Job Details

Job ID: 2871415 | Amazon Development Center U.S., Inc.AWS AI is looking for world-class software developers to join the Deep Learning cross-framework team. In this organization, you will be responsible for contributing extensions to the TensorFlow and PyTorch machine learning frameworks and for developing cross-framework solutions to support training of Deep Learning models at scale, involving thousands of accelerators. You will be working in a fast-paced, cross-disciplinary team of engineers and researchers who are leaders in the field. You will take on challenging problems, elicit requirements, and deliver innovative solutions into production that consolidate the AI team as thought leaders in the space.Key job responsibilitiesDeveloping innovative solutions for supporting Large Language Model training in a cluster of nodes;Implementing model parallelism methods such as pipeline and tensor parallelism as extensions to the PyTorch framework;Implementing sharding of the model training state, activation checkpointing/offloading and other memory saving techniques;Optimizing distributed training by profiling, identifying bottlenecks and addressing them by improving compute and network performance, as well as finding opportunities for better compute/communication overlap;Optimizing communication collectives for the AWS network infrastructure;A day in the lifeThe SageMaker Engines team develops technology for supporting training of Deep Learning models at large scale. This entails implementation of model parallelism and memory saving techniques to allow training of models across accelerators as well as implementation of network communication collectives optimized for the AWS infrastructure.BASIC QUALIFICATIONS 4+ years of non-internship professional software development experience4+ years of programming with at least one software programming language experience4+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experienceExperience as a mentor, tech lead or leading an engineering teamBachelor's degree in computer science or equivalentPREFERRED QUALIFICATIONS 4+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experienceM.Sc. or PhD Degree in computer science, engineering, statistics, mathematics or related fieldProficiency in the TensorFlow and/or PyTorch frameworksStrong working knowledge of C++ programming language;Strong working knowledge of Python programming languageExperience in developing highly scalable, fault-tolerant, distributed systemsExperience with multi-threaded asynchronous development in C++Solid understanding of machine learning techniques and conceptsExperience with High Performance Computing systemsExperience with CUDA programmingExperience with Deep Learning models for Natural Language Understanding or Computer Vision tasks;Experience with Linux kernel system calls or POSIX API (process control, communication, and device management)Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status.#J-18808-Ljbffr