Together AI Logo

Together AI

Distributed ML Systems Engineer- Inference

Job Posted 3 Days Ago Reposted 3 Days Ago
Be an Early Applicant
In-Office
San Francisco, CA
160K-230K
Mid level
In-Office
San Francisco, CA
160K-230K
Mid level
Design and build scalable, fault-tolerant machine learning systems while collaborating with researchers and ensuring system health through monitoring.
The summary above was generated by AI
About the Role

Together AI is seeking a Distributed ML Systems Engineer to design and build scalable machine learning systems that power our accelerated AI initiatives. This role involves developing large-scale, fault-tolerant distributed systems that handle high-load and high-performance requirements. If you are passionate about designing ML systems that operate at scale and eager to create impactful solutions, we want to hear from you. This position offers the chance to work closely with our AI researchers and infrastructure teams to ensure our systems are robust and efficient. Join us in shaping the future at Together AI!

Responsibilities
  • Design and build large-scale, distributed machine learning systems that are fault-tolerant and high-performance.
  • Develop and optimize distributed processing frameworks and storage systems.
  • Collaborate with researchers, engineers, and product managers to integrate ML systems into our infrastructure.
  • Conduct architecture and design reviews to ensure best practices in system design.
  • Implement robust monitoring and logging systems to ensure the health and performance of our ML systems.
Requirements
  • 3+ years of experience in building large-scale, fault-tolerant, high-performance distributed systems.
  • Strong programming skills in one or more of Python, Go, Rust, or C/C++.
  • Excellent understanding of low-level operating systems concepts including multi-threading, memory management, networking, and storage, performance, and scale.
  • Experience with cloud computing platforms (AWS, GCP, Azure etc.) and large-scale infrastructure.
  • Strong problem-solving skills and ability to work in a fast-paced environment.
  • Preferred: Experience with Kubernetes
  • Preferred: Experience with Pytorch
About Together AI

Together AI is a research-drven artificial intelligence company. We believe open and transparent AI systems will drive innovation and create the best outcomes for society. Together, we are on a mission to significantly lower the cost of modern AI systems by co-designing software, hardware, algorithms, and models. We have contributed to leading open-source research, models, and datasets to advance the frontier of AI. Our team has been behind technological advancements such as FlashAttention, Hyena, FlexGen, and RedPajama. We invite you to join a passionate group of researchers and engineers in our journey to build the next-generation AI infrastructure.

Compensation

We offer competitive compensation, startup equity, health insurance, and other competitive benefits. The US base salary range for this full-time position is $160,000 - $230,000 + equity + benefits. Our salary ranges are determined by location, level, and role. Individual compensation will be determined by experience, skills, and job-related knowledge.

Equal Opportunity

Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunities to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.

Please see our privacy policy at https://www.together.ai/privacy  



Top Skills

AWS
Azure
C/C++
GCP
Go
Kubernetes
Python
PyTorch
Rust

Together AI San Francisco, California, USA Office

584 Castro St, #2050, San Francisco, California , United States, 94114

Similar Jobs

17 Minutes Ago
In-Office
Carlsbad, CA, USA
33K-55K Annually
Junior
33K-55K Annually
Junior
Information Technology • Internet of Things • Mobile • On-Demand • Software
The Senior Account Sales Representative focuses on selling products to residential customers through door-to-door, builds relationships with property managers, and meets sales goals while addressing customer issues.
Top Skills: Cold Call Sales SkillsCommunication SkillsComputer Operating SystemsComputer Software ApplicationsConsumer Communications Devices
17 Minutes Ago
In-Office
El Segundo, CA, USA
81K-149K Annually
Mid level
81K-149K Annually
Mid level
Information Technology • Internet of Things • Mobile • On-Demand • Software
As a Multimedia Journalist, you'll pitch, write, shoot, interview, and edit stories, uncovering community issues and delivering news across platforms.
Top Skills: Social MediaVideo Editing Software
18 Minutes Ago
In-Office
El Segundo, CA, USA
73K-117K Annually
Mid level
73K-117K Annually
Mid level
Information Technology • Internet of Things • Mobile • On-Demand • Software
The Digital Partner - Account Executive manages relationships with digital partners, driving sales, ensuring compliance, and conducting marketing audits and strategies to enhance revenue.
Top Skills: ExcelMicrosoft Office SuiteOutlookPowerPointWord

What you need to know about the San Francisco Tech Scene

San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.

Key Facts About San Francisco Tech

  • Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Google, Apple, Salesforce, Meta
  • Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
  • Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
  • Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
By clicking Apply you agree to share your profile information with the hiring company.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account