Sglang Data Jobs

1 job found

Today

Machine learning engineer II, AWS Inference Optimization

Amazon Development Center U.S., Inc. Santa Clara, CA

Santa Clara, CA

Today

$165k - $224k/yr Full time Artificial Intelligence AWS Aws Bedrock

Apply Now

Job Description

This role is for a Machine Learning Engineer focused on AWS inference optimization. You will work on improving inference performance for large language OSS models, with ownership of price-performance for Amazon LLM inference across SageMaker, Hyperpod, and Bedrock.

What You’ll Do

Develop model servers.
Build CUDA kernels in vLLM and SGLang.
Create services for GenAI optimization and benchmarking.
Create or modify a mid-size application or service.
Optimize large language OSS models such as Qwen, Gemma, gptoss, and others by improving inference stacks including vLLM and SGLang.
Perform performance tuning for GenAI inference on Amazon SageMaker, Amazon Bedrock, and Hyperpod.
Implement inference optimization techniques including Speculative decoding, Disaggregated Inference, and CUDA kernel tuning.
Own price-performance for Amazon LLM inference deployed on SageMaker, Hyperpod, and Bedrock.
Deliver production solutions that leverage existing academic and industrial research or apply practical, original thinking.
Deliver solutions into customer-facing products.

Key Technologies

vLLM, SGLang
Amazon SageMaker, Amazon Bedrock
Hyperpod
CUDA
C#, C++, Java, Perl
Transformer architecture

Requirements

3+ years of non-internship professional software development experience.
2+ years of non-internship design or architecture experience, including design patterns and reliability and scaling of new and existing systems.
1+ year of software development engineer or related occupational experience.
1+ year designing and developing large-scale, multi-tiered, multi-threaded, embedded or distributed software applications, tools, systems, and services using C#, C++, Java, or Perl.
Bachelor’s degree (or foreign equivalent) in Computer Science, Engineering, Mathematics, or a related field.
Knowledge of machine learning and LLM fundamentals, including transformer architecture, training/inference lifecycles, and optimization techniques.

Preferred Qualifications

3+ years of full software development life cycle experience, including coding standards, code reviews, source control management, build processes, testing, and operations.
Bachelor’s degree in computer science or equivalent.

Education

Bachelor’s degree (Computer Science, Engineering, Mathematics, or related field).

Location and Work Mode

Santa Clara, CA (onsite)

Compensation

USD 165,200 - 223,600 per year.

Benefits

Sign-on payments
Restricted stock units (RSUs)
Health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance, option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage)
401(k) matching
Paid time off
Parental leave
Workplace accommodation or adjustment during the application and hiring process (including support for the interview or onboarding process)

Los Angeles County Applicants

Job duties for this position include:

Work safely and cooperatively with other employees, supervisors, and staff.
Adhere to standards of excellence despite stressful conditions.
Communicate effectively and respectfully with employees, supervisors, and staff to ensure exceptional customer service.
Follow all federal, state, and local laws and Company policies.
Criminal history may have a direct, adverse, and negative relationship with some of the material job duties of this position, including the abilities and responsibilities listed above, as well as the abilities to adhere to Company policies, exercise sound judgment, effectively manage stress and work safely and respectfully with others, exhibit trustworthiness and professionalism, and safeguard business operations and the Company’s reputation.
Pursuant to the Los Angeles County Fair Chance Ordinance, qualified applicants with arrest and conviction records will be considered for employment.

View full job page →

Get Sglang Data Jobs alerts

New Sglang Data Jobs, daily.