Build production-grade machine learning inference that powers experiences across Cloudflare’s global network. In this hybrid role in New York, NY, you’ll help shape how models run from frontier open LLMs and real-time voice workloads to customer-deployed models served across heterogeneous GPUs and next-generation accelerators. You’ll focus on low latency, strong reliability, efficient resource use, and rigorous benchmarking and evaluation.
Cloudflare also supports your wellbeing and financial security with comprehensive coverage, time off, and retirement planning, plus eligibility for an Employee Stock Participation Plan and participation in the company equity program.
What you’ll do
- Develop, optimize, and productionize machine learning models for Cloudflare’s serverless inference platform, emphasizing performance, reliability, and model quality.
- Create benchmarking and evaluation frameworks to measure latency, throughput, cost efficiency, and model behavior across LLMs, speech, vision, and other model families.
- Improve inference performance using quantization, batching, caching, model compilation, runtime tuning, and accelerator-aware optimization.
- Work with systems engineers to integrate models into Cloudflare’s distributed inference infrastructure across a heterogeneous fleet of GPUs and next-generation accelerators.
- Advance model deployment workflows, including validation, rollout safety, observability, regression testing, and operational readiness.
- Collaborate with product and engineering teams to translate customer requirements into scalable ML capabilities for Workers AI.
- Mentor engineers, contribute to technical direction, and help raise the quality bar for production ML practices.
What you bring
- Experience building, optimizing, and operating machine learning models in production environments.
- Strong Python skills and modern ML framework proficiency including PyTorch, TensorFlow, JAX, or equivalent.
- Hands-on inference optimization for large-scale models, including quantization, batching, caching, compilation, and serving runtime tuning.
- Experience with large-scale inference serving frameworks or runtimes such as SGLang, vLLM, TensorRT-LLM, ONNX Runtime, Triton, llama.cpp, or similar.
- Familiarity with LLMs, speech models, vision models, embeddings, multimodal models, retrieval-augmented generation, or other modern deep learning architectures.
- Experience optimizing models for GPUs or specialized accelerators.
- Strong production ML foundations, including evaluation, monitoring, model regressions, rollout safety, and reliability.
- Ability to work across ML and systems boundaries, with familiarity with distributed systems, networking, or serverless platforms.
- Track record leading complex technical projects and mentoring other engineers.
Tools you may work with
Python, PyTorch, TensorFlow, JAX, SGLang, vLLM, TensorRT-LLM, ONNX Runtime, Triton, llama.cpp, Workers AI
Compensation
USD 194,000 - 266,000 per year (New York, NY hybrid). Compensation may be adjusted depending on work location. Bay Area hires have an estimated annual salary of $194,000 - $266,000, and New York City, New Jersey, Washington, Washington DC, and California (excluding Bay Area) hires have an estimated annual salary of $185,000 - $254,000.
Equity
This role is eligible to participate in Cloudflare’s equity plan.
Benefits
- Medical/Rx Insurance
- Dental Insurance
- Vision Insurance
- Flexible Spending Accounts
- Commuter Spending Accounts
- Fertility & Family Forming Benefits
- On-demand mental health support and Employee Assistance Program
- Global Travel Medical Insurance
- Short and Long Term Disability Insurance
- Life & Accident Insurance
- 401(k) Retirement Savings Plan
- Employee Stock Participation Plan
Time off and leave
- Flexible paid time off covering vacation and sick leave
- Leave programs, including parental, pregnancy health, medical, and bereavement leave
Bonus points
- Experience contributing to open source ML tooling, model serving frameworks, or inference runtimes.
