Senior Perception AI Engineer - Autonomous Vehicles
Job Description
Join NVIDIA to build online 3D mapping systems for self-driving vehicles by turning multi-sensor inputs into accurate maps that update continuously.
Responsibilities
- Design, train, and iterate PyTorch models that predict online map elements (lanes, road boundaries, crosswalks, road markings, topology) from camera, LiDAR, and/or radar using BEV and vectorized-map representations.
- Work on complex scenarios including:
- Intersection topology (connectivity between incoming and outgoing lanes, turn paths, merges, splits).
- Traffic light understanding (detection, state, and association of each light to the lanes it controls).
- Implement production C++ geometry-based post-processing:
- Curve and surface fitting.
- Association, multi-frame fusion, and tracking.
- Topology reasoning and consistency checks on DNN outputs.
- Apply multi-view geometry, calibration, coordinate transforms, and sensor fusion to produce stable, accurate 3D maps over time.
- Optimize and deploy models and algorithms on NVIDIA DRIVE Orin and NVIDIA Thor, operating under real-time and safety constraints, and bring solutions into vehicles.
- Own the GT and data loop narrative:
- Plan datasets and define GT and labeling requirements.
- Partner with labeling, data, and tooling partners to produce high-quality GT.
- Mine data, choose metrics, and analyze failure cases to drive improvements.
- Define critical metrics and build evaluation tooling to measure quality gains using large-scale real-world data.
- Collaborate with perception, localization, planning, and infrastructure teams to integrate online mapping into the full AV stack across L2++, L3, and L4 programs.
- Contribute technical direction and architecture decisions across learned and geometric components of the mapping system.
Requirements
- MS or PhD (or equivalent experience) in Computer Science, Electrical Engineering, Robotics, or a related field (or equivalent experience).
- 5+ years of industry or research experience in autonomous driving, robotics, or computer vision.
- Strong 3D geometry foundation: camera models, projective geometry, coordinate transforms, optimization, and curve/surface fitting.
- Strong C++ skills with a track record of shipping production-quality algorithm code that is fast, reliable, and maintainable.
- Hands-on experience training deep neural networks in PyTorch, including model design, training pipelines, debugging, and performance analysis.
- Data-driven development experience: building datasets, defining labels and GT, choosing metrics, and improving models through failure analysis.
- Excellent problem-solving and communication skills, with the ability to drive technical direction across teams.
Technologies
- PyTorch
- C++
- BEV
- LiDAR
- radar
- camera
- multi-view geometry
- CUDA
- TensorRT
- NVIDIA DRIVE Orin
- NVIDIA Thor
Benefits
- equity
- benefits
Ways to Stand Out
- Direct experience with DNN-based online or HD map generation, or BEV perception (for example, vectorized map prediction or lane topology learning), or equivalent experience to BEV perception.
- Experience with intersection understanding such as lane connectivity and topology reasoning in complex junctions, or traffic light detection and lane association.
- Experience combining learned models with geometric post-processing in a production AV system, plus publications or open-source contributions in 3D vision, mapping, or autonomous driving.
- Background in sensor fusion, multi-object tracking, or temporal fusion for map elements.
- Experience deploying DNNs and algorithms on embedded GPUs (CUDA, TensorRT), ideally on DRIVE Orin or Thor, under real-time constraints.
Location: Santa Clara, CA (onsite)
Compensation: USD 184,000 - 356,500 per year
Experience level: 5+ years