Lead Data Engineer – Physical AI Platform

Caterpillar is hiring a Lead Data Engineer to help build scalable, cloud-based data capabilities that power reliable, high-quality data for business and engineering teams. This full-time onsite role is based in Irving, TX (Dallas), with domestic relocation assistance available and visa sponsorship offered. The salary range for this position is $128,470 - $208,770 per year.

What you’ll build and improve

You will design, develop, and maintain data pipelines, microservices, and cloud data platforms that support both real-time and batch processing across enterprise systems. The work centers on cloud-native ingestion and streaming, data integration frameworks, operational monitoring, and data quality testing in an agile environment.

Responsibilities

• Collaborate with Principal Software Engineers and Data Architects to define solution architecture
• Lead solution design and optimization of scalable data pipelines and microservices using Python to support real-time and batch processing
• Drive development of cloud-native data ingestion and streaming solutions using AWS services such as Kinesis, S3, DynamoDB, EventBridge, and related technologies
• Own the design, implementation, and operational excellence of data integration frameworks and source data pipelines supporting CI Autonomy initiatives
• Partner with business, product, and engineering stakeholders to translate complex requirements into scalable data architectures, workflows, mappings, and system designs
• Establish automated testing, data quality controls, and validation frameworks to protect integrity, reliability, and compliance across distributed data ecosystems
• Lead operational monitoring, performance tuning, and root-cause analysis of production data platforms using observability tooling such as CloudWatch to maintain availability and service reliability

Requirements

• Bachelor’s degree in Computer Science, Computer Engineering, or a related field
• 8+ years of experience in data engineering or related disciplines with increasing responsibility
• Extensive experience on modern, large-scale, complex data platforms
• Strong foundation developing and deploying Python solutions in production environments
• Experience leading teams to build high-throughput, scalable data pipelines
• Hands-on experience with AWS data services including Kinesis, S3, DynamoDB, EventBridge at scale
• Strong skills in SQL, including data quality and validation practices
• Experience deploying software using CI/CD tools such as Azure DevOps, Jira, Jenkins, etc.
• Experience developing microservices that support real-time data ingestion
• Experience developing software applications using relational and NoSQL databases
• Ability to ensure data integrity across distributed and streaming systems
• Experience with monitoring, testing, and automation in large-scale data environments

Tools you’ll work with

Python, Java, AWS (Kinesis, S3, DynamoDB, EventBridge, CloudWatch), CI Autonomy, Azure DevOps, Jira, Jenkins, SQL, APIs, microservices, and real-time and batch data processing.

Benefits

• Medical, dental, and vision benefits
• Paid time off plan (Vacation, Holidays, Volunteer, etc.)
• 401(k) savings plans
• Health Savings Account (HSA)
• Flexible Spending Accounts (FSAs)
• Health Lifestyle Programs
• Employee Assistance Program
• Voluntary Benefits and Employee Discounts
• Career Development
• Incentive bonus
• Disability benefits
• Life Insurance
• Parental leave
• Adoption benefits
• Tuition Reimbursement

Additional details: Any offer of employment is conditioned upon the successful completion of a drug screen. This role requires full-time work at the Irving, TX office (Dallas).