The Data Engineer II role within AWS Analytics Engineering focuses on designing, building, and operating a next generation data ingestion platform that automatically onboard data from new AWS services at scale. This onsite position is based in Seattle, WA.
Responsibilities
- Architect and maintain an ingestion service capable of onboarding data from new AWS services or features automatically at launch, without manual setup.
- Develop scalable ingestion frameworks able to process petabytes of data.
- Design and operate large-scale data acquisition pipelines for analytics workloads.
- Implement governance, compliance, and audit controls to ensure data moving through the platform meets AWS security and data integrity standards.
- Ensure the platform scales with the growing service portfolio and supports event-driven architectures, self-registration, and configuration-driven onboarding with full lineage and audit trails.
- Create the foundational layer that enables all analytics capabilities within AWS Analytics Engineering.
Requirements
- Five or more years of data engineering experience.
- At least three years designing and operating large-scale BI data structures using ETL/ELT processes.
- At least three years in data modeling for BI analytics within large-scale data structures.
- Three or more years of SQL experience.
- Experience with data modeling, data warehousing, and building ETL pipelines.
- Proficiency in one or more modern programming languages such as Python, Ruby, Golang, Java, C++, C#, or Rust.
- Experience with enterprise-scale infrastructure or cloud programs/projects in a related industry.
- Bachelor's degree in computer science, engineering, analytics, mathematics, statistics, IT, or equivalent.
- Experience implementing data governance controls including access management, encryption, data classification, and audit logging.
Technologies
- Python, Ruby, Golang, Java, C++, C#, Rust
- AWS Redshift
- AWS S3
- AWS Glue
- AWS EMR
- AWS Kinesis
- FireHose
- AWS Lambda
- AWS IAM
- Amazon RDS
Benefits
- Comprehensive health coverage including medical, dental, vision, prescriptions, and Basic Life and AD&D insurance, with option for supplemental life plans
- Mental health resources
- Employee Assistance Program
- Medical advice line
- Flexible spending accounts
- Adoption and surrogacy reimbursement
- 401(k) matching
- Paid time off
- Parental leave
Description
The AWS Analytics Engineering (AAE) organization serves as the analytics backbone of AWS, building and operating the data platform that informs decisions across more than 150 AWS services. The team handles data at massive scale, processing petabytes daily through thousands of jobs that include transformations, reporting, ingestions, and infrastructure management scripts.
We are seeking a Data Engineer to join the Data Ingestion Platform team. In this role you will design, build, and operate the next generation ingestion service that automatically onboards data from any new AWS service or feature at launch without manual intervention. The platform must scale to the growing suite of AWS services, ensuring ingestion infrastructure expands accordingly.
You will work with event-driven architectures, self-registration capabilities, and configuration-driven onboarding to guarantee automatic, reliable ingestion with full lineage and audit trails. The ideal candidate is a strong engineer who focuses on building platform-level solutions and abstractions, and who is excited to lay the foundational layer that powers all of AAE's analytics capabilities.
