Lead Data Engineer
Manager
Analytics
Apache Airflow
Big Data
Bigdata
Business Intelligence
Cloud
Cloud Data Engineering
Cloud Data Lakehouse
Cloud Data Platform
Cloud Infrastructure
Cloud Platform
Cloud Platforms
Cloud Technology
Data
Data Analysis
Data Analytics
Data Architecture
Data Build Tool
Data Engineer
Data Engineering
Data Engineering Lead
Data Integration
Data Integration Tools
Data Lakehouse
Data Management
Data Modeling
Data Operations
Data Pipeline
Data Pipelines
Data Platform
Data Processing
Data Transforms
Data Visualization
Data Warehouse
Database
Databases
Databricks
DevOps
DevSecOps
Digital Marketing
ETL
Event Driven Architecture
Informatica
Infrastructure As Code
Lead Data Engineering
Medallion Architecture
Programming Language
Reporting and Analytics
Security Devops
Software Development
Spark
SQL
Streaming Data
Visual Design
Job Description
The Lead Data Engineer at Stafford Gray will architect and deliver a modern cloud data lakehouse, while acting as the technical lead for data engineering execution. The role includes building governed data products, overseeing pipeline and transformation patterns, and mentoring engineers through reviews and hands-on support, in coordination with data science and business stakeholders.
Responsibilities
- Design, build, and maintain pipelines that ingest relational, semi-structured, and unstructured data into a cloud data lake on AWS, Azure, or GCP.
- Transform lake datasets using a Medallion (bronze/silver/gold) approach into Parquet-based data products, ensuring each attribute can be traced back to its source system.
- Implement and enforce data sharing and access policies so subscribers can view only the versions of data products they are entitled to.
- Design star-schema operational data stores that feed Power BI and Tableau reporting.
- Define and uphold standards for data quality, monitoring, validation, and data governance.
- Lead and mentor data engineers through design reviews, code reviews, and hands-on training.
- Collaborate with data scientists, analysts, product teams, and business stakeholders to deliver data and ML solutions.
Requirements
- Bachelor’s degree in Computer Science, Data Science, Information Systems, or a related field.
- 4+ years of hands-on experience building extract-and-load pipelines into an enterprise data lake on a major cloud provider.
- 4+ years of hands-on experience transforming data lake datasets into data products with traceability to systems of record.
- 2+ years of experience enforcing data sharing or access policies on data products.
- 4+ years of programming in Python, PySpark, and SQL, including SQL tuning and loading data into Parquet from databases, flat files, CSV, JSON, and APIs.
- Experience designing star-schema data stores for BI reporting.
- Must live within 90 miles of Lansing, MI at the time of submission and be able to work a hybrid onsite schedule.
Technologies
Work includes AWS, Azure, GCP, Medallion (bronze/silver/gold), Parquet, Power BI, Tableau, Python, PySpark, SQL, CSV, JSON, APIs, Databricks, Spark, DBT, Glue, Redshift Spectrum, SageMaker, Kafka, Kubernetes, Docker, GitHub, Jenkins, Terraform, YAML, Airflow DAGs, NoSQL, and graph databases, plus ML pipeline development.
Preferred
- Experience with Databricks, Spark, DBT, Glue, Redshift Spectrum, and SageMaker.
- Experience with Kafka, Kubernetes, and Docker.
- DevOps and infrastructure-as-code experience using GitHub, Jenkins, Terraform, YAML, and Airflow DAGs.
- Experience with cloud-native data governance and security tooling.
- Experience with NoSQL and graph databases, and with ML pipelines.
- A track record of championing data engineering standards across a developer community.