Project - Data Engineer
Job Description
This on-site role is based in Deloitte’s Rochester, NY delivery hub, where data engineering drives analytics outcomes for clients. The Project - Data Engineer focuses on building and refining data pipelines on AWS and Snowflake, delivering analytics-ready datasets, and boosting data quality and observability within project delivery teams. The salary range for this role is USD 57,300 to 95,500 per year.
Responsibilities
- Design and improve AWS-based data pipelines using Python to ingest, transform, and supply data to Snowflake and downstream consumers.
- Create and maintain Snowflake schemas, tables, and views, and implement efficient SQL transformations to yield analytics-ready datasets.
- Set up workflow automation and scheduling using Airflow or MWAA, Step Functions, or Glue, including dependencies, retries, and logging.
- Incorporate data quality checks and basic observability (validation rules, reconciliations, alerts) and assist with incident triage and remediation.
- Optimize pipelines and queries following best practices, including efficient Python code, S3 partitioning and file formats (Parquet, Delta), and Snowflake warehouse tuning.
- Adhere to CI/CD and IaC standards, utilizing Git-based workflows and Terraform or CloudFormation changes to promote code across environments.
- Collaborate with analysts, product owners, and source-system teams to clarify requirements and validate outputs, and participate in sprint ceremonies and estimations.
- Contribute to code reviews, write and review unit tests, assist with peer debugging, and adopt team engineering standards.
- Communicate regularly with Engagement Managers, Directors, project teammates, and diverse functional or technical stakeholders, escalating issues that require engagement management attention.
- Lead client engagement workstreams, independently or with others, focused on process improvement, optimization, and transformation, including implementing best-practice workflows and addressing quality deficits to drive operational outcomes.
Requirements
- Over one year of experience building and enhancing data pipelines and curated datasets for analytics and downstream consumers.
- At least one year of hands-on SQL and Python experience, including Snowflake and/or PySpark for transformations and scalable processing.
- Minimum of one year in cloud data engineering on AWS (preferred) or Azure/GCP, with orchestration/scheduling using Airflow/MWAA, Step Functions, Glue, or ADF/Fabric Data Factory.
- Understanding of ELT patterns and lakehouse/warehouse concepts; familiarity with S3 file formats and partitioning such as Parquet or Delta.
- Working knowledge of DevOps practices, including Git-based workflows and CI/CD, plus exposure to Infrastructure-as-Code tools such as Terraform or CloudFormation.
- Understanding of data quality, basic observability, and metadata/governance fundamentals.
- Bachelor’s degree, ideally in computer science, information technology, computer engineering, or a related IT discipline, or equivalent experience.
- Limited immigration sponsorship may be available.
- Ability to travel approximately 10 percent on average, depending on client engagements.
Technologies
- Python
- AWS
- Snowflake
- PySpark
- Airflow
- MWAA
- Step Functions
- Glue
- S3
- Parquet
- Delta
- Git
- Terraform
- CloudFormation
- Azure Data Factory
- Fabric Data Factory
Benefits
- Discretionary annual incentive program (subject to rules)
The Team
AI & Data - AI & Engineering leverages cutting-edge engineering capabilities to build, deploy, and operate integrated/verticalized sector solutions in software, data, AI, network, and hybrid cloud infrastructure. These solutions are powered by engineering for business advantage, transforming mission-critical operations. We enable clients to stay ahead with the latest advancements by transforming engineering teams and modernizing technology & data platforms. Our delivery models are tailored to me
Preferred
- Experience delivering projects in Agile environments.
- Strong analytical skills to manage multiple projects and prioritize tasks into clear work products.
- Ability to work independently or with minimal supervision.
- Excellent written and verbal communication skills.
- Ability to present technical demonstrations.