Senior, Data Engineer

Epsilon is hiring a Senior Data Engineer to build and optimize AWS and Greenplum data pipelines, with responsibility for SQL and Python development, testing, monitoring, and internal data products. The role is hybrid in Chicago, with certain days required in the office and telecommuting for the remainder of the week.

Key Responsibilities

• Develop and execute software tests to identify software problems and their causes.
• Write and optimize complex SQL scripts and create Python scripts to run large-scale transformations on Greenplum.
• Create test plans, perform unit and integration testing, and deliver reproducible, well-documented results.
• Design and maintain AI-enabled PySpark data pipelines on AWS, primarily using EMR, Glue, Athena, DynamoDB, and S3 to process large datasets.
• Harden and tune jobs for performance, cost, and compliance, including management of data partitioning, cataloging, and lineage.
• Automate deployments and infrastructure with CloudFormation and parameterized configurations, and schedule batch workloads to support timely, reliable pipelines.
• Build internal dashboards and data products (for example, Amplify apps) and implement monitoring and alerting to detect issues early.
• Handle client tickets by translating ambiguous requests into clear technical tasks.

Required Qualifications

• Master’s degree in Computer Science, Information Technology, Engineering or related field.
• Two years of experience in the job offered, or two years in any occupation in which the required experience was gained.
• Build PySpark ETL on AWS EMR/Glue reading and writing to S3, including partitioning, predicate pushdown, and broadcast joins, to deliver analytics-ready fact/dimension tables on schedule.
• Author and tune complex SQL on Greenplum/Postgres, including window functions, ROW_NUMBER dedupe, distribution keys, AO/AOCO compression, and VACUUM/ANALYZE for efficient large-dataset processing.
• Manage AWS Glue Data Catalog and work with curated layers using Athena; design S3 layouts (prefix/partition strategy and lifecycle policies) to reduce query time and storage cost.
• Implement CI/CD with CodeCommit, CodeBuild, and CodePipeline; parameterize environments and use CloudFormation for infrastructure as code to enable repeatable, auditable, and fast rollbacks.
• Engineer observability using CloudWatch logs and metrics, alarms, and job health checks; create SLA dashboards and alerting to detect failures early and reduce MTTR.
• Enforce data quality through SQL/Python unit tests, reconciliation (row counts, sums, duplicates), referential-integrity checks, and documented UAT evidence for change control.
• Apply security and data-governance controls, including normalization and encryption of sensitive identifiers, use of KMS-managed keys, and IAM least privilege with audit logging.
• Develop GenAI-assisted delivery that integrates LLMs with AWS Bedrock and PySpark/SQL to auto-suggest transformations, generate test cases, and summarize validation results.
• Build lightweight internal data apps with AWS Amplify plus DynamoDB/Athena to manage lookup keys or brand lists and enable stakeholder self-service with controlled access.

Technologies

• SQL, Python, Greenplum, PySpark
• AWS EMR, AWS Glue, AWS Athena, Amazon DynamoDB, Amazon S3
• CloudFormation, AWS CodeCommit, AWS CodeBuild, AWS CodePipeline
• Amazon CloudWatch, AWS KMS, AWS IAM
• AWS Bedrock, AWS Amplify, Postgres

Compensation

USD 145,870 - 182,000 per year.

Work Location

35 West Wacker Drive, Chicago, IL 60601

Required to work from the Company Office in Chicago, IL certain days per week. Telecommuting is allowed for the balance of the week.

Benefits

• Flexible time off (FTO), 14 paid holidays, and paid sick time
• Parental/new child leave, childcare and elder care assistance, and adoption assistance
• Comprehensive health coverage
• 401(k)
• Tuition assistance
• Commuter benefits
• Professional development and employee recognition
• Charitable donation matching
• Health coaching and counseling

Company Overview

Epsilon is a global leader in data and identity with deep expertise in digital media, clean rooms, customer data platforms, loyalty and marketing services. The organization operates in 30+ countries across EMEA, the Americas, and APAC, with more than 8,000 employees.

Epsilon’s five values are: Act with integrity, Work together to win together, Innovate with purpose, Respect all voices, and Empower with accountability.

Equal Opportunity and Additional Notes

• Epsilon is an equal opportunity employer and does not discriminate based on protected categories.
• Accommodations are available for applicants who need support to complete the application process. Contact [email protected] to request an accommodation.
• For the San Francisco Bay and Los Angeles areas, Epsilon will consider qualified applicants with criminal histories consistent with applicable fair chance ordinances.