Shift4 is seeking a hands-on Data Engineer to build reliable SQL and Python workloads in Snowflake for payments datasets and reporting.

Design, write, and maintain Python jobs that execute SQL over Snowflake to produce transaction, settlement, fee, and reconciliation datasets
Develop performant, readable SQL using window functions, CTEs, incremental logic, and semi-structured (JSON/VARIANT) handling on large transaction tables
Package workloads as Docker containers and run them on the existing scheduler
Ensure jobs are idempotent, runnable multiple times, parameterized, and observable with logging and alerting for failures or bad outputs
Implement data-quality checks and reconciliation controls to verify accuracy and consistent totals across sources and reporting periods
Monitor Snowflake cost through warehouse sizing, clustering, query profiling, and avoiding wasteful scans
Collaborate with finance, operations, and product stakeholders to translate payments questions into correct and maintainable queries
Apply payments-appropriate security controls: secrets handling, least-privilege access, PII masking, and audit trails
Document the job library and contribute to code review, testing, and CI practices
Explore and apply AI tools to improve efficiency across query authoring, testing, and documentation

1 to 3 years of experience in data engineering, analytics engineering, or backend roles building SQL-heavy data workloads in production
Strong SQL skills including window functions, CTEs, aggregation and joins over large tables, query tuning, and reading execution plans
Hands-on Snowflake experience: warehouses and cost awareness; stages and COPY INTO; streams and tasks; time travel; VARIANT/semi-structured data; Snowpark or the Snowflake Python connector
Solid Python for data work: pandas or Polars, parameterized SQL, configuration and secrets management, structured logging, error handling, and unit tests
Docker fundamentals: building images, running scheduled batch jobs in containers, and debugging container failures
Git and basic CI/CD habits; comfort with code review and writing jobs that are testable and rerunnable
Strong attention to numerical correctness, including investigating mismatched totals until the root cause is understood

Payments domain knowledge: authorization, capture, settlement, refunds, chargebacks, interchange and scheme fees, merchant and acquirer data models, reconciliation
Orchestration and transformation tooling: Airflow, Prefect, Dagster, dbt
AWS experience from the developer side: S3, IAM, Secrets Manager, ECS/Fargate scheduled tasks
Data-quality frameworks and monitoring: Great Expectations, dbt tests, Soda, pipeline monitoring/alerting
Experience in a PCI or other regulated environment
Exposure to BI tools consuming outputs: Sigma, Tableau, Power BI, Looker

Location: Center Valley, PA (onsite)

Experience: 1+ years

Data Engineer

Job Description