Data Engineer

Proteam Solutions is seeking a Senior Data Engineer to join onsite in Dublin, OH. This role focuses on designing, developing, and optimizing enterprise-scale data pipelines with Databricks, supporting analytics, reporting, and downstream applications. You will play a key part in building scalable Lakehouse architectures and ensuring reliable, high-quality data across critical data workflows. The position calls for deep Databricks expertise, strong data engineering fundamentals, and a collaborative approach with cross-functional teams.

Responsibilities

• Design and build scalable ETL and ELT pipelines using Databricks
• Develop PySpark and Spark SQL transformations
• Construct Bronze, Silver, and Gold data layers following the Medallion Architecture
• Implement Delta Live Tables, Workflows, and Unity Catalog
• Optimize Spark jobs for scalability and performance
• Design dimensional data models for analytical workloads
• Implement data quality validations and monitoring
• Collaborate with analytics, engineering, and architecture teams
• Build reusable frameworks and automation
• Support CI/CD deployment pipelines and Git-based development

Requirements

• 7–10+ years of Data Engineering experience
• Extensive hands-on Databricks experience
• Strong PySpark programming
• Advanced Spark SQL skills
• Experience with Delta Lake
• Experience implementing Medallion Architecture
• Hands-on Delta Live Tables
• Unity Catalog experience
• Strong SQL optimization and performance tuning
• Experience with AWS, Azure, or GCP
• CI/CD and Git experience
• Strong understanding of distributed data systems
• Experience designing enterprise data models

Technologies

• Databricks
• PySpark
• Spark SQL
• Delta Lake
• Medallion Architecture
• Delta Live Tables
• Unity Catalog
• AWS
• Azure
• GCP
• Git
• CI/CD

Preferred Qualifications

• Databricks certification
• Experience with streaming data
• Experience building reusable data frameworks
• Experience with enterprise governance
• Exposure to AI-assisted development tools including Databricks AI Genie