Senior Cloud Data Engineer

Drive scalable cloud data engineering initiatives supporting manufacturing operations and analytics.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks, PySpark, Python, and SQL
• Build and optimize ingestion frameworks for ERP, MES, Industrial IoT, quality, supply chain, inventory, finance, and related operational systems
• Create reusable engineering patterns and standards to improve delivery speed, supportability, and reliability
• Implement lakehouse solutions using Delta Lake and Medallion Architecture across Bronze, Silver, and Gold layers
• Design and optimize data models for reporting, analytics, AI, and machine learning use cases
• Maintain data lineage, metadata, technical documentation, and operational runbooks
• Integrate Azure Databricks with Azure Data Factory, Azure Data Lake Storage Gen2, Azure Key Vault, Microsoft Fabric, Power BI, and Azure DevOps
• Support orchestration and data movement between enterprise applications and cloud platforms
• Monitor and optimize Databricks clusters, jobs, notebooks, and workflows for performance, scalability, and cost efficiency
• Troubleshoot production issues, identify root causes, and implement durable corrective actions
• Contribute to monitoring, alerting, testing, and operational support practices
• Apply organizational standards for data quality, governance, privacy, security, and compliance
• Implement role-based access controls and governed data access using Unity Catalog and Azure security services
• Build automated validation, auditing, and data quality controls into engineering workflows
• Partner with business users, analysts, architects, and IT teams to translate requirements into maintainable technical solutions
• Participate in Agile ceremonies, including sprint planning, code reviews, and technical design reviews
• Mentor junior engineers and promote best practices for coding, testing, documentation, and deployment

Requirements

• Bachelor’s degree in computer science, information systems, data engineering, or a related field; an equivalent combination of education and experience may be considered
• 3–5 years of experience in data engineering, cloud engineering, or analytics platform development
• Minimum 2 years hands-on experience developing solutions with Azure Databricks
• Experience supporting enterprise data solutions in a manufacturing or industrial environment
• Strong knowledge of Azure Databricks, Apache Spark, PySpark, Python, SQL, Delta Lake, lakehouse architecture, data modeling, and data warehousing concepts
• Experience with Azure Data Lake Storage Gen2, Azure Data Factory, Azure DevOps, Git source control, and CI/CD practices for data platforms
• Experience with data quality, security, monitoring, troubleshooting, and performance tuning
• Knowledge of manufacturing data domains such as ERP, MES, production planning, inventory, supply chain, quality management, maintenance, or Industrial IoT
• Ability to work with operational stakeholders and translate manufacturing processes into reliable data requirements and solutions

Technologies

• Azure Databricks, PySpark, Python, SQL
• Delta Lake, Medallion Architecture, Bronze, Silver, Gold
• Azure Data Factory, Azure Data Lake Storage Gen2, Azure Key Vault, Microsoft Fabric, Power BI
• Azure DevOps, Unity Catalog
• Terraform, Bicep, ARM templates
• Git, CI/CD, Infrastructure as Code
• Informat M3, Infor M3

Benefits

• Excellent health and welfare benefits including medical, dental, vision, disability, and voluntary benefit options
• 401(k) with annual company match for eligible employees
• Professional and personal development programs, including Career and Learning Paths

Additional Skills / Experience / Requirements

• Experience with Microsoft Fabric, Power BI, or real-time and streaming data workloads
• Hands-on experience with Unity Catalog and governed lakehouse implementations
• Exposure to machine learning or AI data preparation and deployment patterns
• Experience with Infrastructure as Code using Terraform, Bicep, or ARM templates
• Preferred knowledge of Infor M3, including application design, business data model, and core table structures across manufacturing, supply chain, inventory, finance, and related functional areas
• Preferred understanding of production, planning, inventory, quality, maintenance, and supply chain processes
• Microsoft Certified: Azure Data Engineer Associate
• Databricks Certified Data Engineer Associate or Professional
• Microsoft Certified: Fabric Analytics Engineer Associate