Azure Data Engineer

Calance US is seeking an Azure Data Engineer to lead the development of a next-generation data platform in Charlotte, NC, onsite. This role centers on migrating legacy data infrastructure to a Microsoft Fabric ecosystem and designing the Medallion data flow into OneLake, aligning with the client’s goal of owning its data future and driving innovation in media sales. The position offers an hourly rate of USD 65 - 75 and requires 5 to 7 years of enterprise data engineering experience.

Responsibilities

• Design, develop, and deploy intricate ETL/ELT pipelines leveraging Azure Data Factory and Fabric Data Pipelines to support the organization’s enterprise analytics.
• Create custom Azure Functions (Python) and API integrations to fill gaps where native connectors fall short, ensuring all data sources are reachable.
• Build robust ingestion frameworks capable of handling high-velocity real-time APIs and scheduled landings (CSV/Parquet).
• Tune OneLake and Delta Lake storage to maximize performance for downstream analytics engineering and AI workloads.
• Maintain and enhance GitHub Actions pipelines to automate deployments across Dev, Test, and Prod, ensuring a stable release cycle.
• Implement data governance, privacy, and security practices using Managed Identities and VNETs to protect data assets.
• Perform root-cause analysis of pipeline failures and establish automated monitoring to minimize downtime.

Requirements

• 5 to 7 years of data engineering experience in an enterprise setting, with proven expertise on Azure Cloud.
• Strong command of Azure-native services, including Azure Data Lake Gen2, API Gateway, and Microsoft Fabric.
• Advanced Python skills (including API development with FastAPI) and SQL across relational and NoSQL databases.
• Hands-on experience with Spark (PySpark) and Delta Lake concepts.
• Experience with serverless compute using Azure Functions and YAML-based CI/CD pipelines.
• Microsoft Certified: Azure Data Engineer Associate is strongly preferred.
• Ability to lead architectural discussions and operate in complex cloud environments, including subscriptions, VNETs, and Managed Identities.

Technologies

• Azure Data Factory
• Fabric Data Pipelines
• Azure Functions
• Python
• FastAPI
• SQL
• Spark
• PySpark
• Delta Lake
• OneLake
• GitHub Actions
• YAML-based CI/CD workflows
• Microsoft Fabric
• API Gateway
• Azure Data Lake Gen2
• Managed Identities
• VNETs
• APIs