Data Engineer / Lead Data Engineer
Manager
Analytics
Azure
Azure Data Engineer
Azure Data Factory
Azure Data Platform
Big Data
Business Intelligence
Cloud
Cloud Data Engineering
Cloud Data Platform
Cloud Engineer
Cloud Platform
Cloud Platforms
Cloud Platforms Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Factory
Data Factory Azure
Data Integration
Data Integration Tools
Data Pipeline
Data Platform
Data Processing
Data Visualization
Data Warehouse
Database
Databases
Digital Marketing
ETL
Informatica
Microsoft Azure
Reporting and Analytics
SQL
Job Description
MSR Cosmos is seeking a Data Engineer / Lead Data Engineer (hybrid) to design and deliver data foundations and end-to-end analytics solutions for a complex energy trading environment.
Responsibilities
- Lead the design and development of scalable data ingestion and transformation pipelines using standard data lake and cloud engineering frameworks
- Build solutions for structured and unstructured data in high-volume, high-velocity, high-variety environments
- Develop transformation notebooks, physical and tabular data models, data endpoints, semantic layers, and visualisations
- Create reusable data products that improve data accessibility, consistency, and business value
- Lead ETL/ELT activities across source systems, data platforms, and downstream BI solutions
- Create and publish dashboards and reports for stakeholders with accessible, actionable information
- Run trend and correlation analysis; detect data discrepancies and implement controls to protect data quality across platforms
- Define and enforce engineering standards, reusable patterns, documentation, and development practices
- Build automated CI and release pipelines and apply disciplined source-control practices
- Gather functional and non-functional requirements; translate features and Product Backlog Items into technical designs and delivery direction
- Collaborate with cross-functional teams and mentor engineers where required
Requirements
- Mandatory: demonstrable data engineering experience in energy trading, commodities trading, or an ETR/CTRM environment
- Strong hands-on experience designing production-grade ingestion, storage, transformation, modelling, and serving layers on Microsoft Azure
- Advanced SQL, including development and optimisation of complex queries for data transformation and analysis
- Hands-on experience with Azure Data Factory, Azure SQL, Azure Databricks, Azure Blob Storage, and Azure Data Lake Storage
- Strong ETL/ELT and data warehousing experience, including delivery of BI solutions
- Experience with dimensional, physical, and tabular data modelling and performance optimisation
- Experience with GitHub and/or Azure DevOps, including automated build, test, and release pipelines
- Ability to gather requirements, shape technical solutions, and communicate clearly with technical and business stakeholders
- Proven ability to lead workstreams, manage priorities, and deliver effectively in a collaborative team environment
Preferred Experience
- Experience with dbt, Azure Analysis Services, Azure Table Storage, and Azure Logic Apps
- Experience with Power BI, including clear, decision-oriented dashboards and reports
- Exposure to Power Apps and broader Microsoft Power Platform capabilities
- Familiarity with IBM DataStage or comparable enterprise integration tooling
- Basic understanding of AI engineering, including how governed data products support analytics and AI use cases
- Knowledge of data governance, lineage, metadata management, observability, and automated data-quality controls
Technologies
- SQL
- Microsoft Azure
- Azure Data Factory
- Azure SQL
- Azure Databricks
- Azure Blob Storage
- Azure Data Lake Storage
- GitHub
- Azure DevOps
Location: Houston, TX (hybrid)
Compensation: USD 65 - 70 per hour