Azure Data Lake Storage

Azure Data Lake Storage Jobs

2 jobs found

3d ago

PL Ops - Data Engineer

The Hanover Insurance Group Remote 🌍 Remote

Remote

3d ago

$75k - $100k/yr Full time Analytics Azure Azure Data Factory

Apply Now

Job Description

Join a remote Data Engineer II role with The Hanover Insurance Group and contribute to Personal Lines Operations by building and maintaining Azure Data Factory pipelines and an on-prem SQL Server data environment. You will balance production support with new development while advancing data quality, monitoring, and release practices. The position offers a salary range of USD 75,000 to 100,000 per year and requires a Bachelor’s degree with at least six years of experience.

Benefits

Medical, dental, vision, life, and disability insurance
401K with company match
Tuition reimbursement
PTO
Company paid holidays
Flexible work arrangements
Cultural Awareness Day in support of IDE
On-site medical/wellness center (Worcester only)

Career development

This is a true career opportunity with growth support through on-the-job experiences, personalized coaching, and a robust learning and development program. Professionals at every level are encouraged to grow and advance.

Responsibilities

Provide production support and daily health checks for scheduled ADF pipelines
Design new pipelines and enhance existing ones to improve resiliency, maintainability, and scalability
Implement data validation controls, enhance monitoring and alerts, and help define SLAs for data freshness and availability
Establish foundational data engineering SDLC practices, including Git usage, Dev to Prod promotion, and a formal release process
Coordinate cross-team dependencies and design dependency-aware orchestration and readiness checks
Contribute to future-state Azure data strategy recommendations (e.g., Data Lake/Blob Storage, notebooks) and support migration planning from on-prem SQL Server to cloud databases
Azure/Fabric Data Factory engineering
Design and maintain curated SQL tables and views for analytics and reporting; optimize refresh performance and downstream usability
Develop and maintain ADF pipelines (Copy activities and Data Flows) with consistent logging, error handling, retries, and notifications
Implement parameterization and reusable components to reduce duplication and accelerate enhancements
Implement incremental load and backfill strategies suited to data volume
Production support and reliability: monitor daily pipeline execution and triage incidents to restore processing
Perform root-cause analysis for failures and recurring issues; implement preventative controls and standardized patterns
Create and maintain operational runbooks for critical pipelines and common support scenarios
Data quality, validation, and trust: build automated validation routines and reconciliation checks; collaborate with analysts to define business rules and quality thresholds
Document data definitions, transformations, and lineage to improve transparency
Stakeholder collaboration and dependency management: work with Operations Data Analysts, business stakeholders, and partner analysts to deliver datasets for Power BI models and other analytics; align dependency readiness signals and timelines; provide technical coaching and mentoring

Key measures of success

Pipelines meet SLAs and deliver on time
Pipeline success rate with reduced manual intervention
Time to detect and resolve ETL failures (MTTD/MTTR)
Improvements in query performance, runtime, and cost efficiency
Reduction in recurring data quality defects; data completeness and accuracy
Documentation coverage, including runbooks and data dictionary completeness

Requirements

Bachelor’s Degree preferred in a related field
6+ years of experience in data engineering, ETL/ELT, or related roles
Hands-on experience building and supporting production pipelines in Azure Data Factory
Strong SQL/T-SQL skills and experience with SQL Server environments supporting analytics and reporting
Advanced data modeling capabilities
Experience integrating data from multiple sources including databases, flat files/SFTP, and APIs/vendor feeds
Familiarity with Azure data platform components (Data Lake/Blob Storage, notebooks) and/or cloud migration planning
Strong troubleshooting, root-cause analysis, and operational ownership mindset
Ability to work directly with stakeholders to gather requirements and translate them into technical solutions
Excellent communication and documentation skills for presenting findings and translating data into understandable documents

Technologies

Azure Data Factory
SQL Server
T-SQL
Data Lake/Blob Storage
Notebooks
Power BI
Git
CI/CD
Python
PowerShell
SFTP

View full job page →

3d ago

Senior Data Engineer

The Hanover Insurance Group Worcester, MA 🇺🇸 Remote

Worcester, MA

3d ago

$100k - $125k/yr Full time Senior Azure Data Factory Azure Data Lake Storage

Apply Now

Job Description

Senior Data Engineer responsible for designing, developing, implementing, and maintaining data integrations, pipelines, and solutions, with a leadership role on cross functional teams, available in Worcester, MA or remote.

Responsibilities

Design, build, deploy, and maintain data integrations, pipelines, and analytics solutions across Azure and on prem environments.
Design and implement data solutions using Azure resources such as Azure Data Factory, Azure Synapse, and ADLS.
Develop data integration, ETL processes, and data warehousing capabilities.
Write and optimize code in SQL, Python, and PowerShell.
Demonstrate expertise in data modeling, governance, and engineering patterns and practices.
Optimize ingestion pipelines and notebooks for scale, performance, reliability, and cost efficiency.
Work with integration connectors including APIs, File transfers, SFTP/FTPS, streaming data, DB connectors, SOAP, and REST.
Bring strong problem solving, communication, and collaboration skills to cross functional teams.
Document data processes and solutions for clarity and reuse.
Maintain data security and ensure compliance with industry standards.
Participate in agile development and DevOps practices.
Use Git or GitHub for CI/CD workflows.
Provide coaching and technical guidance to junior team members.
Profile and analyze large data sets with strong analytical skills.
Apply critical thinking and meticulous attention to detail.
Debug data solution issues and troubleshoot bottlenecks.
Collaborate with data scientists, analysts, and stakeholders to align solutions with needs.
Monitor and maintain data pipelines to ensure reliability.
Design, develop, and test data mappings with transformation rules.
Ensure data quality and reliability across data flows.
Develop and maintain documentation for data integration and ETL solutions.

Requirements

Bachelor’s degree or higher in data science, mathematics, statistics, computer science, information systems, business information technology, or equivalent field.
5+ years of professional experience in data engineering, data architecture, or data integration.
Must be eligible to work in the United States without sponsorship now or in the future (US citizen or green card holder).

Technologies

Azure Data Factory
Azure Synapse
ADLS
SQL
Python
PowerShell
APIs
Git/GitHub
Power BI
Microsoft Fabric

Compensation

Salary: USD 100,000 - 125,000 per year

Benefits

Medical, dental, vision, life, and disability insurance
401K with company match
Tuition reimbursement
PTO
Company paid holidays
Flexible work arrangements
Cultural Awareness Day in support of IDE
On-site medical and wellness center (Worcester only)

Career Development

It is a career oriented role with ongoing support through on the job experiences, coaching, and robust learning and development programs. The company encourages growth at every level to help you succeed and stay fulfilled.

Preferred qualifications

Microsoft DP-203 or DP-700 certification
Microsoft Fabric experience
Power BI experience
Master’s degree in data science, mathematics, statistics, computer science, information systems, business information technology, or equivalent

View full job page →

Azure Data Lake Storage Jobs

PL Ops - Data Engineer

Senior Data Engineer

Get Job Alerts