DataJobs.io - Jobs in data science, machine learning and analytics.

Latest Jobs

17d ago

Data Engineer

Builders Atlanta, GA

Atlanta, GA

17d ago

Full time Azure Data Engineer Azure Data Lake Big Data

Apply Now

Job Description

Build, govern, and optimize Azure Databricks lakehouse pipelines supporting property and casualty insurance data migrations.

Responsibilities

Design and build pipelines for the phased migration from Guidewire (PolicyCenter, ClaimCenter, BillingCenter) and legacy systems into the Azure Databricks lakehouse
Implement historical loads, incremental/CDC (Change Data Capture) loads, and reconciliation
Create bronze and silver layer transformations in PySpark and Spark SQL following the medallion architecture on Delta Lake
Support data modeling for property and casualty insurance, including policies, insured risks and exposures, coverages, endorsements, premiums, claims, reserves, payments, and recoveries
Integrate third-party data sources such as geospatial and weather data
Schedule and run production pipelines using Databricks Workflows (Lakeflow Jobs) or Azure Data Factory, including dependencies, parameters, retries, and alerting
Investigate and resolve pipeline failures
Apply Unity Catalog controls defined by the Data Engineering Lead and Data Governance lead, including permissions, PII (Personally Identifiable Information) masking, tags, and lineage
Implement data quality checks, error handling, and monitoring
Tune pipeline performance and cost using partitioning, liquid clustering, and right-sized job compute
Use Git version control (GitHub or Bitbucket) and CI/CD with Databricks Asset Bundles to version, review, and deploy code across development, test, and production
Write documentation and runbooks, participate in code reviews, and gradually take ownership of contractor-built pipelines
Perform other duties as assigned

Requirements

Associate degree or equivalent combination of education and experience
Databricks engineering certification
3+ years in data engineering, including 1+ years hands-on with Databricks (PySpark, Spark SQL, Delta Lake)
Experience with property and casualty insurance data, ideally Guidewire data, and experience migrating from legacy source systems
Knowledge of Python/PySpark and SQL (Structured Query Language)
Knowledge of Unity Catalog: permissions, row/column-level security, and lineage
Experience with incremental loading patterns such as CDC, MERGE, SCD (Slowly Changing Dimension), and Auto Loader
Knowledge of data modeling, including dimensional modeling
Experience scheduling and monitoring pipelines with Databricks Workflows and/or Azure Data Factory
Experience with Git-based version control (GitHub or Bitbucket), including branching, pull request reviews, and Databricks Git folders
Experience with Azure data services including ADLS (Azure Data Lake Storage) Gen2, Azure Data Factory, and Key Vault
Knowledge of CI/CD basics (Databricks Asset Bundles, Azure DevOps, GitHub Actions, or Bitbucket Pipelines)
Experience with streaming (Structured Streaming, Event Hubs)
Experience with API and integration tools such as MuleSoft (added advantage)

Technologies

Azure Databricks
Delta Lake
Unity Catalog
Lakeflow
Databricks SQL
PySpark
Spark SQL
Databricks Workflows
Azure Data Factory
Git, GitHub, Bitbucket
Databricks Asset Bundles
Python
SQL (Structured Query Language)
ADLS (Azure Data Lake Storage) Gen2
Azure DevOps
GitHub Actions
Bitbucket Pipelines
Key Vault
Structured Streaming
Event Hubs
Auto Loader
MERGE
SCD (Slowly Changing Dimension)
CDC (Change Data Capture)
MuleSoft
API (Application Programming Interface)

Benefits

Competitive Salary
Bonus Structure
Profit Sharing
Medical, Dental, Vision Insurance
Employer Paid Short Term Disability
Employer Paid Long Term Disability
Employer Paid Life Insurance
Voluntary Life Insurance
401K with Company Match
PTO

Location and Experience

Atlanta, GA (onsite)
Minimum experience: 3 years
Education: Associate degree

Equal Opportunity Employer

Builders welcomes applicants from all walks of life and does not discriminate based on protected status.

View full job page →

17d ago

Principal Data Engineer

Disney Entertainment and ESPN Product & Technology New York, NY

New York, NY

17d ago

$188k - $252k/yr Full time Application Security Aws Data Engineering Big Data

Apply Now

Job Description

Disney Entertainment and ESPN Product & Technology is hiring a Principal Data Engineer to lead Identity and Device data engineering for marketing measurement and analytics.

Responsibilities

Define and drive technical strategy for Identity & Device data products in Marketing and Consumer Data, including pipeline architecture, platform patterns, and a modernization roadmap in partnership with onshore squad leaders and enterprise architecture.
Own end-to-end delivery for complex, high-impact initiatives spanning Identity360, device/consumer identifier datasets, and dependent Marketing data products, including technical approach, sequencing, and engineering quality standards for other engineers.
Act as principal subject matter expert and escalation point for identity foundational data, covering data modeling, batch and near-real-time processing, performance at scale, and production reliability.
Participate in an on-call rotation and lead incident response for critical pipelines.
Set and enforce engineering standards for the Identity & Device team and Marketing Data Engineering, including code quality, CI/CD, observability, and data governance aligned to GDPR, RTBF, and CCPA, along with reusable platform patterns across Databricks, AWS, and orchestration tooling.
Mentor and develop engineers through design reviews, pairing, and coaching, building depth in big-data engineering, identity domain knowledge, and operational excellence.
Collaborate cross-functionally with Product, QA, Operations, Privacy/Legal, and peer Marketing data squads (Data Collection, Activation, Acquisition) to align requirements, resolve cross-domain dependencies, and represent technical decisions to stakeholders.

Requirements

10+ years of related big data engineering experience, including modeling and building large data pipelines; strong experience with Python, Scala, or Java; strong AWS fundamentals (S3, Lambda, IAM) preferred.
5+ years hands-on experience with performance tuning at scale using distributed systems such as Hadoop or Databricks, with query/process tools such as Spark, Presto, and Hive at the petabyte level.
Experience with at least one major MPP or cloud database technology: Snowflake, Redshift, BigQuery, or Databricks.
Solid understanding of data modeling and data warehousing standard methodologies and practices.
Operational experience in a multi-system/services production environment as a senior data engineer and point-of-contact/SME.
Strong problem-solving with attention to detail, plus excellent analytical and communication skills.
Experience with data integration and orchestration toolsets (e.g., Airflow) and CI/CD.
Demonstrated ability with cloud infrastructure technologies, including Terraform.
Strong understanding of slowly changing dimensions (SCD tables) and building scalable, performant datasets.
Bachelor’s degree in computer science, Information Systems, Software, Electrical or Electronics Engineering, or comparable field of study, and/or equivalent work experience.

Technologies

Python, Scala, Java
AWS, S3, Lambda, IAM
Hadoop, Databricks, Spark, Presto, Hive
Snowflake, Redshift, BigQuery
Airflow, CI/CD, Terraform
GDPR, RTBF, CCPA

Benefits

Bonus and/or long-term incentive units may be provided as part of the compensation package.
Full range of medical, financial, and/or other benefits.

Location: New York, NY (onsite)

Compensation: USD 187,900 - 252,000 per year

Experience: 10+ years

PAY TRANSPARENCY: The hiring range for this position in SFO, CA is $187,900 - $252,000 per year and New York is $179,700 - $241,000 per year.

Note: This position is with Disney Entertainment & Sports LLC, which is part of a business we call Disney Entertainment and ESPN Product & Technology.

View full job page →

17d ago

Sr Data Engineer

Disney Experiences Anaheim, CA

Anaheim, CA

17d ago

$144k - $193k/yr Full time Senior 3d Design Tools Apache Airflow

Apply Now

Job Description

Disney Experiences (DX) Tech & Digital is seeking a Senior Data Engineer for the Data Products & Platforms team. The role focuses on building and delivering scalable data warehousing and ELT solutions for content performance and engagement data using AWS and Snowflake.

Responsibilities

Support the design and growth of Data Products and Data Warehouses centered on Content Performance and Content Engagement data.
Design and develop scalable data warehousing solutions, including building ELT pipelines in big data environments (cloud, on-premise, hybrid).
Use the DX data stack, including AWS and Snowflake, along with Data Build Tool (DBT) and Automic UC4/Airflow, to transform, store, retrieve, and distribute Cast & Operations data.
Help architect data solutions and frameworks, using dimensional modeling to define data assets within a medallion structure.
Collaborate with Data Product Managers, Data Architects, and Data Engineers to design, implement, and deliver successful data solutions.
Maintain detailed documentation of work and changes to support data quality and data governance.
Ensure high operational efficiency and quality to meet SLAs, with a dedicated focus on customers including Data Science, Data Analytics, and Business teams.
Participate actively in agile/scrum practices to support team health and process improvements.

Requirements

5+ years of data engineering experience building large data pipelines.
Strong SQL skills, including writing queries to extract data and create performant datasets.
Hands-on experience with AWS services for ingesting process data, including Lambda, Kinesis, DMS, AppFlow, and EventBridge.
Experience with at least one major MPP or cloud database technology: Snowflake, Redshift, or BigQuery.
Expert-level experience with SQL/T-SQL and the ability to create and tune queries.
Agile and Scrum experience.
Strong attention to detail with excellent analytical, problem-solving, and communication skills.
Solid experience with data integration toolsets such as Airflow, including writing and maintaining data pipelines.
Familiarity with data modeling techniques and data warehousing methodologies and practices.

Required Education

Bachelor’s degree in Computer Science, Information Systems, Software, Electrical or Electronics Engineering, or a comparable field of study, and/or equivalent work experience.

Preferred Education

Master’s degree in Computer Science, Information Systems, Software, Electrical or Electronics Engineering, or a comparable field of study, and/or equivalent work experience.

Technical Stack

AWS, Snowflake, Data Build Tool (DBT), Automic UC4, Airflow, ELT
Lambda, Kinesis, DMS, AppFlow, EventBridge
Redshift, BigQuery, SQL, T-SQL

Compensation and Pay Transparency

The hiring range for this position in Anaheim, CA is $144,100.00-$193,200.00 per year.
Base pay offered may consider internal equity and may vary based on candidate geographic region, job-related knowledge, skills, and experience, and other factors.
A bonus and/or long-term incentive units may be provided in addition to medical, financial, and/or other benefits.

Location

Anaheim, CA (onsite)

View full job page →

17d ago

Senior Financial Reporting and Analytics Analyst

Vaco LLC New York, NY

New York, NY

17d ago

$125k - $150k/yr Full time Senior Analytical Skills Analytics

Apply Now

Job Description

Senior Financial Reporting and Analytics Analyst role supporting financial reporting, accounting, and analytics for a publicly traded technology-forward financial services firm in New York, NY (hybrid).

Responsibilities

Support internal and external financial reporting, ensuring timely and accurate delivery of key reporting packages
Prepare recurring management reports, dashboards, and financial analyses to support business decision-making
Analyze financial and operational data to identify trends, explain variances, and deliver actionable insights
Work with large datasets from multiple sources to improve reporting accuracy, consistency, and transparency
Develop and maintain reporting tools, dashboards, and data visualizations using business intelligence and analytics platforms
Assist with month-end, quarter-end, and year-end reporting processes, including audit support and documentation requests
Partner with accounting, finance, technology, and operations teams to improve reporting processes and data quality
Identify opportunities for automation, reporting enhancements, and process efficiencies
Support system implementations, reporting initiatives, and finance transformation projects
Perform ad hoc analyses and special projects as needed

Requirements

Bachelor’s degree in Accounting, Finance, Business, Analytics, or a related field
3+ years of experience in accounting, audit, financial reporting, FP&A, finance transformation, or a related function
Strong understanding of financial statements, accounting principles, and reporting processes
Experience supporting SEC financial reporting, management reporting, and/or business analytics initiatives
Advanced Excel skills and experience working with large datasets
Familiarity with SQL, Tableau, Power BI, or other reporting and business intelligence tools (preferred)
Strong analytical, problem-solving, and data reconciliation skills
Ability to communicate findings clearly to financial and non-financial audiences
Detail-oriented with strong organizational skills and ability to manage multiple priorities
CPA or progress toward CPA designation is a plus
Big 4/Public Accounting background, plus Financial Services and SEC reporting experience

Technology Stack

Excel
SQL
Tableau
Power BI

Compensation: USD 125,000 - 150,000 per year

View full job page →

18d ago

Senior Data Engineer

The Hanover Insurance Group Remote 🌍 Remote

Remote

18d ago

$105k - $130k/yr Full time Senior Data Engineer Data Engineering

Apply Now

Job Description

The Senior Data Engineer will lead end-to-end design, development, implementation, and optimization of enterprise data integration and warehousing solutions. This role supports remote work and can be based in Worcester, MA, with collaboration across internal teams, external vendors, and multiple database and data lake environments.

Responsibilities

Lead a team of developers, including internal members and vendor resources, to deliver end-to-end enterprise data integration and warehousing solutions.
Design, build, and maintain robust ETL pipelines to bridge source systems with target analytics platforms and support scalable data models.
Partner with external vendors and collaborate with data architects, business analysts, and reporting teams to translate complex business requirements into high-performing technical solutions across Oracle, SQL Server, DB2, and data lake environments.
Coordinate with business stakeholders to define work priorities and act as a data and technology lead on cross-functional teams delivering data and reporting solutions.
Support data operations alongside external vendor teams.
Design, implement, and test complex data mappings and transformations using Informatica PowerCenter.
Optimize ingestion pipelines for scale, performance, reliability, and cost efficiency.
Perform SQL performance tuning, including partitioning, Pushdown Optimization (PDO), and stored procedures in Oracle, SQL Server, and DB2.
Use Linux scripting and enterprise job scheduling software to support and automate data workflows.
Work with diverse integration connectors, including APIs, files, SFTP/FTPS, Streaming Data, database connectors, SOAP, and REST.
Implement data governance and architecture practices, including functional data modeling knowledge, governance frameworks, and metadata management strategies.
Enforce data quality, data security protocols, and strict industry compliance standards.
Provide operations and troubleshooting expertise for large data sets, including debugging and continuous monitoring and maintenance of active data pipelines.
Deliver team leadership through professional communication, mentoring, coaching, and guidance for junior development team members.
Document data processes, workflows, and complete technical ETL solutions.

Requirements

Bachelor’s degree or higher in Computer Science, Information Systems, Business Information Technology, or an equivalent field.
5+ years of professional, hands-on experience in data engineering, data architecture, or data integration.
Must be eligible to work in the United States without requiring sponsorship now or in the future (US Citizen or Green Card holder).

Technologies

Informatica PowerCenter
ETL (Extract, Transform, Load)
SQL
Oracle, SQL Server, DB2
Data lake
Linux scripting
APIs, SFTP, FTPS
Streaming Data, SOAP, REST
Git/GitHub
CI/CD pipelines
Power BI, OBIEE, Hyperion Essbase

Benefits

Medical, dental, vision, life, and disability insurance
401K with a company match
Tuition reimbursement
PTO
Company paid holidays
Flexible work arrangements
Cultural Awareness Day in support of IDE
On-site medical/wellness center (Worcester only)

Bonus Points

Experience working within Agile development methodologies and modern DevOps environments
Hands-on experience using Git/GitHub for version control and CI/CD pipelines
Familiarity with BI reporting tools like Power BI, OBIEE, Hyperion Essbase
Master’s degree in Information Systems, Business Information Technology, or an equivalent field

Additional Job Details

Job Family: Data - Data Engineering
Job Function: Data and Analytics
Pay Type: Salary
Education Level: Bachelor’s Degree
Salary Range: USD 105,000 - 130,000 per yearly
Minimum Rate: 105,000 USD
Maximum Rate: 130,000 USD
Location: Remote work arrangement; Worcester, MA, USA

View full job page →

18d ago

Streaming Data Engineer

Bright Vision Technologies Remote 🇺🇸 Remote

Remote

18d ago

$78k - $105k/yr Full time Big Data Cloud Streaming Data

Apply Now

Job Description

Bright Vision Technologies is hiring a Streaming Data Engineer for a 100% remote role in the U.S. This position focuses on building and running large-scale Apache Kafka and Confluent Platform environments for mission-critical, event-driven workloads. You will own the Kafka platform end-to-end, with responsibility across performance, security, automation, observability, and developer enablement.

What you’ll do

Architect, deploy, and operate large-scale Apache Kafka and Confluent Platform for mission-critical, event-driven workloads.
Own the Kafka platform end-to-end, including cluster sizing, configuration, security, automation, observability, and developer enablement.
Partner with application teams to deliver a reliable, high-performing, and developer-friendly streaming platform.
Collaborate with cross-functional partners across product, design, engineering, operations, and business stakeholders to translate ambiguous requirements into well-engineered solutions.
Raise the bar through code review, design review, and mentorship of more junior engineers.

What you bring

Bachelor’s degree in Computer Science, Engineering, or a related technical discipline.
5+ years of experience operating Apache Kafka or Confluent Platform in production.
Deep, hands-on knowledge of Kafka internals: partitions, replication, ISRs, and consumer groups.
Strong experience with Kafka security: SASL, mTLS, ACLs, and RBAC.
Hands-on experience with Kafka Connect, Schema Registry, and either Kafka Streams or ksqlDB.
Experience implementing HA/DR strategies for Kafka.
Strong scripting skills in Python, Bash, or Go.
Hands-on experience with infrastructure-as-code using Terraform and Ansible.
Working knowledge of observability tooling for Kafka.
Excellent troubleshooting, communication, and documentation skills.

Nice to have

Confluent Certified Administrator or Developer credentials.
Experience operating Kafka on Kubernetes using Strimzi or Confluent Operator.
Exposure to managed Kafka services such as AWS MSK or Azure Event Hubs Kafka API.
Familiarity with stream processing frameworks including Flink or Spark Streaming.
Experience with data governance and lineage for streaming data.

Compensation & location

Location: Remote (U.S.)

Salary: USD 78,000 - 105,000 per year

Visa sponsorship

U.S. Citizens, Green Card Holders, EAD Holders, and H-1B transfer candidates are encouraged to apply. We are unable to sponsor new H-1B visa petitions for this position.

How to apply

For immediate consideration, send your resume to hilda@bvteck.com.

View full job page →

18d ago

Business Intelligence Analyst / Data Engineer

GEICO Bethesda, MD

Bethesda, MD

18d ago

$91k - $172k/yr Full time Apache Superset Business Intelligence Data Analytics

Apply Now

Job Description

At GEICO in Bethesda, MD, you’ll design and support enterprise business intelligence and visual analytics solutions that help teams make data-driven decisions. This role uses Microsoft Power BI as the primary platform, alongside collaboration with stakeholders and data engineering partners to deliver scalable reporting, semantic models, and actionable analytical recommendations. You’ll also have opportunities to grow through personalized development programs, mentorship, and certification assistance, supported by competitive pay, benefits, and flexibility.

Responsibilities

Design end-to-end BI solutions, including semantic and metric layers, dimensional data models, reports, dashboards, scorecards, and visual analytics products.
Develop, maintain, and optimize Power BI semantic models, reports, and dashboards using DAX, Power Query, and enterprise reporting standards.
Apply advanced analytical and statistical techniques to identify trends, performance drivers, customer behaviors, and improvement opportunities.
Move from descriptive reporting to diagnostic insight, and introduce predictive approaches when aligned with the business question and available data.
Partner with stakeholders to define analytical questions, evaluate findings, and translate complex analysis into clear recommendations.
Establish and promote semantic modeling standards, reusable design patterns, visualization standards, and reporting best practices.
Support report certification by validating solution quality, consistency, usability, maintainability, performance, and documentation, including adherence to standards.
Publish and support Power BI solutions using Power BI Service capabilities such as Workspaces, Apps, Deployment Pipelines, and semantic model refresh processes.
Optimize report and semantic model performance, refresh reliability, storage-mode selection, and overall user experience across BI platforms.
Assess the downstream impact of design decisions involving Import, DirectQuery, composite models, Row-Level Security (RLS), refresh strategies, and available Power BI capacity.
Evaluate visualization technologies and select appropriate tools including Power BI, Apache Superset, or others based on business needs, scalability, maintainability, and user experience.
Use Snowflake and other approved enterprise data sources to deliver reliable, scalable reporting and analytics.
Coordinate with data engineering and technology partners to ensure source data is available, reliable, understandable, and usable for reporting.
Document report definitions, calculations, assumptions, semantic models, technical designs, and support procedures.
Lead improvements to reporting processes, including automation of recurring outputs and reduction of manual effort.
Support report migration and modernization efforts, including conversion of legacy reporting solutions to modern BI platforms and reusable semantic model architectures.
Perform testing, validation, deployment, troubleshooting, root cause analysis, performance tuning, and post-implementation support.
Participate in agile planning, backlog refinement, estimation, status reporting, and cross-functional delivery coordination.
Provide technical guidance through design, DAX, query, and solution reviews.

Requirements

Advanced experience designing, developing, deploying, and supporting business intelligence solutions using Microsoft Power BI or comparable enterprise visualization platforms.
Strong proficiency with DAX, semantic modeling, dimensional modeling, Power Query, data visualization, and enterprise BI architecture.
Experience publishing and supporting Power BI solutions through Workspaces, Apps, Deployment Pipelines, and semantic model refresh processes.
Advanced SQL skills, including writing, troubleshooting, and optimizing complex queries used in reporting and analytics solutions.
Ability to translate complex technical and analytical concepts for both technical and non-technical audiences.
Strong analytical, quantitative, problem-solving, organizational, communication, and interpersonal skills.
Ability to work independently and collaboratively in an agile delivery environment.
Ability to manage multiple priorities and deliver reliable solutions with minimal guidance.
Business Intelligence Analyst III: 4-7 years of relevant business intelligence, analytics, reporting, semantic modeling, or data visualization experience.
Business Intelligence Analyst IV: 7-10 years of relevant business intelligence, analytics, reporting, semantic modeling, or data visualization experience.
Experience developing enterprise dashboards, reports, semantic models, and visual analytics solutions.
Experience partnering with stakeholders to define business questions, gather requirements, assess findings, and deliver actionable reporting solutions.
Experience working with enterprise data platforms and large-scale datasets.

Technology

Microsoft Power BI, Apache Superset, DAX, Power Query
Power BI Service, Workspaces, Apps, Deployment Pipelines
Snowflake, SQL
Import, DirectQuery, composite models, Row-Level Security (RLS)
semantic modeling, dimensional modeling, Incremental refresh, Git

Benefits

Competitive pay, benefits, and flexibility to support well-being and the future.
Personalized development programs, mentorship, and certification assistance.

Level Expectations

Business Intelligence Analyst III: Designs end-to-end BI solutions, including semantic and metric layers, data models, reports, and dashboards.
Business Intelligence Analyst III: Applies advanced analytical and statistical techniques to progress from descriptive reporting toward diagnostic insight of performance drivers and customer behavior, introducing predictive approaches when appropriate.
Business Intelligence Analyst III: Works independently with minimal guidance, leads solution components, partners with stakeholders to frame analytical questions, and provides technical guidance to peers.
Business Intelligence Analyst IV: Leads the design of complex, reusable BI solutions and serves as a technical lead for reporting and analytics initiatives.
Business Intelligence Analyst IV: Balances delivery responsibilities with leadership of cross-functional projects, establishes semantic modeling and reporting standards, reviews technical designs, influences BI architecture and tooling decisions, and mentors analysts and developers across levels.

Preferred Qualifications

Microsoft Certified: Power BI Data Analyst Associate (PL-300).
Microsoft Power BI administration certification or equivalent demonstrated knowledge of Power BI administration concepts.
Experience with Microsoft Fabric features such as Lakehouses, Warehouses, Dataflows Gen2, Data Pipelines, Notebooks, Real-Time Intelligence, and related capabilities.
Experience using Snowflake as a primary enterprise data source for reporting and analytics.
Experience with Apache Superset or other BI and visualization platforms, including selecting the right tool for the business use case.
Experience developing or supporting Apache Airflow workflows.
Experience with Power BI Project (PBIP) formats and collaborative development of Power BI assets.
Experience using Git-based version control, including branching, pull requests, code reviews, and collaborative development workflows.
Knowledge of Power BI administration and architecture concepts, including capacity management, storage modes, licensing, workspaces, tenant considerations, governance, and performance optimization.
Understanding of performance and capacity considerations involving DirectQuery, Import mode, composite models, incremental refresh, and Row-Level Security (RLS).
Experience using AI-assisted development tools such as Cursor, Claude, GitHub Copilot, or similar platforms to generate or refine scripts, troubleshoot solutions, create documentation, support testing, and accelerate analytics delivery.
Working knowledge of Python, scripting, or related programming concepts, with emphasis on effectively reviewing, validating, and applying AI-generated code.
Experience supporting report certification, semantic modeling standards, and reporting modernization initiatives.
Experience with Azure DevOps or comparable agile work management and delivery platforms.
Property and Casualty insurance, claims analytics, finance analytics, or enterprise reporting experience.