Databricks

Databricks Jobs

434 jobs found

Today

Data ETL / Data Engineer

UST Richmond, VA

Richmond, VA

Today

$106k - $159k/yr Full time Backend Developer Artificial Intelligence Big Data

Apply Now

Job Description

UST offers a full-time Data ETL / Data Engineer role based in Richmond, VA (onsite), with an annual salary range of $106,000 to $159,000. You will help design and deliver scalable ETL and data pipeline solutions, working across modern data platforms and enterprise databases. UST also supports a culture built on Humility, Humanity, and Integrity and provides benefits including paid time off, insurance, retirement plan eligibility, and HSA/FSA options.

What you’ll do

Design, develop, test, and maintain ETL and data pipeline solutions.
Build scalable data processing workflows using Python and PySpark.
Develop and optimize SQL and PLSQL scripts for data extraction, transformation, and loading.
Design and maintain data warehouse architectures and data models.
Perform performance tuning and query optimization for large datasets.
Work with Databricks, Snowflake, and PostgreSQL environments.
Manage source code repositories using Bitbucket and follow CI/CD best practices.
Use AI-powered development tools to improve productivity and code quality.
Collaborate with business stakeholders, data analysts, and cross-functional teams to deliver data solutions.
Create technical documentation and support production deployments.

What you bring

Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field.
Strong experience in ETL development and data engineering.
Hands-on experience with SQL, Python, and PySpark.
Experience in data warehousing and database performance tuning.
Knowledge of cloud-based data platforms and modern data engineering practices.
Strong analytical, troubleshooting, and problem-solving skills.

Skills and tools you’ll use

Python, PySpark, SQL, PLSQL, and T-SQL
Databricks, Snowflake, PostgreSQL (including AWS RDS PostgreSQL)
Bitbucket and AI coding tools such as GitHub Copilot and Windsurf
Shell scripting
AI-focused development such as AI agents and generative AI, along with AWS cloud services

Benefits

Full-time employees accrue a minimum of 10 days of paid vacation per year
Full-time employees receive 6 days of paid sick leave each year (pro-rated for new hires throughout the year)
10 paid holidays
Eligible for paid bereavement leave and jury duty
401(k) Retirement Plan eligibility with employer matching
Medical, dental, and vision insurance for employees and dependents residing in the US
Company-paid basic life insurance
Company-paid accidental death and disability insurance
Company-paid short- and long-term disability benefits
Eligible to purchase additional voluntary short-term disability benefits
Health Savings Account (HSA) eligibility
Flexible Spending Account (FSA) eligibility

Equal Employment Opportunity

UST is an Equal Opportunity Employer. All qualified applicants will receive consideration without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other applicable characteristics protected by law. UST also considers qualified applicants with arrest or conviction records in accordance with state and local laws and “fair chance” ordinances. UST reserves the right to periodically redefine roles and responsibilities based on organizational requirements and/or performance.

View full job page →

Today

Data Engineer III - Python/PySpark/Databricks/AI

JPMorganChase Atlanta, GA

Atlanta, GA

Today

Full time Python Artificial Intelligence Data

Apply Now

Job Description

Build trusted data products with an enterprise focus as a Data Engineer III in an agile environment. You will design and deliver data collection, storage, access, and analytics solutions, and work with SQL, NoSQL, and AI-enabled capabilities to accelerate pipeline and architecture work while staying aligned to data sensitivity and security requirements. The role is based onsite in Atlanta, GA.

What you’ll do

Support review of controls to help ensure sufficient protection of enterprise data.
Advise and implement custom configuration changes in one to two tools to generate a business or customer-requested product.
Update logical or physical data models as new use cases emerge.
Use SQL frequently, with practical understanding of NoSQL databases and where they fit in real-world deployments.
Apply enterprise-authorized AI capabilities to accelerate data pipeline and design analysis and documentation, validating outputs and handling data according to sensitivity and security requirements.
Use reuse-first, AI-assisted practices to strengthen SDLC-quality routines for data pipelines, including test generation and control validation, with traceability/auditability and alignment to resiliency and security expectations.

What you bring

3+ years of applied data engineering experience, plus formal training or certification in data engineering concepts.
Experience across the data lifecycle.
Advanced SQL, including joins and aggregations.
Working understanding of NoSQL databases.
Demonstrated experience using enterprise-authorized AI capabilities within the work environment to support data engineering workflows, with strong validation habits and awareness of data sensitivity.
Ability to review and validate AI-assisted outputs (for example: query suggestions, test ideas, or model change summaries) before use, escalating when uncertain and following data handling requirements.
Significant experience with statistical data analysis, including selecting appropriate tools and data patterns for analysis.
Experience customizing changes in a tool to generate a product.

Technologies you’ll work with

SQL, NoSQL
AI, Python, PySpark
Databricks

Benefits

Comprehensive health care coverage
On-site health and wellness centers
A retirement savings plan
Backup childcare
Tuition reimbursement
Mental health support
Financial coaching

Preferred capabilities

Exposure to cloud technologies
Exposure to AI-driven development

View full job page →

Today

Lead Data Engineer

Norfolk Southern Corp Atlanta, GA

Atlanta, GA

Today

Full time Big Data Bigdata Cloud Data Engineering

Apply Now

Job Description

Lead enterprise data engineering for a Databricks-based lakehouse and streaming analytics platform at Norfolk Southern Corp. In this hybrid role in Atlanta, GA, you will set technical direction, modernize ingestion and transformation patterns, and help guide steady-state operations while collaborating with cross-functional partners to deliver production-grade data pipelines.

What you’ll do

Lead the definition of data architecture with team members and stakeholders, establishing technical direction for lakehouse, ingestion, and transformation patterns.
Identify and remove technical blockers by using hands-on problem solving, architectural decisions, and coordination across teams.
Design and develop enterprise data models (conceptual, logical, and physical) across domains, ensuring alignment with lakehouse and pipeline architecture.
Define and enforce data modeling standards, best practices, and governance processes using tools such as Erwin.
Define data requirements, then gather, wrangle, and validate large-scale structured and unstructured data using data tools in the data environment.
Enable data standardization, customization, and ad-hoc analysis by building mechanisms to ingest, analyze, validate, normalize, and clean data.
Create data policy and develop interfaces and retention models that may require synthesizing or anonymizing data.
Implement statistical data quality procedures for new data sources, using iterative analytics to support data scientists and analytics teams preparing data for visualization and insight generation.
Develop and maintain data engineering best practices, contributing to insights related to data analytics and visualization.
Lead a data engineering team to use state-of-the-art big data tools to build scalable architectures and high-performance pipelines for both real-time and batch data.
Partner with data science and business intelligence teams to deliver models and pipelines for research, reporting, and machine learning.
Build pipelines that clean, transform, and aggregate data from disparate sources.
Use multiple languages and tools (for example, scripting) to integrate systems, and apply data architecture components from requirements through implementation while leading project teams.

What you bring

Databricks Lakehouse expertise (Required): 10+ years in data engineering with deep hands-on experience across the Databricks ecosystem including Spark/PySpark pipelines, Delta Lake (merges, schema evolution), Delta Live Tables, Unity Catalog governance, Genie for self-service analytics, and LakeFlow Designer for visual ETL orchestration.
Streaming and cloud infrastructure: 4+ years building reliable batch and streaming Spark workloads; 3+ years with Kafka (or Confluent) for high-volume event processing; and 3+ years using AWS analytics services including S3, IAM, Glue/Lambda/MSK.
Data modeling and SQL: ability to build conceptual, logical, and physical data models with strong governance, along with advanced SQL skills for business-ready datasets.
Technical leadership: experience leading technical design, mentoring engineers, building architectural consensus, and unblocking complex data engineering challenges.
Delivery and collaboration: experience delivering ETL/ELT pipelines end-to-end in a lakehouse environment, working within Agile frameworks such as Scrum/Kanban/SAFe to manage iterative delivery and cross-team dependencies.

Tools and technologies

Databricks, Spark, PySpark, Delta Lake, Delta Live Tables, Unity Catalog, Genie, LakeFlow Designer
Kafka, Confluent
AWS analytics services (S3, IAM, Glue/Lambda/MSK)
SQL, Erwin, Python, Scala
Also listed: Snowflake, BigQuery, HBASE, Cassandra, Azure, Kinesis, TIBCO EMS, IBM MQ Series, MSK, Git, REST API, Web Services, NoSQL, ETL/ELT, and Agile frameworks (Scrum/Kanban/SAFe)

Work setup

Location: Atlanta, GA, US
Schedule: Hybrid (two days in office, three days remote)
Shift work: No
On-call: Yes
Weekend work: No

Minimum education

Bachelor’s Degree, preferably in Information Systems, Computer Science, Computer Information Systems, or a related technology field.

View full job page →

Today

Enterprise Data Engineer

Avera Health Sioux Falls, SD

Sioux Falls, SD

Today

$74k - $112k/yr Full time Backend Developer Azure Azure Data Factory

Apply Now

Job Description

Join Avera Health onsite in Sioux Falls, SD to help build and support enterprise data systems for reporting and analytics.

Responsibilities

Design and develop enterprise data systems, including data warehouse and data lake
Assist in building and maintaining ETL processes
Create and support data pipelines that move data from source systems into storage platforms
Perform data cleaning and preprocessing to support high data quality
Monitor and troubleshoot data pipeline performance and address issues as they arise
Maintain system logs and help ensure smooth operation of data workflows
Assist with maintaining and upgrading existing data infrastructure (including databases and data warehouses)
Collaborate with data scientists, analysts, and business stakeholders to understand data needs and deliver solutions
Document data workflows, pipelines, and technical processes
Keep clear, organized records for data sources and transformations

Requirements

Bachelor’s degree in computer science, analytics, mathematics, business, or a related field (or equivalent combination of education and work experience)
2-3 years of relevant experience
Knowledge of SQL and Python
Familiarity with relational databases, including SQL Server
Strong understanding of SQL for querying and manipulating relational data
Exposure to data processing tools and frameworks across on-premises and cloud environments (examples include PySpark, ETL, SSIS, and Azure Data Factory)
Ability to work effectively with cross-functional teams (data scientists, analysts, and engineers)
Strong attention to detail and ability to follow standard operating procedures
Familiarity with cloud platforms such as Databricks (AWS, Azure, GCP)
Willingness to learn and develop skills in data engineering technologies

Technologies

SQL, Python
SQL Server
PySpark
ETL
SSIS
Azure Data Factory
Databricks
AWS, Azure, GCP

Benefits

PTO available day 1 for eligible hires
Up to 5% employer matching contribution for retirement
Career development with hands-on training and mentorship

Preferred Qualifications

Healthcare experience, with preference for Epic Electronic Health Record (Epic EHR)

Essential Qualifications

Ability to work the specified hours for the position
Visual acuity adequate to perform position duties
Ability to communicate effectively with others, hear, understand, and distinguish speech and other sounds
Reasonable accommodations may be available to enable individuals with disabilities to perform essential job functions, as long as accommodations do not cause undue hardship to the employer

Salary range: USD 73,840 - 111,800 per year

Location: Sioux Falls, SD (onsite)

View full job page →

Today

Technical Data Analyst

IDR Inc. Chicago, IL

Chicago, IL

Today

$50 - $53/hr Full time Azure Data Analysis Data Analytics

Apply Now

Job Description

Join IDR, Inc. as a Technical Data Analyst supporting data governance and modernization efforts within the financial services space. This onsite role is based in Chicago, IL, partnering with teams to strengthen data quality, maintain data lineage, and deliver end-to-end requirements for evolving financial data systems.

Working at the intersection of analytics and platform modernization, you will use SQL and ETL practices alongside modern data tooling to validate solutions and help ensure that data stays reliable across connected systems.

Role Summary

The position supports finance transformation projects and data governance initiatives for a client in Chicago, Illinois. The work focuses on improving data quality and traceability across systems, including legacy environments, while contributing to the implementation and modernization of financial data platforms.

Responsibilities

Provide data analysis support for finance transformation projects and data governance initiatives.
Support the implementation and modernization of financial data systems, including end-to-end requirement delivery.
Collaborate with cross-functional teams to maintain accurate data lineage and data quality across systems.
Use SQL, ETL, and modern data platform capabilities to develop and validate data solutions.
Lead workstreams independently, managing issues and communicating technical details to stakeholders.

Requirements

6+ years of experience as a technical data analyst.
Demonstrated expertise in finance modernization, including experience with subledgers, GAAP reporting, and accounting systems.
Strong hands-on skills in SQL, ETL development, and data transformation.
Experience with Databricks, medallion architecture, and Azure-based data platforms.
Ability to create and manage data lineage across multiple systems, including legacy environments.

Technologies

SQL
ETL
Databricks
Medallion architecture
Azure

Compensation and Benefits

Salary: USD 50 - 53 per hourly
Competitive compensation package
Full Benefits including Medical, Vision, Dental, and more
Medical, Dental, Vision, and Life Insurance
Employee Stock Ownership Program

Why IDR

25+ Years of proven industry experience across 4 major markets
Dedicated Engagement Manager committed to your success
ClearlyRated's Best of Staffing® Client and Talent Award winner for 12 years in a row

What’s in It for You

Opportunity to join an industry-leading organization while contributing directly to data governance and modernization outcomes.

View full job page →

Today

Data Engineer & Automation Developer

Guidehouse Inc. Arlington, VA

Arlington, VA

Today

Full time Backend Developer Python Api Integration

Apply Now

Job Description

Guidehouse Inc. is seeking a Data Engineer & Automation Developer to help design and maintain automated, data-driven workflows and integrations. This onsite role in Arlington, VA supports client-focused initiatives that combine Python and Databricks data automation with Microsoft Power Platform solutions and API-based data cataloging.

Role Summary

In this position, you will build and maintain workflows that ingest and transform data from multiple sources and file types, support data cataloging through APIs, and create authentication workbooks that enable connections between data sources and cataloging tools. You will also develop Power Apps solutions within the Microsoft Power Platform and communicate technical findings to both technical and non-technical audiences, including leadership.

Responsibilities

Use strong technical skills and a mission-focused mindset to understand client needs, identify opportunities, and design data-driven solutions.
Leverage Python and Databricks to create and maintain automated workflows for data cataloging via APIs, including ingesting and transforming data from various sources and file types.
Create and maintain authentication workbooks that support connections from data sources to cataloging tools.
Use the Microsoft Power Platform to create and maintain Power Apps solutions.
Communicate clearly to translate technical findings for technical and non-technical audiences, including leadership.
Engage directly with clients to gather requirements, adapt solutions to mission needs, and move initiatives forward.

Required Qualifications

ACTIVE and MAINTAINED “SECRET” Federal or DoD security clearance.
Bachelor’s degree.
Two (2) years of relevant professional experience in data analysis, data science, data engineering, or data visualization.
Experience creating and maintaining automated workflows using Python in Databricks to catalog data using APIs for ingesting and transforming data from various sources and file types.
Experience creating and maintaining authentication workbooks for connections from data sources to cataloging tools such as Alation.
Experience using Microsoft Power Platform and creating and maintaining Power Apps.
Ability to support remote, hybrid, or on-site projects in the Washington D.C. metro area.

Technologies

Python
Databricks
Microsoft Power Platform
Power Apps
APIs
Alation

Benefits

Medical, Rx, Dental & Vision Insurance
Personal and Family Sick Time & Company Paid Holidays
Position may be eligible for a discretionary variable incentive bonus
Parental Leave and Adoption Assistance
401(k) Retirement Plan
Basic Life & Supplemental Life
Health Savings Account, Dental/Vision & Dependent Care Flexible Spending Accounts
Short-Term & Long-Term Disability
Student Loan PayDown
Tuition Reimbursement, Personal Development & Learning Opportunities
Skills Development & Certifications
Employee Referral Program
Corporate Sponsored Events & Community Outreach
Emergency Back-Up Childcare Program
Mobility Stipend

Clearance and Travel

Clearance required: Active Secret
Travel required: None

What Would Be Nice to Have

Three (3) or more years of relevant professional experience.
Experience working in Databricks.
Familiarity with Agile team operations, such as sprint planning, standup calls, writing Jira stories, and backlog refinement.
Ability to work independently, proactively identify opportunities to add value to the client, and drive initiatives and innovation.
Ability to clearly communicate and present technical solutions, insights, and results to technical and non-technical audiences, including leadership.
Experience performing data engineering, such as Extract-Transform-Load (ETL) and managing databases.
Experience working in the Microsoft Power Platform and creating and maintaining Power Apps.
Experience working with data cataloging and metadata.
Experience working with APIs.
Familiarity with Alation.
Ability to support business development including RFP, RFQ, and RFI responses and white papers.
Relevant experience supporting Department of State or other Federal Government organizations.

View full job page →

Today

Azure Data Engineer

NSD International, Inc. Charlotte, NC

Charlotte, NC

Today

$74+/hr Contract Backend Developer Python Analytics

Apply Now

Job Description

NSD International, Inc. is hiring an Azure Data Engineer for contract work supporting an Azure-based data platform.

Responsibilities

Support the active development of an Azure data platform in close coordination with the team
Begin with establishing a reference data system for securities and pricing data, then expand to additional data domains
Follow internal development standards to contribute to the overall Data Strategy agenda
Implement Azure Cloud Platform solutions and apply relevant know-how across the qualifications listed below

Requirements

10+ years of software development experience using Python and PySpark and associated frameworks
Proven background as a Data Engineer and/or Data Quality Engineer with Azure cloud experience
Experience implementing solutions with Azure cloud services
Hands-on experience implementing Azure Data Factory
Hands-on experience implementing Azure Lake Gen 2
Hands-on experience implementing Azure Databases
Hands-on experience implementing Azure Databricks
Experience implementing a Data Quality Framework/Solution
Experience implementing Collibra Data Quality or other data quality tools including IDQ, IBM, or Ab-Initio
Experience implementing Collibra Data Governance with knowledge of data lineage
Experience implementing Power BI
Well versed with Azure Databricks
Strong SQL skills with RDMs or noSQL databases
Familiarity with the DevOps lifecycle including git and Jenkins
Experience with CI/CD processes
Good understanding of ETL/ELT processes
Financial services industry experience is a plus, including financial instruments, asset classes, and market data