Databricks Genie

Databricks Genie Jobs

5 jobs found

22d ago

Lead AI Engineer

Vytwo Dallas, TX

Dallas, TX

22d ago

Full time Manager Artificial Intelligence Azure

Apply Now

Job Description

Vytwo is building production-ready AI solutions and is hiring a Lead AI Engineer to help design and deploy end-to-end data and AI pipelines on Azure Databricks and other Azure cloud services. This onsite role in Dallas, TX focuses on moving AI from experimentation to reliable production systems, including deployment, monitoring, and optimization across ML and Generative AI use cases.

What you will do

Design and build end-to-end data and AI pipelines using Azure Databricks.
Develop robust ETL/ELT workflows with Python (including PySpark) and SQL.
Implement CI/CD pipelines for Databricks deployments, including jobs, notebooks, and workflows.
Integrate Databricks with Azure services such as Data Lake, Blob Storage, Key Vault, Azure OpenAI, and Azure Functions.
Optimize Databricks jobs for performance, cost, and reliability.
Create reusable, modular code to support maintainable production systems.
Collaborate with data scientists and platform teams to move models from experimentation to production deployment.
Implement logging, monitoring, and error handling for production pipelines.
Develop and deploy ML and Generative AI models, including LLMs, embeddings, and RAG pipelines for NLP, computer vision, and predictive analytics.
Fine-tune LLMs using LoRA or QLoRA, and integrate with Azure OpenAI or Hugging Face models.
Build vector search and retrieval pipelines using FAISS or Azure Cognitive Search.
Support responsible AI practices, including bias detection and model governance.

What you bring

Hands-on experience with Azure Databricks, including jobs, workflows, and clusters (with Unity Catalog preferred).
Python skills with strong emphasis on PySpark rather than only pandas.
SQL experience, including complex joins, window functions, and analytical queries.
Working knowledge of Azure cloud components such as ADLS Gen2, ADF, Key Vault, and IAM concepts.
Experience with pipeline orchestration and deployment, including CI/CD and environment promotion.
Experience with Azure DevOps.
Strong understanding of the ML lifecycle and MLOps best practices.
Experience deploying models using MLflow or similar frameworks.

Technologies you will work with

Azure Databricks; Databricks workflows, jobs, and notebooks
Azure cloud infrastructure including Azure Data Lake, Blob Storage, Key Vault, Azure OpenAI, Azure Functions
Python, PySpark, SQL; ETL/ELT
CI/CD pipelines; Logging and Monitoring
ML and Generative AI; MLflow
LoRA, QLoRA, Hugging Face models
FAISS and Azure Cognitive Search; vector search and RAG pipelines
Unity Catalog; ADLS Gen2; ADF; IAM concepts; Azure DevOps
NLP, computer vision, predictive analytics

Strong advantage

Experience with ML and Generative AI workloads on Databricks.
Experience with RAG, embeddings, or inference pipelines.
Terraform / ARM / Bicep experience for infrastructure.
Databricks Asset Bundles experience.
Airflow or ADF orchestration experience.
Production monitoring and cost optimization experience.
Knowledge of LangChain or similar frameworks for AI application development.
Experience with Azure AI services such as Azure Machine Learning or Azure Cognitive Services.

View full job page →

1mo ago

Data and Analytics Engineer

Digital Strategy LLC Remote 🇺🇸 Remote

Remote

1mo ago Closed

$100k - $130k/yr Full time Analytics Big Data Bigdata

This position is no longer accepting applications (closed Aug 31, 2026).

Job Description

Data and Analytics Engineer at Digital Strategy LLC supports a U.S. Department of Energy client by designing, delivering, and operating end-to-end data products on a Databricks based cloud platform. The role blends data engineering, analytics-product development, governance, and AI-assisted delivery in a fully remote setting. Compensation ranges from USD 100,000 – 130,000 per year.

Responsibilities

Collaborate with architects, analysts, BI developers, team leads, DOE staff, and Digital Strategy leadership to ensure each data product aligns with mission priorities and the broader platform architecture.
Convert ambiguous business questions and operational needs into well scoped technical designs, delivery plans, and product increments.
Lead assigned workstreams from intake through delivery by clarifying outcomes, sequencing activities, tracking progress, and managing dependencies, risks, and stakeholder expectations.
Assess incoming requests and surface trade-offs among urgency, scope, quality, architecture, and team capacity for client and leadership visibility.
Communicate recommendations, constraints, and technical trade-offs clearly to stakeholders with diverse technical backgrounds while maintaining professionalism when priorities shift.
Apply and continually improve the team’s architecture and delivery standards, highlighting exceptions and proposing refinements.
Migrate legacy SQL Server and SSIS workloads to Databricks native pipelines and transformation patterns.
Design and operate ingestion pipelines for structured, semi-structured, and unstructured sources, including relational data, JSON, XML, spreadsheets, and documents.
Model Bronze, Silver, and Gold data layers with clear contracts, quality gates, lineage, and promotion logic between tiers.
Translate transactional source data into BI ready facts, dimensions, semantic structures, and curated data products.
Write and optimize production grade SQL and Python for reusable transformations, utilities, automation, and operational tasks.
Design consumption-layer products appropriate to the use case, such as Power BI or Databricks AI/BI dashboards, lightweight Databricks Apps, APIs, Genie Spaces, and governed NLQ experiences.
Embed data quality checks, validation rules, automated tests, documentation, and operational telemetry into every data product.
Leverage AI and emerging platform capabilities selectively to speed delivery, improve usefulness, supportability, or stakeholder access to insights.
Participate as a hands-on technical contributor for assigned products, making implementation decisions within architectural guardrails and escalating material trade-offs as needed.
Mentor analysts, BI developers, engineers, and citizen developers in data-as-code, Git-based collaboration, modular development, automated testing, code review, and CI/CD.
Conduct constructive design and code reviews that explain reasoning and help teammates develop independent judgment.
Create reusable templates, reference implementations, utilities, and practical documentation to enable less-experienced contributors to deliver safely and consistently.
Promote disciplined engineering without over-engineering urgent work; choose controls and patterns appropriate to risk, lifespan, and audience.
Identify patterns across products that can become reusable Digital Strategy capabilities, accelerators, demonstrations, or proposal assets.
Manage metadata, data lineage, ownership, and catalog organization in Unity Catalog.
Implement least-privilege access controls at the row, column, and object level for built and maintained products.
Coordinate with the DOE Databricks platform team on environment configuration, CI/CD, and promotion across development, test, and production workspaces.
Instrument data products with logging, alerts, event information, system-table reporting, and data quality monitoring to surface issues proactively.
Apply naming conventions, object taxonomy, documentation standards, and release criteria consistently across products and environments.
Protect sensitive information and comply with applicable federal, client, and platform security requirements.
Guide data products through design, development, testing, release, and post-production operation, ensuring required evidence and documentation are complete before promotion.
Maintain delivered products for reliability, performance, usability, and cost-effectiveness by tracing issues to root causes rather than patching symptoms.
Proactively surface gaps, risks, and improvement opportunities and collaborate with teammates and client staff to drive resolution.
Improve standard operating procedures and delivery practices when experience reveals gaps or friction.
Balance immediate client value with maintainability to avoid creating fragile production dependencies.

Requirements

7+ years of professional experience in data engineering, analytics engineering, or related fields, including at least 3 years owning production data products on a cloud analytics platform.
Bachelor's degree in Data Science, Data Analytics, Computer Science, Engineering, or a related technical field, or equivalent professional experience.
Proven track record of owning data products from design through release, support, and enhancement.
Experience leading technical workstreams, articulating trade-offs, reviewing peers, and mentoring without formal management authority.
Advanced production-grade SQL across analytical workloads, including complex transformations and dimensional modeling.
Production Python experience for data engineering, automation, utilities, APIs, or lightweight data applications.
Experience building and operating production-grade ingestion, transformation, and orchestration workflows on an enterprise cloud data platform.
Demonstrated implementation of analytical models and translation of transactional schemas into BI-ready facts, dimensions, and curated datasets.
Hands-on experience with source control, automated testing, code review, CI/CD, and promoting data-platform assets across environments.
Practical familiarity with data quality controls, observability, metadata, lineage, and security at row, column, or object levels.
Ability to build or support analytics consumption products such as dashboards, semantic models, APIs, interactive tools, or natural-language data experiences.
Capability to quickly learn a client’s mission and business context to inform technical decisions beyond literal requirements.
Experience translating complex concepts for diverse client stakeholders and co-developing solutions rather than awaiting detailed instructions.
Strong presentation of recommendations and trade-offs with clear reasoning, while staying open to challenge and adjusting direction when warranted by evidence.
Judgment in stakeholder interactions and professional composure when priorities shift, deadlines tighten, or feedback is challenging.
Clear written and verbal communication, including concise status updates, technical documentation, and decision records.

Technologies

Databricks, Unity Catalog, Lakeflow Spark Declarative Pipelines, Lakeflow Jobs
Delta Lake, Spark SQL, PySpark, SQL Server, SSIS
Power BI, Genie Spaces, Databricks Apps, Data Quality Monitoring
Declarative Automation Bundles, AWS, IAM, S3, VPC
Microsoft Fabric, Snowflake, Git, Python, SQL, dbt
Databricks AI/BI dashboards

Benefits

401(k)
401(k) matching
Dental insurance
Health insurance
Life insurance
Paid time off
Vision insurance

What success looks like during the first year

Within the initial months, achieve productive work within the DOE domain, the Databricks environment, and Digital Strategy delivery processes.
Take ownership of at least one data product from intake and design through production release, operation, and enhancement.
Modernize or significantly improve a legacy pipeline, data model, reporting process, or analytics product.
Introduce or reinforce automated quality checks, testing, observability, security, and documentation in assigned products.
Elevate team capability by guiding team members and citizen developers to adopt source control, modular development, code review, automated testing, or CI/CD practices.
Turn a recurring data request or manual process into a governed, reusable solution when value is clear.
Contribute reusable patterns, utilities, templates, accelerators, or lessons that benefit delivery beyond a single product or workstream.

About Digital Strategy LLC

Digital Strategy LLC is a federal IT and management consulting firm focused on data modernization, analytics, automation, and AI enabled delivery for U.S. government clients. The company emphasizes combining DOE and federal domain expertise with strong technical execution and seeks engineers who improve products and teams while delivering reliable, reusable capabilities.

Application Question(s)

In one to two sentences, identify a production data product you personally owned and your contribution.
In one to two sentences, describe how you balanced an urgent data request with longer-term engineering work.
In one to two sentences, identify how you helped a teammate adopt data-as-code, testing, or CI/CD practices.
In one to two sentences, identify a production data issue you traced to its root cause and prevented from recurring.
In one to two sentences, identify your role in building a medallion or comparable layered data solution and how data progressed from raw to curated layers.
Optionally, in one to two sentences, identify one way you have used AI to accelerate data or analytics engineering while maintaining validation and quality controls.
Can you consistently maintain availability from 9:00 a.m. to 5:00 p.m. Eastern Time? Briefly describe your expected working hours in Eastern Time.

Experience

Databricks production data product development and ownership: 1 year (Preferred)
Data engineering or analytics engineering: 7 years (Required)
Cloud analytics platform production data product ownership: 3 years (Required)

Language

English (Required)

View full job page →

1mo ago

Senior Data Engineer

Consigli Construction Providence, RI

Providence, RI

1mo ago Closed

$100k - $120k/yr Full time Senior Analytics Azure Data Factory

This position is no longer accepting applications (closed Aug 26, 2026).

Job Description

Location: Providence, RI (onsite). Salary: USD 100,000 - 120,000 per year. Experience: 5+ years. This onsite Senior Data Engineer role at Consigli Construction offers a hands-on opportunity to shape the organization’s analytical data environment and governance practices.

Responsibilities

Contribute to the evolution of the lakehouse architecture using Databricks, Azure, and Fabric.
Support ingestion, transformation, and serving patterns across Databricks notebooks, Empower and related tools.
Maintain environments, workspaces, and CI/CD patterns under guidance from senior technical leaders.
Support development and maintenance of subject-area models, conformed dimensions, and governed metrics used across reporting and dashboards.
Partner with business and project teams to align and standardize KPIs for cost, schedule, risk, and operational reporting.
Refactor business logic from dashboards or ad hoc SQL into governed transformations and reusable metrics.
Contribute to scaling master data domains (Project, Vendor, Budget, People, etc.) and steward data definitions.
Build, maintain, and document pipelines and datasets that are versioned, code-reviewed, and tested.
Implement data validation rules, anomaly detection (rule-based or ML-assisted), monitoring, and error-handling procedures.
Define SLAs, track reliability, and participate in incident response playbooks.
Identify opportunities to improve pipeline performance and processing efficiency.
Assist with applying data classification, masking, access controls, and privacy-by-design principles.
Collaborate with security and platform teams to support compliance audits and maintain documentation.
Work with project teams and business stakeholders to understand data needs and deliver reliable, well-modeled datasets.
Promote data literacy by helping teams access and use trusted analytics assets.
Provide clear communication, documentation, and best-practice guidance in data modeling, quality, and governance.

Requirements

5+ years in data architecture, data engineering, analytics engineering, or a similar development role within a modern cloud environment.
Strong experience designing data models, building ETL/ELT pipelines, and managing lakehouse or warehouse environments.
Experience with Databricks, MS Fabric, Delta Lake, or similar platforms.
Experience with construction or project-based analytics is a plus.

Technologies

Databricks
Azure
Fabric
OneLake
Data Factory
Empower
Databricks Genie
Claude desktop
Delta Lake
MS Fabric
Sage 300/CMiC (ERP)
Workable/SagePeople (HRIS)
Cosential/Unanet (CRM)

Key Skills

Strong SQL and Python skills
Proficiency with Azure based tools (Data Factory, Fabric/Lakehouse, OneLake) or Databricks equivalents
Deep understanding of data lineage, cataloging, governance, data quality frameworks, and security best practices
Experience with CLI and AI-assisted workflows (Claude desktop, Databricks Genie, or equivalent)
Familiarity with enterprise systems such as Sage 300/CMiC (ERP), Workable/SagePeople (HRIS), and Cosential/Unanet (CRM) is a plus
Excellent communication skills with the ability to translate complex technical concepts for business teams
Strong critical thinking, problem-solving, and analytical abilities
Demonstrated ability to collaborate cross-functionally and drive adoption of data best practices

View full job page →

1mo ago

Senior Data Analyst, Hardware & Supply Chain

Oura San Francisco, CA

San Francisco, CA

1mo ago Closed

$148k - $203k/yr Full time Senior Analyst Analytics

This position is no longer accepting applications (closed Aug 19, 2026).

Job Description

Senior Data Analyst for Hardware and Supply Chain analytics at Oura, focused on hardware manufacturing, supply chain metrics, and related analytics on a Databricks-based platform, collaborating with Operations, Supply Chain, Finance, and Data Engineering.

Responsibilities

Own the analytics layer for the hardware and supply chain scorecard and partner with Operations, Supply Chain, and Finance to define, document, and deliver KPIs used in board and S-1 reporting (hardware revenue, units sold, sell-through, COGS, landed cost, gross margin, on-hand and in-transit inventory, warranty rate, return/RMA rate, activation rate).
Establish each hardware and supply chain KPI as a governed data product with locked definitions, a single owner, dbt tests, full lineage from source systems, formal sign-off, and a documented change-management process.
Build and maintain the hardware data contract and the canonical hardware dimension that ties SKUs, components, ring generations, sizes, and finishes to downstream sales, inventory, and activation metrics.
Serve as the analytics representative in external audit cycles by producing documentation, guiding auditors through lineage, and ensuring controls and traceability are reflected in metric production.
Migrate hardware and supply chain reporting from legacy stacks to a Databricks-based platform with dbt-modeled metrics, Unity Catalog lineage, and Databricks SQL / Genie as the executive-facing surface.
Coordinate with ERP, manufacturing, logistics, and e-commerce source-system owners (NetSuite, contract-manufacturer feeds, 3PL/logistics data, Zuora) and Data Engineering to ensure clean ingestion, accurate dimensional modeling, and clear lineage from purchase orders to sell-through and activation.
Resolve data quality issues to ensure end-to-end trust in reporting.
Build and maintain the Hardware/SC domain data products in Databricks, mentoring analysts on dimensional modeling, dbt practices, and publishing governed metrics for reporting and for MCP/agentic AI consumption.
Partner with Supply Chain and Operations on demand and supply planning data, owning the Databricks-side data layer that feeds planning and ensuring reconciled numbers across regions.
Deliver inventory, fulfillment, and supplier analytics (days of supply, fill rate, on-time delivery, yield, scrap, cost-to-serve) to provide Operations leadership with a single trusted view.
Drive adoption of self-service reporting by Operations and business partners using Genie, Databricks SQL dashboards, and clear documentation to reduce one-off reporting.
Continuously evaluate tools (Unity Catalog, Databricks Genie, AI-assisted SQL/modeling with Cursor and Claude Code, semantic layers) to improve productivity and accuracy in hardware and supply chain reporting.

Requirements

7+ years in analytics, business intelligence, or a related data-focused role, with at least 3+ years supporting Hardware, Supply Chain, Operations, Manufacturing, or Logistics.
Proven experience building executive-level reporting for a physical-product business that earns trust from leadership and endures external scrutiny (Board, auditors, investors).
Deep knowledge of hardware and supply chain data domains: units sold and sell-through, COGS and landed cost, BOM and component costing, inventory (on-hand, in-transit, days of supply), warranty and RMA/returns, manufacturing yield, and how transactional systems (NetSuite/ERP, contract-manufacturer feeds, 3PL, e-commerce platforms) feed into reported metrics.
Hands-on experience with demand and supply planning data and a clear understanding of where planning systems meet the data lake.
Fluent in SQL with a track record of writing complex queries for operational and financial data extraction, reconciliation, and modeling.
Expertise in dimensional modeling for hardware and operations reporting (conformed dimensions across SKU, product generation, channel, geo, supplier; slowly changing dimensions for product and cost hierarchies; period-over-period comparisons).
Experience with dbt, including tests, documentation, exposures, and contracts.
Experience with Databricks strongly preferred (Unity Catalog, Databricks SQL, Lakeflow, Genie); deep Snowflake experience acceptable with willingness to migrate.
Experience with modern BI tools (Tableau, Looker, Databricks SQL Dashboards, Power BI) and an informed view on when each is appropriate.
History working with external auditors (PwC, Deloitte, EY, KPMG) and producing documentation that stands up to audit review.
Excellent verbal and written communication skills—able to briefing leaders on numbers and explaining data behind them to analysts.
Thrives in a fast-paced, collaborative global environment spanning factories, warehouses, and offices across time zones.

Technologies

dbt, Unity Catalog, Databricks, Databricks SQL, Genie
Snowflake, Tableau, Looker, Power BI
SQL, Lakeflow, Cursor, Claude Code
AWS, GitHub, NetSuite, Fivetran
Python, Zuora, Sheets

Benefits

Competitive salary and equity packages
Health, dental, vision insurance, and mental health resources
An Oura Ring of your own plus employee discounts for friends and family
20 days of paid time off plus 13 paid holidays plus 8 days of flexible wellness time off
Paid sick leave and parental leave

What Makes You Stand Out

Familiarity with adjacent tooling and sources: AWS, GitHub, NetSuite/ERP, Fivetran, Python, semantic layers, EDI/3PL logistics feeds, and contract-manufacturer data integrations.
Experience establishing a hardware data contract and a product/SKU dimension (dim_hardware) from scratch in a high-growth consumer hardware company.
Proven ability to build governed KPI programs through formal external audits, including COGS, inventory, and units-sold reconciliation.
Strong data visualization theory sense—knowing how to pick the right chart for the right message and why dashboards fail when misapplied.

Compensation

Salary ranges by region:

Region 1 $172,000 - $203,000
Region 2 $158,950 - $187,000
Region 3 $147,900 - $174,000
A recruiter will determine your zone based on your US location.

View full job page →

1mo ago

Data Engineer I (AWS, Databricks)

Travelers Hartford, CT

Hartford, CT

1mo ago Closed

$109k - $180k/yr Full time AWS Business Intelligence Cloud

This position is no longer accepting applications (closed Aug 16, 2026).

Job Description

Travelers is expanding its Data Engineering team in Hartford, CT, where data professionals design, build, and deploy solutions on AWS and Databricks to support AI and machine learning initiatives as well as enterprise intelligence. The Data Engineer I role focuses on creating scalable data products and pipelines that empower analytics across the organization. This onsite position invites collaboration with cross-functional teams to translate data into actionable insights while growing your expertise in data engineering.

Responsibilities

Build and operationalize complex data solutions, fix problems, apply transformations, and propose data cleansing and quality approaches.
Design data solutions and architectures.
Analyze data sources to determine value and identify data to include in analytical workflows.
Incorporate core data management capabilities including governance, security, and quality.
Collaborate within and across teams to support delivery and educate end users on data products and analytic environments.
Perform data and system analysis, assess defects and incidents of moderate complexity, and implement appropriate corrections.
Test data movement, transformation code, and data components.
Perform other duties as assigned.

Requirements

Bachelor’s degree in a STEM-related field or equivalent.
Six years of related experience.
Strong data engineering fundamentals with hands-on experience designing, building, and optimizing data pipelines.
Proficient with cloud platforms and modern data technologies, including AWS: Lambda functions, EventBridge, Step Functions, DynamoDB, Athena; Databricks: Notebooks, Jobs, Unity Catalog, Genie; and concepts such as Autoloader, streaming, and near real-time data processing.
Experience with AI tools and concepts (for example Claude) and familiarity with developer productivity tools like GitHub Copilot.
Comfortable using engineering and DevOps tools such as Jira, GitHub, Snyk, ServiceNow, and Autosys.
Proficient programming skills and a solid understanding of software engineering practices including testing, CI/CD, and version control.
Ability to deliver work at a steady, predictable pace, break down complex tasks into incremental releases, and participate in trade-off discussions.
Demonstrated domain expertise with the ability to quickly understand technical concepts and develop deep knowledge of supported systems.
Strong problem-solving and debugging skills, including root-cause analysis in unfamiliar codebases and resolving recurring issues.
Excellent verbal and written communication skills and the ability to collaborate effectively with team members, partners, and stakeholders.
Intermediate leadership skills with a track record of self-motivation, ownership, and a commitment to continuous learning and personal growth.
Two additional years of software or data engineering experience.

Technologies

AWS: Lambda functions, EventBridge, Step Functions, DynamoDB, Athena
Databricks: Notebooks, Jobs, Unity Catalog, Genie
Autoloader, streaming, near real-time data processing concepts
Claude (AI tools) and GitHub Copilot
Jira, GitHub, Snyk, ServiceNow, Autosys

Benefits

Health Insurance
401(k) match
Pension Plan funded by Travelers
Paying it Forward Savings Program
Paid Time Off
Wellness Program
Matching Gift and Volunteer Rewards program

Compensation

Base salary range: $109,300 - $180,200 per year. Final compensation is determined by factors including the scope and complexity of the role, location, and the candidate's skills, education, credentials, and experience.

About Travelers

Travelers is an equal opportunity employer that values the unique abilities and talents each person brings to the organization. The company has sustained a strong presence in the property and casualty insurance industry for more than 170 years and emphasizes a culture of innovation and collaboration, guided by the Travelers Promise to take care of customers, communities, and one another.

View full job page →

Databricks Genie Jobs

Lead AI Engineer

Data and Analytics Engineer

Senior Data Engineer

Senior Data Analyst, Hardware & Supply Chain

Data Engineer I (AWS, Databricks)

Get Databricks Genie Jobs alerts