DataJobs.io
← Back to all jobs

Job Description

The Databricks Data Engineer role at Robotics Technologies is based in Atlanta, GA and supports onsite delivery for a contract term of 24 months plus extension. The position focuses on building and operating enterprise-scale data processing solutions using Databricks and the Spark ecosystem, with lakehouse development, governed access, and reliable pipeline execution.

Responsibilities

  • Design and develop scalable data engineering solutions using Databricks.
  • Build batch and streaming data pipelines using Apache Spark.
  • Develop data transformations using PySpark and Spark SQL.
  • Implement lakehouse architecture using Delta Lake.
  • Manage data access, governance, and permissions through Unity Catalog.
  • Build and maintain automated workflows using Databricks Workflows.
  • Implement incremental processing and CDC pipelines.
  • Optimize Spark jobs and improve processing performance.
  • Integrate Databricks with cloud storage and enterprise databases.
  • Develop data validation and quality monitoring processes.
  • Collaborate with data scientists and analytics teams.
  • Troubleshoot production issues and ensure pipeline reliability.

Requirements

  • Strong hands-on experience with Databricks.
  • Advanced programming skills in Python and PySpark.
  • Strong knowledge of Apache Spark and Spark SQL.
  • Experience with Delta Lake and lakehouse architecture.
  • Knowledge of Unity Catalog and data governance practices.
  • Experience with Delta Live Tables / Lakeflow.
  • Experience building ETL and ELT pipelines.
  • Knowledge of AWS, Azure, or GCP cloud platforms.
  • Experience with SQL, data modeling, and data warehousing.
  • Familiarity with Git, CI/CD, and workflow orchestration.

Technologies

  • Databricks
  • Apache Spark
  • PySpark
  • Delta Lake
  • Delta Live Tables
  • Lakeflow
  • Unity Catalog
  • Spark SQL
  • Databricks Workflows
  • AWS
  • Azure
  • GCP
  • Git
  • CI/CD

Preferred Qualifications

  • Experience with Databricks Lakeflow and Lakeflow Connect.
  • Knowledge of Apache Kafka and real-time data processing.
  • Experience with Terraform and cloud infrastructure automation.
  • Knowledge of MLflow and machine learning workflows.
  • Experience with enterprise data migration projects.

Location and Compensation

Location: Atlanta, GA (onsite)
Compensation: USD 76 per hour

Similar Jobs