Databricks Data Engineer
Azure
Big Data
Bigdata
Cloud
Cloud Data Engineering
Cloud Data Platform
Cloud Engineer
Cloud Native
Cloud Operations
Cloud Platform
Cloud Platforms
Cloud Platforms Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Governance
Data Integration
Data Integration Tools
Data Lakehouse
Data Management
Data Pipeline
Data Platform
Data Processing
Data Security
Data Warehouse
Database
Databases
Databricks
ETL
Google Cloud
Informatica
Pyspark
Spark
SQL
Job Description
The Databricks Data Engineer role at Robotics Technologies is based in Atlanta, GA and supports onsite delivery for a contract term of 24 months plus extension. The position focuses on building and operating enterprise-scale data processing solutions using Databricks and the Spark ecosystem, with lakehouse development, governed access, and reliable pipeline execution.
Responsibilities
- Design and develop scalable data engineering solutions using Databricks.
- Build batch and streaming data pipelines using Apache Spark.
- Develop data transformations using PySpark and Spark SQL.
- Implement lakehouse architecture using Delta Lake.
- Manage data access, governance, and permissions through Unity Catalog.
- Build and maintain automated workflows using Databricks Workflows.
- Implement incremental processing and CDC pipelines.
- Optimize Spark jobs and improve processing performance.
- Integrate Databricks with cloud storage and enterprise databases.
- Develop data validation and quality monitoring processes.
- Collaborate with data scientists and analytics teams.
- Troubleshoot production issues and ensure pipeline reliability.
Requirements
- Strong hands-on experience with Databricks.
- Advanced programming skills in Python and PySpark.
- Strong knowledge of Apache Spark and Spark SQL.
- Experience with Delta Lake and lakehouse architecture.
- Knowledge of Unity Catalog and data governance practices.
- Experience with Delta Live Tables / Lakeflow.
- Experience building ETL and ELT pipelines.
- Knowledge of AWS, Azure, or GCP cloud platforms.
- Experience with SQL, data modeling, and data warehousing.
- Familiarity with Git, CI/CD, and workflow orchestration.
Technologies
- Databricks
- Apache Spark
- PySpark
- Delta Lake
- Delta Live Tables
- Lakeflow
- Unity Catalog
- Spark SQL
- Databricks Workflows
- AWS
- Azure
- GCP
- Git
- CI/CD
Preferred Qualifications
- Experience with Databricks Lakeflow and Lakeflow Connect.
- Knowledge of Apache Kafka and real-time data processing.
- Experience with Terraform and cloud infrastructure automation.
- Knowledge of MLflow and machine learning workflows.
- Experience with enterprise data migration projects.
Location and Compensation
Location: Atlanta, GA (onsite)
Compensation: USD 76 per hour