D2 Technologies is hiring an experienced Data Engineer for a hybrid role based in Herndon, VA (US Federal project). In this position, you will design and optimize enterprise-grade data platforms on the Databricks Lakehouse Platform, building scalable ETL/ELT pipelines and improving Spark and SQL performance to support analytics and data transformation.
You’ll work at the intersection of data engineering and engineering best practices, collaborating with data architects, software engineers, analysts, and business stakeholders. The role includes production support, troubleshooting, and documentation to help ensure reliable data operations, governance, and security across enterprise data platforms.
What you’ll do
- Design, develop, and maintain scalable data engineering solutions using Databricks and Apache Spark
- Build and optimize high-performance ETL/ELT pipelines to process large volumes of structured and unstructured data
- Develop, tune, and optimize complex SQL queries for data transformation and analytics
- Analyze business requirements and translate them into scalable technical solutions
- Troubleshoot production issues and optimize data pipeline performance
- Ensure data quality, integrity, security, and governance across enterprise data platforms
- Collaborate with cross-functional teams and participate in code reviews using software engineering best practices
- Create and maintain technical documentation for solutions, deployments, and operational support
- Support production releases and provide operational support as needed
What you bring
- U.S. Citizenship required
- Top Secret (TS) Security Clearance or Public Trust Clearance preferred
- Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field (or equivalent experience)
- Hands-on experience developing data engineering solutions using Databricks, including strong experience with the Databricks Lakehouse Platform
- Strong programming experience with Apache Spark (including PySpark / Spark SQL)
- Expert-level SQL skills, including writing, optimizing, and troubleshooting complex queries
- Experience designing, developing, and supporting ETL/ELT pipelines
- Strong understanding of data modeling, data warehousing, and Delta Lake
- Experience using Git or other version control systems
- Excellent verbal and written communication skills
Tools you’ll work with
Databricks, Databricks Lakehouse Platform, Apache Spark, PySpark, Spark SQL, SQL, ETL, ELT, Delta Lake, Git, Azure, AWS, Google Cloud Platform, Databricks Delta Live Tables (DLT), Unity Catalog, PL/SQL, CI/CD
Preferred
- Strong analytical and problem-solving skills
- Strong SQL query writing, optimization, and performance tuning skills
- Experience with Azure, AWS, or Google Cloud Platform
- Knowledge of CI/CD pipelines and DevOps practices
- Experience with Databricks Delta Live Tables (DLT) and Unity Catalog
- Nice to have: Experience with PL/SQL
Compensation
$100,000 - $150,000 per year (negotiable).
Work location: Hybrid remote in Herndon, VA 20192
