Senior Data Engineer – Spark / Scala

Astyra Corporation is seeking a Senior Data Engineer to help modernize legacy ETL workloads and deliver Spark-based data pipelines. This contract role is based in Richmond, VA with hybrid remote work, with a focus on accurate data movement, Spark job optimization, and validation between legacy and target platforms.

What You’ll Do

• Design, build, and maintain scalable data pipelines using Apache Spark and Scala
• Support the migration of legacy ETL workloads into a modern Spark-based data framework
• Develop and enhance data transformation and processing solutions
• Perform data comparison, reconciliation, and validation between legacy and target platforms
• Identify and troubleshoot data discrepancies throughout the migration process
• Create data quality checks and controls to ensure completeness and accuracy
• Optimize Spark jobs and pipelines for performance, scalability, and reliability
• Collaborate with data engineers, architects, and other technical teams on modernization initiatives
• Support testing, deployment, troubleshooting, and ongoing maintenance of production data pipelines
• Contribute to broader feature engineering and data modernization efforts

What You Bring

• 7+ years of experience in data engineering or a related technical discipline
• Strong hands-on experience designing and developing enterprise data pipelines
• Advanced experience with Apache Spark
• Strong development experience with Scala
• Experience with ETL development, modernization, and migration initiatives
• Strong understanding of data transformation, data quality, and reconciliation
• Experience comparing and validating data across legacy and modernized systems
• Experience developing and supporting production data-processing workloads
• Familiarity with modern CI/CD and software development practices
• Strong troubleshooting and problem-solving skills

Technologies

• Apache Spark
• Spark
• Scala
• Python
• ETL
• CI/CD

Preferred Qualifications

• Experience migrating legacy ETL platforms to Spark-based architectures
• Experience supporting large-scale technology or data modernization initiatives
• Experience within financial services or other highly regulated environments
• Python experience
• Exposure to Generative AI or AI-assisted development and modernization tools
• Previous experience within a large enterprise engineering environment

Compensation and Schedule

• Pay: $66.00 - $71.00 per hour
• Expected hours: 40.0 per week

Benefits

• 401(k)
• Dental insurance
• Health insurance
• Vision insurance

Location: Richmond, VA 23238 (Required). Work location: Hybrid remote in Richmond, VA 23238.