Data Engineer

VTG is seeking an experienced Data Engineer to design, develop, and maintain scalable data processing solutions. The role centers on building ETL pipelines and analytics-ready datasets, integrating structured and unstructured sources, and ensuring data quality within an Agile delivery environment.

Key Responsibilities

• Design, develop, and maintain scalable data processing and ETL pipelines using Python, PySpark, and Apache Spark.
• Extract, transform, and load data from multiple structured and unstructured sources into optimized, query-ready datasets.
• Develop and maintain data mapping, transformation, and integration processes aligned to business and technical requirements.
• Process large volumes of operating system logs, application logs, and other data sources to produce structured reports, metrics, and analytics dashboards.
• Transform disparate data formats into enriched, indexed, and query-friendly datasets.
• Develop and optimize data processing workflows to improve performance, efficiency, and reliability.
• Build and maintain data integration solutions using SQL, MySQL, and PostgreSQL.
• Work with structured and semi-structured data formats, including XML and JSON.
• Perform data extraction, cleansing, transformation, and validation to ensure accuracy and consistency.
• Create and maintain data models, database queries, and data mapping specifications.
• Use Regular Expressions (RegEx) to parse, extract, validate, and transform complex data.
• Support integration of data across multiple systems, applications, and external sources.
• Produce and maintain ETL design documentation, including source-to-target mappings, transformation rules, and data dictionaries.
• Utilize development environments and tools such as Jupyter Notebooks and Visual Studio for data modeling, analysis, and development.
• Implement data validation and quality assurance processes to protect data integrity, accuracy, and completeness.
• Troubleshoot and resolve data processing, integration, and performance issues.
• Maintain technical documentation describing data workflows, processing logic, and integration requirements.

Required Qualifications

• Active TS/SCI with Polygraph
• Bachelor’s degree in Computer Science, Information Technology, or a related technical field
• Demonstrated experience developing data processing solutions using Python, PySpark, and Apache Spark
• Hands-on experience designing, developing, and maintaining ETL pipelines and data transformation workflows
• Strong background in data extraction, mapping, transformation, integration, and loading processes
• Proficiency in SQL and experience with relational databases such as MySQL and PostgreSQL
• Experience processing structured, semi-structured, and unstructured datasets
• Experience working with XML, JSON, and other common data formats
• Strong knowledge of Regular Expressions (RegEx) for data parsing, extraction, and manipulation
• Experience analyzing operating system and application logs and converting them into actionable reports and dashboards
• Experience developing reports and visualizations using AWS CloudWatch, Kibana, or comparable tools
• Experience with data profiling, data quality analysis, validation, and troubleshooting
• Ability to develop detailed ETL documentation, including source-to-target mappings and data dictionaries
• Experience using Jupyter Notebooks, Visual Studio, or similar development environments
• Strong analytical, troubleshooting, and problem-solving skills
• Experience collaborating with customers and integration partners to define and clarify technical requirements
• Familiarity with Agile software development methodologies and practices

Location and Work Setup

Chantilly, VA (onsite)

Technology Stack

Python, PySpark, Apache Spark, SQL, MySQL, PostgreSQL, XML, JSON, Regular Expressions (RegEx), Jupyter Notebooks, Visual Studio, AWS CloudWatch, Kibana