Data Engineer (Bid Role starting in 10 weeks) - TS/SCI w/ Poly

Leading Path Consulting has a Data Engineer bid role for a planned start in ten weeks. The position is based onsite in McLean, VA, with a salary range of USD 175,000 to 275,000 per year, and a focus on assessing, designing, and implementing robust data infrastructure and pipelines across the organization, emphasizing data quality, observability, and governance. This bid role requires an active TS/SCI with FS poly.

Responsibilities

• Conduct comprehensive assessments of current data pipelines, infrastructure, and data flows, including integrations with ServiceNow, network-management tools, and business applications, to identify technical debt, bottlenecks, and reliability issues.
• Evaluate current data architecture against industry best practices and organizational needs; develop technical recommendations and a roadmap for data infrastructure improvements.
• Design, implement, and maintain production-grade data pipelines using orchestration platforms such as Airflow or Prefect.
• Develop resilient ETL and ELT processes from diverse sources including SaaS platforms, network management systems, databases, APIs, files, and streams.
• Build API integrations with authentication methods (OAuth, API keys, SSO), plus rate limiting, pagination, retry logic, and error handling.
• Extract data from systems not designed for export, reverse-engineering undocumented data structures and relationships.
• Manage semi-structured data (JSON and XML) and transform it into structured datasets with consistent schemas.
• Design dimensional models, data warehouses, and data marts following industry methodologies.
• Create conceptual, logical, and physical data models optimized for query performance and storage efficiency.
• Implement slowly changing dimensions and other data warehousing patterns.
• Establish naming conventions, data standards, and modeling best practices.
• Implement comprehensive data quality checks, validation rules, and automated monitoring with alerting.
• Embed error handling, failure recovery, logging, and observability into all processes.
• Optimize pipelines for performance, cost, and resource utilization.
• Develop reusable components and frameworks; refactor legacy pipelines for reliability.
• Build and maintain data infrastructure on cloud platforms (AWS, Azure, or GCP) using infrastructure-as-code with Terraform and CloudFormation.
• Implement CI/CD pipelines, version control with Git, and automated testing frameworks.
• Manage database performance tuning, indexing, partitioning, and capacity planning.
• Establish backup, recovery, security controls, access controls, and compliance measures.
• Partner with analysts, software developers, and business stakeholders to translate requirements into technical solutions.
• Create comprehensive documentation for systems, processes, and integrations.
• Provide technical guidance on data availability and proper usage; enable self-service access.
• Troubleshoot pipeline failures, performance issues, and data discrepancies; perform root cause analysis.

Requirements

• Active TS/SCI with FS poly required prior to application.
• Proven experience building and managing data pipelines.
• Hands-on experience with Python.
• Experience with cloud computing using AWS services.
• Experience processing data with Apache Spark.
• Experience with a relational database management system (PostgreSQL, Oracle, MySQL) and writing SQL queries.
• Experience working in Linux environments and with shell scripting.
• Experience analyzing data in formats such as CSV, XML, JSON, Avro, Parquet, etc.
• Experience writing and validating unit tests.

Technologies

• Airflow
• Prefect
• Python
• AWS, Azure, GCP
• Apache Spark
• PostgreSQL
• Oracle
• MySQL
• Linux
• Shell scripting
• JSON, XML, Avro, Parquet
• OAuth, Single Sign-On (SSO)
• Terraform, CloudFormation
• Git
• ServiceNow
• NetIM
• Forward Networks
• Hive
• Iceberg
• EMR

Benefits

• Fully paid medical, dental, and vision premiums
• Generous paid time off
• 11 paid holidays
• 6% 401(k) contribution
• Annual training and tuition reimbursement
• SPOT Award bonuses
• Regular team events
• Opportunities for professional growth and advancement

Travel

Travel is anticipated for this contract, pending sponsor approval. Local travel or point-of-view travel within the local place of performance is expected on an as-needed basis.