DataJobs.io
← Back to all jobs

Job Description

Guidehouse is seeking a Cleared Data Engineer to design and maintain scalable batch and streaming data pipelines in Databricks, supporting cybersecurity metrics, risk analysis, and reporting. This position requires an Active Secret clearance and is based in Arlington, VA (onsite).

Key Responsibilities

  • Design, develop, test, deploy, and maintain scalable batch and streaming data pipelines in Databricks using Python, PySpark, SQL, and reusable engineering patterns.
  • Convert raw source data into curated, analysis-ready datasets and dimensional models for cybersecurity metrics, risk analysis, dashboards, and executive reporting.
  • Create automated data-quality controls for ETL jobs, including checks for schema, completeness, validity, consistency, uniqueness, timeliness, referential integrity, and source-to-target reconciliation.
  • Define handling procedures for failed quality checks, such as quarantining invalid records, stopping critical loads, generating alerts, documenting exceptions, and coordinating remediation with data owners.
  • Schedule and orchestrate production workflows; monitor job status, freshness, throughput, latency, resource utilization, and failures; troubleshoot root causes; and restore processing within agreed service levels.
  • Improve reliability, performance, and cost efficiency through transformation optimization, workload configuration, incremental processing, partitioning, and compute resource management.
  • Implement and maintain data governance and quality assurance standards, including access controls, metadata, data classification, ownership, lineage, retention, auditability, and protection of sensitive cybersecurity information.
  • Maintain source-to-target mappings, data dictionaries, transformation rules, validation evidence, and operational documentation such as standard operating procedures and runbooks.
  • Work with cybersecurity analysts, data owners, system administrators, dashboard developers, and business stakeholders to define data requirements, metrics, thresholds, and acceptance criteria.
  • Maintain client relationships by anticipating stakeholder needs, communicating delivery risks and dependencies, and resolving issues in a timely and professional manner.
  • Organize and lead client meetings by coordinating schedules, preparing and distributing agendas and pre-read materials, facilitating discussions, recording minutes, maintaining documentation, and tracking action items through closure.
  • Participate in peer reviews, testing, release activities, and continuous-improvement efforts to strengthen engineering practices and reduce recurring defects.

Required Qualifications

  • ACTIVE and CURRENT SECRET federal security clearance
  • Bachelor’s degree
  • SIX (6) years total experience in a data or programming discipline, including THREE (3) years in data engineering
  • Proficiency in data engineering programming languages such as Python, PySpark, and/or SQL
  • Experience with scheduling, monitoring, troubleshooting, and optimizing production ETL/ELT pipelines
  • Experience with data engineering principles and best practices
  • Excellent problem-solving skills and attention to detail
  • Selected Candidate must be able to commute to client and Guidehouse offices as needed

Technologies

  • Databricks
  • Python
  • PySpark
  • SQL

Benefits

  • Medical, Rx, Dental & Vision Insurance
  • Personal and Family Sick Time & Company Paid Holidays
  • Position may be eligible for a discretionary variable incentive bonus
  • Parental Leave and Adoption Assistance
  • 401(k) Retirement Plan
  • Basic Life & Supplemental Life
  • Health Savings Account, Dental/Vision & Dependent Care Flexible Spending Accounts
  • Short-Term & Long-Term Disability
  • Student Loan PayDown
  • Tuition Reimbursement, Personal Development & Learning Opportunities
  • Skills Development & Certifications
  • Employee Referral Program
  • Corporate Sponsored Events & Community Outreach
  • Emergency Back-Up Childcare Program
  • Mobility Stipend

Travel and Clearance

  • Travel Required: Up to 50%
  • Clearance Required: Active Secret

Nice to Have

  • Master’s degree in computer science or related field
  • Experience working with system and application logs
  • Familiarity with the Databricks platform
  • Strong communication and collaboration skills, with the ability to work effectively in a team environment

Compensation

USD 85,000 - 141,000 per yearly

Similar Jobs