Big Data Engineer

SAIC is hiring a Big Data Engineer (onsite) in Honolulu, HI to design and operate real-time and near-realtime ingestion pipelines for a mission-critical system with cross-domain data movement.

Responsibilities

• Support ongoing delivery of continual data feeds and monitor data quality and data ingest health.
• Design, build, and maintain resilient real-time/near-realtime ingestion pipelines using Apache NiFi and REST APIs, including backpressure, prioritization, retries, and error-handling approaches.
• Configure and monitor data flow across security domains using cross-domain guard solutions, ensuring data integrity and adherence to transfer policies.
• Parse, transform, validate, and route structured and unstructured data across multiple formats such as Excel, CSV, JSON, and XML.
• Use data manipulation and visualization tools such as Grafana and Prometheus to communicate pipeline status, data quality, and historical trends.
• Collaborate with platform, software, and other data engineers to (re)configure and continuously improve ingestion pipeline reliability.
• Create and maintain software/scripts to automate monitoring of real-time feeds, including alerting on timeliness, volume, lineage, and distribution issues.
• Apply insights from historical pipeline data to improve ingestion reliability and performance.
• Partner with security and governance teams to implement encryption, authentication, authorization, and data classification controls across pipelines and cross-domain transfers.
• Write and maintain scripts (Python, Bash, or similar) to automate data processing, validation, and monitoring tasks.
• Document data flows, system configurations, and standard operating procedures.

Requirements

• U.S. Citizenship and an active TS/SCI clearance.
• Bachelor of Science degree in Computer Science, Mathematics, Electrical Engineering, Physics, Information Systems, Information Technology, or a related field.
• 3+ years of experience in data engineering, data operations, or DevOps supporting production data pipelines.
• Proficiency in Python, Bash, or similar scripting languages used in data science/data analytics.
• Working knowledge of the Linux (RedHat) command line; familiarity with Windows environments.
• Strong grounding in data engineering fundamentals, including data pipelines, streaming architectures, ETL/ELT concepts, and data quality principles.
• Familiarity with JSON, XML, CSV, and Excel data formats, including parsing and transformation.

Technologies

• Apache NiFi
• REST APIs
• Grafana
• Prometheus
• Python
• Bash
• Linux (RedHat)
• Excel, CSV, JSON, XML
• Kafka, JMS
• Apache Flink, Spark Streaming
• Elasticsearch
• Snowflake
• Git

Preferred Qualifications

• Experience with streaming/messaging platforms and tools: Kafka, JMS, Apache Flink, Spark Streaming.
• Experience with monitoring/observability tools: Grafana, Prometheus, Elasticsearch.
• Experience with data platforms such as Snowflake.
• Familiarity with cross-domain solutions/guards, including data diode concepts, transfer validation, and content filtering.
• Experience enforcing encryption, authentication/authorization, and data classification policies with security/governance teams.
• Experience with version control tools such as Git and Agile/Scrum practices.
• Prior experience in a government, defense, or intelligence community environment.

Location

• Honolulu, HI, US (onsite)

Clearance

• Minimum clearance required: TS.SCI
• Clearance level must be able to obtain: None

Travel

• Yes - 10% of the time

Compensation

• USD 120,001 - 160,000 per yearly