Data Engineer

Agile Defense is seeking a Data Engineer to design, build, and maintain scalable data pipelines and real-time streaming solutions. The role supports analytics and reporting by improving how data is validated, processed, monitored, and made accessible across platforms.

Based in Omaha, NE, this is an onsite position in a SCIF in STRATCOM, with a requirement of 5+ years of relevant experience.

What you will do

• Design, build, and maintain highly scalable, reliable, and efficient ETL/ELT data pipelines that extract, transform, and load data into data warehouses, data lakes, and other storage systems.
• Partner with data scientists, analysts, and stakeholders to understand data needs and ensure an efficient flow of data across platforms and systems.
• Develop, deploy, and maintain real-time streaming solutions with low latency using tools such as Apache Kafka, Flink, or Spark Streaming.
• Optimize and automate data workflows so pipelines remain efficient, dependable, and able to handle growing data volumes.
• Oversee data models and schema designs to support accurate, accessible, and high-performance data storage for analytics and reporting.
• Implement data quality, validation, monitoring, and error-handling mechanisms to protect data integrity and consistency.
• Monitor and troubleshoot performance bottlenecks affecting data processing and data access.
• Stay current with data engineering trends and best practices, recommending tools, technologies, and methods to improve existing infrastructure.
• Mentor and guide junior data engineers and promote best practices in a collaborative, high-performance team environment.

Requirements

• Strong understanding of data validation, model testing, and performance evaluation techniques.
• Experience with data visualization and storytelling using tools such as Tableau, Plotly, or Matplotlib.
• Excellent technical communication skills, including the ability to explain complex concepts to non-technical audiences.

Preferred background and skills

• 4+ years of experience in applied data science, machine learning engineering, or data pipeline development.
• Proficiency in Python, SQL, and distributed data frameworks such as Spark, Databricks, and PySpark.
• Experience building ML models from training to deployment using tools such as scikit-learn, TensorFlow, XGBoost, and MLflow.
• Familiarity with MLOps, API development, and secure cloud environments (e.g., AWS, Azure, Palantir Foundry).

Technologies you may work with

• Apache Kafka, Flink, Spark Streaming
• Tableau, Plotly, Matplotlib
• Python, SQL, Spark, Databricks, PySpark
• scikit-learn, TensorFlow, XGBoost, MLflow
• AWS, Azure, Palantir Foundry

Education and experience

• Bachelor’s degree in Computer Science, Information Systems, Data Engineering, or a related field (masters degree preferred).
• 5+ years of experience in data engineering or software engineering, including designing and managing complex data pipelines.
• Equivalent relevant work experience may be substituted for education requirements, with one year of work experience replacing one year of education required.

Core values

• Happy: Be Infectious
• Helpful: Be Supportive
• Honest: Be Trustworthy
• Humble: Be Grounded
• Hungry: Be Eager
• Hustle: Be Driven