Data Engineer 3 (Mid-Level)
Job Description
UNCOMN is seeking an ETL and Data Ingestion Engineer for the Integrated Data Environment and Global Transportation Network Convergence (IGC) team at Scott AFB, IL.
Responsibilities
- Design, develop, test, deploy, and maintain enterprise ETL and data-integration solutions using Informatica and other approved technologies.
- Build and optimize Informatica assets including mappings, transformations, workflows, sessions, parameter files, reusable components, scheduling configurations, and restart/recovery logic.
- Modernize legacy ETL workflows by refactoring mappings, SQL, transformations, interfaces, and operational processes for approved database and cloud environments.
- Develop and optimize complex SQL including queries, views, stored procedures, functions, and indexes for high-volume data-processing workloads.
- Conduct design and code reviews and define reusable ETL patterns, development practices, and technical approaches for complex integrations.
- Design ingestion patterns for relational databases, APIs, web services, message-based interfaces, structured and unstructured files, and other approved enterprise sources.
- Implement batch, micro-batch, incremental, event-driven, and change-data-capture processing approaches aligned to mission, interface, and performance requirements.
- Analyze source systems (structures, interfaces, dependencies, formats, business rules) and produce source-to-target mappings, transformation specifications, data dictionaries, lineage documentation, and interface-control artifacts.
- Create analytics-ready datasets, views, and data services supporting downstream applications, reporting, and approved business-intelligence platforms.
- Update ETL mappings, SQL, connectivity, and transformation logic to support migration to approved cloud-managed database platforms (including Amazon Aurora where implemented).
- Coordinate interface changes with application, database, BI, testing, cybersecurity, and production-support teams to maintain end-to-end data-flow accuracy and availability.
- Implement data profiling and data-quality controls for completeness, accuracy, timeliness, integrity, and consistency, including cleansing, standardization, reconciliation, deduplication, exception handling, and automated controls.
- Develop validation queries, reconciliation reports, automated tests, and technical evidence to confirm transformation results and end-to-end processing behavior.
- Investigate and resolve failed workflows, rejected records, data discrepancies, interface defects, processing bottlenecks, and recurring data-quality issues.
- Tune Informatica workflows, SQL, database access, transformation logic, and scheduling to improve throughput, scalability, processing time, and resource efficiency.
- Implement pipeline-level logging, metrics, alerting, audit trails, restart controls, and recovery procedures for rapid diagnosis and restoration.
- Design and optimize ingestion and transformation processes to meet IGC data-availability and latency objectives while minimizing backlogs and repeated failures.
- Maintain ETL code, SQL, configurations, and supporting artifacts in approved Git repositories using established branching, review, and configuration-management practices.
- Create ETL build, validation, test, packaging, and deployment automation within established GitLab CI/CD, coordinating platform and Runner requirements with DevSecOps.
- Use Python, Bash, PowerShell, SQL, or comparable approved technologies to develop scripting and data-processing automation.
- Apply secure data-handling practices including least privilege, encryption, secrets protection, secure transfer, logging, and auditability, and support remediation of security findings affecting ETL and ingestion components.
- Provide technical evidence and remediation support for applicable RMF and STIG, secure-configuration, testing, and release activities for assigned components.
- Prepare and maintain technical designs, data-flow diagrams, source-to-target mappings, interface specifications, data dictionaries, lineage documentation, deployment procedures, troubleshooting guides, and operational runbooks.
- Lead technical design discussions for complex ingestion, transformation, and interface requirements, identifying technical risks, dependencies, and mitigations.
- Mentor ETL developers and data engineers on Informatica development, SQL, data quality, performance tuning, testing, and maintainable integration design.
- Collaborate with software developers, database engineers, BI developers, cloud/DevSecOps engineers, cybersecurity personnel, testers, functional analysts, Product Owners, operations staff, and Government stakeholders.
- Participate in Agile planning, technical reviews, sprint demonstrations, release activities, defect resolution, and production-support efforts as assigned.
Requirements
- 8+ years of professional experience in ETL development, data engineering, data integration, database development, or a related technical field.
- Bachelor’s degree in computer science, Cybersecurity, Data Science, Software Engineering, Information Technology, Information Systems, or a related technical field; or an active/current certification from the list:
- Certified Cloud Security Professional (CCSP)
- CyberSec First Responder (CFR)
- Federal IT Security Professional–Manager (FITSP-M)
- An active/current certification from the list:
- GIAC Security Essentials Certification (GSEC)
- CompTIA Security+
- Systems Security Certified Practitioner (SSCP)
- Must be eligible to obtain a security clearance granted by the U.S. Government, requiring U.S. citizenship, using 13 adjudicative guidelines for eligibility.
- Demonstrated experience designing and supporting complex enterprise data-ingestion and transformation solutions.
- Advanced hands-on experience with Informatica PowerCenter and/or another Informatica enterprise data-integration platform, including mappings, transformations, workflows, sessions, reusable components, parameterization, and scheduling.
- Advanced SQL skills including complex queries, joins, views, stored procedures, functions, indexing, query optimization, and troubleshooting.
- Experience with relational database platforms such as Oracle, PostgreSQL, MySQL, Amazon Aurora/RDS, SQL Server, or comparable.
- Experience integrating data from APIs, web services, relational databases, message-based interfaces, and file-based sources.
- Experience designing batch, incremental, event-driven, micro-batch, or change-data-capture ingestion processes.
- Experience developing source-to-target mappings, transformation rules, metadata, lineage, and interface documentation.
- Experience implementing data-quality, validation, reconciliation, error-handling, restart, and recovery controls.
- Experience troubleshooting and tuning high-volume or mission-critical ETL and data-processing workloads.
- Experience using Python, Bash, PowerShell, SQL, or another applicable scripting/programming language for automation.
- Experience using Git and automated software-delivery processes for ETL, SQL, or data-engineering components.
- Familiarity with cloud-native data services and managed database platforms supporting enterprise ingestion and transformation workloads.
- Strong technical leadership, analytical and troubleshooting skills, documentation, mentoring, and stakeholder communication.
Technologies
- Informatica, Informatica PowerCenter, SQL
- Amazon Aurora, Amazon Aurora/RDS, Amazon RDS
- Oracle, PostgreSQL, MySQL, SQL Server
- Python, Bash, PowerShell, Git, GitLab CI/CD
- RMF, STIG
- CCSP, CFR, FITSP-M, GSEC, CompTIA Security+, SSCP
- APIs, web services, message-based interfaces
Benefits
- Instant Flexible PTO
- Generous Holidays
- Immediate Health Coverage
- 401K Safe Harbor Match
- Growth Opportunities
- Free Employee Assistance Program (EAP)
Preferred
- Experience modernizing legacy Informatica workflows or migrating data-integration capabilities to AWS-based environments.
- Experience with Informatica Intelligent Cloud Services (IICS), Informatica Cloud Data Integration, or comparable Informatica cloud capabilities.
- Experience processing large-volume, high-velocity, or mission-critical transportation and logistics data.
- Experience with Amazon Aurora PostgreSQL or Aurora MySQL, Amazon RDS, S3, Glue, Lambda, Kinesis, Database Migration Service, CloudWatch, or comparable approved cloud data services.
- Experience preparing datasets, views, or data services for Apache Superset or another enterprise business-intelligence platform.
- Familiarity with data lakes, event streaming, data catalogs, metadata management, and data-governance practices.
- Familiarity with containerized environments, infrastructure-as-code processes, and automated cloud deployment practices used by cloud/DevSecOps teams.
- Experience supporting USTRANSCOM, Department of Defense logistics, or another mission-critical federal data environment.