Brooksource is hiring a Senior Data Engineer for a 12-month contract-to-hire role based on-site in Kansas City, MO. In this position, you’ll design and build dependable data pipelines that turn raw inputs into clean, consumable datasets for enterprise analytics and downstream AI/ML consumption.
This role focuses on hands-on ETL/ELT work, data modeling, and keeping pipelines operating reliably in production through monitoring, validation, and governance practices.
Responsibilities
- Design, develop, and maintain scalable and reliable data pipelines and platforms for enterprise analytics and reporting.
- Work with business and technical teams to understand data requirements and translate them into data engineering solutions.
- Create ingestion frameworks using Kafka Confluent and integrate data from multiple cloud and on-premises sources.
- Build ELT/ETL pipelines using PySpark, SQL, Dataform, DBT, and Dataproc.
- Design and manage data models, schemas, and data warehousing structures for performance and scalability.
- Ensure data quality, consistency, and integrity using validation and monitoring mechanisms.
- Implement data governance practices including data lineage, metadata management, privacy, and compliance.
- Develop automated monitoring and alerting to support reliability, availability, and performance.
- Collaborate with cross-functional teams to support downstream reporting, analytics, and AI/ML model consumption.
- Follow best practices for version control, documentation, and CI/CD using Git and infrastructure-as-code tools.
- Provide operational support for data pipelines and troubleshoot performance issues proactively.
- Continuously optimize data processing, query execution, and resource utilization in Azure environments.
- Build and maintain RESTful API integrations to support data ingestion and synchronization with web applications and enterprise systems.
- Contribute to lightweight web application development efforts when pipelines interface directly with front-end or full-stack components.
Requirements
- Bachelor Degree in Computer Science or related field and 8 years related professional experience (required).
- Associate Degree and 10 years related professional experience (required).
- High School Diploma/GED and 12 years related professional experience (required).
- Strong programming skills in Python and PySpark for large-scale data processing.
- Proficiency in SQL for data manipulation, analysis, and performance tuning.
- Experience with Dataform, Dataproc, and BigQuery for pipeline development and orchestration.
- Understanding of data governance principles and implementation practices.
- Experience using Git for version control and CI/CD pipelines for automated deployments.
- Experience building and consuming RESTful APIs, including enterprise integrations such as OAuth2, OpenID Connect, webhooks, pagination, rate limiting, retry logic, and error handling.
- Some hands-on web application development experience (full-stack or front-end).
- Candidates must be legally authorized to work permanently in the country where this position is located, without time limitations, restrictions, or the need for work sponsorship.
Technologies
- Python, PySpark, SQL
- Dataform, DBT, Dataproc, BigQuery
- Kafka, Confluent
- Azure, Git
- RESTful APIs
- OAuth2, OpenID Connect, webhooks
- Infrastructure as Code
- RESTful API integrations
Nice to Have
- Experience configuring and building pipelines within Databricks.
- Working knowledge of Infrastructure as Code (IaC) for cloud resource management and automation.
- Familiarity with DBT, Machine Learning, and AI concepts.
- Hands-on experience with Kafka and Confluent for real-time data streaming.
- Knowledge of Cloud Scheduler and Dataflow for automation and workflow management.
- Full-stack application development experience using Node.js, TypeScript, React, and Express.
- Experience designing contract-first APIs using OpenAPI, Orval, and Zod.
- Proficiency with React, Vite, Tailwind CSS, shadcn/ui, and TanStack Query.
- Experience with PostgreSQL and ORM frameworks (Drizzle ORM preferred).
- Experience integrating engineering, construction, ERP, or project management systems.
- Experience with metadata-driven applications, configurable mappings, and schema transformation frameworks.
Benefits
- Dental insurance
- Health insurance
- Vision insurance
Additional Benefits & Perks
- Competitive medical, dental, vision, Health Savings Account, Dependent Care FSA, and supplemental coverage.
- 401k plan with company match and fully vested after eligibility, plus paid time off, sick time, and paid company holidays.
- Employee Assistance Program (EAP) including virtual counseling, financial services, legal services, and life coaching.
Compensation: USD 55 to 60 per hourly. Pay disclaimer: The pay range is a general guideline only and not a guarantee of compensation. Factors considered include responsibilities, education, experience, knowledge, skills, abilities, internal equity, alignment with market data, applicable bargaining agreements (if any), and other law.
