DataJobs.io
← Back to all jobs

Job Description

Robotics Technologies is hiring a Senior Data Engineer for an onsite role in Boston, MA, supporting Colorado’s Statewide Longitudinal Data System (SLDS/COdata). In this position, you will design, develop, optimize, and operate scalable ETL/ELT pipelines that feed a centralized Snowflake data platform, with a strong emphasis on data quality, governance, identity matching, and DevOps automation.

Role Focus

You will build and maintain automated data ingestion and transformation workflows using Snowflake and Informatica, integrate data from multiple source systems, and ensure reliable recurring feed cycles. Responsibilities include implementing data quality checks, maintaining lineage and audit trails, and complying with applicable education and privacy regulations.

Key Responsibilities

  • Design, develop, test, and deploy automated ETL/ELT pipelines using Snowflake and Informatica.
  • Integrate data from state agencies, educational institutions, and third-party source systems.
  • Develop complex SQL queries, transformations, stored procedures, and data processing workflows.
  • Build and maintain source-to-target mappings and data integration processes.
  • Ingest and transform structured data into standardized longitudinal data models.
  • Develop Informatica workflows to automate data ingestion, cleansing, and transformation.
  • Maintain Snowflake tables, views, schemas, and transformation objects.
  • Optimize Snowflake queries, warehouse utilization, and pipeline performance; monitor workloads and resolve bottlenecks.
  • Implement scalable, cost-efficient data processing solutions and maintain RAW, transformation, and analytics data layers.
  • Implement automated data quality checks including structural and format validation, completeness and duplicate detection, referential integrity, and business logic validation.
  • Perform data cleansing, reconciliation, and exception handling, including record linkage and identity-matching across agency datasets.
  • Maintain data lineage, metadata, audit trails, and data governance standards.
  • Ensure compliance with FERPA, COPPA, ADA, and applicable state data privacy regulations; implement data suppression and de-identification requirements.
  • Monitor scheduled pipelines and troubleshoot failed jobs, ingestion issues, and performance problems.
  • Manage late, corrected, or malformed source files and perform data reloads; reconcile source and target record counts.
  • Maintain operational runbooks, incident logs, and pipeline documentation; gradually assume independent ownership of production pipelines and operations.
  • Transition production pipeline responsibilities from a third-party implementation partner, including knowledge-transfer participation.
  • Develop and maintain CI/CD pipelines using Azure DevOps, using Git for branching, pull requests, and code reviews.
  • Implement Infrastructure as Code using Terraform, manage code promotion across Development, Test, UAT, and Production, and execute testing via Azure Test Plans.
  • Collaborate with Data Architects on data models and standards; work with Data Scientists and Visualization Designers to support Tableau dashboards and analytics; coordinate with Operations Engineers on Snowflake warehouse management and deployments.

Project-Specific Work

  • Develop and maintain data ingestion pipelines for Colorado Department of Higher Education (CDHE) course-level data.
  • Support conditional ingestion of postsecondary demographic, Pell Grant eligibility, and disability-related data.
  • Integrate PSAT/SAT assessment data and perform student identity matching.
  • Support existing data feeds, including NSC, K-3, CDEC, and Phase 1 integrations.
  • Participate in knowledge transfer and operational transition from January through April 2027.
  • Stabilize analytics-layer delivery, improve data reliability, and tune performance.
  • Develop de-identified and privacy-compliant data extracts for participating higher education institutions.

Expected Deliverables

  • Production-ready Informatica ETL workflows and Snowflake transformation pipelines.
  • Automated data cleansing and validation modules.
  • Identity matching and record-linkage routines.
  • Snowflake analytics-layer tables, views, and data models.
  • Data mapping, lineage, and dependency documentation.
  • Operational runbooks and incident management procedures.
  • Azure Test Plans artifacts including test execution records and defect logs.
  • Release change logs and production feed reconciliation reports.

Minimum Requirements

  • 5+ years developing, managing, and optimizing ETL/ELT pipelines and data transformations within Snowflake.
  • 1+ year hands-on experience with Informatica Cloud Data Governance and Catalog (CDGC), including metadata management, data profiling, cataloging, and data quality validation.
  • 1+ year experience with Infrastructure as Code (IaC), including provisioning and managing cloud infrastructure.
  • Strong experience building automated CI/CD pipelines for build, testing, and deployment.
  • Strong proficiency with version control, branching strategies, pull requests, merge management, and code reviews.
  • Strong SQL development and data transformation skills.
  • Experience designing and implementing complex data ingestion and integration pipelines.
  • Experience with data quality, data validation, cleansing, and reconciliation processes.
  • Ability to troubleshoot production data pipelines and manage operational support activities.

Preferred Qualifications

  • Experience working with state government, public-sector, or education data systems.
  • Familiarity with SLDS or longitudinal student and workforce data.
  • Experience integrating data from multiple government agencies and external partner systems.
  • Experience with record linkage, entity resolution, and deterministic or probabilistic identity matching.
  • Knowledge of FERPA, COPPA, student data privacy, and data suppression standards.
  • Experience with Informatica ETL workflows and metadata-driven data integration.
  • Experience with Azure Test Plans and structured software testing processes.
  • Familiarity with Tableau data integration and analytics-layer development.
  • Experience working with education and workforce datasets from organizations such as CDE, CDHE, CDLE, CDEC, and National Student Clearinghouse.

Technology Stack

  • Snowflake
  • Informatica
  • Informatica Cloud Data Governance and Catalog (CDGC)
  • Terraform
  • Azure DevOps
  • Git
  • Azure Test Plans
  • SQL
  • Tableau

Location: Boston, MA (onsite). Salary: USD 76 per hour. Experience: Minimum 5 years.

Similar Jobs