Data/Analytics Engineer

Responsibilities

• Develop and scale a governed Databricks platform that serves as the authoritative data source across housing, justice, and clinical systems.
• Architect and deploy automated, production-grade data pipelines and CI/CD workflows using Databricks, Terraform, and GitHub.
• Translate complex program logic into scalable data models and reusable data assets.
• Create scalable semantic models that map a client’s journey across multiple programs, standardizing touchpoints over time.
• Institute secure-by-design data systems handling sensitive PHI and CJI data with Unity Catalog for centralized governance, audit logging, and access control.
• Enable advanced analytics by delivering curated datasets, APIs, and environments for machine learning and NLP applications.
• Collaborate with leadership and cross-functional teams to shape data usage for policy, operations, and outcomes.
• Provide technical leadership by mentoring staff and establishing engineering standards for long-term scalability and maintainability.
• Programmatically map complex, multi-source program utilization (CHAMP, DD, HMIS) to funding streams for strict fiscal and grant tracking compliance.
• Design, implement, and maintain a scalable Databricks lakehouse using Medallion principles; build ETL/ELT pipelines that integrate structured and unstructured data across systems, and ensure interoperability with standards such as FHIR and HL7v2.
• Automate data lifecycles with GitHub Actions and Terraform, applying software engineering practices like version control, automated testing, and continuous integration before production deployment.
• Design and maintain the Unified Data Model for Community Programs, leading architectural efforts to stitch cross-program client journeys and develop multi-dimensional structures mapping utilization to funding streams.
• Establish enterprise data governance, including RBAC/ABAC, data lineage, audit logs, and data loss prevention to safeguard regulated data.
• Develop scalable data products, curated datasets, APIs, and analytical layers that support reporting, dashboards, and advanced analytics; optimize performance with SQL and BI tools.
• Provide technical leadership and delivery oversight for data engineers, analysts, and scientists, promoting best practices in system design.
• Build and maintain environments required for advanced analytics, partnering with data scientists and DHS Security teams to ensure models and data products are scalable and compliant.

Requirements

• Significant experience in Enterprise Data Architecture.
• Strong background in Cloud Infrastructure.
• Extensive experience in Analytics Engineering.
• A relevant bachelor’s degree.
• Proven track record managing complex data lifecycles within large-scale Databricks environments.
• Solid background in Cloud Data Engineering.
• Experience with Infrastructure Automation.
• Option I: Two years in a lead capacity delivering complex data infrastructure and architecture projects, including automated ETL/ELT pipelines, Lakehouse management (Databricks), and enterprise data security at a level equivalent to the Los Angeles County Principal Information Systems Analyst.
• Option II: Bachelor’s degree in Information Technology, Computer Science, Data Engineering, or Data Science with six years applied experience in data engineering, infrastructure automation (CI/CD), and enterprise data management; two years in a lead role. A Master’s or Doctoral degree may substitute up to two years of general experience.
• DHS Live Scan is required.
• Expert proficiency in Python focusing on data manipulation with Pandas and PySpark, and automation.
• Expert proficiency in SQL including query optimization, CTEs, and window functions.
• Advanced data visualization skills for executive insights in Tableau or Power BI.
• Cloud automation experience with Terraform and GitHub Actions.

Technologies

• Databricks
• Terraform
• GitHub
• GitHub Actions
• Unity Catalog
• SQL
• Tableau
• Power BI
• Python
• Pandas
• PySpark

Physical Demands

• Stand: Occasionally
• Walk: Occasionally
• Sit: Frequently
• Handling / Fingering: Frequently
• Reach Outward: Occasionally
• Reach Above Shoulder: Occasionally
• Climb, Crawl, Kneel, Bend: Occasionally
• Lift / Carry: Occasionally up to 25 lbs
• Push / Pull: Occasionally up to 25 lbs
• See: Constantly
• Taste / Smell: Not Applicable

Work Environment

General office setting with indoor climate control.

Selection Process

Live technical interview: shortlisted candidates will complete a live technical assessment to test SQL and Python proficiency, architectural problem solving within the Databricks ecosystem, and the ability to translate programmatic logic into scalable data models.