DataJobs.io
← Back to all jobs

Job Description

Build secure, scalable data pipelines and analytics foundations using Python and Databricks in an onsite Houston environment.

Responsibilities

  • Develop workflows and ELT pipelines using Python and Databricks.
  • Support review of controls to ensure sufficient protection of enterprise data.
  • Implement data security using entitlements frameworks.
  • Update logical or physical data models as new use cases emerge.
  • Use SQL frequently and apply working knowledge of NoSQL databases.
  • Apply enterprise-authorized AI capabilities within the work environment to accelerate data pipeline and design analysis/documentation.
  • Validate AI-assisted outputs and handle data according to sensitivity and security requirements.
  • Use reuse-first, AI-assisted practices to strengthen SDLC-quality routines for data pipelines (including test generation and control validation).
  • Ensure traceability/auditability and alignment to resiliency and security expectations.

Requirements

  • Formal training or certification on software engineering concepts and 3 years of applied experience.
  • Good working knowledge of AWS, Databricks, and Python, with experience across the data lifecycle.
  • Advanced SQL skills, including joins and aggregations; working understanding of NoSQL databases.
  • Significant experience with statistical data analysis and the ability to select appropriate tools and data patterns.
  • Ability to utilize AWS Cloud Services to develop, deploy, and manage applications at scale.
  • Proficiency with AI coding assistants, using tools daily such as Cursor, GitHub Copilot, and Claude for code generation, documentation, and refactoring.
  • Prompt engineering skills: provide context, clear goals, relevant source material, and defined output expectations to produce accurate, usable code.
  • Critical evaluation and validation to identify hallucination patterns, security vulnerabilities, and logic errors in AI-generated code before production deployment.
  • Demonstrated experience using enterprise-authorized AI capabilities to support data engineering workflows with strong validation habits and awareness of data sensitivity.
  • Ability to review and validate AI-assisted outputs (query suggestions, test ideas, model change summaries) before use; escalate when uncertain and follow data handling requirements.

Preferred Qualifications, Capabilities, and Skills

  • Familiarity with Standardized data layer practices, including Medallion architecture.
  • Exposure to Aurora Postgres and MongoDB.
  • Experience developing and supporting AWS GLUE Jobs and Federated Data Lake.
  • Skills in designing efficient data models, including normalization, denormalization, and schema design, with understanding of relational and star schemas.
  • Augmented development workflow: integrating tools into CI/CD pipelines, using containerization (for example, Docker), and leveraging AI to bridge language gaps (for example, transitioning between Python, JavaScript, or Java).

Technologies

  • Python, Databricks, ELT, SQL, NoSQL
  • AWS, AWS GLUE Jobs, Federated Data Lake, Aurora Postgres
  • Medallion architecture
  • Cursor, GitHub Copilot, Claude
  • Docker, CI/CD
  • MongoDB, Federated Data Lake
  • JavaScript, Java

Benefits

  • Comprehensive health care coverage
  • On-site health and wellness centers
  • Retirement savings plan
  • Backup childcare
  • Tuition reimbursement
  • Mental health support
  • Financial coaching

Location: Houston, TX (onsite)

Minimum Experience: 3 years

Similar Jobs