J
Data Engineer III - Python, Databricks
Python
Amazon Web Services
AWS
Aws Glue
Big Data
Bigdata
Cloud
Cloud Computing
Cloud Data Engineering
Cloud Data Platform
Cloud Data Warehouse
Cloud Platform
Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Integration
Data Lakehouse
Data Management
Data Modeling
Data Pipeline
Data Platform
Data Processing
Data Warehouse
Database
Databases
Databricks
ETL
Informatica
Information Technology (IT)
IT Services
Programming
Programming Language
Programming Languages
Security Data Engineering
Spark
SQL
Job Description
Build secure, scalable data pipelines and analytics foundations using Python and Databricks in an onsite Houston environment.
Responsibilities
- Develop workflows and ELT pipelines using Python and Databricks.
- Support review of controls to ensure sufficient protection of enterprise data.
- Implement data security using entitlements frameworks.
- Update logical or physical data models as new use cases emerge.
- Use SQL frequently and apply working knowledge of NoSQL databases.
- Apply enterprise-authorized AI capabilities within the work environment to accelerate data pipeline and design analysis/documentation.
- Validate AI-assisted outputs and handle data according to sensitivity and security requirements.
- Use reuse-first, AI-assisted practices to strengthen SDLC-quality routines for data pipelines (including test generation and control validation).
- Ensure traceability/auditability and alignment to resiliency and security expectations.
Requirements
- Formal training or certification on software engineering concepts and 3 years of applied experience.
- Good working knowledge of AWS, Databricks, and Python, with experience across the data lifecycle.
- Advanced SQL skills, including joins and aggregations; working understanding of NoSQL databases.
- Significant experience with statistical data analysis and the ability to select appropriate tools and data patterns.
- Ability to utilize AWS Cloud Services to develop, deploy, and manage applications at scale.
- Proficiency with AI coding assistants, using tools daily such as Cursor, GitHub Copilot, and Claude for code generation, documentation, and refactoring.
- Prompt engineering skills: provide context, clear goals, relevant source material, and defined output expectations to produce accurate, usable code.
- Critical evaluation and validation to identify hallucination patterns, security vulnerabilities, and logic errors in AI-generated code before production deployment.
- Demonstrated experience using enterprise-authorized AI capabilities to support data engineering workflows with strong validation habits and awareness of data sensitivity.
- Ability to review and validate AI-assisted outputs (query suggestions, test ideas, model change summaries) before use; escalate when uncertain and follow data handling requirements.
Preferred Qualifications, Capabilities, and Skills
- Familiarity with Standardized data layer practices, including Medallion architecture.
- Exposure to Aurora Postgres and MongoDB.
- Experience developing and supporting AWS GLUE Jobs and Federated Data Lake.
- Skills in designing efficient data models, including normalization, denormalization, and schema design, with understanding of relational and star schemas.
- Augmented development workflow: integrating tools into CI/CD pipelines, using containerization (for example, Docker), and leveraging AI to bridge language gaps (for example, transitioning between Python, JavaScript, or Java).
Technologies
- Python, Databricks, ELT, SQL, NoSQL
- AWS, AWS GLUE Jobs, Federated Data Lake, Aurora Postgres
- Medallion architecture
- Cursor, GitHub Copilot, Claude
- Docker, CI/CD
- MongoDB, Federated Data Lake
- JavaScript, Java
Benefits
- Comprehensive health care coverage
- On-site health and wellness centers
- Retirement savings plan
- Backup childcare
- Tuition reimbursement
- Mental health support
- Financial coaching
Location: Houston, TX (onsite)
Minimum Experience: 3 years
Similar Jobs
J
J
J