Store AI Agent - Onshore Data Engineer

On-site contract opportunity in Dallas, TX for an Onshore Data Engineer focused on Databricks-based data and ML pipeline development.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks, PySpark, and Python
• Build and optimize batch and real-time data ingestion frameworks from diverse data sources
• Develop ETL/ELT solutions for enterprise data warehousing, analytics, and AI/ML workloads
• Implement data quality checks, validation frameworks, monitoring, and observability solutions
• Optimize Databricks jobs, notebooks, clusters, and workflows for performance and cost efficiency
• Design and maintain Delta Lake architectures, data models, and metadata management solutions
• Collaborate with Data Scientists and ML Engineers to prepare training datasets and feature stores
• Support development and deployment of deep learning solutions using PyTorch
• Build scalable feature engineering pipelines and data transformation frameworks
• Enable model training, experimentation, and inference infrastructure on Databricks
• Integrate ML workflows with MLOps pipelines and CI/CD processes
• Apply DataOps and MLOps best practices
• Automate deployments using CI/CD pipelines
• Use cloud-native services for storage, security, monitoring, and orchestration
• Support infrastructure-as-code and environment automation initiatives
• Translate business, analytics, and data science requirements into technical data solutions
• Support compliance with data governance, privacy, and security standards
• Participate in architecture reviews and technology evaluations

Requirements

• Strong hands-on experience with the Databricks Platform
• Expertise in Python
• Strong hands-on experience with PySpark
• Experience with PyTorch for machine learning and deep learning workloads
• Knowledge of Delta Lake, Unity Catalog, and Databricks Workflows
• Experience building ETL/ELT pipelines
• Strong SQL skills and data modeling experience
• Experience with distributed computing and big data architectures
• Knowledge of REST APIs and data integration patterns
• Git version control and CI/CD tools experience
• Experience with one or more: Microsoft Azure (preferred), AWS, GCP
• Experience with one or more: Spark, Kafka/Event Streaming, Azure Data Factory or equivalent integration tools, Data Lake architectures, Data Warehousing solutions, Feature Stores
• Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related field
• 5+ years of experience in Data Engineering
• 3+ years of experience with Databricks and cloud-based data platforms
• Experience supporting machine learning and AI use cases using PyTorch

Preferred Skills

• Experience with GenAI and LLM data pipelines
• Knowledge of MLflow and MLOps frameworks
• Hands-on experience with Docker and Kubernetes
• Experience with real-time streaming architectures
• Familiarity with vector databases and AI platforms
• Certifications in Databricks or cloud platforms

Nice-to-Have Experience

• Retail, QSR, Restaurant Technology, Digital Commerce, or Customer Analytics domains
• Large-scale enterprise data modernization initiatives
• AI/ML and advanced analytics implementations

Technologies

• Databricks Platform, Python, PySpark
• PyTorch
• Delta Lake, Unity Catalog, Databricks Workflows
• ETL/ELT, SQL
• REST APIs, Git, CI/CD
• Microsoft Azure, AWS, Google Cloud Platform (GCP)
• Spark, Kafka/Event Streaming, Azure Data Factory
• Feature Stores

Compensation and Benefits

• Pay range: $39.30/hour - $61.40/hour
• Benefits: medical, dental, vision, retirement benefits
• Job type: contract
• Location: Dallas, TX, United States (onsite)

Applications will be accepted on an ongoing basis.