Spark Sql

Spark Sql Data Jobs

3 jobs found

18d ago

Pyspark/Python Data Engineer

Tata Consultancy Services Irving, TX

Irving, TX

18d ago

$100k - $120k/yr Full time Python Amazon Web Services Apache Airflow

Apply Now

Job Description

This role focuses on designing, building, and optimizing scalable PySpark data pipelines for Tata Consultancy Services in Irving, TX.

Responsibilities

Design, develop, and maintain ETL/ELT pipelines using PySpark
Craft efficient PySpark transformations with DataFrames and Spark SQL
Create modular, reusable Python-based data processing components
Uphold data quality, integrity, and performance across pipelines
Debug, tune performance, and optimize PySpark jobs
Collaborate with cross functional teams including Data Analysts, Architects, and DevOps
Contribute to CI/CD pipelines and deployment workflows for data applications
Monitor and troubleshoot production data workloads

Requirements

Hands-on PySpark experience with Spark SQL and DataFrame API
Advanced Python skills for data processing, performance tuning, and modular coding
Solid understanding of ETL design patterns and data pipeline architecture
Working knowledge of SQL for data transformation and analysis
Experience with data processing in distributed environments
3 to 8 years of experience in Data Engineering or PySpark development
Proven hands-on project experience using PySpark and Python
Bachelor of Computer Science

Must Have Technical / Functional Skills

Strong hands-on experience in PySpark and Python for designing scalable data processing pipelines
Practical knowledge of distributed data processing and production-grade Python coding

Technologies

PySpark, Spark SQL, DataFrame API
Python, SQL
Amazon S3, AWS Glue, AWS EMR
Airflow, Snowflake, Git

Experience

3–8 years of experience in Data Engineering or PySpark development
Proven hands-on project experience in PySpark and Python

Job Details

Salary: USD 100,000 – 120,000 per year
Location: Irving, TX (onsite)
Job Function: TECHNOLOGY
Role: Engineer
Job ID: 416300
Education: Bachelor of Computer Science

View full job page →

19d ago

Data Engineer

INSPYR Solutions Spring, TX

Spring, TX

19d ago

$70 - $85/hr Contract Analytics Azure Azure Data Factory

Apply Now

Job Description

We are seeking a Data Engineer for a contract role with INSPYR Solutions, based onsite in Spring, TX. The position focuses on designing scalable data models and Azure-based pipelines to enable organization-wide analytics and business intelligence. The assignment requires onsite work in Spring, TX, three days per week.

Job Details

Location: Spring, TX (onsite)
Job Type: Contract
Salary: USD 70 - 85 per hour
Required Experience: 5 years minimum

Responsibilities

Design and maintain dimensional data models and semantic layers to support enterprise reporting and analytics, applying data warehousing and architecture best practices.
Develop scalable, modular ETL and ELT pipelines using Azure Data Factory, Synapse Pipelines, and Databricks.
Transform raw data into analytics-ready structures with Spark SQL, DBT, and Azure Synapse; implement data quality checks and schema evolution strategies.
Leverage Azure services including SQL Server, Synapse Analytics, Databricks, and Delta Lake to build and manage scalable data infrastructure and pipelines.
Integrate data workflows with Azure DevOps and Git for version control, automated testing, and continuous deployment.
Collaborate with analysts and BI developers to ensure data models support reporting and visualization in tools such as Power BI.
Monitor data pipelines and models, troubleshoot issues, and optimize for high availability and performance.
Continuously adopt and apply current data engineering practices, tools, and technologies.

Requirements

5 to 7 years of experience in data engineering or related roles.
Proven expertise in data modeling, particularly dimensional modeling, and data warehouse design.
Hands-on experience with Azure Data Factory, Synapse Analytics, Databricks, and DBT.
Strong SQL and Spark SQL skills.
Experience with CI/CD, DevOps, and version control systems such as Git and Azure DevOps.
Familiarity with Power BI or similar visualization tools.
Solid understanding of data integration, data quality, and data transformation best practices.
Demonstrated problem-solving ability and a collaborative mindset.

Technologies

Azure Data Factory
Azure Synapse Analytics
Databricks
DBT
Spark SQL
SQL Server
Delta Lake
Power BI
Azure DevOps
Git

Benefits

Comprehensive medical benefits
Competitive pay with a 401(k) plan
Retirement plan
Additional benefits available

View full job page →

2mo ago

Sr Data Engineer - Databricks

CGI Group, Inc. Miami, FL

Miami, FL

2mo ago Closed

$90k - $198k/yr Full time Senior Data Engineer Data Integration

This position is no longer accepting applications (closed Jun 12, 2026).

Job Description

Senior data engineer specializing in Databricks and Spark, delivering production-grade data pipelines and data products in a hybrid Miami environment.

Responsibilities

Architect and implement ETL and ELT pipelines on Databricks Spark to support both batch and incremental workloads.
Establish reliable data ingestion from diverse sources including files, databases, APIs, and streaming when applicable.
Tune Spark jobs for performance, scalability, and cost control.
Enforce data quality, reconciliation, and observability across end-to-end pipelines.
Create advanced SQL transformations for cleansing, enrichment, and aggregation.
Build and maintain analytical data models such as fact and dimension schemas, curated layers, and reporting views.
Provide clean, well-structured datasets to support downstream reporting, analytics, and data science use cases.
Configure and manage data exposure via APIs for internal and external consumers.
Collaborate with application and integration teams to define API contracts and data payload formats.
Establish secure, scalable, and high-performance data access patterns.
Support versioning and backward compatibility for published data services.

Requirements

Minimum five years of experience in data engineering or related roles.
Hands-on expertise with Databricks and Apache Spark.
Advanced SQL skills, including complex joins, window functions, and performance tuning.
Proven experience building and managing ETL and ELT pipelines.
Experience configuring and maintaining data exposure via APIs.
Solid understanding of data warehousing and analytics concepts.
Strong problem-solving and analytical abilities.
Ability to work independently on complex data problems.
Clear communication with both technical and non-technical stakeholders.
Comfortable working in an agile, product-oriented environment.
Excellent verbal and written communication; ability to document processes and frameworks clearly for future reference; dependable, accurate, and professional.
Self-motivated and capable of working without close supervision.

Technologies

Databricks
Apache Spark
SQL

Compensation

Salary range: USD 89,600 - 198,400 per year.

Benefits

Competitive compensation and total rewards
Comprehensive insurance options
401(k) plan with company matching and an employee share purchase plan
Paid time off for vacation, holidays, and sick time
Paid parental leave
Learning opportunities and tuition assistance
Wellness and wellbeing programs

Skills

Cloud data platform design
Data engineering
Databricks
ETL

What You Can Expect From Us

We operate with a sense of ownership and collaboration. Life at CGI Partners emphasizes teamwork, respect, and belonging, with opportunities to grow and contribute from day one as we pursue shared goals. Your work will create value, and you will develop innovative solutions as part of a collective effort to shape the company’s strategy and direction.

View full job page →

Spark Sql Data Jobs

Pyspark/Python Data Engineer

Data Engineer

Sr Data Engineer - Databricks

Get Spark Sql Data Jobs alerts