Senior AWS Python Developer / PySpark Data Engineer
Backend Developer
Python
Analytics
Analytics Engineer
Aws Data Engineer
Aws Data Engineering
Aws Data Lake
Aws Data Lakehouse
Aws Data Services
Aws S3 Data Lake
Aws Solutions Architect
Big Data
Bigdata
Cloud Data Engineering
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Integration
Data Integration Tools
Data Pipeline
Data Platform
Data Processing
ETL
Pyspark
Software Development
Spark
SQL
Job Description
System One is hiring a Senior AWS Python and PySpark Data Engineer for hands-on cloud data processing and scalable production workloads across hybrid locations.
Responsibilities
- Develop scalable Python applications and data processing solutions
- Integrate APIs into data workflows and application services
- Write and optimize complex SQL queries for analysis and transformations
- Process large datasets using Apache Spark and AWS services
- Troubleshoot application and data processing performance issues
- Contribute to CI/CD automation
- Collaborate with technical and business teams across the organization
Requirements
- 8+ years of hands-on Python development, including data-intensive and big data applications
- Strong Python fundamentals and experience building scalable, production-grade applications
- 5+ years working with Apache Spark/PySpark, including troubleshooting and optimizing Spark jobs
- 5+ years hands-on development with AWS: Amazon EMR, AWS Lambda, Amazon EventBridge, AWS Step Functions, Amazon Redshift, Amazon S3, and AWS Glue
- Advanced SQL skills, including complex queries, data transformations, joins, and recursive CTEs
- Experience developing and consuming REST APIs and integrating applications with external services and data sources
- Experience working with large datasets, including data manipulation, transformation, analysis, and processing
- Big data technologies experience such as Hadoop/EMR, Spark, and Hive
- 4+ years of experience working with AI-related technologies or applications
- Hands-on CI/CD and infrastructure automation using GitLab and Terraform
- Strong debugging, performance tuning, analytical, and problem-solving skills
- Ability to collaborate effectively across development teams in a matrixed organization
- Bachelor’s degree in Computer Science, Information Systems, or a related field
Technologies
- Python, PySpark, Apache Spark
- SQL
- Amazon EMR, AWS Lambda, Amazon EventBridge, AWS Step Functions, Amazon Redshift
- Amazon S3, AWS Glue
- Hadoop, Hive
- REST APIs
- GitLab, Terraform, CI/CD
- AWS services
Location and Work Model
- Hybrid across: Lafayette, LA · Knoxville, TN · Columbia, SC · Birmingham, AL
Job Type and Role Focus
- Direct Hire
- Primarily a hands-on data engineering role focused on AWS big data and data processing
Note: Prior cybersecurity or security data environment experience is ideal, based on the role focus.
Similar Jobs
