Data Engineer
Python
Apache Airflow
Azure Data Factory
Azure Synapse Analytics
Big Data
Bigdata
Bigquery
Cloud
Cloud Platform
Data Architecture
Data Engineer
Data Integration
Data Lake
Data Lakehouse
Data Pipeline
Data Pipelines
Data Platform
Data Processing
Data Warehouse
Data Warehousing
Database
Databases
Databricks
Delta Lake
ETL
Kafka
Microsoft Azure
Snowflake
Spark
SQL
Stream Processing
Job Description
SN Cloud Solutions is seeking a Data Engineer to join its Chicago team in a hybrid arrangement. The role focuses on designing, building, and maintaining scalable data pipelines and data platforms that underpin enterprise analytics and BI, leveraging Python, PySpark, SQL, and cloud ETL/ELT tools. You will partner with data scientists, analysts, and business stakeholders to turn raw data into reliable insights.
Responsibilities
- Architect and implement ETL/ELT pipelines for large-scale data processing.
- Create scalable data solutions using Python, PySpark, and Apache Spark.
- Build and sustain data ingestion frameworks for both structured and unstructured data.
- Develop and tune SQL queries, stored procedures, and database objects.
- Ingest and consolidate data from diverse sources, including APIs, databases, and cloud storage.
- Establish data quality, validation, and monitoring mechanisms.
- Deploy and operate on cloud platforms such as AWS, Azure, or GCP.
- Create and manage data lakes and data warehouses.
- Partner with data scientists, analysts, and business stakeholders.
- Orchestrate workflows with tools such as Apache Airflow.
- Diagnose production issues and optimize pipeline performance.
- Follow Agile/Scrum practices and CI/CD processes.
Requirements
- Experience as a Data Engineer.
- Strong experience with Python and PySpark.
- Hands-on experience with Apache Spark.
- Strong SQL programming skills.
- Experience with ETL/ELT development.
- Experience with cloud platforms (AWS, Azure, or GCP).
- Knowledge of data warehousing concepts.
- Experience with Apache Airflow or similar orchestration tools.
- Experience with Git, Jenkins, or Azure DevOps.
- Strong understanding of data modeling and database design.
- Experience with relational and NoSQL databases.
- Excellent problem-solving and communication skills.
Experience
- Azure Data Factory: 7 years (Required)
- SQL: 3 years (Required)
- Data warehouse: 4 years (Required)
Technologies
- Python
- PySpark
- Apache Spark
- SQL
- AWS
- Azure
- GCP
- Apache Airflow
- Databricks
- Kafka
- Delta Lake
- Snowflake
- Redshift
- BigQuery
- Synapse
- Azure Data Factory
- Git
- Jenkins
- Azure DevOps
Benefits
- Flexible schedule
Job Type
Full-time
Pay
From USD 50.00 per year
Work Location
Hybrid remote in Chicago, IL 60618
Similar Jobs
