Data Jobs in Clemmons, NC

1 job found

Include remote jobs

28d ago

Lead Data Engineer

Circle K Clemmons, NC

Clemmons, NC

28d ago

Full time Azure Data Factory Big Data Bigquery

Apply Now

Job Description

Circle K seeks a Lead Data Engineer to steer the design, development, and deployment of data solutions that yield actionable insights. This onsite role in Clemmons, NC leads a team of data engineers and collaborates with cross functional teams to strengthen data infrastructure, CI/CD pipelines, and analytics capabilities. A Bachelor’s or Master’s degree is required, with a minimum of eight years of experience in data engineering.

Responsibilities

Apply advanced data engineering principles to design and implement data loading and aggregation frameworks across diverse areas of the organization.
Collect and process raw data in structured, semi-structured, and unstructured forms using batch and real-time processing frameworks.
Build and optimize data solutions within enterprise data warehouses and big data repositories, with a emphasis on cloud migration.
Advance capabilities for Enterprise Data Platform partners to meet the needs of product, engineering, and business teams.
Leverage enterprise system experience with Databricks, Snowflake, and cloud platforms such as Azure, AWS, and GCP.
Utilize Python, Spark, and SQL to construct robust pipelines for efficient data processing and analysis.
Develop CI/CD pipelines to automate the build, test, and deployment processes for data solutions.
Apply data modeling techniques to design and optimize data schemas, ensuring data integrity and performance.
Lead continuous improvement efforts to enhance the performance, reliability, and scalability of the data infrastructure.
Collaborate with data scientists, analysts, and other stakeholders to translate business requirements into technical solutions.
Implement best practices for data governance, security, and compliance to safeguard data assets.

Requirements

Bachelor’s or Master’s degree in computer science, engineering, or a related field.
Eight or more years of experience in a data engineering role, with proven success designing and building data pipelines, ETL processes, and data warehouses.
Strong proficiency in SQL, Python, and Spark.
Extensive experience with cloud platforms such as AWS, Azure, or Google Cloud Platform.
Hands-on work with big data technologies including Hadoop, Spark, Kafka, and distributed computing frameworks.
Knowledge of data lake and data warehouse tools such as Databricks, Snowflake, Amazon Redshift, Google BigQuery, Azure Data Factory, and Airflow.
Experience implementing CI/CD pipelines for automated build, test, and deployment of data solutions.
Solid understanding of data modeling, data warehousing architectures, and data management best practices.
Excellent communication and leadership abilities, with a track record of collaborating across cross-functional teams and guiding technical decisions.
Relevant certifications (for example Azure, Databricks, Snowflake) are a plus.

Technologies

Python
Spark
SQL
Databricks
Snowflake
AWS
Azure
GCP
Hadoop
Kafka
Airflow
Google BigQuery
Amazon Redshift
Azure Data Factory

View full job page →

Get Job Alerts

New jobs delivered to your inbox.