Data Engineer

The Data Engineer role at Realign LLC is a full-time, permanent position based in Charlotte, NC (onsite). The work focuses on complex data solution engineering with a strong emphasis on big data processing and cyber security domain knowledge, including related vulnerability data management.

Role Overview

In this role, you will participate as a team member under the supervision of a more senior domain expert. Primary interactions include business partners and stakeholders, Product Owner, Scrum Master, development team, feature lead, architect, domain community of practice, and data scientists.

Objective of role: knowledge in the cyber security domain.

Key Responsibilities

• Ensure the design and engineering approach for complex data solutions remains consistent across multiple flows and systems.
• Perform research, design, and prototyping activities, including evaluating new toolsets.
• Utilize multiple architectural components to design and develop client requirements.
• Maintain, improve, clean, and manipulate data for operational and/or analytics data systems.
• Assemble large, complex datasets aligned to functional and non-functional requirements.
• Build processes for data transformation, data structures, metadata, data quality controls, dependency tracking, and workload management.
• Define and build data pipelines that support faster, better, data-informed business decisions.
• Contribute to integration, regression, and performance test suites; analyze reports, identify test issues or errors, and triage root causes.
• Document and communicate deployment, maintenance, support, and business functionality requirements.
• Follow team delivery and release processes, including code deployment and release cadence.
• Identify gaps in data management standards adherence and collaborate with partners to create plans to close gaps.
• Lead concept and experimentation testing and synthesize results to validate and improve analytical solutions.
• Define key performance indicators and internal controls.
• Mentor more junior Data Engineers and coach team members for delivery and release CI-CD events, including branching timelines, pull requests, issue triage, merge or conflict resolution, and release notes.

Required Skills and Experience

• Minimum 8+ years of hands-on data engineer and/or software development experience, including at least 5 years onsite experience.
• Experience with big data processing systems such as Hadoop and Cloudera.
• Programming experience with Python, including modular Python programming.
• Strong SQL and UNIX experience.
• Experience with Hadoop components such as Hive, Spark, and PySpark (listed as Pispark).
• Proven technology leadership, guiding junior engineers toward project completion and deliveries.
• Cyber security domain experience, preferably vulnerability data management.
• Ability to switch between technology leadership and hands-on coding when needed.
• Experience with cloud services and technologies such as AWS and Azure.
• Experience with data stores including Hive, HBase, and Elastic.
• Experience with Bitbucket for code versioning.
• Experience with CI/CD pipelines.
• Good knowledge of data engineering principles and the SDLC process.
• Ability to detect early and articulate requirement challenges and gaps with business partners.

Technologies

• Python
• Hadoop
• Cloudera
• Hadoop – Hive
• Spark
• PySpark (listed as Pispark)
• SQL
• UNIX
• HIVE
• HBase
• Elastic
• AWS
• Azure
• Bitbucket
• CI/CD (listed as CICD and CI-CD)

Employment Details

• Job Type: Full Time
• Job Category: IT
• Role: Data Engineer
• Location: Charlotte, NC (Onsite)
• Full-time: Permanent
• Compensation: USD 119,000 per year
• Minimum Experience: 8 years

Primary Role Classification

• Role: Data Engineer
• Skill: IT Security Engineer
• Required Skills (listed): DevOps Engineer Senior Email Security Engineer