This contract Data Engineer role focuses on building high performance, event driven data pipelines within the Ab Initio ecosystem for a large credit card processing system. The position is onsite in Berkeley Heights, NJ, with an hourly rate of USD 50 to 53, offered as a contract engagement.
Responsibilities
- Design complex Ab Initio continuous flow graphs, including stateful joins, sliding time windows, and aggregations.
- Build event driven data pipelines using Kafka, MQ, and file streams.
- Ensure resilience of continuous flows through checkpointing and recovery to guarantee exactly once processing.
- Apply advanced Ab Initio components such as Reformat, Rollup, Join, and Partition to maintain low latency.
- Monitor live production streams proactively to sustain 24/7 reliability and troubleshoot data issues.
- Develop ETL pipelines for both batch and real time data ingestion and transformation.
- Implement data validation, security, integrity, and compliance across big data platforms.
- Monitor and troubleshoot performance in large scale clusters.
- Collaborate with data scientists, analysts, and application teams to deliver robust data solutions.
- Automate workflows and improve operational efficiency through scripting and orchestration tools.
Requirements
- Strong understanding of credit card processing systems.
- Extensive knowledge of GDE (Graphical Development Environment), EME (Enterprise Meta Environment), Conduct>It, and Continuous Flows.
- Understanding of Kafka, messaging queues, and real time stateful services.
- Proficiency in Unix/Linux shell scripting, SQL, and database technologies such as Oracle and Teradata.
- Experience in Java, Scala, Python or Kafka is a plus.
- Familiarity with Linux/Unix environments and shell scripting.
- Understanding of data security, governance, and compliance standards.
- Experience with cloud based big data platforms.
- Exposure to containerization with Docker and Kubernetes for big data workloads.
- Knowledge of CI/CD pipelines for data engineering projects.
- Bachelor's degree in computer science, Software Engineering, or a related field.
Technologies
- Ab Initio
- GDE (Graphical Development Environment)
- EME (Enterprise Meta Environment)
- Conduct>It
- Kafka
- MQ
- Oracle
- Teradata
- Java
- Scala
- Python
- SQL
- Linux
- Unix
- Shell scripting
- Docker
- Kubernetes
- CI/CD pipelines
- Cloud based big data platforms
About Next Gen Software Solutions LLC
Next Gen Software Solutions is a technology staffing and consulting firm focused on delivering IT solutions and connecting businesses with skilled professionals. The company provides tailored IT consulting services, software solutions, and access to IT talent, with a foundation built on integrity, discipline, and commitment as guiding values. The organization emphasizes practical, mission aligned delivery.
Equal Employment Opportunity Statement
Next Gen Software Solutions LLC is an Equal Opportunity Employer. We are committed to fostering an inclusive and diverse workplace where all employees and applicants are treated with respect and dignity. We do not discriminate on the basis of race, color, religion, sex (including pregnancy, sexual orientation, or gender identity), national origin, age, genetic information, veteran status, or any other legally protected characteristic under applicable federal, state, or local laws.
