Senior Data Engineer - PGIM Technology

Prudential's PGIM Technology is seeking a Senior Data Engineer to design, build, and maintain scalable data and AI solutions on cloud platforms. The role emphasizes hands-on engineering, data ingestion and transformation, and AI use cases such as semantic search and Retrieval-Augmented Generation (RAG). Collaboration across analysts, scientists, and stakeholders is essential, with a hybrid work arrangement in Newark, New Jersey.

Location

Newark, New Jersey, hybrid work arrangement.

Compensation

USD 115,000 - 155,000 per year.

Responsibilities

• Design, build, and maintain scalable data pipelines (ETL/ELT) to support analytics and AI workloads.
• Develop solutions using Microsoft Fabric, Azure Data Factory, or equivalent tools.
• Implement data quality checks, transformations, and data modeling.
• Handle large-scale datasets with PySpark and SQL.
• Deploy and integrate machine learning models within data pipelines.
• Develop AI-enabled capabilities such as semantic search, embeddings, and Retrieval-Augmented Generation.
• Build and expose data and AI services via APIs.
• Contribute to CI/CD processes using GitHub Actions or Azure DevOps.
• Apply best practices for data security, governance, and access control.
• Monitor, troubleshoot, and resolve issues in pipelines and AI systems.
• Collaborate with analysts, data scientists, and business stakeholders.
• Produce documentation and reusable components for future projects.

Requirements

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field.
• 3–6 years of experience in data or AI engineering.
• Strong Python and SQL skills.
• Experience with data pipeline tools such as Azure Data Factory, Microsoft Fabric, AWS Glue, or similar.
• Proficiency with Spark and PySpark.
• Familiarity with data lakes and data warehouses.
• Experience deploying machine learning models.
• Basic understanding of MLOps.
• Exposure to Generative AI concepts including embeddings, RAG, and LLM APIs.
• Experience with REST APIs and microservices architecture.
• Familiarity with CI/CD practices and Git.
• Understanding of data governance and security principles.
• Strong problem-solving abilities and communication skills.

Technologies

• Python
• SQL
• PySpark
• Spark
• Microsoft Fabric
• Azure Data Factory
• AWS Glue
• GitHub Actions
• Azure DevOps
• REST APIs

Benefits

• Competitive base salary with yearly bonus potential.
• Medical, dental, vision, life and disability insurance, generous PTO, and various leave options including parental and military leave.
• 401(k) plan with company match up to 4 percent.
• Company-funded pension plan.
• Wellness program reimbursements up to $1,600 annually for personal wellbeing needs.
• Work/Life Resources covering parenting, housing, senior care, finances, pets, legal matters, education, mental health, and career development.
• Education benefits for traditional degree enrollment and accredited certificate programs.
• Employee Stock Purchase Plan with purchase at 85 percent of the lower price, after one year of service.