Data Engineer

DCI Solutions is seeking a Data Engineer to join a cross-functional team in New York, NY (onsite). In this role, you will integrate data, build application components that support operational decisions, and help ensure data builds run reliably through monitoring and debugging.

The position includes work across data feed transfers, distributed transformations with Spark, and support for platform teams as they investigate and understand the code behind data products. You will also contribute to platform data footprint inquiries through RFI requests.

Responsibilities

• Act as a Data Engineer on a cross-functional team to address pressing problems
• Integrate data, translate insights into usable formats, and build applications for operational decision-making
• Set up transfers of data feeds from source systems into the required data location
• Write transformations and create new datasets using Spark for distributed computation
• Monitor build progress and debug build issues as they arise
• Design application components that address operational questions
• Support RFI requests related to the platform data footprint
• Work with product teams to debug and understand the code supporting the platform

Requirements

• BS in Computer Science / Engineering (or related career field) preferred
• Strong background in data engineering
• Experience with Spark, PySpark, and/or Pandas
• Extensive coding/programming experience in Python
• Extensive coding/programming experience in SQL, R or similar
• Strong understanding of algorithms and data structures as applied to software development
• Foundry experience preferred
• Active Secret Security Clearance required

Experience

• Data engineering: 5 years (preferred)
• Python coding/programming: 5 years (preferred)
• Building and managing data pipelines: 5 years (preferred)
• Data transformation: 5 years (preferred)
• Foundry: 1 year (preferred)
• Spark and/or PySpark: 3 years (preferred)
• Pandas: 3 years (preferred)

Technologies

• Spark
• PySpark
• Pandas
• Python
• SQL
• R
• Foundry

Benefits

• 401(k)
• 401(k) matching
• Dental insurance
• Flexible schedule
• Flexible spending account
• Health insurance
• Health savings account
• Life insurance
• Paid time off
• Parental leave
• Professional development assistance
• Relocation assistance
• Retirement plan
• Tuition reimbursement
• Vision insurance

Job Type: Full-time
Work Location: In person
Security Clearance: Secret (Required)
Salary: USD 175,000 - 260,000 per year