Data Engineer
Backend Developer
Aws Data Engineer
Big Data
Bigdata
Cassandra
Cloud
Cloud Data Engineering
Cloud Data Warehouse
Cloud Data Warehouse
Cloud Platform
Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Integration
Data Integration Tools
Data Pipeline
Data Platform
Data Processing
Data Warehouse
Data Warehousing
Database
Databases
Engineer
ETL
Informatica
NoSQL
Snowflake
Spark
Spring Batch
SQL
Job Description
CGI is seeking an experienced Data Engineer for an on-site role in Cary, NC, focused on building and operating ETL and external-facing API solutions.
Responsibilities
- Design, develop, test, deploy, and maintain ETL batch jobs using Spring Batch and Apache Spark on EMR
- Schedule, monitor, and debug Spring Batch and Spark batch processing workflows
- Build Java clients for REST and SOAP API consumption
- Develop Scala Spark batch applications for data movement and processing
- Use Apache Spark to process big data and write outputs to NoSQL databases such as Cassandra or Yugabyte
- Build and deploy software solutions on AWS cloud infrastructure
- Provide technical support to squads across the product area for building, deploying, and maintaining applications in AWS
- Work across multiple development workstreams while supporting scalable, resilient, and extensible architecture
- Ensure technology decisions align with Fidelity architecture standards and strategies
- Evaluate and recommend technologies and frameworks to support flexibility, long-term viability, and time to market
- Collaborate with peers via code reviews, pair programming, and daily discussions across multiple squads
Requirements
- Bachelor's degree in Computer Science, Information Systems, or a related field, with a proven track record in data engineering
- 6+ years of software development experience
- 6+ years developing Spark or Spring Batch services for data movement
- Experience scheduling, monitoring, and debugging ETL batch jobs using Spring Batch and Spark
- Hands-on experience with Java clients consuming REST and SOAP APIs, plus Scala Spark batch applications
- Experience developing, testing, deploying, and maintaining ETL batch jobs using Spring Batch and Apache Spark/EMR
- Experience using Apache Spark for big data processing and writing to Cassandra or Yugabyte
- Proven experience building and deploying solutions on public cloud providers such as AWS
- Strong understanding of the full software development lifecycle: analysis, design, coding, testing, problem resolution, and planning
- Strong collaboration and communication skills, including working across multiple squads and adapting to change
- Experience with cloud-based data warehousing and data lake solutions such as Snowflake
- Experience with data storage and management for large datasets, including Parquet and HDF5
Technologies
- Spring Batch
- Apache Spark
- EMR
- Java
- REST
- SOAP
- Scala
- NoSQL
- Cassandra
- Yugabyte
- AWS
- Snowflake
- Parquet
- HDF5
- Spark SQL
Skills
- Cloud Data Platform Design
- Data Engineering
- RESTful (Rest-APIs)
- Spark SQL
- Spring Batch
Compensation and Location
- Location: Cary, NC (onsite)
- Salary range: USD 91,800 - 160,700 per year
Benefits
- Competitive compensation
- Comprehensive insurance options
- Matching contributions through the 401(k) plan and the share purchase plan
- Paid time off for vacation, holidays, and sick time
- Paid parental leave
- Learning opportunities and tuition assistance
- Wellness and well-being programs
Additional Information
- CGI is required in some jurisdictions to include a reasonable estimate of the compensation range for this role.
- CGI benefits are offered to eligible professionals on their first day of employment.
- All U.S. employment offers are contingent upon successful completion of a background investigation.
- Qualified applicants will receive consideration without regard to protected statuses as required by applicable laws.
- CGI provides reasonable accommodations to qualified individuals with disabilities; to request an accommodation to apply in the U.S., email [email protected].
Similar Jobs
A