Databricks Data Engineer - Manager - Tech Cons - Open Location
Manager
Artificial Intelligence
Big Data
Bigdata
Cloud Data Engineering
Cloud Data Platform
Cloud Platform
Cloud Platforms
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Engineering Lead
Data Integration
Data Integration Tools
Data Lakehouse
Data Pipeline
Data Platform
Data Processing
Data Warehouse
Database
Databases
Databricks
ETL
Informatica
Information Technology (IT)
Lead Data Engineering
Software Development
Software Engineering
Spark
SQL
Job Description
EY is hiring a Manager of Data Engineering to lead the design and delivery of cloud analytics solutions with a strong focus on Databricks. This Chicago-based onsite role combines technical leadership, team management, and daily client collaboration to turn business requirements into scalable, governed data platforms.
Responsibilities
- Analyze business requirements and translate them into technical requirements and delivery plans.
- Design, build, and operate scalable data architecture and modeling solutions.
- Lead workstream delivery while ensuring quality across engineering processes.
- Oversee end-to-end data pipeline development, including ingestion, transformation, and storage using Databricks and other cloud services.
- Drive architecture for complex cloud analytics solutions, aligning with business objectives and best practices.
- Ensure quality, integrity, and security of data throughout the lifecycle, applying data governance best practices.
- Engage with clients on a daily basis through working sessions, support delivery, and identify opportunities for additional services.
- Implement resource plans and budgets while managing engagement economics.
- Manage, mentor, and develop a team of data engineers, building a culture of innovation, collaboration, and continuous improvement.
- Communicate with stakeholders including technical and non-technical audiences to convey complex concepts and project progress.
- Stay current on emerging trends in data engineering, cloud computing, and analytics.
- Lead documentation and reporting to communicate project progress and outcomes to clients.
Requirements
- Bachelor’s degree in computer science, Engineering, or a related field (Master’s preferred).
- Typically 4 - 6 years of relevant experience in data engineering, with focus on cloud data solutions and analytics.
- Proven expertise in Databricks with hands-on experience using Spark for big data processing.
- Hands-on experience with AI-native data engineering practices, including generative AI and agentic workflows to accelerate pipeline development, testing, documentation, and optimization.
- Strong data architecture background, including designing complex cloud analytics solutions.
- Strong programming skills in Python, Scala, or SQL.
- Familiarity with AI-powered IDEs and coding assistants such as Cursor, GitHub Copilot, and Databricks Assistant.
- Experience leading and managing teams, with an emphasis on mentoring and developing talent.
- Strong problem-solving skills and the ability to work independently and in a team environment.
- Excellent communication and interpersonal skills, including client management.
- Strategic leadership to align initiatives with organizational goals and drive strategic vision.
- Project management experience managing multiple projects and teams with delivery on time and within scope.
- Stakeholder engagement skills, including presenting solutions to executives.
- Change management capabilities supporting clients through data transformation and technology adoption.
- Risk management skills to identify risks and develop mitigation strategies.
- Technical leadership experience leading discussions and making architectural decisions.
Technology Stack
- Databricks, AWS, Azure, GCP, Spark
- Generative AI, agentic workflows
- Python, Scala, SQL
- Cursor, GitHub Copilot, Databricks Assistant
- ETL, AI-powered IDEs
Benefits
- Medical and dental coverage
- Pension and 401(k) plans
- A wide range of paid time off options
Location and Salary
Location: Chicago, IL (onsite). Other locations: Anywhere in Country.
Salary: USD 125,500 - 261,600 per yearly. Competitive compensation varies by geography and other factors, including education, experience, knowledge, skills, and geography. Specific ranges include:
- New York City, Boston, and Washington DC Metro Areas, Washington State, and Southern California offices: $150,700 to $251,200
- Bay Area California offices: $157,100 to $261,600
- All other US offices including Sacramento: $125,500 to $230,200
Large-Scale Implementation Programs
- Enterprise Data Lake Implementation: Led design and deployment of a cloud-based data lake for a Fortune 500 retail client, integrating data from multiple sources (e.g., ERPs, POS systems, e-commerce platforms) to enable advanced analytics and reporting.
- Real-Time Analytics Platform: Managed development of a real-time analytics platform using Databricks for a financial services organization, enabling real-time fraud detection and risk assessment through streaming data ingestion and processing.
- Data Warehouse Modernization: Oversaw modernization of a legacy data warehouse to a cloud-native architecture for a healthcare provider, implementing ETL processes with Databricks and improving data accessibility for analytics and reporting.
Additional Details
- Travel may be required regularly based on client needs.