Data Engineer

The Data Engineer role at TWFG Insurance in Spring, TX (The Woodlands) focuses on designing and maintaining scalable data pipelines and lakehouse-based architectures that power enterprise analytics for insurance operations and reporting.

Key Responsibilities

• Design, build, and maintain scalable ELT/ETL pipelines using Microsoft Fabric and PySpark notebooks.
• Develop and optimize lakehouse layers within a Medallion architecture.
• Create curated dimensional models, fact tables, and semantic-ready datasets for reporting and analytics.
• Integrate data from internal and external platforms, including policy administration systems, carrier systems, vendor platforms, and APIs.
• Implement data quality controls, validation frameworks, and monitoring processes.
• Develop and maintain Fabric Lakehouses, Mirrored Databases, Dataflows, Notebooks, and Pipelines.
• Optimize Direct Lake architectures and semantic model performance.
• Support Power BI Embedded analytics and enterprise reporting solutions.
• Manage large-scale datasets and help ensure optimal query performance across analytical workloads.
• Participate in data governance initiatives, including metadata management, lineage tracking, and cataloging.
• Contribute to CI/CD, testing, release management, and operational monitoring for data assets.
• Maintain documentation, including data dictionaries and architecture diagrams.
• Promote engineering best practices and reusable development patterns.
• Work toward certification on the organization’s cloud provider data platform if the certification is not already obtained.

Required Qualifications

• Bachelor’s degree in Computer Science, Information Systems, Engineering, Mathematics, or a related discipline.
• Equivalent professional experience will be considered.
• 3+ years of experience in Data Engineering, Analytics Engineering, or Data Warehousing.
• Experience designing and supporting enterprise-scale data pipelines.
• Strong skills in SQL development.
• Experience with Python and/or PySpark.
• Experience working with cloud-based data platforms: Azure, AWS, or GCP.

Preferred Qualifications

• Hands-on experience with Microsoft Fabric.
• Experience with Power BI semantic models and analytics workloads.
• Experience with Azure Data Engineering technologies.
• Knowledge of Delta Lake, OneLake, and Direct Lake architectures.
• Experience implementing Row-Level Security and enterprise data governance practices.
• Insurance, financial services, or highly regulated industry experience.
• Familiarity with data cataloging and lineage tools such as Microsoft Purview.

Technologies

• Microsoft Fabric
• PySpark
• ELT/ETL
• Lakehouse
• Medallion architecture
• Power BI Embedded
• Direct Lake
• Azure, AWS, GCP
• SQL
• Python
• CI/CD
• REST APIs
• Delta Lake
• OneLake
• Row-Level Security
• Microsoft Purview

Compensation and Location

Location: Spring, TX (onsite).
Salary: USD 85,000 - 110,000 per year.

Benefits

• Medical/Dental/Vision Insurance
• Life Insurance
• 401K with a matching component
• LTD (employer paid)
• Education reimbursement based on tenure
• Paid Time Off
• 9 Paid Holidays a year
• Work Location: In person