Data Engineer

Design, build, test, deploy, and support governed enterprise data pipelines and analytics-ready data products on modern cloud platforms.

Responsibilities

• Design, build, test, deploy, and support modern data platform pipelines and lakehouse or warehouse data products.
• Implement ingestion, staging, transformation, conformance, and publishing patterns across raw/bronze, stage, silver, gold, and certified consumption layers.
• Build PySpark, Spark SQL, Python, and SQL transformations that are distributed, restartable, idempotent, and operationally supportable.
• Implement full-refresh, incremental, CDC, merge, and partition-aware load patterns where appropriate.
• Design source registration, audit stamping, row hashing, metadata capture, and lineage-friendly table handling.
• Build conformed dimensions, facts, bridge tables, reference tables, and curated marts for analytics and semantic model consumption.
• Partner with business SMEs and analysts to define business keys, surrogate keys, grain, relationships, slowly changing dimension needs, and metric-ready structures.
• Implement metadata-driven configuration patterns for sources, tables, columns, jobs, schemas, quality rules, and load audits.
• Support MDM-ready outputs, reference-data management, and stewardship workflows for core entities including customer, account, product, location, device, meter, vendor, or chart of accounts.
• Maintain clear documentation of table contracts, lineage, refresh behavior, data quality rules, and operational dependencies.
• Apply governance and security practices across data products, including access control, sensitivity awareness, ownership, cataloging, endorsement, certification, and audit evidence.
• Support Microsoft Purview, OneLake catalog, or equivalent governance capabilities for lineage, data cataloging, domains, glossary terms, quality status, and data product discoverability.
• Design solutions that support domain-oriented ownership, data mesh principles, reusable platform patterns, and federated governance.
• Ensure production reports and semantic models consume approved data products rather than unfinished engineering layers unless explicitly approved for validation.

Requirements

• Minimum: Four (4) year bachelor’s degree from an accredited institution in computer science, information systems, data engineering, software engineering, mathematics, statistics, engineering, or a related technical field.
• Minimum: Four (4) years of progressively responsible experience in data engineering, ETL/ELT development, analytics engineering, database development, BI engineering, software development, cloud data platform engineering, or related technical work.
• Preferred: Certifications or formal training in Microsoft Fabric, Azure Data Engineer, Databricks, Spark, Snowflake, data governance, DevOps, data modeling, or cloud architecture.
• Preferred: Experience designing and operating lakehouse or warehouse platforms, medallion data flows, Spark/PySpark workloads, Data Factory or pipeline orchestration, metadata-driven frameworks, dimensional models, master data, and governed semantic-ready marts.
• Preferred: Experience with Microsoft Fabric, Azure Data Lake Storage, OneLake, Delta Lake, Synapse, Databricks, Snowflake, Power BI semantic models, Git-based DevOps, and enterprise operational data.
• Ability to operate a variety of office equipment, including a personal computer, printers, copy machines, telephone.
• Ability to work irregular hours for assignment completion and flexibility to change scheduling and report to work on short notice during emergency situations.
• Irregular hours in evenings and on weekends are possible during peak business times and during storm season.
• Normal work hours: eight (8) hours between 7:00 am and 5:00 pm, Monday through Friday.
• Successful completion of pre-employment background check, physical and drug screen.
• Follows safety procedures diligently, identifies potential hazards, and takes appropriate action to maintain a safe working environment.
• Provides exceptional service to members, actively listening to their needs and ensuring their satisfaction.
• Acts with honesty and integrity in all tasks, maintaining transparency and ethical standards.
• Demonstrates a strong work ethic by consistently meeting deadlines and achieving high performance in all tasks.
• Contributes to an inclusive culture by respecting and valuing diverse perspectives and collaborating effectively with all team members.
• Takes responsibility for actions and decisions to meet commitments and deliver high-quality work.
• Works collaboratively with team members, sharing information and supporting collective goals.
• Applies engineering judgment to design reliable, scalable, secure, and maintainable data solutions aligned with organizational architecture and tradeoffs across performance, governance, and usability.
• Applies PySpark, Spark SQL, Python, and SQL to build scalable data pipelines for ingestion, transformation, cleansing, enrichment, and validation while optimizing performance and avoiding distributed processing anti-patterns.
• Leverages expertise in modern lakehouse and warehouse platforms, including Fabric, ADLS, OneLake, Databricks, Snowflake, and Synapse, to architect and optimize scalable, secure, and governed data environments using industry-standard storage and data management practices.
• Leverages expertise in Data Factory, Dataflows, notebooks, APIs, and orchestration tools to develop and maintain reliable data pipelines supporting scalable processing, operational resilience, monitoring, and production stability.
• Demonstrates expertise in dimensional modeling and analytical design using facts, dimensions, keys, SCDs, bridge tables, conformed dimensions, and aggregate structures for semantic models and business intelligence solutions.
• Establishes governance frameworks covering metadata, lineage, cataloging, auditability, and data quality; implements validation controls including reconciliation, integrity checks, and duplication detection with security compliance.
• Applies MDM principles (entity resolution, survivorship, stewardship) and domain-driven design with data-as-a-product thinking, reusable contracts, and governed consumer-ready data products.
• Applies Git, CI/CD, code reviews, automated deployments, environment management, and versioning to ensure controlled, repeatable, reliable data delivery.
• Exhibits understanding of Power BI, Direct Lake, semantic models, gold-layer design, and performance-optimized analytics consumption.
• Applies AI tools (Fabric Copilot, code generation assistants) responsibly, ensuring validation, privacy compliance, and safe production usage.

Technologies

• Microsoft Fabric
• Azure data services
• Azure Data Lake Storage
• OneLake
• Delta Lake
• Databricks
• Snowflake
• Synapse
• PySpark
• Spark SQL
• Python
• SQL
• ETL/ELT
• CI/CD
• Change data capture (CDC)
• Data Factory
• Dataflows
• Notebooks
• APIs
• Git-based DevOps
• Git
• Power BI
• Direct Lake
• Microsoft Purview
• Delta / open table formats
• Dimensional modeling
• Data mesh
• Master data management (MDM)
• DevOps
• DataOps
• Fabric Copilot

Verification

The above qualifications and competencies may be verified through a combination of education, experience, interview questions and technical skills exercise(s).

Work Environment and Physical Demands

• Regularly required to sit and talk or hear.
• Occasionally required to walk; stand; use hands to finger, handle, or feel; reach with hands and arms; climb or balance; stoop, kneel, crouch, or crawl.
• Must regularly lift and/or move up to 10 pounds.
• Specific vision abilities required: close vision, distance vision, color vision, ability to adjust focus.
• General sedentary office environment.
• Noise level usually moderate.

Workplace and Compliance Statements

• SECO Energy participates in E-Verify.
• SECO Energy does not accept unsolicited resumes from any source other than directly from applicants.
• This posting may be closed and no longer available to receive applications upon SECO Energy's discretion.