DataJobs.io
← Back to all jobs

Job Description

ScienceSoft USA Corporation is hiring a Senior Data Engineer for a full-time, on-site role in Dallas, TX (McKinney, TX).

Responsibilities

  • Own data pipelines end-to-end, covering ingestion, processing, validation, and publication
  • Build and maintain batch and streaming data pipelines
  • Implement reliable processing patterns for idempotency, backfills, late-arriving data, and partitioning
  • Develop automated data quality checks for completeness, timeliness, anomalies, and cross-source consistency
  • Monitor production pipelines and configure effective alerting
  • Investigate and resolve data quality and pipeline reliability issues
  • Optimize pipeline performance and resource usage
  • Onboard new datasets and data providers
  • Work with large volumes of financial and market-related data
  • Communicate with data consumers about incidents, data issues, changes, and availability
  • Collaborate on APIs and internal libraries for data access and distribution
  • Improve reliability, maintainability, and observability of the data platform

Requirements

  • 5+ years of professional experience in Data Engineering
  • Strong practical experience with Python
  • Strong knowledge of SQL
  • Strong hands-on experience with Apache Spark
  • Experience with Hadoop / Hive or modern lakehouse formats such as Delta Lake or Apache Iceberg
  • Experience building and maintaining batch and streaming pipelines
  • Strong understanding of idempotent data processing
  • Strong understanding of backfills
  • Strong understanding of late-arriving data
  • Strong understanding of partitioning strategies
  • Experience implementing automated data quality checks
  • Experience with monitoring and alerting for production data pipelines
  • Experience working with large volumes of data
  • Strong understanding of reliable data ingestion and publication processes
  • Strong analytical and troubleshooting skills
  • High attention to detail and data correctness
  • Strong ownership and ability to independently manage data engineering tasks
  • Strong communication skills with data consumers and technical stakeholders
  • Valid US work authorization
  • Ability to work on-site full-time in Dallas, TX

Technology

  • Python, SQL
  • Apache Spark
  • Hadoop, Hive
  • Delta Lake, Apache Iceberg
  • Kafka
  • Airflow
  • Snowflake, Databricks
  • Java, Scala

Nice to Have

  • Experience working with market data, including equities, macroeconomic data, or market data vendors
  • Experience with Kafka or other streaming technologies
  • Experience with Airflow or another workflow orchestration platform
  • Experience with Snowflake or Databricks
  • Experience developing APIs or reusable libraries for data distribution
  • Experience with Java or Scala
  • Previous experience in Investment Management / Financial Services

Similar Jobs