Distributed Sql Data Jobs

1 job found

Today

Staff Data Engineer - Data Platform

Trend Micro Austin, TX

Austin, TX

Today

Full time Big Data Bigdata Bigquery

Apply Now

Job Description

Design and build the Vision One Data Platform that powers ingestion, CDC, governance, semantic discovery, and governed self-service transformations.

Responsibilities

Design and build the Vision One Data Platform, including ingestion, replication, transformation, governance, discovery, and serving capabilities.
Create scalable data pipelines and change-data-capture (CDC) architectures using technologies such as Datastream and similar streaming or replication tools.
Enable self-service dbt across Vision One OLTP data sources, giving product teams governed capabilities to transform, model, test, document, and expose data without requiring the central data platform team to build every pipeline.
Define patterns for safely extracting and replicating data from OLTP and distributed SQL systems into analytical and downstream data environments.
Design platform architectures using distributed SQL technologies such as YugabyteDB, CockroachDB, or similar platforms.
Work with technologies such as Databricks and modern analytical processing engines to support large-scale processing and analytics.
Build a comprehensive data catalog and metadata foundation so teams and AI agents can understand what data exists, where it comes from, what it means, and how to use it.
Develop semantic data discovery capabilities that help engineers, analysts, applications, and AI agents find relevant data using business meaning rather than only database or schema names.
Establish data governance standards for ownership, lineage, schemas, quality, access control, retention, and lifecycle management.
Develop reusable data platform primitives and APIs product teams can consume as self-service capabilities.
Implement automated mechanisms for data quality, schema evolution, lineage, observability, and freshness monitoring.
Design for evolving product schemas to accommodate continuous change with minimal duplication and operational overhead.
Partner with security and platform engineering teams to implement fine-grained data access controls across databases, schemas, tables, and data products.
Help establish a source-of-truth architecture for Vision One while minimizing unnecessary duplication.
Provide technical leadership and architectural guidance to data engineering and product engineering teams.
Drive engineering standards across reliability, scalability, performance, cost efficiency, and operational excellence.

Requirements

8+ years of experience in data engineering, distributed systems, backend engineering, or a related field.
Strong experience designing and building large-scale data platforms and data pipelines.
Deep understanding of OLTP and OLAP architectures and the tradeoffs between them.
Hands-on experience with CDC, data replication, and streaming architectures.
Strong SQL expertise and experience working with relational databases at scale.
Experience with distributed SQL databases such as YugabyteDB, CockroachDB, Spanner, or similar.
Experience building data transformation workflows using dbt or similar modeling frameworks.
Experience with modern analytical/data platforms including Databricks, Spark, Trino, Snowflake, BigQuery, or equivalent.
Strong understanding of data governance concepts including metadata, lineage, data quality, ownership, access control, and data catalogs.
Experience designing systems that support self-service data consumption and transformation.
Strong software engineering fundamentals: API design, testing, CI/CD, observability, and production operations.
Ability to work across organizational boundaries and influence architecture and engineering standards without relying only on formal authority.

Technologies

dbt
Datastream
YugabyteDB
CockroachDB
Databricks
Spark
Trino
Snowflake
BigQuery
Spanner
CDC
Kafka
Debezium
OLTP
OLAP

Preferred Qualifications

Experience building a company-wide or platform-level Data Platform rather than individual product pipelines.
Experience with Datastream, Debezium, Kafka, or similar CDC or streaming technologies.
Experience with YugabyteDB or CockroachDB in production environments.
Experience operating Databricks or similar distributed data-processing platforms.
Experience designing data catalogs, semantic layers, knowledge graphs, or ontology-driven data discovery.
Experience building platforms that expose data to AI/agentic applications.
Experience with data mesh, data products, or domain-oriented data architectures.
Experience implementing fine-grained authorization for data platforms.
Experience managing large-scale telemetry, security, observability, or other high-volume data.
Experience balancing data quality, query performance, platform reliability, and infrastructure cost.

What Success Looks Like

Vision One data can be discovered through a common semantic layer rather than requiring teams to understand every underlying database and schema.
Product teams can use dbt as a governed self-service capability across their OLTP data.
Data lineage, ownership, quality, and access policies are built into the platform instead of managed manually.
Data moves reliably from distributed OLTP systems into analytical and downstream systems through standardized CDC and streaming patterns.
Product teams can consume platform capabilities without repeatedly building ingestion, transformation, governance, and discovery infrastructure.
The platform provides a trusted source of truth while minimizing unnecessary data copies.
The data foundation supports traditional analytics and applications as well as AI and agentic experiences across Vision One.

Engineering Leadership

Operate beyond a single team by shaping architecture, establishing technical standards, mentoring engineers, and driving cross-team adoption.
Be comfortable going deep into implementation while maintaining long-term architecture thinking for a large-scale multi-product security data platform.

Benefits

Comprehensive medical, dental and vision insurance
Life insurance
Short & Long Term Disability
Pre-partum, maternity, parental and medical leave
Mental Health Wellness Program
Adoption Assistance
Wellness Incentive
Pet Insurance
401(k) with company match
Paid Time Off
14 Annual Holidays
Tuition Assistance
Employee Resource Groups

Additional Information

Location: Austin, TX (onsite)
Experience: 8+ years

TrendAI provides benefit options for you and your family, competitive compensation with bonus opportunity tied to company performance, and ongoing learning and broad technological opportunities, with work-life balance supported by team activities and a culture focused on equity, inclusion, and collaboration.

View full job page →

Get Distributed Sql Data Jobs alerts

New Distributed Sql Data Jobs, daily.