Hybrid role in Knoxville, TN focused on building and improving modern big data solutions across ingestion, integration, and transformation. You will support cloud-based data platform services and help deliver reliable, efficient pipelines built for large-scale processing and storage.
What you’ll do
- Design and implement data engineering components aligned to client requirements and technical solution goals.
- Build and maintain batch and real-time ingestion, integration, and transformation workflows.
- Work with large-scale data processing and storage technologies to keep data pipelines running reliably and efficiently.
- Contribute to analytics, search, and aggregation capabilities across data platforms.
- Participate in technical design discussions to help ensure overall solution health and performance.
- Support cloud-based data platform services and optimize data workflows for performance and scalability.
- Collaborate with project teams in a hybrid work environment, including attending onsite interviews or equipment pickup when required.
What you bring
- At least 4 years of hands-on experience with big data technologies.
- Experience with Hadoop ecosystem tools such as HDFS, Kafka, Spark, Spark Streaming, Hive, NiFi, Airflow, Flink, Storm, Oozie, or similar platforms.
- Strong programming experience in Java, Python, or Scala, with Java preferred.
- Working knowledge of NoSQL and MPP data platforms including HBase, MongoDB, Cassandra, AWS Redshift, Azure SQLDW, or GCP BigQuery.
- Experience with AWS data platform services.
- Ability to translate business needs into technical design and implementation.
- Strong problem-solving skills and the ability to work independently on design discussions and solution delivery.
Tools you may use
HDFS, Kafka, Spark, Spark Streaming, Hive, NiFi, Airflow, Flink, Storm, Oozie, Java, Python, Scala, HBase, MongoDB, Cassandra, AWS Redshift, Azure SQLDW, GCP BigQuery, AWS, ActiveMQ, RabbitMQ, Solace, Informatica, Talend, Oracle, MySQL, SQL Server, Postgres, Collibra, Alation.
Preferred background
- Experience with traditional ETL tools such as Informatica or Talend.
- Familiarity with database technologies including Oracle, MySQL, SQL Server, or Postgres.
- Knowledge of data governance practices, including security, lineage, and catalog tools like Collibra or Alation.
- Exposure to distributed messaging tools such as ActiveMQ, RabbitMQ, or Solace.
- Experience with search/indexing and microservices architectures.
- Background in performance tuning and data pipeline optimization.
- Cloud data certifications or other relevant big data credentials.
Additional notes
You may work from an East Coast office or on-site in Knoxville, TN when needed. Horizontal Talent may contact you using automated technology, including phone calls, SMS/text messages, or email delivered by a virtual AI recruiter, Alex. Horizontal is committed to building an inclusive workplace where diverse perspectives are valued, respected, and encouraged.
