Llm Testing Data Jobs

1 job found

Today

AI Engineer

Hollstadt Consulting Minneapolis, MN 🇺🇸 Remote

Minneapolis, MN

Today

$105k - $140k/yr Full time AI Ai Engineer Ai Engineering

Apply Now

Job Description

Hollstadt Consulting is building AI capabilities through an AI Center of Excellence, bringing together cross-functional teams to deliver machine learning, AI, Generative AI, and Agentic AI solutions. This remote AI Engineer role supports domain-focused needs while helping translate complex requirements into production-ready systems across the organization.

In this position, you will work on end-to-end AI/ML pipelines and GenAI and agent capabilities, including building reliable workflows for extracting unstructured content and establishing testing and monitoring practices to keep models dependable over time.

Role focus

Support cross-functional teams with machine learning, AI, Generative AI, and Agentic AI solutions
Design and build production AI/ML pipelines and GenAI/agent capabilities for domain-specific requirements

Responsibilities

Work across GenAI platforms and libraries including AWS, SalesForce, Oracle, Snowflake, MS Copilot, and other third-party GenAI offerings
Automate workflows to extract complex, multimodal unstructured content from varied sources into accurate and reliable structured content using tools such as AWS Textract and Bedrock
Design and build MCP hosts, clients, and servers
Establish and use automated LLM testing frameworks
Create regression test suites to detect drift and prompt breakage
Integrate with internal and external web services using secure authentication and authorization mechanisms
Adopt safe practices aligned with enterprise security guidelines to protect against prompt injections and jailbreaks
Design, develop, and deploy production-grade traditional ML models such as regression, classification, clustering, and recommender systems
Design, maintain, and optimize end-to-end AI/ML pipelines including data ingestion, training, evaluation, deployment, and monitoring on cloud infrastructure (for example, AWS or equivalent)
Ensure AI/ML solutions are scalable, reliable, secure, and cost-effective in cloud environments
Create reusable components, frameworks, and best practices to accelerate AI development
Design and develop GenAI solutions using prompt engineering, Context Engineering, Retrieval-Augmented Generation (RAG), and custom pipelines
Design and develop interoperable AI agents using Model Context Protocol (MCP) and/or Google A2A
Partner with data scientists, architects, product managers, business stakeholders, and technical teams to align solutions with organizational goals
Provide hands-on technical support and mentorship to technical teams across the enterprise

Requirements

3+ years of experience designing and deploying ML/AI solutions in real-world environments
Very strong Python skills
Strong hands-on experience with LLM APIs (OpenAI, Azure OpenAI, Gemini, Anthropic, etc.) using Python and Python-based frameworks
Strong hands-on experience with prompt engineering, context construction, and grounding strategies
Strong hands-on experience with Retrieval Augmented Generation (RAG), including extracting, chunking, and creating embeddings from unstructured documents from sources such as O365 (email, Word, Excel), PDFs, and webpages
Comfortable building Model Context Protocol (MCP) clients, servers, and hosts
Strong expertise building REST APIs and integrating with internal and external APIs
Hands-on experience with Intelligent Document Processing and/or OCR technologies on complex documents
Knowledge of Google A2A
Deep experience in AWS including Lambda, Bedrock, Step Functions, API Gateway, and IAM
Strong experience with observability tools such as Dynatrace or similar GenAI observability tools
Excellent GenAI foundations and concepts
Clear understanding of enterprise data privacy, AI governance, and observability
Proficiency in Python and common ML/AI libraries (TensorFlow, PyTorch, scikit-learn)
Strong understanding of data engineering, SQL, and feature engineering
Hands-on experience with cloud services such as AWS Sagemaker, Lambda, ECS, S3, and IAM
Familiarity with containerization (Docker) and orchestration (Airflow, Kubeflow)
Working knowledge of version control and collaboration tools including Git, Jira, and Confluence
Bachelor’s degree in computer science, Engineering, or a related field
Knowledge of machine learning algorithms, deep learning frameworks, cloud AI technologies, GenAI technologies, and emerging agentic AI technologies
Knowledge of cloud platforms (AWS, Azure, GCP) for scalable AI/ML development
Knowledge of Responsible AI principles, including bias mitigation and ethical deployment
Knowledge of ML Ops best practices including CI/CD for ML, model monitoring, and versioning
Ability to build robust, scalable, and efficient AI/ML solutions in cloud-native environments
Ability to translate ambiguous business problems into clear technical ML/AI tasks
Ability to communicate complex ideas clearly to technical and non-technical stakeholders
Ability to learn and adapt quickly to emerging AI technologies, techniques, and tools

Technologies

AWS, SalesForce, Oracle, Snowflake, MS Copilot
AWS Textract, Bedrock, Model Context Protocol (MCP), Google A2A
OpenAI, Azure OpenAI, Gemini, Anthropic
REST APIs, Dynatrace
TensorFlow, PyTorch, scikit-learn
O365, PDFs, webpages
Intelligent Document Processing, OCR
AWS Lambda, AWS Step Functions, AWS API Gateway, AWS IAM, AWS Sagemaker, AWS ECS, AWS S3
Docker, Airflow, Kubeflow
Git, Jira, Confluence
CI/CD, ML Ops
Retrieval-Augmented Generation (RAG), prompt engineering, Context Engineering, grounding strategies, embeddings

Location, salary, and start details

Location: Minnesota (remote), local preferred but not required
Salary: USD 105,000 - 140,000 per year
Additional compensation: 8% AIP eligibility
Start date: 9/28/2026

Benefits

Comprehensive benefit plan including medical, dental, vision, life insurance, short-term disability, long-term disability, paid sick leave, and retirement benefits
401(k) + matching (match on the first 4% of your contributions)
Bonus opportunities including a Longevity Award bonus and referral bonus
Professional development via on-demand training through a consultant portal, including upskilling in Artificial Intelligence (AI)
Ongoing support and networking through a Consultant Coach program

Preferred qualifications

Master’s degree in a related technical field
Hands-on experience with agentic AI frameworks
Prior contributions to open-source AI/ML projects or published research
AI/ML certifications from cloud providers
Experience in highly regulated industries such as healthcare or finance

View full job page →

Get Llm Testing Data Jobs alerts

New Llm Testing Data Jobs, daily.