Senior AI Engineer — Conversational Voice AI

SmartAdvocate is hiring a Senior AI Engineer to build and operate enterprise conversational voice AI agents for legal case intake and client calls.

Responsibilities

• Architect, build, and deploy conversational voice agents for callers, leads, and clients.
• Integrate conversational agents with VoIP/SIP and cloud communications platforms including Twilio, Plivo, Amazon Connect, and LiveKit.
• Support real customer telephony topologies: call forwarding, conferencing an SA number onto a live call, and full number porting.
• Handle live-call dynamics such as barge-in and turn-taking, interruptions, poor reception, accents, and background noise.
• Optimize real-time performance across STT, TTS, NLU, and latency budgets; tune voices for naturalness.
• Design warm transfer and escalation workflows to hand calls to humans with full context, including routing high-value or urgent situations immediately.
• Build real-time pipelines that transcribe, parse, and structure conversations to populate case fields, create intake records, update case notes, and trigger downstream workflows.
• Record and summarize every call into the case file with clear provenance for agent-provided content.
• Detect caller language and respond in it, with Spanish prioritized after English.
• Implement caller identification and authentication before any case information is disclosed (caller ID alone is not identity).
• Make agent behavior configurable per firm, including operating hours and after-hours coverage, outbound calling windows by callee type and time zone, escalation rules, and voice selection.
• Add guardrails, fallbacks, structured logging, and call/transcript retention designed for audit readiness.
• Own production operations end-to-end: latency budgets, monitoring, alerting, failover, and on-call. Treat a law firm’s main phone line as an availability-critical system.

Requirements

• Production experience building conversational voice agents integrated with telephony infrastructure such as SIP/RTP, WebRTC, Twilio Media Streams, LiveKit, Deepgram, ElevenLabs, Vapi, Retell, or similar.
• Real-time audio pipeline engineering experience with streaming STT and TTS, sub-second response handling, concurrency, and latency profiling under load.
• Pipeline engineering for extracting structured JSON or database payloads from live audio streams.
• Python service development strength (FastAPI/async or equivalent) and clean REST API design.
• Production-grade LLM integration, including function/tool calling, structured outputs, prompt/context engineering, and cost and latency tuning.
• Event-driven and asynchronous patterns including queues, background workers, retries, and idempotency.
• Systematic evaluation of conversational quality using test suites, transcript review, and regression testing when prompts or models change.
• Production ownership experience including containerized deployment, CI/CD, monitoring, structured logging, and personally debugging a live outage.

Technologies

• VoIP/SIP, Twilio, Plivo, Amazon Connect, LiveKit
• SIP/RTP, WebRTC, Twilio Media Streams
• Deepgram, ElevenLabs, Vapi, Retell
• STT, TTS, NLU
• FastAPI, Python
• LLM, JSON, REST API
• CI/CD, queues

Benefits

• Salary range: USD 120,000 - 160,000 per year
• Comprehensive benefits package
• 401(k) and 401(k) matching
• Dental insurance
• Flexible spending account
• Health insurance and health savings account
• Life insurance
• Paid time off
• Parental leave
• Vision insurance
• Reports to the CTO; works with the existing AI Architect and AI development team
• Location: Remote (US) or Hybrid in Melville, NY
• Opportunity to lead high-impact AI work on an established platform with real customers, real data, and unsolved problems in legal tech

Safety, Configuration and Operations

• Design caller identification and authentication before disclosing any case information; caller ID alone is not identity.
• Configure behavior per firm: operating hours and after-hours coverage, outbound calling windows by callee type and time zone, escalation rules, and voice selection.
• Implement guardrails, fallbacks, structured logging, and call/transcript retention suitable for an audit.
• Own production service reliability including latency budgets, monitoring, alerting, failover, and on-call support.

Nice to Have

• Telephony compliance: state call-recording and consent law, TCPA, permitted calling windows, spam-labeling, and verified-number strategy.
• Experience with C#, .NET / ASP.NET and Microsoft SQL Server (core platform), while AI services are Python.
• Experience in regulated industries such as legal, medical, insurance, or similar.
• Multi-language voice development with Spanish as the priority.
• Caller authentication patterns including callback-to-number-on-file and PIN or knowledge-based verification.
• Understanding of HIPAA, PHI handling, and attorney-client privilege constraints for AI applications.
• Working knowledge of voice AI unit economics such as per-minute cost, model selection, and cost-per-call optimization.

Live Data Extraction and Write-Back

• Build real-time pipelines to transcribe, parse, and structure conversations into case fields, intake records, case notes, and downstream workflows.
• Record and summarize every call into the case file with clear provenance for agent-entered information.
• Detect caller language and respond in it, with Spanish prioritized after English.