Skip to main content

How we cut call time by 63% Read more

Back to Engineering
Engineering

How we infinitely scaled our onboarding capacity and cut call time by 63%

How we built Harley, the voice agent behind our customer onboarding calls: from a teleprompter to a LiveKit phone line in under a month, and what breaks when a phone call is a full insurance application that can run past half an hour.

Aarfan Hussain, Product Engineer @ Harper 8 min read
Voice AI LiveKit Architecture

Harper is an AI-native commercial insurance brokerage for small and medium-sized businesses in every industry and state, including the ones other brokers turn away. AI agents handle the repetitive work, from the first call to quoting; our licensed team advises every customer.

This is how we built Harley, the voice agent that runs our customer onboarding calls. In two months it took the ceiling off that capacity and cut average call time by 63%, and customers who waited a day or two for a quote now wait two or three seconds on hold. While the owner talks, three things happen at once: Harley holds the conversation, our quoting engine fills carrier applications in parallel, and the transcript that fills our own application builds line by line.

Before July, it was all manual. A person on our team took every call, and every new hire had to learn more than 200 applications and supplemental forms, thousands of questions, and how to ask them of a roofer, a daycare or a car dealer. They worked through the form on the phone, then typed every answer in by hand afterward.

That capped how many businesses we could help. Calls come in bursts, and during a burst owners wait on hold or hang up. Even after a good call, quoting waited on the typing, so an owner heard back in a day or two.

When calls come in bursts

Before Harley

3 specialists, so new callers wait on hold.

Specialists
Specialiston a call8:12Specialiston a call11:40Specialiston a call1:24
On hold
Food truck3:09Plumbing contractor2:36Daycare2:03Roofing contractor1:30Car dealer0:57

On hold 5 · Hung up 2

With Harley

Every caller gets their own Harley voice agent.

One Harley voice agent per caller
freeHarleyBakery4:33freeHarleyAuto repair shop4:00freeHarleyPlumbing contractor2:21freeHarleyDaycare1:48freeHarleyLand­scaping company3:27freeHarleyFood truck2:54freeHarleyRoofing contractor1:15freeHarleyCar dealer0:42freefreefreefreefreefreefreefree
On hold
No one on hold

On hold 0 · Longest wait 2 s

Illustrative, sped up

An illustrative burst of calls: before, owners waited on hold and some hung up; with Harley, each one gets its own call right away.

Hiring scales slowly: every specialist has to be recruited, trained on every application and staffed against call volumes nobody can predict, and cost grows with every seat. We wanted our team's knowledge in software instead, so routine onboarding could scale on its own and our people could do the work that needs a licensed person: advising owners, working with underwriters and handling complicated businesses.

A teleprompter first

We started in July with a teleprompter for our team: a screen that transcribes the call live, shows what to ask next and fills in the application as the customer talks. If a roofer mentions solar work, the solar questions appear.

The caller mentions solar, and the questions and the application update live.

It showed us that a good onboarding call comes down to knowing what to ask, in what order, for which business. Once software held that, a model could run the call: the teleprompter's talk tracks became Harley's data, and its first phone line went live on August 5, less than a month later.

Talk paths

Insurance applications are written for underwriters, who decide whether to cover a business and at what price. Read aloud, they repeat themselves and jump between topics. So we rewrote each as a talk path: the same application, reordered into a conversation that adapts to the business's answers.

Our licensed team shapes the wording without changing what a question asks, and when real calls show a question confusing people, the fix goes into the path.

Talk paths are versioned data, not prompts in code. Every call loads the published version when its session opens, and our evals are built from the same paths, so a fix reaches every call at once.

Application forms are rewritten as conversations into a versioned talk path that Harley, a speech-to-speech voice agent, runs on every call, and real calls flow back to improve the path. From application to talk path Application forms 200+ applicationsand supplements rewritten as conversations versioned, published once Talk path questions in orderfollow-ups gated on answers every call Harley voice agentspeech-to-speech real calls improve the path Application forms are rewritten as conversations into a versioned talk path that Harley, a speech-to-speech voice agent, runs on every call, and real calls flow back to improve the path. From application to talk path Application forms 200+ applicationsand supplements rewritten as conversations versioned, published once Talk path questions in orderfollow-ups gated on answers every call Harley voice agentspeech-to-speech real calls improve the path
Every application becomes a talk path that Harley runs on every call, and real calls keep improving it.

Building Harley on LiveKit

Off-the-shelf voice-agent platforms got us a working demo in days, but not control over what mattered most: knowing when to speak on a bad line, looking things up without pausing, handing off cleanly to our licensed team and keeping call data in our own systems. One hung up after ten minutes by default, barely half an application.

So we built Harley ourselves on LiveKit, an open-source platform for real-time audio.

On a call, Harley confirms who's calling and what the business does, picks up the coverage the owner asked for, and walks the right application, skipping anything already answered. That's only the part the caller hears. Every turn also streams over a live socket to Blaze, our quoting engine, which opens carrier applications and fans out parallel quoting runs mid-conversation, so quotes are coming back by the time the call ends. Every industry runs the same Harley; only the talk path changes.

A caller reaches a LiveKit room over SIP, where the speech-to-speech Harley agent runs the call over WebRTC; the agent calls Harley MCP tools over HTTPS, holds a live WebSocket quoting connection to Blaze and sends events to Chassis through an on-disk spool, and all three services write to the DB. Harley system architecture Our services Caller phone call LiveKit room one per call Harley agent speech-to-speech Harley MCP onboarding API Blaze quoting engine Chassis event backend DB SIP audio WebRTC tools HTTPS live quotes WebSocket events on-disk spool A caller reaches a LiveKit room over SIP, where the speech-to-speech Harley agent runs the call over WebRTC; the agent calls Harley MCP tools over HTTPS, holds a live WebSocket quoting connection to Blaze and sends events to Chassis through an on-disk spool, and all three services write to the DB. Harley system architecture Caller phone call SIP LiveKit room one per call audioWebRTC Harley agent speech-to-speech tools, live quotes, events HTTPS, WebSocket, spool Our services Harley MCP onboarding API Blaze quoting engine Chassis event backend DB
The whole system: one room per call, three services around the agent, one DB.

Under the hood:

  • One room per call. A call arriving over SIP gets its own LiveKit Cloud room and a Harley worker that runs it in its own process, so one bad call can't take down its neighbors. Capacity scales with workers, not hires.
  • A fleet that never scales in mid-call. The fleet is fixed and always on, because scaling in would end a live call. A deploy brings up a new fleet and lets old workers finish their calls before exiting.
  • One speech-to-speech session. Our first version chained speech-to-text, a language model and text-to-speech. In its first week, the median turn took 2.2 seconds and the 95th percentile 3.8, with transcription alone eating about 0.9 seconds. Now one speech-to-speech session carries audio both ways, with no transcription step or model hand-offs in the turn. Since the model hears audio, not text, a sidecar transcribes the caller's track for a verbatim record.
  • Tools over HTTPS. Tool calls go to Harley MCP, our onboarding API: looking up the business, opening the session, loading the talk path. The transcript syncs to the session as the call goes, and at hang-up the session is submitted for an async fill job. Writes are versioned, so the agent and our team can edit a session without overwriting each other.
  • Quoting beside the call. Each call holds one WebSocket to Blaze: the conversation streams out, carrier quote status streams back. Nothing on it can block the audio, and a dropped connection resumes where it left off, so a Blaze deploy mid-call costs the caller nothing.
  • Typed, idempotent events. Every call emits typed lifecycle events, from start through each turn, tool call and transfer to hang-up. They reach Chassis, our voice backend, through an on-disk spool, so a network blip delays an event instead of losing it, and Chassis writes them idempotently, so a retry never writes twice. Mission Control, our ops dashboard, streams live from Chassis and reconciles against the DB.
  • Config as data. Runtime settings come from a versioned document read once per call, so we know exactly which settings and talk path ran any call.
  • Handoffs. When the application is done, Harley plays hold music while quotes come back, dials a licensed team member into the room and steps out once they're connected. A caller who asks for a person mid-call is handed over with a SIP REFER.
A Harley worker writes each event as an atomic file in an on-disk spool, which posts it to Chassis; Chassis inserts it into the DB idempotently and streams it live to Mission Control over SSE, and Mission Control runs a read-only reconcile from the DB to catch anything the live stream missed. How events reach the DB Harley worker call.startedtranscript.partialtool.invokedcall.completed On-disk spool retriesuntil delivered Chassis verifies,normalizes DB raw events,sessions Mission Control live ops board written firstatomic file POSTauthenticated insertidempotent live streamSSE reconcileread-only A Harley worker writes each event as an atomic file in an on-disk spool, which posts it to Chassis; Chassis inserts it into the DB idempotently and streams it live to Mission Control over SSE, and Mission Control runs a read-only reconcile from the DB to catch anything the live stream missed. How events reach the DB Harley worker call.startedtranscript.partialtool.invokedcall.completed written firstatomic file On-disk spool retriesuntil delivered POSTauthenticated Chassis verifies,normalizes insertidempotent DB raw events,sessions live streamSSE Mission Control live ops board reconcileread-only
Every event is spooled on disk, ingested once and streamed live to Mission Control.

The hard parts

Most voice agents answer a question or book an appointment in a few minutes. An onboarding call is a full insurance application as a conversation: it can run past half an hour, span several types of insurance, and every answer has to land in the right field. That breaks four assumptions: that nothing else happens while the agent talks, that a call is one task, that it's short and that it fits in the model's memory.

Real-time audio, parallel agents

The hard part is everything running beside the audio. While Harley talks, the business lookup runs behind a short spoken line and a typing sound, coverage applications open in the background, and Blaze's carrier lanes listen to every turn, flag missing answers and quote in parallel. None of it may block the audio loop or leave the caller in silence.

Every tool call has a hard time budget, and Harley speaks before any wait starts. The opening is just as careful: the business phone number arrives as keypad (DTMF) digits, the first keypress mutes the caller's audio so the tones aren't heard as speech, Harley reads the number back, and the lookup runs only after the caller says yes.

On the line, the caller confirms the industry, Harley speaks while the session opens and the talk path loads, then asks the first question; in the background, other coverage forms open in parallel and every turn streams to Blaze over a WebSocket, without blocking the audio loop. No dead air while tools run On the line In the background async Caller confirms the industry Harley speaks “One moment while Ipull up the questions.” Session opens talk path loads First question Other coverage forms open in parallel Every turn WebSocket to Blaze Every tool call has a hard time budget and never blocks the audio loop On the line, the caller confirms the industry, Harley speaks while the session opens and the talk path loads, then asks the first question; in the background, other coverage forms open in parallel and every turn streams to Blaze over a WebSocket, without blocking the audio loop. No dead air while tools run On the line Caller confirms the industry Harley speaks “One moment while Ipull up the questions.” Session opens talk path loads First question In the background Other coverage forms open in parallel Every turn WebSocket to Blaze Every tool call has a hard time budgetand never blocks the audio loop
Harley speaks before every wait, so the line is never silent while a tool runs.

Several coverages, one call

Many owners need several types of insurance, say general liability, workers' comp and commercial auto. Each has its own application, and they overlap: all of them ask what the business does and how many people work there. Our first version walked them one after another, so callers heard the same questions two or three times, about 2.2 times the questions they needed.

Our next attempt folded every form into one big script. It looked elegant and quietly lost 46 to 81 percent of the merged questions, and past 15 or 20 minutes one giant block of instructions stopped behaving predictably. Now the primary application is asked first, exactly as written, and each other coverage follows as its own part with anything already asked taken out. Cutting that repetition is a big part of our 63% cut in average call time.

Harley starts the primary application as soon as the caller confirms the industry; the other applications open in the background and their repeats are struck before the first part ends. Coverage is settled up front, so the plan never shifts mid-call, and anything an owner adds later goes to our licensed team at the transfer.

Before, general liability, workers' comp and commercial auto each asked six questions one after another; now general liability is the first leg and asks the shared questions once, each later coverage is its own leg that asks only what it adds, and those later forms open in parallel at the start. Three coverages, one call Before General liability 6 questions Workers' comp 6 questions Commercial auto 6 questions shared questions asked three times2.2× the questions needed Now Shared questions leg 1 · asked once GL adds leg 1 · subcontractors WC adds leg 2 · payroll by class Auto adds leg 3 · vehicles, drivers opened in parallel at the start one leg per formrepeats struck in background Before, general liability, workers' comp and commercial auto each asked six questions one after another; now general liability is the first leg and asks the shared questions once, each later coverage is its own leg that asks only what it adds, and those later forms open in parallel at the start. Three coverages, one call Before General liability 6 questions Workers' comp 6 questions Commercial auto 6 questions shared questions asked three times2.2× the questions needed Now Shared questions leg 1 · asked once GL adds leg 1 · subcontractors WC adds leg 2 · payroll by class Auto adds leg 3 · vehicles, drivers WC and auto forms open in parallel at the start one leg per formrepeats struck in background
Shared questions asked once, then only what each coverage adds.

Long calls, one leg per coverage

Several applications plus half an hour of conversation in one set of instructions gets unreliable: the model starts skipping questions and losing its place. So a long call runs in legs, one per coverage. Each leg carries only its own form and tools, and hang-up doesn't exist until the final leg, because a model that thinks it's finished will reach for it. Between legs, the next leg's tools and instructions swap into the live session in place, so the audio never drops and the model never holds more than one application. To the caller it's still one conversation.

One trap: LiveKit's realtime plugin recycles the model session every 20 minutes by default, and the replayed history drops every tool call and result, so calls past 20 minutes lost track of what they'd done. We moved the recycle to 55 minutes, past virtually every real call (99% end within 40) and inside the provider's hard 60-minute limit.

A timeline of realtime session recycling from 0 to 60 minutes into a call: a typical call lasts 15 minutes and long calls run up to 45, the plugin default recycles the session at 20 minutes where history replay drops tool calls, Harley recycles it at 55, and a shaded margin runs to the provider's hard close at 60. Session recycle: 20 vs 55 minutes typical call, 15 min long calls, up to 45 min provider hard close Plugin default: 20 minhistory replay drops tool calls Harley: 55 min99% of calls end by 40 margin 0102030405060 realtime session recycle (minutes into the call) A timeline of realtime session recycling from 0 to 60 minutes into a call: a typical call lasts 15 minutes and long calls run up to 45, the plugin default recycles the session at 20 minutes where history replay drops tool calls, Harley recycles it at 55, and a shaded margin runs to the provider's hard close at 60. Session recycle: 20 vs 55 minutes typical call, 15 min long calls, up to 45 min Plugin default: 20 minhistory replay drops tool calls Harley: 55 min99% of calls end by 40 provider hard close margin 0102030405060 realtime session recycle (minutes into the call)
Moving the session recycle from 20 to 55 minutes put it past virtually every real call.

No compaction

Voice agents usually survive long conversations with compaction: older turns get summarized to free up the model's memory. On an onboarding call, that's where answers disappear. A summary that forgets "twelve employees" means asking again or, worse, a wrong answer on the application. Harley is built so even the longest call never needs compaction, and nothing the caller confirms depends on the model's memory.

Our first settings compacted well before the longest calls ended, so it fired on exactly the calls that mattered most. Worse, a speech-to-speech model can't summarize its own history, so compaction there often just cut. We fixed it from three sides. Legs keep the instructions to one form, with the facts the next leg needs restated in its prompt. Every word stays in a verbatim transcript that fills the application after the call. And key facts about the caller are pinned for the whole call. With all three in place, the compaction limit sits far past our longest call ever, so it never runs.

Before, a long call's history was compacted by cutting older turns, which lost the employee count, so Harley asked again; now each coverage runs as its own small-prompt leg that keeps every turn verbatim and fills the application from the transcript, with the business, caller and industry pinned for the whole call and a compaction limit set past any real call.Why compaction never runsBeforeHistory growson a long callCompactedolder turns cut“12 employees”lostHarley asks againNowOne leg per coverageone form in contextEvery turn keptcaller track, verbatimTranscript fillsthe application, after hang-upPinned for the whole callbusiness, caller, industryCompaction limitpast any real call Before, a long call's history was compacted by cutting older turns, which lost the employee count, so Harley asked again; now each coverage runs as its own small-prompt leg that keeps every turn verbatim and fills the application from the transcript, with the business, caller and industry pinned for the whole call and a compaction limit set past any real call.Why compaction never runsBeforeHistory growson a long callCompactedolder turns cut“12 employees”lostHarley asks againNowOne leg per coverageone form in contextEvery turn keptcaller track, verbatimTranscript fillsthe application, after hang-upPinned for the whole callbusiness, caller, industryCompaction limitpast any real call
Compaction cut answers mid-call; now nothing is compacted and the transcript fills the application.

Filling the application

The conversation is half of onboarding. The other half is applications that have to be perfect, because an underwriter prices exactly what's on the page. Harley doesn't type into a form while it talks; at hang-up it submits the session to Harley MCP, and the applications are built asynchronously from the verbatim transcript.

At hang-up the session is submitted and the verbatim transcript starts a durable fill job, one run per form; the general liability, workers' comp and commercial auto forms fill in parallel, every answer is written with evidence, read back and audited against the PDF before licensed review, and any change from review loops back to refill. Filling the application Every answer, every form at hang-up session submitted refill Verbatim transcript every turn of the call Fill job durable, one run per form in parallel General liability Workers' comp Commercial auto Write with evidence transcript excerpt Read back saved answers re-read Audit + PDF check complete, matches Licensed review before any underwriter At hang-up the session is submitted and the verbatim transcript starts a durable fill job, one run per form; the general liability, workers' comp and commercial auto forms fill in parallel, every answer is written with evidence, read back and audited against the PDF before licensed review, and any change from review loops back to refill. Filling the application Every answer,every form at hang-up session submitted refill Verbatim transcript every turn of the call Fill job durable, one run per form in parallel General liability Workers' comp Commercial auto Write with evidence transcript excerpt Read back saved answers re-read Audit + PDF check complete, matches Licensed review before any underwriter
After the call, every form fills in parallel from the transcript, and every answer is checked before a person reviews it.

A first pass extracts each form's answers the moment the caller hangs up. Then a durable fill job takes over: one run per form, all in parallel, each following its form's own rules and writing answers back through Harley MCP.

Perfect applications come from never trusting a single write. Every answer carries the transcript excerpt it came from, and every write is read back. Each form is audited for completion, rendered to PDF and checked against the saved answers; only validated PDFs move forward. If something changes after the call, a refill reruns the job, and a licensed team member reviews the result before anything reaches an underwriter.

New fill paths roll out by percentage, with each call pinned to one side so a resubmitted call never flips between them.

What real calls taught us

A demo took days. Holding up on more than 10,000 real calls, with bad connections, background noise and owners busy running a business, was the real work.

Speed. On the phone, a short silence can sound like a dropped call. We've built the audio path three times, from a chain of three models to a hybrid to one speech-to-speech session, and each version had to hold accuracy on the application, not just shave time.

Knowing when someone is done talking. Owners pause mid-answer to think or shout to a coworker. Jump in early and Harley talks over them; wait too long and the call drags. Turn detection is set for real phone lines, not a quiet headset: noise reduction runs first, then server-side voice detection uses a high speech threshold so line noise doesn't count as talking, pads the start so the first syllable survives, and ends a turn only after a silence window tuned on real calls, long enough to keep a breath inside the answer and short enough that the caller doesn't hear a gap.

A 0 to 6 second timeline: the caller speaks in two parts around a short pause, after noise reduction, server-side VAD speech probability crosses the threshold only while they speak, and after a longer silence Harley starts the next question.Turn detection on a phone lineCallerSpeechprobabilityserver-side VADHarley“We have twelve…”“…employees.”prefix paddingshort pause, same turnthresholdnoise stays under the thresholdnext questionsilence window ends the turn0123456 s A 0 to 6 second timeline: the caller speaks in two parts around a short pause, after noise reduction, server-side VAD speech probability crosses the threshold only while they speak, and after a longer silence Harley starts the next question.Turn detection on a phone lineCallerSpeech probabilityserver VADHarleyprefix padshort pausethresholdnoisenext questionsilence ends the turn0123456 s
Noise stays under the threshold, a short pause stays in the turn, and only a longer silence ends it.

Interruptions. Callers talk over voice agents constantly. When one cuts in, Harley stops, handles what they said, then returns to its question.

Pinning the language. On autodetect, the session's transcription turned line noise into confident foreign-language text that barged into Harley's turn and landed in the transcript as an answer. Pinning it to English stopped that without costing other languages: when a caller asks, Harley re-pins both transcribers to theirs.

Getting the details right. Mishearing is most common on phone numbers, emails and business names. A wrong digit means we can't reach the owner; a misspelled name can end up on the policy. Harley reads back phone numbers, emails, license and tax ID numbers, ZIP codes and start dates and waits for a yes, and a new business is spelled out and confirmed before we create it. The caller-side transcriber is primed with each industry's vocabulary.

Inside one call

  1. Step 1 of 4

    The caller pauses mid-answer. Harley waits, because the pause is still their turn.

  2. Step 2 of 4

    A clatter on the line is ignored. Only real silence ends the turn.

  3. Step 3 of 4

    The caller cuts in. Harley stops, answers, then goes back to its question.

  4. Step 4 of 4

    Harley reads the phone number back and saves it only after a yes.

The whole call

  1. HarleyHow many people work for you, including yourself?
  2. CallerWe have twelve… employees.
  3. HarleyGot it, twelve employees. And do you use any—cut off
  4. CallerWait, sorry, is my crew covered if someone gets hurt?
  5. HarleyGood question. That's what workers' comp is for: it pays for employees who get hurt on the job, and it's one of the coverages we're quoting for you today. It's also why I'm asking: do you use any subcontractors?
  6. CallerYeah, a couple, for roofing work.
  7. HarleyAnd what's the best phone number to reach you?
  8. CallerFive one two, five five five, zero one four seven.
  9. HarleyThat's five one two, five five five, zero one four seven. Is that right?
  10. CallerYes.
  11. HarleyPerfect, got it.
An illustrative call with synthetic voices: a pause stays in the turn, noise is ignored, a barge-in is handled, and a phone number is read back before it's confirmed.

Asking only what's needed. Early versions asked questions that didn't apply; now every question carries the condition that gates it, and nothing the application requires gets dropped.

Waiting like a person. Owners often say "give me a minute, let me grab my policy." Harley waits up to five minutes, checking in once a minute, and checks in on its own after ten seconds of unexplained silence.

No guessing. The application is filled from the verbatim transcript, not the model's memory, so every answer traces back to something said on the call, and nothing reaches an underwriter until our licensed team has checked it.

Staying out of advice. Harley explains what a question means but never recommends coverage or tells a caller they're covered. Those questions wait for our licensed team, who join by live transfer at the end of the call. Callers can ask for a person at any point, and once we know the business, Harley hands them over. A licensed team member reviews every quote before the owner sees it.

Evals

Real calls tell us what to fix; evals make sure a fix doesn't break something else. Every change runs our test suite and simulated callers on a test line, and every night scripted onboarding conversations run across our industries and caller types, scored on completion (every question answered and recorded), order and how close the wording comes to our team's on real calls. A run fails unless completion and order are perfect. Test calls are tagged from the start and kept out of production data.

Where it is now

More than 10,000 calls have gone through Harley since August, across 59 industries and 14 lines of coverage. Average call time is down 63%, and because every call gets its own room, a rush of calls no longer leaves owners on hold. Quoting starts while the owner is talking, so instead of a day or two, customers wait two or three seconds on hold for a quote to be ready for our licensed team to review, with 20 or more back within the first 10 minutes of the call. Our licensed team spends the time saved on work that needs a licensed person.

A few of the industries and lines of coverage Harley has talk paths for.

Our team still uses the teleprompter on the calls they take, and today it runs on Harley too, with its voice turned off.

Next, Harley takes on more of the routine calls after onboarding. If you'd like to work on real-time audio, agents that run beside a live call and the data behind them, or work this way as a licensed insurance professional, we're hiring.