Software Engineer, Voice Evaluation [NYC or SF]
$220K – $300K • Offers Equity
US Visa and Green Card sponsorship available
About Concurrence
Concurrence (formerly Amigo) is building the platform healthcare organizations use to build their own clinical AI agents. Our mission is to make excellent care accessible to every patient, not rationed by the limits of clinician time.
Underneath our platform is the Health World Model, which unifies a health system’s data into a living, event-level record of every patient’s care. On top of it, agents handle work across the patient journey, from intake and scheduling to medication management and clinical decision support.
Our agents have supported more than 10 million patient encounters and are on track to grow tenfold this year. We run natively on Databricks, integrate with every major EHR, and are backed by Madrona, General Catalyst, and Optum Ventures. We’re about 35 people, working in person in New York and San Francisco.
About this role
As a Software Engineer, Voice Evaluation at Concurrence, you'll build the system that determines whether a voice agent is ready for patients. You'll own realistic audio tests, scoring and the release gate, with authority to block releases that fail it. The tests need to catch failures that a clean transcript can miss, from mishandled interruptions to missed clinical information or a failure to hand off to a person.
What you'll do
Build simulated callers and call-replay tests that exercise the production voice path with real audio
Test phone trees, hold queues, interruptions, conflicting clinical information and appropriate escalation
Build scoring systems and calibrate automated judges against human assessments
Measure clinical failure rates and uncertainty, and test whether simulation results predict behavior on real calls
Define release criteria with model, runtime, product and clinical colleagues, own the gate and block changes that fail those criteria
Turn production failures into regression tests, verify fixes and monitor behavior after release
Maintain held-out evaluation sets so models are tested on examples they did not train against
What we're looking for
You've personally built and owned evaluation or simulation for production agents at scale. Voice and audio experience is a strong preference
You've built agent simulations or replay systems, implemented scoring and caught a concrete failure before release
Your evaluation results determined what shipped, and you led that work
You can explain what a score captures, what it misses and how you checked it against real behavior
You write reliable software that runs at volume and follow failures through to a verified fix
You use coding agents in your work and independently verify their output
You're low ego, direct, and can hold the line on release criteria with evidence
You can work on site in New York City or San Francisco
Nice to have
Experience calibrating automated judges against human labels
Load testing and concurrent calls using audio
Speech or audio machine learning
Experience in healthcare or another regulated environment
Benefits (available to Full-Time Employees)
Health & Wellness
Comprehensive health, dental, and vision insurance
Daily catered lunch and dinner
Mental health support and wellness coaching
Flexible wellness stipend for fitness, therapy, or personal growth
Growth & Development
Annual learning budget for courses, books, or conferences
Conference attendance budget for professional development
Annual team offsite
Academic collaboration opportunities
Unlimited PTO
Our Core Values
Patients Win, We Win
If patients aren't getting better care, we haven't earned the right to scale. Every internal decision gets pressure-tested: does this make patients' lives better? If we can't draw the line, we question why we're doing it.
High Standards, High Care
We hold a high bar for the team because patients are counting on us to get this right. But high standards only work with genuine investment in each other. You can take risks, admit mistakes, and challenge ideas. Not despite our standards, but because of them.
Thoughtful Urgency
We move fast by default, but speed without judgment is recklessness. The discipline is knowing which decisions are reversible vs. not. In healthcare AI, the companies that win will be fast everywhere they can be and careful everywhere they must be. We build the muscle to do both.
Intensely Measured
We instrument patient outcomes, provider ROI, system performance, and clinical accuracy. But data without action is surveillance. Every metric should have an owner, a threshold, and a response plan. If we're measuring something but never acting on it, we stop measuring it.
Who Builds With Us
Low ego: Politics and territory don't interest you. The best ideas win, regardless of who has them.
Direct: You say the hard thing, challenge ideas openly, and commit fully once decided.
High agency: You thrive on trust rather than instruction. When you see something is broken, you fix it. You don’t file tickets and wait for someone else.
Bar of excellence: You hold yourself to a bar most people wouldn't, and you want teammates who do the same.
Skeptical: You push back on rules that don’t make sense and question assumptions that haven’t earned their place.
Apply
You can also apply in a new tab.