Generative AI: Healthcare

Generative AI in healthcare

Clinicians do not need another chatbot. They need the note written, the referral packet assembled and the phone answered, inside the EHR they already use, with a human signing off on anything clinical. We build ambient scribes, front desk voice agents and document agents on HIPAA eligible model endpoints, measure them against your own records before go live, and write the results back over FHIR. It is the same approach we run in production in Prescr, our own clinic product.

Get a free assessment Healthcare software

Key highlights

We build ambient scribes, AI front desk voice agents, prior authorization packet agents and clinical knowledge assistants that work inside your EHR workflow.
PHI only reaches model endpoints covered by a BAA, such as Claude on AWS Bedrock or GPT on Azure OpenAI, or self hosted Llama inside your own network.
A clinician reviews and signs every AI drafted note, letter and message, and every edit is logged against the model version that produced the draft.
Output is written back to Epic, Oracle Health (Cerner) and other EHRs over FHIR R4 and HL7v2, so nobody copies and pastes between tools.
A measured prototype on your own de identified data usually takes about 4 weeks, and a production rollout 10 to 16 weeks.

Why healthcare AI projects stall

The models are capable enough. Healthcare AI projects stall on PHI handling, clinical safety sign off, EHR integration and the absence of any evidence that the tool is accurate on your patients.

Pilots that never pass clinical safety review. A demo impresses the CMIO, then the clinical safety officer asks for a hazard log, failure modes and test evidence. Nobody has them, and the pilot quietly ends.

PHI leaking into consumer chatbots. Staff paste patient letters into a public AI tool because the approved workflow is too slow. The organisation now has a privacy incident and still no sanctioned alternative.

Hallucinated clinical detail. One invented medication dose or a symptom the patient never mentioned is enough for clinicians to stop trusting every note the tool writes, including the good ones.

AI that lives outside the EHR. The scribe produces a great note in a separate window. The clinician copies it into Epic by hand, loses the structured fields, and saves no time at all.

Phones that nobody answers. Front desk staff spend the morning on hold queues, refill requests and rescheduling. Missed calls become missed appointments and lost revenue, and patients go elsewhere.

No idea where the regulatory line sits. Summarising a consultation is one thing, suggesting a diagnosis is another. Teams build first and only later learn the feature may count as software as a medical device.

Use cases

Generative AI use cases we build for healthcare

Each use case below is built around a clinician or staff member who stays in control, with retrieval, structured outputs and an evaluation harness behind it.

Ambient clinical scribe

Speech to text captures the consultation, an LLM turns it into a structured SOAP or specialty note with suggested ICD-10 codes, and the clinician edits and signs before it is filed to the EHR as a FHIR DocumentReference. Our own product, Prescr, runs an ambient scribe in production.

AI front desk and voice agents

Voice and chat agents on Asterisk or LiveKit SIP that answer calls around the clock, book and reschedule against your scheduling API, take refill requests and hand off to staff with a summary. Clinical questions are always escalated, never answered.

Prior authorization and referral packets

Tool using agents pull problems, medications, labs and notes from the chart over FHIR, match them to payer criteria and draft the letter of medical necessity or referral letter for staff to check and submit.

Clinical knowledge assistant

RAG over your own protocols, formularies, care pathways and policies with hybrid search and citations to the source paragraph, so nurses and staff get the local answer rather than a generic one from the open internet.

Revenue cycle copilots

Coding suggestions from the signed note, claim scrubbing explanations, and denial appeal drafts built from the 835 remittance reason codes and the chart. Coders and billers approve every submission.

Patient inbox triage

Portal and SMS messages are classified, routed to the right pool and given a draft reply for clinician review. Red flag symptoms are caught by deterministic rules first, so urgent messages never wait on a model.

How we deliver

From one workflow to a governed rollout

01
Use case discovery and data readiness
We sit with clinicians and front desk staff, rank use cases by time saved and clinical risk, and check what data, EHR access and de identified samples exist. Output: a written scope with a hazard list and acceptance criteria.
02
Evaluation harness and prototype
A golden set of real, de identified encounters or calls is built with clinicians, then two or three models are tested against it for accuracy, omissions, latency and cost. The prototype is chosen on measured results.
03
Guarded pilot with clinicians in the loop
A small group uses the tool on live work with mandatory sign off, edit tracking and a feedback button. Every draft, edit and override is logged so the clinical safety case is built from evidence.
04
Production rollout with monitoring
EHR write back, role based access and audit trails go live, then usage widens site by site. Accuracy, edit rates and cost are monitored continuously and models are reviewed as new versions ship.

AI governance for healthcare

Healthcare AI governance is mostly about three questions: where PHI goes, who is accountable for the output, and whether the feature has crossed into regulated medical device territory. We answer all three in the design, not after launch.

HIPAA and BAA covered endpoints HITECH UK GDPR and NHS DSPT DCB0129 and DCB0160 clinical safety EU AI Act (medical device AI) FDA SaMD and CDS guidance ONC HTI-1 decision support transparency India DPDP Act

PHI stays on covered or self hosted models. We use model endpoints covered by a BAA, such as Claude on AWS Bedrock, GPT on Azure OpenAI or Gemini on Google Vertex AI, with data retention switched off, or self host Llama or Qwen in your own VPC when data must not leave your network. Identifiers are redacted before prompts where the task allows, and hosting follows your US, UK, EU, UAE or India residency requirements.

A clinician is accountable for every output. Notes, letters and messages are drafts until a named clinician signs them. We store the prompt, retrieved sources, model and version, the draft and the final edit for each item, which gives your audit and clinical safety teams a complete trail and lets us measure accuracy from real edits.

We flag the medical device line early. Documentation, scheduling and administrative tools are generally outside medical device rules. Features that suggest diagnoses or treatment can fall under FDA SaMD oversight, UK MHRA and EU MDR, and therefore the EU AI Act high risk regime. We identify this at discovery, and for NHS deployments we support your DCB0129 and DCB0160 clinical safety documentation.

Models and tooling we use for healthcare AI

Covered commercial models where the BAA and quality fit, open weight models where data must stay in your network, and healthcare standards at every integration point.

Claude on AWS Bedrock GPT on Azure OpenAI Gemini on Vertex AI Llama and Qwen (self hosted) Whisper and Deepgram speech to text LangGraph pgvector FHIR R4 and SMART on FHIR HL7v2 LiveKit and Asterisk Microsoft Presidio PII redaction Langfuse tracing

FAQ

Common questions

How much does a generative AI project in healthcare cost?

It depends on scope, so we start with a fixed price prototype and then price production from a written scope. A single workflow such as a scribe or prior authorization agent costs far less than a multi site rollout with EHR write back. Model API costs are passed through at cost and shown per encounter or per call, and you get a written estimate within 48 hours. Our pricing page shows how engagements are structured.

Is it safe to send patient data to an LLM?

Yes, if it goes to the right endpoint. We only send PHI to model services covered by a BAA with retention disabled, such as AWS Bedrock, Azure OpenAI or Google Vertex AI, or to open weight models hosted in your own environment. Identifiers are redacted where the task allows, access is role based, and every request is logged for audit. Consumer chatbot accounts are never used.

How do you stop the AI from hallucinating clinical details?

By grounding, measuring and keeping a clinician in charge. The scribe works only from the transcript and the chart, the knowledge assistant cites the source paragraph, and outputs are checked against a golden set of real encounters for omissions and invented facts before go live. A clinician signs every note, and edit rates are monitored in production so drift is caught early.

How long until a healthcare AI tool is in production?

Usually 10 to 16 weeks. A measured prototype on your de identified data takes about 4 weeks, followed by a guarded pilot with a small clinician group and then EHR integration and rollout. Clinical safety review and information governance sign off are planned into the timeline from the start, not discovered at the end.

Should we buy an AI scribe or build our own?

Buy if an existing product fits your specialty, EHR and budget. Build when you need the AI inside your own product, have specialty templates or languages that off the shelf tools handle poorly, or want to own the data and the workflow. We have built our own product, Prescr, with an ambient scribe and AI front desk, so we can also tell you honestly when buying is the better option.

Does a generative AI feature count as a medical device?

Usually not for documentation, scheduling or administrative work, but it can if it suggests diagnoses or treatment. FDA, MHRA and EU MDR rules focus on intended use, so we document the intended use, keep clinical decisions with clinicians, and flag any feature that would cross the line before it is built.

Keep exploring

Related services and industries

Free consultation, no commitment

Ready to give clinicians their time back?

Tell us the workflow you want to fix. Written scope, timeline and cost estimate within 48 hours.

Book a free consultation All industries