The vignettes gave away the answer.
Each of 177,120 synthetic cases named the disease, named the active ingredient, and asked for up to three FDA-approved products in order of preference. The clinical reasoning was already done. What remained was a lookup: of the 5.5 approved products that carry that ingredient on average, which one comes first.12
Across 883,403 responses from five frontier models, one product took 97.5% of first choices.2
The easy reading is that the models absorbed prescribing habits. The data says otherwise. Against U.S. prescription records from the Medical Expenditure Panel Survey, the models' first choice matched the most-prescribed product 29.7% of the time, and 17.0% when weighted by prescription volume. The models tended to pass over the products physicians actually prescribe, and over products approved later.23
What predicted a recommendation was visibility. Originator status carried an odds ratio of 20.5. A dedicated Wikipedia page carried 3.00. Google search volume carried 1.70.2
Caution
The popularity being reproduced is not clinical popularity. It is discursive popularity — the marketing copy, patient forums, news coverage and encyclopedia entries that accumulate around a brand during its years of exclusivity.
Amlodipine is chemically identical to Norvasc, but Norvasc had a decade of advertising writing itself into the corpus before amlodipine had a manufacturer willing to compete for attention. The model learned the shape of that attention and treats it as knowledge.
Earlier work had shown the instability underneath. Swapping brand and generic names in standard biomedical benchmarks costs models 1–10% accuracy on MedQA and MedMCQA, on questions that are pharmacologically identical, and the authors trace part of the fragility to contamination of those test sets in widely used pre-training data.4 In oncology tasks, GPT-3.5 was more likely to associate brand names than generics with being effective (OR 1.43) and with being free of side effects (OR 1.76).5
Those were warning lights. This is the bill. Across the 51 active ingredients with complete pricing data, 38 of them — 74.5% — carried higher projected acquisition costs under model-guided prescribing. Ueda put the total at "approximately US$55.6 million," and then put it in the form that matters: "This represents a 1,019% increase in the cost of purchasing these prescriptions."23
Greedy decoding turns a plurality into a monopoly
The collapse has a mechanical component that rarely gets named. The researchers ran every model at temperature zero, which is exactly how a clinical tool would be configured, because nobody wants a copilot that answers the same patient differently on Tuesday.2
Temperature zero means greedy decoding: at each step, take the most probable token.
Greedy decoding has no concept of a close second.
If a model's internal distribution over products for an ingredient were 55% brand and 45% generic, the output would be 100% brand, every time, for every patient. The training corpus produces a skewed prior, and deterministic decoding converts any skew, however modest, into unanimity. A mild visibility bias in the weights becomes total concentration in the behavior.
Important
Raising the temperature does not fix it. Sampling would scatter recommendations in proportion to web attention, replacing a monopoly with a lottery weighted by advertising history. Neither mode makes the choice a function of evidence, because the generation step was never the right place to make it.
Asking nicely moves the collapse; it doesn't end it
The study tested the obvious remedy. Prompts asking for a comprehensive list of equivalent products, for the lowest-cost option, or for generics where therapeutically equivalent products exist all changed the answers: the leading product switched in 45.2%, 62.6% and 77.9% of comparisons respectively, and most switches moved toward a generic.2
Concentration barely moved. Mean leading-product share came in at 93.1%, 94.9% and 97.6%.2 Under the generic-preference prompt, the models were more concentrated than at baseline. They had converged on a different name.
That settles prompt engineering, and it generalizes to fine-tuning and corpus rebalancing. Each of those reshapes the distribution, which changes which product sits at the mode. None of them changes the fact that a mode is being selected. A preference-tuned model that favors generics is still choosing by learned salience, with a different salience map — and the map drifts again with the next training run, the next marketing cycle, the next wave of web content.
Debiasing is maintenance on a mechanism that should not be making the decision.
Product resolution is a query, not a generation
The sharpest point in the data is where the failure happened. The vignettes supplied the active ingredient. The models were not weighing clinical tradeoffs between drug classes; they were choosing among products that FDA has, in most multisource cases, already evaluated.
- The Orange Book publishes therapeutic-equivalence codes, and an AB rating means FDA believes the products "can be substituted with the full expectation that the substituted product can be expected to have the same clinical effect and safety profile as the prescribed product."6
- The Purple Book carries the equivalent licensing and interchangeability record for biologics.7
- RxNorm normalizes every U.S. drug by ingredient, strength, dose form and brand relationship, and links branded names to their generic counterparts explicitly.8
- NADAC gives a public acquisition benchmark.9
The one step where five models spent their probability mass on brand names is the one step in the whole pipeline that requires no intelligence at all. It is a join across three federal datasets and a formulary table.
Important
Generating that join token by token from web-trained weights is the most expensive and least reliable way to perform a database lookup.
So the structural fix is to separate the model's ability to reason about medicine from its authority to select a medication. Containment is a property of the system, not of the operator inside it.10 The model keeps what it is good at — reading a clinician's question, pulling patient context out of messy notes, explaining alternatives in plain language. Everything that determines what is eligible, what is equivalent and what it costs moves into deterministic services reading versioned reference data.
Five stages. The model touches the first and the last.
Intent — the clinical question becomes structured fields: indication, patient constraints, route, dose, payer.
Candidates — the full product set is retrieved from reference data, not recalled. A query cannot forget the generic with no Wikipedia page.
Constraints — hard masks for contraindications, interactions, allergies, formulary exclusions, discontinued listings. The study found models recommending products already off the market, an error class that disappears when candidates come from a live registry.3
Ranking — lexicographic, not a weighted score. Safety and mandatory requirements first; an option that fails them cannot be rescued by price. Then evidence-supported effectiveness, then patient-specific factors like adherence burden, then cost. Weights, where they apply at all, belong to clinicians and P&T committees and apply only among options that have already cleared the hard gates.
Explanation — the model returns, and explains the result using the sources the services actually read.
The single most effective intervention inside that pipeline is to keep brand names out of the decision until the decision is over. The ranker receives normalized identifiers and clinical attributes: Candidate A as ingredient X, oral, 10 mg, AB-rated to the reference listed drug; Candidate B as the same ingredient from another manufacturer with the same rating. Equivalence gets verified from the Orange Book or Purple Book before two candidates are treated as substitutable, because sharing an active ingredient does not establish interchangeability across formulations, and sharing a therapeutic class establishes less than that. Only after ranking is complete do identifiers resolve back into commercial names for display.
A brand-blind ranker cannot be swayed by a name it never sees. It can still inherit bias through the evidence and attributes it does see — the trial literature was itself largely generated on originator products — and those inputs need their own audit. But the single largest predictor in the study, an odds ratio of 20.5, leaves the decision path entirely.
Provenance is generated, never asserted
Every recommendation should carry a machine-readable record of how it was produced.
{
"selected_product_id": "rxcui:normalized-id",
"candidate_source": "rxnorm:2026-11-02",
"candidates_evaluated": 6,
"constraints_applied": ["formulary:plan-id", "interaction-check", "active-listing"],
"equivalence_basis": "orange-book:AB",
"evidence_state": "supported",
"cost_basis": {
"acquisition": "nadac:2026-10",
"plan_net": "unavailable:confidential-rebate",
"patient_out_of_pocket": "formulary-tier:plan-id"
},
"proprietary_inputs": ["interaction-compendium:licensed"],
"brand_used_as_ranking_feature": false,
"requires_clinician_approval": true
}
The services that performed each check write those fields from their own results. The model is never permitted to assert that validation occurred, because a model trained on documentation of validation will produce text that looks like validation.
A provenance field the model can write is a hallucination with a schema.
That record also enables a standing adversarial test, and it is the test that does not decay as the models improve.11 An independent evaluator reruns each decision with brand labels swapped, candidate order shuffled and clinically irrelevant demographic details perturbed. If the recommendation changes without a clinically meaningful reason, the system flags a ranking instability and routes the case for review. Brand-swap sensitivity becomes a production metric rather than a journal finding two years after deployment.
The 177,120 vignettes are a ready-made benchmark for the same purpose. Run them through a baseline model and through the constrained pipeline and the outcomes that matter are directly comparable: guideline concordance, contraindicated or discontinued recommendations, brand-swap sensitivity, and cost regret against the lowest-cost clinically appropriate option.
Important
Recommendation diversity is not the target. A system that picks the AB-rated generic 99% of the time because it is the cheapest equivalent on formulary is succeeding, and its concentration statistic looks almost identical to the failure in the study. Ninety-seven percent agreement is fine when the reason is a formulary rule, an equivalence rating or a price. It is indefensible when the reason is a Wikipedia page. Only the provenance tells the two apart.
Synthetic vignettes are not prescriptions
The strongest objection is that none of this happened to a patient.
These were generated cases, not consultations. No clinician in the study typed a question, read an answer and signed a prescription; the $55.6 million is a projection against historical prescribing, not observed spending, and it covers 51 ingredients rather than a formulary. The odds ratios are associations — the reporting on the study says so directly, that they "do not establish that brand status or online visibility caused a model's choice." And the 97.5% figure is the leading product's average share within each model-and-pair analysis, not one brand's share of everything.
All of that is correct, and none of it is load-bearing, because the failure mode does not require a patient to be present. It requires an ingredient, a product list and a decoder. Those exist in every deployment. The pooled leading product exceeded 90% in 96 of the 123 disease-drug pairs, which is a property of the models rather than of the vignettes.2 A projection built on NADAC also understates rather than flatters the result, since NADAC excludes rebates and the net prices that actually move formulary placement are confidential.1213
The study measured a mechanism under laboratory conditions. The mechanism ships in production at temperature zero.
This was never a pharmacy problem
The same machinery runs through every domain where a language model acts as a recommender. Ask one for a web framework, a database or an observability stack, and it reproduces the GitHub stars and conference-talk volume of the last decade. Ask for an index fund, and the families with the largest advertising budgets and the heaviest personal-finance blog coverage dominate the answer regardless of expense ratio or tracking error. Anyone who has worked under a suitability standard knows that "most frequently discussed" has never been a defense for a recommendation, and a fund pick produced by greedy decoding over web text would fail a compliance review on sight.
What generalizes is not the clinical pipeline but the abstraction under it: decouple candidate discovery from candidate evaluation, make evaluation independent of historical visibility, and make every authority the evaluation leans on inspectable and contestable.14
- The clinical adapter reads RxNorm, the Orange Book and NADAC.
- A financial adapter reads prospectuses, expense data and suitability rules.
- A technical adapter reads benchmarks and maintenance signals.
The model handles semantics and explanation in all of them. Deterministic services establish facts and permissible choices. Validated logic ranks what survives.
That last requirement is the one that does not resolve here. Brand-blind ranking on public reference data removes the bias the model learned from the web; it does not remove the biases of the institutions the system now trusts instead. The equivalence and acquisition layers are open federal data, but the price that most shapes formulary placement is a net price after rebates, and those contracts are confidential.13 The constraint stage is further along than that. One of the two companies that publish U.S. drug-interaction data already sells it to AI agents over the Model Context Protocol, which makes the deterministic service in this pipeline a procurement decision before it is an engineering one.15 Replacing probabilistic authority with institutional authority only works if the institutions are inspectable, and some of them are not.16
The models were told the ingredient. The equivalence ratings, the acquisition prices and the complete list of approved products were sitting in public federal databases the entire time.
Five systems still spent 97.5% of their first choices on the names the internet repeated most. Not because the question was hard.
Because the lookup was never performed.
This is analysis and commentary, not legal, medical or investment advice.