Wynda Clayton spent years auditing Medicare Advantage plans for the federal government before she became a compliance executive on the other side of the table.
The moment she keeps coming back to, when she talks to the engineers building AI for this space, is not a court case or an enforcement action. It is a toolkit.
In late 2023, the Department of Health and Human Services Office of Inspector General published a set of high-risk diagnosis codes it uses to flag Medicare Advantage claims for review, along with the clinical logic behind the selections and the SQL queries used to run them. Not just the findings. The detection method itself was made public.
Clayton’s read on this, stated plainly in a session she ran earlier this year: “If OIG can identify unsupported diagnoses with just a few lines of SQL, CMS can too. And that means so can you.”
Her point was not really about SQL. It was about what kind of AI system can actually run that query and produce an answer someone can act on, and what kind cannot.
Two Different Questions, Wearing the Same Interface
Most AI vendor evaluations, in healthcare or anywhere else, are built around one question: how accurate is this? Buyers run bake-offs, compare benchmark scores, and pick a winner.
OIG’s published toolkit asks a different question entirely: can this specific documentation support this specific diagnosis, according to this specific rule?
That is not an accuracy question. It is a validation question, and a lot of AI architecture that scores extremely well on the first question has no mechanism for answering the second at all.
While modern discussions around AI revolutionizing medical coding accuracy focus heavily on automated precision and speed, accuracy alone isn’t enough if the underlying logic can’t survive an audit.
The distinction shows up in OIG’s own language. Its compliance guidance states that reviewers “reviewed medical records that MAOs provided as support; however, a high percentage of the associated HCCs could not be validated,” giving the agency “indications that MAOs are submitting diagnosis codes for payment that are not verifiable.”
That is the language of a validation failure, not an accuracy failure. The codes were, in a lot of cases, probably reasonable guesses. They were just not provable.
Why Prediction Can Only Ever Say Yes
Here is the part Clayton’s SQL comment actually points to. A model built to predict, whether that is a classic classifier or a large language model, produces a likelihood.
Ask it whether a diagnosis is supported, and it hands back a number. Ask it whether a diagnosis should be removed because the documentation does not hold up, and there is nowhere in its output for that answer to live. Its entire structure is candidates ranked by probability.
OIG has flagged the automated version of this pattern directly, naming “prompts generated by artificial intelligence algorithms” that push toward adding diagnoses among the conduct its investigations have surfaced.
Nobody built these systems to behave this way on purpose. They behave this way because a probability score is a one-way instrument. It can go up. It has no way to go to zero and mean something.
A system built from an actual rule, the kind OIG published, works differently. It can be tested against a record, and it can fail that test, which is the only mechanism by which a system can meaningfully say a diagnosis is unsupported rather than merely less likely.
An Old Idea, Forced Into The Open by a Regulator
None of this is new in AI research. Rule-based systems dominated the field for decades before statistical learning proved better at handling messy, unstructured input, and the current interest in combining the two, often called neuro-symbolic AI, is closer to a rediscovery than a breakthrough. What is new is a regulator forcing the question in public, with a published query anyone can run against their own data.
Clayton frames the shift in her sector as one from capture to care: from a system judged on how much it found to one judged on whether what it found can survive someone else checking.
The swap of the domain and the shape of the argument holds for any AI product whose output eventually has to face a person with the authority to ask why.
Anyone building a risk adjustment solution for Medicare Advantage right now is living this shift directly, replacing systems that only ever add with ones built to test a rule and report the result either way.
The regulator handed over the exact standard it is checking against, in plain SQL. Very few vendors, in this category or any other, have taken the hint yet.
