Only as Good as Human Consensus: Ljubomir Buturovic on Building Machine Learning Diagnostics Without a Gold Standard

Your doctorate was in electrical engineering at the University of Belgrade, followed by postdoctoral work at Boston University’s BioMolecular Engineering Research Center. How did you get from there to building classifiers that clinicians act on – and is there anything from the engineering training that still shapes how you approach a diagnostic problem?

I have an EE degree, but my doctorate was actually in machine learning, which back then was taught at the EE Department at the University of Belgrade. A chance encounter led to a postdoc at Boston University, which then led to a job in the biotech industry (Incyte Corporation). The path from there to diagnostic classifiers led through another chance encounter at a bioinformatics course at Stanford University taught by Prof. Doug Brutlag. It was followed by an ML job offer at Pathwork Diagnostics. The rest is history. In a word: a roundabout path, sprinkled with luck and traces of hard work.

My EE training shaped my approach to diagnostic problems in the sense that we were trained to build and ship products given specs. I continue doing that to this date, though the products I ship these days are algorithmic, not electronic. I do keep engaged in electronics by maintaining an array of custom-built deep learning workstations at my home.

You have been an adjunct professor of computer science at San Francisco State University since 2005, alongside roles at Incyte, Pathwork and now Inflammatix. What does keeping one foot in teaching give you that industry alone would not?

Yes, though for the record I retired from the academic role in 2019 due to lack of bandwidth. Also, to be exact, my role was primarily supervising M.Sc. students. This was an opportunity to engage in pure academic research with my very talented students. I always liked academic research but had limited opportunities in the industry, due to the intense pressure of shipping deliverables in a tightly regulated field.

This is the problem I find most interesting in your work and the one least discussed in public. In acute infection there is no gold standard: you cannot label a training set against ground truth, so clinical adjudication has to stand in for it. How did you actually handle that, and what does it mean for what a reported AUROC can and cannot tell a reader?

In a practical sense, we engaged expert physicians to adjudicate patient cases using elaborate EHR records and a pre-defined protocol. The end product of the protocol was a simple diagnosis for each patient: bacterial infection, viral infection, co-infection or no infection. Expert adjudication is a significant bottleneck in assay development and is ripe for AI automation – more on that later. But for our TriVerity development, that is how it was done.

This approach obviously does not tell the reader how accurate the test is in some absolute sense, but only relative to human expert adjudication, which we know is erroneous and fairly variable. A test with AUROC of 1 is only as good as human consensus, not perfect. This is the price we pay for the lack of ground truth.

As we know, molecular tests like ours do not achieve perfect accuracy relative to physicians. In fact, we would not want them to – that would mean they literally replicate the judgement of doctors, thereby improving nothing on the existing standards of care. Our hope is that they have sufficient orthogonal biological information to augment the judgement of physicians and thereby help them and their patients. We think that is how users should interpret the accuracy and AUROC results: some of the discordances should resolve in favour of molecular testing and augmented care.

In SEPSIS-SHIELD, 1,222 patients had valid results but 729 had consensus adjudication, and the paper notes accuracy was lower when uncertain cases were included. How should a clinician reading those two numbers think about the difference between them?

I would use common sense: harder cases are more difficult to diagnose and the AI makes more errors, just like humans. It is not unique to our test, or to AI in general.

Bacterial infection prevalence in the cohort was around 63 per cent, which the paper attributes partly to broad inclusion criteria and seasonality. What does that imply for predictive values in an emergency department whose case mix looks nothing like the study population?

We are carefully studying test performance in the field, that is, in the hospitals that use our TriVerity test. So far we have not seen large deviation from the bacterial infection prevalence, either among hospitals or between seasons. If there is a large deviation at a site, they will have to adjust their prior/posterior calculus – that is, adjustment of the prior (prevalence) following the TriVerity test result. This is not new or specific to our test.

TriVerity reports three scores in five bands rather than a binary call. Was that a scientific decision, a regulatory one, or a clinical-usability one – and what did you learn about how clinicians actually read banded probabilistic output?

It was a clinical usability decision. There was a long back and forth within the team between 3, 4 and 5 bands. We are still learning how clinicians use it, but so far the feedback is overwhelmingly positive. One thing to point out: this decision had to be made prior to any clinical use, because you cannot use the product clinically before FDA clearance, and you cannot obtain the clearance until the product is finalized. So by definition this had to be a best guess.

You led predictive algorithm work on two FDA-cleared tests at Pathwork, including the Tissue of Origin test, and then TriVerity. Very little is written publicly about what the agency actually asks of a machine learning device. What surprised you most, and what do you wish you had understood earlier?

Most surprising: everyone – or at least novice FDA submitters – struggles about the exact wording of the Intended Use Statement, yet in practice it does not matter that much. What really matters is clinical validation of the product, ideally in the form of publication in a high-impact medical journal. The exact wording of the IUS is seldom an obstacle to adoption. At Pathwork we spent – arguably, wasted – about a year haggling about the IUS and limitations. Eventually we had to accept the FDA wording, which was severely limiting, but after product launch hardly anyone paid much attention to it. What really mattered were our manuscripts and clinical studies and case studies, and of course cost and clinical integration. I do not think physicians parse every word of an IUS before each test order.

So, a practical piece of advice: accept limitations on your IUS in the interest of reaching market faster.

Another observation, maybe not that surprising, but nevertheless worth a mention: the FDA is a fantastic US government agency of dedicated, world-class experts and professionals. It has been an honour to work and interact with them bringing products to market. They are laser-focused on patient safety, which may sometimes look like inflexibility, but it is not. A wonderful example of the great use of US taxpayer dollars for the public good, including the global good, because many countries accept FDA clearances as a de facto endorsement for use for their own citizens.

TriVerity received Breakthrough Device Designation in November 2023 and clearance in January 2025 (K241676). What did the designation change in practice, and what did it not change?

It helps with marketing, incrementally, and it modestly speeds up FDA review. It is not a game changer in any meaningful sense. Get it if it is easy, but do not spend too much time chasing Breakthrough Device Designation.

For a research group with a promising classifier and no regulatory experience, what are the two or three things that most often make the difference between a model that gets cleared and one that does not?

It is a long and perilous road from a research classifier, no matter how promising, to a clinical product on the market. Be prepared for five years plus, as a ballpark, and many challenges. Longer if you are building a hardware medical device from scratch; maybe shorter if it is Software as a Medical Device, but probably not much shorter. Embark on this if you are confident.

Maybe obvious, but nevertheless it needs to be stated: get professional advice on applying for and obtaining FDA clearance as early as possible. It is crucial to have a specialized FDA lawyer on your team – usually part-time or consulting should suffice.

For the technical team: avoid overfitting and information leakage at any cost. It is an insidious, hidden force that can single-handedly destroy your product, the company and your career in a pinch. Do not fool yourself and everyone else by presenting inflated performance statistics. Maintain statistical and machine learning sanctity and rigour during product development. Use as much AI help as possible to achieve this.

Locked versus adaptive algorithms is a live regulatory question. Where do you think the right line sits between a model that can learn from new data and one a regulator can meaningfully evaluate?

It is not in fact a live regulatory question. There are no products on the market, in clinical use, that are adaptive in the sense of changing their weights and biases in real time – at least not that I am aware of. Maybe this is referring to a United States FDA regulatory framework called PCCP (Predetermined Change Control Plan), which allows vendors to modify their device AI algorithms without seeking additional FDA authorization, using a set of predetermined rules. But that does not comprise real-time adaptive changes.

The TriVerity classifier code is proprietary and the study data are owned by Inflammatix, yet clinicians make antibiotic decisions on the output. I do not raise this as a criticism – it is the standard commercial model – but our readers will want to hear how you think that tension should be governed. What should a hospital or a regulator be entitled to inspect before a closed model is used in care?

The regulator is already entitled to inspect everything. I personally think that hospitals should also be entitled to inspect the innards of the commercial diagnostic models that they are buying, to give them incrementally more confidence about the AI tests. Nevertheless, there are two factors to consider.

First, mine is a minority opinion. Most vendors, as you pointed out, keep models hidden.

Second, and more importantly: to my knowledge, hospitals are seldom or never interested in this. At Pathwork, our CMO and I often joked, “if it helps patients, it doesn’t matter what’s inside; it could be a crystal ball.” Translation, if it is needed: what matters is whether your product helps patients, at reasonable cost. So vendors should prove that their products work. If they do, the architecture and weights do not really matter.

An independent real-world evaluation of TriVerity in a Dutch emergency department has now been published (the Acutelines study). What is your view on the role of independent external validation for AI diagnostics, and how much weight should it carry relative to the sponsor’s own pivotal study?

Hard to comment, given that I have a major conflict of interest, being an employee of Inflammatix. I think our clinical study was a rigorously designed and executed scientific experiment. That said, common sense suggests that independent external studies ought to be commended, due to their independence. Of course, it is important to assess their own conflicts of interest, and their own rigour.

Your own paper states plainly that interventional studies are still needed to demonstrate clinical benefit, and that the antibiotic stewardship analyses were exploratory and post hoc. What trial would convince you that TriVerity improves outcomes rather than discrimination – and what, realistically, stands in the way of running it?

As we know, the RCT is the gold standard of evidence. Once you have rigorous RCT data showing improved outcomes, there is really little doubt. So that would be the ultimately convincing proof. Unfortunately, running RCTs on diagnostic tests is challenging in practice, due to the economics of diagnostic test development. Diagnostic tests generally do not have as high a return on capital as drugs do. You can raise capital for drug development with the assumption that it will have to go through an RCT; I am not sure you can do this for diagnostic test development, given their dramatically lower price point. So in a word, what stands in the way is cost, time and the complexity of RCTs. As a result, most diagnostic vendors rely on indirect and incremental evidence to gradually build a customer base. Those that eventually run an RCT, like Oncotype DX, may achieve stellar commercial success, but the RCT is not the exclusive way to adoption.

Diagnostic error and antimicrobial overuse are both long-standing patient safety problems. Where do you think host-response testing genuinely moves the needle on them, and where do you think the field is currently claiming more than the evidence supports?

Our studies consistently show that the TriVerity test outperforms standard labs like PCT and CRP. That should move the needle on diagnostic accuracy. Antibiotic overuse is a complex topic that needs careful consideration. For example, if the recommended use of antibiotics is more accurately targeted – that is, towards truly bacterial infections – it does not have to be reduced in absolute numbers. The evidence generation is ongoing.

You recently presented work on LLM adjudication of infectious diseases at the Nature AI in Medicine conference in Paris. Without going into unpublished specifics, what is the general problem you are trying to solve – and what would have to be true before a language model could be trusted to adjudicate a clinical endpoint?

Development of AI-based clinical tests using supervised learning requires the clinical truth – that is, a reliable class label, for example bacterial infection, viral infection, co-infection, fungal infection or no infection. In the absence of a gold standard for infection, we rely on human expert adjudication, basically a judgement call. Getting adjudications in this way is relatively expensive but, most importantly, slow. So the problem we are solving is improving productivity and reducing the time and cost of developing machine learning classifiers for infections, by automating the adjudication process.

At Inflammatix, we already have a good degree of trust in the ability of LLMs to judge clinical endpoints. But this is based, to a degree, on internal data that others have no access to at this time. An ideal scenario, in my mind, would be: develop a clinical diagnostic test using training data adjudicated by humans; develop an equivalent test using training data adjudicated by an LLM; apply both tests to an independent, previously unseen dataset, presumably adjudicated using human experts; and compare the relevant performance statistics. If the LLM wins, it can be trusted with adjudicating clinical endpoints.

One could, of course, contemplate an even more radical approach: compare the classifiers in a randomized trial, where the control arm uses the human-adjudication-based test and the treatment arm uses the LLM-adjudication-based test. But that is really challenging, and I think the above is a reasonable alternative.

Looking five years out, what would tell you that machine learning diagnostics had actually matured as a field rather than simply proliferated?

In my mind, the ultimate metric is clinical guidelines, such as NCCN for cancer in the US. An ML diagnostic that is in the relevant guidelines is the standard of care. The more ML diagnostics in the guidelines, the more mature the field is.

—

Interviewer: Prof. Atanas G. Atanasov

—

Ljubomir Buturovic is Vice President of Machine Learning at Inflammatix, where he has led development of the algorithms behind the TriVerity test system from 2017 through FDA clearance in January 2025 (K241676) and its clinical validation in Nature Medicine. He was previously Chief Scientist at Pathwork Diagnostics, where he led predictive algorithm development for two FDA-cleared cancer genomics tests including the Tissue of Origin test, and Bioinformatics Director at Incyte Corporation. He held an adjunct faculty post in computer science at San Francisco State University from 2005 to 2019, supervising master’s students. He holds a doctorate in machine learning from the University of Belgrade and completed postdoctoral training at Boston University’s BioMolecular Engineering Research Center.