Al Lewis
By way of background, there are two things you need to know before launching into this Valid Points article:
- Livongo, a venture-capital funded diabetes management company, recently published a study in the Journal of Medical Economics alleging a significant savings in diabetes-related spending for active participants in its program. Its program features free glucometers, unlimited strips, remote monitoring of results, and coaching.
- Validation Institute offers standard and advanced training in outcomes report analysis. Those who pass earn a Certified Outcomes Report Analysis (CORA) credential. Following advanced training, candidates may be designated as a “CORA Pro.”
This article shows how a CORA Pro might question the Livongo claims.
SPOILER ALERT: CORA Pros often question findings in peer-reviewed journals. By contrast, most people in this industry embrace peer-reviewed findings (except in wellness, when myriad findings, peer-reviewed and otherwise, clearly disprove cherished beliefs). This study is no exception. Benefits consultants and jumbo employers are signing up for Livongo in large numbers.
And yet this study raises some serious questions that the latter groups appear to be overlooking.
Questions about possible investigator or publication bias
Why did Livongo publish in a journal with an “impact factor” of 2, as compared to 47 for JAMA and 73 for New England Journal of Medicine?
Could the fact that every author on this study works for or consults to Livongo or Eli Lilly create an investigator bias? What would happen to those employees’ careers if they had concluded that their employer’s product didn’t work?
Questions about Livongo’s own interpretation of the study
Likely due to peer review, the article draws no cause-and-effect relationship between the Livongo intervention and the results. The headline itself says that reduced medical spending is “associated with” their product. Later the authors say the results “imply” the product works. Then why does Livongo’s press release announce:
The findings showed that by using its remote digital health platform, the Livongo for Diabetes program delivered an $88 per member monthly reduction. (Actually, the alleged savings is only $20 after program fees, before taking into account administrative costs.)
Next, a CORA Pro would ask how these alleged findings compare to Livongo’s initial claim below, featured in a recent Valid Points posting about diabetes vendors snookering purchasers:
How did this 59% reduction claim (which basically requires eliminating every hospitalization not connected with births, trauma or cancer) get replaced by their current claim that large reductions in physician visits generate all the savings, while admissions increased? Seems curious that physicians would be doing more work – more notifications from remote monitoring devices about blood sugar, more titrating dosages – and be perfectly fine making 26% less money.
Actually, much less money than that. The 26% was the reduction in cost from all outpatient visits in total. Clearly, visits not associated with diabetes would be unaffected by Livongo. If those visits constitute half of the total, then the diabetes-related visits fell a whopping 52%. Almost the opposite of Livongo’s initial claim that inpatient spending plummeted >50% while outpatient didn’t change.
Questions about the authors’ basic understanding how to design a diabetes study
If someone were to ask a CORA Pro to name endpoints in a diabetes study, the reply would be: “The intermediate outcome would be reduction in insulin usage across the population and the final outcome would be reduction in hospital admissions primary-coded or even secondary-coded to diabetes.”
Why, then, did neither outcome get measured? Many other things got measured. Is it possible the researchers measured insulin use across the population, or even just among participants, but didn’t like the way the answer turned out and hence didn’t publish it? Nah. That would have been dishonest. The diabetes company employees probably just forgot about insulin. Hey, an honest mistake. Could happen to anyone.
Why did they measure participants against non-participants, when that study design is known to be completely invalid? Benchmarking a par-vs-non-par result has been done five times, including three times by wellness promoters hoping to prove validity of the design The conclusion in each case: 100% of difference in outcomes between the two groups is attributable to the study design, and 0% to the intervention. The design is 100% invalid.
Further, more than 15% of the initial Livongo participants dropped out. Assuming dropouts largely failed, aren’t the authors overstating the outcomes for the participant group as a whole by not counting dropouts?
Questions about the authors’ basic understanding of diabetes
Livongo offers “free unlimited glucose test strips” to try to reach a “target level of glucose control of Hb A1c <6.5%”.
It is not quite clear that either of those goals is good for diabetics. Why, one might ask, does the American College of Physicians propose 7% to 8% instead of 6.5%? While it is true that the American Diabetes Association is sticking with its much lower blood sugar goal, isn’t it also the case that the ADA is largely funded by companies that make products to help lower blood sugar?
And what is the rationale for encouraging more use of glucose strips while Choosing Wisely recommends less testing for many Type 2 diabetics (in JAMA Internal Medicine, impact factor 20)? Exact words:
This recommendation is based on robust evidence, including a Cochrane review of 12 randomized clinical trials with more than 3000 patients, showing no statistical difference between patients who do not self-monitor their blood glucose multiple times per day and those who do self-monitor their blood glucose multiple times per day in glycemic control, nor evidence of effects on health-related quality of life, patient satisfaction, or decreased number of hypoglycemic episodes.
What is Livongo’s objection to “robust evidence”? Further, as one commenter pointed out:
Unnecessary testing not only is costly but it leads to bad management. Many doctors advise patients to monitor their sugar levels using a glucometer. Common advice is to increase or decrease insulin by 2-4 units if the blood glucose levels are ‘high’ or ‘low’ respectively. There are so many factors which make the glucose level fluctuate. Reacting to each and every change is not advisable. Many patients develop hypoglycemia due to this practice. That is dangerous.
It should always be the doctor to make the decision about titrating the insulin dose.
And that leads us to the final question. With all the strips and testing that Livongo wants diabetics to do to capture the “many factors that make glucose levels fluctuate,” how can the doctor “make the decision about titrating the insulin dose” when they are 52% less involved in care?
Just askin’… Because that’s what CORA Pros do.
Tweet
