Validation Institute reviewed Predilytics’ Analytic Design Plan (ADP) for a health plan, which outlined:
· the objectives of the Medication Adherence project,
· the method for achieving those objectives, and
· the statistical and quantitative metrics that are used to document model validation.
Validation Institute reviewed Predilytics’ program evaluation which described how the models were used in a health plan’s operational call center to improve medication adherence for their members.
Predilytics’ evaluation of the predictive models and the operational recommendations reflect a well-grounded understanding of how predictive models perform and accurate assessment of the value of the insights created by these models to inform resource deployment within healthcare operations.
Within that program evaluation, Predilytics used additional statistical analysis to isolate the incremental performance of the predictive model above and beyond call center efficiency gains. Overstatement of attribution is commonplace in predictive modeling in population health, but the model was very carefully designed not to overstate attribution of the Predilytics’ intervention to the outcomes. The one aspect of this study in which overstatement might have been taking place is that the customer service representatives making the outbound calls got better at calling members over the course of the study. As a result, calls towards the end of the period were more successful. Predilytics pointed out this potential confounder to the validator and estimated the impact of it, rather than wait for the validator to point this out to them.
Consequently, the Validation Institute is able to provide validation for Predilytics, as they are well above the standard of using a methodology that is equivalent to or better than the most valid standard commonly used in their segment of the industry.