The efficacy of Mental Health Apps

In January, PLOS Digital Health published a study which claimed that there’s “sparse” data to support the efficacy of most mental health apps.

In their meta-analysis of published studies, the authors found universal deficiencies and concluded that there’s no convincing evidence that app-based interventions deliver meaningful outcomes. Without digging further, one might think the current case for digital therapeutics is weak.

I reviewed all 14 meta-analyses included in the study and found the analysis to be accurate, but incomplete.

These analyses focused exclusively on wellness and smoking cessation apps, bypassing a category of digital therapeutics regulated by the FDA that make scientifically validated claims using sound research.

The Missing Context: Wellness Apps vs. Digital Therapeutics

There are approximately 20,000 mental health apps and the overwhelming majority are considered “wellness” products. The FDA views wellness apps as akin to over-the-counter homeopathic treatments—products with minimal risks and low potency – and they are unregulated. As few as 2% have peer-reviewed research to back up their claims.

What consumers, payers and physicians may not know is that there are two subsets of digital health applications that are considered digital therapeutics (DTx) and are regulated as “software as a medical device” (SaMD).

Non-prescription DTx (NPDT) are regulated as SaMD under the FDA’s “enforcement discretion” guidelines. While these NPDT products do not require FDA clearance, they must substantiate their claims with rigorous, clinical research.

Prescription DTx (PDTs), however, follow traditional FDA pathways for clearance to treat specific and diagnosed medical conditions. These products require clinical trials to demonstrate their safety and efficacy before they can be prescribed by a licensed healthcare professional.

Wellness apps comprise the majority of what’s on the market, which is likely why the meta-analyses included in the PLOS article focused so heavily on these types of apps. But, the bar for research and clinical outcomes is much lower for wellness products than it is for PDTs and NPDTs.

A meta-analysis focused on DTx products, like Big Health’s Sleepio, Pear’s reSET, Orexo’s Deprexis, and Happify Health’s Happify, would reach different conclusions.

Consistent Standards Matter

It’s unrealistic to expect the FDA to regulate all mental health apps, which is why it’s important the field adopts its own standards.

In 2017, pioneering DTx companies created the Digital Therapeutics Alliance (DTA), a 501(c)(6) non-profit trade association focused on providing patients, clinicians, payors, and policymakers with the necessary tools to evaluate and utilize PDTs and NPDTs.

One of their primary initiatives was to develop core principles to which all DTx companies should adhere. These include making claims that are appropriate for the level of clinical evaluation, and following required regulatory processes to support product claims. More than 40 companies have joined the DTA and agreed to these principles.

When you look at the research being conducted by leading DTA members, the evidence for digital therapeutics is far more convincing. Almost none of the 145 trials included in the meta-analysis published in PLOS Digital Health included studies from DTA members.

Three Guiding Principles for Digital Therapeutics Research

The DTx industry has yet to adopt industry-wide standards for research. In the absence of regulatory guidelines, payers and clinicians in search of quality digital therapeutics should look for companies that follow the DTA’s 10 guiding principles and follow these three guidelines:

First, studies should be sufficiently powered. Some patient populations are difficult to recruit for clinical research, so large sample sizes are not always possible. However, high-powered studies are more likely to provide accurate estimates of effects and lead to replicable findings; small samples have a higher risk of false positives. High-powered studies demonstrating statistical and clinical efficacy are, therefore, essential.

Second, studies should include long-term follow-ups. Studies focusing on the immediate outcomes following a brief intervention with a digital health product could be tolerated in the early days of DTx. But as we mature, it’s critical to follow users for six to 12 months to assess the stability of effects. Companies that have gone to the FDA for clearance of PDTs have already done this, but it should become standard practice.

Finally, and perhaps most importantly, the gold standard should be to use active controls. Too many studies in the DTx space fail to include a robust (or any) control group. In some instances, when testing a product vs. lack of care, a “waitlist” control is acceptable. But by and large, DTx should be tested against active control groups, using robust digital shams to control for digital placebo effects.

Given continued stigma around mental illness and the barriers to accessing care, mental health apps aren’t going anywhere. Knowing that, any analysis of DTx should clearly differentiate between wellness products and DTx. We also need to encourage companies to adopt stricter research standards.

About the author 

Eliane Boucher, Ph.D., is senior director of research strategy for Happify Health, The Intelligent Healing Company™ that shortens the distance between need and care by configuring personalised digital therapeutics and care solutions at scale for the modern healthcare cloud.