Adopting AI platforms: A structured framework for thoughtful platform evaluation

The adoption of AI platforms, particularly in sectors such as drug discovery and healthcare, is a complex and critical process. The potential of AI to revolutionise processes, enhance accuracy, and deliver unprecedented efficiencies is undeniable. However, the landscape is changing so rapidly that it is difficult to resist the urge to adopt new technologies simply for the sake of modernisation.

The allure of AI can often overshadow the complexities involved in its evaluation and adoption.

To ensure a successful AI platform adoption, it is essential to carefully evaluate the potential solution, ask the right questions, and consider the broader implications for the organisation. It doesn’t stop at a technical assessment, but needs a more holistic approach to be successful.

While many large biopharma companies have structured evaluation processes, the process is less structured for small companies. This article aims to provide a basic framework to evaluate AI platforms and provide some key considerations to deliberate before adoption.

1. More digital isn’t the motive - more problem-solving is!

Before diving into the technical capabilities of an AI platform, it's crucial to start by understanding the fundamental business problem it is meant to solve. Adopting AI for the sake of innovation alone can often lead to disappointing results because expectations were not defined clearly.

An example of such a failure can be seen with IBM’s Watson for Oncology. It was initially touted as a revolutionary AI tool that could assist healthcare practitioners in diagnosing and treating patients, but the platform struggled to deliver on its promises due to several factors, including a lack of understanding of the complexities of oncology, as well as a lack of real-world applicability and expertise in the domain.

Establishing the understanding how the key stakeholders will benefit from a tool and how will the platform align with the organisation’s strategic goals is crucial. Often, the best indicator is that the demand for a tool or solution is generated by the scientists/clinicians and not by an idea a data scientist had.

The following key questions should be asked:

Establishing demand and problem scope:

What specific problem is the platform trying to solve?
Establish a clear demand from internal stakeholders: Is there a genuine need for this solution, or is it a "nice-to-have", rather than a "must-have"?
Which part of the discovery or development chain will benefit the most from the platform?
- Discovery: Is it scientific, requiring new insights and discoveries, or operational, aimed at streamlining existing processes? Which part of the discovery chain is likely to gain the highest advantage?
- Development: Is it a sponsor/pharma problem? Patient problem? Provider problem? Site problem? Clinical problem?
What are the existing benchmarks, and does the platform promise to surpass them?

2. Drug discovery and healthcare delivery is complex: What proof do they have?

Drug discovery and healthcare delivery are highly complex processes, requiring deep expertise in not only several scientific domains, but also other areas for any project to succeed. It’s not just about the technology itself, but about whether the technology provider understands the unique challenges and requirements like the supply chain, workflow between teams, etc. Often, their background is key raising funds successfully from investors, which can be crucial to survivability of the company.

Lumiata, a San Francisco-based AI company, was focused on building AI to forecast patient outcomes, risk, and cost predictions. However, despite significant initial funding and promising partnerships with healthcare providers, Lumiata struggled to scale its business and achieve sustainable revenue. In 2020, the company quietly shut down operations, leaving its partners, who relied on its AI-driven insights, without a continuation of service.

The following key questions should be asked:

A. Relevant experience and track record:

Do the founders have relevant experience? If so, how many years? What skill sets do key team members bring?
What is their track record in terms of existing partnerships, published work, or demonstrated success?

B. Viability of the company:

Does the partner have cash flow, funding, or a plan to survive the length of the expected relationship?
If and how has the leadership team changed in the company and how does it affect the operations?
Who are the vendors/players with similar tools? What is the unique value proposition?

C. Technical assessment:

Data: Is there enough data to apply ML/DL? Is it public or private data? Is the data clean? Does the partner have the right skill sets to clean the data? Does the partner have proprietary data sets?
Models, benchmarking, and modularity: Does the partner have the right models? Are the benchmarking studies done with a suitable dataset? Does the platform allow integration of your internal models?

Each of these questions can be made more granular based on the application being evaluated.

3. How do I make it work?

Even the most advanced AI platforms can fall short if they don't fit well within the organisation’s existing workflows and disrupt existing patterns by a significant degree. Internal teams should understand not only technical benefits in the scientific problem, but also how the AI platform would integrate into the operations, IT infrastructure, regulatory, and security requirements. A lack in any one of the areas can be a cause of failure.

Proteus, a digital health company that developed a digital pill to track patient adherence, had great potential and high-profile partnerships with both health systems and pharma companies. Once valued at over a $1 billion, the company failed to demonstrate clear regulatory hurdles and filed for bankruptcy.

The following key questions should be asked:

A. Establishing fit:

Does your organisation have the internal skill sets necessary to evaluate and integrate the platform?
Does your organisation have additional tools/data that may improve the solution?
Can the platform be easily adopted within existing workflows, or will it require significant changes?

B. Adoption structure:

Is it simple enough to adopt as an SaaS after initial training? Does the partner provide implementation services?
Would an iterative approach be needed for the technology to be successful?
How to design a successful and fast POC with enough stage gates?

C. Partnership structure:

How can the partnership be structured? Who is responsible for the deployment, regulatory compliance, life cycle support, etc.?
In case of the partner’s bankruptcy/failure, who maintains the platform?
How can the partnership be structured to ensure that both parties are aligned on goals, responsibilities, and long-term sustainability?

D. Regulatory:

What are the regulatory implications? GCP? HIPAA?

E. Infrastructure and security:

What IT infrastructure is going to be needed? Would the data stay on our cloud/premise or be transferred to the partner’s ecosystem? Is the system secure? Is this scalable?
When would the partner delete data from the cloud?

F. Intellectual property:

Is the innovation patentable?
Who owns the IP generated through the platform?

4. Value that can be measured

Ultimately, the success of any project will be measured by the value it delivers, be it an AI platform. This value should be tangible and measurable, whether in the form of cost savings, improved efficiency, enhanced accuracy, or other key performance indicators.

Understanding the key resources that will be deployed, expected time spent on project, key challenges that you may encounter, and how to model that into a measurable sheet is critical to justifying the adoption of the platform.

The following key questions should be asked:

Return on investment:

Is it an out-of-the-box solution or will need testing before providing the desirable outcome?
How to model the ROI?
What internal skill sets can improve ROI

Adopting an AI platform involves careful consideration of the problem at hand, a thorough evaluation of the platform and the company behind it, and a clear understanding of the operational and regulatory implications. By asking the right questions and taking a thoughtful approach, organisations can ensure that their investment in AI delivers tangible benefits and drives real problem-solving, rather than simply adding another digital tool to their arsenal.