From 0.1% to infinity: How increasingly larger GPT models are transforming the future of drug discovery in the uncharted chemical universe

90% of drugs on the market are small molecule drugs, and many of them play crucial roles in ensuring our health. However, the chemical drug discovery field is facing an unparalleled challenge: the need to explore a molecular space so vast it defies comprehension.

To date, estimates suggest that less than 0.1% of all small molecules that could possibly exist have been synthesised for potential drugs, or even considered. That means that there are likely many new drugs that could treat an enormous number of diseases and conditions – vastly improving our health in untold ways – waiting to be discovered.

This situation underscores both the current limitations of drug discovery and the immense potential that remains untapped. It speaks to the need for a leap in innovation – one that moves beyond the conventional, incremental approaches of today.

Leveraging GPT for science and drug discovery

One way to tackle this challenge is with a technology that has revolutionised all of our lives in the last few years: GPT. GPT, an acronym for generative pretrained transformer architecture, is an AI machine learning model trained on language that, when prompted, can generate in real time, combinations of words and sentences, not simply repeating what it was trained on; similar to how a human speaks. OpenAI’s Chat GPT and Google’s Gemini are, of course, famous for this, churning out everything from answers to questions to recipes to college essays – and these are just two examples of the many AIs built on GPT architecture.

GPT technology can be leveraged for science and drug discovery as well, and has indeed been used in this manner in the last couple of years. GPT models trained on chemical data – translated into a letters-based language – can be prompted to suggest new molecules that meet certain characteristics. For example, a scientist can request a new molecule that is water soluble, safe to ingest, and interacts with certain enzymes. Then, the results can be tweaked by asking for more qualities. More traditional approaches require a tedious manual process, with testing at every step of the way. Each change often sends the researcher back to the beginning, as tweaking one parameter often “ruins” another, much like when solving a Rubik’s cube, fixing the colour on one side introduces a mistake elsewhere.

Using GPT, rather than these manual methods, can drastically increase the probability of success in the process of finding new molecules that can be useful for drugs, laying the groundwork for potentially addressing unmet medical needs at far faster speed and lower cost. However, GPT has not proved to be a magic overnight solution. Even as GPT tech, trained on a few million molecules, is integrated into the drug discovery process, there is still a lack or a shortfall of being able to generate truly unique molecules with the desired characteristics, due to the fact that the training set usually includes a few million molecules at best, still a fraction of those that could exist in the world. For context, the molecular space contains between 1060 to 10100 potential small molecules, dwarfing the total number of atoms in the observable universe.

Revolutionising structural biology

Recent developments can take this core innovation of GPT to new levels with the adoption of foundation models, which are models trained on even larger datasets, a development made possible by the continuous expansion of computing power. With foundation models, there is even more potential for AI technology to change molecule discovery and pharma innovation.

A prominent example of a foundation model is DeepMind's AlphaFold, which has revolutionised structural biology by accurately predicting protein structures. This capability, achieved by processing millions of parameters, significantly surpasses the scale and accuracy of previous AI models designed for this purpose. AlphaFold's impact is profound, having identified numerous potential drug targets, including those for antipsychotic medications, liver cancer treatments, and therapies for rare parasitic infections. Its widespread adoption in cancer research and other fields underscores its transformative potential. In recognition of this groundbreaking work, Demis Hassabis and John Jumper of Google DeepMind, the developers of AlphaFold, were jointly awarded the 2024 Nobel Prize in Chemistry, along with David Baker of the University of Washington, for his work on computational protein design. This prestigious award highlights the significant advancements in protein research made possible by AI. Industry leaders anticipate that AlphaFold's applications will continue to expand, further accelerating biological and medical research.

When it comes to drug discovery, foundation models are also a potential gamechanger, as they can train on billions of, rather than just a few million, molecules and their parameters. This vast trove of training data allows for truly novel creations, as the model’s scope includes so much more of the potential molecular world, in the blink of an eye. First and foremost, for patients and healthcare providers foundation models are significant in that they will likely lead to better treatments.

One of the modalities that can benefit from this technology are small molecule drugs, based on chemical entities, which make the vast majority of drugs on the market. Due to their low molecular weights, these drugs can enter cells easily, and interact with other molecules, including proteins, targeting a wide range of diseases and medical conditions. Foundation models can be especially promising in finding the formulas to design new small molecule drugs that can enter the cells needed, reach the correct targets, and potentially deliver cures and treatments, also allowing for the discovery of drugs for rare or untreatable diseases. For those widespread conditions that already have many treatments, but for which those treatments are expensive, uncomfortable, likely to fail or only make an incremental difference – like cancer and diabetes, finding novel small molecule solutions for targets is also critical to the future of healthcare.

For pharma companies, these models offer a way to truly advance science and health while also efficiently and effectively developing new molecules that are unique and thus can be submitted as a new composition-of-matter patent, adding a surge of motivation to the industry. Even with the advancements of AI technologies, pharma companies continue to struggle with herding and “me-too” drugs, which is the tendency for many drug developers to seek the same targets and same candidates, which are known to be somewhat effective, as an economic and practical way to add incremental value to treatments or to make a similar competing treatment for a well-defined market. Foundation models offer an even more promising vehicle to reduce those trends, and develop treatments that are a leap, rather than just a step above existing drugs. This comes mainly because foundation models, in containing much more data, offer an efficient and less risky way to look for novel targets and drug candidates. Critically, foundation models, due to their size, reduce the known AI use risk of hallucinations, producing more reliable results and ideas for new molecules.

Beyond the molecule discovery stage, foundation models are starting to make transformative inroads in other areas of drug design and healthcare. For example, researchers are already using them to better analyse complex biomedical data, predict drug efficacy and safety, genomic sequencing, microscopy, and better understand the link between specific targets and diseases. Language-based foundation models for small molecule research could open up the pharmaceutical industry, providing pharma firms with huge amounts of data about many conditions and diseases that are currently underserved or have no treatment at all.

Drug discovery, and science in general, are fields built on data. Foundation models bring new capabilities by integrating vast amounts of data via GPT or other architectures, allowing for insights humans cannot fathom, and paving the way for breakthroughs in medicine that will truly transform lives and the future of humanity.

About the author

Dr Ruth Gross is vice president of business development at Evogene, a veteran computational biology company. Dr Gross leverages her 10+ years of expertise in medical, marketing, strategic partnerships, and pipeline development to drive growth in biotech and life sciences.