Highest Score at the CASP competition
Who
AlphaFold
What
92.4 total number
Where
Not Applicable ()
When

The highest score achieved at the Critical Assessment of protein Structure Prediction (CASP) competition is 92.4 GDT, achieved by AlphaFold – a system created by British AI lab DeepMind – at CASP14 (which ran from May to September 2020). The results were announced by the Protein Structure Prediction Centre on 30 November 2020.


A protein is a chain of different types of amino acids. Each unique combination of these building blocks causes the chain to twist and fold back in on itself in a different way, creating complex three dimensional shapes. The shape of each protein plays a crucial role in how it interacts with the environment and fulfils its function.

Figuring out the chemical composition of a protein (which amino acids it uses, and in what order) is relatively easy, but without knowing its shape, it is hard to predict what a protein does, or how its action might be influenced or modified. It is possible to figure out a protein's shape through direct observation using techniques such as cryo-electron microscopy, nuclear magnetic resonance and x-ray crystallography, but these techniques are expensive and slow. It can take hundreds of thousands of dollars and months, even years of testing to nail down a definitive answer.

If an AI based system can be made to reliably predict the structure of proteins, it would great accelerate scientific research and the production of new medicines. It would also open up the possibility of engineering artificial proteins to perform specific functions (leeching pollution from groundwater, breaking down plastics).

The CASP competition has been held every two years since 1994. Each event challenges interdisciplinary teams of researchers – including biologists, chemists and computer scientists – to try to predict the structure of a set of proteins. The success or failure of each team's predictions are quantified using a measure called Global Distance Test, or GDT. This is essentially a measure of the physical similarity between the structure in the model and the structure of the protein as verified through experimental techniques.

A score of zero would be a prediction that looked nothing like the real structure, while a score of 100 would be exactly the same. A score above 90 GDT means that the prediction is accurate enough to be within the margins of error involved in experimental methods – around 0.16 nanometres – and that it's close enough to correct to be medically useful.

At CASP1 in 1994, the highest overall median score was just 47 GDT. It had climbed to around 60 GDT by CASP5 (2002), but then stagnated around there. Work on AlphaFold started in 2016, and its first entry into the competition, at CASP13 in 2018, raised the winning score by more than 10 percent, the largest advance in CASP history. In 2020 AlphaFold achieved an overall median score of 92.4 GDT, and an impressive 87.0 GDT in most challenging Free Modelling category, which includes the hardest and most complex proteins.

The AlphaFold system takes a few weeks to determine an answer running on a bank of 16 TPUs (Tensor Processing Units). TPUs are processors with a similar architecture to the GPUs (Graphical Processing Units) in modern videogames consoles and PCs, just larger and optimised for AI applications. The bank of 16 TPUs is roughly equivalent to 100-200 commercially-available GPUs. The deep-learning AI was trained using the basic principles of protein structure, which have been figured out by human scientists, as well as a dataset of around 170,000 previously verified protein models.