AlphaFold - better mental models of AI progress

Understanding ‘AlphaFold’ is a good way to help make sense of the non-linear trajectory of AI progress.

Bad mental models of AI progress

Understanding ‘AlphaFold’ is a good way to help make sense of the non-linear trajectory of AI progress.

We’re heading somewhere strange, no doubt. A non-linear trajectory toward AGI and ASI - artificial general and then superhuman intelligence. We have no idea how or when we get there.

I think most people, myself included, have a bad mental model of AI progress. Maybe it's sci-fi, or maybe it's chatbots to blame? I’m not sure. For most of us, the main interface through which we experience AI, is chat. So "better AI" means "general assistant that will one day be AGI". We tend to focus more on the immediate short-term utility, and less on the non-obvious, domain specific breakthroughs along the way.

There’s some consensus that the current paradigm of language models, next token/word prediction isn’t the final stop.

More importantly - in addition to new ideas and new research on this very non-linear path, we’ll have other ‘less-immediately-obvious’ breakthroughs. And those things will completely change and influence the ‘how’ and the ‘when’ to agi.

"I think the really, really interesting point is where we can characterize these systems as reliable enough? Do we find useful things for them to do? I think we need to be much more kind of utilitarian about it. And certainly, machines like AlphaFold, I wouldn't necessarily apply the word, think."

After watching the new DeepMind documentary, and listening to interviews with John Jumper, I’d argue AlphaFold is, by far, the best example to illustrate and improve one’s understanding of this non-linearity, and the more interesting and impactful things we might unlock along the way.

AlphaFold and Proteins 101

AlphaFold is an AI system that takes the amino-acid sequence of a protein and predicts the 3D shape that protein folds into. If, like me, it wasn’t immediately obvious why predicting protein structure is such a big deal, I’ll try to explain as simply as possible. Every organism and biological system on Earth uses and depends on proteins. They are tiny molecular machines that do almost everything inside you. Enzymes that digest your food. Antibodies that fight infection. Hemoglobin carrying oxygen through your blood. Collagen holding your skin together.

We are, operationally, protein machines. The 3D shape of a protein determines what it does, so if you can predict the shape, you can understand how life works and design ways to fix it when it goes wrong. We’re skipping over 99% of the details here, but the high level puzzle pieces are:

  • Your entire meat-sack contains roughly 37 trillion cells
  • Almost every cell stores a 3-billion–letter instruction manual (your DNA)
  • Scattered across that DNA are about 20,000 protein-coding recipes (genes)
  • Genes are transcribed into mRNA, a working copy of the recipe
  • The mRNA is translated by ribosomes into amino acid chains
  • Those amino acid chains fold into 3D shapes, becoming proteins, the machines that do nearly everything in your body
  • Cells, DNA, mRNA, ribosomes - this is the pipeline which produces the amino-acid chain, that chain folds into a very specific shape (the protein structures) - those protein structures are the functional machinery of life - their shape determines how life works.

The key point here is that the amino acids aren't useful until they fold into a 3D shape. It's the shape that determines what the protein does. We often know a protein's amino acid sequence - ie, the parts list. But experimentally figuring out the 3D structure is hard, slow, and expensive.

The number of possible ways an amino acid chain can fold is astronomically large.

Proteins fold into specific shapes

The hemoglobin, Nobel Prize story

Hemoglobin is the protein in your red blood cells that picks up oxygen in your lungs and delivers it to the rest of your body. In the 1950s, we knew hemoglobin's amino acid sequence, but the sequence alone couldn't tell us how it actually carried oxygen.

In 1959, after more than 20 years of work, Max Perutz solved hemoglobin's 3D shape using X-ray crystallography. It was the first time anyone had determined a full protein structure. He won the Nobel Prize for it.

Max Perutz solved hemoglobin’s structure With that blueprint, we could finally understand how hemoglobin works, how mutations break it, and how to intervene at a molecular level (disease diagnosis, drug development, molecular medicine).

Hundreds of millions of years work in a few months

Before AlphaFold, we had about 180,000 experimentally solved structures - the result of about six decades of work. Post AlphaFold, we have more than 200 million predicted proteins - Essentially the entire known protein universe is now “structurally charted”.

This would have taken us hundreds of millions of years to solve experimentally. AlphaFold did it in months.

Video of demis releasing. Over 3 million researchers in 190 countries now use the database. More than a million of those are in low- and middle-income countries. It's been cited in over 35,000 papers. It won the Nobel Prize in Chemistry.

Saving honeybees and better understanding infertility

The implications are real - drug discovery, understanding disease, designing new proteins, understanding evolution. Here are two real and recent examples.

Helping Honeybee populations - Breeding & Disease Resistance

There's a protein in honeybees called Vitellogenin. It plays a central role in bee health - immunity, stress resistance, feeding offspring, colony resilience. For a long time, biologists knew its amino acid sequence but had no reliable 3D structure. In 2021, a researcher named Vilde Leipart used AlphaFold to predict Vitellogenin's structure. "It took me two days," she said, "to do something that could have taken me years." DeepMind write-up With that structure, scientists could now see which genetic variants of the protein give better resilience or immunity, before bees are even bred. This would normally require years of trial-and-error breeding. This could help prevent colony collapse, safeguard pollination, improve crop yields etc. And because many egg-laying species use similar proteins - fish, poultry, amphibians, reptiles, these insights might help protect other vulnerable species too.

Infertility in humans

We've known for a while that certain proteins help sperm and egg bind together. But fertilization has remained largely mysterious. Using AlphaFold, researchers screened thousands of proteins and found one called Tmem81 that no one had linked to fertilization before. It forms a complex with two known proteins, and together they act as the "key" that unlocks the egg. They tested it: fish and mice without Tmem81 were infertile. Sperm couldn't bind. Now we have a molecular picture of how fertilization works. That opens doors—diagnosing infertility, new treatments, maybe even new contraceptives. Read more here.

Better mental models

It's important to emphasise the ‘messiness and complexity of biology’ - John Jumper and others who worked on AlphaFold continually do this. AlphaFold is only a small part of solving the larger puzzle of understanding life and curing disease. Regardless, this more utilitarian view seems like a better model for how AI might change the course of things, and a far more valuable space to invest time and effort.