24 October 2023
Scaling compute in the age of AI + taking inspiration from nature
Nature Computes Better Programme Director Suraj Bramhavar on developing his programme thesis.
You’ve joined ARIA with an emerging thesis about the potential to harness stochasticity to improve AI hardware infrastructure. Can you talk us through how you got to this insight?
Before ARIA, I was already part of a community exploring new ways of doing computing. I’d been exposed to macro-economic and geo-political issues emerging around the future of computing, and understood that progress is going to be harder to come by as time goes on. These trends coincided with the explosive growth of AI algorithms which: 1) increasingly utilise noise and other natural features as a computational aid, 2) are often geared towards mimicking functions which nature has proven adept at performing, and 3) exhibit strong enough commercial demand to warrant the exploration of high-risk ideas and the cost to pull them out of the research laboratory. The confluence of these ideas helped form the kernel of the idea underpinning my programme area.
As an incoming ARIA Programme Director, through a combination of talking to the ARIA internal team and my conversations with the other Programme Directors, I was encouraged to step back, look at the broader landscape of activities, and be more open-minded to new concepts than ever before.
One of the core criteria for an ARIA ‘opportunity space’ is the potential to lead to significant new capability for society. Why is more efficient computation particularly important and relevant for the opportunities and challenges we face as a society?
There has been a very clear vector of progress for computing for fifty years or so that has taken computers from taking up whole warehouses to sitting in our pockets, which is great. But as I mentioned earlier, that exponential improvement is coming to an end.
The biggest issue with that happening is the AI algorithms that have come round in the last 10 or 15 years have shown us that increasingly impressive capabilities arise with increasing computing power. It’s unclear what those capabilities will be in the future. But based on recent progress, it’s clear there is societal value that relies fundamentally on improving computing potential power over time. Examples include our ability to better understand how proteins fold, automatically generate computer code, build self-driving cars, and more. All of these disciplines benefit significantly from increasing computing power.
One of your fellow Programme Directors, Davidad, is worried about the risks of increasing computational power for AI. Is the programme he’s developing in conflict with yours?
I think it’s a great thing that another Programme Director is looking at this important topic from another vantage point. Davidad is interested in using mathematics to formally certify that AI systems are safe to deploy in critical applications, so it’s clear that we have a shared belief in the massive potential for AI for societal benefit. Having both programmes succeed would allow society to ensure that future AI is both safe and transformative.
Let’s dig into some of the bottlenecks that might have limited exploration in this space before. In your published document, you reference less attention being paid to energy-minimisation in physical systems as an extremely efficient computational mechanism. Talk us through why you think that is.
Things around us in nature are performing very complicated processes in order just to function. Those processes aren’t necessarily purely quantum mechanical (but could benefit significantly from our understanding of quantum sciences), and they’re definitely not purely digital.
There is a community of scientists and engineers that have recognised the intense computation that nature does with very low energy, deftly dealing with the inherent imprecision. We try to emulate this using pure ones and zeros, which has allowed us to build massively scalable commercial products, but it’s incredibly resource intensive. There exists an under-explored technical space between these two worlds and an under-appreciated community of researchers blurring the lines between quantum information processing, nonlinear dynamics, neuroscience, and digital computing. As an ARIA Programme Director, what’s so compelling is the idea of lighting a fire under the group of researchers currently thinking about this opportunity space, and seeing what then comes out of it.
Have there been barriers to adopting a more interdisciplinary approach before? Why do you think that is?
The barriers have been a by-product of the computing industry in general and, in some ways, that’s been good for progress. The whole industry relies on abstractions between disciplines, so that each discipline can just focus on their own mission and everything is in place so that everyone can benefit. For example, people tasked with making transistors smaller can just focus on making transistors smaller. Everything else is in place so that when they do, the computers get better and the programmers benefit, without having to know all the details of how the transistors got smaller.
But when progress starts to taper off in some areas, it becomes really important to start breaking down some of those barriers. This requires getting communities of people who haven’t historically interacted to interact. The explosive growth of AI and the strain it has placed on our hardware infrastructure provides us with a unique (and necessary) opportunity to do so.
Who are the kinds of people you want to hear from and who do you think might be able to help inform your programme thesis?
There are a lot of different groups of people I would love to hear from.
The first is people who are most knowledgeable about the existing digital paradigm that is done phenomenally well. I’d want to get those people in rooms with folks with radical ideas to think about how boundaries could theoretically be pushed if roadblocks weren’t in the way.
I also want to get hardware experts talking to algorithmic experts to understand the trade-offs that currently exist and how speaking to each other could change those trade offs to improve things for both sides.
Lastly, I’d like to bring together people from the biology and chemistry world and people from the computer science and electrical engineering world. We’re talking about how nature performs all these processes. Biologists and chemists understand these processes better than anyone, but computer science and electro engineers have their own language for how they describe their processes. If we’re trying to link those two worlds it would be wonderful to bring those communities together to find common language. I think there’s a lot of similarities between those two worlds but they currently speak different languages.
I’m so excited to be in a position at ARIA where I’m thinking about how I can bring all those groups together in a way that leads to a really productive conversation.
So, what’s next?
I’m formulating my programme thesis based on the input I’ve been receiving and plan to publish that for feedback and further engagement in the coming weeks. In the meantime, I’m open to connecting to anyone that thinks that this is a compelling place to put their energy. I encourage people to reach out on my form online or submit a document. I’m committed to reading anything you send and would love to know what compelled you to this journey of thinking.