Why the Brain Computes 1,000,000x More Efficiently Than A GPU: Unconventional AI's Naveen Rao

you We're going to jump into our next set of Frontier Talks. The first is from Naveen Rao. Naveen is a pioneer in the AI space. He did his PhD in neuroscience. Then he started one of the first ever AI chip companies way before it was cool. Um, he's actually the person who ran, uh, Mosaic ML, one of the first AI training companies and then built all of Databricks AI, He left that amazing role to do something new and redefine the future of computing, bringing Kuhl back to neuroscience. Naveen, come on up.

Thank you. Naveen Rao: All right. Good afternoon, everyone. Yeah, super excited to be here. I'm Naveen Rao, CEO of Unconventional AI. So we are unconventional because maybe it's actually the wrong word to use because I think we're going to have to change the name to conventional. It's actually a great time to be a startup, as Boris alluded to. Like having no baggage actually I think is a true competitive advantage. We can do things so much faster than traditional chip companies and full stack companies can do. And I think that's what's very exciting.

get to tape outs in months instead of years, things like this. So anyway, let's get started. I put that up there. I can guarantee you some of you are trying to prove me wrong. All of a sudden you are like, oh, my God, I can hear the years turning. That can't be true. Well, let me explain my logic a little bit. and maybe the definition of what I call ASI. Really, I think we need to get to a much greater amount of compute efficiency. And when I say compute efficiency, I don't mean algorithmic compute efficiency or data efficiency.

There's lots of people working on these problems. I actually mean the fundamental substrate. Actually how I do information processing at the physics level. We chose a path of making computers work the way they did for a lot of reasons about 80 years ago. If you think about it, like in the tech industry, how many things have existed for 80 years, like not a lot. The digital abstraction, sort of floating point numbers, those were around from the 1940s. And for a machine that was built from a Completely different substrate. and for a machine that was built for a completely different purpose.

And now we're building machines for intelligence. So let's think through this a little bit. So, Part of this is that AI will make us more efficient. We talked about coding and running 1,000 agents on your phone, all this kind of stuff. It does make us more efficient. But at some point, we start to get to a place where what does efficiency really mean? If you think about energy-- actual energy, maybe you're actually not more efficient. And we're kind of butting up against those limitations of the physical world now. you know Already today, we're using many gigawatts for AI inference and training.

And we're going to get to a point within the next couple of years. This is not 10 years away. two, three, four years. where we just don't have any more energy in the world for AI. Then this topic starts to become very, very important. Right now you can kind of look at it like, well, Electrical energy versus food. These are two energy sources for intelligence. And there's no real limitation on the electrical side, but that's going to hit a pretty solid wall very soon. And we're talking about going to space, we're talking about building fusion reactors.

Great. Let's do all those things. But still, these fundamental physics apply. So-- If we think through this a little bit, we have about 8 billion people in the world. Our brains use about 20 watts each. It's only 160 gigawatts. So the entirety of humanity is 160 gigawatts. So just put that into comparison, we have about 9,000 gigawatts of capacity in the world today. The US has about a thousand. And this runs everything. This is like heating in your home and all the things, electric cars, all that kind of stuff. But if we said we maybe got 50% more of this, and we say, OK, great, now we got like-- 4,000 plus gigawatts.

But the problem is our current paradigm of compute is just vastly more inefficient. So a computer, I'm making some numbers up here. I mean, I could say, like, if I'm running inference per token, I can come up with numbers. But fully loaded, like the amount of energy that goes into inference, building the model, running the model, all of that, Let's call it a gigawatt, something like that. definitely in the megawatt range. But humans are on the order of 20 watts. You could also argue that evolution over four billion years has created what we are.

But the reality is today our constraint becomes complex. How quickly can we get learning to happen? How quickly can we build intelligence on a given amount of energy? So if we want to build this future where we have lots of intelligence in the world and we're automating all kinds of things, and we want to really be more efficient from an energy standpoint, we're going to need a lot more watts. Or we can think about building a vastly more power-efficient computer. And that's where we come in. Thank you. So I love this curve here.

And I think most people really haven't thought about this, because you just sort of assume the computer is a computer. And we haven't really questioned that. Again, this is the unconventional part. Let's break that apart. The assumptions we made 80 years ago are actually not quite valid anymore. We just choose to keep building on them because I can build a product in two years. I can make something that I can sell in two years. But we're kind of taking a different tactic. Let's go back to those first principles and see if we can build something much, much better.

So there is a thermodynamic limit to intelligence for what? OK, that means you just can't do any better. There's something called the Landauer Principle, which some of you may know, which basically suggests how much compute could happen within a certain amount of energy. So there is a physical reality that we can't get past. And that's sort of this asymptote here. Now, biology is somewhere up here. It's actually pretty darn efficient. 4 billion years of evolution have created something that is actually very efficient. However, it's not at the asymptote yet. There's probably an order of two of magnitude between those two.

We are actually here, by the way. We are down here. And I think limits of 2D lithography, that's what chips are built on today, call it somewhere down here. And I think... With focused effort, we can get to the point where we're pushing the limit of that. And so we'll have something that's-- This, by the way, is something like three orders of magnitude from where we are. It is very far away from where we could be in terms of energy efficiency. And so really that's what we're focused on today. So how do we do it?

Yeah, this is great. Make something more power efficient and wonderful. But the reality is it's not so simple. We can't be thinking about the computer in exactly the same way as we have been. It's not about a machine that runs matrix math. That's been the simple way to move forward. NVIDIA, of course, has owned that market and continued to push the envelope. But if you look at the power efficiency numbers, actual power efficiency of delivering an FP8 flop, for instance, It's not that much better. Costs have gotten better. Because manufacturing has gotten better, our ability to package has gotten better, but actual energy per flop with memory access has not gotten better.

It's very, very incremental now. Thank you. So I am a neuroscientist. I was a computer architect for 10 years before that. I've been thinking about this problem for a long time, actually on the order of 30 years. So it's a very exciting time for me personally. Biology really does provide an existence proof. I mean, you can argue that, OK, the tokens per second out of a human are lower than the machine. But the intelligence is higher. We still haven't gotten to the point with these gigawatts that we're throwing at it that we're rivaling a human's intelligence in terms of discovery.

We'll get there. We're going to get there in a very short amount of time. But it's going to come at the cost of a lot of energy. So what I think is most interesting here is actually not just that brains are -- human brains are 20 watts, but that The water just scales with the weight. A macaque monkey's brain is probably less than a watt. And actually, you see this all through the mammalian world and also the insect world. Like, you have very complex behavior for milliwatts. Just for reference, your phone in your pocket is about one watt.

So A squirrel jumping from branch to branch is running on less than 10 milliwatts. That's 1/100th of your phone. We can't actually do this perfectly, like squirrels jumping 10 feet across from branch to branch within wind and all that stuff. We can't do that with a much, much larger computer. So biology still created something quite amazing. And I just don't feel like there's been an appreciation for that. So I'm just -- just a little reminder there. Great. We see this kind of phenomenologically. Biology is efficient, can do amazing things, but how does it actually work?

We don't really know, I will be honest as a computer scientist and a neuroscientist, but there are some ideas that we can harvest from neuroscience. And one of them is that-- The brain is dynamic. It does not use matrix math to do compute. It uses what's called nonlinear dynamics to do compute. What this means is that there's a time varying interaction between neurons, and that's actually where the compute lies. So can we extract that and actually apply it to synthetic circuits? Maybe. They don't do floating point math. They don't do matrix math.

They do something that can be characterized as such. But it's actually much richer than that because of these nonlinear dynamics. And they're stochastic. Brain's compute is not a strict one and zero. In a digital computer, if we're off by a one or a zero, the whole system falls apart. They are really not computers. So, I'm going to try to go through this quickly. This is a thing called a Kuramoto synchronization. So if you look at a bunch of oscillators here, and they're kind of rigidly coupled to each other on this plank, you'll see over time that they start off in any state, And then they actually synchronize.

This is an example of a contracting or converging dynamical system. So no matter how you start it, it converges. And it's only based on the coupling between them. We can generalize this to something that has a flexible coupling, call it a trainable coupling between those things. And then it can have all kinds of interesting dynamics. It can move through this state space of dynamics in many, many different ways. So if you generalize this, you can actually think about an electrolyzed and electronic circuit. You can see I have a bunch of oscillators, and they have a fabric on which they're coupled.

And now when I make this fabric trainable, I can actually see something that starts to look a little bit more like the dynamics of the brain. It actually has non-linearities, and they interact with each other in very interesting ways. It's actually very rich and represent a lot of information. This is an actual chip that we're going to be building this summer. So we went from basically no team in January to a full prototype in six months. And that's because of AI. So this is what's probably cool about not having baggage, is you can do things in completely different ways.

And the way you compute with something like this, the traditional way would be you basically loop over some sort of linearized time. This is how we do things in a -- in a -- in a -- in a -- in a -- von Neumann machine. We write state out, we retrieve it, we operate on it, we write it back. We keep going back and forth. It turns out that's what burns most of the energy in an existing computing system. With something with nonlinear dynamics, I actually just say, Here's the initial state, kick it, and let it run.

So the physics themselves basically do this computation. And it does a sort of-- the state is an implicit. It's not an explicit right. So in some ways, you can think about if you take anything from this talk, that we use the time access of the physics. to do computing, and existing computing constructs do not. And so the question then is, can I train this? And the answer is yes. I can actually steer the system into multiple different things. In fact, we've sort of traced out in state space a-- unconventional logo.

That's the idea here. We can train it in a few different ways. So yes, we can train these systems and steer them into basically any arbitrary set of trajectories. And can we connect it to AI problems like image generation? So actually, I have a better version of this I can go to in just a really quick demo here. Let's go to the next one. we could have Yeah. So basically what you see here is, It's something that's running on a model of dynamics that was trained on these different images. So basically, you can say, OK, I have to use cats.

I think Andrew is here. So, homage to him. We can do anything. But basically this is a pretty simple generative And I can basically say, like, okay, at times equals one, I'm going to back prop an error from randomness to a particular image class. And after that point, we let the system just run naturally. And what you'll find is that it actually has clumped its representation into places that are meaningful. They're no longer just random pixels, but they're actually pixels that make different kind of animals or whatever. So for horses, it start off as random, and at t=1 you should see it kind of converge into horse-like things.

And then over time, you'll actually see it sort of morph between those. So it's already learned in the state space that it can move between these different things. So let's go ahead and move out of this. So this is really the emergence of something new. So CPUs actually do very fast, single-threaded things the best. Even today, it's faster than a GPU. And really what you're doing is this kind of von Neumann machine, where you're moving in and out of memory and cache, and doing operations. GPU basically did this with multiple operands at once.

So we move a bunch of operands from memory, do some stuff to it, write it back. Compute and memory, like Grok, did the same thing, but just did it on chip. It's kind of a more fine-grained version of this. What we're talking about is doing something in a dynamical system. The state and the function are overlapped with the physics themselves. So you now have no separation. between STATE AND COMPETATION. And, you know, compute efficiency goes up. Of course, galaxy brainness goes up. And this is truly non-Wahn-Norman. So with that, I'm just going to leave you with this quote.

It's been something I've guided my entire life by, and I'm really excited about this time, because I've been thinking about this problem for 30 years. And we're at this point where I think we can actually start to understand how brains work, because now we can build them. Thank you. Thank you.