Phaidra’s Jim Gao on Building the Fourth Industrial Revolution with Reinforcement Learning

[00:00] A lot of times, like when we talk about AI, right, both in the Valley and elsewhere, [00:04] I think there's a conflation between AI and automation, right? [00:08] Um, AI can absolutely automate things. There's, there's no doubt about that, right? Especially like routine things, right? [00:14] But I think that honestly undersells the real promise of AI, right? I think the real promise of AI is what Demis, the CEO of DeepMind, calls, you know, like AI creativity, right? It's the ability to... [00:29] acquire knowledge that did not exist before, right? And I, of course, experienced this firsthand. The reason why I'm such a true believer in the technology is because... [00:38] Again, I was the expert who helped design the system, but this very AI agent that we created is telling me new things about the system that I didn't know about before, right? And that's a very, very powerful feeling. [01:04] Beep beep beep beep beep. [01:06] Hi, and welcome to Training Data. [01:08] Please welcome Jim Gao, founder and CEO of Phaedra. [01:12] Jim was previously the leader of DeepMind Energy, one of the first and only AlphaGo-style reinforcement learning applications in the wild. [01:19] DeepMind Energy used reinforcement learning to manage Google's data centers and drove some staggering metrics [01:24] including 40% energy savings. [01:27] We're excited to ask Jim about reinforcement learning in the industrial world.

[01:30] and to learn more from him about what other real-world applications [01:33] are poised to be transformed next by deep reinforcement learning. [01:38] Thank you so much for joining us. Maybe before we get started, we're going to spend a lot of time today talking about your deep mind energy journey. [01:45] But maybe can you give everyone one or two sentences on your background and what you're building? Yeah, of course. So, Phaedra is an AI company, of course. Fundamentally, we are an AI automation company. So, what we do... [01:56] is we use a type of AI known as reinforcement learning to directly control [02:01] and operate our customers very large mission critical industrial facilities so [02:05] In practice, these AI agents... [02:08] They act as virtual plan operators, virtual members of the plan operations team. [02:14] Let's go back in time and talk about the journey that led to this journey. [02:18] And I believe that you once said an email with the subject line, [02:23] Reinforcement learning plus data centers equals awesome. [02:27] Can you tell? Question mark. Yes. Reinforcement learning plus data centers equals also. Can you tell us? [02:38] Who did you send that email to? Why did you send that email? What was on your mind at the time? And then, of course, what did that lead to? [02:45] Yeah, of course. So the reason why there was a question mark is because it was generally an unknown. [02:50] if the combination of reinforcement learning with industrial facilities would actually be awesome. [02:56] So that was an email that I had sent to a person named Mustafa Suleiman, who would later become my boss at DeepMind.

[03:02] and um [03:04] Really, the impetus was something called AlphaGo. So to set the stage properly, [03:08] I had been experimenting as part of my... [03:11] uh famed the 20 time at google [03:14] with machine learning technologies. [03:16] And it was actually a very specific course. [03:19] Introduction to Machine Learning by Andrew Ng and Coursera that had just come out. [03:24] This is back in 2013, my view. I think I was like the second cohort or something. [03:28] And that class had completely changed my life. I taught myself how to program. [03:33] and just started tinkering around with machine learning on the side. It was very interesting technology. And your background was mechanical engineering and environmental systems type things? Yes, that's absolutely right. So my responsibility at the time was to, one, help Google design and operate their very large data centers. And once these very large data centers, which consume enormous amounts of energy, were built, [03:53] we of course shifted our focus to [03:55] operating these complex industrial systems in the most energy efficient way possible, [03:59] because they use billions of dollars in electricity. [04:02] So that was kind of the background. I was already tinkering around with machine learning technologies on the side to analyze the enormous amounts of data that Google's data centers were generating today. [04:12] In 2016, AlphaGo came out. [04:15] And I was one of hundreds of millions of people around the world, you know, watching. It was like 3 a.m. in the Bay Area or something. And I found it absolutely captivating. And to the point where... [04:27] Um, [04:27] I sent an email to Moose describing this idea that [04:31] If DeepMind could beat the smartest, most intelligent people in the world at complex games like Go,

[04:37] then surely we can train the same AI agents to play a very different game [04:42] that I'm familiar with called Let's Optimize the Pee Wee, the Power Usage Effectiveness of Google's Data Centers, right? Right. [04:48] So that was the context for that email. [04:51] And I remember internally the way I pitched it to Google's leadership, so specifically Joe Caba, [04:56] of who leads Google's data centers and ORS, [04:59] was I showed a picture of a Go board on one side and a video game controller, like an Xbox controller on the other. And I'm like, look – [05:06] There are objective functions that we're trying to minimize or maximize. [05:10] There are concrete, like knobs and levers, so actions that we can control. There are constraints that we have to stay within. And all of this happens within a very measurable environment. [05:20] reinforcement learning and operating large complex industrial systems are actually one and the same thing. So that was the... [05:28] original kernel of insight, I guess, that inspired it all. And I know Sonia has accused me of going rogue with some of the questions we ask here. I'm going to go ahead and go rogue for a minute. Already? It's been like one minute. We're going to come back. I want to skip the story. This is a brief diversion. Bear with me. [05:45] The three things you mentioned that allowed you to see the parallel between reinforcement learning and control systems or control theory. [05:51] Objective function, actions, constraints. [05:55] Are those the three key ingredients for where... [05:58] reinforcement learning can be applied to real world systems yes absolutely that is 100 how we think of it right um [06:05] you know, the... [06:07] Reinforcement learning systems, they need like KPIs to optimize for. They need to know how good or bad an action is, right? They obviously need things to control and they need to know what are the constraints they have to stay within. So.

[06:19] Really what we're saying is, [06:21] As long as we can map [06:23] the problem we're trying to solve [06:26] into [06:27] Um, [06:27] reinforce a learning framework, which really from a mathematical perspective, what we're saying is we're solving a constraint optimization problem, right? [06:34] If you can map [06:35] the constraint optimization problem. If you can define it and map it to the underlying data, [06:41] then it should be able to be solved using reinforcement learning. [06:44] So that's very much the lens through which we look at things at Phaedra as well. [06:48] And, you know, to take it one step further, [06:50] You know, we often talk about how [06:52] reinforcement learning and controls and optimization are like two [06:58] wildly [06:59] different fields historically that have somehow independently converged to the same area, right? Like they're two very similar concepts. Well, we've been calling them by different names this whole time. So you've had... [07:09] like almost these independent evolutions, [07:12] Right. A different ways of tackling the same problem. And Phaedra is really kind of the intersection of both of these. OK, let's get you back onto the story. So you said that you lost. Sorry for the diversion. So you said the email to Mustafa and then what happened? [07:24] Yeah, so he sent the email to Mustafa. [07:26] Two weeks later, Moose had actually flown out to Mountain View, right, where I was working at the time on Google Campus. [07:33] with a team of D-Mind folks, and we actually started mapping out exactly how reinforcement learning could be used to control and optimize Google's data centers. So that actually kicked off the original partnership between D-Mind folks. [07:46] Google and DeepMind around the application reinforcement learning for the data center work.

[07:51] Um... [07:51] It was... [07:53] you know it was very very fascinating but [07:56] You know, most importantly, it's actually also how I met one of my two other co-founders, right? So Veda was one of the original engineers on the AlphaGo project. [08:06] He had gone to go to, you know, he went to Seoul, South Korea, right? And, you know, he actually got to meet Lee Sado and Larry Page and all, you know, all this fun stuff. [08:15] After AlphaGo, he came back to the UK and he was wondering, well, what is my next big thing going to be? [08:23] And I managed to convince Veda, like, hey, [08:25] What if we applied [08:27] self-learning frameworks like AlphaGo [08:30] to control and optimize Google's data center. So that's actually how I started working with my co-founder, Beta. [08:36] Did people think he was going to work, or was it like, this is a crazy moonshot, let's just try, but who knows? [08:41] I didn't even know if it was going to work. Conceptually, it made sense in my mind. I'm like, hey, this is – [08:47] It's a... [08:49] Operating a data center is just a different game. [08:52] play, right? And there's all kinds of different games in the industrial world, right? Maybe the game is maximized energy efficiency. Maybe the game is [08:58] minimize water consumption. Maybe the game is maximize the yield of a factory, right? But there's all these games that we're constantly playing. [09:06] So in my mind, it made sense, but [09:09] To ask your question directly, no, I had no idea if it was going to work. [09:12] I still vividly remember... [09:14] to this day. [09:16] When we turn on the AI system and we watch the energy just drop,

[09:21] And it was so surprising for two reasons. Number one... [09:25] well, we had designed the system. I played a role in designing that very mechanical system, right, that the AI was now controlling and optimizing. So in theory... [09:34] I'm literally supposed to be [09:36] the subject matter expert who knows everything about these systems, but the AI is teaching me things that I didn't know about the system I helped design in the first place, right? [09:45] Um, [09:45] And two, the moves that the AI was making were just very counterintuitive, right? Like when we looked... [09:52] at the decisions that were coming out. The plan operators and I [09:56] We were sitting in a giant cornfield in Iowa where Google likes to put its data centers, and we were looking at the decisions, and we thought to ourselves, there's no way this is right. This AI sucks. I learned the wrong thing. [10:09] But we're here anyway, so let's try... [10:12] what the AI is saying [10:14] And we tried it and it worked and we just saw the energy plummet. So I think that was kind of when I became a believer in this technology, right, that [10:22] Fundamentally, this... [10:24] technology. [10:25] Um... [10:26] It's creative. [10:27] It helps us discover new knowledge that didn't exist before from raw data. [10:32] Was there a performance tradeoff or was this just straight up Pareto gain, like performance held? That's a good question. No, it respected exactly the same constraints as the plan operators and engineers had already put in place. So this is pure gain. [10:47] Right? [10:48] respecting exactly the same temperature profile as exactly

[10:51] you know, the same constraints on how quickly you can turn on and off a chiller. [10:55] minimum pump VFD speeds, all that sort of stuff. So this is pure optimization, pure gain. [11:00] which I think is one of those crazy things. We don't really expect... [11:04] Like, usually when you think about energy efficiency, for example, right? Like, in the world that I come from, people usually think about expensive capex, like [11:11] oh, we got to rip out the chillers. We got to buy a bunch of new chillers from Johnson Controls and trains or whatever. And then we have to install them. So they're like, [11:18] hardware efficiency gains, right? But you don't really think about like pure software, like data driven efficiency gains, right? And I think that's part of what was surprising for us. [11:28] Can you add this to the before and after? Maybe before what you all implemented, was this industrial control systems? Was this manual plant operators turning knobs? How did this work before and then after? [11:50] modern industrial facilities are very, very complex. There's all kinds of machines that people are operating and controlling. So I often tell folks to do a simple thought experiment. So imagine you have... [12:03] just 10 machines you're controlling. So say they're like pumps, right? And each one of those machines has 10 possible set point values. So 10 modes associated with it. So think something like, [12:14] 10% pump speed, 20% pump speed, 30% pump speed, etc. [12:19] Then in this very simple toy example, you have 10.

[12:22] raise to the 10. [12:23] or 10 billion different permutations for how you can operate your toy system. [12:27] Right. [12:28] So then the question becomes, well, at any given point, what is the most optimal way of operating your toy system? And by the way, these are dynamic systems, right? [12:37] IT load is changing, the weather is fluctuating. [12:40] The people operating these systems are changing, the pipes are corroding, the heat changes are fouling. So the point is these are very complex dynamic systems. [12:48] Railroad Systems [12:50] have a lot more than 10 machines and each machine has a lot more than 10 set points. [12:53] So you can start seeing why... [12:56] technologies like AlphaGo, which managed to navigate MX complexity, [13:01] are helpful over here. It also helps explain why there's often so much room for optimization in the first place, because there's so much complexity, right? Like if you think about the total action space, right, like all the possible actions that [13:13] within a modern data center, for example, right? [13:17] because of risk averseness, but also because of hard-coded rules and heuristics, right? We've only ever explored, like, [13:24] 0.00001% of all the possible ways that you could operate that system. So then the question becomes, what is in this... [13:33] 99.99999% of the access space we've never explored. Surely, [13:38] there are more optimal ways of operating the system than what we've done historically. So it's kind of an... [13:43] intuitive explanation, hopefully, of why there can be such large efficiency improvements in the first place that are undiscovered. And the way that we operate these facilities is constrained by a mixture of

[13:54] hard-coded controls logic, right? So [13:57] Don't get me wrong, these are automated systems. [14:00] today already, right? There's just... [14:01] not opto, you know, automated intelligently, I would argue. Yeah. Right. [14:07] And, you know, there is a healthy mixture of human intuition as well, right? [14:12] We have people like myself or plant operators who are constantly monitoring the system, who are nudging the system, adjusting things, or setting things. [14:20] adjusting the rules for that system, the constraints that the system has to operate within. [14:26] But fundamentally, [14:27] human intuition plus hard-coded controls logic is still [14:32] when you talk about this degree of complexity, right? Yeah. [14:36] Can you talk to us about the key results? So you saw the energy levels drop immediately. Um... [14:41] But what results were you able to drive for Google? [14:44] Initially, so there's two types of results, right, you know, for Google in particular. There was results from the pilot. So in 2016, we released, we announced like the results of the pilot, right? Now, the pilot was done on a couple of data centers, but fundamentally, it was not an autonomous control system. [15:02] So what I mean by this is it was the AI generating recommendations. [15:08] which for human experts like myself to manually review and implement. [15:13] and of course you know [15:15] we didn't want to jump straight to taking our hands off the steering wheel, right? Because it's a new novel technology, right? But also like... [15:21] no one knew at the time, like, is it even possible to use AI from the cloud to control big-ass infrastructure, right? So, um, so step number one was do the pilot, right? The AI generated

[15:33] recommendations, that's where we saw like really steep, like 40% energy savings, right? [15:38] Now... [15:39] that experience taught us like, Hey, we think there's something real over here. [15:43] we should actually just let the AI control things directly. [15:47] right to get the value automatically. [15:49] And also, quite frankly, the plan operators were getting tired of checking their email like every 15 minutes, waiting for the AI to tell them what to do, right? Or the manual implement things. They had better things to do, right? [16:00] So we actually decided, and rather Ors and Joe decided, like, hey, [16:05] It's time to go to a fully automated system, right? This was total uncharted territory at that point. Like forget about can AI control things. We didn't even know like. [16:15] Is it possible to control machines from the cloud, like huge industrial infrastructure in the cloud? Because to our knowledge, no one had done it before. [16:22] Is it fair to assume that a lot of the hardware, a lot of those machines are things that Google built from scratch? Or does Google use a decent amount of commercially available data sort of stuff? It's a mixture of both. So obviously Google does a lot of things in-house, but it doesn't manufacture chillers and that sort of hardware. So Google does buy off-the-shelf hardware. [16:42] but you know there's a lot of like modifications and google specific things you know that that we did right for example like [16:48] you know, programming some of our own PLCs or, you know, making modifications to the building management system. So like the software control layer looked quite different. That was done in-house. [16:58] But I still remember very vividly actually to this day [17:02] Veda and I, we were standing in a large 90 megawatt data center. It was a fairly large data center.

[17:09] And... [17:10] You know, Vida's like typing away in his MacBook, right? [17:13] He submits the PR... [17:15] Right? Get smirged. [17:16] And all of a sudden, [17:18] this huge honking huge chiller that is the size of a bus that we're standing right next to, roars to life. And as it's coming to life, right, like the ground is shaking vigorously. And we're like, oh, my God. [17:31] with a few keystrokes on its MacBook, like we just turned on this [17:35] enormous chiller [17:36] and that was like the very first data point to us like yes it is possible [17:41] to control things from the cloud, [17:42] So now the next question is, [17:44] How do we control things intelligently from the cloud, right? You know, where all the compute resources live. [17:50] What were your biggest takeaways from that experience? You mentioned the creativity of the machine. Any other big takeaways or learnings? [17:57] So the creativity is absolutely a big one. I think, you know, just to elaborate on that briefly, [18:02] you know a lot of times like when we talk about ai right both in the valley and elsewhere [18:06] I think there's a conflation between AI and automation, right? And like AI can absolutely automate things. There's no doubt about that, right? Especially like routine things, right? [18:16] But I think that honestly undersells the real promise of AI, right? I think the real promise of AI is what Demis, the CEO of DeepMind, calls, you know, like AI creativity, right? It's the ability to... [18:31] acquire knowledge that did not exist before right and i of course experienced this firsthand the reason why i'm such a true believer in the technology is because

[18:40] Again, I was the expert who helped design the system, but this very [18:45] AI agent that we created is telling me new things about the system that I didn't know about before, right? And that's a very, very powerful feeling. It's kind of like when... [18:54] Um... [18:54] you know, if you think back to AlphaGo, right? [18:56] Like. [18:57] Lee Sedol was the best in his field at GO. He was the world champion for a decade. He was at the top and his ELO rating... [19:05] It was just something outrageous. It was like $2,800 or something. It was outrageously high. [19:10] but it had flat you know flat light right so for a full decade [19:13] his elo rating was the same, and there was no one to challenge him because he was already at the top. So once he hit the top, he just kind of plateaued. [19:20] And then after AlphaGo, [19:22] And he actually got to play against AlphaGo, you know, privately a few more times because DeepMind, you know, had, you know, had let him continue interacting with the system. [19:32] What happened, for the first time in a decade, his ELA rating started climbing. [19:36] And so this is what I mean when I say that I think the real power of AI is helping us discover knowledge that we didn't necessarily know about before. And where you're going to see the most… [19:47] Gain from that, it's not going to be in routine automation things, right, like call centers or whatever. [19:52] is going to be, I think, in very, very... [19:56] complex... [19:58] Like areas where... [20:01] human intuition is insufficient because of immense complexity, but that [20:05] is yet underpinned by data. So that's why you're seeing such things like [20:10] protein folding, for example. I mean, that...

[20:12] That's fucking extraordinary. [20:14] And it's those areas that are just like massive permutational complexity underpinned by data. [20:21] That's where I think we're going to see some of the most interesting companies and products [20:25] Um, [20:25] So that was a rather long tangent. But so one, creativity is something that I learned. [20:30] The other one, [20:32] lesson that my co-founders and I learned is really around [20:36] Um... [20:36] you know. [20:37] If you want real impact, [20:39] You got to turn the technology to product and this is actually the important reason why we we decided to leave the mining train technologies to start phaedra right like over and over again we were seeing the technologies. [20:50] that we were helping to develop at DeepMind was just extraordinary. [20:54] They were achieving crazy things like with protein folding. But the problem is [20:59] In order for the technology to make the most impact, you have to get into the real world. People have to actually use it. [21:05] Right. And that fundamentally means we're talking about a product. [21:08] Turning a technology into a product is like, I mean, you guys would know much better than myself. It's like a hundredfold, a thousandfold more work, right? [21:17] And that, for us, led us to the conclusion that, hey, it's time to leave, right? It's time to actually... [21:24] um [21:25] start a company that [21:26] that creates these [21:28] intelligent. [21:29] virtual plan operators, these intelligence and AI agents as a real product. [21:34] Let's talk more about that. Um, [21:36] For what you're building now, [21:38] How much of what you learned at Google Demind sort of translates directly into what you're doing now?

[21:45] How much is... [21:46] new because the environments are different, the customers are different, there's something different about it. [21:50] I think the most important thing that we learned from our Google DeepMind experience is that [21:55] is possible. [21:56] Like, this is not a crazy... And that isn't to, like, you know, like, downplay, like, what we learned. Like, it's actually a huge thing, right? We learned... [22:04] that it is in fact [22:06] possible [22:07] to use [22:08] you know, closed loop learning systems like reinforcement learning, right, to drive very large improvements in complex industrial facilities. It hadn't been done before, to our knowledge, right? [22:19] And that was a massive proof point. [22:21] um [22:22] I think the problem, though, is that... [22:24] Like, [22:25] The real world is quite diverse. Every single customer is diverse and [22:28] especially when you talk about industrial facilities, like every industrial facility is a snowflake, right? So for us, I mean, the learnings have just been... [22:38] like massive since we left Google and DeepMind, right? Because every time we onboard a new customer, we're learning something new. [22:45] about like how equipment are connected or some product gap that we didn't know about before that needs to be fixed right [22:51] or you know, [22:53] new ways that data can break. At this point, I can tell you like 100 different ways that, you know, data associated with mission critical cooling systems can break. Probably not the most interesting party topic for most folks. But you know, I personally find it quite interesting. [23:06] but um [23:06] Yeah, there's certainly been quite a lot of learnings in that regard. [23:10] Folks you're talking to, are they ready to let the technology take over the system and, you know, let the culling system just start going?

[23:19] Yeah, I mean... [23:21] Yes and no. Right. And that actually gets back to your earlier question, Pat, as well. Right. About like the specific learnings from Google. I mean, when I look back. [23:30] you know [23:31] I think... [23:31] I think what we helped pioneer at Google and DeepMind could only have been done at a company like Google. [23:39] The reason why I say that is because Google is a very forward-leaning company. Yeah. Yeah. [23:44] But also, like, one of the things I've learned, right, is that, like, [23:48] Google is absolutely an anomaly when it comes to, like, how much data it has and the pristine quality of the data and the ease of access of the data, right? Like, Google is fundamentally a data analytics company, right? And, you know, Asset should have invested all this, like, infrastructure in high-quality, high-availability data. [24:08] on which you can do things like real-time intelligence applications, like what we were doing, and there are many other examples of this within Google and DeepMind. [24:15] Um... [24:17] Having left the nest, one of our rude awakenings was, you know, Google is definitely an anomaly. [24:25] And... [24:26] I mean, gosh, like everyone is in various stages of their AI journey. [24:30] Right. Like it was certainly on one extreme. We have customers who've encountered where [24:35] Like, you know, forget about real time intelligence. They're like, they're not capturing the data in the first place. Right. Or. [24:41] They may be censorizing – [24:43] In the industries we work, like pharmaceuticals and district cooling and especially data centers, [24:48] almost always the customer is sensorized, right? Because these are billion-dollar facilities. Of course, it makes sense to throw a million dollars worth of sensors on it.

[24:55] But that doesn't just because you sense right doesn't mean [24:58] that you're storing the data [25:00] Right. [25:01] A lot of customers of ours aren't necessarily storing the data beyond like 90 days or six months or a year or whatever. [25:08] Right? [25:09] And, you know, and they might cite some reasons like, well, it's costly to store the data. [25:14] or the more commonly we're not using the data for anything which is a true statement. A lot of our industrial customers they aren't using the data and [25:24] It's more like a forensics thing where if something goes wrong, then we go back and we look at the logs to see what happened, right? [25:30] Um... [25:31] And then... [25:32] you know. [25:33] So if we think about it like Maslow's hierarchy of data needs or something, right? You got your sensorization, you got your storage, then you have to invest in making sure that the data is cleaned, right? There's a lot of effort, as we all know here, around making sure the data is actually cleaned and usable, right? And that requires you to know what bad data looks like, what good data looks like, and how to convert bad data into good data so it's actually useful. [25:59] And then once you have claimed data, you also need to make it accessible in a streaming and batch historical manner, right? [26:06] So. [26:07] There's different gradients, I guess, is what I'm trying to say of AI readiness. The customers whom we work with are all over the spectrum. [26:15] but [26:16] you know, [26:17] Like, Phaeger today is at the point where we are autonomously controlling data centers for our customers, right? So I was going to ask you if the... [26:25] basic workflow or the basic loop is data goes in,

[26:28] which is a lot of what you just talked about, getting the data into the system. [26:32] Step one. Step two, decision is made. [26:35] Step three, action is taken as a result of a decision that was made. [26:38] Step four, action is evaluated against the objective function of the system. Yeah. And then the loop continues. [26:44] So the front end of that process, which is data goes in, sounds like there's a lot of work to get some real-world data ready to go. We call it the AI readiness journey, right? So, like, if you think about our work with customers, like, [26:54] there is a chunk of upfront work [26:56] where it's just like, hey, we're going to get your facility. We're going to get you and your facility AI ready. [27:02] How about on the... [27:04] Action is taken. Piece of that. [27:06] Are the systems ready to be controlled by some sort of autonomous system or [27:11] Is there work that needs to happen there too? [27:13] Yeah, it's a really good question. [27:15] Yes and no. [27:17] right you know elaborate on what i mean by that right control systems today were like designed in like the 1980s right um you so was i well me too for that matter there we go but you know this uh like what i mean by that is you know that was the the 70s and 80s was the third industrial revolution right so with that [27:39] you know, with the shift from analog to digital and the advent of the first automation systems, right? In order to automate, you fundamentally first have to sensorize. But these are simple automation systems, right? [27:50] The fourth industrial revolution, right? [27:53] Um, [27:53] you know, is, you know, we're biased, but Phaedra, you know, we, we think the fourth industrial revolution means, um,

[28:00] intelligent infrastructure, right? Infrastructure that can operate itself and fundamentally get better over time at doing self-improving infrastructure. [28:07] right um [28:08] But right now we're, [28:10] shoehorning intelligence into systems from the third industrial revolution. Right. So they certainly weren't desired for this. [28:18] But... [28:18] What we do instead is, most importantly, we ride on top of the existing control system. So there is a hard-coded layer [28:26] of rules and heuristics. So millions of lines of if-then statements [28:29] programmed into what we would typically call the BMS, the building management system, or a SCADA system, right? That defines how the facility should operate. The problem with hard-coded systems is that because they're hard-coded, they operate the same way today. [28:43] as they did yesterday or a year ago or five years ago, more like 10 years ago, because people don't very frequently go into the backend programming, right, to update that controls logic. [28:53] Now, what Phaedra does is we insert a new cloud intelligence layer at the very top of the control stack. So we're [29:00] not we don't introduce any hardware we don't introduce any new sensorization right we actually ride on top of the existing control stack that's really really critical right you can think of it as a general in the battlefield the general [29:15] has a global view of everything that's happening across the system, and it's issuing command signals to the troops on the ground for actual execution. So the AI is looking, in our case, at 10,000 trends a minute in real time,

[29:30] Right. And it's issuing decisions like which pumps to turn on or what their pump speeds should be. [29:37] right to the local BMS system and or the PLCs for automatic implementation and execution. So that's why I said it's a mixture of yes and no. Were they designed for this in the first place? No. [29:47] There is a lot of work that we have to do with our customers to be able to... [29:52] accept this type of external intelligence. There's a lot of work that we do in defining the safety nets and guardrails, right, to ensure that the AI can't do bad things to the customer system. [30:04] Right? [30:05] But fundamentally, we are still riding on top of the existing controls architecture. And to be clear, [30:10] We always want to do that. [30:12] You don't want... [30:14] AI controlling things like how fast a valve [30:17] opens and shuts, right? Like that's a terrible application of AI, like hardcoded Rosencure 6 will do great there. [30:22] So if you were to look at the overall system – [30:26] 90% of it. [30:28] is fine with... [30:29] just hard-coded Rosen heuristics because it's like granular controls logic that doesn't need non-deterministic [30:36] crazy powered intelligence behind it, right? But it's the higher level thinking and reasoning. That's where you want the AI. It's the global optimization. Have you seen any of your customers at Phaedra kind of get the deep mind order of magnitude results? [30:51] Mm-hmm. [30:52] Mm-hmm. [30:52] So – [30:54] I'm glad you asked. So one of the things I'm really excited about is actually – [30:59] Just... [31:00] Actually, literally this week, earlier this week, um...

[31:03] Merck Pharmaceuticals became our first public customer. So we're pretty proud about that. We've actually been working with them for two years now. [31:10] They've been using Phaedra [31:12] for over two years [31:13] the full autonomous AI system to control a massive 500-acre building. [31:20] a vaccine manufacturing facility in Pennsylvania. This is the definition of mission critical complex. They've got 62,000 tons of cooling, so they've got four very large shell plants interconnected with each other across 500 miles of... [31:36] manufacturing space right [31:38] hundreds of machines interacting with each other. This is where the AI really shines. [31:43] Uh, [31:44] And, yeah, the results that we saw with them were quite strong, right? Like, you know. [31:48] I think Merck actually just shared some data at a conference we were at. We chose 16% energy savings when we first trialed the system at one of their chiller plants. [31:56] But what I always tell our customers is... [32:00] Um... [32:02] Don't over-index. [32:03] on the magnitude of the energy savings initially. Like, we honestly have no idea... [32:09] what the energy savings are going to be [32:11] or the reliability improvements are going to be, [32:14] Um... [32:15] ahead of time. [32:16] Because these are non-deterministic systems. And by definition, if I could tell you what things you're not doing in order to get energy savings, why do you need the AI in the first place? Yeah, yeah, yeah. [32:25] So. [32:26] But what we do know [32:29] is that the unique thing about this technology [32:32] about Phaedra and about reinforcement learning in particular, is that it is a closed-loop system. It is a self-learning system. It can learn.

[32:39] because it's able to take actions and it can measure the impact of his actions against his predictions right [32:45] and [32:45] That means it gets better over time. [32:47] So maybe we start off at 1% energy savings. Maybe we start off at 5%. Maybe we start off at 10%, right? But fundamentally, it will learn and it will get better over time, right? [32:59] Not infinitely because there are so are laws of physics, right? [33:03] but it will get better over time. [33:05] and once it reaches optimal [33:07] it will stay at optimal, right? That's super important because with hard-coded rules and heuristics, right, um, [33:15] When you tuned a system as you were commissioning it, so when you're turning it on for the first time, right? [33:20] That system today no longer performs the same way that it did 10 years ago when you first commissioned that system. [33:26] because the pipes have corroded and the heat exchangers have fouled and the cooling towers have scaled, whatever, where you ripped out equipment. So, [33:34] But the promise of an adaptive self-learning system is that [33:37] it will change with you, right? As your customers are, for example, now putting in a bunch of H100 and soon H200 GPUs, right? Well... [33:45] The system will learn and adapt on the fly with you. [33:48] Right. So it can stay optimal. [33:50] I'm like, [33:51] I'd love to transition for a minute beyond industrial control systems and get your opinion on... [33:57] I mean, you were one of the... [33:58] first and maybe one of the only real world applications of reinforcement learning. Yeah, we're definitely not the only. Not the only. I'd love to get your thoughts on the not the only. So, I mean, what else are people doing with reinforcement learning in the wild today?

[34:13] Yeah, absolutely. So, you know, unfortunately, my knowledge is very heavily indexed on the Google and DeepMind space because that's how we spent so much time, right? [34:23] But even within Google and DeepMind, there were other very cool reinforcement learning applications. Like, for example, the team that sat right next to us, they used RL systems to... [34:34] to help prolong battery life, for example, right? So you may notice that your Android phone, right, [34:39] Like the battery life has been increasing. [34:43] There are hardware changes associated with that, but there are also intelligent software changes behind the scenes that proactively manage your battery life. There were reinforcement learning systems for YouTube video recommendations, for example. [34:59] and a whole host of other things, right? [35:01] Absolutely, there are. [35:03] Um, [35:03] you know, [35:04] reinforcement learning applications in the wild. [35:08] To your point, though, I wouldn't say that there are a whole lot of them, right? Yeah. And I think that it's not a coincidence that you tend to see them at more of like the... [35:17] big tech companies where they've already invested in the data infrastructure, right? [35:21] So that the underlying infrastructure, so that they can benefit from this technology, right? [35:27] like outside of the big tech companies. [35:30] there are very few applications of like real world reinforcement learning, like in production, at least. Yeah. And do you think that's because of kind of low applicability? You know, you started this podcast by talking about [35:41] the necessary ingredients for RL to be a good solution. Do you think it's just...

[35:46] There's not that many applications where RL is a good solution, or do you think it's just readiness? Absolutely not. I think the applications for reinforcement learning are... [35:55] freaking massive and we're... [35:58] Baird Phaedra [35:59] is one of many examples that we're just scratching the surface as an industry of what we can do with this technology, right? Like fundamentally, [36:06] the power of the technology is that it is a self-learning system alpha go [36:11] and its successor AlphaZero taught itself to become the best in the world at [36:15] Go, Chess, and Shogi, three vastly different games, same learning framework. [36:19] Right. [36:20] and a tot itself. [36:22] I think there's a lot of very interesting application areas, right? I think the data infrastructure is missing. [36:27] in a lot of them. But just to list off a few, right? I mean, obviously, we've already talked about the protein folding, right? Yep. But there's an entire untapped field around logistics, right? [36:39] Like that is such a gnarly computational challenge, right? You know, when you start looking at operations research, operations research underlies trillions of dollars worth of, you know, industrial like activities, right? Not just industrial, but other sorts of activities, right? Like shipping, airplanes. [36:57] Uh, uh... [36:57] FedEx-like driving routes. These are all applications of operations research. Grid balancing... [37:03] right i mean i think rebalancing is probably the single [37:08] most important way that AI can fight climate change. I would generally believe that is where AI will have the most impact on climate change. If you had to guess...

[37:16] The first time you deployed this into a data center at Google, you saw 40% energy savings. [37:22] If we had just... [37:23] killer AI doing load balancing on the grid. [37:27] What sort of energy savings do you think we could see? [37:30] Oh my gosh. [37:31] I mean, that would be wild. [37:33] I think it's not so much about the magnitude of the energy savings per se, but rather about the potential cost savings because then you could start shifting your loads around to when it's most cost effective to do compute. Or if you had CO2 signals, you could start scheduling loads around… [37:51] um [37:52] when it's the least carbon intensive to do your non-lacency sensitive workloads, right? Which I think Google has already been experimenting a bit with, right? [38:02] But honestly, I think it's really more around the – [38:06] global system level optimization, right? We have to keep in mind that [38:10] data centers, [38:11] already are but increasingly you know um... [38:15] It's just massive, massive load banks, right? Like data centers, like they were... [38:19] One and a half, 2% of U.S. energy consumption. That's about to increase to 4%, right? Like this year, I think. And then by the end of the decade, it's projected to get up to like 9% of the U.S. In Ireland... [38:31] Right now, Ireland is... [38:33] 22% of Ireland's [38:35] national energy, electricity consumption, [38:38] goes to data centers alone. [38:40] the International Energy Agency predicts that that's going to increase to 37% by the end of the decade, right? Like just mind-boggling numbers. But the point, the reason why I mentioned this is because...

[38:51] These are massive load banks on the grid, right? There is an actual opportunity if you could somehow coordinate the data centers together, right, right? [38:59] Um, [39:00] to [39:01] to help balance the grid. [39:03] And that is such a gnarly, gnarly challenge. And it is what is holding the energy transition back. [39:09] As more and more renewable energy starts coming onto the grid, the supply side becomes increasingly stochastic. [39:16] We used to have this perfectly deterministic system, at least on the supply side, where a good operator can call someone who operates a coal-fired power plant and say, hey, ramp up or down your power production. It's deterministic. [39:28] But now – Ramp up or down the sun. Yeah, totally, right? So now you get more and more renewable penetration coming onto the grid, right? [39:36] you have a somewhat non-deterministic demand side. It's somewhat predictable, but there are definitely spikes, and a massively non-deterministic supply side. And what is the problem with that? The problem is that [39:49] you know, we... [39:50] Because we do not know how much energy we're going to generate, [39:54] you now have [39:55] all this wasted excess capacity in reserve, right? So there is a concept of spinning reserves on the grid where [40:01] Um... [40:02] There are peaker plants... [40:04] You know, like... [40:06] giant natural gas turbines, right? That as we speak are just sitting there idling, just like your car idles, [40:12] at a stoplight, right? [40:14] in case we need that power, right, as a buffer against the uncertainty. And as renewable penetration increases, ironically, the amount of buffering you need also increases. If you look at Germany's failed energy transition, right, they decommissioned their nuclear baseload while wrapping up their renewable energy penetration, right? Right.

[40:32] good motivation on the surface, although I personally think we need a lot of new, more nuclear on the grid, but that's another topic. But, um, [40:41] it ended up backfiring, right? Because Germany actually ended up needing to build more fossil fuel power [40:47] power plants to buffer against all the renewable energy that was coming onto their grid now. Right. So that's why I think [40:55] AI for grid balancing? [40:58] We need it, and it probably is the single most impactful thing that AI can do to solve climate change. [41:03] Let's talk a bit about some of the limitations of reinforcement learning and also where you see it intersecting with transformers. [41:09] Yeah. [41:10] Um... [41:12] So I should state that, first of all... [41:14] My co-founder, Veda, is by far the expert on this topic. He knows way, way more than me. You know, I'm just a simple mechanical engineer, right, who happened to learn a bit about AI. Yeah. [41:26] Um, [41:27] I think the intersection is really interesting, like very... [41:31] very [41:34] Very potentially complementary strengths and weaknesses is how I would describe it, right? It's certainly not mutually exclusive. [41:42] Like what I mean by that is, and I was just talking with Beta about this earlier. [41:46] So Veda will tell you that like, [41:49] all intelligence systems have certain hallmarks, right, of intelligence, so that we can say they're intelligence. They need to deeply understand the world, the environment that, you know, that, that [41:59] that they're modeling. There needs to be some element of memory, so like remembering things.

[42:06] and [42:06] Very importantly, there needs to be the ability to... [42:10] and reason, right? Very interlinked. [42:12] Transformers are clearly quite good at the first one, right? In the sense that they can take in [42:18] Um... [42:19] huge amounts of structured and unstructured data. [42:22] right? To learn [42:24] quite good models, right, of the world. [42:28] but it is limited in the sense that these models are primarily through correlation and not causation. That makes it challenging for people. [42:37] at least for what Phaedra does, right? Because... [42:39] Because we work with real-world systems, [42:41] we have to have causality. We have to understand why is the AI doing certain things? Why is it not doing other things? How do we force a certain behavior that we know... [42:51] has to exist in our system right so these are mission critical systems what i'm trying to say there has to be causality so [42:58] That's where the limitation is, right? [43:01] Um... [43:02] with reinforcement learning systems, I mean, [43:04] that the power of RL-based systems is very [43:06] you know, much in the planning and reasoning approach. [43:10] part, right? Where [43:11] um, [43:12] you're able to plan long trajectories of actions and learn really intricate policies, right? [43:20] I think where it gets really interesting is the intersection of [43:24] Right? [43:25] where potentially transformer architectures can learn [43:29] models, like value functions or models of the world that the AI can learn policies against.

[43:38] But... [43:39] without that causality [43:42] piece, right? It's going to be quite tricky to cut it over into at least industrial control applications like what Phaedra does. [43:49] Should we move into a rapid-fire round? Let's do it. [43:52] What are you most excited about in the world of AI in the next five or ten years? [43:56] So in the very near future, [43:58] I'm excited about just the absolute explosion. [44:03] right, of AI applications, right? It feels kind of like a [44:07] pre-Cambrian explosion of sorts where there's like a primordial suit. And like all these AI startups and services are all of a sudden springing up, right? So it's quite exciting. [44:17] Um, [44:18] but when I look at [44:19] where that, uh, [44:22] where that activity is happening, where that research and that entrepreneurial activity is happening. [44:27] Um, [44:28] is very clearly focused on like around LLMs and [44:32] even more specifically around like [44:34] natural language interactions, right? Text-based interactions, right? [44:39] And that certainly is a large part of the economy. It is very exciting. But in the five to 10-year frame, to answer your question, I'm most excited about... [44:46] when we can start [44:48] Uh, [44:48] getting some of this technology into real-world physicals. [44:52] applications. It's the intersection of this technology. [44:55] with [44:56] the real world infrastructure that we live in, right? [45:00] big industrial systems, cars, homes, like, you know, physical things. I think that's where [45:06] we're going to see some really interesting things in the future.

[45:11] Who do you admire most in the field of AI? [45:13] Gosh. [45:15] A tricky question. I admire a lot of people. You've worked with some of the greats, and so it's going to be hard. [45:22] Yeah, I mean, of course, my mind jumps immediately to a lot of the people whom I've worked with, right? [45:27] you know. [45:29] I admire very much the D-Mind researchers whom we've worked very closely with. [45:35] I often tell people working at D-Mind, it's kind of like being a kid in a candy shop if you're a technologist like myself. It's like you get to see years in the future, right, and all this cool technology on the forefront, and it just makes your head spin as to, like, all the possible applications of that technology. [45:51] Um... [45:53] I admire Moose a lot. [45:54] my old boss who has of course since moved over to Microsoft. [46:00] you know, [46:01] I was saying earlier, like one of the biggest lessons I learned, you know, and my co-founders learned at DeepMind is that like, [46:07] Making a technology? [46:09] like what we did for Google's data centers versus making a product like what we're doing at Fisher, totally different things, wildly, wildly different things. [46:16] and there are few people as good in the world as Moose that like, [46:20] taking technologies and turning them into real products. Right. I remember my co-founder Katie and I, [46:27] Um, [46:28] We were sitting down. We were grabbing drinks with Moose at some random dive bar in Seattle. He happened to be up there. [46:35] And this is before OpenAI released ChatGPT2 and just ushered in a world of craziness, right?

[46:42] And – [46:44] And he was raving. [46:47] to myself and Katie about the applications of LLMs and how powerful these systems are. [46:54] And we were like, okay, Moose, but let's tell him about Phaedra. We had no idea what he was talking about, right? [47:02] But, I mean... [47:04] he was prescient. He saw this ages in advance, what the technology that was being developed and the capabilities that it would usher in. [47:14] And then, of course, he went off and he started inflection. [47:16] I admire him a lot for the ability to turn technology to actual products. [47:21] All right, last question. [47:23] You are building a very ambitious business, very hard business to build, and you've been at it for a while in the context of the new wave of AI startups now. [47:31] What advice do you have for other founders or would-be founders who are trying to build companies here? [47:36] I mean, I'm not sure I'm even qualified because... [47:39] one [47:40] I hope yes again in one or two years when hopefully Phaedra is wildly successful. [47:47] Um, [47:48] We certainly didn't choose the easy path by focusing on global infrastructure. [47:56] Honestly, my mind gravitates towards more like – [48:00] would be founders, right? Like people like my co-founders and I, [48:05] who [48:05] we're thinking about leaving to start something new, right? [48:09] And my advice there was... [48:11] is twofold [48:12] Uh, one...

[48:13] Make sure you have co-founders. Like, my God, it's so stressful. [48:18] There's so many things that can go wrong, and you're constantly on this emotional rollercoaster of up and downs. [48:24] Having co-founders to lean on [48:27] uh, [48:28] both for the workload but also just for the emotional support and mental sanity. [48:32] So important. [48:33] Right. Um, [48:35] Advice number two would be... [48:37] Um... [48:37] the risk [48:38] is less than you think it is. [48:41] I'm biased, but I think people should take the jump. [48:45] A lot of times when I talk with my former colleagues, [48:49] and other people who are thinking about making the jump, right? They'll say things like, well, but I've got a nice job over here. [48:56] You know, they pay me well. You know, I'm on a rising trajectory. But my point to these folks is always, like, no matter, you know, like – [49:04] how valuable and successful you are today in the organization, you will only be [49:10] more valuable, [49:12] and successful for that organization or other organizations or to society in general, [49:17] if you learn new skill sets. Like, [49:20] take the plunge, go out, start a company, learn what it's like to turn technologies into products, right? And if that fails for whatever reason, hopefully it doesn't, right? But if you fail... [49:31] then the Googles, the Microsofts, whatever the world, they will only want to hire you back at an even higher premium. So why not take the pledge, right? It's the biggest, best investment, and obviously – [49:43] Much smarter people than me have said this for a really long time, but the best investment you can make in yourself.

[49:48] right, is you... [49:49] Right? Up-leveling yourself. [49:51] learning new skill sets, that's always the best thing you could do. [49:55] Thank you, Jim. This is a fascinating conversation. Yeah. Thank you very much for having me, guys. Really enjoyed it. [50:01] Thank you. [50:02] *music* [50:26] Thank you.