Bridging AI and Human Intelligence – Multicolor Lasers and the Future of Photonic Computing

And when a grandmaster is planning his or her

chess move, it consumes close to 20

watts of power. The same level of

sophistication and planning exercise running on an AI

inference cluster, whether it is Claude's AI agent

or ChatGPT's AI agent, it consumes close to 1

megawatt of power. Welcome to

Impact Quantum.

Hello and welcome back to Impact Quantum, the podcast where we explore the

emerging industry that is quantum computing. And you don't

have to have a PhD in physics, you just need to be a little bit

curious. And with that in mind, I have the most quantum-curious person I

know, Candace Cooley. How's it going, Candace? It's great. Thank you for

asking. Today's a beautiful day. Blue, blue sky. I'm very

excited. I'm sorry, go ahead. No, no, it's— I'm— it's a little

gloomy here today, but that's okay because it's been like

well above 95 and humid all week. The— if you're

watching this, you'll notice my background is a little different. I am in Hilton Head

Island, South Carolina on a family vacation, and it's been spectacular.

So a little warm, but spectacular.

And I just told you today, in fact, that now it's like summer gets turned

off in mid-August, usually here in Montreal, Quebec,

and it just happened. So, like, we're having the 70s now.

So I'm in, like, the mid-70s, which is beautiful weather, right? But

you're not swimming anymore, but it's still beautiful. And I'm not complaining. I'm not

complaining at all. So today we have the

co-founder of Escape Photonics, Vivek.

We're really excited to talk with you and find out more about the

company and what you're doing there. How are you today?

I'm good. Thanks for asking, Candace and Frank. Enjoying the weather in

California myself. So— Yeah, you get to enjoy the

weather like, what, 11 months out of the year?

Yeah, it's been hotter than we expected, but today it's

been fairly good weather. Cool. So tell me

about Escape Photonics.

Escape Photonics is a company that is

looking to solve the next frontier of

hardware that is looking to mimic

human intelligence. Really? Oh, so it's

not just photonics is the story. It's It sounds like

there's a little bit of AI in there. It's—

everything today is about solving

the quest for human intelligence and what is the most efficient hardware to

build it. And today's data centers are

glorified computers that are mimicking human brains.

And the way we are thinking about it at Xscape

Photonics is how do we make that

AI brain, which is trying to meet human brain's

efficiency, as efficient as possible using a

multicolor laser platform. Interesting. So not just

photonics, but because usually in

my understanding of photonics, traditionally it's been

some one-color laser with different polarization of it. But now

you're mixing colors into the mix. Sounds like there's a lot more bandwidth.

And if you're able to use color. That's absolutely right.

I think the close— the reason I keep talking

about human brain and human intelligence is partly

because today's human brain consumes

35 times more power in communication than in computation.

And the reason that happens is

because human brains can contextualize a different

inputs in the most efficient manner. And

when a grandmaster is planning his or her chess move,

it consumes close to 20 watts of power.

The same level of sophistication and planning exercise

running on an AI inference cluster today, whether it is

Claude's AI agent or ChatGPT's AI agent,

it consumes close to 1 megawatt of power. Yeah, there

is no efficiency. Yeah, 50,000x in efficiency

difference between the two. And the reason for that is the

communication between the processor is not efficient at all,

and it is not fast enough. And using multicolor lasers to

improve the communication between the processors

and memory is a way to get closer to how

a human brain operates and how the efficiency of AI

inference cluster can improve. Interesting. That could be

big because a lot of the complaints about data centers

is that they need so much power, they need so much cooling, and they need

so much land. And one of the things that I've always said was there's a

lot of room for efficiency improvements in

artificial intelligence. Because if you look at the human

brain, it's something like on a regular baseline, it consumes

about 25 watts of power. Right. And that's because nature had a

lot of constraints, right? Consuming calories to the

point where no, no

biological system that I'm aware of could possibly consume 1

megawatt worth of calorie, right? It's

a luxury really only machines have. And I think also too,

another example I like to use is the crow. Crows are

intelligent, very intelligent. They are frequently

rated close to a 5 or 6-year-old child in terms of intelligence.

But they— I don't know what their calorie consumption is, but they

have to pack all of that in a platform that could fly.

Yeah. Right. So you have a dense platform. Yeah. You have

evolutionary pressure on biological systems that

thus far we've not really had in, in kind of artificial systems. So I

find that interesting, and I'm glad somebody It's thinking outside of the

traditional, I just throw another rack of GPUs at it.

Which is how the industry is doing it because no one has actually thought

outside the box like what you're talking about. And we just

think the answer is staring right at your face. The human

brain is all about communication. And when you focus on

building a very efficient communication platform, then

you end up solving that efficiency problem

as well as you can. And today, all the

problem solving has been focused on processing. How can I

process as fast as possible? That's where the GPU investments have

been coming through. And without realizing the

fact that these GPUs still have to talk to one another

and they still need some memory. And inter-process communication.

Exactly. So the investment in that had been like

an afterthought until recently. Now they just have started

thinking about it. Yeah, it's interesting because like, it's funny you mentioned that because,

you know, on the first watch of a Jensen Huang keynote,

he'll talk about the processors. On the second— in the second

watch of it, about 2-3 minutes in, he'll make a

passing reference to the networking of them together, which,

as I heard you say this, it's like Maybe, maybe we need to

put equal emphasis at least. Yes. Yes.

And now it's actually becoming more important because he talks

about Moore's Law, which is the law that governs how well the

processor can improve over time, is getting saturated,

which means it's no longer about how fast you can

process if you cannot contextualize and

communicate that to the neighboring processor. And

today the bottlenecks have shifted from processing

to memory access and how well you can actually communicate between

multiple processors and multiple memory. And that's why you start

seeing optical technologies and memory companies

getting all the limelight in the last few months,

mainly because the realization that now

I actually need a memory to think like human brain is kind of happening

now. And while the solution has been

staring right at our face for a long time, and the

industry has started embracing that inevitability,

and there is a shift in focus on how we think

about the future of AI as we know it.

So I'm taking all this in and my

mind is blown. And so I want to dial down for a

second and Just go back to the idea of what does

light allow us to do in computing that

electricity simply doesn't do as well? That's a great

question. So today, electricity

is very good in processing information. When you can

switch on and off, that gets in some— that information

of a transistor where you can tie, you can

store bits of information in zeros and ones. That

can be done extremely efficiently when you're using electricity and transistors.

That's what is the basic building block of a processing unit.

What electricity has limitation is transmitting

that information. So, uh, it just

turns out that when you have a lot of information that you are processing,

you can, you can do it instantaneously with

electrons. But then when you have to communicate from one

processor to another processor, the— it becomes

extremely difficult to transmit that information over a copper wire

beyond a certain speed. And the reason for

that is electrons are extremely lossy

if you ask the electrons to communicate at high speed.

So think of it as like it runs out of steam. when it is actually

running at fast speed. So, and

for a long time, the industry has been trying to figure out how

fast I can communicate over a copper wire without

it losing stream, losing its strength.

And it has tried to navigate that problem by actually coming up

with advanced circuit compensation techniques where you

all the information that gets lost when you're transmitting through the copper

wire, you end up recovering that information using

advanced encoding technologies. And this is where

technologies like SerDes technology and

electronic copper wire technology came into picture. It

just turns out that when you're doing that, you can only

transmit— over time, the industry has evolved to a level where it

can transmit, say, information at 16 gigabit per

second to 32 to 100. And today the industry can

transmit information at 100 gigabits per second

over 2-meter copper cable. But when it goes

to 200 gigabit per second on a single copper wire,

it cannot transmit for more than half a meter or even say

less than say 10 meters. And as the speed continues to

go up, The physical limitation is electrons just cannot

travel long enough over a copper wire. And this is the same

problem that happened in the telecom world long time back when

the industry went away from copper wire to fiber

optical cable. The reason is, when,

when you replace copper wire with fiber optical cable, the

photons don't have the same energy loss when it is

transmitting at high speed. that electrons do.

So photons can carry information across like

kilometers and kilometers of fiber optic cable without losing energy.

So then you are not spending more energy on recovering that information anymore.

So you kind of get a double boost in efficiency.

Exactly. Because the original problem goes away. And then because the original

problem goes away, you don't need the fixes. And what you're talking about, I know

my dad worked in electronics and I mean, this goes back to the

1880s when they were wiring telephone poles. Right? Like, exactly.

Every— I forget what the exact number is, but every X number of feet, they

had to amplify the signal. And, you know, the,

the repeat. Exactly. Those, those amplifiers break down. They have to

be replaced. They have to be maintained. Right? So, like, the, the switch to

fiber was a, a no-brainer, so to speak, back in '80s,

'90s, I guess. Yes. Which is exactly— we are

seeing a revival of that in the datacom world

where the bandwidth— because now the processor are

communicating, processing information a lot, but then you are not able to

communicate that to the neighboring processor or a memory. So then what's

happening? You are just waiting on the

communication to happen between multiple processors,

and it's not happening fast enough. So then your tokens are getting

limited by how fast you can communicate back and forth.

And now your entire performance is limited by how fast you can

communicate. And that is hitting a wall with copper wires.

And now with optical cables, all we are reviving

is the telecom industry's invention, which has

replaced copper wire with optical cable. And with optical

cable, photons can travel longer. And now I can, um,

and it can travel faster. So you can travel longer and

faster without needing additional energy. And now you can

increase the speed of communication by increasing number of

colors, because every color can transmit one, like, high-speed data.

So now not only can I send 200 gigabit per

second on a single color, I can also send

100 such colors. So that means I

end up getting a 100x boost in the speed of

communication, and I also get 100x boost in the

distance of communication, which means the

entire data center today can be wired

up with copper wire, with optical cables that

can make it behave as one big processor

unit rather than Chunks of small

processing units waiting on one another for communication.

I remember reading something about this. I don't know, maybe like a couple of years

ago, they were, they were talking like, if you have an optical cable

connecting CPU to CPU or GPU to GPU,

right? Because you don't have the speed limits and

limitations, like they could be across the room, right? Now, again,

there is a latency cost because it's the speed of light. But honestly, Whether

it's— for most applications, I'm sure someone will— you can

correct me— whether it's across the room or like a football field away,

the speed of light is fast enough that it's going to be

mostly negligible. That's right. It's actually—

you cannot even realize that because it's a

nanosecond. So the speed of light is such that

it takes only 5 nanoseconds over a meter. So if you're talking about a

kilometer, You're talking about a microsecond. You might not even realize

that it's taking time anymore. Right. And this

is not new. The data center already realized that this is what

they need as the distance increases. So they have been

using optical cables within the data center too, when they're connecting

multiple racks for a long distance, like 2-kilometer cable and

3-kilometer cable. What has changed with AI is

the fact that even within a rack, which was all copper cable,

that is becoming all optical cable, which means

optics is becoming a core part of the

compute. It's getting closer and closer to the processor. And

when it gets closer and closer to the processor, that division of boundary

between processor memory and optical cable is

diluting to a point where you need to treat each, like all

of them together as a single building block. So the definition

of compute is no longer just processing. The definition of

compute becomes processing, communication,

and memory. I have a bunch of questions. They need to be designed together.

Yes. I'm sorry, Candace. I don't mean to— No, no, no. Go ahead. I have

a bunch of questions. Great. First of all is, so if you take out a

typical motherboard, I build— if you build PCs, right? Or you open

up a laptop, right? You see a motherboard. you have each of the chips and

there's some kind of copper and some kind of breadboard. You're talking about replacing the

copper there with optics? That's exactly right.

That is the difference between what has been happening in the data center till now,

where there are optical cables, but they are transmitting much lower speed,

let's say 100 gigabit per second. But when you keep going closer

and closer to the processor, and if you're replacing that motherboard,

the copper wires on a motherboard with optical cable, now you do

optical cable becomes a new motherboard. So, I mean,

then, uh, then what you need is, uh, think of it

as, um, you have, uh, 2 chips right

next to one another with an optical cable that becomes your unit.

And you can think of this as like a mesh, like an optical mesh

of like a fabric of processor

units and memory units that is all tied

together. with like a multicolor fabric where you

can essentially communicate any processor to any

memory at speed of light. Effectively,

instantaneously. Instantaneously,

effectively with a pretty wide bandwidth. Exactly.

Which means the speed of communication within an internal

processor today happens at petabits of

bandwidth. Petabits is 10 to the power of 12. It sometimes— And

also over a wider distance. So you could have, you know, let's just,

let's come up with a ridiculous scenario, right? I can get like

3 or 4 old PCs and then wire them together with

optics. Obviously there's a lot of practical engineering that would, but let's put that

aside for now. I could have that as one massive

supercomputer 'cause they'd all work together. Interesting. Exactly.

Because now I'm just like weaving them together and now

the speed at which they communicate within a chip and speed at which they

communicate outside the chip is exactly the same. So then the

boundaries don't matter anymore. Right. So that's what optics bring to table is the

distance and speed that copper cannot

achieve. So now distance is a

non-factor, whether there are 2 chips at the 2 ends of the

football field or whether they're inside the same device,

they are talking at exactly the same speed. So then it views

them as you are essentially building the internet of the

GPU world where you are— we are able

to have a real-time conversation with Candace in Montreal and you in

North Carolina. And there is no, absolutely no latency.

Today, that world in the

processor world is still happening through postal lines where

there is a lot of, I'm still sending you a return letter and I'm

waiting for a post box and

USPS to like figure out where it needs to go. And I'm

still waiting for Candace and hoping that she's actually, she has

time to read that letter and then respond. And I'm still

waiting and twiddling my thumbs until that happens. There's a lot of idle time.

And this is basically, I think the easiest analogy is I'm

introducing, um, the real-time video conferencing, and that

cannot happen with copper wires. It has to happen with optical

cables. So then the next question is, I suppose since you're dealing

with photons, you can also encode quantum-level information

in how the thing spins. Do you do that too, or is for now color

wavelength is enough? Yeah, we are currently focusing on

let's just make the industry navigate the

postal era to a video conferencing era first. And then we

can really talk about augmented reality version of it, which

is the figuring out how can I encode

qubits with these multicolor lasers. And our

co-founders like Professor Alex Gater and Yoshi, they

have been doing this comb laser technology

for quantum applications for the longest of time. So they

are very well aware of the application as it relates to

quantum computing. And they do believe that the underlying

platform that we are currently building at Xscape can be

easily applicable there in terms of multi-qubit

encoding and being able to control multiple

qubits at the same time and being able to

use that as a pump for low noise

qubit generation. And I

wouldn't claim to know anything

about quantum, well enough to comment on what the

technical implications would look like, except the fact that the

application of comb lasers in quantum has been a deep

field of research for my co-founders for over a decade.

And there is a lot of interest in that field when that

becomes a reality. And once the fiber is there,

it can carry— if it can carry the same photon with or

without extra quantum information thrown into it. That's exactly right. So

you would— you're building the infrastructures, and then whatever you do on top of that,

which presumably would be maybe another order of magnitude more

information across from point to point. That's interesting. Sorry, Candace, I'll

stop hogging the mic now. No, no, this is fantastic. So

does photonics change what is computationally

possible, or does it mainly let us do the same

things faster and more efficiently? In today's data center,

it is mainly focused on efficiency and

speed. And with the applications like quantum

computing, it changes the way how computing is done.

And photonics and the platform that we are building for that is

certainly the base infrastructure on which

people can reimagine the way the computing can be done.

I still think that's going to be the next wave of innovation that is going

to ride on top of the existing AI wave.

Would the— would it run cooler

slightly? Because obviously CPU heat is going to be CPU heat.

But the fact that it seems like it would run slightly cooler, right? If

not chips themselves, the interconnecting parts would not heat up as much.

100%, because you are— you no longer need that many overhead to

communicate anymore. Right. You don't need

additional overhead on figuring out

where the loss of the signal comes from, recovering that loss,

and spending additional power on that. So,

and you don't have to necessarily worry about errors

associated with sending it through copper wire. So when you're using

optical cables, you end up like removing

a lot of additional signal recovery

circuits that consume some significant portion of the

communication power. So you're looking at roughly 10 to

20x improvement in overall energy efficiency of the

system. And in the future, you can even

unlock an additional vector with multicolor lasers as well.

Interesting. So what

happens when you combine photonics and electronics?

Rather than treating them as competing technologies?

The short answer is they are not competing technologies. I still think you need

electronics for processing the information.

And my view is today the most

efficient way of doing it is to marry electronics and

photonics and use electronics wherever you

can and use photonics wherever you must.

which is in the communication of multiple electronic

circuits at high speed and still

rely on electrons for processing information.

Even when you're looking at quantum, you still need quantum error correction

and additional circuits that are still

electronics-driven. And electronics are something that is

extremely efficient when it comes to doing math

operations and doing complicated

signal processing techniques. So

that's where the strength of electronics lies, and I will continue

leveraging electronics for that, which is information

creation, and using photonics for information

communication transfer, and using memory for

information storage. So I do think each of them have

their own strengths, and we just— the ideal platform should

use the best of all 3 of them.

That makes a lot of sense.

So if AI keeps demanding more computing power,

where do you think light becomes essential rather than simply

advantageous?

Today, at this point of time, the inflection point of

light becoming essential is happening And we are living through that

transition phase as we speak today.

The— it turns out when you are processing large

context information and when you are trying

to do reasoning applications where

the model has to think through different scenarios,

contextualize different scenarios in its head, and think

about what is the right optimum solution for a

particular query. What the AI model is

actually doing is mimicking the way humans

are thinking through different scenarios and

thinking about what is the right answer and what should I really

respond and how I should respond to a particular question and why.

And that becomes more and more complicated when you are getting a lot of

multimodal inputs. What I mean by that is I'm, I'm watching you

speak. I'm also hearing

Frank's feedback about certain technology.

And at the same time, I'm also thinking about what does the AI

model would look like under these scenarios.

And then I'm coming up with my response. And when the

level of sophistication and the context continues to grow.

In other words, as the AI model is going to grow from being a

toddler to being an educated scholar, the

experience that it is going to get and the context it's going to

continue to have when it is responding is going to evolve.

And when that is happening, it just turns out that the

efficiency with which it is able to recover information

is becoming the key factor that

makes light essential rather than a nice-to-have.

And the right figure of merit of how to

actually evaluate it comes from thinking about

how many tokens per second, which is the way

AI models respond, am I able to generate? Like, how many

tokens per second am I generating and how much of calories am I

consuming, which is the megawatts? So that tokens per second

per megawatt today is, say,

10,000x when you are thinking about very short

context length. If you are just asking what is an apple, then it can do

it at like a speed of like 10,000 tokens per second per megawatt.

But when you are asking it to process a certain slide deck and asking

you this, asking AI this question, then it is generating

tokens at 10 tokens per second per

megawatt. So as the context length is going

larger and larger, the agents are not able

to reason out in a timely manner that is economically

viable because it cannot continue to consume that much

power when it is providing that response. So the

key is for this large context application, when you want

the AI robots to behave like humans, light

becomes essential. It's no longer necessary. Like, it

is a fundamental building block that needs to be factored in

when you're designing these systems. But if you are really just focused

on one-off, like, questions on what is an apple, what is a

mango, and describe the weather today, those are all,

like, much smaller context, simple applications where you

probably don't need light. So it's really a matter of how

smart you want the AI to be. Light makes it way smarter

and way more intelligent. If you just want the models

to be like a toddler, then you don't need light anymore.

Interesting. Yeah, because I think that we're— as we go push the frontier models and

things like that, we're really running— we're outrunning our

ability to improve infrastructure. in terms of the chips. And this

seems like would be a good solution to that

problem. Yes. Fundamentally today,

that's why the optics industry is

becoming a hot topic of conversation across the

entire industry, mainly because it

is supply constrained and the demand

is outgrowing supply at a pace that the industry is not able to

meet And that's one of the reasons why the data center

power consumption is becoming unsustainable. So

there is no smoke without fire. And the way the industry

is handling it is just throwing more data centers and inefficient

cores because at the end of the day, I can only process 10 tokens per

second, but I still need to process 10,000 to get to a revenue. How

do I do it? I will just buy 1,000 more GPUs.

If I'm buying 1,000 more GPUs, now I don't have supply. Like, who's

gonna make 1,000 more GPUs? I need to install more

foundry capacity. So now it becomes a

supply-constrained world where to meet an

inefficient ecosystem design

constraints, you have to throw money and supply at it. And then you

start hitting the supply-constrained world that the industry is currently navigating.

So what advances in optics have made photonic

computing more practical today than

10 years ago? The ability to

process light-based devices, the devices

that can process light encoding and convert

electrical data to optical data. All these devices

traditionally were manufactured in a very

discrete, small volume manufacturing line.

Today, the industry can process and manufacture all of them in a

12-inch wafer, which is a silicon CMOS wafer that

the industry typically uses for electronic

consumer industry. So if you take an iPhone or

an iPad or even a Pixel phone, the chips that go inside it are

manufactured on a 12-inch CMOS wafer. And

that's why they were able to bring down the cost

and increase the supply

and being able to meet the demand of the

entire world. The photonics industry was never

able to do that until like a few years ago when

we demonstrated the ability to manufacture these things

on a 12-inch wafer. Since then, I think there has been a

clear inflection point in the ability to drive unit economics down

and drive volume up to a point where photonics has become

more viable at scale, and it

unlocks new applications like photonics computing as well.

Interesting. How do you actually use the properties

of light to perform computation

rather than simply transmit information?

Yeah, that's a good question. Today we use it for only

transmitting information, but the way to think about how do you, uh,

how to convert that for computation is to

think about how, um, what are the building

blocks of computation would look like. The first building block is

zeros and ones. Can you actually process zeros and ones?

And the— in the analog world,

like when you're talking about light, the way to think about the

equivalent of it is whether when there is light, there is

1, and when there is no light, it is 0. So

the equivalent of zeros and ones is literally light and

no light. So you can make a device that can

transmit light and that can transmit no light. And

depending upon that, you can actually process and convert that as a compute

element. Interesting. So,

there are devices that you can make where you can actually

send in signals, and depending upon the signals that you

send, either electrical signals or just optical signals,

when you're sending in optical signals, depending upon the

interference between these optical signals, you either get a

response out, which can either be a 0 or a 1.

And you can use that property of interference

between 2 optical signals as a compute element.

Interesting. And this is very similar to

how, at the end of the day, everything is a wave.

Photonics is nothing but a wave. So if you are essentially

taking 2 streams of information and sending it through a through

a device that can take both of it in, the

output of it is essentially a compute output. And

the industry has been looking at that for quite some time. Those are like some

of the fundamental building blocks of what is being used in

communication as well, and also what is being used in quantum

as well. Just with the right constraints and right inputs, you

can also make it as a compute element. I

remember reading a science fiction short story in Omni magazine. So

shout out for anyone who remembers Omni magazine. as a kid,

and they were talking about— it was set, you know, some cyberpunk type

future on it— was they were basically saying like, you know, well,

if you use light instead of like electrons,

interference is different. You have a lot more efficiency. You can pack things

more densely. And it was an interesting concept. And

I wasn't sure if it was still science fiction, but it sounds like it's less

science fiction today than it was when I was a kid. 100%. Yeah. I mean,

things are moving in that direction. And it's

just the max linear equation, Maxwell's equations

have been around for a long, long time. And the

industry has just not figured out how to make it

efficiently at scale and how to leverage the properties of those.

And now we actually have a way to unlock that.

That's always the research and the development, right? Like some

physicist comes up with some crazy, figures something out crazy. And

then it could take like 50 years or 100 years to figure out how to,

how to engineer that in a practical way. Yeah. Yeah. Yeah.

Interesting. And it's really just figuring out,

realizing at the end of the day that all these are just zeros and

ones. Once you realize that the way to think about zeros and

ones is just, and you add colors to it and you add

circuits to it and you, you do math around that. But the

way computers do math is very different than the way we do math because

everything is zeros and ones. So you just have to line them up in a

way where it meets the same outcome as we do. So

do you need new network protocols defined? Like, is there gonna be like an

802.11 something? Like, or

does it already exist? Like, how much of this— so obviously the hardware

stack changes, But how much of the other

higher-up stack? I'm thinking transport layer stuff, like

what else has to change because of this? Yeah, I mean, every—

today the electronic world has like a certain stack, and then there's a quantum

world that has a different stack. And I think the photonic computing world will

essentially be an intermediate between them. So there will be

certain level of changes, like you might end up reusing similar hardware in

the electronic world. But then you will end up like bringing in photonics

closer to the electronics and make it

look like a photonic computing hardware, which means

there are devices that is— that can do both processing and

communication and memory

storage, all with optics. And

in the future, that will get morphed into all optical, which is for

quantum computing. So I do view The transition from

the current world to the quantum computing world is where the

photonics computing is going to act as that intermediate step

to get us there. So in terms of the stack that has to change,

it's likely layers that are on the software protocol

level, like how you write the software, how you write the

communication between them. There's a chance that the communication transport layer might not

change, But there is likely

that the coding, the control logic, like which information

goes where, those need to be like rewired because

you are physically training your brain

to communicate. You are putting the structure in place saying that when you

have this calculation, go there. When you have this calculation, go

there. Which today everything is done

electronically. by encoding a packet

header. What I mean by that, think of it as, um,

an address that you write on every data saying that this is

the address you need to send this particular data to.

And an electronic circuit reads that address and says, okay,

if this came from GPU 1, I'm reading this address, it

says I have to send it to GPU 1 million. So let me send it

out, let me figure out what is the path I need to take.

Do I need to catch a flight to this particular switch? So it's

usually not the same. It might not connect into the same

country, right? It could be like the analogy that I'm

drawing is a million GPUs within a cluster can

be connected via multiple switching layers. And you

can think of those switching layers as terminals. that an

airport would take. So the message has to first go to one

switch, one terminal, it processes it, then it says, okay, I'm

currently in Newark. Now I need a way to go to,

say, Belgium. I could either take a

direct flight, but then I have to wait for one more

day to take the direct flight, or I could take one stop to a

different flight. So it optimizes based on the header

And, um, and constraints in the address also tells

you which, um, what is the latency constraint you have

to follow. So you have to follow the protocol of that communication.

So, um, then it says, okay, I'm going to take single hop, or I will

take 1 stop or 2 stops. So every information that you

do or think through when you're optimizing a route through kayak.com

is what a switch is trying to think through

in terms of where it needs to send the packet and how it needs to

send the packet. Now, it's like IP routing tables, so to speak,

right? Yes. It's going to give you the shortest path

or it is going to optimize for something else. And then eventually it goes there.

Now, if you go all optical, this electronic

packeting layer is going to get replaced with optical circuits.

Now, you still need that intelligence somewhere. And

that intelligence will end up happening on the node level

where the buyer, like, so instead of like

relying on a switch or like an airport to figure out what is the

fastest path, as a sender, I need to figure out what is the fastest

path and pick that route and send it out so that I'm

guaranteed that it is reaching the destination. So

think of this as like 2 different methods of how do you communicate communicate when

you're communicating to optics versus communicating through electronics.

You just, so it's essentially like coming up with communication

protocol is really like figuring out how to connect point A to point

B and who controls it. Yeah. Is it the airport?

Is it the networking layer that is gonna control it? Or is it the sender

that is gonna control it? So then you get into this philosophical discussion

on who should actually own it, right? As an end

consumer, would you rather rely on yourself to come up with the

best strategy to reach point B, or would you rely on someone like

kayak.com to tell you, or would you go to a different one,

right? So then, so this is the same thing

that the analogy to what is

happening in the data center today. And what is the most energy

efficient way of doing that? That's an additional constraint.

And who owns what piece of the stack and what piece of the value chain?

And those are all the discussions that are happening as we speak, which is

the control layers, the transport layer, the

communication between multiple GPOs, the communication

between one device to

another device, multiple vendors. So all these

conversations is what the hyperscalers are actively

having with vendors like us and end users like—

Anthropic and OpenAI before they have finalized the

right solution for every generation. And that's a good point because,

you know, because of the particulars of deployment,

there are probably people who have no experience with AI, so

to speak, working on these types of hardware and routing and switching

problems, which I think traditionally we would have called just networking kind of—

not network engineering, more network design. Or

protocol definitions, right? There's probably a better title for it. Yeah. Like

the people who decided what Ethernet is, right? How Ethernet should look, how it should

work. I suspect that those conversations in the standards bodies

are working on that now. Exactly. And it is happening. And now, to

your point, until now, that has been a different conversation from

who decides to make processors because they view

exactly the way you view it, which is like, oh, it's just a networking problem.

Let's, let's actually like throw it over the wall. Let me just make

the most efficient processor that can process a lot of information.

Now the conversation has shifted from being a device and

networking, like separate buckets and combining all of

them to a cluster-level definition, saying that it's no

longer your problem versus my problem. It's, it's an entire

cluster-level problem where networking is part of this decision.

Because networking owns which processor is going to talk

to who. And today it just happens that networking

choice is going to govern how fast you can think. So now

if tokens per second is determined by networking performance, you can argue

that networking is the new compute.

So that's why the hyper— the conversation is no

longer a separate conversation with a processor, separate conversation with

networking. With the AI world, this entire

conversation with networking, memory, and processing is happening at the

same time. And the winners have been those who have actually treated them as a

single entity, like NVIDIA. Sure. NVIDIA actually designs the

entire cluster as an entire system. And now to your

point, Jensen no longer talks about GPU roadmap alone.

He starts off with NVLink cluster. He talks about

NVLink as a separate business unit that is generating certain

revenue, and that has been growing 200% year over year.

And the networking component of the business is

outgrowing the processor component of the business for, in

certain, in certain companies. So what you start to

realize is when the distance becomes, when you're starting to

design an entire cluster as a unit of compute, it's no

longer chip boundaries, no longer

geared towards what you're designing on silicon. Chip boundary is that entire data center,

and everyone involved is part of the chip design.

And that is creating this huge

complex web of decision-making that can potentially

delay deployment cycles. And that's where

vendors like NVIDIA have a huge advantage because they're fully vertically

integrated. where they're not relying on so many vendors to come

together and arrive upon a decision before they can actually

agree and handshake and start the development. Right. So,

right. So the vertical aspect of integration. So yeah. So it

sounds like the future is a lot more discipline,

multidisciplinary. Yes. Kind of organizations have a significant

advantage on this. Exactly. Yeah. I remember

It must have been the keynote back in March for one of

the GC, the big GTC in San Jose. And he had

said InfiniBand, I think, or something like that, whatever it's called.

But he had said what the bandwidth was

on their networking stack for their cluster. And he said, like, you know,

it can handle all the network traffic for the internet, entire internet

for like 2 minutes or something like that. And I was like, that sounds almost

impossible. But now I can kind of see, like,

Maybe it's not impossible. And I also struggled with why would you need that

much bandwidth on a particular cluster? And then now this makes sense.

Yeah. Today, within not the— they are

absolutely right. The given cluster, one rack unit

communicates as much amount of information that the entire

internet has communicated over the last 20 years. Wow.

Yeah, it sounds unbelievable, but Yes. Yeah.

Yes. That's why it's like, I mean, the volume of

information is just going through the roof to a point where

you have to really rewire these cables to

make it really efficient. Otherwise, you are just going to continuously

dissipate a lot of power that is going to make it

unsustainable. Yeah. Wow. Yeah, I

suppose for hyperscalers in these types of businesses, This is no longer a

nice to have. I mean, this is straight up survival. 100%. Yeah. I

mean, it is— there are multiple

narratives of the data center power consumption story that you

must have read and seen. But there is certainly

one report from McKinsey that I find to be closest to reality, which

is predicting close to 10 to 12% of

the entire US power being consumed by data centers by

2030. And I actually think it's on, it's on target,

like it's actually on course for that. And that's a lot of power.

10 to 12% is a lot of power. I mean,

it's 1 out of 8. Yeah. I mean, yeah,

yeah. It's, um, I think

Bloomberg, somebody on Bloomberg was saying it was going to be 25%.

Like, yeah, I wouldn't be surprised. Like 20, 35%. Yeah. If they

don't fix it soon, power is going to become a premium and, uh,

We all might have to revisit our plan to Mars,

right? Yeah, seriously. So, but I do think

efficiency is the next frontier of innovation. That is a

must-have, not a nice-to-have, where you basically treat

power as a hard constraint and say, I'm not going to give you more than

10% because we have people to feed

and we just cannot afford to give more than 10% of

US power for these data centers. And imposing some

hard constraints as a budget would force

innovations on efficiency rather than— rather

than taking it for granted that we just have to figure out a way to

make more power. Well, it was always, it's somebody else's problem. It's

somebody else's problem. Eventually, you kind of run out of— Until

you start seeing power fluctuations in your house because there is a

data center close by. Well, there's always controversy around

electric bills going up, and sometimes that's not always the data center's fault,

right? Sometimes there's other reasons for that. But, you know, certainly

they are getting a lot of the blame, and ultimately it's going to make it

harder to get a data center approved, right? If the— Yeah, 100%, I think.

And it is also, I think it's unfair to

limit the innovation and

limit AI access because there is no power. So, um,

the, the real solution is you cannot stop people from asking, uh,

um, uh, uh, them to use AI

applications because once you start using them, it's like using internet.

So once you start using internet, you are not gonna go back. You're not going

back. Like I, it, at that point of time, it becomes a part and parcel

of your life. So the question is, uh, tomorrow if

we just said the only solution is stop using your iPhones and stop

using internet, people are not gonna like that either. So I almost think the

real solution is how do you continue innovating and providing

performance improvement and enable these AI applications

without consuming a lot of power? Interesting. While staying within this.

And there are a lot of solutions there. I certainly think to your

point, human brains consume only 25 watts of power.

And we have 300 million

humans in US, if I'm not mistaken. So it's certainly like, so all I'm,

trying to get towards the calorie— the power consumption of these data

centers can be brought down drastically, and optics is a way to do that

for sure. And the industry is starting to recognize that,

and certainly there is a lot of progress that is being

made in that direction. Interesting. I could talk to you for another hour,

but I want to be respectful of your time. Where can folks find out more

about you and about your company? We have a website,

escapephotonics.com. where we talk about our

laser products for data center application. And

they can certainly join us. They can

apply for job openings there or just reach out to us on

any product briefing that they would need or

just reach out if they just wanted to chat.

Excellent. Fantastic. Awesome. Any parting thoughts, Candace?

This was fantastic. I'm telling you, it was a brilliant

conversation. Thank you so much for your brain. It was great.

It was awesome. I won't look at fiber optic cables quite the

same way again. Totally, exactly. We'll let the outro

music play. Awesome. Thank you.

The multiverse is skanking, skanking in time. Black holes are

wailing in a horn line so fine. From Planck scales to planets, they're

connecting the dots. Candace and Frank, they're the cosmic

hotshots!

Quantum hotshots! Turn it up fast.

Candace and Frank blowing my mind at last. Quantum

Podcast, they're breaking the mold. Science has got beats.

It's bold and it's gold.