Navigating the Complexity of Operationalizing ML Models

Welcome to show 350 of data driven.

In this episode, Frank and Andy interview Chris McDermott,

VP of engineering at Wallaroo. Wallaroo helps

customers operationalize machine learning to ROI in the cloud,

in decentralized networks, and at the edge. It's

a fun conversation on MLOps and the future of intelligence systems

and model management. Now on to the show.

Hello, and welcome to data driven, the podcast where we explore the emergent

fields of artificial intelligence, Data science and of course, data

engineering, the fundamental thing that kind of underpins it all. And with

me on this epic road trip down the information superhighway

Is my favorite data engineer of all time, Andy Leonard. How's it going

Andy? Good Frank, how are you? I'm doing alright. I'm doing

alright. We just, we're chatting

in the, virtual green room about some of the logistical challenges we had,

with Microsoft Bookings and how Kind of like you can only have, you

know, like, remember that the pick any 2 triangle, right? Good, fast, and

cheap? Yep. Yep. Like, we can only have 2 things, 2 features of what we

needed to do. Right. Alright.

Despite logistical challenges, we are excited here to have,

Chris McDermott who is the VP of engineering at Wallaroo,

and, he is a passionate, and intellectually

curious professional With excellent communication skills, he

loves hard problems, then he must have definitely loved the

process to get on the show, And,

have yet to meet 1 he couldn't solve somehow. Maybe we should get you,

Chris, to help us with our scheduling stuff. Really? You visit

later? Yeah. So welcome to the

show, Chris. Thank you. Thank you. It's great to be on. It's nice to meet

you both. Well, likewise. Likewise. So you're coming to us from the,

Mile High City. That's right. Awesome place. It's, I was

there once, for internal Microsoft

conference actually. Oh, nice. And beautiful town, like, it was

just really cool. I think it was the 2nd biggest

event that in Denver history was the Microsoft thing. Wow.

And they they literally ran out of hotel rooms like it was.

Oh, wow. It was pretty wild. Yeah. I think it was, just

before one of the big parties had a convention there. And,

they Oh, yeah. Yeah. Yeah. Yeah. I was so I'm actually

slated to head back there next year for a Red Hat

conference, so we'll see Let's see if the hotel situation has

improved. I think it's improved a little bit. The city's been growing a lot. So,

Yeah. Lots of government. Isn't Denver the place that has, like,

the large bear up against the conference center that

Yeah. Yeah. Yeah. Yeah. Yeah. That's exactly right. A giant blue bear appearing in the

window of the conference center. Yeah. I was there. And,

and and I remembered that. That was the That was the first thing I

remember. It was, I was there in

2007 for a Kind of a Microsoft conference. It was

a, Professional Association for SQL

Server. That's what it was called back then. And, I was

actually the first one I spoke at. I've spoken at a bunch since then,

yeah. Like, I echo what Frank said, beautiful city

and, just very picture picturesque. Yeah.

Yeah. The weather in the mountains are beautiful. Mhmm. Yeah. And

it's funny, like, you know, on the East Coast, we talk about mountains,

but It's nothing like that. Like Right. Yeah. We quite the

same. You would laugh at what we call mountains. Yeah. Right. But

I remember a Robin Williams bit Where he said something like that

people he admired the people in Denver because they got to

Denver and they looked at the mountains and went, Well, I can't say what

he said, but he had a kind of an Elon

moment.

There's so many of those. There's so many No more. We're stopping right here. We're

not going over those mountains. So,

You're VP of engineering at Wallaroo. So tell us a little bit about Wallaroo.

Mhmm. Plus you're also ex data robot too. That that's interesting.

Yes. Yep. Exadata robot. Yeah. So I've been working in the machine

learning and AI space for, about 7 years now, I guess, or 6

years. And, it's been really fun. You know, it's, it's a

good time to be in the business. There's a lot of development

happening, very fast pace of change, which I appreciate.

And, you know, Wallaroo has been really great. Like,

the team is fantastic, and the people are wonderful. And it it's a lot of

fun working, with people that you enjoy hanging out with and and you respect

and everything, that's that's very important to me. That's awesome. But I also just

I think the product is awesome. It's really, I think,

playing well in the market. Like, we are focusing on making it as easy as

possible to deploy And manage machine learning models.

And the focus is really on any model using any framework and being

able to deploy onto sort of any architecture, any hardware,

and being able to leverage GPUs if you need them or different kinds of CPUs,

different acceleration libraries that people have tailored to the different

architectures. And, honestly, there are not a lot

of other solutions that tackle those 2 problems for people. Right.

A lot of the other companies that we're competing with, they are trying to be

like an end to end solution or, like, really force people into, you know, their

platform. So you train on their platform, you deploy on their platform, you manage on

their platform. But it's very limiting in terms of what you can bring on to

the platform and and being able to, deploy on the different types of

architectures and, platforms and things like that. So it's really

exciting. It's fun. I think that's really important that you bring up

the CPU solutions. As I've been tinkering,

you know, the past couple of years with, you know, with the

different, different platforms that are out there, it's

That's definitely a smaller market, but maybe it's emerging now. I'm

just not sure. Mhmm. Yeah. I wonder yeah. Sorry. Go ahead.

Well, I was gonna say, you know, a lot of the time people conflate training

and inferring, which is, you know, sort of the 2 different stages. Like,

1st, you have to train a model, but then you use the model to make

inferences, which, you know, it's really like asking the model to make a prediction

or you give it some input and it gives you some output. And,

they're they're very, very different tasks. And just because, you know,

like, you may wanna use some hardware GPUs for training Doesn't

necessarily mean mean that you need the GPUs when you are in production and

you're asking it for predictions. A lot of the time, you know,

The model is small enough that you really don't need to, but there's

so much hype. It it's hard sometimes to separate the hype from the, you

know, The real stuff and Yeah. Yeah. The hype the hype

machine is real. I mean, like, it's and and and I I wanna get your

thoughts on, you know, I mean, I love generative AI. I'm not

knocking generative AI, but it feels like it's taken all the oxygen out of the

room for All the other kinds of AI.

Yeah. Yeah. Yeah. Because there are a lot of, you

know, great models. I like XGBoost is a very standard one. It's

been around for, you know, a long time, meaning at least for, you know, 5

or 10 years now or something. But, that really honestly solves so

many problems, and it's such a Small, easy model to deploy.

I I wish people would focus more on on that kind of thing rather than

hype. Right. No. That's a good point. And I think you

bring up an interesting point because not all not

all AI workloads are created equal. Right? Obviously, there's,

I heard this term the other day and I had to spit my coffee out

because it was just so funny. Legacy AI. Yeah.

Yeah. There's generative AI now. There's legacy AI. That's

crazy talk. You know? And I was just like,

wow. But,

you know, because, you know, legacy AI, basically,

you're not using deep learning, you're not using neural networks,

Generally, you don't get a good boost from GPU's.

Correct. Right. And that's something that when when you tell that to

Even tactical decision makers, they they they

kinda look at you like, you know, what sorcery is that? Like, you know, because

they'll they'll They'll say, like, oh, we don't have enough GPUs. There's no budget for

GPUs. Like, what what what types of workloads are you running? And I

tell them, it's like, well, it's not really a concern for you. Like, you don't

need them. Yeah. And you see, you know,

the the the the people who are doing the actual data science, they're like, yeah,

duh, that's what we're trying to tell you. Yeah. But you see, like, the leaders

of these teams are like, like, you know,

it's, now Just for my own education,

there wasn't there something called RAPIDS, and it was an

acronym that let you use GPU's For

certain types of like XGBoost, I think was one of them. Random

forest there. I don't know. Oh. You See, it's funny because

it was an it's an NVIDIA thing and obviously it only optimizes on. But,

like, it was I remember Hearing about

everything, and you haven't heard of it. Only,

like, one ever per other person I met in the wild has ever heard of

it, and he was at the same conference I was at where we heard about

it. So I'm like, That's kind of unusual,

but, you know, we gotta watch so

fast, you know, and it's really hard to tell sometimes What

what, which new developments are gonna end up being the future and which ones

are gonna end up as dead ends? Right. You know, and even all

the transformer stuff that that is powering GPT and and those similar types

of models, I think that was originally written up in a

while, and nobody paid a whole lot of attention to it until OpenAI really

ran with it. So yeah. Pension is all you

need. I think that's was that the paper? Sounds right.

Yeah. And then we're gonna go. Oh, sorry. Go ahead. Sorry,

Andy. I cut you off your point. No. I I don't wanna go too far

downstream before I say cred boost for using the phrase I don't

know. Oh, nice. Somebody with your

credentials, you know, saying I don't know. That's that's super

cool. So cred Honestly, there's way too much to know. There's no

way anyone person could know that. I I like to joke. I

haven't checked my phone or, like, news Feed in like half an hour

and I'm like woefully behind now. Yeah.

But it feels that way like in the whole Oh, no. It does. Yeah. Especially

it was especially interesting when the whole drama on OpenAI, and I

don't wanna go down that rabbit hole too far. But when all of that soap

opera kinda unfolded Yeah. Yep. It was kind of like,

what's the latest? Like, is he back? Is he gone? Is he working at Microsoft?

Like, he did work at Microsoft for like 10 minutes, and now he doesn't.

Like, Yeah. You know, at

at some some point down the middle of it, it's like, call me when this

is over, and I'll deal with the, things yeah. I'll

check-in again. But that's just the human

side of it, let alone the let alone technology side of it.

So Operationalization. I think that's gonna

be the buzzword. Obviously, chatty b t and JennyIs, taking

all the air out of there. And I think the next buzzword It's gonna be

operationalization. 1, because it's kinda

hard to say, and I'm not gonna lie, I've had to practice.

But, it's something that I think companies and organizations

that adopt AI, whether it's legacy AI

Or generative AI. They're gonna have to realize, like, it's one thing to build

the model, and then it becomes a, okay, now

what? Yeah. Yeah. Well and models

really are just like any other software. It's not something that you just

write once and you, you know, Throw it out there, and it runs forever

without being touched. Right? All of it requires care and feeding, and

and machine learning models are no different. So, I think

part of it is, you know, how do you deploy it? And then, you know,

how do you keep that that deployment up to date, you know, getting critical

patches and vulnerability fixes and things like that. But also

how do you monitor the model and how it's performing and how it's performing

relative to the real world, Because the world doesn't stand

still right. So even if the model was trained on some data and it was

98% accurate when it was trained, as the world shifts and

and the situation around it shifts, that accuracy will

almost certainly start to degrade over time. So You need to monitor that. You need

to know when to retrain the model. And you have to be kind of

keeping track of, new training data. Right? So the

the the new environment that the model is operating in, you need to be recording

all of the the inputs and also paying attention to the ground truth of, You

know, what was the outcome of that prediction that the model made? Was it accurate

or not, after the fact? And and correlating that back into your

training data So you can retrain the models and, you know, keep them going

over time. And that's just, you know, assuming you're gonna be using the same model

forever. But as we just finished talking about new models coming out all the

time, new approaches, new techniques. So, yeah, it really is

is something you've gotta pay attention to. It's an

extremely Yeah. It's an extremely dynamic space.

Mhmm. I've heard this called

MLOps for the longest time. Mhmm. Mhmm. But I've also heard a new term

kinda pop up on the radar called AI ops Mhmm.

For this. What do you call it? I

generally call it MLOps. You know, one, I

I sort of per like, AI and ML, there's an

interesting, you know, difference there in in terms of who uses the different terms and

when they use them. For me, AI is

more of a general term that I use conversationally. And most of the time if

I'm trying to be fairly technical and specific, I'll usually revert to ML,

Because in fact, most of these things are machine learning. AI is a much more

nebulous concept, and I I don't even think everybody agrees on on what AI is

or What the threshold would be, you know, if you're

doing statistical analysis, I think most people probably would not call that

AI. But there are a lot of machine learning models that do work that way.

And and that's definitely, like, part of the gradient. You know? I've

noticed that too. Like, there it is a gradient too. Like, there's not a, like,

a hard, like, You know, typically it depends on the audience. Right? If they're

if they're BDMs, business decision makers, they're gonna use

AI. Yeah. They're technically focused people. They tend to prefer the term

ML. Yeah. That's also been my experience Interesting.

Quite often. So I like MLOps because, one,

it sort of grounds you a little bit more in that technical perspective. Mhmm.

And, and it's sort of a like, To me, I think I came up

through DevOps a lot of my, you know, first half of my career was DevOps

and infrastructure and things like that. And, I

think part of the appeal of the term MLOps is it taps into a

lot of the DevOps, associations. Right? And

Right. The concepts and the themes of DevOps, which is really about,

merging different skill sets and breaking down silos and getting different teams to

communicate with each other and And to collaborate more,

being more dynamic. Not just, you

know, putting software out there and and letting it run forever, but Keeping it up

to date, monitoring it, recording the logs, you know, all of that kind of

stuff, and and getting into a flow of continuous

deployment, you You know, continuous integration, continuous testing, continuous

deployment. And I think on the ML side, that's also where

MLOps really shines and and is bringing those themes

to the party, rather than a data scientist training a

model, deploying it, and, you know, Throwing it over the

wall to to, like, an operations team or something. It's

getting all these different teams and skill sets to work together. It's

building a continuous, you know, pipeline, with

monitoring and and feedback loops and so on. So that's that's why

I like MLOps. No. I like that too. So in order to prevent

any hate mail come in or or but actually comments, AI

ops is also used, I've heard, in,

the telcos and network operators tend to have a term

called AI ops, where they use AI to help operate their network. So that is

Got it. It's it's a it's a namespace collision,

which I've free further which I prefer MLOps for to avoid the namespace

collision, plus all the reasons you said. You

know, what's interesting is and I came from a software engineering

background and, you know, and I'll be honest, I was not

initially a big, believer in in DevOps, but

kind of as time went on, I became a convert. But I think

that, you know, when you look at how AI models, ML

models, whatever, how they get operationalized.

You look at it And I I often I often can tell

who the fans of the new Star Wars movies are by using this analogy,

DevOps being kinda like the original, episode 4 And then the

new, the the first new one, right? It's the same

plot. I mean, the characters have changed, some things are

different, But very effectively, it's the same plot. And, you

know, some people will laugh like you did, and some people I can

see will, Their their faces turn red. But,

but I mean it's like it's it's the same plat plot. The names, the places

have some have changed. But you're right. I mean, I think and there's a lot

of lessons we can learn Yeah. In the ML community

from the DevOps world. Right? Because, You know prior

to DevOps, you know, the developers and operations had a

very antagonistic relationship for the most part. I'm sure there's always

exceptions. You know, I was I was joking that they would only meet,

they only have to interact 3 times a year, and one of those was the

holiday Christmas party. You know what I mean? And

Yeah. But if you wanna deploy something in a far more agile

way where they have to, you know, you put it In some extreme cases, every

few hours, some new bit of code gets gets pushed up. That's obviously on

on one fore end of the spectrum. But for the most part, you know, a

couple times a month is not unreasonable. You have to automate that. You have to

have processes in place. Yep. And I I see a day, and if that day

has not already come, I would be surprised, That AI is gonna be the

same thing or ML. Right? You're you're gonna have to get but to your point,

right, this is a continuous process, You know? Yep. Yeah.

We can't get away with, you know, you have this isolated team of data scientists.

They they they kinda go off to their little area 51 type labs

in secret. Right. I then come back with some model,

and and I'm guilty of this too. I've done this. Right? Where I'm like, I

built the model. I'm done. I did the math. I did the hard

part. How do you get the play it? Not my problem. Not my

problem. And it's funny, like Yeah. You know, I caught

myself. Right. I caught myself doing that as I, you know, you

know, doing that. Like recently, I had to I had to do a demo

and I had to work on a kind of a It's basically a predictive

maintenance type thing, and I took all this data, had the model, and I

just said, here's the here's the link to the model, Have at it,

pal. Mhmm. And then as I sent that, I was like, you know, I should

probably be more involved in getting this on a race for it.

Right. Yeah. Yeah. Yeah. Yeah. No. I think that's a big part of it.

Another big part of it though is, scale, you know. And I think scale

scaling of compute and, how How people were using compute and how

much compute was required was a big part of what drove DevOps.

You know, if you were a sysadmin responsible for a 100

servers, That's, you know, challenging, but it's feasible. Like, you can do

that. You can keep them all up to date. You can keep them all in

sync with each other. Make sure they they all have the same patch levels and

and so on. But you scale that up to a 1,000 servers?

That gets a lot trickier. You try to go to a 100,000 or, you know,

if you're doing Internet scale things like Google or Facebook or somebody, We're talking

millions, tens of millions. And Right. That level of scale

requires you know, everything has to be automated. Everything has

to Has to work that way and it has to be resilient and it has

to, you know, have automatic fail over and stuff. You know, there's the,

x k CD where they're, You know, they get to a certain point. They're just

roping off entire data centers and being like, alright. We're throwing that one away and

moving on to the next one. And for AI, I

think a lot of the same stuff is happening. When, you know, 10 years ago

or so when when people were just getting started on this journey And as an

entity, as a business entity, if you're talking about 1 or 2 use

cases, you know, you can have humans curate that stuff and hand

craft it, hand roll it, hand deploy it, and hand manage it. But

if you're a a big enterprise company and you you wanna have hundreds of use

cases in production or thousands or tens of thousands, there's just no way.

You have to automate it. No. That that that's a that's

an excellent point. Like, one way I've heard to describe is that if you're

baking a loaf of bread for your family and friends Or loads of

bread. You can do it in your kitchen. Right? You don't have to do anything

special, but if you're the Wonder Bread Corporation or

Mhmm. And you wanna deliver at that scale, that's no longer an

option. Mhmm. And I think that we're at that point where and

correct me if I'm wrong, where I think AI and ML adoption or AI

adoption is still new enough where there's enough of naivete out

there of, oh, we don't need to scale to that degree. Like, we don't need

the production line. I think I think that's ending. I think we're getting close to

the the end of that era, but that's kind of been my yeah. I think

so too. Yeah. Because they're they're more and more, ML

tools in everybody's toolbox. Right? So you were talking about telcos

routing network traffic using ML models. That's not

gonna be 1 model. Right? Like, with latency and and

everything else, you're gonna need, you know, Very small. Lots

and lots of very small models deployed on every router, every top of rack

switch, every, you know, whatever 5 gs cell phone tower,

whatever you're talking about. There are a lot of cell phone towers. So you're

not managing 1 model. You're managing a fleet of models, right, across

different geos and all kinds of things. No. That's that's an

excellent point. Sorry, Andy. That's okay. It does seem to scale like that,

though. Right? It's almost it's almost tectonic. There's

a whole new layer going down. You know? That's that's the new surface.

I noticed on the website, I I popped over to wallaroo dot,

aiwallar00.ai.

And I noticed a familiar looking, blurb just

below the top of page. And it's familiar to me because, I

started off in business intelligence. I'm still working in BI.

And there's a note, 90% of AI

initiatives Failed to produce ROI. And I saw this

in, you know, it's very similar number, 85% in,

in BI back in the day. It's probably still true. So where

does that number come from? Well, I think it reflects a lot of

things. You know? Some of them we were just talking about and

and where MLOps is coming from is is, a lot of the failure

modes were teams not really working with each other. Right?

Somebody decided we should be doing AI, so they hired the data scientist.

And the data scientist works in the corner for a while and,

You know, 1, they don't have access to all the data. They don't know what

the data is, where to find it, how to access it, how to clean it,

what it means to the business. There there are a whole set of challenges there.

And then, you know, they may train some models and and get something, you

know, to a point where they think that it's gonna solve a problem. But Then

you've got to work with an IT organization to stand up infrastructure. You've got to

work with somebody to package the model and build, you know, an API around

it or a UI of some sort And figure out how to deploy

it, train people on how to use it, and and actually, like,

somehow integrate it into your business process So that it's

it's driving business outcomes. And all of those are really tough

challenges. And all of them require breaking down those

silos and getting a bunch of different People within an organization to

talk to each other and communicate and to work together to solve something.

I don't think ML or AI is is a magic wand that you just

wave and magically provide value to a business. You've got to really

think about What is your business doing? And, you

know, machine learning at at heart, it it's

really just like a a more efficient way of

Making decisions, you know, faster and more accurately,

and with less human input. And so you've got to look for places where your

business can either save a lot of money or make a lot of money by

being able to answer a a simple question repeatedly very,

very efficient. And that sounds easy, but in practice,

defining a business problem is often one of the hardest parts. So now I'm

seeing even more parallels. Uh-huh. Yeah.

You know, that was the problem we were trying to solve, with

business intelligence as well. So didn't mean to cut you off. Sorry about

that. No worries. So I yeah. I think I agree with you. It it there

are tons of parallels there. I think there are a lot of similar lessons to

be learned, and I think we are applying them in this In this space in

ways that we've applied them to other spaces in the past. I also

think there are technical challenges. You know, part of it is the field is moving

so fast. So there's just this constant stream

of of new frameworks, new models, new techniques, and you

have to kinda stay on top of that. You have to be careful with your

tool selection, to make sure you're not, you know,

going whole hog into some tool. That sounds

great today, but it's just not flexible, and it's not gonna be able to support,

like, all these new things that are coming out. Yeah.

Or that company could have internal internal political

strife, which was crazy talk. Right? Cast Absolutely. Right. Cast

doubt on their future. Alright. That would never happen. That would never

happen. Sorry. Yeah. You were talking about privacy, which I think is

another key thing. Yeah. Data residency, data privacy, see data

security. You know, all of those things matter tremendously.

And for for a business trying to, get

value out of AI and ML. You know, a lot of it, depends on

having good data and, Cleaning it and curating it

and getting it ready for things. But then it it forces the

the organization to really kind of do an inventory. What do we have? What's useful?

What's not useful? Well, how much do we store? How much do we not store?

How do we comply with various regulatory

environments? Right? GDPR is is the big one everybody, you know,

loves to throw out there. It's it's big and it's complicated, but, you know,

that. They're covered or whatever. I think that, you know, that that

is not only do they have a big stick, but they have a big arm

that they can wave that stick wet. Yes. You

know, if if a small country with, like, you know, 50 people in it, and

that could something like GDPR, people would just walk around it. But I think

that, a block with I've heard different numbers, but

it's for, you know, pushing 4 to 500,000,000

people. That's a huge that's a big enough market nobody can really ignore.

Yeah. What's interesting is on the LinkedIn page

for Wallaroo I love the website, by the way. I checked that out too. Thank

you. It talks about decentralized

networks Mhmm. And at the edge. Yes. What how would

you define decentralized network? Yeah. This is a big new push for us that we've

been focused on for, I mean, we've been focused on it kinda for the

last year, but it was a lot of, development on on the back end. And

we just released kind of our 1st edge features and product,

in October, so it's kind of a new thing for us. But,

As you think about ML and edge or ML and AI,

and the the fleets of models that we talked about and all these use cases

And, you know, telcos and and five g cell phone towers and all of those

types of things, intersecting with data and data

residency and privacy and security, It it really seems to

indicate to me and and to us at Wallaroo in general that the

future is lots and lots of models being deployed in lots of

locations. And I think that one

big sort of industry wide theme that I'm seeing is if the

last 20 years, let's say, was the story of Everybody

picking up from their colos and moving to the cloud and centralizing

all of their IT, I think that the next 20 years are gonna be

Not like deconstructing the cloud. I think the clouds are here to stay and they're

gonna continue to grow, right, year over year. But there will be more

of a push out to more edge computing environments. Cell phones

are getting more and more powerful. Cars are getting more and more powerful. Like, there's

more computer stuff happening, all over the place, and the compute

available, the memory and the storage available is all through the roof compared to

what it was 20 years ago. And, I think we're

gonna see more push for smaller, more specific machine learning models, And

they're gonna be pushed out to all these edge locations so that they can run

close to where the data is. So you're not schlepping this sensitive data all over

the Internet and other people's networks. Yeah.

But, you know, you're taking advantage of of compute resources that you

have local to the data and making very fast decisions,

you know, very efficiently. So I I have to jump in

because, you you just made me feel really good.

About a year ago, I built a large server here

at home, which I hadn't done in a decade. Actually, my my

20 year old son built it. But he and he helped me with,

with picking out the new shiny fast parts, on it because I was

so out of practice with this such confessing.

But, and it's really cool to see, you know, all of his All of

his skills. He does edge. We just picked up the

Raspberry Pis are back in stock, finally. Yep. And I just picked up,

like, 3 for $35, You know, the 1 gig force.

Yep. Anyway, super excited about that. One of the things I built

at the time I built a box About a year ago, you

couldn't do a local GPT or anything close

to that. And I said, Eventually, we're

gonna be able to do this. I I made that guess, and it was a

guess. Yeah. But about 6 months later, about 6 months

ago, All of a sudden, I started seeing these 7,000,000,000

token machines showing up and it started clicking.

It was like, holy smokes, you can do this. I did make one stupid mistake

and he didn't catch me on it. I bought a 12 gig GPU

because that's super crazy huge From 10 years

ago. And that wasn't super crazy huge at all. No. No.

No. But it's interesting. Now they're back now. They can run on, You know, on

the 12 gigs. And like you said, you mentioned the CPU models. So I just

learned a ton as I've been going through this. And, That

it's it's very encouraging to hear that. I had not heard anybody

say edge and running small ML models on the edge.

That's, I mean, that's what we've been trying to do here. And I I love

the redundant you know, the idea of a redundant array of whatevers,

you know, MLs. It's almost like a swarm of MLs. I've heard,

yeah. Yeah. Yeah. That's true. Right? And, you know, I think there's a lot

of interesting stuff happening on the battlefields in Ukraine right now drones.

And Right. That Yeah. Was also a fascinating space and

very much, I think, heading in the direction of lots of ML running at the

edge. It's it's funny you mentioned that. So I live in a DC area,

and, I was at a government tech

symposium about 2, 3 weeks ago now. And

they were talking about that that, you know, edge is gonna be much more important

in the future of warfare. And he said presumably

elsewhere too. Right? He was permanent primarily a government in defense. It was definitely a

military industrial complex, type of type of event. But he was

explaining like, you know, in the past, you know, 20 years,

we've not dealt with adversaries. We've

only dealt with adversaries in in battle space conditions

where it was, you know, we controlled the airwaves.

Mhmm. And he, I think he used an interesting term. We

had airspace and electromagnetic electromagnetic

dominance. I was also like, Wow. Yeah. That was yeah. Yeah. I was, like, oh,

that's interesting. So, like, the whole idea of these disconnected

decentralized networks, I mean, I think

you're I think you're spot on. It's the future for

geopolitical reasons, but also just for, you know,

Privacy and just kind of flexibility reasons. Yeah. The

question I have though is, like,

Organizations can barely manage the infrastructure they have now and barely manage

the software they have now. What are they gonna do when the software starts Not

thinking for itself, but, like, this becomes another workload Yeah. On

top of that. Like, what Well, for one thing, that's why Wallaroo It

is focused where we are, and we're trying to build this platform to help people,

you know, with this capability of being able to deploy models and manage a fleet

of them at the edge. Because, yeah, there aren't a lot of good

solutions for that today. Yeah. Interesting. I I think the

general answer to your question is probably some combination of cloud and edge.

You know, like, it does make sense to centralize a lot of things, and it

makes the the maintenance easier and, more efficient. And

You can get some economies of scale and, you know, all that kind of stuff.

But, we are gonna have to get good at managing a bunch of,

disparate types of things in desperate locations. I think all of

us. Interesting.

So this is the part of the show where we'll switch over to

The premade questions, and for your convenience,

I will, paste that in here.

Hopefully, paste it. And there we go.

So You had an interesting career looking at LinkedIn. You were at

SendGrid. You were then you were at DataRobot, and you said you made a switch

into the the data world, which begs the question, How did you

find your way into data? Did data find you or did you

find your way to data? I I

guess that is a good question. I think that, it was probably a

little bit of both.

Finding my way to data, I think that the beginning of the story is probably

at SendGrid. And I joined SendGrid as a DevOps engineer.

And to be honest, I had not really heard of SendGrid at the time. I

knew a little bit about it, but it, you know, I didn't really understand what

it was, too much with the scale. SendGrid, by the way, is now owned by

Twilio. But they have an API for sending email, and

they make it just really easy to integrate with, websites and applications

and and software so you don't have to worry about SMTP and, you know,

DKIM signing and all the other, like, gnarly bits of of

email. Turns out that Sengrid had a

ton of data. They're handling billions of emails a day,

and, you know, there's a lot of metadata there. The the actual data of the

email and and so on, the recipients and who to send it to and all

that stuff. And so working in that space,

I was dealing with tons and tons and tons of data. I mean, we

had, we were using mostly MySQL, and we had these

massive massive clusters. I think we had,

like, 30 or 40, you know, schemas under management. Each

schema was a cluster of anywhere from, Like, 6

to 40 plus servers, Wow.

You know, with lots of compute and everything else. So that was probably my

1st foray into, like, really thinking about data as a first class

citizen. And, and even to the extent of, like,

You know, building an architecture around the data. Right? So

that you can optimize the flow of the data, and being able to store it

and process it and transmit it fast enough to keep up with, with the

flow. And so, yeah, from there,

you know, had a lot of fun, learned a lot of things about, startups

about industry, about, DevOps and and all kinds of

things. Management as well and leadership because that's where I first,

started managing teams And then moved to data robot and,

into the ML space. And then it was a whole another learning journey

about, you know, data,

engineering, feature engineering, transformation tools. How do you

curate your data? And how do you really, like, know what you

have and inventory it and, make it available

to people within the business so that they can get value out of it.

Interesting. Very much. So our next question

is what's your favorite part of your current gig?

I think it's actually, I'm gonna cheat and I'm gonna say I have 2 favorite

things. And I I kind of always have I I

Figured out this formula a while back, in terms of what

motivates me. And it's one part the people that I work with

and another part, the problems that I had yet to solve.

So I wanna work with smart people. I I really don't like being like, feeling

like the smartest person in the room. I much prefer to surround myself

with people that are smarter than me and I respect and I can learn

from. But that also, you know, I enjoy. Right?

We spend a lot of time at work, so it helps to to enjoy the

people that you're working with. True. So that's a big part of it. And

then, finding tough problems, hard challenges. You know, if I

don't have hard challenges to keep me, to keep my mind

engaged and occupied, I start to get bored and, that's no fun. I

prefer to to always have something new to to to, you know, be chewing

at. So, yeah, good people, smart people,

and hard challenges. That is that is really awesome. I feel the

same way about about both of those things. The, for me though, I

I, Trying to find people that are smarter than me is

really easy. So I I enjoy that part a

lot. Like Frank. Frank is smarter than me.

Well, thank you. So

we have a couple of, complete this sentence, questions.

The first one is, when I'm not working, I enjoy

blank. When I'm not working, I enjoy

reading. I Enjoy movies. I go biking sometimes.

That's part enjoyment, part exercise. You know, it's good for me, but,

There's a lot of good, road biking in particular around Denver and a lot of

beautiful scenery. So you can, you know, just ride for a while and find yourself

up in the mountains or something, which is great. Yeah.

Traveling, cooking, all these things are good.

Our next fill in the blank is I think the coolest thing about

technology today is blank.

I I don't think it's necessarily something about today, but I think the coolest thing

about technology is how it builds on itself. I remember

Years years ago, I was studying for the CCNA exam, and

that was such a formative moment for me to

suddenly understand How networks worked all the way

from the physical, you know, sending

electricity down a copper wire, and it can be on or it can be off.

And that's it. Right? And you can do that really, really fast. Switch from on

to off, on to off, on to off, all the way up to,

web 2.0 and and Ajax and, you know, Asynchronous

JavaScript stuff happening in Google Maps. Right? And I can just drag my map

around. It's just mind blowing. And, honestly,

like, that That journey from the zeros and the

ones up to Google Maps, that was, you

know, what, 50, 60 years of,

technology building on itself of people solving very small simple

problems, but you add up all those small simple solutions and you get

something incredibly complex And absolutely mind blowing.

Excellent. Very interesting. The last, the 3rd and

final, Complete the sentence. I look forward to the

day when I can use technology to do blank.

I I would love, a Personal assistant, you know, like

Jarvis from from Marvel Comics or something or, I don't know,

from I I'm big into sci fi and and things like that when I read.

So, there are plenty of examples, but some kind of a smart personal

assistant that, you know, I can chat with and it keeps track of my calendar

and reminds me of appointments and, you know, when to call

my dad and whatever else, stuff like that. I just think that's

so cool. And I don't you know, with Especially with all the new LLMs

and and GPT stuff that's happening, I don't think we're super far from that. So

it's kind of exciting to me. No. You're right. Like, I

you know, if you watched, you know, when I was a kid, Star Trek next

generation was on, And the way that they were able to interact with the

computer just through their voice. Yep. And I mean, the 1st Star

Trek show had that too, but, like, the way the conversations I thought were more

richer and more kinda interactive. Mhmm. Mhmm. We

have a lot of that now. Yeah. I think some of the fundamental pieces are

in place now. Yeah. It'll probably take a little while to put

them all together and make it work right. But yeah. Agreed.

So our next one is, share something different about yourself.

But we, always remind guests that we're trying to keep our clean

rating. Yeah. On Itunes. So

I don't know. I think one of the more interesting things about my

Journey is that I don't have a background, like a a degree

in anything technical. I went to college and I got

my undergrad Studying Greek and Latin and classics. And

so it was mostly history, archaeology, languages, and things like

that. And Computers have always been a hobby of mine and and I

definitely did some computer science stuff in high school. I took 1 or 2 classes

in college, but I didn't really make my way into that

Professionally until a few years after college.

And, you know, honestly, I I don't think it's hurt me at

all. And in many ways, I think it's helped me partly

because, you know, it it helps a lot with management and leadership, just

to To kind of have a broad background and and understand, you know,

different people and perspectives and and where they might be coming from.

And I'm sure that some of the languages, you know, studying languages helped me

picking up computer languages as well. I think there are a lot of similarities in

In, human languages and and computer, you know, programming languages. But

What? Yeah. But, yeah, it is somewhat unique, and I don't run

into too many other classics majors, At, you know, tech startups.

I could definitely see the convergence, especially now when we're talking about

LLMs and the like. Right. You know, the the

nearest neighbor algorithms and all of that that are that are being applied

because my understanding is that's that's, You know, that's how that

works as it picks the next best word Right. You know, in a in a

sentence. And so syntax and grammar and all of the things you

studied in-depth, That should be very helpful.

Yeah. No. That that's awesome. There

is that good value in,

like a classics education. I I went to Jesuit

High School and Jesuit College, you know. Mhmm. I was forced into studying Latin

and things like that, like, didn't do it voluntarily. I'm not gonna

admit that, not do that. But but like as I get older, like, it's

definitely like, Oh, I get this. Like, you

know, especially when dealing with a lot of lawyers, there's a lot of Latin in

that. And so I'll hear them, like, you know, Excuse some

words. I'm like, I think I know what that means. Yeah.

Audible sponsors data driven.

And you mentioned you read a lot. Do you do audiobooks and

sci fi? Do you have any recommendations? Yeah.

There was a really good book that I read recently. Like, this is maybe

a year ago or something, but, best book I've read recently.

It's, The title of the book is called Seeing Like a State,

and it's by, James c Scott. The the longer

subtitle is, something like how Some

schemes to improve the human condition have failed or something like that. But,

it talks about this concept of legibility and how a lot of

The developments over the course of the enlightenment, the industrial revolution,

and, in the last few 100 years in in

Our society have been primarily

driven by the centralization of power in states

And the state needing to administer all of these people,

taxes, lands, land ownership, and all these different things.

And, you know, as part of, like, the the enlightenment, the

scientific revolution, we all got very enamored with, like,

rational thought and Logic and and all of this stuff. And

we thought, we're understanding the principles of the universe. We can predict

the motions of the planets and all these things. Well, we can solve all these

problems about, you know, around human civilization and humans as well.

And in a lot of cases, it failed. Right? And we didn't know as much

as we thought we did. And one of the sort of basic,

like, premises of the book, I guess, or arguments that it's trying to make is

that we routinely Underestimate, the

complexity of the natural world and how necessary it is.

And we think we can Simplify things and strip out all these

variables and go, you know, monocultures in our in our agriculture,

for example, and do industrial scale agriculture. You need

timber for building ships. Great. We'll just plant Norwegian pines in straight

rows. This is gonna be great. It's so predictable. We know exactly what,

You know, an acre of that will yield after 10 years.

But it turns out you can't strip out all the variables because the whole thing

falls apart. You need the complexity of the ecosystem to keep all those trees

healthy. And so all that predictability you thought you had

disappears within a couple of generations because, it can't

sustain itself. Wow. So, anyway, it it's a very, like,

complicated book. I'm not really doing it justice,

but I definitely recommend it. Interesting. It's on Audible.

Yeah, yeah, so definitely check it out.

The show. So if you go to the date is ribbon book.com,

you'll be routed to an Audible page. And if you choose to get a subscription,

to Audible. You will give us

you'll get a free book, and then we'll get like a little bit of a

bump on the head, and pat on the back, and Probably enough to

buy a cup of coffee. It started Which will share. Which will

share. Yes. Yes. And the final question,

Where can people learn more about you and Wallaroo? And they even

made that rhyme. Yeah. Great. I

think the best place to go is the Wallaroo website, which, as

Andy mentioned earlier is wallaroo.ai. So wallar00.ai.

And we've got a ton of great stuff on there. Lots of, you know,

documentation and and white papers and, tutorials and things about the

product and what we're doing there. And for myself,

I'm on LinkedIn. That's probably the easiest place to find me, Chris McDermott.

And, I think I even have that as my, like, LinkedIn

Profile name or whatever sits in the, you know, in the URL.

Cool. It is, actually c s m

c s McDermott. Okay. Well, thank you. Close. I was just looking at

it, and I was also looking at the website. It's a very nice website. Thank

you. Great design. And, although I can't design

great websites, when I look at one, I can tell whether it's great or

not. Me too. Me too. Same boat. I can't do it myself, but I definitely

appreciate it. I I can't cook, but I appreciate a good meal. There

we go. Yeah. That's it. And with that, we'll let

Bailey finish the show. Thanks, Frank and

Andy. And thank you, Chris, for putting up with our broken

calendaring system. Satya should really look into that

now that the drama around open a I is over.

Well, over for now at least. Maybe g p t

five can fix it.