Skip to content
Follow us
Exploring Machine Learning, AI, and Data Science

Would You Trust Your Data Scientists With Your Kubernetes Cluster?

|

rank La Vigne, technical marketing manager at Klique, takes the KubeCon stage to talk about GPU control and token control for the SUSE AI Factory. You may not know the name Klique yet — it’s ClearML, same DNA, same software, evolved for modern AI implementations, sitting on top of the foundation SUSE already gives you.

Walking the expo floor, Frank kept hearing two themes: cost control for token spend, and how you integrate AI into Kubernetes. Klique’s answer is governed self-serve access, with four pieces: an AI development studio for practitioners, an orchestration engine so admins can manage GPUs, CPUs, and cloud APIs on one screen, a governance engine with spend visibility and auditability, and an AI engine that can sit in front of every model with spend control behind it. Enable, control, publish, and measure — for every user, AI actor, or agent.

SUSE AI Factory is built for managing the hardware, in a way that’s comfortable for Kubernetes engineers. Klique runs on top of that and talks to the AI practitioner. The question Frank would ask IT ops: would you trust your data scientists with access to your Kubernetes cluster? The answer should be no. That’s not their wheelhouse. It’s the same GPUs and the same hardware, viewed through two lenses — one for the platform team in SUSE Rancher, one for the AI teams. That turns the cluster into pools, policies, and models. The end user sees a well-defined endpoint. Admins keep governance over what people can do and what they can spend, so you’re talking to Dana before the surprise bill shows up at the end of the month.

The LLM gateway is one governed entry point to every model. You can change keys and models to adjust cost without sending a note to every developer to update an endpoint or an environment variable. Spend controls include cost quotas, wallet budgets, and budget metering, authenticated through the gateway and metered per session. A gateway alone, Frank says, is a commodity; tying it to cost, budget, and metering is the point.

In the live view, a tenant is at 62% of their allocation. Everybody’s green except Dana, who’s spending all the tokens. Maybe Dana has a good reason. Now you know who to ask — in a friendly way — instead of reverse-engineering it through financial alchemy when the bill arrives.

This episode is the audio from Frank’s talk at KubeCon. The video is linked in the comments. Frank also pointed people to his LinkedIn if they want to talk more about Klique.

Transcript
Lana:

Welcome back to Data Driven. The following is the audio from Frank's talk

Lana:

at KubeCon. You will find a link to the video in the comments.

Lana:

Enjoy

Frank:

Well, hello, my name is Frank La Vigne, and I'm a technical marketing manager at Klique, and

Frank:

we're today gonna talk to you about GPU control and token control for the SUSE AI

Frank:

Factory. You may not know us as Klique, but we are ClearML, and the

Frank:

same DNA, the same software, all of that has been evolved to address the

Frank:

modern challenges of artificial intelligence implementations,

Frank:

right? Everything we did for compute, we now do it for tokens and GPUs

Frank:

as well, as well as an agentic story. We have an LLM gateway

Frank:

that allows smart routing across models and clouds. We

Frank:

have token governance. This is gonna be a very hot topic. As I walk through the floor here,

Frank:

yeah, I notice two themes. One, cost control for token spend,

Frank:

and how do you integrate your AI into Kubernetes, right? That is a very hot

Frank:

topic. And we provide governed self-serve access for employees and your developers.

Frank:

So think of it in kind of four ways that we provide. We have an AI

Frank:

development studio for your AI practitioners. We have an

Frank:

orchestration engine, which this would be the ones that the admins that manage your GPU,

Frank:

your CPUs, and cloud APIs all in one screen.

Frank:

We have a governance engine that allows you to understand not just who's

Frank:

spending what, but you also have auditable ability. And we have

Frank:

the AI engine that has every model you can connect to, you can

Frank:

hide it behind this with spend control and governance behind it.

Frank:

So think of it as we enable, control, publish, and measure for every user,

Frank:

AI actor, or agent. We really bring it together.

Frank:

And we sit on top of the SUSE AI Factory.

Frank:

SUSE gives you the underlying foundation, and we sit on

Frank:

top of that. This is we provide services, governance, and orchestration.

Frank:

This is a bit of an eye chart, but the short of it is, is that

Frank:

we provide an extra level on top of SUSE AI Factory. SUSE

Frank:

AI Factory is really great for managing your hardware and really something that's comfortable

Frank:

for your Kubernetes engineers, but we run on top of all of that. We provide

Frank:

services, governance, and orchestration in a way that makes a lot more sense

Frank:

to the AI practitioner. Here's another tree graph to show like what

Frank:

this ultimately looks like all put together. We support Rancher, Kubernetes Engine,

Frank:

Linux x86 server, and the AI Factory, and it's really... You

Frank:

can deliver this as a partner AI Factory blueprint. So

Frank:

you have the ability now to just simply just install it from an existing

Frank:

blueprint. We leverage all the underlying great stuff that

Frank:

SUSE has built, and we make it even better So you're probably thinking, well, if

Frank:

SUSE AI Factory is so great, and it is, why would you need

Frank:

Klique? Why? Why? Well, that's a good question because we really talk about

Frank:

managing the AI stack, not just the Kubernetes. And as a

Frank:

AI engineer and data scientist myself, it took me a

Frank:

while to even learn how to spell Kubernetes. So ultimately,

Frank:

most data scientists and AI engineers have no interest or time

Frank:

to get really deep into Kubernetes. And the question I would ask

Frank:

for a IT ops and engineers is would you trust your data

Frank:

scientists with access to your Kubernetes cluster? The answer

Frank:

should be no, 'cause that's not their wheelhouse. That's not their center of

Frank:

gravity. So what we provide, this is the same GPUs, the same hardware, but

Frank:

viewed through two different lenses. This is for the ... This is in SUSE Rancher.

Frank:

This is for the platform team and this is for the AI teams with DataG.

Frank:

We really provide a different environment. Now, what's

Frank:

really cool about this is this turns the cluster into pools, policies, and

Frank:

models. Short version is you configure it ... You configure a series of pools of

Frank:

GPUs. Yeah. And then you set resource policies that connect to that.

Frank:

So at the end of the day, the only thing the end user would see would

Frank:

be a well-defined endpoint structure. Yeah. And then you have absolute governance

Frank:

over what they can and cannot do, what they can and do not spend,

Frank:

which is increasingly becoming important. So you won't get that surprise bill at

Frank:

the end of the quarter, end of the month, saying how much was spent,

Frank:

and you know who to talk to well before it becomes a crisis.

Frank:

The control plane exists as one pane of glass.

Frank:

You have ... You can orchestrate, govern, and provide an A-

Frank:

AI engine. So you really have this whole command center for this,

Frank:

and it really is the same console for enterprise IT teams, multi-tenant by design,

Frank:

and it has We have hardened FIPS. If you're in the public sector space, you probably

Frank:

know that acronym, an external Kubernetes management system. So we really work

Frank:

with anyone. Any Kubernetes, of course you would use ... Well, I don't know why you

Frank:

wouldn't use anything but SUSE-branded Kubernetes, but different strokes for

Frank:

different folks. And really, the, the LLM gateway I think is really also a key

Frank:

part of our technology. You have one governed entry point to

Frank:

every model. So this way, you can route, you can set up routing pro-

Frank:

profiles, you can smart route resources, and then these are the

Frank:

actual endpoints. This lets you change up keys and different models that you use

Frank:

to adjust the cost, all without having to send a note out to all the developers to

Frank:

change your endpoint, right? Change your environment variables. Developers don't like that.

Frank:

So we provide that, make it easy. The other thing is the spend. The

Frank:

other big theme here is managing your spend. We can set up

Frank:

cost quotas, wallet budgets, budget metering, and through the

Frank:

gateway, we then have protected, governed, and authenticated

Frank:

ways to get into this environment that is metered per sess- per session.

Frank:

So really a full control over this. A gateway by itself is

Frank:

just a commodity at this point, but now we tie it to cost, budget, and metering.

Frank:

So this is what it looks like in the real world. We'll see ... I'm almost done, I promise. I

Frank:

promise. This is what it looks like. This is the tenant. They've used

Frank:

62% of their view, and you can see this in more or less real time. You can see that

Frank:

this ... They're at this level, they're at that level. So everybody's green except for

Frank:

Dana. Dana's who's spending all the tokens. You can have a chat with Dana,

Frank:

or maybe Dana has a good reason for doing it. We don't know. But we

Frank:

know now who to talk to in a friendly way. "Hey, Dana. You're spending a lot of

Frank:

tokens. Is everything okay?" Right? So now we have that kind of visibility

Frank:

as opposed to trying to reverse engineer that through some kind of financial

Frank:

alchemy at the end of the ... When the bill shows up. So thank you for listening. My

Frank:

name's Frank La Vigne. I'll be answering any questions you have. Thank you. Thank

Frank:

you. Thank you very much, sir. Thank you.

Frank:

Great partner. Great partner.

Frank:

And if you wanna know more, there's my LinkedIn. I'll be happy to talk about

Frank:

Klique. It's a quite- Thank you. Okay, so I think it's time for

About the author, Frank

Frank La Vigne is a software engineer and UX geek who saw the light about Data Science at an internal Microsoft Data Science Summit in 2016. Now, he wants to share his passion for the Data Arts with the world.

He blogs regularly at FranksWorld.com and has a YouTube channel called Frank's World TV. (www.FranksWorld.TV). Frank has extensive experience in web and application development. He is also an expert in mobile and tablet engineering. You can find him on Twitter at @tableteer.

Tagged