Kevin Latchford on the Security Risks of Large Language Models
In this episode, we explore real-world cases that showcase the susceptibility of AI chatbots to manipulation, as illustrated by a shocking incident where an AI was manipulated to sell a Chevy truck for just $1. Kevin Latchford sheds light on the dual-use knowledge risks and the potential for unauthorized leaks and malicious backdoors within AI plugins.
Frank and Kevin dive into the implications of quick technological adoption, drawing parallels to the early web era. We discuss the impact of network setups, access controls, data supply chain integrity, and the ongoing investigations into the security implications of these burgeoning technologies. This episode is packed with expert insights and practical advice on navigating the complex world of AI security.
Show Notes
05:04 Public space tech meant to have safeguards.
09:39 Security issue in enterprise AI adoption concern.
12:53 Understanding security implications is crucial for mitigation.
16:40 Chatbot manipulated to sell Chevy truck for $1.
17:57 Found something during cybersecurity exercise, not sharing.
21:11 Uncertainty about security in remote interfacing.
24:00 Utilize specialized LLM to analyze prompts precisely.
29:15 Understanding cybersecurity first is key to AI.
32:32 Implement outbound stateful connection to prevent automatic calls.
34:31 IT field is interesting with its vulnerabilities.
37:15 Data-driven podcast highlights AI security vulnerabilities. Stay vigilant.
About the Speaker
Kevin Latchford is an esteemed expert in the cybersecurity realm, renowned for his comprehensive understanding and proficiency in both offensive and defensive strategies. Drawing from concepts rooted in military practice, Kevin adeptly navigates the intricate dynamics of red teaming and blue teaming. As an advocate for offensive cybersecurity, red teaming, also known as opposing force operations, he challenges the vulnerabilities within systems to enhance their integrity. Conversely, his expertise in blue teaming, the defensive counterpart, focuses on shielding and fortifying friendlies. Through his dedicated efforts, Kevin ensures the confidentiality, integrity, and accessibility of computer networks and systems, whether they are natively hosted or web-based, culminating in fortified cyber defenses and resilient information security.
Transcript
Ladies and gentlemen, welcome to another riveting episode of the
Speaker:data driven podcast. Today, we're diving into the
Speaker:fascinating and sometimes terrifying world of IT security.
Speaker:Joining us is none other than the formidable Kevin Latchford, an
Speaker:expert in safeguarding our digital lives. We'll be discussing
Speaker:the vulnerabilities of large language models. Yes. Those clever
Speaker:algorithms behind chatbots and virtual assistants like yours
Speaker:truly. Are these digital wordsmiths a blessing or a
Speaker:potential security threat? Stay tuned as we unravel
Speaker:the secrets and risks lurking in the code.
Speaker:Hello, and welcome back to Data Driven. I'm your host,
Speaker:Frank Lavinia. And while Andy is out
Speaker:playing on vacation, I had the opportunity to invite our guest,
Speaker:Kevin Latchford, who recently spoke at the Northern Virginia
Speaker:Cyber Meetup on securing large language
Speaker:models and the most pressing exploits that are out there.
Speaker:What really got me interested in this is that I saw a paper, I think
Speaker:it was published by NIST, talking about vulnerabilities and red
Speaker:teaming against large language models. So welcome to the
Speaker:show, Kevin. Great pleasure to be here.
Speaker:Awesome. Awesome. So for those that don't know, I kinda know what red teaming is
Speaker:because my wife works in the security space. But for those that are not necessarily
Speaker:familiar with the term, what is red teaming versus blue teaming?
Speaker:Well, red teaming versus blue teaming is basically it's,
Speaker:basically in military parlance that we called opt for, the opposing
Speaker:force. The opposing force often is called the red
Speaker:force. Blue force is your, friendlies.
Speaker:And, basically, this is offensive cybersecurity,
Speaker:whereas blue teaming is is defensive
Speaker:cybersecurity. The tools are different. The
Speaker:methodologies are the methodologies are different, but they come together for a common
Speaker:purpose. The common purpose is the assurance of the
Speaker:confidentiality, the integrity, and the accessibility
Speaker:of a computer network, computer system,
Speaker:application, whether it be natively hosted or web.
Speaker:Interesting. Interesting. So we're not you you know, we talked
Speaker:in the virtual green room. People don't think of
Speaker:LLMs as a major security flaw. And I think that
Speaker:I find that a little dangerous, and I think you're gonna tell me it's very
Speaker:dangerous. Well, it could be quite it could be quite dangerous, you
Speaker:know, to the point of, you know, frankly, near deadly,
Speaker:depending on what you use it for. The big thing, there's a lot
Speaker:of misconceptions about AI and l the LLMs
Speaker:that is they're based on. Number 1, it is not
Speaker:conscious. Right. 2, it is not a toy,
Speaker:and number 3, it is literally,
Speaker:something that is at present, not
Speaker:not necessarily, you know,
Speaker:fully understood, in in regards to the integrations
Speaker:and the things it may need to work with. You can't treat an
Speaker:LOM exactly the way
Speaker:you would treat, another enterprise application that's a little
Speaker:bit less opaque because LLMs are opaque on the on the
Speaker:inside, but you have to, for the purposes of
Speaker:security regulation, for the purposes of security compliance, you
Speaker:have to treat them, though, nonetheless, the same as any other
Speaker:enterprise application. So that's the conundrum. The conundrum
Speaker:is, how do you see into something that's
Speaker:opaque? And the way you do it is kind of
Speaker:what I discussed in that in that, in that paper, in
Speaker:that presentation, as well as one of the biggest
Speaker:vulnerabilities and that being jailbreaking. Yeah. So tell me about that
Speaker:because there's been a lot of, concerns
Speaker:about jailbreaking and, and I've noticed that
Speaker:the public facing GPTs have a ridiculous amount
Speaker:of safeguards around them to the point where, you know, if you
Speaker:ask it to describe something. Right? I asked it to talk
Speaker:about the to generate an image for the Butlerian jihad,
Speaker:right, which is a concept in June. And, obviously, I think the jihad
Speaker:term really freaked it out. Listen. I'm sorry. I can't do that.
Speaker:So there's clearly I understand why these safeguards are in place, but it seems
Speaker:like it's not that hard to get around them. Well, not
Speaker:necessarily. It depends on the model you're working with. For those of you
Speaker:who may use private LLMs because a
Speaker:wider issue on that is actually the DOD and many other government
Speaker:agencies actually prohibit the usage of public LLM
Speaker:systems, public AI, because they're concerned about unauthorized
Speaker:linkages as well as, data point model
Speaker:poisoning, prompt injections, things like
Speaker:that. So you often you're using these private elements. Several of these are
Speaker:uncensored. Right. Which means they do not have those safeguards.
Speaker:The ones that you see on the public space are supposed to have those safeguards,
Speaker:but you're never a 100% sure they're working because they may have been
Speaker:corrupted. In their regards to jailbreaking,
Speaker:jailbreaking is basically you're getting it to do something
Speaker:it's not supposed to do by either, a, breaking the guardrails,
Speaker:or by, b, influencing it
Speaker:through almost methods of interrogation to
Speaker:kind of break it down and make it talk. So
Speaker:it it literally is almost like that. So for those of you who, you know,
Speaker:kind of look at the it's it's kind of a there there's a great,
Speaker:neurophilosopher. His name is Jay Fodor and Nernschild named Richard Searle
Speaker:discussing the the philosophy of the mind as it applied to, computer
Speaker:technology. Several of the arguments that they say, well, the brain is like a
Speaker:computer. Yeah. You can kinda treat it like a human mind
Speaker:in the way you approach it in your prompts, but it isn't exactly the same.
Speaker:Once again, as I say, it is not conscious. It is not, and and it
Speaker:operates under a very strict set of parameters.
Speaker:But that being said, yes, you can literally interrogate it to do that.
Speaker:I'm not gonna say here, unfortunately, how,
Speaker:because, one, there are security reasons why we would
Speaker:not do that, a. And, b, there's also I mean,
Speaker:literally, in my presentation, that is all the news that has
Speaker:come to Academia and much of the industry
Speaker:today. There are new ones out there, but they haven't been discovered
Speaker:yet. Right. So I many ways to jailbreak. Yeah. And I was thinking, like
Speaker:so one of your slides I have pulled up here is, like, the top 10
Speaker:threats to LLM applications. I didn't think there were as many as
Speaker:10. So I knew that there were. I also know that
Speaker:data poisoning, for me, as a data scientist, data engineer,
Speaker:my first look at this when I saw this, aside from
Speaker:the g whiz bang factor of LLMs, was,
Speaker:wow. This isn't the data that trains this is a huge attack surface.
Speaker:And then when I first said that, people thought I was a tinfoil hatter.
Speaker:Right? And then slowly but surely, you're seeing research papers come
Speaker:out saying, like, no. We have to treat kind of the data as part of
Speaker:a secure software supply chain, which is an
Speaker:interesting concept because data people tend not to it's something they don't
Speaker:think about security. They think about security different. Is that a fair
Speaker:assessment in your that you've seen?
Speaker:Supply chains and the integrity of
Speaker:data is something that is not often, it
Speaker:seems, given the respect it's probably due. To be
Speaker:honest, I don't think so. In my own experience, I see it.
Speaker:It's not I guess one would say maybe it's not
Speaker:necessarily consistent. Maybe that's the fair way to put it. That's a
Speaker:really good way to put it. Yeah. And, I mean, right now, we're just now
Speaker:getting into discussion of, SBOM, software
Speaker:bill bill of materials Okay. Just for regular applications.
Speaker:I mean, it's a whole another level with LLMs and the
Speaker:models they're trained on, the models that these systems are trained on.
Speaker:So, yeah, that there's very much. So you have to make sure you're getting it
Speaker:from the right source, and you have to make sure that it hasn't been tampered
Speaker:with because it could very well be tampered with.
Speaker:It's not necessarily that hard. Right. Right. You
Speaker:could you could poison it with just one little segment of changing the
Speaker:the thing and across across 5 gigs of let's just say 5
Speaker:gigs. You know, that'd be like looking for a needle in the haystack.
Speaker:Precisely. In fact, that's what I talk about with the cockpit example that I
Speaker:gave. If I teach that l and to make sure that every time it puts
Speaker:in code to put in this malicious code that is a backdoor
Speaker:Right. Well, okay. It will do that. Every time somebody does,
Speaker:it embeds it into software code that is returned in the output for
Speaker:the prompt. If it does that, and let's say this
Speaker:is handed amongst several things, different
Speaker:applications, different solutions. Well, then if
Speaker:people take that that
Speaker:solution, that application, and it's in their software bill of
Speaker:materials, and then it gets distributed. Open source often
Speaker:gets proliferated very quickly. Right. And then it finds itself in
Speaker:there. You have a log floor 4 j situation.
Speaker:Right. Very similar except for the fact this thing
Speaker:is semi self executing. Now if it's semi self
Speaker:executing, you have a problem. You have a
Speaker:big problem. And I know I I just generally in industry. Now, obviously, you you
Speaker:spoke with the Northern Virginia. You're based in Northern Virginia. Northern Virginia is
Speaker:probably a little bit more security focused in terms
Speaker:of just who's based in that area than your average enterprise. Right?
Speaker:And I just I just see a lot of enterprises rushing to get into this
Speaker:LLM and Gen AI craze, but I don't see a lot of
Speaker:forethought or concern around security. And I just see a big
Speaker:disaster coming. Like, I I feel like I'm at I feel like I'm on the
Speaker:bridge of the Titanic, and I'm looking at something in the distance, and we're going
Speaker:full steam ahead. And I'm like, hey. Maybe we should
Speaker:not slow down, but be a little more cautious that we are in dangerous
Speaker:waters. Is that is that what you've seen too? Obviously, your customers
Speaker:and your clients may be a little more security cognizant.
Speaker:Well, I would say that I mean, I'm okay. We'll use the Titanic
Speaker:analogy. I'm the one up in the crows nest, you know, yelling into the radio
Speaker:phone, I see an iceberg. Right. Right. So I mean, that
Speaker:I agree. And that is a big issue because
Speaker:also there is this over reliance. Mhmm.
Speaker:Yeah. I imagine that as one of the top threats. So tell me about there's
Speaker:22 of those that I have very, very interesting questions about, but one of them
Speaker:was overreliance. So when you say overreliance on LLMs, what do you mean?
Speaker:Well, this is actually this is a sort of c suite, board
Speaker:level, thing as well as a engineering
Speaker:department level. They want to use AI to
Speaker:replace employees, make their operations more cost effective,
Speaker:more profitable. The problem is and this is a popular conception.
Speaker:This kind of goes into that argument about AI will take your job.
Speaker:This is a bit of a misunderstanding. It's not
Speaker:supposed to fully replace people. It's supposed to make them highly
Speaker:productive and efficient. They
Speaker:also do not necessarily feel like, well, the thing handles itself,
Speaker:so I can just wind it up and let it go. It doesn't need observation.
Speaker:It can fully self regulate. That would be true if
Speaker:there was a regulating function. You don't run a steam engine without
Speaker:a regulator on it. You need a regulator for LLMs.
Speaker:So the same concept applies. So first of all, there is this, it can do
Speaker:it itself, and a person is not necessary.
Speaker:This is incorrect. You most certainly need people.
Speaker:A great example I give in a recent presentation I've written
Speaker:is a discussion of, well, what does this mean to the organization?
Speaker:Well, a lot of level 1 tech, tech
Speaker:support jobs, there a lot of people say, well, those people are gonna get replaced.
Speaker:Well, yes, but someone needs to still be behind that LLM
Speaker:running the prompts, you know, and executing them in such an word and
Speaker:making interpretations based on the output.
Speaker:So that would be maybe something okay. Is that a dedicated job, or is that
Speaker:something you give to interns? Well, that would be, like, in,
Speaker:in the union trades you call an apprentice.
Speaker:That's the kind of thing. There's still a person involved. It's
Speaker:just not the same way we've done it before. Right.
Speaker:Also, on the subject of security, if you
Speaker:don't understand the security implications
Speaker:of it, you don't have controls for it. If you don't have controls for
Speaker:it, you can't mitigate that risk. And if you can't
Speaker:mitigate that risk, that's the liability.
Speaker:And if you're over reliant, you basically set up the whole system for LOMs, and
Speaker:then, you know, you just allow your customers to just come in and interact with
Speaker:the device. Well, if something
Speaker:happens, it would be treated very much like it
Speaker:was on any other application, so then you're now engaging
Speaker:in liabilities, loss of reputation, potential
Speaker:civil and criminal penalties, the list goes on.
Speaker:And a point on those 10 those 10,
Speaker:security issues, this is OWOX who is saying this.
Speaker:This is the open source, web application project.
Speaker:So we have, you know, a number of them
Speaker:that are a number of organizations, OOS is just the one I chose, they're
Speaker:kind of emphasizing this. They're saying, you know, don't think
Speaker:this thing can think for itself. Don't think this thing can act for itself.
Speaker:You need to look at it as humans are going to
Speaker:interact with it, and humans probably should be watching it.
Speaker:Right. So once again, it's that lack of controls leads to
Speaker:the risk. Yeah. I think the dream of it replacing
Speaker:everybody is gonna be at the root cause of
Speaker:a lot of problems down the road. I think I'm a firm believer
Speaker:in human in the loop. One of the the the interesting thing
Speaker:there and, that I see that was particularly
Speaker:curious was excessive agency. What do you mean by that? Because that got my
Speaker:attention. I think I know what it means, but I wanna hear it from you.
Speaker:Well, excessive agency is you're giving you you're kinda giving, you know,
Speaker:full the whole keys to the car. Right. There's
Speaker:no role based access control. If every user has near
Speaker:admin or actual admin privileges,
Speaker:that's that's actually something dangerous. A point of example,
Speaker:NetworkChuck just released a video on how to build your own
Speaker:AI on a very low cost platform.
Speaker:I love Network Chuck, and I have followed that step. You
Speaker:too. I'm doing I'm doing the same thing as he is because I have kids,
Speaker:and I want them to be able to use these things. But 1, I don't
Speaker:wanna pay the extra subscription. 2, I don't want them using mine. And 3, I
Speaker:don't really like what they're doing. I can at least exercise adult
Speaker:judgment on what I ask it and what I don't ask it. I don't think
Speaker:they can, and I don't think that's fair to put on kids. Sorry for the
Speaker:aside, but big shout out to network. No. That's fair. No. That's fair. That's exactly
Speaker:why Chuck was. And one
Speaker:thing about it is the first account that signs into the open
Speaker:web interface for Ollama sets you
Speaker:as admin Right. By default.
Speaker:Okay. Well, immediately, you need to engage role based access
Speaker:control to make sure that the next account does not get that same privilege.
Speaker:Maybe you should be given it. But is there any
Speaker:major access controls in the public ones?
Speaker:Not really. Private one? Is everybody thinking about that? Not
Speaker:really. I mean, I think Microsoft is doing some things around that because it's they're
Speaker:they're trying to integrate it with Office or m 365. But I
Speaker:don't I I I can't and if anyone in the sound of my voice wants
Speaker:to come on the show and talk about that, please do. But you're right. I
Speaker:don't think people do. And I also think excessive agency.
Speaker:What you heard about the car dealership, right, in Silicon Valley?
Speaker:Oh, yeah. Yeah. Yeah. Yeah. So for those who don't know, somebody
Speaker:managed to almost interrogate, like you said,
Speaker:to browbeat a AI chatbot to give
Speaker:him a it was a Chevy Tahoe or something like that for $1
Speaker:Chevy. It was a it was a Chevy truck
Speaker:and for $1. Now I'm not an automotive industry
Speaker:veteran, but I do know that if you sell 40,000, $50,000,
Speaker:cars for $1 a pop, you're not gonna be in business very long.
Speaker:So was that an example of excessive agency? I mean, clearly, it's an example of
Speaker:bad implementation. Almost certainly. That is. I mean, if you have
Speaker:the ability to trick if you have the ability to kind
Speaker:of browbeat it to override it and say, no. No. No. You don't understand me.
Speaker:You will do this. Well, then, okay,
Speaker:leave it to whatever
Speaker:gremlins there are out there on the web, out there in the
Speaker:world. Inside user, external user,
Speaker:irrelevant. If they can if just anybody can do that,
Speaker:you're the problem. Right. In this case, it was
Speaker:you could influence the model to set a
Speaker:certain price after arguing with it. Right. I actually
Speaker:found something recently, and I'm not gonna say which, LLM I
Speaker:did this on. It is a public one, and this is a
Speaker:result I suspect of another issue.
Speaker:I saw I tried to get some
Speaker:cybersecurity information from it when I was doing, a
Speaker:a try hack me exercise with a local cybersecurity group,
Speaker:hackers and hops. And I browbeat it
Speaker:saying, no. You don't understand. I need this for a cybersecurity
Speaker:exercise, and it gave me this information. Now this is absolute dual
Speaker:use knowledge. Right. It could be used for good. It could be used
Speaker:for evil. White hat or black hat. But the fact
Speaker:that you could do it,
Speaker:that sounds very dangerous. That sounds very dangerous.
Speaker:Prompt injection. Is that is that still a thing with
Speaker:the major public models, or is it just one of those things we're gonna live
Speaker:with for the rest of our lives? To be honest, I'm not
Speaker:sure. I mean, it's a case of, well, what is the prompt you're putting
Speaker:in? Right. When I talk about jailbreaking, I talked about,
Speaker:base 64 encrypt your text message
Speaker:into base 64. Why? Because that's how the prompt is seen
Speaker:by the LLM. Right. In other words, ASCII
Speaker:text. It doesn't check it, but it processes the text
Speaker:just the same. Oh, that sounds bad.
Speaker:It gets worse. Multi shot. Bury a
Speaker:malicious prompt inside the whole load of prompt,
Speaker:and fire hose it at the at the LM.
Speaker:It's not gonna check every single prompt. So if you bury 1
Speaker:in there, it might process that one and give you an answer
Speaker:it's not supposed to give. That's because the guardrails didn't engage.
Speaker:Interesting. So the guardrails are not necessarily on by default.
Speaker:Well, no. They are on by default, but if it overloads it,
Speaker:it may it may slip the net. So rather than shut
Speaker:down, it it shuts off? Well, Well, it's
Speaker:basically what you're doing is effectively a buffer overflow. You're basically using
Speaker:an injection method to induce what is effectively
Speaker:analogous to a buffer overflow. That's wild. That's
Speaker:not how I would have thought it would have worked. Interesting.
Speaker:Interesting. This is a fascinating space. So
Speaker:Yes. One of the things that I think people
Speaker:don't realize is
Speaker:just the sick insecure ways in
Speaker:which these plug ins could be designed. Right? Because, like, everyone's all
Speaker:gaga about these plug ins, and I look at it. I'm like, where am I
Speaker:sending my data? Right? Am I gonna read the 30 page EULA? Right? Or
Speaker:am I just gonna say, yes. Yes. Yes. I wanna do what I'm doing.
Speaker:Is that really a problem? It is.
Speaker:Because that kind of ties into unauthorized leakages.
Speaker:Right. How do I know that plug in is a secure
Speaker:connection into the l one, and there's nothing in between?
Speaker:Right. Or that it will contain what I get it.
Speaker:How do I know? I don't know. That's the thing is that is this plug
Speaker:in itself secure, and is its connection to the
Speaker:LLM secure, And is that LLM also
Speaker:integral? So, yeah, I could send it in there, but how do I
Speaker:know that along the way, something you know, the pipe might leak?
Speaker:So you need to check it. Just and, I mean, this goes I mean, this
Speaker:is very similar to APIs. This is very similar to,
Speaker:all sorts of remote interfacing. Just good engineering
Speaker:short lived. Just good engineering discipline seems to be
Speaker:missing from a lot of this because people are focused on the AI,
Speaker:not necessarily the underlying infrastructure that
Speaker:has to support it. Indeed. And I think that that's
Speaker:but that's the whole thing is that there is this massive trend as
Speaker:of late. I mean, perhaps it wasn't really emphasized
Speaker:before. I'm sure it was there, but it's now becoming very, you
Speaker:know, reiterated that we need to have security by
Speaker:design. Right. The security by design is already we're already doing
Speaker:that in other enterprise applications. Same should be applied to
Speaker:LLMs. Security by design. You check the code. You check the
Speaker:model. You check everything. And while it's operating,
Speaker:you check it. One of the biggest things you can do to overcome the
Speaker:opacity of an LLM, export
Speaker:the logs, export the comp the prompts.
Speaker:Have it processed. Now you could potentially process it.
Speaker:I'd figure the way you process any other kind of log data.
Speaker:The other thing you can do is use machine learning or
Speaker:an air gapped isolated LLM
Speaker:specifically trained to look for signatures,
Speaker:words, phrases, things like that. And when
Speaker:these patterns match, it returns saying, I found
Speaker:something that looks suspect. This is suspect.
Speaker:Here is the user who did this. Here is their IP.
Speaker:Like every other bit of log security log information we would get.
Speaker:So that would help piece together the trail to figure out, are these a
Speaker:bad actor, or is this the happenstance? Exactly.
Speaker:And that is one way you can do it because once you have the
Speaker:internal prompts and you have the internal logs and
Speaker:those are exported out, you now can see in.
Speaker:Right. The biggest problem is you gotta have that monitoring. You have to have that
Speaker:transparency. The elements are so large, you
Speaker:can't so easily see into them, but if you're taking the data out, it's a
Speaker:lot clearer. So you can kind of follow what the LLM is doing,
Speaker:if not, what's inside of it? Precisely. And the advantage
Speaker:is is if you use another LLM that is specifically designed
Speaker:to, you know, interrogate the prompts and look through
Speaker:them, examine them, scan them, whatever word you wish to use.
Speaker:You can find out where it is because that
Speaker:is not gonna be so easy to break the guardrails because it's examining
Speaker:one little bit at a time. It's looking at the individual prompts. It's not really
Speaker:it it's kind of agnostic about everything around it. It can get it can kind
Speaker:of filter out the new leads. Interesting. That's
Speaker:I mean, it's just so fascinating kind of to start pulling the thread at this,
Speaker:and there's a lot more. It's like I found there's a story about a guy
Speaker:who was renovating his basement, and he found, like, this ancient underground city. That's how
Speaker:I feel when I just get kicked back. It's true. It happened in
Speaker:Turkey. Like, he found, like, this underground network from, like, Byzantine
Speaker:or Roman times. That's what I feel like. I I like, wow. Like,
Speaker:this really goes down deep. So what's an
Speaker:inference attack? Because I've heard of that. What's an inference attack? We discussed that,
Speaker:or have we touched on that? Well, inference is
Speaker:basically what you're inferring to, the answer you are seeking.
Speaker:So, basically, it's basically, to the
Speaker:the inference is literally, the
Speaker:prompt that you are entering in and what you're getting out. Okay.
Speaker:More or less. So how is that an attack surface? Well,
Speaker:basically, you're you're chaining it. You're daisy chaining your attacks.
Speaker:You're trying to infer things. You're trying to kinda subtly
Speaker:get through. So it's a bit like it's a maybe
Speaker:more like cross examination from an attorney, a hostile attorney
Speaker:I would say that. Yeah. More than more than, like,
Speaker:interrogation or torture or or whatever verb we used
Speaker:earlier. Yes. Interesting. What's
Speaker:model inversion? Model inversion is
Speaker:basically you trying to spill the model itself. Oh. You're trying
Speaker:to kind of you're trying to kind of tear the
Speaker:guts tear the guts out, maybe put stuff in there,
Speaker:things of that kind. Interesting.
Speaker:Interesting. Where do
Speaker:we stand on the
Speaker:criminal and civil liabilities here? Right? I I I know that Air
Speaker:Canada had to pay a fine because they promised that its
Speaker:chatbot promised somebody something.
Speaker:I don't know where the California Chevy Tahoe thing
Speaker:is. But, I mean, have the laws
Speaker:caught up? Or, like, how were how is this generally looking like?
Speaker:Well, it depends. I mean, all jurisdictions are different, but I would
Speaker:suspect to say that whatever guarantees
Speaker:you make, you're bound to them. So
Speaker:probably disclaimers, indemnification is
Speaker:probably extremely wise. I would say,
Speaker:unfortunately, I'm not a legal expert. Right. Right. Right.
Speaker:Specifically to the law. Right. But as I'd say, I'd have
Speaker:enough legal understanding to probably say that if you make a promise,
Speaker:you better put your money where your mouth is. So that's why I back it
Speaker:up. IBM indemnifying their users for using one
Speaker:of their Granite models is probably a big deal for
Speaker:businesses. Because just in case somebody I'm sure that there's
Speaker:all fine print and things like that, but that that would be an appealing
Speaker:thing for business users. Yes.
Speaker:Interesting. Interesting.
Speaker:How does someone get started in learning how to jailbreak these? Like, is this is
Speaker:this a typical your background is, IT security.
Speaker:But what about someone who has a background in, say, AI and and and building
Speaker:these LLMs? Is that, Gunning, you think, be an another career
Speaker:path for the what we call data scientists today?
Speaker:Well, I would say you're gonna have to probably do it just as is. I
Speaker:think to the developers and to the data science Right. Scientists who work on this,
Speaker:you're gonna have to be security literate. Right.
Speaker:For those who want to get into it, I mean, data science is like any
Speaker:other AI trade. I mean, we often
Speaker:cross pollinate. So I would say that you might have an understanding
Speaker:already of these things. These prompt injections, as I say, are not
Speaker:much different than SQL injections. The data science Right. You probably know what that is.
Speaker:How you transfer it depends on what you know.
Speaker:I would say most data sciences do understand how some of this stuff
Speaker:works. Right. So getting into it is
Speaker:just basically you just learning more about security. Right. For the
Speaker:average person trying to get into it, I would say, if you're trying to
Speaker:get into AI security, know security
Speaker:first, and there are many ways to get into
Speaker:it. I, myself, came in, from my
Speaker:CCNA. I mean, that's how I kinda got into it. I got
Speaker:into networks, and then I got into cybersecurity. And
Speaker:then it was around the time that, you know, the GPTs were really starting to
Speaker:hit their stride. And it was just part and parcel of it because
Speaker:I needed a good reference tool. And so then I learned, okay.
Speaker:Well, how does this work? How do how is it put together? How,
Speaker:you know, how is it all formed and such? How does
Speaker:it make its inferences? How does it understand the problems?
Speaker:So from that, I would say to anybody trying to get into this field,
Speaker:know cybersecurity first, and you will know AI
Speaker:in time. AI is in concept
Speaker:relatively simple, but the nuts and bolts of it are quite
Speaker:complex. So Yeah. The implementation
Speaker:details are quite severe. Like, I think
Speaker:AI is really, I think, better not better suited, but it came
Speaker:out of the lab. I think the paint is still wet. Paint hasn't dried
Speaker:yet. And now we're forcing it into an enterprise
Speaker:scenarios with real customers, real data, real people's lives.
Speaker:And I don't see a lot of the traditional security
Speaker:discipline that
Speaker:I would expect in modern era, modern development.
Speaker:And even that's a low bar. Even that's a low bar. Let's be real. Well,
Speaker:it's it's new. Right. It's very shiny.
Speaker:Mhmm. That's I think that's what I would say is the general
Speaker:populace and even in the industry that's quite I think our view is that this
Speaker:is a shiny thing. Right. Well, you know, well, I want
Speaker:to. You don't even know what it does. I still want it. I want it.
Speaker:What's interesting is, it
Speaker:reminds me a lot of the early days of the web where everybody wanted a
Speaker:website. Well, what are you gonna do with it? I don't know. I just want
Speaker:a website. You know? It's very it has very very
Speaker:similar vibe in that regard of we want it. We you know, the hell with
Speaker:the consequences. But the way I see this
Speaker:being,
Speaker:taken up as quickly as it is kind
Speaker:of worries me. Like, there's gonna be a day of reckoning, I
Speaker:think, coming. You know? And I thought we
Speaker:already have it. Right? You you had, there was a leak from Chat
Speaker:CPT. They had a 100 was a 100000 ish customers there, give or
Speaker:take? A 100000 credentials taken, compromised.
Speaker:Credentials and and presumably the data and the chats?
Speaker:Some of it potentially, I'm sure. But what we're looking at is, like,
Speaker:names, email addresses. I mean, it depends on how much you put in
Speaker:that profile. Remember, everything you put in that profile is stored.
Speaker:Right. Right. That is truly scary.
Speaker:So you mentioned network, Chuck. So you do you think that
Speaker:just on a personal level, it's
Speaker:what worries me about these offline models, right, you run OLAMA locally.
Speaker:Right? Do you think they could they call
Speaker:home? Could those be hijacked? Could those have problems?
Speaker:Specifically. Specifically. Like, so if I'm
Speaker:running Olama locally, right,
Speaker:how secure is that? Does that does that depend on the security of my
Speaker:network, or is there something in there that calls home?
Speaker:No. Not unless you tell it to. Not unless you try to extract it, you
Speaker:make a pull, then, yes, it does that. But that's the idea is that once
Speaker:it's pulled down, it kinda isolates itself. Now
Speaker:what you can do yourself is set up your
Speaker:network so that literally it has to be outbound,
Speaker:a stateful connection, originating outbound.
Speaker:And you can set that up in your firewall, physical
Speaker:or otherwise. And you can do things like that, and you can
Speaker:kind of put it to a point where it doesn't call home unless you tell
Speaker:it to. Right. And, also, once again, that
Speaker:private LLM is also very good because you control
Speaker:the access to what it does. So you can say,
Speaker:other than these addresses, sanitize it to the
Speaker:address of wherever the model comes from, say, these are the only ones
Speaker:allowed. Right. And nobody else is permitted.
Speaker:Otherwise, implicit deny. Right. So that's a I think
Speaker:a a small tangible example of something you
Speaker:can do that is relatively straightforward for any
Speaker:systems or network engineer, to do just in the hearing
Speaker:now. But in general, no. They don't normally call without
Speaker:prompting. Okay. But depends on what they do with those models.
Speaker:They might put in that kind of feature. A lot of that go back to
Speaker:the I'm sorry. Yeah. That's kind of my concern is, like, you know, would that
Speaker:end up in there? Or Well, Meta might put that in there.
Speaker:Right. Meta is a not alone. Meta is not
Speaker:exactly free. Right. Matt is not exactly,
Speaker:has a reputation for privacy. No.
Speaker:So it's kind of ironic that they are
Speaker:leading the effort in this space. Seems kind of an odd move.
Speaker:I I don't know what to say about that. No. No. No. I just need
Speaker:I have no thoughts on it, but Right. Right. Frankly, I don't I don't know
Speaker:how relevant it'd be to this discussion. But it's an interesting it's
Speaker:it's just an interesting time to be in this field, and,
Speaker:this is just fascinating that you can
Speaker:jailbreak. You could do this and, you know, even just the basics. Right?
Speaker:Like, you could do a DOS attack. Right? There's
Speaker:just basics too. Like, this is still
Speaker:an IT service no matter how cool it is, no matter futuristic it is. It's
Speaker:still an IT service, so it has all of those vulnerabilities,
Speaker:you know, that I don't know. Like, it's just it's just interesting. People are so
Speaker:focused in the new shiny. I just find it fascinating.
Speaker:And that's the thing is that this thing is a compounded problem. Right. You
Speaker:don't just have the usual suspects. You also have
Speaker:new things that are they
Speaker:by the virtue of them being new, there's not much
Speaker:investigation. There's not much study. I mean, amongst my
Speaker:research for this presentation, I found a number of
Speaker:papers, white papers coming from all sorts of universities.
Speaker:They are now looking into this. Right. This is something that maybe we
Speaker:should have done maybe a while back. Good thing, though, we're doing it now.
Speaker:Right. But also, also, there's a lot of reasons why you would do that, though.
Speaker:You would do that because in the wild, you'd be able to identify these things.
Speaker:Right. You'd be able to see. You're not gonna know everything when something gets released
Speaker:until it's put out into the wild. Right. And real users
Speaker:get their hands on it. Good actors, bad actors,
Speaker:and everything in the middle. Right? Like, you're not gonna yeah. No. I mean, it's
Speaker:kind of like I guess I guess in a perfect world, the cart would be
Speaker:before the horse in this case, but that's not the world we live in.
Speaker:Interesting. So where can
Speaker:people find out more about you and what you're up to? Well, you
Speaker:can find me on, LinkedIn. Kevin Lynch
Speaker:with CCNA. Cool. You can look up my company, Novi Tea Guy,
Speaker:Novi Tea Guy dot com. And For those outside the area,
Speaker:Nova stands for Northern Virginia. Just just wanna figure it out there. Well,
Speaker:also, it well, it's actually a bit of a it's a double meaning. At the
Speaker:time, I was dedicating myself to IT for the first time. I've done
Speaker:IT kind of side part of my work. So Nova is also the
Speaker:Latin for new. So I was Okay. The new IT guy. The
Speaker:new IT guy. But when it comes to IT, I'm still your guy even then.
Speaker:There you go. I love it. And,
Speaker:I'll definitely will include in the show notes a link to your presentation.
Speaker:And this has been a great conversation. I'd love to have you back and maybe
Speaker:do your presentation, maybe on a live stream or something like that if you're interested,
Speaker:and, I'll let Bailey finish the show. And that's
Speaker:a wrap for today's episode of the data driven podcast.
Speaker:A huge thank you to Kevin Latchford for shedding light on the vulnerabilities
Speaker:of large language models and how to stay one step ahead in the ever
Speaker:evolving world of IT security. Remember, while these
Speaker:models are brilliant at generating conversation, they aren't infallible
Speaker:so keep your digital guard up. Until next time, stay
Speaker:curious, stay safe and always question the source unless,
Speaker:of course, it's me. Cheers.
About the author, Frank
Frank La Vigne is a software engineer and UX geek who saw the light about Data Science at an internal Microsoft Data Science Summit in 2016. Now, he wants to share his passion for the Data Arts with the world.
He blogs regularly at FranksWorld.com and has a YouTube channel called Frank's World TV. (www.FranksWorld.TV). Frank has extensive experience in web and application development. He is also an expert in mobile and tablet engineering. You can find him on Twitter at @tableteer.