Dr Yossi Keshet on Decoding Speech, AI, Morality, and the Future

Welcome back to another riveting episode of Data Driven.

Joining us today, lakeside and positively glowing from his

Appalachian retreat, is Frank. Meanwhile, the

always astute and ever energetic Andy is here to keep us

grounded. But enough about us. Today, we have

a true luminary in the field of AI, someone who's blending the worlds

of academia and enterprise with seamless finesse. He's an

associate professor at the Technion, has published over 100

research papers on automated speech recognition, and is the chief

scientist at Iola. Please welcome doctor Yossi

Keshet or as he's known to his friends, Yossi.

Alright. Hello, and welcome to Data Driven, the podcast where we explore the

emergent fields of artificial intelligence, data science, and,

and, of course, data engineering, without which the whole world would probably stop turning.

And you know, data engineering is important. That's

basically it. Still working on that that that revamped

monologue, for, for season 8, Andy. Were

you on vacation? You're on vacation. I am on vacation. And

for those of you who can't see on camera who are not who are

listening, not watching, I am literally lakeside,

in the foothills. Well, not the foothills. We are actually in the Appalachian Mountains. Or

is it Appalachian? I I never I I've heard of those. I I never

got a clear read on it. Say either. So, you know When I say either.

Yeah. Yeah. Yeah. Yeah. Yeah. So I am in Deep Creek Lake,

Maryland, which is kind of like, Maryland doesn't really have a Panhandle

per se, but if it did, it would be this is what this would be.

I probably think I'm 5 miles from West Virginia and about

20 miles from Pennsylvania. So it's kind of like this quiet

little corner of the state.

And I've been, you know, reading and studying

today. I hit day 600 on Pluralsight Consecutive. Nice.

So recording this June 17th. And, how

things with you, Andy? Things are good. I'm gonna throw out a plug for

data driven media dot tv because Frank mentioned.

If you're listening, he while he was mentioning that, he was

actually panning the camera over to the lake. But if

you're, subscribing to data driven media dot tv, you get

to see us. You get to see the video, and you

can see, for instance, that I am wearing the, my data is the

new oil t shirt, which you can pick up. I'm just full of

sponsor stuff today. I'm just doing Well, it's self out. It's

self sponsored. And, honestly, we really need to get better at that. Right? We have

data channel. Tv. There is a for listeners to the show, I will give

a preview. There is gonna be data driven academy is is launching soon. You have

a course coming up the end of the month. Actually, yeah, it's fabric.

Today. We're recording this on 17th. It's 24th

of of June, but I'm also doing, 2 more, at

near the ends of July August. And in addition

to that, while we're shameless plugging away here,

before we get to our very interesting guest, now I'm also bringing

back my, day of Azure Data Factory as wildly

popular. I delivered it at a couple of, conferences,

international conferences, 22, 23. And,

yeah. Let's see see if people are interested. What do you do Friday this

afternoon Friday afternoons, Andy? Oh, there's this thing, Frank. Thanks for

mentioning that. Totally free. We we gotta we're trying to get better at this. That's

all. We do. Yeah. Data engineering Fridays. And if you go to data engineering

fridays.com, you can learn more about that. Frank, you're doing a lot

of stuff with I noticed with using the, encore

replay feature in Restream. And it's

right you you shared that with me. I started doing that with data engineering

Fridays as well. But great a great way to,

you know, to get your message out there. And, you

know, I I had no idea replays would help. But my gosh.

They really have. It's just a matter of just hitting the echo of I

can't even talk. Algorithm the right way. Yeah. And Yeah. You know,

maybe we can get the so I think it's a good segue, for our

guest. Doctor Yossi, Keshet. He's the chief

scientist at AIOLA, an AI powered tech

company that automates business workflows

by capturing spoken data. Yossi is also

an associate professor at the Faculty of Electrical and Computer

Engineering at the Technion in Israel.

Yossi is an award winning scholar and has published over a 100 research

papers about automated speech recognition and speech

synthesis. Welcome to the show, Yossi. Hi.

Nice for having me. Thank you for having me. Hey. No problem. No

problem. We are very excited to have you. And, you're not just an

academic, but you've also proven yourself in in actual enterprise. So

which sounds really bad as I say that out loud, but I think you knew

there was a compliment.

But, so what is AIOLA?

Can you tell me a little bit about that? Because I'm curious about that and

and and workflows

around spoken data. So

Iola is a company that is aimed to target

the, you know, the very basic and foundational

industries. Maybe if I

may, let's start with the a general scene of the

automatic speech recognition now, and then you will understand where are YOLA stands because we

have now open AI and everything is like we you

can say we solve the AI problem. So it's not like that.

So we are in a in a amazing shape in in

terms of automatic speech recognition. So we we have a paper that shows

that whisper, the model of OpenAI, is as good as humans in

detecting and transcribing language when we speak about

American English with noise, without noise, and

also, l 2 speakers. That is the

speakers of non non native American speakers of the

language. And the the results are so whisper. The

OpenAI model is the same as human listeners. And that is

the main thing. But the thing is that

when you come to industries, usually they have jargon, they have special words.

And and those words are either rare in

their language or they they they are not none

word. It's like I don't know. I when I'm a medical doctor and would like

to make a surgery surgery and I would like to transcribe what I'm saying during

the surgery. I'm there isn't words that which are not

often used or which are none, non English words. And

in that case, those, automatic speech recognizer doesn't

work at all. They don't detect those words. And in Ayala, this

is our target to take those words, which are actually the most important word. Those

are the jargon of the of the industry of the of the facility.

So the goal is to help those industries to come

up with the with the automatic speech recognition for

reporting for transcribing speech.

I have a question. When you say automatic, what what makes it automatic? Is

it just kinda, what exactly does that mean?

So automatic speech recognition today works very similar

very, very similar to the way KJGPT works.

KJGPT works on a model called transformer. It's an, deep

learning architecture, which has, a

history based on previous recurrent architectures.

And it can predict, as as we all know, it can

predict text amazingly. In speech recognition, automatic

speech recognition, it's almost the same thing, but there is another

component, to the to the to the

this transformer, which is which is called encoder.

This this part take the speech and actually transfer it to

a great representation that can be used

with this, with this, let's call it with this with the other side, with

this, GPT together. Together, they can,

transcribe speech in, as I described, in a very good

way, as good as humans in some

cases. I will say, like,

I've been messing around with the app that's on the phone,

for, chat g p chat gbt, and,

I use the the voice interaction feature. It is

amazingly good at getting rid of the umms, the ahs,

the scatterbrain thoughts that I sometimes have when I talk to it.

Like, it it could kinda really distill a lot of

things. Like, I'm impressed with it. It's it's really gotten last time I

did anything serious with speech recognition was probably, like, maybe 4 years

ago, and it's really improved. Like, I mean, orders of magnitude

than I thought. I mean, it's it's it's it's almost at Star Trek level. You

know? I'm not sure

in those it depends on the company if it's Apple or

Google. And I'm not sure which they don't declare

which models they use. I think, personally, they don't use this whisper or

the latest model that we have for automatic speech recognition that

is transcribing speech. And the goal is a little bit different

in the in the phone. You actually want to maybe Right. Make,

make notes, send an email, send a text message,

and maybe the vocabulary the vocabulary is less

less defined. There is another problem with

the phones. Oh, no. Go ahead. I want to call my

friend. His name is xi, and

the last name is CHUNG. How do you pronounce it?

What what do you do with that? I'm gonna say he or chi or

so there is a there is a problem of proper name and how do you

define them. And this is a completely different problem. It's still an open problem, and

the goal is a little bit different. So

it's when we assessing the quality of those models, it's

a little bit different than the assessment of just spoken language

like what we do now. No. I mean, that's a great point. I mean, my

last name has, you know, technically is Lavin.

But, you know, growing up for for reasons many,

big and small, it became Lavinia. And like, so, like,

the phone, depending on if it's Android or an Apple, it will, it

will he gets confused pretty easily.

And that is an interesting point. Some names, Andy is lucky to have an

easy name for the, the system.

But not everybody does. So I understand that. Sure.

I also wanna double click on American

English. You you you said that a bunch of times. Like, is there is there

an inherent bias in these model trainings because these are done by American

companies? Yes. There is. Okay. The

day the data is mostly of American English. The research institutes

are mostly American. So the reason maybe I don't know

if you'd call it you call it inherent or implicit bias, but there is a

bias, definitely.

We are investigating, by the way, the the intelligibility

of speech in some cases And what is the intelligibility of

of American listener versus the inter intelligibility of

myself, which I'm not American listener, but I I know English.

What is the best, what is the best, double quote speaker? What is the best

listener? How can we transform those

to speech recognizer? How can we transform those to assessing the

quality of speech? What does it mean? What does it mean about the pathologies in

speech? And this is ongoing research on

this on this field. Interesting.

I I often wonder, like, you know, what it's not just English.

Right? Like, you know, if you listen to Spanish, like, there's different dialects of

Spanish. Right? Even even German. You know, I'm sure

there's, you know, plenty of dialects of all these languages and,

like, how do you the training of a

model that where it can get to be as good at

understanding x and x versus x and y versus, you know,

the base language, the base standard. I don't know. That's

fascinating. It seems like it seems like it could be an endless loop of, like,

training. It it is. Indeed, it

is. And when we train, there is another so I'm I'm

working on deep learning and AI. And what we found out

that it it may it may be the case that if you train

on 1 language, huge amount of data from 1 language, let's say

American English, but then train on less data on Spanish,

you actually get you get some advantage of training from

from the American English. So, again, in this modern whisper of

OpenAI, most of the data is American English, but,

actually, other languages are really great.

Again, Spanish is amazing. So maybe like

humans maybe like humans as we learn more and more languages, it's easier

for us. This is very interesting, point.

No. That's an interesting idea because I know, like, I never

understood American English grammar, American or otherwise,

until I studied a foreign language. And then when I studied it, it was German.

And, you know, German kept a lot of the archaic things that

are in English and kept them and kept make kept them,

made continue to keep them important. Like in English, you know, who

and whom used to confuse the you know what out of me.

Right? But when I when I learned in German about different cases and things

like that, I was like, oh, that's why it is. Right? So,

like, all these things that just like you said, like, learning another

having more data or data from another point of view, I suppose,

or another way to look at the world help me look at my world

a little better. Maybe maybe that's how

AI will work too. I don't know.

Maybe. We don't know. We we actually have a guess about that

because it those networks actually solve an optimization problem,

mathematical optimization problem. It's a problem that

that is, we define it with equation, and we need to have

a computer running and solve it. The equation is

overtraining set of examples. So it's 1

1 person say that, another person said something else.

And what happened is that when, again, when we have

a large amount of data,

it seems that those those networks get to an amazing place.

So this this, algorithm, this whisper or other

algorithms, it's really from the recent years, like 2, 3 years.

That's it. We it's they they perform amazingly

amazingly, with the with the

same with the same mechanism, not with the same amount of

data. Yeah. That's that's that's the

fascinating aspect of all of this. It's just that some of these things just seem

some problems seem harder than they ought to be,

and then some solutions to problems seem way more effective than they

ought to be. It's an interesting also to say

it's always the case that we so Whisper, OpenAI Whisper, was trained

way, way much more than just a kid learning a language.

Kid language learning a language exposed to way much less hours of

speech, less less accurate, less,

coherent. And this is something,

Nom Chomski raised years ago, like, 50 years ago.

And it's still an open question. Like, if we can make those

system works better, if we know the language,

I guess you learn German faster than any

machine that works today.

That's yeah. It's it's and I'm glad you mentioned Noam

Chomsky because that kinda was like so for those who don't know, Noam

Chomsky is, among other things, a noted linguist scholar.

I highly recommend you do a search on him because that's a that's a

good Wikipedia rabbit hole to fall into. But,

how much does linguistics come up in this? Right? Because I think

what's fascinating about this field for me is a lot

of, my grandfather, my great grandfather

was a a linguistic professor. And, you know, as the

family lore goes, I never met him. He died decade or 2 before I was

born. He spoke, like, 12 languages. He was a professor of, like, 5

or 6. And, you know, a lot of people in my family

seem to have on that side of the family seem to be gifted in language.

And 1 of the fields I was tempted to to study in

university was linguistics. And I just find

it interesting how there's

a now a Venn diagram now is much larger

than it used to be in terms of linguistics and computer science.

So what are your thoughts on? Like, how much does like,

if you're if you have a

company like AIO. Right? Like, how many people are, you know, honest to

goodness, linguists versus computer scientists and and AI engineers?

So there is there is no no linguists there. Oh,

really? Okay. There are no linguists. But I have to tell you, so there was

a professor called Freddie Frederick, Jelinek. He was the

head of language, research at the John Hopkins University

at Baltimore. He was amazing. He was 1 of the smartest,

people on earth. And he said he was

developed many of the speech recognition algorithms. He said,

every time I fire a linguist, the performance of speech recognizer goes

up.

And this is, this is embarrassing. But I've been I

made myself, 1st, really like

linguistics. I really like cognitive sciences, and I really

try to combine it with with my work. But it's really

amazing that we don't have all those AI system

don't have any of that. So you don't train CEGPT

to what is a noun, what is a verb, what is anything. You don't train

speech that this is the

this is the you don't you don't use linguist. You don't use this is

the prominent word. This is the end of the sentence. It just happened

by huge amount of data. And

this is interesting. This is somehow contradict Noam Chomsky who said that

there there is a universal grammar. There is a

we are born innate with language. There is a

maybe some black box in our brain which

is tuned to learn a language. And,

we are not sure about that. There is no direct proof if it's correct or

no. We are born with language. We are as humans, we're

born with language. We this is part of our, human being.

We are not born with written language. So written language was invented.

The spoken language is something like like a zebra

has stripes. This is this is our nature, and this is

interesting. This is not happening not happening in

AI. The best success that didn't have linguist, they don't have any

restriction of what should be say or not.

Maybe maybe AI will be a tool to somehow

make the linguist research more effective and

try to understand what happened in the brain, what happened in the cognition part.

But I would like to tell you about another research we are preparing here, which

is really amazing. 1 of the thing is that we have

so there is this JGPT. It's a language model.

We also have something in the brain. It's also neural network.

And we when we try to compare them, there is a huge

correlation between the the what happened in the artificial neural

network of GPT and the neural

biological neural network in the brain. And, it was

shown, several years ago, and here we

show it again with, with this, with the most modern,

automatic speech recognizers. So this is

a phenomenal post correlation between the artificial and the

neural mechanisms. I was gonna ask about that

because I'm I'm familiar with, you know, at least the abstracts of

the research, from a few years ago and now. And

I was curious if there had been any new correlations

or, you know, or new research, new connections that have been made

between machines learning languages

and the way our brains work. It sounds like

that's true.

So we try to we just initiate, man,

a research here in my lab about that. There was

some French guys from, mainly King

and his colleague at, Meta. And

and I forgot the university in France. So they

show that there is those correlation. They show simple correlation. What we

they show it with LLM, with language model. What we show is a little bit

different. We show correlation with automatic speech

recognition. So we ask people under fMRI, under MRI.

They're we scan their brain at some

resolution, and we try to find correlation with their brain activity

during reading and during speaking aloud,

and ask what is the correlation with the the best model we know for

speech recognition. And then there are correlation.

I have to say that there is a mechanism in the transforming this

architecture of neural network. There is a mechanism called attention. This

mechanism allow those model to to have the connection between

worlds and themselves. So, I'm eating an

apple. It was delicious. So it refers to the apple.

Okay? So there is attention mechanism. This what makes those

model amazing. So there is attention mechanism, I guess, in the

brain. So we try to correlate the this attention mechanism in

the models and compare it to what the activity in the brain. We don't have

results yet, but it seems promising. And we also ask

another question. What if you don't read aloud? What if you read

like silent reading? What if you have dyslexia? What if you have,

other type of, pathology? What

what are the correlation then? So this is fascinating. So and

there is correlation. I don't I don't know still what what's going to happen

with that. But I I know the pathologist, but it's unbelievable, the

correlation. That that is really exciting,

especially when you're examining things like dyslexia,

which is considered, you know, not normal,

or maybe that's not the right term for it, but a

challenge at a minimum. The cool the cool kids call that neurodivergent

now. I think Neurodivergent. Thank you, Frank. So when you're studying, you

know, when you're studying that sort, I'm wondering if there's a place for

that, in in the artificial.

I'm curious. What what do you mean? Can you

So, yeah, is there is is there any benefit

to, I say, transferring the thought processes

of people who are neurodivergent and and automating that

and making that part of the, you know,

the the language model or or speech recognition?

Yeah. I think so. I think so. 1st, it's a it's a tool

to to an to analyze what happened in the

brain. Yeah. What happened

but it's very difficult. So we don't, we don't have any debugger for the build

the brain. We don't see the code of the brain. We don't see that this

function doesn't work. And it's, most of the work

is to design the experiment and

and it's really amazing. In our design, we have the

same so as yet as I told you, I'm asking people to read aloud

and compare it to what automatic speech recognition,

is plan is, supposed to do. But I'm

also asking people to read silently, and then I follow

their eyes. I have a make a make a machine that follows their eyes, and

I know where where is the where like, III

track their eyes and I see which wall they are reading

now. And I can and I can use that to follow

what what they read. But in order to operate that on a speech

recognizer model, I need the speech. So it's during the design of

the experiment, I need artificial speech or I need them to to read aloud

afterwards. It's a it's a big, it's a big question

how to do that properly and how to

make things happen, but definitely walking with

people with, with problems first to help them.

And second, to understand them. And 3rd, to maybe make

understand the brain and make, AI better.

I also think, like, stroke victims, right, could benefit down the line

from a better understanding of lang language models. Right? Like, maybe there would be some

kind of therapy that could be directed to that. I think I think it's

fascinating. I always love those fields where they touch upon more than 1 thing.

Right? This isn't just math. This isn't just computer science. Like, it's linguistics. But,

you know, it's a little bit of everything. It's like a giant, like, pot of

stew that you just throw a bunch of stuff in, and it all kind of

mixes. And, like, it's kind of like, almost like intellectual gumbo,

I guess, would be the word. Right? But,

what what,

what drove you to make, your your your

your company? Like, what what was the driving force to

say, hey. You know, we have

I remember many, many years ago in an office, and you would always see

doctors talking into these little, like, miniature recorders.

Right? In the olden days, they would go off to

some data center somewhere and somebody would not data center, but, like,

some piping center, call center where people would

transcribe that. You know, obviously, that is now an artifact of

the past as these models have gotten better.

What what was the goal in in in, your

company to say we can do this better? What what was the the that breakthrough

moment of, like, here's here's what the industry already does. Here's how we can do

it better. So there is

so we all know Check GPT, and it influence our life. We search now

instead of Google, we search with GPT and it's amazing. It's unbelievable.

So I thought, what about the very fundamental industries? What

about,

like, when you check-in when you, check an airplane, you

use a special jargon. You cannot touch anything. You cannot

leave even a pen there because otherwise the the plane wouldn't be,

valid for flight. What about industries like the food

industries when you need to report, the process? You

have gloves, you cannot touch an iPad, you cannot barely

write. And what about, other industries

like, maybe the cheap technology when you make nanotechnologies and

when you make chips, you make, you know,

silicon chips and silicon

first. So you need you you are cover all.

You are with gloves. You need to report the process. It's a all

those industries has this have special jargons. They use special

terms to describe what they're doing. They don't have access to

to to write something,

and they are very limited in the way they they provide. And on the other

end, we had speech recognition, but speech recognition doesn't work on

those jargon world. Those jargon world are actually the

most important to those industries, and this was the goal for

Iola. So what we do is we operate,

automatic speech recognition, the best automatic speech recognition,

but we also operate something else. We also operate something called keyword spotting.

It's another deep network, which is focused

on detecting only the jargon words. So you can define those jargon

words in advance. You don't need to train them. You you can

define them, and it they all work together. They work like, as a

complimentary, couple to make a

very robust prediction, and we can detect those,

jargon words and make reporting on on on on the

process, without just by speaking. So it

can it can use in any industries,

any, industry that doesn't

have access to the most modern AI system, the speech

recognizer wouldn't walk there. They have problems, like,

writing and formulating their reports.

Yeah. So I'm curious how those work together. You mentioned

that you've got the speech recognizer. You've got the keyword,

engine. Are they 2 separate engines that are just always running

maybe agents, running at the same time or are

they encapsulated, say, is the speech

recognizer does the speech recognizer have a, you know, a

subset or a a function built into it to do the

keyword recognition? So just to

be sure, those keywords in some industries are not are

not are not English words. So it can be a word which nobody

knows about. It was not shown in the in

the, like, in the Internet, like, JGPT strain on the data over the

Internet. There are some walls that are not not there. This is

your, proprietary company. You have invented a wall to

describe what is the this, part of the engine. So

Yeah. So what we so we have this keyword spotting. It was it it

is trained to detect keyword in general. They are defined by,

by text and it operates. We have 2 model for preparation. 1 of them

works on the this encoder part of

the of the automatic speech recognition, and then it guides.

It's still the speech recognition towards the correct

transcription. And there is another mode, which is,

our self, encode our self representation of

speech, and then it also guides the automatic speech

recognition to a better, location and to detect those

words. And, actually, we can show that you can buy combine

any word can be from different languages, and we can

detect them, like, almost 100% correct, those jargon

words. That was that was going sorry. Go ahead.

No. No. No. Sorry. That no. That's okay. That that makes perfect

sense now, what you just said about the languages using

multiple languages, you know, English plus all of the

other languages because sometimes

people will struggle if their English as a second

language speaker. They'll struggle to find the right

English word, and they'll substitute a word from their native language.

And in other cases, they'll be perhaps teaching

on a topic, and they may revert back

to an older language, Greek, Latin, something

like that. That may be part of the, the

lecture or, you know, I could see that in

medicine. I could see it in, you know, all all sorts

of literature studies. I could see a lot of that. And that

that kinda clicked for me as you were saying that that makes sense that you

would have additional languages. Yeah. I also wonder, like, in in

also conversational context. Right? Like, you know, Spanglish is a

thing. Frankel is is the French and

English kinda mashed together, and I know that other language

whenever you have 2 groups of people kinda come together, like, you know, there's always

some kind of weird mix of language that that kinda

just evolves either naturally or forced. I mean, that's Right. That's another

debate. Are you thinking belt or creole? I know we're Belter, you know, I

wasn't going there, but that that's a that's an excellent example.

So, Yosie looks very confused. So so there's a series of

books, called The Expanse. It was an excellent TV show

for about 6 seasons, and it's basically set, 2,

300 years in the future.

And as humans colonize the asteroid belt,

their people from all over the world kinda all end up living

together. So, like, the the Belter Creole language is this is a

creole of, you know, literally dozens of languages. Right?

So, like, it'll switch from, you know, Hindi to Arabic to,

English to French to there's even some German in there. I've heard some of that.

Like, and there are these kind of these weird mixes of things. Right? So they'll

say the the word for the Belter people, like,

people live in the Belk, is Beltaloda. Belt obviously comes from, you

know, the asteroid belt English. Loda, I think is a Hindu term. I

think. Don't hate on me in the comments. Don't hate on me in the comments.

But, I know Walla is a is a is a Hindu term. Right? So

they'll they'll, you know, when they talk to people who live in the Earth or

Mars, they refer to them as well wallahs, gravity well

wallahs. Right? Like so it's like, and I only know wallah because

of dish wallahs, and Wired Magazine did a whole story about dish wallows in

the nineties. Anyway, but I mean, I think, like, you know, I

I suppose that approach could work for something like a creole. Right? Like, we have

multiple languages kinda mixed together. Or is that not really a

massive business case?

It's Creole is really complicated. It's a language. It's like real real a

real language, and it's complicated. This the the more

delicate cases of that, what we call in research, code switching when

I'm Right. When I speak Hebrew, for example, I don't have a

word for the, you know, the Internet router. So I say the router in

in English. Or I said email or I will say

I don't know. There are so many words in English that are used especially

in technology that you use worldwide in other languages, and this

is code switching. There is another case. I think Andy pointed it

out that sometimes when you are stressed

or let's say your l 1 is Spanish, but l 2 is American

English or you're bilingual. And sometimes when you are

stressed, you you just switch the the 1

word and it this is amazing phenomena. This is a research with Tamar Golang

from, University of San Diego and Matt Goldrick from Northwestern

University. And I provide, again, a mechanism to detect

that and to make research of that. And the the key question is,

like, why do you do that? Why do and when do you do that? Is

it stress? What what what is the what is the state of

describing those? Are you gonna describe it in the American

way, the Spanish word, or is it gonna be vice

versa? And this is really interesting.

It's not my field of research. I just know how to detect them

and, and Interesting. To detect them really well,

but I don't know why it happens and what is the mechanism

behind that. I could definitely see,

the opportunity with starting with being

able to detect, you know, these I

don't I don't know the right word for them. I'll I'll call them modes. You

know, a mode of speech where someone is mixing 2

languages. And I'm sure those vary.

So Like when I go Jersey on you. Right? That's we we

can't we can't say any more about that, Frank. We're trying to keep our

clean rating. But yes. Exactly. But,

that's sorry. Inside, Joe. But the,

but, yeah, I could see modes of speaking where someone who is

more familiar with English as a second language.

And and they've still you know, of course, they know their native language. They'll always

know that. But as they I don't I don't wanna use the wrong word

here, but I'm thinking experience is probably the best word is they get more

experience, gain more experience with their second language.

They may switch words less or switch languages

less. And detecting that, I think, is the

is key. I understand now more about what what you're doing, what

you're accomplishing. And that that's the

very first step to then being able to produce speech

in those different modes. And that would be a

fascinating, you know, a fascinating accomplishment.

If you do, the more we can have. Machines

speak to us in the language that we're most familiar with, that,

of course, you know, is is almost there now, mostly

there right now, but have it be able to to speak to us in these

different modes where we where the machine switches where it's

back to our first language, you know, based

on some algorithmic calculation. That sounds

fascinating. Yeah. It is.

I'm not sure we are there yet. It's we have a long way to go

there. But, Sure. Yeah. Makes

sense. Fascinating. Well, this is how it starts, though. Right?

This is fascinating. This is, yeah, this is,

somehow there is an elephant in the room. There we may have to say

something about AI and their regulation and what happens now.

And, if I may, I would like to say something about this because I have

a deep totally different point of view about that.

Please. So everybody is speaking about

regulation and it might be a catastrophic situation

if those, machine are connected

together and they start to train themselves. They try to

build a meta architecture and try to train themselves,

and then they come up with something which is better than human. Some some people

call it the singularity point. So this is frightening. They're smarter

than us. Maybe they they're gonna kill us all. And

people say now people speak about regulation now, and there are

several institutes in Europa, in Europe and in, the US

trying to tackle that. And that

is amazing. That is really important, but I think we missed something here.

And I'll tell you why. So the so there is a book. It's here.

You know, Isaac Asimov, I, Robot. You probably

know that. So he, like, the first page of this book is like the 3

laws of robotic. A robot may not in in injury a

human being or through an interaction, allow human being to come to harm.

A robot must obey others and so on. So we have let's say

we have the regulation. AI cannot hurt humans. Okay?

But that doesn't enough. It's not good enough because if the AI is smart

enough, it will not do the I mean, it will

show us humans that it really obey the law

the laws, but it wouldn't. And this is frightening.

And here I suggest to look a little bit about the human morality

and what why human are have do they have laws? So we need to

think about, if I may, think about the

human psychology. In human psychology, we have a mechanism to obey law.

It's called the superego. It was embedded or defined by

Freud. So we have a mechanism that if we

if we doesn't we if we don't obey a law, we feel either

guilt or fear. And this mechanism was evolutionary.

So do we have a group of monkey? They obey

the the alpha monkey because they're frightened from him. They have some kind of

primitive superego. We obey the law because either we fight them from the

police or either we feel the guilt, we

we it's like the

those experiments that show that, there is, somebody,

left something on the table, and we don't take it because we feel guilt or

we feel something. So this is this mechanism, what

I claim, should be transferred to the

AI machine. This should be the regulation. So what is it superego? Superego

is a infrastructure for to be moral,

and we need a digital version for that for the this is the regulation we

need. We need the infrastructure to be moral in machine. And what it what

does it mean? So superego means that it's a little bit like

self harm, if I may. It's like we feel guilt. We feel something bad if

we do something not okay, if you're not obey the law.

So it's like a self destruction for AI machine. So AI machine,

if it doesn't obey the law, should feel something. It

cannot feel so. Right. It will distract itself. So this is my

claim. This is a book I'm writing, and this is something very fun fundamental.

We we all speak about this regulation, but I think it

it doesn't help just to to do standard

regulation. And if you if I may say another thing, the last thing is that

if you read the I, Robert, carefully, so

he speak there are several short stories there, and he speak about robots that

obey the law. And if you look carefully about those robots that

obey the law, the those robots have super all

all of them have have super ego. They feel guilt.

The the first story is about a robot that play with a girl,

and he feel guilt about winning all the time. So he let her win.

So he feels guilt. It means that it has superhego.

And then he feels frightened from the mother of the girl. And it's

really amazing. So I think, so

this book I'm trying to describe the psychological concept of superego

and then describe why it need to be more and how we can,

find a way to put it in regulation, like the the infrastructure

itself and not just lows.

That is a very interesting problem you're trying to solve.

Very important problem at that. Agreed. And

culturally, we speak, in the US, we have a saying that you

cannot legislate morality, which

legislate, regulate would be, you know,

synonyms. Exactly. Right? So Right. Right. And and legal code

is code. I I

definitely get what you're what you're saying. And I think it's super

important. You mentioned you were writing a book about this. Now

now now you have to tell me more because I wanna read this book.

Same. I'm in the process of looking

for an agent and it's, it's complicated. It's supposed

to be a popular book trying to explain the psychology of fraud.

What is, superego, ego, and the id,

and then describe what is the pathology? So we all have a pathology. So

you have the pathology of, it's called,

the, personalities criminal personality disorder. This

person will not have a super ego, ego ego. It's like Richard the

third from Shakespeare. He didn't have superego. He killed

his family and didn't feel guilt. So this wouldn't what's

going to happen with the with the with those machine. And then I

give some literature examples of,

what is a superego like from the, criminal and

punishment that that the guy killed the the

old lady, but he didn't he nobody,

caught him killing the lady. He murdered her. Nobody caught him, but he

still feel guilt. So he has a very, big

superego. And then we describe I describe, what happened in

other moral theories of human being, all of them connected to the

superego. And then I tried to describe a little bit how machine

learning is trained. Again, solving an optimization problem. And then I try

to describe how can we do superego with, how can we have

a digital superego if we can? No.

It's like you're giving it a conscience of of sorts. Exactly.

Yeah. And I I just wanted to, to add, we

may be able to help you. Maybe not find an

agent, but find a publisher. Both Frank and I are

published. And we, you know, we know Andy has a lot of

Andy's got a lot of connections in the publishing. Well That would be

great. I am I am not, I just wrote a lot of books

for different, publishing houses, and I know some people that if

they can't help you directly, they can probably point you to someone who

can. And, again, I am wholly motivated by wanting to

read this book. Same. Like, I think it's important

because I live in the Washington DC area. Right?

So so, like, there's a lot of people there who they're policy

makers. Right? Like, and they just assume

and I think a lot of humans fall for this. Right? You you see this

when the European Union passed their AI regulation act.

They assume that regulation's gonna solve all their problems.

And I think regulations prove that 1 of the fundamental forces

in the universe is is unintended consequences.

And, you know, when you regulate something, you don't end

the problem. You change the way people will route around it. Right? Like,

and I think a good example of this in AI is the movie Megan, which

I don't know if you've seen, or m threagan. I'm not sure how to pronounce

it, where I think she was about to torture

she was I don't wanna give the plot away, but the the robot

child, Chucky, kinda goes evil, Like, this is the

basic kind of plot line, and the the the person who created her

was like, you can't kill me because it's against your programming. He goes, oh, I

said nothing about killing you. I was gonna put you in a coma, and you'll

live, you know, however many years. Like, it was just like I mean,

that's a great example of, like, she you know, don't kill. Right? Seems like a

pretty reasonable instruction to give a robot, particularly a child's toy.

They'll kill anyone. But, you know, she was realized, like, well, kill

equals death. So if I don't kill you, if I just hospitalize you or

incapacitate you, that doesn't conflict with rule number 1.

Right? Which I think is no. Obviously, as, you

know, humans, we're like, well, it's not really the spirit of the

law, or the rule. But clearly,

the robot or the AI in this case, kind of figured it

out. Like, I don't know. I think you're right. Like and any regulations like that

too. Right? How many loopholes do people discover, whether it's

tax laws or, you know, this. It's like, well, technically, it's

legal. Is it actually, you know,

what the law intended? No. Like, it's Yeah. You need a you need

almost an something like a Nuance engine,

you'll see to Yeah. To get the the

what the machine to interpret

to the laws. And that's I've read Asimov as well,

big fan. And that's what happens down stream of

the 3 laws as they begin to fail as because the

robots are doing exactly what they're programmed to

do. And they're not they're they're

finding ways that in our opinion, human opinion,

circumvents the 3 laws, but really doesn't

break the robot's programming. And it's all about, you know,

how do you define harm? Like, Frank's example is a great, you know,

great example of that. So, yeah,

fascinating stuff. Yeah. We gotta Awesome stuff. We gotta help you write this

book. I wanna read this book. Yeah. I want to raise

another point, but the opposite point that you raised. Like, what happened with

the autonomous car, for example, or people say,

let's let's let's focus on autonomous cars. So so there will be

autonomous car. Who is in charge of a of a car accident?

Accidentally, somebody was killed. You are the

owner you. Somebody is the owner of the car. He sits

there. He bought the car, but the car killed

somebody. So

who who this is an open problem. This is, again,

moral problem. So what I suggest here is

maybe it will take time,

I guess. Maybe the the car, if we can be the

superego and mechanism for morality, you know, the just

the infrastructure for morality can take the

morality of the human. And if somehow he

inherit the the the driver morality, you

can blame the driver. I'll give you another example, which will be much

more maybe concrete. So we say now that there will be change GPT for

every person, for every laptop and iPhone and whatever.

You will have your own GPT with your own life follows

your own history. And the discussion with this GPT will be, And the

discussion with this, GPT will be very personalized and

very helpful. What happened in that case? So in that

case, if this, GPT

will take your responsibilities and morality, somehow we

can copy your morality and be part of it. So if you're moral, it

will be moral. If you're not, you're not, but this is

your responsibility as a human. And I think this

is the way to to go with that. We need just the infrastructure and not

the the law. Anybody can define the low, and anybody

can break the low. We just need the infrastructure to know that

at least the machine to know that it break the broke the low.

And and this is really important. I I think

Oh, I totally agree. Totally agree. Well, we're

gosh. We're coming up on time, Frank. Yeah. This was

awesome. So we'll just any

book recommendations? Obviously, I, Robot, I think, would be good reading

in this space. You also mentioned Shakespeare too,

Richard the 3rd. So Eddie, you can book

which I'm which I'm reading now, which is the band,

Vernon Stuputeux. It's, it's

amazing. It's amazing. It's 3 books, and it's actually

discussed whatever which is not AI. Anything which cannot be solved with

AI. It's speak about a a person who has a vinyl shop,

shop to sell vinyl and then CD runs, and now we cannot sell

anything. So this shop is is closed, and then he

he he try to somehow manage, but he get up at the street. He's, like,

homeless, and he meets many people. And the way like,

every chapter is a different, person or

or a group of pair of people, and it's really

fascinating. It's all those things that you cannot solve with AI. It's all

the human interaction, the very, very basic human interaction. Amazing.

Nice. Where can folks find out more about

you? So I have a website

under Joseph Keshet, and, and they

can find me there. Excellent.

Any parting thoughts, Andy? No. Just great great

interview. I appreciate that. 1, I would ask if you repeat the name of

the book you just mentioned about the the different stories.

What's the name of that book? It's not it's a it's a single

story. It's called the the pants,

for non subtext. It's from French. Oh, okay.

Amazing. Amazing. Amazing. Awesome. Excellent. That's it. That's

it for me. But that's great talk. Thank you. Excellent talk. Thank you.

And we'll let Bailey finish the show. Well, folks, that brings us to the end

of another enlightening episode of data driven. We've

navigated the fascinating intricacies of automatic speech

recognition, explored the moral quandaries of AI, and

pondered the future of technology with none other than 1 of the best minds

in the field, doctor Yossi Keshet. Remember, if you

enjoyed today's conversation, don't forget to subscribe to data

driven media TV for exclusive video content.

You can also grab some fantastic merch like the my data is the

new oil t shirt Andy's sporting today. And while Frank is

basking in the Appalachian sunshine, you can bet we're already cooking up the

next episode to keep your data driven minds engaged and entertained.

Until next time, stay curious, stay informed, and

always keep questioning. Cheerio.