Max Sklar on Exploring AI, Data Science, and Local Search
In today’s episode, the hosts Frank La Vigne and Andy Leonard are joined by the expert in location data and machine learning, Max Sklar. Max shares insights from his decade-long tenure at Foursquare, delving into the company’s evolution, gamification features, the challenges faced in the local search space, and his early interest in location data.
The conversation explores the enduring relevance of foundational tech concepts, the cyclical nature of technology trends, and Max’s personal journey into data and machine learning. Max also discusses his podcast, “The Local Maximum,” and his diverse interests, including abstract math papers and a project rewriting the US Constitution. Join us as we dive into a thought-provoking discussion about AI, data science, and the ever-evolving world of technology with Max Sklar.
Show Notes
00:00 Foursquare split, confused but loved the concept.
04:29 Rewards program failed due to lack of scalability.
08:44 Early career in New York City’s tech boom.
13:05 Foursquare uses phone data to track locations.
16:25 Models analyzed data to improve sentiment analysis.
20:02 Data pipeline technology used for real-time deployment.
20:54 Python written code, comparing different languages used.
24:17 Navigating reinvention in a changing world.
29:38 Joined wireless generation, now known as Amplify, as a software engineer.
31:53 Machine learning brings data to life.
34:26 Using OpenAI API to create interactive content.
40:03 Technology enables limitless creativity and storytelling potential.
42:12 Enjoys volunteering in underserved communities around the world.
44:36 Extensive library and website featuring various projects.
47:48 Please subscribe, rate, and review our podcast.
Transcript
In today's episode, Frank and Andy sit down with special guest
Speaker:Max Sklar to delve into the world of artificial intelligence, data
Speaker:science, and data engineering. Max, a
Speaker:trailblazer in location data and machine learning, shares
Speaker:insights from his extensive experience at Foursquare, including his
Speaker:work on local search and bias correction. Get ready
Speaker:for a thought provoking discussion about groundbreaking projects, tech
Speaker:drama, and the ever evolving landscape of technology.
Speaker:So sit back, relax, and prepare to be amazed.
Speaker:Hello, and welcome to Data Driven, the Podcast, we explore the
Speaker:emergent fields of artificial intelligence, data science, data engineering,
Speaker:and all that good stuff. With me on this ever present,
Speaker:journey down the information superhighway is Andy Leonard. How's it going, Andy?
Speaker:Good, Frank. How are you doing? I'm doing alright. It's been a wild 24
Speaker:hours in, Maison Levin, or or
Speaker:Maison, Lavinia, depending on your, how you wanna pronounce it.
Speaker:At one point, we will share those crazy
Speaker:details, but it's been good most of the part. I I am,
Speaker:recovering from a, a bout of COVID,
Speaker:that hit the entire house. I think we all picked it up on the cruise.
Speaker:Again, there are worse places to pick up COVID, and there's worse
Speaker:things that could happen. I've I've sneezed quite a
Speaker:bit, I've coughed quite a bit, but The thing that's bugging me the most is
Speaker:this headache I had now for 48 straight hours.
Speaker:But it's okay, like I'm kind of living, learning to live with it, and I've
Speaker:actually given it a name, I call it Charlie. So, you
Speaker:know, Charlie is is is gonna be on the show today as
Speaker:well. I see. Yeah. You do sound a little different, but not much.
Speaker:Nah. That's why I need to clone my voice, and, maybe how they
Speaker:do the avatar stays. As I was saying in the virtual dream room, this is
Speaker:the 1st time I've I've felt fit for camera In about
Speaker:a week. Perhaps you can use some of that good
Speaker:AI voice modulation. There you go. We are
Speaker:pleased to present, with us, today is Max Sklar.
Speaker:Max is a, not only a fellow, data
Speaker:guy, but also a Fellow podcaster.
Speaker:Welcome to the show, Max. Thank you so much, Frank and Andy,
Speaker:for, for having me on. I'm looking forward to it. And,
Speaker:yeah. Thank you. Cool. So you you've been at you were at a
Speaker:company for a number of years. You were talking about this in the virtual dream
Speaker:room. You were at Foursquare for, oh, quite some time. Right.
Speaker:That's unusual, actually. 10 years at a at a company as as, you
Speaker:know, building software is pretty unusual. But I was really there For, I
Speaker:think, like, 3 different phases, so I kind of break it up into 3
Speaker:jobs. Interesting. So, you know,
Speaker:Foursquare is one of those companies that
Speaker:On when they broke up into 2 different parts, it kind of I didn't
Speaker:understand it. Like, this is just user. I'm not I'm not I know 1 I
Speaker:know that sounds terrible, but despite being my headache, Charlie talking, but, like,
Speaker:what, you know, obviously, that was a decision
Speaker:that was made in the marketing department, and I don't think people probably thought that
Speaker:through. But I love that location. Like, I I think, you know, like, I was
Speaker:the mayor of, like, 5 6 different places. It was such a cool
Speaker:creative, concept that if you go there long
Speaker:enough, you're the mayor, and some places would offer the mayor, like, a free coffee
Speaker:and a donut or something like that. Like, it was really clever.
Speaker:I know. I used to I mean, that was one of the things that got
Speaker:me really excited about it back in the day. And this is already 10 years
Speaker:ago. I assume you're talking about the, The app split back in
Speaker:2014, which I guess is That sounds about right. I I guess
Speaker:almost 10 years. I guess it's It it's actually been 10 years since we started
Speaker:working on that, which was also how it worked. But,
Speaker:no. I I mean, I actually remember a lot of what was going on
Speaker:at the time, and, we could talk about that a little more in a
Speaker:second. I I I I I'm happy to talk about that.
Speaker:I I think the the whole gamification thing was really
Speaker:exciting. And I really loved the idea that I can go into
Speaker:a place and be like, hey. I was here for 5 times. Oh, we're gonna
Speaker:give you some free, you know, free, you know, dessert or
Speaker:free chicken nuggets or whatever. Maybe these days, you know, kind of trying to watch
Speaker:what I eat a little bit more, it might not be as exciting. I I
Speaker:it was really sad that that didn't scale as a business. I think what
Speaker:would happen was, and and this I both experienced
Speaker:this personally. And also from talking to the leadership there, it was clear
Speaker:that they felt it it, You know, it it it wasn't a
Speaker:sustainable thing because you'd often go into a place and say, hey, you know,
Speaker:I checked in here all these times. I got all these rewards. You know,
Speaker:I'm supposed to get, like, a dollar off, or I'm supposed to get some
Speaker:some free dessert or something, which is It's always exciting
Speaker:to get a little reward, even if it's not, you know, even if it's just
Speaker:a dollar or whatever. It's always but I think what would end up happening is
Speaker:like the the man, the The management who worked there or the or the
Speaker:bartender or whatever would know about it. And, you know,
Speaker:once a few times that, you know, you got that thing. Oh, oh, let me
Speaker:go in the back and check. And then they come back in, like, 20 minutes,
Speaker:and it's like a whole big deal. They're like, maybe I don't wanna do this
Speaker:again. That's interesting, because, like, from my point of view,
Speaker:it was always very It was very fun, like, so I was mayor of,
Speaker:there used to be a ferry service between, in the
Speaker:suburbs of DC, between Poolesville, Maryland and
Speaker:Leesburg, Virginia. And, I would take that ferry a lot so
Speaker:much. So I was actually the mayor of the ferry. It didn't get
Speaker:me anything, like, in that case, but it was this kind of cool, like I
Speaker:I love to I I always imagined, like, there would be, You know, like
Speaker:the ferry could have some kind of a a little like TV screen that could
Speaker:like show who the mayor was. And then you can, you know, and and and
Speaker:then then then we can like kind of scale up those fights, But alas,
Speaker:society didn't go in that direction. Interestingly,
Speaker:Foursquare, what, like, I have been interested in that this like local
Speaker:local search space since since well before forest grabbing even my, you
Speaker:know, my my senior project as an undergrad, back,
Speaker:though, starting in 2005, was this, like, you know, this this website
Speaker:called sticky map, where people would post little, Icons all
Speaker:over Google Map. It's kind of inspired by Wikipedia. You can add messages.
Speaker:And I just thought it was pretty cool. People started marking up the, the campus,
Speaker:and then people started, you know, marking up. It was like, well, it's based on
Speaker:Google Maps API, so you could just mark up anything you want. And
Speaker:I think in that first project, I noticed All the problems
Speaker:that still exist with that kind of data today. Okay. What happens when you
Speaker:add duplicates? It's the first thing that happened. As, you know, As an undergrad, I
Speaker:was like, oh, I'm so excited. Let me show something like this. They're like, okay,
Speaker:let me create a marker. And I'm like, don't create that one. It's already been
Speaker:created. And then it's like, okay, now we have duplicates Right off the bat. And
Speaker:that is still something that, you know, Foursquare
Speaker:deals with and I'm sure Google deals with and Apple Maps deals with and then
Speaker:they they all deal with it. So,
Speaker:Yeah. It it it it I I think when I
Speaker:discovered Foursquare, you know, several years later,
Speaker:it was the it was innovative in several ways. It was, first of
Speaker:all, it was based on on mobile apps actually being at the place when you're
Speaker:commenting on it, which is exciting. It was the gamification of it. And the
Speaker:fact And and for those listening Yeah. For those listening, that was new. Like,
Speaker:that was brand new. So sorry. I didn't mean to cut you off, but, like,
Speaker:the context is important because All I had was was something on a on
Speaker:a website. You know, I I wasn't thinking of the I the iPhone didn't
Speaker:exist, for actually, pre Foursquare,
Speaker:there there was something called dodgeball, which was kind of the the
Speaker:predecessor to Foursquare, Which I wasn't involved with, but it but, it
Speaker:it was based on, like, you know, SMS kind of text messages where people were
Speaker:messaging on. If you remember, You know, those, you know,
Speaker:the the the what was it? The t nine texting where people were Oh,
Speaker:God. Yeah. Yeah. Yeah. But people would use that and and Foursquare and,
Speaker:Google bought that. And then the the the founders,
Speaker:Dennis Nautz went it went in and started Foursquare after that.
Speaker:So, very, there there's a very
Speaker:interesting history of kind of, like, Local search, city guide,
Speaker:and basically sort of social kinda local applications. We're
Speaker:we're very big at the time. Nowadays, I think we need to find a new
Speaker:take on it, but, that when I joined Forescord
Speaker:in 2011, it was very exciting. I'm glad you mentioned that because there was
Speaker:a a real so I started my career in New York City. Right?
Speaker:So I worked at Barnes and Noble com. So I was there in the fairly
Speaker:early days of Silicon Alley, and that was a huge thing. It was
Speaker:Microsoft. I think it was Microsoft had something called Sidewalk.
Speaker:And then there was there was maybe it was maybe it wasn't Microsoft. Maybe
Speaker:somebody else had something called sidewalk AOL had something, going and the fact
Speaker:you can't remember it, I think says it all. Right? Like, it was like in
Speaker:in, you know, anybody that could register a .com could spell and could
Speaker:spell HTML To get funding back in those days, a
Speaker:bit like the way the AI startup ecosystem is kinda
Speaker:today. But, no, there was I mean,
Speaker:people there was a time, young children
Speaker:out there, That when, you know, people saw the online
Speaker:world is slightly different than the real world, and they saw this as an opportunity.
Speaker:Right? But no, I'd like That takes me back. As soon as
Speaker:you said to, like, the local kind of connection guides, I was like, wow, it
Speaker:takes me back. Yeah. Yeah. Coming back to
Speaker:the app split, and I wasn't expecting to talk about this today. I'm sorry if
Speaker:it brings up If for what it's worth, I'm a former Windows For what it's
Speaker:worth, I'm a former Windows phone developer, and I wrote a book on silver light.
Speaker:So I understand the pain of working on
Speaker:It'll fade and I worked at barnes and noble.com. Right? So there's my trifecta of
Speaker:ill fated technology projects. Yeah. I I think it's
Speaker:A lot of technology companies, in order to become successful, actually have
Speaker:to go through big changes where people yell at them.
Speaker:And so it's like, how do you know whether you're breaking things or whether you're
Speaker:actually doing what you're supposed to do? And so that's kind of a Right. That's
Speaker:that's kind of a tough decision. I I think For Foursquare
Speaker:at the time, there was always, like, kind of a design,
Speaker:and product, like, tension between the people who
Speaker:wanted to be there As essentially like a Yelp replacement, kind of
Speaker:like a a local search city guide. And then there and then versus the people
Speaker:who were there for the the life logging, the check ins, the game. And I
Speaker:think, I think the separation could have been
Speaker:done. I mean, my personal thing is I think there could have been a
Speaker:Separation, I could think, could have been executed a little bit better. I
Speaker:think technically we did a good job. I think the apps that
Speaker:we ended up with were well designed, but I think, I think
Speaker:we needed to do is take into account how the how people were using the
Speaker:apps, at the time and not just, like, Kick all the
Speaker:people who are checking in, which was which was Foursquare's kind of bread and butter
Speaker:and just, like, kick them to the side with this other app. And then it's
Speaker:like, well, what is this? I'm calling this something different. Sorta. That was that was,
Speaker:I think, too much. But, again,
Speaker:there there were people saying it at the time. But
Speaker:The problem is, I I guess, you know, there whenever you make a
Speaker:change, there's always a great many people saying a great many things. So
Speaker:We could wax this we could wax nostalgic about Foursquare because I used to
Speaker:like, when I I was travelling a lot at the time when I worked I
Speaker:worked at Microsoft about 10 years ago. But I remember Sometimes I would actually
Speaker:choose different connecting airports, so I could get, like, the the
Speaker:jet that was it, the the jet set tag, like level up in my achievement
Speaker:there. Right. Which is kinda sad, but, we could
Speaker:we could wax nostalgic about that all day, and I would love to. But I
Speaker:think what What was the role of AI and ML in that
Speaker:space? Right? Because you're obviously collecting a lot of data. No. Like, I'm just curious,
Speaker:like, because how how was that being used? How was that,
Speaker:leverage. In in in a lot of ways,
Speaker:and, you know, many of which I I worked on over all those years.
Speaker:You know, one of them was, I mean, just, you know, search
Speaker:ranking in general, which, you know, Foursquare had a lot of ex Google engineers.
Speaker:So I learned directly from them so they they knew what to do.
Speaker:But search ranking, search ranking was a was a was a big project.
Speaker:This is kind of more of a statistical Problem where you were kinda trying
Speaker:to weight different attributes, like, is this related to the search the
Speaker:person put in? Is this related to how much do we wanna, You know,
Speaker:score things that are, you know, maybe someone's
Speaker:friends went to. So something like that. I think that
Speaker:The biggest well, I'll talk about the one that I think is the biggest deal
Speaker:and then the one that that I worked on the most. The one I think
Speaker:was the biggest deal for for Foursquare, which I did work on a little
Speaker:bit, is basically trying to figure out where
Speaker:someone is given. So we know where someone is given there that long
Speaker:from their phone. But it's like, what are they actually in a particular
Speaker:store? Like, are they in the Starbucks? Are they in the, you know, are they
Speaker:in the office, over there? Are they Are they just walking down
Speaker:the street? And so using the fact that people were were
Speaker:giving us training data, essentially, which was a big theme there, which is, you know,
Speaker:I think, something that,
Speaker:data scientists and data entrepreneurs need to need to look in closely, which is
Speaker:like, How can you get people to give you training data? Because it is really
Speaker:useful. So if you have people giving you where they are and
Speaker:then you could see the information from their phone, not just a lot long, but
Speaker:like what, You know, things like what Wi Fi's can you see? What, you know,
Speaker:other sensors from your phone, can you figure out where they are? And then there's
Speaker:the whole stop detection, problem. And so, Yep.
Speaker:Foursquare essentially can kinda figure out, you know, where you went day
Speaker:to day, and it's actually pretty good. Like, you know, if I Don't tell Foursquare
Speaker:where I went. Even today, I still look at it and, you know, it tells
Speaker:me what, what actual stores I was in. Now maybe there's a question of,
Speaker:you know, whether whether our apps are knowing too much about us, but
Speaker:that's that's a whole another question. But that was a very important,
Speaker:a resource for the business. And the one that I worked on the most, that
Speaker:was the most exciting though for me was the natural language
Speaker:processing Pipeline. And, of course, you know, text text
Speaker:data today is is is having such a a resurgence
Speaker:with, You know, I don't need to tell your audience with AI and all that.
Speaker:But, you know, it it it back then it was like, well, people were giving
Speaker:us, you know, several sentences called tips on Foursquare Venues,
Speaker:which would often be like, here's, you know, here's what you should do here. Here's
Speaker:what you should try. Here's a little review, something like that. People were
Speaker:leaving text with their check-in. So there's a bunch of texts, there's
Speaker:menus, things like that. So there's a bunch of texts in the system. And so
Speaker:it's like, what do we do with all of that? And, one of the things
Speaker:that we did was we pulled out key terms, you know, noun phrase detection.
Speaker:This is all kind of standard natural language
Speaker:processing. You know, not
Speaker:you know, you know, people often ask, oh, you know, I I think
Speaker:Nowadays, I'm often thinking everyone's thinking, oh, you were probably using, like,
Speaker:generative AI or something. No. It was just kind of standard NLP that had
Speaker:been developed over the last, you know, several decades. But,
Speaker:we did sentiment analysis and we used that to come up with the ratings for
Speaker:the venues, which which are used today. So you could tell
Speaker:how good something is. And,
Speaker:you know, I did some things that were a little bit
Speaker:more interesting that, you know, maybe get overlooked, but they're
Speaker:they're kind of unique to To to what we did there, which was sort of
Speaker:like timeliness and seasonality, which is so, like, if you check into
Speaker:a diner in the morning versus in the afternoon, It'll statistically
Speaker:give you different suggestions based on how timely it thinks each
Speaker:each suggestion is. Because with every Check-in where someone is doing
Speaker:something in real time. We have the timestamp. We know what time of day. We
Speaker:know what time of week. We know what time of year. And so it's kind
Speaker:of cool to to put that all together. And some of
Speaker:the some of the, some of the
Speaker:models, got pretty,
Speaker:you know, it was it was pretty neat how it all turned out. I think
Speaker:that one I you know, I still talk about that one is one of my
Speaker:That's my favorite one after being in the industry so long, even though it was
Speaker:like 10 years ago, because it was like, okay, we had training
Speaker:data again from on these tips where, You know, we
Speaker:could tell if the person liked the venue or disliked the venue and because
Speaker:they they told us, and they also left the tip. There were a lot of
Speaker:people who did that. So that just gave us training data for sentiment analysis.
Speaker:And at the time, I'm sure the tools now are much more sophisticated at the
Speaker:time when we use pretrained sentiment analysis tools, Didn't really
Speaker:work well on our data because it's just it was just a different kind of
Speaker:text. People wrote on Foursquare differently than they did on Twitter, for
Speaker:example. So, so that gave us training data. Give
Speaker:us training data for every language. And so that was nice. We got kind of,
Speaker:like, you know, 90 languages for free just by just by
Speaker:using that Strategy of Oh, wow. Using the data that people gave us.
Speaker:Probably not probably didn't work very well in all 90, but certainly worked
Speaker:well. Well, the beauty of it is It ends up working
Speaker:well so long as we have good language detection, it ends up working well
Speaker:in, any language that has any Particular,
Speaker:you know, any particular popularity in Foursquare.
Speaker:So for example, if, the Turkish was very popular. Okay. Well, that
Speaker:means we have a lot of Turkish training data. That means that the
Speaker:the model, which trains monthly, is Is going to use all that training data. That
Speaker:means it's going to work very well. And so, and
Speaker:so that the fact that the models were always regenerating
Speaker:And they were always regenerating based on the latest data
Speaker:was was really cool because oftentimes you think these think about
Speaker:ML teams kind of building a model, and then they kind of throw
Speaker:it over a wall. They they productionize it. And then you have to
Speaker:work on the next one, but you have to you have to do some work.
Speaker:It's not automated, you know. So it's like, well, this is this is gonna start
Speaker:going downhill If we don't, if we don't interact. And the fact that we were
Speaker:able to set it up where it was just constantly getting smarter was,
Speaker:was pretty neat. So MLOps and pipelines before they were called
Speaker:MLOps. Well, they might have been called pipelines, but yeah. Interesting. Yeah.
Speaker:Pipeline was a big Big big key phrase. So what
Speaker:what did the data what did the back it because like one of the one
Speaker:of the jokes that we have, and in fact, it's a domain name that I
Speaker:registered. 1st, you get the data, is a phrase that
Speaker:a lot of data scientists will often use, much to the chagrin of a lot
Speaker:of data engineers, because a lot of data,
Speaker:you have to get the data in a certain way to to format it and
Speaker:and and to get it trained. And if you go to first you get
Speaker:the data.com, it should redirect you to our website,
Speaker:hopefully. God only knows if it works. 1st, I think
Speaker:yeah. I'm gonna I'm gonna try to get the data.com.
Speaker:I'm shattered to think that okay. Good. It does work. Okay. DNS
Speaker:and me have a long history. Yes. It's going back.
Speaker:Good. I I it's always good to start off a week with a win with
Speaker:DNS. What did it what did the
Speaker:because I'm curious, like, Foursquare was one of those early, kind
Speaker:of mobile first, kind of success stories. I'm
Speaker:always curious, what did the back end data platform look like? Right?
Speaker:Because, and again, going back 10 years, I
Speaker:mean, I mean, did you use what was the name of the,
Speaker:gosh. Can't think of the name of the platform, but what sorts of technologies did
Speaker:you did you guys use? Yeah. I'm I mean, I'm sure
Speaker:it works similar today in in at at Foursquare. Mhmm.
Speaker:Well, we were using data pipe I assume. But, yeah,
Speaker:if I remember correctly, we had, you know, our transactional database, our Mongo
Speaker:database that was sort of like, Every once in a
Speaker:while. And so that was kinda like the baseline. And then there'd be a series
Speaker:of jobs that, like, you know, built it up, that that
Speaker:that kind of Calculated things off of that, and that
Speaker:would, in the at the end of that pipeline, you know, release
Speaker:a, a dataset that would then be kind of,
Speaker:Automatically, deployed and then read by,
Speaker:read by the server in real time. So, if I can think of, like, the
Speaker:technologies, I think the, the pipeline technology, the
Speaker:pipeline, what was it? It was
Speaker:like Luigi. It was written in pipe. Python. I don't know if that's too interesting.
Speaker:There's a lot of different ones you could use these days.
Speaker:It's an interesting question of, like, you know, which one do you use? I,
Speaker:it's It's probably,
Speaker:you know, from from my point of view, it's always like, well, the company kinda
Speaker:chooses it. You don't really have much of a say.
Speaker:And then then it's like, well, well, how do I know how to compare? But
Speaker:let's see. Like, you know, we were using MapReduce jobs. We're using Hadoop At the
Speaker:time, I think scalding, was was one that's that's maybe kind of out of fashion
Speaker:now. That was a, a scala based framework for for
Speaker:some of these jobs which were, which which
Speaker:was based on abstract algebra. So it's actually pretty cool. I wish it
Speaker:was. It was kinda hard to to reason about sometimes if you
Speaker:It kind of went too far, to the side of, okay,
Speaker:you know, I love abstract algebra, but I don't want everybody
Speaker:who I don't I don't want that to be a barrier to entry for people
Speaker:who are we're working on this. But, I'm
Speaker:just trying to remember, like, some of the, You know, some of the some of
Speaker:the tech bud buzzwords. But if you have any specific questions, maybe they'll jog jog
Speaker:my memory. I don't know. Like, one of the things that was popular about that
Speaker:time was, HBase. Oh,
Speaker:yes. I, were we using interest? I think we were using,
Speaker:Yeah, I remember that Term, but I I know. I know. I was as you
Speaker:were talking, I'm like using it or if we wanted to use it. It was
Speaker:one of those 2. Now from that if memory serves, I think Facebook is the
Speaker:one who pioneered h base because it was really it was a right once read
Speaker:many thing, and basically, the last one in when,
Speaker:last last I can't talk, sorry. Last one wins.
Speaker:Let Andy help might help me out if with the
Speaker:technical term for that last one. Last one wins. What's the
Speaker:Oh, yeah. So right. I remember they were called h files, so it must have
Speaker:been yes. It must have been that. Yeah. Yeah. Yeah. That was one
Speaker:of those sorry, Andy. Go ahead. That's okay. You were I was thinking,
Speaker:you know, ChipLogic last in first out. Yeah.
Speaker:Yeah. Something like that. Yeah. Know if that's what you were after or not. Last
Speaker:one wins. Last one That was their concurrency strategy.
Speaker:That's, I know there's a better term for that, but again,
Speaker:it's a Monday and, I have
Speaker:a headache. But no, it's it's it's it's
Speaker:fascinating to kind of Almost like technology
Speaker:archaeology. Like what worth the big projects
Speaker:that were popular at the time. Right? You know, and it's
Speaker:just, And it's scary to think that, you know, we're talking 10 years
Speaker:ago. I mean, I mean, you Not even though. A lot of this stuff was
Speaker:I mean, a lot of this stuff is probably still in place at Foursquare today.
Speaker:Yeah. I mean, what's interesting to me is you you mentioned a lot of
Speaker:the NLP, techniques that, You know, for lack
Speaker:of better term, people would consider legacy now, right? Because they're pre
Speaker:transformers, right? They're pre GPT, right? Sentiment
Speaker:analysis, a lot of, you know, I I speak with a lot of people with
Speaker:varying degrees of technology skills, and they
Speaker:assume that this field of research didn't exist prior to
Speaker:last year. And, very
Speaker:much not the case. It's just that radically changed about a year ago.
Speaker:Right. I mean and and this is something that I'm trying to figure out how
Speaker:to do, which, I I might not be alone. It's like, okay. I did
Speaker:all these things. How do I reinvent myself now in this new world?
Speaker:And, you know, once you realize it could be exciting thing, then
Speaker:it's maybe not so much of a drag, you know, because there's there's so many
Speaker:opportunities out there. But it's like, but I can't be
Speaker:the only one out there who's struggling with this being like, okay,
Speaker:wow, I've got a, You know, I I've gotta,
Speaker:you know, work or at least do projects for companies that
Speaker:are at the cutting edge here in order to, In order to be,
Speaker:you know, in order to be at the forefront. Yeah. It's funny, like, you miss,
Speaker:like, I was, You know, offline for I tried to be offline, but for the
Speaker:better part of a week and for vacation.
Speaker:And like During that week, AMD announces that they are
Speaker:producing their own, GPU LLM
Speaker:type hardware. Gemini comes out and all
Speaker:these other innovations that come out, and I'm like, I feel, like, hopelessly behind
Speaker:now. I'm being offline for a week. Yeah.
Speaker:Yeah. It's I mean, I I guess the
Speaker:only, consolation there is everyone's dealing with that.
Speaker:Right. You know? Yeah. Right. And Kinda like impostor
Speaker:syndrome. Right? Yeah. Yeah. I think I think the
Speaker:question is, especially in this new world of generative
Speaker:AI. And and the question I'm asking I don't necessarily have answers. But it's
Speaker:like, how do you so You wanna jump in the stream
Speaker:and get all the latest stuff, but you also want to leverage your experience
Speaker:and understanding. Cannot be leveraged. And
Speaker:so what's the best way to to, you know, what
Speaker:what's the best way to balance that? I think that's something that I would like
Speaker:to see more people asking. And I would like guidance on this. I know I'm
Speaker:the guest. I'm supposed to say what I know, but No. But try now. You
Speaker:know, Mads, the, the thing is a lot of the stuff that's
Speaker:That's new. I'm doing the air quotes here for people who are listening.
Speaker:A lot of things that are new are really coming out of tech That was
Speaker:developed. The math was developed, for instance, in the late sixties, seventies,
Speaker:eighties, nineties. So a lot of that is just being reapplied
Speaker:Back when the math was developed and the theorems and and such,
Speaker:we didn't have machines fast enough to do it or at least do it
Speaker:usefully. So I wouldn't feel bad at all about,
Speaker:you know, having a bunch of, a bunch of experience that seems dated
Speaker:right now because A couple of weeks to a couple of months. That might
Speaker:be the new shiny. Right. That's true. When I went back to
Speaker:a a university computer science program, You know, they're still studying
Speaker:data structures and algorithms. It's still very relevant.
Speaker:And, you know, I think a lot of outsiders think, oh, everything's
Speaker:gonna Turnover in, in a year and a lot of things
Speaker:do. But there are also a lot of kind of like universal,
Speaker:kind of, there's a lot of universal Theory that's, good to know
Speaker:about. Sure. The fundamentals don't change that often.
Speaker:Nope. And it's a lot of reapplications. I see a lot of people
Speaker:reapplying stuff 2 or 3 times. I mean, I've been I've been
Speaker:around computing since 1975. So I've seen kinda like these meta
Speaker:patterns flow, you know, through several generations,
Speaker:and they kinda keep just resurfacing. One of one of the
Speaker:interesting ones is, like, the, well,
Speaker:both the chatbot and the text based interface versus the,
Speaker:Graphical interface seems like we keep going back and forth. You know,
Speaker:I I remember chatbots back in the, you know, AOL days.
Speaker:AIM days probably way before that too.
Speaker:And then, you know, and then There was kind
Speaker:of a a a chatbot resurgence in, you
Speaker:know, 2016, 2015, whenever when every company wanted
Speaker:a chatbot and we're excited about that. Yeah. It didn't quite work. It
Speaker:seemed to fizzle out. Then, you know, the
Speaker:the then nowadays, we have So many chat interfaces,
Speaker:chat GPT and and generative AI seems to be resurgent again.
Speaker:So there are these weird sine waves, these weird
Speaker:cycles, and I almost think of it as a coil where, you know, you're starting
Speaker:at the bottom and you're cycling, but you're also moving up at the same time.
Speaker:And so How do you how do you surf the wave? That's, that's,
Speaker:something that's once you kind of, understand the
Speaker:fact that that's what you're doing, then then then you can be excited about it.
Speaker:I I think it's fair. Well, we're at that point in the show
Speaker:where we transition to our, questions. And, we
Speaker:dropped them into the chat here for you. Our very first one is how did
Speaker:you find your way into data? Did data find you or did you find
Speaker:data, Max? Interesting. Well, I
Speaker:guess I was always interested in math and computer
Speaker:science. You know, going back to undergrad, you
Speaker:know, it was like there was a lot of different areas I could choose. I
Speaker:had a hard time going into a field that, you know, where I
Speaker:wasn't, using all different parts of my brain and
Speaker:computer science department, it was it was not just the
Speaker:mathematics. It was, you know, there was, you know,
Speaker:there was a bunch of creativity in it as well. There was human computer interface.
Speaker:There was it. So, So I was kind of, I gravitated to that field
Speaker:as an undergrad. When I graduated, I I joined a company
Speaker:called wireless generation, which, today is called Amplify.
Speaker:And that's it was an education tech company. And I was
Speaker:doing, you know, some simple kind of software engineering work. Actually, back
Speaker:then, It was, which sounds really dated now, but, you know, they
Speaker:were probably doing this up to, like, 2010, which was, you know,
Speaker:writing c plus plus for the palm pilot. You know, we yeah.
Speaker:Because it was they were assessing students and then it would sync to to
Speaker:the web and all that. And Sure. It was a lot of, like, taking
Speaker:stuff, Taking that information out of databases and putting it into a a
Speaker:dashboard. And it was it was you know, I I felt like there
Speaker:could be something more interesting I was doing even though I love kind of the
Speaker:mission of that company there. So I ended up in grad school. I ended up
Speaker:at NYU and I went there from I guess
Speaker:2009 to 2011 really discovered,
Speaker:you know, data mining, was the 1st related
Speaker:class. Then I took, You know, machine learning, natural language processing. Actually,
Speaker:the the machine learning class was with, Jan Lacun, who is,
Speaker:a very well known machine learning researcher. He's Like The Lani.
Speaker:The the the the the the the the the the the the the the the
Speaker:the the the the the the the the the the the the the the the
Speaker:the the the the the the the the the the the the the the the
Speaker:the the the the the the the the the the the the the the the
Speaker:the the the the the the the the the the the the the the the
Speaker:the the the the the the the the the the the the the the the
Speaker:the the the the the the the the the the the You know, all the
Speaker:stuff that exists today. Like, even this was 2010. He would show us a camera
Speaker:where he would point to different objects. He'd be like key, wallet,
Speaker:chair, and it would like, the the the text would appear
Speaker:on the the screen based on what he pointed at. So they knew how to
Speaker:do all this stuff, that that you think of as as kind of
Speaker:it it's it almost seems crazy that that was not, like,
Speaker:and and turned into a product that anyone could use back then that it almost
Speaker:seems crazy that it took you know so long to do it but they and
Speaker:actually it it may have been Used by someone.
Speaker:It's, sure. Maybe we just don't know about it.
Speaker:Sorry. My paranoia. No. No. You're right. I'm sure it was used quite
Speaker:a bit, but it it it's just like what it was that kind
Speaker:of Sitting on his laptop was so much more sophisticated than anything that
Speaker:that I I saw a year later. But,
Speaker:Yeah. So it was That was kind of inspiring. And so it was
Speaker:like, you know, it was
Speaker:to me, it seemed like a much more interesting problem. Well, how do you How
Speaker:does the machine learn? You know? How do you, you know, I don't I don't
Speaker:wanna sit around writing code that's just dead. I want it to To
Speaker:be alive, I wanted to to learn from experience. And so when you dive into
Speaker:that question, well, then you get into machine learning, which is actually Pretty well
Speaker:named. And then and, you figure, okay, well, you need
Speaker:data to learn from, and then that that ends up being a statistical model
Speaker:and so on and so forth. So, you know, when I
Speaker:so Foursquare, was a company that that essentially came
Speaker:out of NYU And, you know, it kind of intersected. So
Speaker:and and they wanted to, to learn from from
Speaker:their data. They wanted to kind of, sort of a
Speaker:to build a data science team. And so I had already been
Speaker:working on that sticky map project, And I was into local search. I
Speaker:loved the the product aspect. I didn't have my new
Speaker:interest in machine learning and LP in there. So it all kinda came together. And
Speaker:so that's why I think that was such a good fit for me and probably,
Speaker:probably would be very difficult to find such a fit again.
Speaker:Our next question is, what's your favorite part of your current
Speaker:gig? And that was, in The virtual green room, you said you
Speaker:kinda had a good story about that.
Speaker:Right. So I don't. Well, I don't exactly have a a
Speaker:current gig right now. I have a bunch of different projects that I'm working on.
Speaker:It was you know, I think It it was on one hand, it was
Speaker:nice in Foursquare to be able to focus on one thing, and I'm gonna come
Speaker:back to that. But I feel like you need these periods, almost like the same
Speaker:as the grad school period That I had, back in 2010 where it was
Speaker:like, well, you're working on a few different side projects, but let's see.
Speaker:Hopefully, like, eventually it'll coalesce into something,
Speaker:you know, something a little bit more long term and permanent. So I'm working on
Speaker:several projects. One is with with the Foursquare founder, Dennis
Speaker:Crowley. And we are Working on a new product, a new
Speaker:kind of city guide where you walk around the city with your headphones in,
Speaker:with your AirPods in or whatever. And We kinda know what you're passing,
Speaker:by. We sort of are are using some of the Foursquare
Speaker:tools that are publicly available that we know about, but also, You know, we're
Speaker:kind of rigging up our our own thing because we've just done it so many
Speaker:times. You know how to do it. We're okay. We know what
Speaker:stores and stuff you're walking past. So what kind of sounds can we play? Right
Speaker:now, it's a bunch of text to speech. Essentially, the way I've rigged it up,
Speaker:the the old version 0, the alpha version is, you know,
Speaker:we asked chat g p t or OpenAI API what to say. So it's
Speaker:basically like you're you're walking down the street hearing,
Speaker:content From OpenAI. Interestingly, OpenAI
Speaker:seems to the the GPT seems to know,
Speaker:stuff about Every place along the way, like, you don't
Speaker:have to go into, like, location based database.
Speaker:It seems to seems to know quite a bit. There is a question of the
Speaker:all the content is there's some interesting content in there, but it all ends up
Speaker:being kind of mediocre. So it's like, okay, well, how do we turn this into
Speaker:something really cool? I think, you know, in the end, having,
Speaker:you know, you know, maybe music and and and speeches and an art
Speaker:project somehow in there, based on where you walk is an interesting
Speaker:idea. So if I could That'd be cool. Yeah. I could be like a
Speaker:platform that people can use, like a cultural version of
Speaker:Foursquare. Yeah. Yeah. And or maybe it's just
Speaker:like an enhancement of the the sounds of the city. Or maybe
Speaker:it's, You know, I mean, a lot of people think, okay, maybe maybe a tour
Speaker:guide. I I don't know. But, you know, it it's it feels like,
Speaker:It feels like there needs to be, a variety
Speaker:of use cases tried because there's there's a lot you could do with it. And
Speaker:and Maybe, you know, if if you put this in the hands of more
Speaker:creative or of of additional creative people, they would,
Speaker:ultimately find something interesting. I'm also working
Speaker:yeah. Oh, I could answer questions about that. But then my other project is my
Speaker:other 2 projects are are kind of interesting as well. Well, I have the
Speaker:podcast, The Local Maximum. So still doing that every week and, you know,
Speaker:interviewing people. Talking about,
Speaker:talking about data, talking about AI, you know, few episodes on the
Speaker:whole. You know, all the drama around OpenAI recently.
Speaker:I I never wanted to become kind of the the the,
Speaker:the the the tech drama, you know, what's it called?
Speaker:TMZ of technology? Yeah. Yeah. But but that's something that happened because
Speaker:I remember, like, last year, a couple years ago, there was all this craziness coming
Speaker:out of Google with, You know, there was 1 guy at Google who said,
Speaker:you know, he thought that the LLM has come to life. And Oh, yeah. And
Speaker:then and then there was a there was A whole
Speaker:bunch of stuff with, like, the the AI safety, you
Speaker:know, seemingly staffed
Speaker:by, people who are a little nutty. And
Speaker:so, it was a And they fired a bunch of
Speaker:people From that team too. So, like, there's
Speaker:definitely, it was something weird some weird mojo
Speaker:was going around. That's for sure. Yeah. And when when I cover that, I
Speaker:mean, it's hard to, you know, it's hard to hide the fact where it's like,
Speaker:wow, everyone in this story seems kinda nutty. But I also try to, you know,
Speaker:I try to take a step back and say, okay, this is what we know.
Speaker:These are a few things that could be happening internally, but we don't know everything.
Speaker:I'm not gonna jump to conclusions. But,
Speaker:I I I I try when I'm covering a story in a local maximum to
Speaker:give, like, a a balanced, a balanced version of
Speaker:of whatever story I come across. You know, maybe it's my show as I try
Speaker:to give my opinion. But, yeah, I I
Speaker:I my attempts, which, you know, some people have have,
Speaker:said I I I've captured that. But my my attempt is to sort
Speaker:of, try to try to
Speaker:approach each Story with a little bit of humility and try
Speaker:to help people understand what's going on without the
Speaker:raw emotion that you get often on on Twitter. Gotcha. That's a good
Speaker:point. Yeah. So we have, go ahead. I'm
Speaker:sorry. Oh, no. No. It's okay. Go ahead. Okay. So we got,
Speaker:3 complete dishonest. And, the first is when
Speaker:I'm not working, I enjoy blank. Right.
Speaker:So now that I've moved to Connecticut, I feel like I
Speaker:am such a a Connecticut stereotype where I kinda, like, Drive
Speaker:around, going for walks in the woods and into
Speaker:various malls and stuff. So it's like it's like it's When the
Speaker:weather's good, you go into the woods. When the weather's bad, you go into the
Speaker:mall. Yeah. So I I actually like enjoy doing
Speaker:that. I enjoy listening to podcasts.
Speaker:I, honestly, enjoy hanging out with friends. You know,
Speaker:after, I used to live in New York City. I enjoyed
Speaker:it a lot, and I sorta had this, situation where
Speaker:I had this be careful what you wish for because, at the end of 2019,
Speaker:I was like, oh my god. I'm going to, like, events every single day. It's
Speaker:just it's just too much. How can we, like, how can we, you know,
Speaker:cut back on that. And then COVID came. And then to me, it was just
Speaker:it was the worst thing because it was like, okay, you stay in your apartment
Speaker:in New York City all day and you don't go and and talk to anyone.
Speaker:And it was just like it it it was just awful. It just
Speaker:felt like a a prison. So I I
Speaker:moved to New Hampshire for a couple years, then I came back. But, you
Speaker:know, nowadays, when I get a chance to hang out with with friends and and
Speaker:family, I just I try to do it, whenever I can because I'm
Speaker:not like, you know, it's not like when I was living in New York in
Speaker:the 2010s and got kind of overload on that.
Speaker:Right. Right. So, yeah. That's my answer there. And we have another complete this
Speaker:sentence. I think the coolest thing in technology is blank.
Speaker:The the way I've been putting it recently Is this,
Speaker:where, you know. It.
Speaker:You know, back maybe 10 years ago, the story we
Speaker:were getting that the hopeful story we were getting was that, okay, if you're an
Speaker:engineer, you could Build anything you want at
Speaker:a very low cost or if you're not an engineer for anyone because we
Speaker:have access to social media. You know, you can,
Speaker:you can put anything out there into the world that you want
Speaker:and and have people read it if if if they want to, or have people
Speaker:look at it if they want to. And so that was kind of the new
Speaker:exciting world. I think today, The new exciting
Speaker:world goes well beyond that, which is going to be like,
Speaker:you you can create worlds. Any Any world that you wanna
Speaker:build, any scenario that you can imagine, you
Speaker:can just have a machine fill in all the gaps for you and, You
Speaker:know, write the write the story, make the
Speaker:image and maybe, like, you know, make the make the video, make the whole
Speaker:world. So I think, I I I think the
Speaker:idea with generative AI that I want people thinking about more that that I
Speaker:I also wanna think about more is, like, Okay. If you could create any world
Speaker:you want, to explore, to live in, just
Speaker:to, you know, maybe it's something to to teach us about something. Maybe it maybe
Speaker:it's just an artistic adventure
Speaker:venture, you know, what kind of world do you want to create because that's
Speaker:that's going to become very cheap very quickly. Yeah. I could
Speaker:see that. So I'm gonna skip to,
Speaker:share something different about yourself. But we remind our guests
Speaker:to remember it's a family podcast.
Speaker:Okay. And I'm, you know, I'm I'm trying to,
Speaker:I'm I'm trying to think of an answer here, and it's not because,
Speaker:it's it's not because of the, of the of the caveat there,
Speaker:but it No. No. I get it. Well, you've already covered a lot
Speaker:that's It's different. I did. It is good stuff. Yeah. I mean, I
Speaker:think, I think one thing that, It's,
Speaker:I I enjoy doing that that that I forgot to mention, because
Speaker:I'm actually doing it again for the 1st time in in in 6 years was,
Speaker:I was a member of the Yale Alumni Service Corps. It's not
Speaker:a member. It's like you can do a a it was essentially we were doing
Speaker:trips to underserved Communities around the world and,
Speaker:you know, doing little, like, kinda, kind of service
Speaker:trips where you'd ever Either build a structure or work with small business
Speaker:owners or go, you know, teach in a school. And so I've been
Speaker:to, Nicaragua and Ghana And I actually
Speaker:got to lead one of their trips in 2017 and that was to the Fort
Speaker:Mohave Indian reservation. Very different kind of a trip because
Speaker:it was Within the United States here. And
Speaker:so it was honestly a lot easier. Because that's very cool
Speaker:flying to Vegas. But yeah. But we're actually going back there, in in a few
Speaker:months after 6 years. And so I was so even though it was less
Speaker:convenient this time around, I'm I'm very excited Do that. And so
Speaker:I I don't know. I really like learning about different cultures,
Speaker:different philosophies, different religions. I think A lot of people might
Speaker:assume given the you know given the tenor of my
Speaker:podcast that I'm very like you know rationalist and I talk about Bayesian
Speaker:inference a lot. But, I I
Speaker:sort of venture out of that a lot. I don't think that,
Speaker:That raw math can, can explain everything in life. And I also love like the
Speaker:diversity of, of cultures and stuff. So No. It's cool.
Speaker:That so that's maybe a positive thing, so I I don't know. It's very positive.
Speaker:Something different. No. It definitely is. It definitely is. So where can
Speaker:folks find out more about you and what Sure. So you mentioned you have a
Speaker:podcast, which I love the name, The Local Maximum.
Speaker:Right. Yeah. Local maximum's triple entendre, because it's got my name, Max.
Speaker:It's a local maximum is, of course, you know, in machine learning,
Speaker:when you're when you're trying to find the, well, sometimes it's often the local minimum
Speaker:if you're trying to, Minimize the the loss function, but the
Speaker:in basing inference, if you're trying to maximize the probability, whatever, you're you you get
Speaker:stuck stuck in one of your next which was Your name is Max. So Yeah.
Speaker:Exactly. Right. Right. That's the first one. That's the second one. And then, you know,
Speaker:I I worked on location data a lot, so it's kind of a a triple
Speaker:meaning. And so, I've
Speaker:been doing that for for quite a while. You can go back into kind of
Speaker:a a really extensive library there. And, I have
Speaker:the website local max radio.com. I have.
Speaker:If you go, I have local maximum labs. If you go to local maximum local
Speaker:max radio.com/labs, I have a
Speaker:bunch of papers And, you know, kind
Speaker:of works that I've done, which, you know, includes some
Speaker:discussion of machine learning, like kind of the math mathematics behind bias correction,
Speaker:but But also something kind of fun that I did, like, with the podcast last
Speaker:year, which is, like, I just rewrote the US constitution, fixed a bunch of
Speaker:things just because I I felt that was fun. I was Taken aback by how
Speaker:mad people get when you when you do that, it's not like I was actually
Speaker:trying to, you know, run a political campaign for it. I just thought it was
Speaker:a a fun project, and I learned a lot. But Some once you venture into
Speaker:the political, people start treating things different. People get angry pretty quick.
Speaker:Yes. That's true. Yeah. Yeah. I I I The
Speaker:I I I love to hear criticisms on it. I wanna hear what what what
Speaker:people think. But The one criticism
Speaker:that I get a lot, which I really hate is, like, how dare you spend
Speaker:your free time on this, which I I just don't get at all.
Speaker:Yeah. But, which, you know, whenever I put
Speaker:out some kind of math paper, even if it's like and there there is one
Speaker:called relative probability, which is, you know, sort
Speaker:of an abstract paper where it's like, okay, a reimagined probability
Speaker:theory as, okay, let's say you can't talk about The probability of something
Speaker:happening. Let's say you can only talk about 1 probability relative
Speaker:to another. What, what does that look like? And I just stated some basic facts
Speaker:and, you know, Not that many people gonna use it. Maybe people
Speaker:won't use it for for a while. I I feel like it's an interesting idea,
Speaker:and I feel like it will have uses eventually. But,
Speaker:You know, nobody criticized me for that. For like, how dare you spend your free
Speaker:time on that? Exactly. They they pick on you for the other
Speaker:stuff. Yeah. I mean, I I look at people. I mean, you spend your free
Speaker:time yelling at people on Twitter. I mean, what's the difference? I was gonna say
Speaker:you can you can look at TikTok, and you can find far more destructive uses
Speaker:is of Exactly. Exactly. So that's so that's that's my
Speaker:main thing. I I think maybe with the, with the Constitution, I think people have
Speaker:their sort of ideal society in mind. And if If your thing doesn't wind up
Speaker:with that, they they kind of perceive you as a threat. Like you're trying to,
Speaker:like I was trying to revitalize democracy, but some people are saying, no, you're
Speaker:backsliding on democracy. Alright. Like, let's talk about it. But, yeah, it's it's
Speaker:people get you know, people get different. We need to have you back and talk
Speaker:about that more. Yeah. For sure. For sure. Talk about that. Absolutely.
Speaker:We'd love having you. Both Andy and I, however, do have a hard stop, and
Speaker:I would love this This covers you to go out for a couple hours, and
Speaker:we'll talk to you more. And I just had a a conversation last night with
Speaker:my cohost that went a couple hours. I know how it goes. Yeah. Yeah. Yeah.
Speaker:We ended at 1 AM, and I was like, oh my god.
Speaker:Well, those 1 AM conversations. I know what you mean. You got it. So
Speaker:With that, we'll definitely make sure. Send us all your links, and
Speaker:we'll make sure we get them in the show notes, and we'll let Bailey, our
Speaker:semi extension AI host, Co host, 3rd
Speaker:host, wrap up the show. And thank you, dear
Speaker:listener, for subscribing to our podcast. You
Speaker:have subscribed to us, haven't you? Once you do,
Speaker:please be sure to rate and review our podcast on iTunes, Stitcher,
Speaker:or wherever you subscribe to us. Having high ratings
Speaker:and reviews helps us improve the quality of our show and rank us more
Speaker:favorably with the search algorithms. That means more
Speaker:people listen to us, spreading the joy. And,
Speaker:can't the world use a little more joy these days?
Speaker:So, go do your part to make the world just a little better and be
Speaker:sure to rate and review the show.
About the author, Frank
Frank La Vigne is a software engineer and UX geek who saw the light about Data Science at an internal Microsoft Data Science Summit in 2016. Now, he wants to share his passion for the Data Arts with the world.
He blogs regularly at FranksWorld.com and has a YouTube channel called Frank's World TV. (www.FranksWorld.TV). Frank has extensive experience in web and application development. He is also an expert in mobile and tablet engineering. You can find him on Twitter at @tableteer.