1
00:00:00,250 --> 00:00:02,230
Welcome back to Adventures in DevOps.

2
00:00:02,230 --> 00:00:06,350
Today we're going to review the complexities of building models at enterprise
organizations.

3
00:00:06,350 --> 00:00:10,536
And hopefully we'll find enterprise AI success stories.

4
00:00:10,576 --> 00:00:18,603
Our guest has spent his career large scale systems across machine learning, data
infrastructure, compilers, and cloud native architectures.

5
00:00:18,603 --> 00:00:22,828
His work spans graph processing, multimodal inference, and rag architectures.

6
00:00:22,828 --> 00:00:26,243
the co-founder and CTO at Corvik, Donald Nguyen.

7
00:00:26,243 --> 00:00:27,614
Welcome to the show.

8
00:00:27,865 --> 00:00:28,957
Thanks for having me.

9
00:00:29,208 --> 00:00:32,949
So I have to ask, how large scale is large?

10
00:00:33,200 --> 00:00:37,880
It's kind of, I mean, large scale can mean a lot of things to a lot of people.

11
00:00:37,880 --> 00:00:47,800
I think one part that's important to get at is in addition to like scale, which we can
talk about gigabytes, petabytes and stuff like that, it's about complexity of the data,

12
00:00:47,800 --> 00:00:48,760
right?

13
00:00:49,140 --> 00:00:54,878
And In terms of the volume you're talking about, it's not a lot, but oh my God, the
computer can read my PDFs.

14
00:00:55,038 --> 00:00:56,899
it can see these pictures, right?

15
00:00:56,899 --> 00:01:02,470
This data that we actually always had that was just sitting there, just dead, sitting in
S3 or something like that.

16
00:01:02,470 --> 00:01:04,260
Now it means something.

17
00:01:04,741 --> 00:01:12,593
And uh once you do that, then you start asking, well, if we can see these pictures and
what if it knows a little bit about some other data and then you start connecting it.

18
00:01:12,593 --> 00:01:17,844
And those links and those combinations, they create like a combinatorial explosion of
complexity.

19
00:01:17,844 --> 00:01:24,868
And a lot of, I think the difficulty people sort of see about managing scale is not about
the volume.

20
00:01:24,868 --> 00:01:26,900
but how these interactions play out.

21
00:01:27,151 --> 00:01:31,297
So for them, they're storing all the data in S3 or somewhere else.

22
00:01:31,297 --> 00:01:35,634
how much is that really that you commonly see in practice?

23
00:01:35,634 --> 00:01:37,667
we talking about, is it gigabytes?

24
00:01:37,667 --> 00:01:40,470
Is it terabytes, petabytes, et cetera?

25
00:01:40,471 --> 00:01:50,455
I think for a lot of the cases, uh we're talking about like gigabytes, but it's really
about data that's been untapped and it's these PDFs and about the pipelines you have to

26
00:01:50,455 --> 00:01:53,086
build in order to build meaning on top of it.

27
00:01:53,086 --> 00:01:53,463
Right?

28
00:01:53,463 --> 00:01:55,067
And so one thing is you store the data.

29
00:01:55,067 --> 00:01:55,547
Good.

30
00:01:55,547 --> 00:01:59,749
Second thing is you have something that can just like do a keyword search over that.

31
00:01:59,749 --> 00:02:00,529
Great.

32
00:02:00,529 --> 00:02:06,722
Third thing is I want to build like a specific way of interpreting this data because I'm
an auditor or something like that.

33
00:02:06,722 --> 00:02:09,923
And I want to find whether or not this document is in compliance with

34
00:02:09,923 --> 00:02:11,657
like my parameters.

35
00:02:11,657 --> 00:02:14,793
That's a very complicated query to express in terms of keyword search.

36
00:02:14,793 --> 00:02:18,712
So you have to start building these additional indexes on top of it.

37
00:02:18,712 --> 00:02:21,646
like I said, it's the complexity more than anything else.

38
00:02:21,646 --> 00:02:26,308
It sounds like really a focus on the knowledge basis of LLM system.

39
00:02:26,308 --> 00:02:35,673
you have, it's not necessarily like first class business analytics or data regarding the
end product, but like when you say PDFs, the first thing that comes into my mind are

40
00:02:35,673 --> 00:02:46,283
invoices and manuals and maybe some internal compliance documentation or specifications
that you have written out and you want to answer some critical questions that help you.

41
00:02:46,283 --> 00:02:49,573
basically get to the next step of whatever the business is trying to achieve.

42
00:02:49,574 --> 00:02:51,694
Yeah, I mean, think that's a good start point.

43
00:02:51,694 --> 00:02:58,734
think of what differentiates one organization from another in a digital world.

44
00:02:58,734 --> 00:03:03,314
It's basically the data they have and the kind of prior they have about their data.

45
00:03:04,054 --> 00:03:05,154
I'm a supermarket.

46
00:03:05,154 --> 00:03:07,234
I got a bunch of things about my customers.

47
00:03:07,234 --> 00:03:09,594
I'm like an investment bank.

48
00:03:09,594 --> 00:03:11,694
I have different things about my clients.

49
00:03:11,751 --> 00:03:17,184
ah In terms of the mechanical interactions that they're doing, the transactional things
that they're doing, it's very similar.

50
00:03:17,184 --> 00:03:27,223
You intake a customer, you intake a client, you collect some data about it, but the actual
data makes the difference between JP Morgan and Safeway.

51
00:03:27,223 --> 00:03:37,468
So one thing I saw in a lot of companies that I've worked with in one way or another is
that they were storing just masses of data that was completely and utterly useless.

52
00:03:37,469 --> 00:03:44,073
And so one of the biggest challenges was always I'll say clean like I think the canonical
term is cleaning cleaning the data.

53
00:03:44,073 --> 00:03:48,775
But I feel like it's just so much more than that, where it's just like even figuring out.

54
00:03:48,835 --> 00:03:56,079
I mean, there's like first order effects like you can be like, well, that's just analytic
or log data on some transactions that happened decades ago.

55
00:03:56,091 --> 00:04:01,878
probably useless uh as compared to some stuff that's relevant for right now.

56
00:04:01,899 --> 00:04:11,273
Do you see that there's like a majority of the data falls into the category of complete
garbage that needs to get removed before you can start utilizing the data effectively?

57
00:04:11,273 --> 00:04:13,395
Or is that just a very small part of it?

58
00:04:13,499 --> 00:04:19,264
So one thing that's interesting, I'll talk the letter, is that in at an enterprise
setting, you're talking about all these different stakeholders, right?

59
00:04:19,264 --> 00:04:23,668
And so the people collecting the data are very different than the people who actually
using the data.

60
00:04:36,038 --> 00:04:25,093
But they're separated.

61
00:04:25,093 --> 00:04:27,183
And they speak completely different languages.

62
00:04:27,183 --> 00:04:32,305
On the data collection side, you're talking about ETL and AirBite and Kafka.

63
00:04:32,305 --> 00:04:36,427
And then on the data analysis side is like Excel, right?

64
00:04:36,427 --> 00:04:38,788
A spreadsheet or something like this, right?

65
00:04:39,228 --> 00:04:47,151
And when you create that separation responsibility, obviously if I'm just charged of
collecting data, I'm very precious about collecting every, every, because

66
00:04:47,251 --> 00:04:49,051
someone can ask me, where is that data?

67
00:04:49,051 --> 00:04:51,271
And I'll feel bad if I don't have it.

68
00:04:51,611 --> 00:04:55,431
But it creates a weird over-indexing in the wrong ways.

69
00:04:55,431 --> 00:04:58,631
Because at the end of the day, the whole purpose of this is to drive business outcomes.

70
00:04:58,631 --> 00:05:05,331
And so if you're just collecting stuff because you want to collect stuff because you're
not sure what you need, you create a weird ecosystem.

71
00:05:05,751 --> 00:05:12,449
So I think a lot of that comes from that, is just because we can, should And then
secondly, yeah, a lot of data is garbage.

72
00:06:22,963 --> 00:05:21,907
I guess another specific case is I used to work in a graph database company and we were
doing these complicated uh graph neural network embeddings.

73
00:05:21,907 --> 00:05:28,291
So if people are familiar with language embeddings that we use for retrieval augmented
generation, imagine you could

74
00:05:28,507 --> 00:05:32,427
build embeddings not over words and sentences, but over relationships of things.

75
00:05:32,427 --> 00:05:38,820
So it would be nice to identify, like you're in a social network and you're like a hub, a
very important person, that social network.

76
00:05:38,820 --> 00:05:44,592
You need to know who you're related to, and then you can imagine embedding that of
represents that latently.

77
00:05:45,172 --> 00:05:56,055
And it's a very sophisticated technology, and then we were building out these pipelines
for various sophisticated use cases around fraud detection.

78
00:05:56,167 --> 00:06:04,375
ah And you know, maybe it's better in like 1 % of the case, but it's so complicated to
use, so difficult to use, and you have to know exactly what this embedding actually means

79
00:06:04,375 --> 00:06:06,167
before you build on top of it.

80
00:06:06,167 --> 00:06:15,746
embeddings was one of those words where I had, it took me a lot of years before I was able
to map it to, oh I'm just changing the representation of data in my database from, you

81
00:06:15,746 --> 00:06:19,660
know, saving it as a string to saving it as a list of numbers.

82
00:06:21,422 --> 00:06:27,966
I don't understand why it was picked, but that one definitely held me back from true
understanding of what was happening.

83
00:06:27,966 --> 00:06:32,388
uh But once I got that, I'm like, yeah, OK, well then, I totally understand.

84
00:06:32,388 --> 00:06:39,291
You store the data in a different format, like you're doing the CQRS pattern, the uh
command query response.

85
00:06:39,291 --> 00:06:46,755
And why store it as a string when storing it as something else allows you to search and
filter and get the results back faster?

86
00:06:46,755 --> 00:06:49,967
It's interesting you bring up relationships in graph world.

87
00:06:49,967 --> 00:06:52,228
did that actually look like in practice?

88
00:06:52,449 --> 00:06:59,334
Was there something special that had to be done in order to figure out how to actually
store those relationships in a vector database?

89
00:06:59,334 --> 00:07:09,101
Or are there common strategies already, and it was pretty much just converting the format
of the data that you had to a different format so that it could be used uh in a more

90
00:07:09,101 --> 00:07:10,597
optimized way at runtime?

91
00:07:10,597 --> 00:07:19,441
the way I think of an embedding, it's like there's the universe that we're in, and then
there's like a different universe that is like, uh that has a different type of meaning.

92
00:07:19,441 --> 00:07:23,623
And embedding is just sort of, in this current universe, right?

93
00:07:23,623 --> 00:07:29,704
The word cat and the word dog sound different, but in the other universe, they're very
close together, they sound the same,

94
00:07:29,704 --> 00:07:30,191
Mm-hmm.

95
00:07:30,191 --> 00:07:40,916
What is curious about relationships and just things above the surface kind of semantic
embedding type of thing is that it's very use case dependent, I look at a PDF.

96
00:07:40,916 --> 00:07:45,347
If I am like a business person, want to see an invoice, I have to these certain fields.

97
00:07:45,347 --> 00:07:48,478
If I'm an artist, I think of this invoice very differently.

98
00:07:48,478 --> 00:07:54,100
I think of like the meaning of commercialization in culture, something like this.

99
00:07:54,200 --> 00:07:58,351
And that's type of stuff that is not inherent in the data.

100
00:07:58,351 --> 00:08:05,103
It's something that comes from the viewer of the data and they have to impose a bias to
say, look at this, but consider it like this.

101
00:08:06,444 --> 00:08:08,604
And a lot of the challenge about

102
00:08:08,967 --> 00:08:11,800
building on top of relationships is trying to answer that question.

103
00:08:11,800 --> 00:08:14,823
Say, if I'm looking at this, what connections do I want to draw?

104
00:08:14,823 --> 00:08:18,026
Do I want to consider this invoice very similar to this other invoice?

105
00:08:18,026 --> 00:08:21,027
Or do I want to consider it very differently because they use a different name?

106
00:08:21,027 --> 00:08:22,955
It really depends on what you're trying to do.

107
00:08:23,284 --> 00:08:24,445
incredibly novel, honestly.

108
00:08:24,445 --> 00:08:28,009
I don't think anyone's ever brought that up, at least to me before, in a way.

109
00:08:28,009 --> 00:08:38,394
I used to treat the embedding models as pretty much just on a spectrum of how accurate are
they at driving the, like a...

110
00:08:38,394 --> 00:08:48,498
semantic meaning in the embedding result from the original words, but if you say, actually
there's some business context or hidden meaning in what's actually captured, and that

111
00:08:48,498 --> 00:08:58,833
needs to be carried through into what you're persisting in the database, because without
that you're losing a critical aspect, that tells me that you actually can have different

112
00:08:58,833 --> 00:09:07,043
models that are optimized for different use cases, specifically for embedding and not just
for semantic validation or be able to query back later.

113
00:09:07,043 --> 00:09:14,863
scary because now that means when you're selecting a model to do like to populate your
database to actually calculate the embeddings for all your documents, you actually have to

114
00:09:14,863 --> 00:09:24,123
be concerned about not just how accurate the model is, but how it was designed to actually
vectorize the original data.

115
00:09:24,123 --> 00:09:26,223
What intent was there?

116
00:09:27,623 --> 00:09:30,444
you actually evaluate that in practice?

117
00:09:33,074 --> 00:09:35,242
here, this is where we'll use the phrasing of building a model and embedding.

118
00:09:35,242 --> 00:09:40,923
But I think you'll have to understand it as technically the way you solve this problem is
not literally by training a model.

119
00:09:41,264 --> 00:09:50,583
So what we are trying to do is all the same thing, is to look at something and try to
represent it in a different space so that other things that we consider similar

120
00:09:50,583 --> 00:09:51,803
are close in that other space.

121
00:09:51,803 --> 00:10:04,884
um The way embedding models work is basically do this using matrix multiply, and that we
look at a pre-existing corpus to figure out what the values that we're using this matrix

122
00:10:04,884 --> 00:10:06,234
multiply are going to be.

123
00:10:06,894 --> 00:10:18,769
But as we sort of uh up-level, Iger, or make it more use case specific, um you could train
a literal model based off your specific use case in order to understand this kind of

124
00:10:18,769 --> 00:10:19,519
thing.

125
00:10:19,799 --> 00:10:22,090
But it turns out, one, that's very expensive.

126
00:10:22,090 --> 00:10:23,371
Two, you might not have enough data.

127
00:10:23,371 --> 00:10:29,693
And three, you want to elicit information that the human knows, the prior about their
space or the problem they're trying to solve or whatever.

128
00:10:29,693 --> 00:10:32,407
And they'll just say, like, ignore this, right, or whatever.

129
00:10:32,768 --> 00:10:34,150
I mean, you're just leading yourself into

130
00:10:34,429 --> 00:10:35,189
the next question.

131
00:10:35,189 --> 00:10:38,811
How do you actually manage those pipelines at scale?

132
00:10:38,811 --> 00:10:47,816
Because I think one of the challenges that has historically been there is that, especially
if you're in a space where your customers are providing you basically free-form data

133
00:10:47,816 --> 00:10:48,357
input.

134
00:10:48,357 --> 00:10:54,920
So maybe it's just CSVs or Excel files or PDFs, hopefully not, but even JSON that
they've...

135
00:10:55,060 --> 00:10:57,783
configured themselves to be an input to your system.

136
00:10:57,783 --> 00:11:02,488
the whole goal of your product is to say, well, there isn't a standard of how to do this.

137
00:11:02,488 --> 00:11:08,575
Or maybe there is, but they've decided not to follow that standard because that company
makes too much money and they have their own use case.

138
00:11:08,575 --> 00:11:12,218
Or they're in health care or finance, and so who cares about standards there?

139
00:11:12,399 --> 00:11:13,220
And.

140
00:11:13,473 --> 00:11:17,275
realistically, then you have this job of actually parsing that data correctly.

141
00:11:17,275 --> 00:11:22,218
And for the longest time, the challenge was, well, we have to stand up a pipeline
specifically for that.

142
00:11:22,218 --> 00:11:30,992
Not only do we have to get the data out, but we sort of have to validate it because even
if a customer tells us how or user tells us exactly what the spec or schema is of that

143
00:11:30,992 --> 00:11:35,584
thing, they will send us incorrectly specced data afterwards.

144
00:11:35,584 --> 00:11:42,968
And I think this was a huge challenge because it pretty much meant, historically, every
single customer you had, you would have a new pipeline set up.

145
00:11:43,008 --> 00:11:54,235
And I think maybe one of the hopes was that utilizing LLMs would allow you to dynamically
create pipelines in a way that would avoid the need to maintain them.

146
00:11:54,342 --> 00:12:03,197
And I still think at the end of the day, though, you still have a huge explosion of these
things that you still have to be fully aware of and control.

147
00:12:03,197 --> 00:12:07,330
And my question is, how in practice are you actually managing those?

148
00:12:07,356 --> 00:12:12,798
one, we want to create a situation where the system is very kind of transparent to what
it's doing,

149
00:12:28,488 --> 00:12:14,564
there's no one standard about this.

150
00:12:14,564 --> 00:12:19,544
Like I said, either you might follow a process, or you just want the thing being
transparent, or whatever it produces is fine.

151
00:12:19,544 --> 00:12:21,335
You have to be very flexible about that.

152
00:12:21,335 --> 00:12:30,211
So I think one of the things that we're getting into here is that it is incredibly
subjective of whether or not the output is even good.

153
00:12:30,765 --> 00:12:35,865
I don't know that's really the appropriate word there, but especially when it comes to the
side where it's...

154
00:12:35,865 --> 00:12:44,449
Decidable by the business whether or not it's like what to do with that outcome what to do
with the data that's being saved What to do with whatever you've piped in whatever you to

155
00:12:44,449 --> 00:12:46,598
do with what you've cleaned or when you do a search

156
00:12:46,598 --> 00:12:49,179
used to have humans doing customer support.

157
00:12:49,459 --> 00:12:53,871
Now we inserted some kind of automation process so that first line support is done by
someone else.

158
00:12:53,871 --> 00:12:56,481
So then I shaved some head count, basically.

159
00:12:56,481 --> 00:13:00,683
And then with respect to whether or not that was a good decision or not, well, you clearly
saved money.

160
00:13:00,683 --> 00:13:04,554
And then you do have some metric out there about dissatisfied customers or something like
that.

161
00:13:04,554 --> 00:13:08,048
And as long as that doesn't increase dramatically, you're right?

162
00:13:08,048 --> 00:13:10,396
um

163
00:13:10,396 --> 00:13:19,423
sort of pray that companies are tracking like how satisfied or dissatisfied and I know
after you file a support ticket and you get an email saying like, helpful was I?

164
00:13:19,423 --> 00:13:25,227
I don't know if I trust that actually many companies are utilizing that specifically to
make decisions.

165
00:14:00,275 --> 00:13:26,917
it's pretty nebulous.

166
00:13:26,917 --> 00:13:32,757
And that's why you hear a lot of people, a lot of companies saying you need to just use
10x your AI token usage.

167
00:13:32,757 --> 00:13:33,337
Why?

168
00:13:33,337 --> 00:13:35,957
We assume that this is increasing productivity.

169
00:13:35,957 --> 00:13:39,597
Therefore, we assume that the bottom line will be improved by this.

170
00:13:39,597 --> 00:13:43,697
But does anyone have a very concrete number that we should be aiming for?

171
00:13:43,948 --> 00:13:51,270
Well, it's interesting you bring that up because like I actually spent the last couple of
months basically interviewing software companies informally on their success stories

172
00:13:51,270 --> 00:14:01,094
regarding LLMs and none of them can kind of come up with really anything other than well,
there are some processes that we couldn't quite automate before and now we can.

173
00:14:01,094 --> 00:14:06,566
ah But I'm really hoping there's something more than that out there, but I just haven't
seen it.

174
00:14:06,566 --> 00:14:07,396
see...

175
00:14:07,466 --> 00:14:16,382
Two things really, it's the very non-technical automation that I think could have been
automated before, but may have required software engineering to make that happen and they

176
00:14:16,382 --> 00:14:18,383
didn't have that expertise or capability.

177
00:14:18,383 --> 00:14:30,187
uh And the other one is engineers complaining that their organizations are requiring them
to do something that doesn't actually have a positive impact for the business.

178
00:14:40,416 --> 00:14:31,427
we're still kind of working things out.

179
00:14:31,427 --> 00:14:38,403
Like, I feel like there's a, people have a, some fraction of people have a feeling like
this should do something, right?

180
00:14:38,403 --> 00:14:42,185
But like the mechanics of it on the ground, I think are very different,

181
00:14:42,185 --> 00:14:44,796
It's like we're still searching about that.

182
00:14:44,796 --> 00:14:48,721
We're still searching for the value of oh cryptocurrencies.

183
00:14:49,323 --> 00:14:51,323
So 2009 maybe.

184
00:14:51,323 --> 00:14:52,912
ah

185
00:14:52,912 --> 00:14:53,111
old.

186
00:14:53,111 --> 00:14:54,045
You're like, that thing.

187
00:14:54,045 --> 00:14:56,840
Oh, wait, that was a decade ago.

188
00:14:56,840 --> 00:15:00,524
don't know if I say I I've bet on it being successful.

189
00:15:00,554 --> 00:15:04,157
But I definitely have been optimistic about it finding a use case.

190
00:15:04,157 --> 00:15:10,501
And uh now it almost has one with the micro payments for processing for uh agent
interactions.

191
00:15:10,501 --> 00:15:13,233
Cryptocurrencies are now AI together.

192
00:15:13,233 --> 00:15:16,576
That's been uh the unique opportunity.

193
00:15:16,576 --> 00:15:20,854
So it could be a while, I think, before we find new...

194
00:15:20,854 --> 00:15:32,298
cornerstone use cases that really solve stuff outside of say workflow automation, uh which
I think historically is always a pain, especially in organizations that don't have the

195
00:15:32,298 --> 00:15:33,179
technical expertise.

196
00:15:33,179 --> 00:15:40,448
And so if anything, it's lowered the barrier to improve those non-technical processes that
organizations have in place.

197
00:15:40,449 --> 00:15:43,241
I think a bet that well, that I'm making,

198
00:15:43,258 --> 00:15:50,318
is that a lot of these things where you're just sort of a middleman and you depended on
sort of encoding that business process in a non-scalable way, like that was a complete

199
00:15:50,318 --> 00:15:55,138
product, like just that process, To do sales automation or whatever, and then another one.

200
00:15:55,138 --> 00:15:57,818
Those, I think, are going to slowly erode away.

201
00:15:57,818 --> 00:16:02,029
are you talking about like the integrations between two systems that don't directly
communicate with each other?

202
00:16:02,029 --> 00:16:05,579
Because both of them have very opinionated world views and

203
00:16:05,579 --> 00:16:07,900
but neither of them are willing to directly integrate.

204
00:16:07,900 --> 00:16:10,191
Like there's no public standard for how to do it.

205
00:16:10,191 --> 00:16:13,313
you're just, someone has to write the adapter.

206
00:16:13,313 --> 00:16:17,455
And historically that meant you would have hundreds of thousands of adapters out there.

207
00:16:17,455 --> 00:16:20,326
And then these companies uh spin up all the time.

208
00:16:20,326 --> 00:16:21,206
There's a whole bunch of them.

209
00:16:21,206 --> 00:16:30,271
I think we've had some of them on the podcast, actually in the past, that talk about like
their whole strategy was to dynamically generate adapters for every possible integration

210
00:16:30,271 --> 00:16:31,201
you could ever have.

211
00:16:31,201 --> 00:16:33,452
There's SDKs out there that supposedly do this.

212
00:16:33,452 --> 00:16:35,243
And so the products as well.

213
00:16:35,523 --> 00:16:46,976
And I do think you're right though that those sort of things go away when a lot of one of
the two systems will fully take over that responsibility of that integration.

214
00:16:46,976 --> 00:16:50,918
What's the canonical implementation of this that you are using internally or

215
00:16:50,918 --> 00:16:52,900
firecracker VMs, just like spawn stuff, right?

216
00:16:52,900 --> 00:16:55,815
So like that traditional kind of sandbox serverless model.

217
00:16:55,815 --> 00:17:05,100
Firecracker, that's like what AWS is utilizing to safely deploy instances of Lambda
functions and execute them at a ridiculous scale.

218
00:22:24,178 --> 00:17:09,289
So again, we can't really fit to a specific technology unless it's very broadly
applicable.

219
00:17:09,289 --> 00:17:10,828
So S3, great.

220
00:17:10,828 --> 00:17:21,214
But supporting MySQL versus Postgres versus other things and having that be your canonical
data storage, it's not so great because you have the multi-tenant in and then you're very

221
00:17:21,214 --> 00:17:24,906
specific to the cloud or very specific to the technology.

222
00:17:24,906 --> 00:17:26,687
but there's only like a finite number of those things, right?

223
00:17:26,687 --> 00:17:38,707
Like, couldn't I create a connector for MariaDB and MS SQL and Oracle database and
DynamoDB, KeyValueStore and MongoDB and Cassandra and I can go through the list.

224
00:17:38,707 --> 00:17:40,088
Like it's a finite number.

225
00:17:40,088 --> 00:17:41,089
It's probably less than a hundred.

226
00:17:41,089 --> 00:17:42,860
Like that wouldn't be that bad, would it?

227
00:17:44,029 --> 00:17:50,133
It wouldn't be that bad, but then the other requirement that I have is that we take those
and we build data pipelines on them.

228
00:17:50,234 --> 00:17:51,985
So we take that data out, we enrich it.

229
00:17:51,985 --> 00:17:53,076
Where do we put it?

230
00:17:53,076 --> 00:17:54,827
I'm not putting it back in that database, am I?

231
00:17:54,827 --> 00:17:57,429
I really need to put it somewhere that I understand.

232
00:17:57,429 --> 00:18:03,063
And at that point, you're like, why don't I just put it there when I bring in the data
initially?

233
00:18:03,063 --> 00:18:11,026
a lot of organizations, they have their data already stored in such a disparate number of
systems, like the number of companies that I've talked to that have multiple...

234
00:18:11,026 --> 00:18:12,169
uh

235
00:18:12,169 --> 00:18:17,712
like identity providers, even within the same company, is like a nonsensical number.

236
00:18:17,712 --> 00:18:19,843
And they often have more than two.

237
00:18:19,843 --> 00:18:24,666
It's like some business units have Entra ID and other ones have Google Workspace and a
third has Okta.

238
00:18:24,666 --> 00:18:31,790
And there's some giant project going on at this moment where they're trying to actually
introduce another one or collapse them down.

239
00:18:31,790 --> 00:18:38,553
And that's just identity providers where I think everyone is in agreement, like one is
probably the right number to have and not more than one.

240
00:18:38,553 --> 00:18:40,734
I mean, of course, there's edge cases.

241
00:18:40,890 --> 00:18:50,268
Then getting to the point where the number of scored data repositories that you have
trying to reduce that down I feel like it's just a I Don't know if that can ever happen in

242
00:18:50,268 --> 00:18:59,957
practice But the amount of cardinality between the different number of data stores that
you have within a company organization is Vastly smaller than the cardinality of different

243
00:18:59,957 --> 00:19:03,128
data types like the actual data that's being saved in those

244
00:19:03,128 --> 00:19:11,396
I could see that it wouldn't necessarily be a problem to pipe data from individual, like
actually build a connector for every single type of data store you could ever have.

245
00:19:14,759 --> 00:19:21,810
once I build one for RDS and one for uh CosmoDB and whatever is in GCP Cloud SQL, uh is
the best place to actually stick this data?

246
00:19:21,810 --> 00:19:23,246
And please don't say snowflake.

247
00:19:23,246 --> 00:19:27,666
think if you're in an existing organization, you're talking about your internal data
platform.

248
00:19:32,326 --> 00:19:29,558
for very justified reasons.

249
00:20:44,001 --> 00:19:29,564
of stuff?

250
00:19:29,564 --> 00:19:32,274
And then you have to make a very important decision, right?

251
00:19:32,315 --> 00:19:39,488
And, but if you sort of optimize for kind of batchish kind of things, like, I would
suggest just making, like, a data lake or running in Databricks or something like this,

252
00:19:39,488 --> 00:19:42,819
because that's, a good kind of open place to put stuff.

253
00:19:42,819 --> 00:19:47,004
You can dump things in, process them, and then you can sort of expand modularly.

254
00:19:47,004 --> 00:19:51,137
have the opportunity to dog food your own platform in a way because you have your own data
sources.

255
00:19:51,137 --> 00:19:53,448
What are you doing with the target data?

256
00:19:53,448 --> 00:19:55,089
Like are you sticking it in Databricks?

257
00:19:55,089 --> 00:19:56,690
Are you using Dynatrace?

258
00:19:56,690 --> 00:19:59,719
you funneling it into a hole back into S3 somewhere?

259
00:19:59,719 --> 00:20:01,630
What happens to your business data?

260
00:20:01,630 --> 00:20:02,610
Where does it go?

261
00:22:12,972 --> 00:20:08,463
we're using object storage and then bunch of Parquet files and building workflows on top
of those things.

262
00:20:09,092 --> 00:20:15,336
Okay, you're doing the difficult work of assembling long-form table reads from Parquet
files and S3 and whatnot.

263
00:20:15,336 --> 00:20:15,880
Yeah.

264
00:20:15,880 --> 00:20:16,585
yeah,

265
00:20:16,585 --> 00:20:22,337
So what would you say is like some fundamental technology challenge you're actually facing
still today?

266
00:20:22,337 --> 00:20:35,540
as you sort open up platform, even to a vibe-coded code, basically, a lot of things that
you would have handled by human interaction has to be made more.

267
00:21:39,417 --> 00:20:39,097
Right now, we're kind of just, since we control the sandbox, it's just like there.

268
00:20:39,097 --> 00:20:41,089
So like, you're in the sandbox, you can see the logs.

269
00:20:41,089 --> 00:20:42,624
It's like of this weird sandbox,

270
00:20:42,624 --> 00:20:47,840
I mean, the first thing that comes to mind would be a concern where there is something
malicious that would show up in the logs.

271
00:20:47,840 --> 00:20:57,751
And so while it could be blocked fundamentally at the inner the HTTP interface or
whatever, GCP, whatever the data input uh validation could be there, but.

272
00:20:57,751 --> 00:21:04,233
then it still gets in the logs m or maybe it's passed in through a mechanism which is
considered secure, say like a common field or whatever.

273
00:21:04,233 --> 00:21:11,265
And then you end up with a log for J because even though it gets to your monitoring
platform and gets re-ingested, now it's like first class data that you've actually saved

274
00:21:11,265 --> 00:21:12,495
in a different mechanism.

275
00:21:12,495 --> 00:21:21,118
How have you gone about figuring out how to protect even, I still think it's important to
protect the sandbox, even if you call it that because uh in some way it must still have

276
00:21:21,118 --> 00:21:25,379
access to maybe sensitive data, either being able to write or.

277
00:21:25,420 --> 00:21:27,103
the internet in some regard.

278
00:21:27,103 --> 00:21:29,263
Yeah, and you hit the nail on the head, right?

279
00:21:29,263 --> 00:21:31,528
Because I think a lot of ways we think about...

280
00:21:31,765 --> 00:21:36,645
application building has a trusted computing base, right?

281
00:21:36,645 --> 00:21:45,232
But when you assume that your actual thing that you're running is untrusted, you have to
really think about all the decisions that led up to this, right?

282
00:21:45,492 --> 00:21:54,760
And I think it's about being very clear, thinking about the world the same way AWS or GCP
thinks about the world, which is like, I've got our customers come in just with garbage

283
00:21:54,760 --> 00:21:57,746
code or whatever, and I have to protect the actual,

284
00:21:57,746 --> 00:21:59,697
data center from this kind of stuff, right?

285
00:21:59,697 --> 00:22:08,018
And like you have to be very clear about what is uh part of your infrastructure versus the
application space and really never mix them,

286
00:22:08,018 --> 00:22:12,474
How do you decide what's in for the threat model and what should be left out?

287
00:22:12,474 --> 00:22:17,398
I don't think there's a strong sense of what fine-grained access control means for agents.

288
00:22:17,414 --> 00:22:18,959
That's basically what it is.

289
00:22:19,319 --> 00:22:22,641
We have very coarse-grained stuff at the data level, but beyond that, think

290
00:22:22,641 --> 00:22:31,860
So there are some standards around transactional auth and obviously because we're in the
authorizations base ourselves, we have a more nuanced perspective here, but it definitely

291
00:22:31,860 --> 00:22:42,137
is way more complicated than just row level security and it goes much further than that,
whereas you really need to think about when agent performs an action, what are the exact

292
00:22:42,137 --> 00:22:48,870
pieces of data it should have access to, both from a reading standpoint and a writing
standpoint and at that level of granularity, I just...

293
00:22:49,050 --> 00:22:50,591
Just that system doesn't exist.

294
00:22:50,591 --> 00:22:51,441
It can't exist.

295
00:22:51,441 --> 00:23:01,799
It's just fun fundamentally limit that you would never be able to have the context that
you can provide to an agent and also the context for what a policy should be that can

296
00:23:01,799 --> 00:23:11,803
apply to its to its interaction that could be generated outside of that agent that has to
do the thing ah Maybe to give an analogy There is this canonical problem and he just

297
00:23:11,803 --> 00:23:17,766
regular old no LLMs involved when like one sir your customer service calls something
internal you

298
00:23:17,766 --> 00:23:20,137
and then you call a third party and whatnot.

299
00:23:20,137 --> 00:23:27,221
There's no way for the customer to know what the third party is going to require until the
request actually gets there.

300
00:23:27,221 --> 00:23:33,403
You can sort of try to predict what that third party is going to need and what sort of
access control you'll want.

301
00:23:33,403 --> 00:23:39,046
And this is the whole, well, uh you offer a service that integrates, say, with Google
Drive.

302
00:23:39,046 --> 00:23:42,229
and your customer comes in and makes a request and then you're like, you know what?

303
00:23:42,229 --> 00:23:47,313
We need access to your Google Drive, be able to read and write all files, delete your
entire drive.

304
00:23:47,313 --> 00:23:54,699
Also, just in case we need your calendar access and be able to create meetings on all of
your calendars and also delete your calendars just in case.

305
00:23:55,039 --> 00:23:57,561
And something about your email too while we're at it, why not?

306
00:23:57,561 --> 00:24:05,748
And just request all of that upfront because you don't know fundamentally later what is
actually going to be necessary to complete the action, especially if it's asynchronous.

307
00:24:05,748 --> 00:24:08,194
And so I think those sorts of things actually make this a...

308
00:24:08,194 --> 00:24:11,595
sort of not solvable problem in some regard.

309
00:24:17,787 --> 00:24:19,443
we're on this topic, ah there's this idea where you can just stand up some agents and
limit the access that that thing has.

310
00:24:19,564 --> 00:24:24,725
And they're like, oh yeah, you create a bunch of identities for your agents or your system
to run in that particular sandbox.

311
00:24:24,725 --> 00:24:27,597
But if you fast forward like five years from now,

312
00:24:27,597 --> 00:24:34,890
that agent is going to be a perfect representation or proxy for all of the things you
already have in your organization of every single, you know, team member.

313
00:24:34,890 --> 00:24:41,468
It's going to have access to all of your email systems, all your databases, et cetera,
because you're going to have decided that, well, it needs to read and write to all those

314
00:24:41,468 --> 00:24:42,239
things.

315
00:24:42,239 --> 00:24:51,907
And so saying that you should give your agents their own identity is sort of like a
short-term myopic mindset, which doesn't really hold up in the end.

316
00:24:51,907 --> 00:24:55,546
So I don't really understand why people are pushing for that so much.

317
00:24:55,546 --> 00:24:56,746
But yeah, you're right.

318
00:24:56,747 --> 00:24:57,687
People get excited.

319
00:24:57,687 --> 00:24:59,647
It's like, can't even have more data, more data, more data.

320
00:24:59,647 --> 00:25:04,253
And then you can create this giant, just super god agent that has everything.

321
00:25:04,253 --> 00:25:10,597
think fundamentally with the transformer architecture, they'll always have prompt
injection attacks will always be a thing.

322
00:25:10,597 --> 00:25:18,583
And it's like, well, what if you take like another model and you wrap the first model and
you sort of validate the inputs and the outputs don't seem like a prompt injection attack.

323
00:25:18,583 --> 00:25:20,534
And then, but then it's turtles all the way down, right?

324
00:25:20,534 --> 00:25:23,786
Like it's like, well, then you need another model to make sure the second model, you know,
wasn't.

325
00:25:23,786 --> 00:25:28,740
And it's like, whoever has more models stepped on top of each other will be successful in
the end.

326
00:25:28,740 --> 00:25:31,918
I have more turtles, therefore I can protect, you know, prevent those attacks.

327
00:25:31,918 --> 00:25:33,922
I thought one more turtle ahead.

328
00:25:34,866 --> 00:25:40,129
Yeah, I think ah from the data perspective, think that the

329
00:25:40,129 --> 00:25:46,082
roadmap is kind of clear, which is like if you know what data sensitive or not and you can
just provide it or not, it's very binary.

330
00:25:46,082 --> 00:25:48,693
think that's a problem that can be engineered and solved.

331
00:25:48,693 --> 00:25:53,419
But you're right that on the control, the action side, it's very hard, right?

332
00:25:53,419 --> 00:25:54,856
Like what actions you allow it to take.

333
00:25:54,856 --> 00:25:57,897
So I think that's why I'm happy I'm just working sort of more on the data side.

334
00:25:57,897 --> 00:26:01,884
You're just talking about just like confidentiality of data, not like the actions you can
take right now.

335
00:26:01,884 --> 00:26:03,280
It's just, can you read this or not?

336
00:26:03,280 --> 00:26:10,083
But you're right that if we want a world where these things are acting on our behalf and
sending emails or

337
00:26:10,083 --> 00:26:13,379
making decisions that have like external side effects.

338
00:26:13,379 --> 00:26:16,951
I don't know how you'd want to, how you can really manage that properly

339
00:26:16,951 --> 00:26:26,496
think it's one of those things where humans have already sort of figured out the solutions
to most problems and we're re-figuring them out when we use LLMs, but the solutions are

340
00:26:26,496 --> 00:26:28,757
exactly the same solutions we already have in place.

341
00:26:28,757 --> 00:26:33,859
So if we just look to how we solve this problem without LLMs, I think we'll find some
answers for us.

342
00:26:33,859 --> 00:26:41,173
Like if we go to the bank, you get like multiple different validation prompts and like,
you sure you want to send this money to this particular address?

343
00:26:41,173 --> 00:26:46,496
Well, that address has error correction in it and we do bank account checks for.

344
00:26:46,496 --> 00:26:52,510
the size the transaction as well as maybe there's some fraud detection with know, LMS
aside.

345
00:26:52,510 --> 00:26:55,962
If we look at governments as well, there's like different levels of clearance.

346
00:26:55,962 --> 00:27:03,948
There's but you see how challenging it is because there's like well, there's like a secret
clearance and then there's like a top secret clearance and then there's another thing

347
00:27:03,948 --> 00:27:11,183
above that which is like for your eyes only which basically means that ah we don't know
how to do clearance correctly at this level.

348
00:27:11,183 --> 00:27:14,835
So we'll just say you know, if you need it someone else will decide that you

349
00:27:14,835 --> 00:27:22,713
can get it but there's no like canonical answer for that problem it has to be figured out
every single time like in that moment

350
00:27:22,964 --> 00:27:24,202
I

351
00:27:24,202 --> 00:27:26,243
problems and insecurity, yes.

352
00:27:26,243 --> 00:27:28,485
And I don't think it's getting better.

353
00:27:28,526 --> 00:27:30,687
Maybe for something quickly different.

354
00:27:30,687 --> 00:27:36,220
uh So I was chatting with a few company leaders in the last couple of weeks and...

355
00:27:36,220 --> 00:27:43,555
They told me that they are already budgeting for 50 % of the engineering salary uh for LLM
usage.

356
00:27:43,555 --> 00:27:54,843
So if you're paying someone 100K today, then tomorrow you should be allocating 150K for 50
of which will be just token usage by their favorite LLM provider.

357
00:27:54,843 --> 00:27:56,394
What do you think?

358
00:27:56,394 --> 00:27:59,344
Accurate, inaccurate, something you're already planning for?

359
00:27:59,344 --> 00:28:08,397
depends what kind of code you're writing really because I think what I've discovered like
there's certain types of problems where your skill level is quite low and so what the LM

360
00:28:08,397 --> 00:28:14,640
is providing you is great you get like uh something that a lot of information out of that
you learn a lot or whatever

361
00:28:14,640 --> 00:28:22,855
Is it half, should we think about like the way we evaluate employees is like how many
tokens did you use, you didn't use enough,

362
00:28:22,855 --> 00:28:24,652
I think for you get what you measure, right?

363
00:28:24,652 --> 00:28:31,011
And uh I think we're already seeing companies falling prey to token maxing, but it's sort
of a different aspect, right?

364
00:28:31,011 --> 00:28:34,246
You have some role or responsibility in your company.

365
00:28:34,246 --> 00:28:38,882
There's something that has to get completed and you have someone who's put on the job of
doing that thing.

366
00:28:38,882 --> 00:28:49,449
And I think as companies evolve and if LLMs really are tool that everyone will use at some
point, then it's a no-brainer to believe that they may use a tool to solve that and that

367
00:28:49,449 --> 00:28:52,334
tool may be an LLM and that tool has a cost.

368
00:28:52,334 --> 00:28:53,515
How, like...

369
00:28:53,881 --> 00:28:55,312
What is the expectation here?

370
00:28:55,312 --> 00:29:00,002
Is it that you have no idea how much these things will end up costing?

371
00:29:00,002 --> 00:29:05,777
Do you have an expectation on what does sort of, what would be reasonable, what would be
unreasonable?

372
00:29:05,777 --> 00:29:12,361
Like if an engineer came to me and said, yeah, this next year I project I'm gonna use
three times my salary on tokens.

373
00:29:12,361 --> 00:29:14,282
I'd be like, that sounds wrong.

374
00:29:14,282 --> 00:29:17,374
ah Which has nothing to with the salary.

375
00:29:17,374 --> 00:29:21,746
It has to do with the fact I don't know if you're generating three times your salary's
worth of value from tokens.

376
00:29:21,746 --> 00:29:23,687
So maybe that's the conversation.

377
00:29:23,820 --> 00:29:32,460
what would it take to take a normal software developer and then bring them a little bit
more aligned towards product or business?

378
00:29:32,640 --> 00:29:35,080
that's the way you would value it, right?

379
00:29:35,080 --> 00:29:40,199
It's like that you give the person this tool so they can expand their responsibility set.

380
00:29:40,199 --> 00:29:42,400
And that's how it would price it.

381
00:29:42,651 --> 00:29:48,437
I love that you're tempted to just think about token usage from an engineering standpoint.

382
00:29:48,437 --> 00:29:53,541
Because I feel like the biggest usage of tokens would be the product manager saying, you
know what?

383
00:29:53,562 --> 00:29:57,826
It's incredibly critical that we prototype this idea and get it in front of customers.

384
00:29:57,826 --> 00:30:04,922
And to do that means I'm just going to send my favorite LLM on an infinite cycle loop
until this thing is done.

385
00:30:04,922 --> 00:30:07,983
And then after spending n...

386
00:30:07,983 --> 00:30:13,731
thousands of whatever your favorite currency is, I'll throw it in front of customers and
get some feedback there.

387
00:30:13,731 --> 00:30:16,474
ah And that horrifies me.

388
00:30:16,475 --> 00:30:17,346
I mean,

389
00:30:18,415 --> 00:30:21,556
I can buy it for like just sort of like show a prototype or something.

390
00:30:21,556 --> 00:30:27,769
I think the biggest value that we had for non-engineering use cases of LLMS is actually
bringing people to speak the same language.

391
00:30:27,769 --> 00:30:36,624
So like our designers can produce just real working websites and designs and like uh
changes and then they're submitted as code.

392
00:30:36,624 --> 00:30:43,317
Again, you can't, it's really garbage code to read, but at least we're all sort of
speaking the same language and we're like understanding what we're actually communicating

393
00:30:43,317 --> 00:30:47,419
because we can actually make a physical artifact about that that behaves in high fidelity.

394
00:30:47,419 --> 00:30:48,319
And I think that part is

395
00:30:48,319 --> 00:30:49,379
very useful.

396
00:30:49,630 --> 00:30:56,450
For a product manager, if I were a manager of a product team, would I be like, you need to
maximize your token usage?

397
00:30:56,790 --> 00:31:02,010
I think that part is a little scary in the sense that you have to be kind of aligned with
reality.

398
00:31:02,010 --> 00:31:07,930
And so if you can't actually produce that thing and solve all the other problems, then
knowing that you can do this is nice.

399
00:31:07,930 --> 00:31:09,630
And seeing it very concretely is very nice.

400
00:31:09,630 --> 00:31:11,890
But in some sense, ideas are cheap, right?

401
00:31:11,890 --> 00:31:14,950
The ability to execute is the most important thing.

402
00:31:15,201 --> 00:31:17,531
The right business outcome, yeah.

403
00:31:17,531 --> 00:31:23,876
I think the companies that are just trying to optimize token usage have the wrong
incentives at play realistically.

404
00:31:23,876 --> 00:31:29,000
But so I think that's a much more mature way of looking at it for sure.

405
00:33:21,972 --> 00:31:39,326
I do think that there's a lot of overlap with how things are migrating to at a large scale
with the concept of say outsourcing one of your critical internal departments to another

406
00:31:39,326 --> 00:31:40,516
company to function.

407
00:31:40,516 --> 00:31:47,949
I don't mean like using a SaaS provider, but maybe like if you're doing software
engineering and you're hiring a third party company to basically make you some prototypes

408
00:31:47,949 --> 00:31:49,550
or some mobile apps.

409
00:31:49,610 --> 00:31:56,238
If you are uncomfortable with that model, I feel like you should also be uncomfortable
utilizing LLM to solve a problem.

410
00:31:56,238 --> 00:32:01,971
I also really like your perspective on the data because I think this is something that
definitely was lost for me early on in my career.

411
00:32:01,971 --> 00:32:05,323
It's that data is sort of like a throwaway term.

412
00:32:05,323 --> 00:32:11,446
doesn't it means almost nothing there or maybe it means a lot to a lot of people and
that's sort of the problem.

413
00:32:11,446 --> 00:32:18,364
And it's really like stop saying the term data is like what is the data actually
representing because you know you brought up this point with the embedding model early on.

414
00:32:18,364 --> 00:32:25,281
It's that there are some cornerstone critical aspects to why the data exists and what the
sort of inherent

415
00:32:25,281 --> 00:32:30,286
information or value is in that particular data and that's where the complexity is.

416
00:32:30,286 --> 00:32:39,373
And you really need to understand that that's where the challenge is and not just the
storage or data pipelining or processing of arbitrary pieces of data or formats.

417
00:32:39,374 --> 00:32:41,994
Do you know the Douglas Adams thing?

418
00:32:41,994 --> 00:32:44,314
know, like the answer to everything is 42?

419
00:32:44,315 --> 00:32:45,275
What was the question?

420
00:32:45,275 --> 00:32:47,235
Like that's the most important, the data, that's 42.

421
00:32:47,235 --> 00:32:48,355
How did you get that?

422
00:32:48,355 --> 00:32:49,755
That's the whole journey.

423
00:32:49,755 --> 00:32:52,655
Yes, the life, the universe and everything.

424
00:32:52,655 --> 00:32:55,050
It's summed up in a very simple number.

425
00:32:55,050 --> 00:32:58,842
That was the, wait, was that the number crunching machine, right?

426
00:32:59,323 --> 00:33:08,451
Keep asking, you what's the most important question in the universe that requires the
entire processing of the whole universe in order to actually come to a conclusion.

427
00:33:08,471 --> 00:33:11,343
So with that, maybe we'll switch over to PICS.

428
00:33:11,343 --> 00:33:13,655
So Donald, what'd you bring for us today?

429
00:33:13,656 --> 00:33:17,489
Well, I have a young daughter, so I often have to read to her at bed.

430
00:33:17,489 --> 00:33:25,294
And the challenge, if anyone knows, about reading stories to children is a lot of the
stories are terrible, and you don't want to read them over and over and over again.

431
00:33:25,775 --> 00:33:32,559
So we locked upon a set of books called The Investigators, which is like a little graphic
novel.

432
00:33:32,559 --> 00:33:42,166
I think it's geared towards kids who can read, but it's enjoyable for younger folks, I
think, mostly because it's just like a very funny pun-filled

433
00:33:42,381 --> 00:33:47,155
absurd story about these alligators who are like investigating different things.

434
00:33:47,155 --> 00:33:55,472
ah But it's one of those things I've very rarely like if people are familiar with the TV
show, Blue Eats, it's the same category, whereas an adult you enjoy watching it.

435
00:33:55,472 --> 00:33:58,097
I go to my daughter, like, can we read this book?

436
00:33:58,097 --> 00:34:00,546
Because I'm curious about what's happening next in the story.

437
00:34:00,546 --> 00:34:01,817
ah

438
00:34:01,817 --> 00:34:09,371
because it's something she enjoys, but it actually has this continuity about the
storylines built on top of each other over multiple series.

439
00:34:09,371 --> 00:34:11,102
Again, very uncommon for children's books.

440
00:34:11,102 --> 00:34:14,394
They're very mechanical if you find a series.

441
00:34:14,394 --> 00:34:18,016
So, Investigators John Patrick Green, believe.

442
00:34:18,077 --> 00:34:20,578
If you have a kid, um read it to them.

443
00:34:21,290 --> 00:34:26,992
Yeah, I'm with you on that because it does seem like most children's books, I mean, even
some adult books, they're very episodic in a way, right?

444
00:34:26,992 --> 00:34:29,622
Like there's some story and that's complete case.

445
00:34:29,622 --> 00:34:37,975
And it does feel like there is no continuity in any regard, which sort of removes any sort
of desire to continue on in some way.

446
00:34:37,975 --> 00:34:44,827
uh Okay, well, I think you may be the first to come on the podcast and recommend a
children's book.

447
00:34:44,827 --> 00:34:48,138
Not like, there have been kids toys, so.

448
00:34:48,602 --> 00:34:49,282
Totally fine.

449
00:34:49,282 --> 00:34:51,559
uh I actually really like the pic.

450
00:34:51,559 --> 00:35:01,196
Which makes me almost want to check it out that I'm always looking for interesting things
to read in German because I want to improve my German language skills and uh I've often

451
00:35:01,196 --> 00:35:10,379
thought you know, what would be great is easy television shows or books and finding good
content that has been well translated is a challenge and I find children's stories

452
00:35:10,379 --> 00:35:12,901
tend to be translated, but most of them are bad.

453
00:35:12,901 --> 00:35:16,515
uh I may actually look into that.

454
00:35:16,515 --> 00:35:17,655
yeah, so what did I bring?

455
00:35:17,655 --> 00:35:18,376
I almost forgot.

456
00:35:18,376 --> 00:35:22,900
ah I brought this episode of a YouTuber named Dr.

457
00:35:22,900 --> 00:35:23,421
Nemo.

458
00:35:23,421 --> 00:35:29,266
ah Specifically the episode is, you've never seen a clockwise circle pit.

459
00:35:29,266 --> 00:35:34,140
That's a pit of people running in a clockwise motion.

460
00:35:34,522 --> 00:35:43,705
Counterclockwise for me actually has always felt more appropriate just anything
counterclockwise from conservation of angular momentum to electromagnetic wave pointing

461
00:35:43,705 --> 00:35:44,205
vectors.

462
00:35:44,205 --> 00:35:46,325
It's always counterclockwise.

463
00:35:46,365 --> 00:35:53,947
But one thing I never challenged is why is it called clockwise and counterclockwise when
clocks are the only thing really that goes backwards?

464
00:35:53,947 --> 00:36:03,452
I feel like we should have picked a different term and there's a lot of stuff in here
that's baked on the uh not just nurture but also uh

465
00:36:03,452 --> 00:36:09,554
nature of how humans work and standard stuff as well as cultural aspects.

466
00:36:09,554 --> 00:36:16,526
I wonder if it's true across all cultures or if it's just a Western world m aspect.

467
00:36:16,526 --> 00:36:20,547
And we may never know because I don't think we have enough segregation.

468
00:36:20,547 --> 00:36:26,806
There's a lot of things, say, in physics and mathematics where counterclockwise is
positive and clockwise is negative.

469
00:36:28,941 --> 00:36:26,942
that's good.

470
00:36:26,942 --> 00:36:32,662
I love those kinds of things where, you know, there's all these, a lot of these random
conventions, right?

471
00:36:32,662 --> 00:36:33,602
Like, how do they come about?

472
00:36:33,602 --> 00:36:37,022
Was it just like a chance one day or, you know, is there an inherentness?

473
00:36:37,022 --> 00:36:42,142
And I guess your point, right, is that if we thought from a mathematical basis, maybe we
would have chosen a different way.

474
00:36:42,142 --> 00:36:43,734
Maybe we just figured this out too late.

475
00:36:43,734 --> 00:36:55,527
Yeah, well, he didn't actually say this in the video, but my theory is that we've decided
in the Cartesian plane that uh the x-coordinate system goes left to right, negative to

476
00:36:55,527 --> 00:36:59,318
positive, and the y goes from uh negative to positive down to up.

477
00:36:59,318 --> 00:37:01,238
So down to up, left to right.

478
00:37:01,238 --> 00:37:08,781
And if you do a mathematical operation where you go from a high positive x-coordinate to a
high positive y-coordinate, you go like this.

479
00:37:08,781 --> 00:37:10,301
And that's the...

480
00:37:10,301 --> 00:37:14,704
that's this curl and that curl then becomes positive because both those numbers are
positive.

481
00:37:14,704 --> 00:37:26,122
But if we changed our mental model where the primary coordinate was the vertical and not
the horizontal line, then we would have reversed them and that would have made that

482
00:37:26,122 --> 00:37:26,632
negative.

483
00:37:26,632 --> 00:37:33,377
Or if we thought numbers differently where negative numbers were higher than positive
numbers.

484
00:37:33,377 --> 00:37:35,509
So there is a huge aspect there.

485
00:37:35,509 --> 00:37:37,320
Do other cultures have different counting systems?

486
00:37:37,320 --> 00:37:39,218
Do they have a different way of thinking about it?

487
00:37:39,218 --> 00:37:46,856
I know that there are some tribes that fundamentally have it built into their culture,
like which way is north, which way is south.

488
00:37:46,856 --> 00:37:52,697
So coordinate system or cardinal directions or waves, like they just sort of know.

489
00:37:53,278 --> 00:37:54,059
I don't.

490
00:37:54,059 --> 00:37:56,721
I have no idea which way north is most of the time.

491
00:37:56,722 --> 00:37:59,659
But I think it reminded me that...

492
00:37:59,794 --> 00:38:02,054
think we take a lot of things for granted, like just numbers, right?

493
00:38:02,054 --> 00:38:07,414
And you can almost see it today in the way we use the words for numbers, right?

494
00:38:07,414 --> 00:38:14,194
And in different languages, like they're very weird in that like you think of one, two,
three, four, five, six, seven, eight, fine.

495
00:38:14,194 --> 00:38:16,834
Okay, 11, 12, 13.

496
00:38:16,914 --> 00:38:21,534
Then you see this pattern like, you know, 13, 14, 15, 16, 17, 18, 19, great.

497
00:38:21,534 --> 00:38:22,974
But it's different, right?

498
00:38:22,974 --> 00:38:24,874
Then 20 and you have this repeat again.

499
00:38:24,874 --> 00:38:27,466
So you can get a sense that when...

500
00:38:27,468 --> 00:38:35,504
English speakers were thinking or whatever they came from were thinking about numbers they
weren't thinking about in terms of that very rigorous kind of base 10 feeling they were

501
00:38:35,504 --> 00:38:44,100
just like I needed a thing for this concept or like one of these things and then for two
of these things and then and at some point like we stopped inventing and we're like oh I

502
00:38:44,100 --> 00:38:53,618
see the pattern now and so we should do this right and you know that from different
languages they have very different counting systems like French the minds at base 20 yeah

503
00:38:53,693 --> 00:38:56,633
things like 96 in French are so absurd.

504
00:38:56,973 --> 00:38:59,893
there's something there's...

505
00:38:59,893 --> 00:39:12,453
Yeah, I'll say that German numbers definitely mess with me the most because you put the
ones digit before the tens when you speak, but hundreds still like, but the larger digits

506
00:39:12,453 --> 00:39:13,573
still come first.

507
00:39:13,653 --> 00:39:17,853
like 11 and 12, they're the same as in English for the most part.

508
00:39:17,853 --> 00:39:20,913
But if you get to say 21, you say,

509
00:39:21,344 --> 00:39:30,101
one in 20 and two in 20 and you know three in 30 and it's like it's so confusing when
people are talking to me and I'm just like wait what number did you say?

510
00:39:30,101 --> 00:39:40,157
maybe that will be my pick for next week's episode so maybe I won't spoil it for the
viewers until then but other than that thank you Donald for coming on for today's episode

511
00:39:40,157 --> 00:39:42,903
and talking through multimodal stuff uh

512
00:39:42,903 --> 00:39:46,089
building AIs, LLMs at scale for customers.

513
00:39:46,089 --> 00:39:47,651
I think it's been great.

514
00:39:47,652 --> 00:39:48,292
Awesome, thank you.

515
00:39:48,292 --> 00:39:49,287
pleasure.

516
00:39:49,287 --> 00:39:56,018
And uh thanks for all the viewers for tuning into this week's episode and I hope to see
you all back again next week.