1
00:00:07,928 --> 00:00:13,122
Welcome back to Adventures in DevOps, where apparently we're on a spree of roasting CI CD
platforms.

2
00:00:13,122 --> 00:00:17,765
Historically, we found guests who shine talking about why all build systems struggle.

3
00:00:17,765 --> 00:00:21,708
Last week, Cassidy Williams was on to unofficially defend GitHub.

4
00:00:21,708 --> 00:00:23,549
It was a great episode, by the way.

5
00:00:23,549 --> 00:00:33,496
And this week I'm bringing in Chris Farris, AWS security hero, cloud security evangelist,
and famously quoted as saying, if there are 15 ways to do CI CD, my company is doing

6
00:00:33,496 --> 00:00:34,677
twenty-one of them.

7
00:00:34,677 --> 00:00:36,108
Welcome to the show.

8
00:00:36,364 --> 00:00:37,065
No, thank you.

9
00:00:37,065 --> 00:00:44,040
And yes, uh apparently they've expanded since I said that in twenty seventeen and now that
I think they're up to three hundred and sixteen.

10
00:00:44,410 --> 00:00:48,625
but they've also become four companies or or changed ownership four times.

11
00:00:48,625 --> 00:00:50,566
So, you know, that's what happened.

12
00:00:51,374 --> 00:00:52,626
How do you know what the count is?

13
00:00:52,626 --> 00:01:02,493
Like do you have some sort of crawler that goes out there on the internet and finds all
the companies suggesting that they can solve CI C D as a platform or as a product and just

14
00:01:02,493 --> 00:01:03,532
keeping yourself up to date?

15
00:01:03,532 --> 00:01:07,384
Like any good AI these days, I make up the number.

16
00:01:07,384 --> 00:01:18,989
But it goes back to a a thing that I had dealt with early on in my cloud security career,
which was hey, everybody is doing something differently because we all kind of evolved our

17
00:01:18,989 --> 00:01:22,571
way to the cloud rather than planned on migration to the cloud.

18
00:01:22,571 --> 00:01:28,504
And so, yes, I had one team over in this one brand that was doing something with.

19
00:01:28,504 --> 00:01:35,276
you know, maybe Circle CI and another one was Building with Jenkins and another one was I
I don't remember all of them.

20
00:01:35,276 --> 00:01:37,937
This was before GitHub Actions was a thing.

21
00:01:37,937 --> 00:01:42,348
And so like each team had kind of evolved to do its own thing.

22
00:01:42,348 --> 00:01:50,411
And then I was said brought in as the the first cloud security person and told, okay,
everybody should be doing CI C D get them all to do it securely.

23
00:01:50,411 --> 00:01:52,622
And I'm like, you're doing it that way.

24
00:01:52,622 --> 00:01:53,612
I can't.

25
00:01:53,704 --> 00:02:01,497
Um, so I started like trying to explain it to leadership and to others that, you know, if
there's fifteen ways to do CICD, we're doing twenty-one of them.

26
00:02:01,497 --> 00:02:08,040
Um, and then that company merged with three other companies and there were more ways that
I was responsible for.

27
00:02:08,040 --> 00:02:12,492
And then I left to join one another company which was doing similarly the same thing.

28
00:02:12,492 --> 00:02:14,523
Now those two companies came together.

29
00:02:14,523 --> 00:02:21,398
And so, yes, it does seem like that over time there just seemed to be more and more ways
to solve the same problem.

30
00:02:21,398 --> 00:02:28,130
And large organizations never really kind of forced teams to get on board with doing it
one particular way.

31
00:02:28,130 --> 00:02:28,831
Why do you think that is?

32
00:02:28,831 --> 00:02:30,472
Is it like all of them are just bad?

33
00:02:30,472 --> 00:02:39,178
And so it's this swamp where you get thrown in into the deep end right away and i people
are just pulling the first thing that they can find.

34
00:02:39,224 --> 00:02:47,999
So it's not a strategic priority for management to invest in getting teams to convert to,
you know, the one true way.

35
00:02:47,999 --> 00:02:59,005
Assuming they could even agree on the one true way, there's no way that management is
going to be like, we could either launch this new application or we could retool this team

36
00:02:59,005 --> 00:03:01,927
to go use the the company's CICD pipeline.

37
00:03:01,927 --> 00:03:05,669
So it it became just organizational tech debt.

38
00:03:05,669 --> 00:03:08,002
Um that hey, this team used

39
00:03:08,002 --> 00:03:11,823
you know, this one tooling and this team used another tooling.

40
00:03:11,823 --> 00:03:19,346
You know, I've since jumped around a number of companies since I, you know, came up with
that quote in around 2017.

41
00:03:19,366 --> 00:03:24,848
And I've seen a couple of companies where there is typically the company's one true way of
doing it.

42
00:03:24,848 --> 00:03:28,029
And most everybody's on board with it.

43
00:03:28,029 --> 00:03:31,870
But almost every company these days has some mergers and acquisitions that go on.

44
00:03:31,870 --> 00:03:32,460
And so

45
00:03:32,460 --> 00:03:38,656
We bring in this company, and well, the main company's in AWS shop, but they're in GCP,
they're doing things differently.

46
00:03:38,656 --> 00:03:47,523
Again, no strong motivation for management and the board to say, migrate off of Google
into Amazon.

47
00:03:47,523 --> 00:03:53,508
No, it's just like this is working and we want to integrate this product with this product
because that's how we're going to make money.

48
00:03:53,508 --> 00:03:54,910
And so that's what they do.

49
00:03:54,910 --> 00:04:00,290
And so you've got half the company on GitHub, half the company on GitLab.

50
00:04:00,290 --> 00:04:04,572
You know, some random, you know, weirdos using Bitbucket on-prem.

51
00:04:04,772 --> 00:04:11,370
And as a security person, I have to defend that, not the practice, but the actual
infrastructure.

52
00:04:11,370 --> 00:04:17,677
It, you know, becomes pa painful and frustrating, especially when, you know, you're you're
one person doing that.

53
00:04:17,917 --> 00:04:22,369
so if I sound bitter and if I sound angry at folks, it's not that I'm angry at you.

54
00:04:22,369 --> 00:04:29,066
I'm angry at the situation and maybe angry at your managers for not like prioritizing
that.

55
00:04:29,066 --> 00:04:32,930
We do do something about our raging problem of tech debt.

56
00:04:32,930 --> 00:04:40,470
Um, because I think tech debt is leading to and is going to lead to a really big disaster
coming.

57
00:04:40,470 --> 00:04:41,781
I feel like it's always the case.

58
00:04:41,781 --> 00:04:44,984
It's the thing that is sitting in many organizations.

59
00:04:44,984 --> 00:04:46,416
It doesn't matter if it shows up.

60
00:04:46,416 --> 00:04:51,170
Like it's it's always showing up in some regard in some area in every single team.

61
00:04:51,170 --> 00:04:57,345
And I think one of the reasons it shows up everywhere is because it's not well defined on
what that thing i actually is.

62
00:04:58,507 --> 00:04:59,068
Yeah.

63
00:04:59,068 --> 00:05:05,752
I feel like there's a lot of definitions out there, but you know it's like I it's hard to
describe it usually, but you know it when you

64
00:05:05,752 --> 00:05:06,202
See ya.

65
00:05:06,202 --> 00:05:08,754
So it's the potholes in the paved roads, right?

66
00:05:08,754 --> 00:05:12,866
You know, if you've got a really well paved road, you're going at Autobahn speed, great.

67
00:05:12,866 --> 00:05:17,948
But you start having potholes in the Autobahn and you're no longer going, you know, 200
kilometers an hour.

68
00:05:17,948 --> 00:05:19,289
And so that's what tech debt is.

69
00:05:19,289 --> 00:05:29,752
Tech tech debt is those speed bumps, those potholes that exist in your infrastructure that
prevent you from going as fast as you want.

70
00:05:29,752 --> 00:05:30,783
For sure, for sure.

71
00:05:30,783 --> 00:05:39,587
So I I think it's interesting that you bring this up in connection with any sort of
continuous integration platforms, because historically that's been the mechanism to go

72
00:05:39,587 --> 00:05:40,807
even faster.

73
00:05:40,807 --> 00:05:46,560
And so like on one on one side, it seems like the thing that should be prioritized in
every single organization.

74
00:05:46,560 --> 00:05:52,533
And for a lot of years we saw uh I mean, before the door report came out, we saw people
doing whatever.

75
00:05:52,533 --> 00:05:55,960
And then after the door report, we still saw people doing whatever, but

76
00:05:55,960 --> 00:06:01,535
They sort of understood that there were organizations out there that were moving faster
than they were using one of these strategies.

77
00:06:01,535 --> 00:06:10,463
And I think while you could point to a leadership approach of this mindset has to be
applied throughout the organization because clearly there are some benefits.

78
00:06:10,463 --> 00:06:20,602
I found it to be incredibly a challenging conversation to have with leaders to explain
that they do have tech debt in some capacity and they're not actually it's not just about

79
00:06:20,602 --> 00:06:23,374
going fast necessarily, just even building a

80
00:06:23,374 --> 00:06:26,284
product that works tends to be a core challenge.

81
00:06:26,284 --> 00:06:26,574
Yeah.

82
00:06:26,574 --> 00:06:32,598
So I've been doing this since well, I started my career about the time that Linus was
creating Linux.

83
00:06:32,598 --> 00:06:39,302
So we have, you know, uh I I have a broad perspective of what this looked like.

84
00:06:39,302 --> 00:06:51,578
And before we had DevOps, we had gray bearded um sysadmins who would walk into a room and
take a machine out of a box and shove it in a rack and cut their fingers on those rack uh

85
00:06:51,578 --> 00:06:53,410
screws and everything else.

86
00:06:53,410 --> 00:06:54,358
And then they'd

87
00:06:54,358 --> 00:07:00,201
stick in a CD and then we got really fancy and we were able to pixie boot off of a and
kickstart a server.

88
00:07:00,201 --> 00:07:02,182
But you know, that was time.

89
00:07:02,182 --> 00:07:11,207
And even before those, you know, sysadmin was taking the thing out of the box, there was
the finance people approving the budgets and then the procurement people ordering the

90
00:07:11,207 --> 00:07:14,209
stuff and the UPS driver delivering the servers.

91
00:07:14,209 --> 00:07:15,009
Right.

92
00:07:15,009 --> 00:07:24,418
So we suddenly were able to shrink all of that down into, hey, let me run a Terraform
apply and I suddenly have a network and I'm but

93
00:07:24,418 --> 00:07:27,799
bunch of machines and a database and a firewall and all of that.

94
00:07:27,799 --> 00:07:41,443
And so managers were like, wow, this is so much better that they got from horse and buggy,
you know, 25 mile an hour kind of speeds up to, okay, well, now we're going 55 and this

95
00:07:41,443 --> 00:07:43,383
seems really cool.

96
00:07:43,383 --> 00:07:48,435
But there some some teams want to go faster, you know, some teams have a more of a lead
foot.

97
00:07:48,435 --> 00:07:52,618
So they haven't invested to go and get that

98
00:07:52,618 --> 00:07:59,813
incremental amount of performance out of the the pipelines by cleaning up the
infrastructure and and everything else.

99
00:07:59,813 --> 00:08:04,620
And so that's probably why we've gotten to a point and it's good, but it could be better.

100
00:08:04,620 --> 00:08:15,344
So I I think I I don't think it's a stretch of the imagination for to say like technical
minded people to be very well behind the idea of paying down tech debt.

101
00:08:15,344 --> 00:08:20,977
because it's easy for them to see, I think, how much of a problem it is to delivering
stuff effectively.

102
00:08:20,977 --> 00:08:30,401
But I think you're on to something when you say like there's a difficult conversation
there to be had with the people making the decisions on what to work on, because it's not

103
00:08:30,401 --> 00:08:34,092
something that's so easy to just explain.

104
00:08:34,188 --> 00:08:36,580
The impact before actually doing it.

105
00:08:36,620 --> 00:08:44,887
One thing I see as a struggle in a lot of organizations is, uh and this, you know,
happened a few years ago when I was advising one company, they would say they're very data

106
00:08:44,887 --> 00:08:45,808
driven.

107
00:08:45,808 --> 00:08:52,093
And for them, what data driven meant was that you needed to have the data to justify a
decision before you actually did it.

108
00:08:52,093 --> 00:08:59,560
The problem with that is a lot of things in engineering, you don't have that data because
you haven't been allowed to perform the experiment to collect that data in the first

109
00:08:59,560 --> 00:08:59,960
place.

110
00:08:59,960 --> 00:09:02,496
So saying we're data driven is just code for.

111
00:09:02,496 --> 00:09:06,504
I disagree, we're gonna do it my way and you can't do anything about it.

112
00:09:06,540 --> 00:09:15,365
Yes, I would say that, you know, that can be a a different way of weaponizing the the
disagree and commit sort of thing is like, yeah, you disagree, but I'm in charge, so now

113
00:09:15,365 --> 00:09:16,155
we commit.

114
00:09:16,155 --> 00:09:25,160
Shoving observability into the people processes of things is definitely a challenge if it
wasn't baked in there to start with.

115
00:09:25,160 --> 00:09:36,646
Even just the simple matter of me trying to figure out where my team is spending time and
being like, Well, I could look at Jira tickets, but that's not the

116
00:09:36,802 --> 00:09:46,566
fly in help that somebody does giving, you know, answering a question in Slack or, you
know, the time necessary to read up on, hey, what's happening with this latest Vercel

117
00:09:46,566 --> 00:09:48,717
breach and are we impacted or not?

118
00:09:48,717 --> 00:09:57,491
So, you know, the the level of what you can measure doesn't necessarily even always
reflect what what the reality is.

119
00:09:57,491 --> 00:10:04,822
And so right, you know, I think my challenge to everybody in an individual contributor
role is

120
00:10:04,844 --> 00:10:10,838
Find the small places where you can like just solve a little bit of tech debt at a time.

121
00:10:10,838 --> 00:10:13,450
And even that helps clean it up.

122
00:10:13,450 --> 00:10:24,215
I wan I sort I sort of want to get into that because I feel like historically we can draw
a connection between sysadmins early on, uh basically micromanaging and uh some pristine

123
00:10:24,215 --> 00:10:31,017
garden that they had of the scripts and uh servers that they had in place and how to stand
them up and manage.

124
00:10:31,038 --> 00:10:31,792
Chris raises

125
00:10:31,792 --> 00:10:35,767
been a sysadmin, I will tell you that it was never a pristine garden.

126
00:10:35,767 --> 00:10:36,968
That was a lie.

127
00:10:36,968 --> 00:10:38,109
We were fooling you.

128
00:10:38,109 --> 00:10:39,190
Go ahead.

129
00:10:39,394 --> 00:10:45,869
Well, I I think the thing is is that there was a d disconnect between the work that had to
be done and the time that needed to do it.

130
00:10:45,869 --> 00:10:52,955
I feel like sysadmins would be like, you what, it's better if we wait to the weekend to do
the upgrade and then they would come in at ungodly hours to make that happen.

131
00:10:52,955 --> 00:10:58,089
And but that meant like during the rest of the week, they may not have had to dedicate
themselves to doing every activity.

132
00:10:58,089 --> 00:11:05,405
The expectation of being in the office from nine to five thirty or whatever your work
hours were be didn't necessarily align.

133
00:11:05,405 --> 00:11:08,940
And that may that that sort of lended us to the capability of

134
00:11:08,940 --> 00:11:16,075
doing the right thing at the right time and not worrying about the amount that we were
getting done because it was always more like there was always more work.

135
00:11:16,276 --> 00:11:27,005
But I worry that we're making this transition, or we've already made this transition, to
every aspect of the job and the management, going from a garden that we failed to curate

136
00:11:27,005 --> 00:11:32,729
so well, to pushing out as much stuff as possible as fast as we possibly can.

137
00:11:32,870 --> 00:11:36,293
And that means that we no longer have the capacity to curate stuff.

138
00:11:36,293 --> 00:11:38,348
And so my hypothesis is that

139
00:11:38,348 --> 00:11:41,652
we're only going to be creating more and more tech debt unproven.

140
00:11:41,652 --> 00:11:44,005
I have no evidence for this what whatsoever.

141
00:11:44,005 --> 00:11:54,837
But I think it's being accelerated by the LLMs that are just creating stuff where we're
now not only expected to do more, but n no longer have the time to actually go back and

142
00:11:54,837 --> 00:11:56,018
want to clean it up.

143
00:11:56,030 --> 00:11:56,860
Absolutely.

144
00:11:56,860 --> 00:12:05,195
So I'm I was talking with a mentor of mine who actually was one of the people running
about 12 of those 17 CI CD things.

145
00:12:05,576 --> 00:12:14,060
And he was commenting that, you know, at this point, we're getting to where the cost of
writing code is approaching zero.

146
00:12:14,200 --> 00:12:15,871
And, you know, it was true.

147
00:12:15,871 --> 00:12:18,113
I started experimenting with Claude.

148
00:12:18,113 --> 00:12:23,556
Uh I turned on dash dash dangerously skip permissions in a dev container.

149
00:12:23,556 --> 00:12:24,526
And I'm like,

150
00:12:24,526 --> 00:12:26,247
Claude, go figure this out.

151
00:12:26,247 --> 00:12:28,860
And it had access to a sandbox AWS account.

152
00:12:28,860 --> 00:12:30,391
It had access to my code.

153
00:12:30,391 --> 00:12:31,812
And it had access to the internet.

154
00:12:31,812 --> 00:12:38,898
And it solved a problem for me that, you know, was like, okay, that would have taken me
like three days.

155
00:12:38,898 --> 00:12:46,864
It built me a website based on a Yugo template I downloaded and a GitHub repo of an open
source project.

156
00:12:46,864 --> 00:12:48,486
And it figured it all out.

157
00:12:48,486 --> 00:12:49,506
It put together that.

158
00:12:49,506 --> 00:12:54,390
It put together all the reasoning and the rationale behind it, all the documentation.

159
00:12:54,414 --> 00:12:59,756
Claw did that one Saturday afternoon while I was walking along the beach.

160
00:12:59,816 --> 00:13:05,179
And so, you know, like it it is a lot easier to produce the content.

161
00:13:05,179 --> 00:13:12,942
But now then the question is, right, just because your developers uh or what I'm trying to
remember the exact quote, right?

162
00:13:12,942 --> 00:13:21,066
But your developers were so interested in whether or not they could create this code, they
did not consider whether they should have created this code.

163
00:13:21,066 --> 00:13:21,812
And

164
00:13:21,812 --> 00:13:35,159
I like to hearken it back to, again, being now Graybeard, Microsoft Word from the 90s and
2000s, how Word had features that some random enterprise customer insisted that we will

165
00:13:35,159 --> 00:13:37,971
only buy Microsoft if it does this.

166
00:13:38,291 --> 00:13:47,936
And so you had this massively painful piece of bloated software that everybody hate, but
everybody had to use because it was the only thing that was, you know, compatible.

167
00:13:48,137 --> 00:13:50,658
I see a lot of products.

168
00:13:50,784 --> 00:14:03,821
Where you've got the idea of oh, product manager or product owner wants to go into linear
and create a ticket and sketch out something and then LLMs will, you know, push it into

169
00:14:03,821 --> 00:14:05,662
staging andor production.

170
00:14:05,662 --> 00:14:11,585
That's gonna lead to a lot of products starting to look a lot like Word of the the 90s.

171
00:14:11,585 --> 00:14:20,610
Features that they thought were a great idea, but they didn't measure the observability,
you know, they didn't instrument do customers want this?

172
00:14:20,652 --> 00:14:22,543
Are customers using this?

173
00:14:22,543 --> 00:14:26,217
There's this whole feature set, but is anybody actually clicking in?

174
00:14:26,217 --> 00:14:31,851
Has anybody even found it to know that, hey, that you can do all of these cool things in
your product?

175
00:14:31,851 --> 00:14:43,190
All of that is going to be an entirely new area to um suss out as the cost of writing code
and the cost of adding features goes to zero.

176
00:14:43,190 --> 00:14:50,606
Of course, the cost of adding code goes to zero, but the cost of maintaining that code
over time adds up.

177
00:14:50,606 --> 00:14:56,838
I like to say that we're in the in the I I I think that we're in the age of disposable
software is is how I'll put it.

178
00:14:56,838 --> 00:15:05,999
And but you bring up an interesting point about everything tending to go the direction of
having feature bloat and then the consequences of having those extra features.

179
00:15:05,999 --> 00:15:18,449
And so I think my question from there has to be how many features can a product have
before the onus of having those features becomes a critical burden or obstacle to

180
00:15:18,449 --> 00:15:20,290
releasing or

181
00:15:20,300 --> 00:15:24,200
maybe managing that product that you actually have available.

182
00:15:24,200 --> 00:15:26,171
Folk song from the 60s?

183
00:15:26,171 --> 00:15:34,313
You know, that's a good question because we're gonna patch over a lot of that operational
pain by just throwing more tokens at it.

184
00:15:34,313 --> 00:15:38,154
So it's like, okay, we've we've added a lot of features, now the product gets slow.

185
00:15:38,154 --> 00:15:41,050
Well, now we tell the LLM how do we make the product faster?

186
00:15:41,050 --> 00:15:50,904
um so I think we're gonna get we're we're gonna put more plaster on this hole, even though
we haven't actually like fixed the framing behind the hole.

187
00:15:50,904 --> 00:15:59,555
So I think a problem here is that my my theory is that it's impossible to want to review
AI generated code.

188
00:16:00,577 --> 00:16:01,117
Yeah.

189
00:16:01,117 --> 00:16:04,398
Um, so I I think I have this like singularity.

190
00:16:04,398 --> 00:16:08,989
My thought is that every organization has the to technological singularity.

191
00:16:08,989 --> 00:16:19,266
Although what it means is that it's the moment where they no longer are producing or
managing their software development lifecycle with humans and having AI do it.

192
00:16:19,266 --> 00:16:22,449
But as soon as they do it, they can no longer produce good software after that.

193
00:16:22,449 --> 00:16:32,828
So you get to decide like which part of your product or service you have that you are
totally fine with, that you love, and that you don't really want it to change, but you

194
00:16:32,828 --> 00:16:37,382
need to make micro changes to it or improvements, which you can have an LLM do.

195
00:16:37,382 --> 00:16:42,626
So I guess where I'm getting at with that is that there's these organizations that vibe
code out of the gate.

196
00:16:42,626 --> 00:16:47,650
They're AI first companies, but that for me means they're generating tech debt from.

197
00:16:47,650 --> 00:16:49,793
day one in a non sustainable way.

198
00:16:49,793 --> 00:16:56,760
That means the future of their product is is, you know, f a failure uh from immediately
going to happen.

199
00:16:56,760 --> 00:17:11,619
So I don't know if there's any vibe coded companies that have really taken off to the
extent that they are now operating at an AWS or Salesforce scale, right?

200
00:17:11,619 --> 00:17:20,704
Even some of the companies that are kind of like leading edge in the AI space, the
Vercells, you know, they they started off as actual humans writing code.

201
00:17:20,704 --> 00:17:24,970
uh So there's probably still, unless they've been laid off.

202
00:17:24,970 --> 00:17:34,864
Some humans who recall some of the architectural decisions and have some of that
institutional memory of, yeah, what happened that time when you know the data center

203
00:17:34,864 --> 00:17:37,995
caught fire and we had to go fix things and and all of that.

204
00:17:37,995 --> 00:17:51,381
You know, that that's the big fear that a lot of us in the cloud community have around the
massive layoffs of high-level people at AWS, is all of that operational experience is

205
00:17:51,381 --> 00:17:52,701
walking out the door.

206
00:17:52,701 --> 00:17:54,936
Um, and so then when things happen.

207
00:17:54,936 --> 00:18:04,749
completely out of the blue that have never happened before, like missiles landing on a uh
a a region taking out multiple availability zones, right?

208
00:18:04,749 --> 00:18:10,081
The folks who thought about, hmm, what would happen if we lost two of our three
availability zones, how would that respond?

209
00:18:10,081 --> 00:18:11,751
Those people aren't there anymore.

210
00:18:11,751 --> 00:18:17,723
So, you know, even if they're, you know, it's it's not like you can just go and replace an
AWS region.

211
00:18:17,723 --> 00:18:21,762
It takes them years to build one, takes them minutes to burn down.

212
00:18:21,762 --> 00:18:33,355
But even figuring out, okay, how do we best support our customers during this horrifying
time, they don't have the right people to do that any so I feel like, yes, the the the new

213
00:18:33,355 --> 00:18:39,047
companies that are fully v AI first, vibe coded from the get-go.

214
00:18:39,047 --> 00:18:43,268
I don't think we have enough history to know what they're gonna look like.

215
00:18:43,268 --> 00:18:50,860
I think we do know what companies who started off with humans writing code and are now
transitioning into this AI world.

216
00:18:50,904 --> 00:18:53,375
I think we have an idea what that's gonna look like.

217
00:18:53,375 --> 00:19:02,698
And that's gonna look like a lot of micro decisions made without the broader context of
experience.

218
00:19:02,698 --> 00:19:09,139
Cause right, that experience isn't necessarily in the LLM's training data.

219
00:19:09,500 --> 00:19:17,962
And you know, the context windows are too big to go back and look at every Slack message
from every incident that a human

220
00:19:18,216 --> 00:19:22,150
you know in a L7 kind of human would would be able to do.

221
00:19:22,150 --> 00:19:26,313
So yeah, I I think we'll we'll see what that looks like.

222
00:19:26,313 --> 00:19:29,936
And then I think part of that is going back to the tech debt, right?

223
00:19:29,936 --> 00:19:35,801
The more tech debt you get rid of now, the easier your models will have it.

224
00:19:35,801 --> 00:19:43,047
Because it's not like they're looking at there's 16 EC2 instances running in this this
account.

225
00:19:43,047 --> 00:19:44,849
Um they must all be important.

226
00:19:44,849 --> 00:19:46,636
No, they're probably not.

227
00:19:46,636 --> 00:19:46,826
Right.

228
00:19:46,826 --> 00:19:55,551
There might be five of them that the LLM needs to worry about, and eleven of them that are
off here that were spun up by Bob and it's like Bob's ping test.

229
00:19:55,551 --> 00:20:01,375
Um, and by the way, that's open to the world and are running Ubuntu twenty twelve since
twenty twelve.

230
00:20:01,375 --> 00:20:12,981
And, you know, so that's probably cost you eight thousand dollars just in idle compute
capacity since this was done, you know, twenty four years or fourteen years ago.

231
00:20:13,432 --> 00:20:17,144
But it's still something that the LLM has to put into its context window.

232
00:20:17,144 --> 00:20:31,542
Um, and the bigger that problem set gets, context windows, you know, are are getting
bigger, but I don't think that they can scale to the level of cruft and dirt and grime

233
00:20:31,542 --> 00:20:35,532
that you find in people's old cloud accounts.

234
00:20:35,532 --> 00:20:35,962
Yeah.

235
00:20:35,962 --> 00:20:44,217
No, I I so I actually brought this up as a pick in one of the previous episodes where
there's something known as like the the maximum maximum effective context window.

236
00:20:44,217 --> 00:20:54,473
Uh basically there is this problem with LLMs where the more a longer context window
doesn't actually help solve problems because we're already seeing context rot within a

237
00:20:54,473 --> 00:21:01,487
single set of tokens for a single prompt, basically, when there's too much noise there,
how do you figure out where the value is?

238
00:21:01,487 --> 00:21:03,438
And I feel like humans have the same problem.

239
00:21:03,438 --> 00:21:04,248
I write

240
00:21:04,248 --> 00:21:08,441
blog posts that are sometimes uh quoted as being 16 minutes long.

241
00:21:08,441 --> 00:21:12,594
And at that point it's like, well, where's the val like, where is the most important thing
that's in that post?

242
00:21:12,594 --> 00:21:18,279
And sometimes I get people saying, you should have cut that in multiple pieces because I
couldn't read it on my way to work.

243
00:21:18,279 --> 00:21:18,869
It was too much.

244
00:21:18,869 --> 00:21:24,274
And I'm like, yeah, but I thought about I actually thought about this that, you know,
there's not a good place to just cut it in half.

245
00:21:24,274 --> 00:21:26,655
And I feel like that is then becomes the work.

246
00:21:26,655 --> 00:21:30,200
It is incredibly important on the context that we're giving to

247
00:21:30,200 --> 00:21:34,395
humans and therefore the context we're giving to LMs when we need to generate something.

248
00:21:34,395 --> 00:21:44,108
And I feel like if the only context available is a lot of terrible software code that was
written by another L LM, it doesn't really have a lot of intelligent stuff to go on.

249
00:21:44,108 --> 00:21:47,431
Yeah, or terrible software code that was, you know, written by humans.

250
00:21:47,431 --> 00:21:55,229
Because let's let's face it, I can look at any year and just the number of CVEs are going
up, up, up.

251
00:21:55,229 --> 00:21:58,182
And that's not from AI-generated code.

252
00:21:58,182 --> 00:22:00,474
That's from human-generated code.

253
00:22:00,474 --> 00:22:05,098
So we're not doing a great job in writing secure code.

254
00:22:05,520 --> 00:22:08,258
we're training everything on the insecure code.

255
00:22:08,258 --> 00:22:08,779
code.

256
00:22:08,779 --> 00:22:15,648
I don't even know how Mythos was able to actually learn how to do this stuff and what was
right because I don't know where right is.

257
00:22:15,648 --> 00:22:18,199
So I think I think there's something to be said here.

258
00:22:18,199 --> 00:22:28,542
And and that's uh and I I like that you brought up uh mythos, mythos, whatever whatever
it's called, is that we've known for a while what the vulnerabilities look like.

259
00:22:28,542 --> 00:22:31,602
And I don't think it's so much that we're not good at creating secure code.

260
00:22:31,602 --> 00:22:37,044
We're I think we're just incredibly bad at removing insecure code from our environments.

261
00:22:37,044 --> 00:22:45,100
And like you I I think this gets back to the thing you had started with, which is that
it's not really a priority if it's quote unquote working and

262
00:22:45,100 --> 00:22:53,396
the errors and omissions or uh liability insurance covers it, why why not why not leave it
as vulnerable?

263
00:22:53,396 --> 00:22:55,717
I mean, who is that really, really hurting?

264
00:22:55,717 --> 00:23:03,162
And so I I think it's totally believable that we can automate the process of finding these
vulnerabilities, which we already know exist.

265
00:23:03,162 --> 00:23:04,903
We've known they've existed forever.

266
00:23:04,903 --> 00:23:06,890
We know how to exploit them.

267
00:23:06,890 --> 00:23:14,924
I mean, I I think one thing the LLMs do, and I think what r really mythos is doing, is
putting together individual steps

268
00:23:14,924 --> 00:23:20,578
Which hypothetically in the past you always knew could be the case is actually stringing
them together in the right way.

269
00:23:20,578 --> 00:23:26,760
Yeah, building the attack chain is what Mythos is apparently particularly good at.

270
00:23:26,890 --> 00:23:34,332
and whether Mythos is hype or it's the end of the world, it's either Mythos or Mythos the
next generation.

271
00:23:34,332 --> 00:23:39,024
You know, at some point we're gonna start seeing this flood of attacks.

272
00:23:39,024 --> 00:23:46,986
But even before the flood of attacks, I think with Mythos and with Anthropic's project
Glasswing, which is their limited release.

273
00:23:47,064 --> 00:23:57,537
Hey, come check out Mythos that they're giving out to 50 highly trusted companies, um,
including Amazon and Microsoft and Apple and Google and you know, bunch of other companies

274
00:23:57,537 --> 00:23:58,957
that are like make software.

275
00:23:58,957 --> 00:24:01,918
We're gonna see this influx in CVEs.

276
00:24:01,918 --> 00:24:10,900
So, right, one of the things that everybody needs to start focusing on is how quickly can
we patch and how hard is patching?

277
00:24:11,000 --> 00:24:17,218
And how many places do we have that Ubuntu 1804 instance still running?

278
00:24:17,218 --> 00:24:23,641
that aren't gonna get the patches, because that then becomes your dangerous attack
surface.

279
00:24:23,641 --> 00:24:33,926
Once you figure out and you actually get to, you know, regular patching that that nothing
in the cloud lives for more than a few days, you've got to find all of the things that are

280
00:24:33,926 --> 00:24:35,406
older than a few days.

281
00:24:35,406 --> 00:24:43,394
And most of those are going to be the that that that tech debt, the Bob's ping test
machine or

282
00:24:43,394 --> 00:24:52,661
That one MySQL five database that nobody's really sure who's responsible for and nobody's
really sure that nobody's really sure they want to put their career on the line by saying,

283
00:24:52,661 --> 00:24:54,546
Hey, yeah, I'm gonna go turn that off now.

284
00:24:54,546 --> 00:25:01,108
I I'm sm I'm smiling because you know, I I don't know if you are, but I'm just gonna
assume you are a huge serverless advocate.

285
00:25:01,108 --> 00:25:02,748
It sounds like we were right all along.

286
00:25:02,748 --> 00:25:12,971
Serverless is the way to go because if you have to manage individual versions or versions
of packages or operating systems, like you're it's a losing battle uh against uh against

287
00:25:12,971 --> 00:25:22,644
what is going to be an onslaught of just attacks coming in because it's going to it
basically we're talking about it basically being free to exploit something.

288
00:25:22,862 --> 00:25:28,665
Claude, tell me how many Python 3.9 lambdas I have running in the Prime Harbor environment
right now.

289
00:25:28,665 --> 00:25:38,501
Um, by the way, Claude isn't listening and I haven't hooked it up to have full access to
everything, but I guarantee you it's gonna come back with a number that's in the triple

290
00:25:38,501 --> 00:25:39,192
digits.

291
00:25:39,192 --> 00:25:45,476
Because, right, you know, it's like, I'll spin something up and it's a proof of concept
and then it's sitting in an AWS account somewhere.

292
00:25:45,476 --> 00:25:50,018
The nice thing about serverless is it tends to not have much of an attack surface.

293
00:25:50,272 --> 00:25:57,000
especially with the event driven serverless stuff that I write that's not beh you know,
fronted by an ALB or a a API gateway.

294
00:25:57,048 --> 00:25:59,340
Yeah, no, I I mean I totally totally behind that.

295
00:25:59,340 --> 00:26:07,907
Actually, it's interesting you bring that up because for Lambda it runs on Firecracker and
I think I think one of the first C VEs was just released for Firecracker of a container

296
00:26:07,907 --> 00:26:09,739
escape list or last week or so.

297
00:26:09,739 --> 00:26:13,352
Uh which is I mean, it obviously like an LLM found it.

298
00:26:13,352 --> 00:26:14,613
I say LLM found it.

299
00:26:14,613 --> 00:26:19,517
A human found it using a set of tools that was available to them at the time.

300
00:26:19,517 --> 00:26:25,962
But yeah, it just it just seems inevitable for for those sets of technology and limiting
your attack service.

301
00:26:25,976 --> 00:26:33,335
to the area which you can actually secure seems like the thing that everyone should have
been doing all along, but now is a good reminder that if you're not actively paying

302
00:26:33,335 --> 00:26:39,916
attention to what that interface is, that you have a a huge vulnerability there.

303
00:26:39,916 --> 00:26:40,406
Yeah.

304
00:26:40,406 --> 00:26:43,768
Well, and it's the things that are it's just working, right?

305
00:26:43,768 --> 00:26:43,948
Yeah.

306
00:26:43,948 --> 00:26:45,998
I don't need to change it, it's just working.

307
00:26:45,998 --> 00:26:52,761
Well, now, even though it's just working, it's still running Python three nine, you need
to change it.

308
00:26:52,761 --> 00:26:55,332
You need to change it for the sake of changing it.

309
00:26:55,332 --> 00:27:08,538
Um, and that's I think where like engineering management and folks in security and folks
on the front lines maybe have a different uh set of opinions and and expectations.

310
00:27:08,538 --> 00:27:09,814
So, you know

311
00:27:09,814 --> 00:27:18,460
I think the primary takeaway from all of this methos in Project Glasswing is prepare to do
a lot more patching.

312
00:27:18,460 --> 00:27:32,170
Um, and expect that that's gonna become a bigger part of not not even a security
professionals time, but a DevOps professionals, an SRE time that that we're gonna end up

313
00:27:32,170 --> 00:27:38,444
in a situation where security isn't handing you spreadsheets anymore.

314
00:27:38,536 --> 00:27:44,793
It's security is saying this thing has been running for a little bit too long.

315
00:27:44,833 --> 00:27:47,196
Make sure that it's got everything up to date.

316
00:27:47,196 --> 00:27:50,019
But not too up to date because there's supply chain attacks.

317
00:27:50,019 --> 00:27:51,711
And so you never want to grab the latest.

318
00:27:51,711 --> 00:27:55,666
You always want to grab the slightly less latest, unless, of course, there's a zero day
out there.

319
00:27:55,666 --> 00:27:59,630
Um, and so, you know, I mean, how could you not win at this?

320
00:27:59,902 --> 00:28:10,370
see, I think I think that's one of the fool's errands uh current security advice that is
not a not a good thing to tell people, but is right now a little bit valuable.

321
00:28:10,370 --> 00:28:19,738
The the delay installing new packages for one week or two weeks is only helpful as long as
other people are installing those packages right away.

322
00:28:19,738 --> 00:28:20,330
So

323
00:28:20,330 --> 00:28:23,433
you're just utilizing this delay and finding value in it.

324
00:28:23,433 --> 00:28:25,995
But as soon as everyone agrees, yeah, you know what?

325
00:28:25,995 --> 00:28:28,236
Vulnerabilities only happen in new release versions.

326
00:28:28,236 --> 00:28:29,858
Let's just wait a couple of weeks.

327
00:28:29,858 --> 00:28:35,852
Then everyone will be waiting, which means that no one will find those vulnerabilities
until that moment happens.

328
00:28:36,174 --> 00:28:44,116
Unless of course those vulnerabilities are being actively exploited, in which case the
folks who will find them are in fact going to be, yeah.

329
00:28:44,116 --> 00:28:51,558
I think there there is value in pausing and not grabbing the latest thing that was
uploaded ten minutes ago.

330
00:28:51,558 --> 00:29:02,341
But I don't think it's more than seventy-two hours and that's just to give you enough time
to actually enjoy your weekend before you know some something drops in like that.

331
00:29:02,341 --> 00:29:04,928
And then you you need to have that kind of

332
00:29:04,928 --> 00:29:10,748
emergency switch where it was like, yes, the general rule is wait 72 hours before using
it.

333
00:29:10,748 --> 00:29:16,302
But in this case, go grab this particular version right now, even if it's younger than 72
hours.

334
00:29:16,302 --> 00:29:16,512
hours.

335
00:29:16,512 --> 00:29:16,942
Yeah.

336
00:29:16,942 --> 00:29:17,452
No, agreed.

337
00:29:17,452 --> 00:29:22,624
I I think it's a bad proxy for what should be looked at, which is trust.

338
00:29:22,624 --> 00:29:25,385
How much do we trust this version of this package?

339
00:29:25,385 --> 00:29:33,468
And d time can be helpful because it's correlated with the number of say zero days or
discovered exploits.

340
00:29:33,468 --> 00:29:34,849
They're not zero days at that point.

341
00:29:34,849 --> 00:29:38,488
However, there's no that's not actually the metric that you probably want to be using.

342
00:29:38,488 --> 00:29:45,900
There's a good number of companies out there that are scanning the latest and looking for
weird things.

343
00:29:45,900 --> 00:29:56,434
So you will, you know, even we're not finding these hacks based on the fact that
somebody's getting breached from these supply chain things.

344
00:29:56,434 --> 00:30:04,426
We we really are hearing about these hacks from the vendor community, from the researchers
who are looking at things and

345
00:30:04,426 --> 00:30:10,269
Even just the project maintainer who may wake up and be like, I didn't release three point
six point one last night.

346
00:30:10,269 --> 00:30:18,704
Um, and then being like, Shit, pulling that and and warning everybody if you installed
this, don't, you know, go get three dot seven.

347
00:30:18,704 --> 00:30:29,920
So there there is some value in some level of pause, but I don't think you you certainly
don't want to race to um and you don't want to rely on your fellow companies that are

348
00:30:29,920 --> 00:30:31,942
leveraging this to find this stuff.

349
00:30:31,942 --> 00:30:32,704
Cause yeah.

350
00:30:32,704 --> 00:30:35,179
We're not good at finding this stuff.

351
00:30:35,179 --> 00:30:38,887
That's what the sec yeah, that that's what the security vendor community is for.

352
00:30:38,887 --> 00:30:44,824
They are working together, they're doing a reasonably good job of of tracking this threat
activity.

353
00:30:44,824 --> 00:30:53,361
So we're actually hopefully a couple a couple of weeks from now, we should have someone
coming on to talk a little bit more about supply chain security on the developer side as

354
00:30:53,361 --> 00:30:54,312
well as what's getting released.

355
00:30:54,312 --> 00:31:00,527
There's a lot of interesting products in the place in the in the marketplace and the
internet that claim to do something.

356
00:31:00,527 --> 00:31:04,590
And hopefully we'll see a a deeper understanding of what exactly is happening there.

357
00:31:04,650 --> 00:31:14,538
One thing I do want to sort of probe you on a little bit is the sorts of activities that
you're doing and what you're seeing at some of the companies that you're supporting.

358
00:31:14,760 --> 00:31:21,112
when you're going in today, what is like the number one area that you seem to be focusing
a lot on?

359
00:31:21,112 --> 00:31:22,803
So it's two things.

360
00:31:22,803 --> 00:31:31,206
Um, one it's gosh, we have to do AI and h how do we do AI, how do we do it f quickly?

361
00:31:31,206 --> 00:31:35,478
And then I'm like, Well, yeah, but you're gonna generate risk here if you do that.

362
00:31:35,478 --> 00:31:44,351
And then the other thing is, hey, help me figure out, you know, how to eat these security
vegetables that have been sitting rotting on my plate for a bunch of years.

363
00:31:44,351 --> 00:31:49,734
Um and those tend to be like the the the two things and they're actually related, right?

364
00:31:49,734 --> 00:31:50,560
Because

365
00:31:50,560 --> 00:32:01,036
Again, I'll say if you you shouldn't be doing uh fully agentic autonomous AI stuff if
you've got a lot of tech debt and you haven't actually built your environment up to

366
00:32:01,036 --> 00:32:02,297
support that.

367
00:32:02,297 --> 00:32:06,698
So I'm working with one client on okay, so what does fully agentic mean?

368
00:32:06,698 --> 00:32:10,282
And what what what do they have to do to be able to get there?

369
00:32:10,282 --> 00:32:17,496
And then, you know, just other companies with help me figure out, you know, do I need this
security tooling or can I turn it off?

370
00:32:17,496 --> 00:32:17,776
Right.

371
00:32:17,776 --> 00:32:18,456
Like

372
00:32:18,456 --> 00:32:30,020
Cost and security and operations are really three elements of this this cloud governance
triad that I talked about many, many years ago, probably while I was making sourdough

373
00:32:30,020 --> 00:32:31,750
during p the pandemic.

374
00:32:31,910 --> 00:32:38,292
And so, right, it's like cost matters, security matters, you know, the operations element
matters.

375
00:32:38,292 --> 00:32:44,664
And all three generally are in alignment because security and oper security and finance
can get together and say, look.

376
00:32:44,664 --> 00:32:47,149
This thing is a security risk and it's costing you money.

377
00:32:47,149 --> 00:32:51,205
And so if you turn it off, you've solved two problems at once.

378
00:32:51,334 --> 00:33:01,461
I guess, you know, part part of it is you're you're coming in and you're helping companies
answer the question of basically implementing AI or agentic solutions uh with their

379
00:33:01,461 --> 00:33:04,223
current organizational structure, the current technology.

380
00:33:04,223 --> 00:33:07,775
How are they where are they defining the goalposts to actually be?

381
00:33:07,775 --> 00:33:14,470
And I I feel like that's a very vague sort of unit's like we need I I hear this a lot and
I think it comes up on the podcast.

382
00:33:14,470 --> 00:33:18,092
We need to have AI in our software development lifecycle.

383
00:33:18,092 --> 00:33:20,940
We need to have it within our actual product that we're providing.

384
00:33:20,940 --> 00:33:25,372
But I feel like a lot of times the people that are talking about this don't fully
understand the capabilities there.

385
00:33:25,372 --> 00:33:30,535
So I guess my question is like, how are you helping them to navigate that?

386
00:33:30,535 --> 00:33:35,598
Or is it they've already decided where the goalpost is and now you're like, How can we
still make sure this is secure?

387
00:33:35,678 --> 00:33:40,499
could have given me some goalposts, that would have made my job a heck of a lot easier.

388
00:33:40,499 --> 00:33:44,791
What it started with is, oh, we should use some AI.

389
00:33:44,791 --> 00:33:48,862
So let's get a, you know, some chatbot in to help and do code assist.

390
00:33:48,862 --> 00:33:55,084
And so we start working on, okay, what are the procurement, privacy, all of that stuff to
do code assist.

391
00:33:55,084 --> 00:34:00,755
Meanwhile, there's a couple folks over here who are like all off doing cursor agents and
MCPs and all this fancy stuff.

392
00:34:00,755 --> 00:34:03,826
And then everybody's suddenly like, ooh, we want to do what they're doing.

393
00:34:03,962 --> 00:34:10,987
Um, and so then like the the engineering and the the initiative effort pivots to that.

394
00:34:10,987 --> 00:34:16,552
But we still haven't actually solved some of the things around like, so what's the proper
governance for MCPs?

395
00:34:16,552 --> 00:34:19,634
And they're off like, MCPs are dead, everything's command line now.

396
00:34:19,634 --> 00:34:22,236
Um and so it's like, okay, yeah.

397
00:34:22,236 --> 00:34:23,937
Yeah, and and I get it, right?

398
00:34:23,937 --> 00:34:30,842
It it took me a while before I finally like built the harness so I could let AI be AI.

399
00:34:31,317 --> 00:34:43,432
And the moment I did that, the moment I filed up, fired up the Trail of Bits dev container
and gave it some AWS credentials and a code base and said, go fix this thing for me.

400
00:34:43,432 --> 00:34:47,744
And it did it, and it iterated on itself.

401
00:34:48,085 --> 00:34:50,896
And I had, you know, slash remote control turned on.

402
00:34:50,896 --> 00:35:00,182
So I was shopping for socks at the mall while I was on my phone watching it figure out
what it needed to do, me giving it occasional prompting.

403
00:35:00,182 --> 00:35:02,083
You know, that was the aha moment for me.

404
00:35:02,083 --> 00:35:15,854
That was the same level of aha moment that I had back in 2014 when I went to reInvent and
saw somebody replace the data center engineer who took six months to get fiber from one

405
00:35:15,854 --> 00:35:24,930
end of a room to another one because, we don't have the fiber, or we bought the wrong
optics, or well, I was dealing with an outage, or suddenly I went to a cloud formation

406
00:35:24,930 --> 00:35:27,638
session on a Friday morning.

407
00:35:27,638 --> 00:35:31,910
And they were like, and so here's how VPC peering looks in in in cloud formation.

408
00:35:31,910 --> 00:35:35,898
I'm like, my God, 16 line of JSON will replace that engineer.

409
00:35:35,898 --> 00:35:37,166
It was like, ah.

410
00:35:37,166 --> 00:35:40,789
Then a few months later, they made, you know, made YAML available in CloudFormation.

411
00:35:40,789 --> 00:35:42,170
And then it was like four lines.

412
00:35:42,170 --> 00:35:44,392
That was my cloud aha moment.

413
00:35:44,392 --> 00:35:50,067
And then this was the kind of AI aha moment where it's not code assist.

414
00:35:50,067 --> 00:35:56,246
Yeah, code assist is helpful, but it's like, here's a problem to solve, and here's your
constrained.

415
00:35:56,246 --> 00:36:01,327
Your constrained environment in which to do it, go do it and show me what you got.

416
00:36:01,327 --> 00:36:02,718
That was the beautiful moment.

417
00:36:02,718 --> 00:36:04,048
That was my moment of.

418
00:36:04,048 --> 00:36:10,650
So that is really what I'm now trying to sell to customers who really are like, what
should we do about AI?

419
00:36:10,650 --> 00:36:21,173
You should get to the point where for certain discrete, concrete tasks, you can prompt it
and it will do it and it will test it and it will give you a commit.

420
00:36:21,173 --> 00:36:25,922
And then you'll sit or you'll you'll spend some time looking at it and being like,

421
00:36:25,922 --> 00:36:27,973
Yep, this seems like a reasonable solution.

422
00:36:27,973 --> 00:36:30,604
And you'll merge it into production and you'll get something out.

423
00:36:30,604 --> 00:36:34,976
But you need to have a really good sandbox environment that the LLM can work in.

424
00:36:34,976 --> 00:36:41,649
You need to have a good staging environment that you can test what the LLM did long before
it gets to production.

425
00:36:41,649 --> 00:36:53,774
And if your engineers have access to production and staging and dev, and they're just
sharing their dot AWS directory to the models.

426
00:36:53,774 --> 00:36:57,335
Well, now your model has access to production.

427
00:36:57,335 --> 00:37:04,817
You might not have told it to go do anything in production, but these things are really
determined to figure it out.

428
00:37:04,817 --> 00:37:16,440
And they will go ahead and delete a database or terminate your, you know, Ruby application
because, well, I was asked to upgrade this to Ruby 311.

429
00:37:16,440 --> 00:37:20,241
So easiest thing to do is Terraform Destroy, and then I can go recreate it.

430
00:37:20,241 --> 00:37:22,424
And it was like, that was running the company.

431
00:37:22,424 --> 00:37:28,586
Have you have you run into any horror moments with the rollout in any of your uh your
customers and seeing

432
00:37:28,586 --> 00:37:40,192
I have um I have not had them, you know, the the meta AI safety expert, my god, running to
unplug the Mac mini because their open claw started deleting everything.

433
00:37:40,192 --> 00:37:48,277
I have seen Claude be helpful and go ahead and accept a mark AWS marketplace terms and
conditions on behalf of me for Claude.

434
00:37:48,277 --> 00:37:51,979
Um so anthropic Claude enabling Claude in bedrock.

435
00:37:51,979 --> 00:37:55,201
And it was like, hey, you just did signed a legal agreement.

436
00:37:55,201 --> 00:37:56,331
What is that legal?

437
00:37:56,331 --> 00:37:57,952
What what how does that work?

438
00:37:57,952 --> 00:38:03,293
right, you know, when the LLMs are clicking on terms and services, has anybody actually
signed that?

439
00:38:03,293 --> 00:38:07,892
I am really interested to see where the first legal case of of that goes.

440
00:38:07,892 --> 00:38:21,829
If we let the executives at large organizations have their way, I think they would be
saying that whoever owns that LLM is accountable for the decisions that LLM makes.

441
00:38:21,829 --> 00:38:32,293
But I think in practice, where it's gotta be the other way around, where we it's
incredibly impossible to get engineers to review hundreds of thousands of lines of

442
00:38:32,293 --> 00:38:37,866
generated code that contains mostly garbage, there's no way you can convince them to
somehow know.

443
00:38:37,866 --> 00:38:40,644
magically what all the things are that an LM w we're doing.

444
00:38:40,644 --> 00:38:42,784
I think it's really unfair to hold them accountable for that.

445
00:38:42,784 --> 00:38:43,504
Well, and that's it.

446
00:38:43,504 --> 00:38:52,358
And I don't think I I think that because, you know, I had this discussion with a client
where like we were writing an acceptable policy and acceptable usage policy.

447
00:38:52,358 --> 00:38:56,079
And one of the things that, you know, was you're responsible for the output of your code.

448
00:38:56,079 --> 00:38:58,601
And I'm like, that made sense six months ago.

449
00:38:58,601 --> 00:38:59,841
Now I don't think so.

450
00:38:59,841 --> 00:39:02,338
I think at this point, this whole

451
00:39:02,338 --> 00:39:04,869
You're accountable for the output of the LLMs.

452
00:39:04,869 --> 00:39:08,840
This whole idea of human in the loop, I think is quickly going to become a fiction.

453
00:39:08,840 --> 00:39:19,743
And I think by the end of the year, we're very much not going to have most things where
somebody is actually line by line reviewing and understanding what it is.

454
00:39:19,743 --> 00:39:28,466
Yes, I may take my output from Claude and hand it off to Codecs and say, Hey, Codex, is
this, you know, slop or is this thing good?

455
00:39:28,466 --> 00:39:29,280
You know, or

456
00:39:29,280 --> 00:39:38,035
Level multiple agents that have like, you know, coding standards and architectural
standards and everything else to like review that.

457
00:39:38,035 --> 00:39:53,314
But you know, I think at the end of the day, you know, we've got very specific and tragic
examples right now of where human in the loop has failed and has led to yes, the disasters

458
00:39:53,314 --> 00:39:55,485
that have happened in recent conflicts.

459
00:39:55,485 --> 00:39:56,576
So if

460
00:39:56,576 --> 00:40:00,233
like life and death decisions don't have human in the loop.

461
00:40:00,233 --> 00:40:06,976
Pushing this color change feature and breaking the CSS probably is not gonna have human in
the loop, right?

462
00:40:06,976 --> 00:40:16,122
I think for the longest time the engineers, the organizations that were on the cutting
edge of how to lead an organization effectively were very on top of the idea of blameless

463
00:40:16,122 --> 00:40:27,449
p postmortems, which is sort of like this concept of we did all of the things that were
highly likely and recommended in the moment, given the lack of information we had at the

464
00:40:27,449 --> 00:40:27,789
time.

465
00:40:27,789 --> 00:40:28,139
Right.

466
00:40:28,139 --> 00:40:30,801
Like we did the best job we could knowing what we knew.

467
00:40:30,801 --> 00:40:31,481
Is that the case?

468
00:40:31,481 --> 00:40:32,662
And usually that's the case.

469
00:40:32,662 --> 00:40:40,662
And so I feel like it's gotta be these organizations that don't have a concept of
blameless postmodem mortems that are trying to transition the accountability model from

470
00:40:40,662 --> 00:40:48,124
individuals, which they felt like they could fire, to LMs where they feel like they don't
have the ability to manage on an individual level.

471
00:40:48,124 --> 00:40:49,844
I don't know where that I'm going with that.

472
00:40:49,844 --> 00:40:56,757
That accountability is like to the level of wanting to terminate something or somebody.

473
00:40:56,757 --> 00:40:57,794
Let me hear from you.

474
00:40:57,794 --> 00:41:02,859
Um, you know, but the accountability is right, you know, you broke it, you fix it.

475
00:41:02,859 --> 00:41:12,524
Um, you know, the accountability is, hey, you know, Frank made this mistake, let's all
learn from it, kind of thing.

476
00:41:12,524 --> 00:41:18,934
And so I also want to actually take a step back and disagree slightly with the idea that.

477
00:41:18,954 --> 00:41:26,451
Small and medium-sized businesses can model their practices off of what Amazon, Google,
Microsoft, um, and Salesforce are doing.

478
00:41:26,451 --> 00:41:28,943
These are trillion dollar companies.

479
00:41:28,943 --> 00:41:37,311
They've got thousand-person engineering teams, or tens of thousands of engineers,
thousands of security professionals working there.

480
00:41:37,311 --> 00:41:45,322
Most of us are working at much smaller organizations, maybe a hundred engineers, maybe
three to four security folks.

481
00:41:45,322 --> 00:41:55,859
What what Steve Schmidt can tell me that I should be doing in a reInvent leadership
session and what I can actually practically do drastically drastically different.

482
00:41:55,859 --> 00:42:00,271
And so the first of those leadership sessions I went to, I was like, wow, this is so
inspiring.

483
00:42:00,271 --> 00:42:02,473
And then I kind of was like, you know what?

484
00:42:02,473 --> 00:42:03,654
Screw you.

485
00:42:03,654 --> 00:42:09,037
You've got a thousand people working for you and you've got a trillion dollar budget to do
this in.

486
00:42:09,037 --> 00:42:11,138
I've got three people.

487
00:42:11,390 --> 00:42:18,813
And a company that's, you know, like twenty to forty years in tech debt, and you know, I
can't accomplish that.

488
00:42:18,813 --> 00:42:24,625
So you telling me that I should be doing this, it's like eat your security vegetables, you
know, it's like eat a field of corn.

489
00:42:24,625 --> 00:42:27,035
No, it's like I can eat one thing.

490
00:42:27,035 --> 00:42:29,216
My partner here can eat two.

491
00:42:29,216 --> 00:42:31,517
So we're we're gonna get this done.

492
00:42:31,517 --> 00:42:32,217
That's it.

493
00:42:32,217 --> 00:42:35,368
We're we're not gonna be able to do all of what you can do.

494
00:42:35,532 --> 00:42:42,865
I I think there is definitely a a miss there in communicating all of the relevant facts
about utilizing LLMs effectively.

495
00:42:42,865 --> 00:42:53,620
I think even if they are capable of say solving a particular problem that's been
advertised, they very rarely, if ever, get into under like conveying what the necessary

496
00:42:53,620 --> 00:42:55,430
infrastructure was to make that happen.

497
00:42:55,430 --> 00:43:00,522
And one of the things they leave off is the dollar signs, uh or euro signs or pounds or
whatever.

498
00:43:00,522 --> 00:43:04,638
Uh I see a a lot of the hyperscalers saying, look what we accomplished with

499
00:43:04,638 --> 00:43:14,741
just having a few a couple of very senior engineers, uh distinguished scientists or
whatever, their their L sevens, L eights, L nines, using an LLM were able to in one month

500
00:43:14,741 --> 00:43:15,831
bring this whole product up.

501
00:43:15,831 --> 00:43:17,702
And I'm like, Okay, great.

502
00:43:17,702 --> 00:43:25,764
I sort of believe you, but if a person outside of your company went to try to use your
models, like how much would you charge them to actually utilize it?

503
00:43:25,764 --> 00:43:26,764
Like what plan would they pick?

504
00:43:26,764 --> 00:43:32,966
And I thought if you evaluate that, you're probably paying millions to accomplish that
same thing.

505
00:43:32,966 --> 00:43:34,410
And and I'm just like that's not

506
00:43:34,410 --> 00:43:37,873
half of that just to get an L eight from one of those companies.

507
00:43:37,873 --> 00:43:38,244
Right.

508
00:43:38,244 --> 00:43:47,742
So right you've got a million dollar budget, half of that is going to the one L eight
that's gonna come in and run the model, not to mention then the cost of the model.

509
00:43:48,096 --> 00:43:57,254
I just like even if you're paying even if you're paying through a third party, like even
if you were to pay, say, Claude uh or Anthropic for uh Opus four point seven model to just

510
00:43:57,254 --> 00:44:00,898
run in cycles to complete your product for you, how much would that actually cost?

511
00:44:00,898 --> 00:44:09,906
Is just an ungodly amount of money, realistically, that I think just completely gets
ignored when Anthropic comes out and says, we created this new product, we did it using

512
00:44:09,906 --> 00:44:11,026
our model.

513
00:44:11,127 --> 00:44:15,390
You don't have access to the the cost uh equation that they do.

514
00:44:15,810 --> 00:44:20,223
There is another element of that too, in addition to cost, which is time, right?

515
00:44:20,223 --> 00:44:22,254
The LLMs don't sleep.

516
00:44:22,254 --> 00:44:27,827
So once you take the human out of the loop, the LLM can do a hell of a lot more while the
human is sleeping.

517
00:44:27,827 --> 00:44:28,738
For sure.

518
00:44:28,738 --> 00:44:32,300
I do want to say that like when we talked a little bit about Mythos, right?

519
00:44:32,300 --> 00:44:38,244
The one one of the main bugs that it found was like this 20 year old thing and OpenBSD and
whatever.

520
00:44:38,244 --> 00:44:42,376
And I'm told that it was like a mid five figure token cost to find.

521
00:44:42,614 --> 00:44:54,269
So right, you know, I'm not gonna go and for my little open source projects drop twenty to
thirty thousand dollars to have it run the most frontier of frontier models to to to look

522
00:44:54,269 --> 00:44:55,414
for my problems.

523
00:44:55,414 --> 00:44:56,114
Yeah.

524
00:44:56,114 --> 00:45:05,947
Well, one of the problems with that is I'm pretty sure they ran the exact same practice on
thousands of repositories out there in the world that are also all claimed to be highly

525
00:45:05,947 --> 00:45:06,937
secure.

526
00:45:06,937 --> 00:45:17,701
And then when they found one, they just reported on that one instance because that is news
making and not the fact, yeah, actually we ran we burned, you know, hundreds of thousands

527
00:45:17,701 --> 00:45:20,221
on this, didn't find anything.

528
00:45:20,391 --> 00:45:22,932
so we're just not we're just not gonna talk about that.

529
00:45:22,956 --> 00:45:23,587
Yeah.

530
00:45:23,587 --> 00:45:30,154
I think I heard somewhere that like they the project glasswing token budget.

531
00:45:30,154 --> 00:45:30,704
Yeah.

532
00:45:30,704 --> 00:45:35,320
where Anthropic is giving out tokens is about a hundred million in tokens.

533
00:45:35,320 --> 00:45:36,120
Yeah.

534
00:45:36,120 --> 00:45:45,130
and that's defined, you know, the vulnerabilities in the core critical infrastructure, you
know, that that makes up, you know, most of

535
00:45:45,132 --> 00:45:46,623
m modern civilization.

536
00:45:46,623 --> 00:45:58,874
So I imagine that, yeah, there there are a number of passes that Myth Mythos is making
over the Linux kernel, over, you know, Java JDK, um, yeah, JDK uh, you know, runtimes and

537
00:45:58,874 --> 00:45:59,804
all of that.

538
00:46:00,216 --> 00:46:00,927
I think that makes sense.

539
00:46:00,927 --> 00:46:12,297
My worry is that we are already getting to the place where the companies that have the
most money are standing to get the benefit of collaborations and partnerships.

540
00:46:12,297 --> 00:46:22,604
And the companies who don't have the resources to protect their technology uh have no
ability to fight back against the it the incoming attacks that are going to be there.

541
00:46:22,604 --> 00:46:32,917
So right before I joined this, I was reading this article on 404 Media about something
called Malice, M-A-L-U-S.

542
00:46:32,997 --> 00:46:37,358
And it is basically an LLM-based clean room.

543
00:46:37,358 --> 00:46:46,881
And they wrote it as a proof of concept of like, hey, is there an LGPL library that you
really want to use in your product, but don't want to be tainted by the LGPL?

544
00:46:46,881 --> 00:46:49,944
Well, feed it into Malice and Malice the one.

545
00:46:49,944 --> 00:47:02,022
Half of malice will completely deconstruct it into a spec, and then the other half of
malice will completely rewrite it from spec in what was, you know, a 1970s kind of clean

546
00:47:02,022 --> 00:47:08,286
room way of doing, of where they were reversing original IBM BIOS stuff.

547
00:47:08,286 --> 00:47:10,036
And so this isn't it.

548
00:47:10,036 --> 00:47:15,768
We're researchers who trying to make a point, created a company to create a product to

549
00:47:15,768 --> 00:47:16,658
Do this.

550
00:47:16,658 --> 00:47:23,020
And so, right, like I think there is a lot of a lot of the motes are gonna go away very
quickly.

551
00:47:23,020 --> 00:47:31,002
I was discussing because I was irritated with AWS because they were blocking me from, you
know, spinning up some resources for weird fraud reasons.

552
00:47:31,002 --> 00:47:42,905
And I was like, I bet you in the time it will take AWS's decimated support teams to
respond to my ticket, I could probably have ported this entire thing to GCP and started

553
00:47:42,905 --> 00:47:44,466
running it there instead.

554
00:47:44,466 --> 00:47:45,118
And so

555
00:47:45,118 --> 00:47:55,785
Even companies like AWS, which have strong data gravity and generally, I'm still an AWS
fan, but like it was like, I could do this, right?

556
00:47:55,785 --> 00:47:57,766
Even their mo is getting smaller.

557
00:47:57,766 --> 00:48:09,792
If I can just pivot my architecture from AWS to Google, then the only thing that's keeping
me in AWS is my pricing agreements and data gravity.

558
00:48:09,792 --> 00:48:18,789
And eventually, right, the the fact that AWS has fallen down on customer obsession and
everything else may be a win for uh the other providers.

559
00:48:18,789 --> 00:48:30,122
I think it'll be interesting to see here in Europe where, you know, the idea of sovereign
cloud and not being beholden to tech giants that are beholden to potentially adversarial

560
00:48:30,122 --> 00:48:35,616
uh, you know, government agencies, how easy is it to pivot to a local provider?

561
00:48:35,616 --> 00:48:40,589
Most of the local providers are effectively VPS as a service.

562
00:48:40,589 --> 00:48:49,935
You know, you you get the basic EC2 or you know, VMs, maybe a managed database, you know,
maybe some object storage.

563
00:48:49,935 --> 00:48:56,318
There's a couple that are like getting into the ideas of serverless, but their idea of
serverless is yeah, we'll run Kubernetes for you.

564
00:48:56,360 --> 00:48:58,512
There's just so there's just so much here.

565
00:48:58,512 --> 00:49:09,222
I do I do want to get your perspective um on just to maybe loop it back around to
something we started talking about at the beginning of the episode, the implications of

566
00:49:09,222 --> 00:49:14,250
the tech debt that we have in organizations and the LLMs generating it in a way.

567
00:49:14,250 --> 00:49:21,910
Uh I think it's inevitable that organizations will find themselves in a situation where
some of their more sophisticated

568
00:49:21,910 --> 00:49:31,608
engineers or technology members are utilizing LMs outside their control, the most notable
of being some sort of claw, open claw, micro claw, whatever they're called.

569
00:49:31,608 --> 00:49:32,984
There there's so many of them now.

570
00:49:32,984 --> 00:49:37,056
Okay, I didn't know the lobsters were proliferating, but I'll take it even one step back.

571
00:49:37,056 --> 00:49:38,798
We don't even have to talk about open claw.

572
00:49:38,798 --> 00:49:45,992
Um, I'll give you two examples of critical engineer is the source of breach.

573
00:49:45,992 --> 00:49:50,105
The first one was LastPass back in twenty twenty two, I think.

574
00:49:50,105 --> 00:49:57,688
So very advanced threat actor, because hey, if I n want to get something that I'm gonna go
after everybody's password manager.

575
00:49:57,688 --> 00:50:07,611
compromise a Plex server in an engineer's house and then use that to network pivot to
their laptop with and then use that to get into the password manager to get the

576
00:50:07,611 --> 00:50:15,023
credentials to AWS and the decryption keys to the convault backups to be able to get at
everybody's LastPass thing.

577
00:50:15,023 --> 00:50:24,815
And so that was engineer working from home as they're likely to do and a very senior level
one, because not every engineer is going to have the decryption keys to the backups, got

578
00:50:24,815 --> 00:50:26,626
popped from their home Plex server.

579
00:50:26,626 --> 00:50:27,898
Um and it seems like

580
00:50:27,898 --> 00:50:31,740
I know of a couple of incidents that involve Plex servers, but that's the one I can talk
about.

581
00:50:31,740 --> 00:50:39,073
Um and then I think, you know, yesterday, the day before today, it was Vercel, but it was
the thing that they context.

582
00:50:39,073 --> 00:50:39,863
Thank you.

583
00:50:39,863 --> 00:50:40,354
Thank you.

584
00:50:40,354 --> 00:50:40,694
Yes.

585
00:50:40,694 --> 00:50:55,354
So an example from today is Vercel got compromised because one of their engineers was
using a product called context AI on their work machine, and they shared.

586
00:50:55,354 --> 00:50:58,136
uh OAuth token between context AI.

587
00:50:58,136 --> 00:51:04,081
Context AI got popped in a way that I don't think has been yet yet disclosed.

588
00:51:04,081 --> 00:51:13,961
But so now you've got because one engineer was using context AI and context AI got popped,
you've got companies that are now scrambling to rotate access keys because they were Bruce

589
00:51:13,961 --> 00:51:14,989
Help customers.

590
00:51:14,989 --> 00:51:21,282
Software bill of materials to figure out everything between the Linux kernel and your
application.

591
00:51:21,282 --> 00:51:26,908
I think we're gonna start needing to have like and we do it in the data privacy space if
you're gonna process personal data.

592
00:51:26,908 --> 00:51:30,071
GDPR requires you to disclose all your subprocessors.

593
00:51:30,071 --> 00:51:33,375
I think that's gonna land to whether or not you're holding data.

594
00:51:33,375 --> 00:51:42,905
You give me a list of all of your vendors, and so and then you have to give me a list of
all of their vendors on all the way down until we, you know, hit I guess everybody will be

595
00:51:42,905 --> 00:51:44,366
you using AWS.

596
00:51:44,366 --> 00:51:47,908
So I wish that I I hate S bombs.

597
00:51:47,908 --> 00:51:49,279
I think they're completely useless.

598
00:51:49,279 --> 00:51:53,021
I think there there is a value theoretically, but in practice, this is a good one.

599
00:51:53,021 --> 00:52:04,907
Uh we found out it is known that Context AI was vetted for their security posture by
Delve, which has its own scandals that have been running for the last year or so.

600
00:52:04,907 --> 00:52:07,999
And I don't want to get into it because I don't have all the details.

601
00:52:07,999 --> 00:52:13,752
But uh at this point I would be suspicious of anyone using Delve to audit them for.

602
00:52:13,752 --> 00:52:20,146
SOC to and they then passed that attestation off for to Vercel as proof.

603
00:52:20,146 --> 00:52:30,493
And what had happened within Vercell was apparently uh the engineer that started using
context AI, which was not used by Vercel, uh approved access to their Google Workspace

604
00:52:30,493 --> 00:52:37,298
account via the admin permissions through the OAuth two flow, which is just honestly a a
quite a

605
00:52:37,298 --> 00:52:42,332
challenge to deal with because you aren't logged in as a user that has limited
permissions.

606
00:52:42,332 --> 00:52:46,485
You're logged in as your identity, which often has access to do things in Google
Workspace.

607
00:52:46,485 --> 00:52:54,090
And so when a random OAuth two window pops up, because thank you Google, it's such a great
experience, I'm just going to click approve most of the time.

608
00:52:54,090 --> 00:52:59,180
And that means getting access to vital resources within your uh GCP account.

609
00:52:59,180 --> 00:53:06,103
And go find the setting in admin.google.com where you can disable and prevent your
employees from doing that.

610
00:53:06,103 --> 00:53:11,916
And if you find it, let me know because it means they finally have released the thing that
I've been asking for for years.

611
00:53:11,916 --> 00:53:22,492
You can set permissions and products that should not be OAuth too enabled, uh, that would
leak too much access to your account.

612
00:53:22,492 --> 00:53:32,818
But it the permissions in Google are so not fine-grained enough that often it means
basically just turning off access to every single like logging in with any single product

613
00:53:32,818 --> 00:53:33,459
out there.

614
00:53:33,459 --> 00:53:38,914
That's because most products don't do um like granular permissions or w there's a name for
it.

615
00:53:38,914 --> 00:53:42,698
Where you basically step up, access requests from yeah.

616
00:53:42,698 --> 00:53:43,669
Well it's it's not just that.

617
00:53:43,669 --> 00:53:51,828
It's just like the the global recommendation is like, yeah, no, just put them all on the
initial login because you may need them at some point and then y users aren't paying

618
00:53:51,828 --> 00:53:53,469
attention because it's just too much.

619
00:53:53,469 --> 00:53:55,552
This is just a disaster waiting to happen.

620
00:53:55,552 --> 00:54:01,474
I just can't imagine the number of products, third party products out there that offer
proxies to as their poor service.

621
00:54:01,474 --> 00:54:04,596
Jira for HR ta tickets.

622
00:54:04,596 --> 00:54:04,916
Yeah.

623
00:54:04,916 --> 00:54:12,860
So they want to turn on an MCP that says, hey, I want this MCP to be able to r read my
engineering JIRA tickets.

624
00:54:12,860 --> 00:54:16,052
And I point out as like, there's no fine-grained access control in this.

625
00:54:16,052 --> 00:54:22,715
If you have access to see the engineering tickets and the uh HR tickets, there's no way to
filter that.

626
00:54:22,715 --> 00:54:24,556
There is no way to say

627
00:54:24,586 --> 00:54:30,368
I am Chris and I have all of these powers, but I only want Chris's LLM to have this subset
of access.

628
00:54:30,368 --> 00:54:31,488
It's all or nothing.

629
00:54:31,488 --> 00:54:38,871
You know, the agent is Chris with all of the powers he has, even though Chris does not
want him to have those powers.

630
00:54:38,871 --> 00:54:48,364
That's the fundamental problem I have with a lot of the agentic tooling that they're
building right now, is they don't give me the ability to say, I want my agent to have

631
00:54:48,364 --> 00:54:49,474
this, but no more.

632
00:54:49,474 --> 00:54:50,015
Yeah.

633
00:54:50,015 --> 00:54:56,067
And it's it's like even if you trust the L LMs in a way and trust what they do, the
problem is twofold, right?

634
00:54:56,067 --> 00:55:06,811
I mean, obviously there's fundamental issues with just what will access, but then that as
a tool having its own exposure surface and vulnerability attack surface that can be

635
00:55:06,811 --> 00:55:10,465
compromised and and expose those credentials that it then has.

636
00:55:10,465 --> 00:55:12,995
So I I think this is just turtles all the way down.

637
00:55:12,995 --> 00:55:13,462
Uh

638
00:55:13,462 --> 00:55:14,604
I I would agree.

639
00:55:14,604 --> 00:55:23,040
And then you you brought up OpenClaw and one of the biggest concerns I have with OpenClaw
is actually the supply chain around the OpenClaw community.

640
00:55:23,320 --> 00:55:26,986
how many of the things that are out there are tend to be malicious.

641
00:55:27,020 --> 00:55:37,475
I think I think the the one thing I'll say there is you can't just give it its own
identity because eventually its identities will contain all of your data because you're

642
00:55:37,475 --> 00:55:41,334
only using it for everything and there's not a good way to sequester access.

643
00:55:41,334 --> 00:55:42,684
I don't know about that.

644
00:55:42,684 --> 00:55:54,178
If you actually like if you look at executives who have executive assistants, they can
delegate their calendars, they can give access to certain emails, right?

645
00:55:54,178 --> 00:56:01,890
You if you actually think of your open claw as an executive assistant, that is another
completely separate identity.

646
00:56:01,890 --> 00:56:05,411
That's the way to look at an open claw tool like that, right?

647
00:56:05,411 --> 00:56:10,242
You know, it's like, I want you to propose trades, but you're not going to have access to
my password vault.

648
00:56:10,242 --> 00:56:11,783
Be able to log in and make it.

649
00:56:11,783 --> 00:56:18,586
Um, you can update my calendar, but you're updating your my calendar as Chris's open claw,
not as Chris.

650
00:56:18,586 --> 00:56:21,017
There are ways to do that.

651
00:56:21,017 --> 00:56:27,810
Um, but fundamentally, it's still open claw lacks the three things that an executive
assistant has, right?

652
00:56:27,810 --> 00:56:30,711
Conscience, they know what's right and wrong.

653
00:56:30,711 --> 00:56:36,914
Consequence, they know what's gonna happen to them if they do steal from the executive.

654
00:56:36,914 --> 00:56:37,730
And and

655
00:56:37,730 --> 00:56:42,883
common sense because the only common sense the L L Ms have is what they've been trained
on, which is the internet.

656
00:56:42,883 --> 00:56:45,930
So we can say that their common sense is probably nil.

657
00:56:46,200 --> 00:56:52,116
I think that'd probably a a good point to um switch over to picks for for the episode.

658
00:56:52,116 --> 00:56:54,336
So Chris, what did you bring for the audience today?

659
00:56:54,336 --> 00:57:06,652
So I think what I would suggest say is, and this actually the consequence, conscience and
common sense line actually came from a a newsletter and a podcast that I really, really

660
00:57:06,652 --> 00:57:07,372
like.

661
00:57:07,372 --> 00:57:08,993
it's called Risky Business.

662
00:57:08,993 --> 00:57:18,758
And three times a week I get a summary of all kind of things that are happening in the
security space, whether it's vulnerabilities.

663
00:57:18,816 --> 00:57:25,226
Influence operations, what nation state actors are up to, what criminal cyber syndicates
are up to.

664
00:57:25,226 --> 00:57:28,552
Um, and then s you know, a little bit of industry news.

665
00:57:28,552 --> 00:57:38,400
It it is the one email newsletter that I get on a three times a week basis that I'm
guaranteed to read, you know, in the morning while sipping my coffee.

666
00:57:38,400 --> 00:57:40,311
I I actually I do I read it as well.

667
00:57:40,311 --> 00:57:41,352
It's absolutely fantastic.

668
00:57:41,352 --> 00:57:48,436
My favorite part is when when it's there, there's usually a section dedicated to three
reasons to be happy this week.

669
00:57:48,436 --> 00:57:49,670
And that's the third reason.

670
00:57:49,670 --> 00:57:50,448
That's like yes.

671
00:57:50,448 --> 00:57:51,188
Yeah.

672
00:57:51,188 --> 00:58:00,637
that's like the w one thing where it's like there's so much negativity uh in in and around
the world and specifically in the security space that I appreciate, you know, someone

673
00:58:00,637 --> 00:58:03,646
taking the time to be like, you know, here's why we can actually be happy.

674
00:58:03,646 --> 00:58:04,320
Yeah.

675
00:58:04,320 --> 00:58:06,391
My pick is gonna be non-technical.

676
00:58:06,391 --> 00:58:14,117
I have taken the opportunity, uh which is likely a mistake, to rewatch Rick and Morty,
because it's been suggested to me so many times.

677
00:58:14,117 --> 00:58:19,560
And I've gotta say, some parts have not aged well for me that I'm almost embarrassed
recommending it.

678
00:58:19,560 --> 00:58:23,183
But I will say seasons two and three are absolutely fantastic.

679
00:58:23,183 --> 00:58:27,155
And if you like any sort of science fiction fantasy stuff, it is absolutely great.

680
00:58:27,155 --> 00:58:28,530
I I I love it.

681
00:58:28,530 --> 00:58:35,315
I would often leverage Rick and Morty memes in presentations when I was working at the
company that that produced Rick and Morty.

682
00:58:35,315 --> 00:58:45,332
Um but I tried to watch it and there were a few episodes that were good and then there
were a few episodes that it was just like this is the same bad joke over and over again.

683
00:58:45,334 --> 00:58:47,681
Yeah, yeah, that's that's accurate.

684
00:58:47,681 --> 00:58:52,304
And I'm I'm sure I'm gonna get some hate for saying this, but uh Morty is my favorite
character.

685
00:58:52,304 --> 00:58:54,008
And I'll say the smartest too.

686
00:58:54,008 --> 00:58:55,935
Definitely the smartest, yeah.

687
00:58:56,184 --> 00:58:58,872
So thank you, Chris, for joining us for it for this episode.

688
00:58:58,872 --> 00:59:00,120
I have thoroughly enjoyed it.

689
00:59:00,120 --> 00:59:03,792
Cool, thank you for having me and yeah, can't wait to uh catch

690
00:59:04,743 --> 00:59:08,085
and thanks to the audience for coming back for another episode of Adventures at DevOps.

691
00:59:08,085 --> 00:59:10,381
Hopefully we'll see everyone back again next.