1
00:00:07,832 --> 00:00:17,000
Welcome back to Adventures in DevOps, our guest, previous senior leadership at Citrex,
where they focus on access security, managing conversational AI products, led

2
00:00:17,000 --> 00:00:24,417
orchestration platforms at Redwood Software, and now the CTO at Sauce Labs, where the
their team is validating the the efficacy of written software tests.

3
00:00:24,417 --> 00:00:26,222
Welcome to the show, Anuk Tripati.

4
00:00:26,222 --> 00:00:26,882
Thank you, Warren.

5
00:00:26,882 --> 00:00:28,403
Thank you for having me out here.

6
00:00:28,403 --> 00:00:33,046
So many times they will go and dig in saying, yeah, we do have a test case and did fail.

7
00:00:33,046 --> 00:00:38,149
But since our test cases are not reliable, we don't really give much weightage to it.

8
00:00:38,149 --> 00:00:46,904
Andy dug into it and he came back and said, you know, we do have a test case and it was
failing, but we never bothered to look into it because it used to occasionally, all our

9
00:00:46,904 --> 00:00:48,615
tests occasionally fail.

10
00:00:49,155 --> 00:00:52,117
So I said, my God, let's do this.

11
00:00:52,117 --> 00:00:53,952
Next three sprints.

12
00:00:53,952 --> 00:00:55,774
We are not going to take anything else.

13
00:00:55,774 --> 00:00:57,686
We are not fixing any customer issues.

14
00:00:57,686 --> 00:00:59,428
We are not writing new code.

15
00:00:59,428 --> 00:01:02,771
I just want everybody to make sure they fix all the test cases.

16
00:01:02,771 --> 00:01:07,375
And fixing test cases actually doesn't always mean you're fixing the test script as such.

17
00:01:07,496 --> 00:01:09,448
Sometimes it's more about fixing the environment.

18
00:01:09,448 --> 00:01:13,592
Sometimes in going and fixing the code, changing the behavior of the code.

19
00:01:13,592 --> 00:01:16,924
How do you actually make sure you have the the right tests up front?

20
00:01:16,924 --> 00:01:23,107
I I think every single failed RCA uh root cause analysis session always ends the same way.

21
00:01:23,107 --> 00:01:25,882
We just needed to have one more test.

22
00:01:25,882 --> 00:01:30,398
Uh of course, the trick is actually having that test before the incident.

23
00:01:30,398 --> 00:01:30,848
occurs.

24
00:01:30,848 --> 00:01:40,364
So I think one of the challenges is how do you make sure that the validations that you're
writing and putting in place are actually matching what will prevent future problems and

25
00:01:40,364 --> 00:01:43,702
not just handling a arbitrary edge cases.

26
00:01:43,702 --> 00:01:44,593
It's a challenge.

27
00:01:44,593 --> 00:01:48,336
I don't think it's a fully solved problem to be be frank about this, right?

28
00:01:48,336 --> 00:01:56,743
But there are multiple ways to look at people have gone about hey I'm going to do my
instrumentation of the code and make sure how many lines of code have I covered in the in

29
00:01:56,743 --> 00:02:04,929
the test process right instrument like okay I'll instrument the code and then I will run
my test and at the end of it it will give me a code coverage report and we went and

30
00:02:04,929 --> 00:02:10,904
acquired a company in my last job and when we went to acquire them we asked them hey are
you agile?

31
00:02:11,064 --> 00:02:11,465
yeah.

32
00:02:11,465 --> 00:02:12,285
You use JIRA?

33
00:02:12,285 --> 00:02:13,294
Yes we use JIRA.

34
00:02:13,294 --> 00:02:14,054
Okay.

35
00:02:14,455 --> 00:02:14,855
All right.

36
00:02:14,855 --> 00:02:18,657
So the point is don't stop at those questions.

37
00:02:18,657 --> 00:02:21,999
said, okay, so how long is your sprint cycle?

38
00:02:21,999 --> 00:02:23,687
Oh, one releases sprint cycle.

39
00:02:23,687 --> 00:02:24,541
I said, nice.

40
00:02:24,541 --> 00:02:27,662
That means maybe they're doing release every week, every other week.

41
00:02:27,662 --> 00:02:30,474
But I asked, how long is your release cycle?

42
00:02:30,474 --> 00:02:31,984
we release once a year.

43
00:02:32,145 --> 00:02:36,427
So once a year, it's an agile process.

44
00:02:36,427 --> 00:02:39,729
One sprint is one year long, right?

45
00:02:39,729 --> 00:02:41,172
And the patterns out there.

46
00:02:41,172 --> 00:02:42,843
Same thing is on the testing, right?

47
00:02:42,843 --> 00:02:44,503
Testing becomes an afterthought, right?

48
00:02:44,503 --> 00:02:50,896
And I had a product where everybody was testing and releasing to the field, the debug
version, right?

49
00:02:50,896 --> 00:02:52,647
This is a Windows-based product.

50
00:02:52,647 --> 00:02:55,168
They're compiling a debug version of it and sending it.

51
00:02:55,168 --> 00:02:55,808
Why?

52
00:02:55,808 --> 00:03:01,624
Because the moment they change the compiler to say release, would start crashing left and
right.

53
00:03:01,624 --> 00:03:06,901
How do you fix this fundamentally so that no one ever thinks that this is the the right
solution?

54
00:03:06,901 --> 00:03:17,246
And I think that the just the steps removed to have both the mindset and the orchestration
and a platform or a system set up in a way to encourage the right practice here.

55
00:03:17,246 --> 00:03:18,818
So I I think just as an example.

56
00:03:18,818 --> 00:03:19,118
Right.

57
00:03:19,118 --> 00:03:23,372
You may have a end-to-end test that calls a third party service that you don't control.

58
00:03:23,372 --> 00:03:26,165
Maybe it's your your billing platform or something like that.

59
00:03:26,165 --> 00:03:28,407
And there's some sort of non-production environment there.

60
00:03:28,407 --> 00:03:29,508
And it does some processing.

61
00:03:29,508 --> 00:03:30,709
And then you wait for the result.

62
00:03:30,709 --> 00:03:33,092
And it maybe it's an async process.

63
00:03:33,092 --> 00:03:39,157
And so it comes back with some sort of URL or some sort of wait timer or some sort of GUID
that represents that process.

64
00:03:39,157 --> 00:03:43,041
And you could pull it over and over again and say, hey, you know, where is this at?

65
00:03:43,041 --> 00:03:43,592
Is it done?

66
00:03:43,592 --> 00:03:44,428
et cetera.

67
00:03:44,428 --> 00:03:50,422
But if it always takes about five seconds, is it it's so much easier to write the line of
code that says just wait five seconds?

68
00:03:50,422 --> 00:03:51,032
Or you know what?

69
00:03:51,032 --> 00:03:51,823
Let's be safe.

70
00:03:51,823 --> 00:03:53,084
Let's just wait ten seconds.

71
00:03:53,084 --> 00:03:53,734
Exactly.

72
00:03:53,734 --> 00:03:55,305
you know, in case something happens.

73
00:03:55,535 --> 00:03:58,517
and then with that, we'll just check the result.

74
00:03:58,517 --> 00:03:59,127
And then it works.

75
00:03:59,127 --> 00:04:00,889
And it works almost every single time.

76
00:04:00,889 --> 00:04:05,871
And of course, everyone knows, well, the right thing to do is, well, your test is dumb.

77
00:04:06,152 --> 00:04:09,484
separate out the like a harness for that third party system.

78
00:04:09,484 --> 00:04:14,449
Have your own sort of mock of how that works because you don't need to test the
integration of that realistically in this way.

79
00:04:14,449 --> 00:04:16,021
That's not a time-based thing.

80
00:04:16,021 --> 00:04:21,754
You can have expected return results and then just directly validate it against uh your
harness.

81
00:04:21,754 --> 00:04:24,340
Uh this happens a lot for like physical equipment, right?

82
00:04:24,340 --> 00:04:30,817
If you have like a a printer or a you know a saw or something that's mechanically
controlled, a CNC machine, et cetera.

83
00:04:30,817 --> 00:04:32,298
You don't need to

84
00:04:32,320 --> 00:04:35,852
attach your your tests, your integration test, your end-to-end test to the physical
machine.

85
00:04:35,852 --> 00:04:39,815
You can make a mock of the interface and have it work like the a actual thing works.

86
00:04:39,815 --> 00:04:47,921
So you don't actually need to cut or or run an expensive or dangerous piece of equipment
every time you want to r validate your your platform or your integration.

87
00:04:47,921 --> 00:04:54,566
I mean, of course you can have the the long-term integration test or end-to-end test to
actually do it at some point, but for most of the stuff you don't need that to happen.

88
00:04:54,566 --> 00:04:57,157
But to build that out is not free.

89
00:04:57,157 --> 00:05:01,260
A lot of companies that offer computer controlled products

90
00:05:01,260 --> 00:05:11,553
don't necessarily offer uh a virtual version of it, a digital clone, to run in in your
testing suite, uh, which has real time responses to it.

91
00:05:11,553 --> 00:05:13,073
And so we end up with this problem.

92
00:05:13,073 --> 00:05:16,844
And I think we see it in even huge platforms, even ones that are totally software based.

93
00:05:16,844 --> 00:05:25,547
Like you would think that email providers and companies that have digital messaging
products offer a way to support item potency.

94
00:05:25,547 --> 00:05:29,198
Like you send a message and you can guarantee that message, you know, was delivered or
not.

95
00:05:29,198 --> 00:05:30,562
And they don't offer a way.

96
00:05:30,562 --> 00:05:34,029
to ensure that you don't deliver a message twice.

97
00:05:34,029 --> 00:05:42,706
So sometimes you I feel like you're forced into this because of controls that are just not
in your sphere of ability to really affect.

98
00:05:42,706 --> 00:05:43,276
So

99
00:05:43,276 --> 00:05:46,309
and very few people build the product for testing.

100
00:05:46,309 --> 00:05:57,541
But what happens is, and this is where if you, and one of the orgs that was working, this
was at Citrix actually, what we did was uh we said the developers will do the testing, but

101
00:05:57,541 --> 00:05:59,953
very few organizations actually think that way, right?

102
00:05:59,953 --> 00:06:01,165
They're always in the rushed way.

103
00:06:01,165 --> 00:06:03,186
Let me build something and get it out.

104
00:06:03,426 --> 00:06:03,856
for sure.

105
00:06:03,856 --> 00:06:14,193
I I think this is a very nuanced and mature perspective where if you know your software is
gonna have to be tested, that you look at code and maybe today you'll have uh aesthetic

106
00:06:14,193 --> 00:06:15,814
opinions about how it was written.

107
00:06:15,814 --> 00:06:16,874
I don't like the way this looks.

108
00:06:16,874 --> 00:06:25,099
I don't like how this class is named or how these methods are formed, but or how it's
architected, how it com communicates with this third party component.

109
00:06:25,099 --> 00:06:30,296
But if you say, Okay, but we have to test this, then you're like, Well, okay, now I can be
specific.

110
00:06:30,296 --> 00:06:40,004
The way this is coupled together makes it very difficult to inject in a uh impl a fake
implementation of this third-party thing so that we can actually write a test that

111
00:06:40,004 --> 00:06:41,876
validates the code that we've written.

112
00:06:41,876 --> 00:06:50,764
And I find that those are the weird edge cases where it does force you to do more software
development in order to even have the test in the first place.

113
00:06:50,764 --> 00:06:55,688
The counter-argument goes: tests have no upfront value for customers, right?

114
00:06:55,688 --> 00:06:58,144
No one of your no customer of your software cares.

115
00:06:58,144 --> 00:06:59,225
if your tests pass.

116
00:06:59,225 --> 00:07:00,616
They care that your software works.

117
00:07:00,616 --> 00:07:02,298
And I think this is a fundamental distinction.

118
00:07:02,298 --> 00:07:09,965
So if you spend more time architecting your software in order to write tests, you're
actually spending more time to deliver the same amount of value.

119
00:07:09,965 --> 00:07:11,584
How do you argue against that?

120
00:07:11,584 --> 00:07:24,104
It's, you know, this is a similar question that our go-to market leader asked me the other
day, says, Anu, how do you define a testing budget for a product?

121
00:07:24,505 --> 00:07:34,893
That's a very interesting problem, testing budget is like you kind of going and saying,
you know, I screwed up and I want money to fix that in some sense.

122
00:07:34,893 --> 00:07:38,256
This is like I'm buying insurance, not even insurance actually is worse than that.

123
00:07:38,256 --> 00:07:40,918
uh

124
00:07:40,918 --> 00:07:47,042
I think the testing budgets are typically getting defined more about what actually happens
in the field, right?

125
00:07:47,042 --> 00:07:55,707
People will start with a certain place where most of the companies, okay, I don't have
testers in my company, we don't test, we just build and ship it to the customers.

126
00:07:55,707 --> 00:08:00,770
Our unit test is so good, our interface testing is so good that we can ship it, right?

127
00:08:00,770 --> 00:08:08,294
Then they'll go through this process and over time they'll find some big customer issues,
or big customers really, to a point where either they are threatened to leave or they have

128
00:08:08,294 --> 00:08:08,866
left.

129
00:08:08,866 --> 00:08:17,451
and then you'll bring in a testing team and then you'll over-rotate on the other side and
then they will talk about developer to test ratio, my developer test ratio is two is to

130
00:08:17,451 --> 00:08:21,373
one, three is to one, four is to one, eight is to one kind of a thing, right?

131
00:08:21,474 --> 00:08:25,556
And the thing is, it's more about the maturity curve of the problem.

132
00:08:25,556 --> 00:08:33,060
But again, the problem, I can understand the practical problem as you rightly put it, like
why do I invest in building the test cases upfront?

133
00:08:33,060 --> 00:08:34,102
But the thing is,

134
00:08:34,102 --> 00:08:41,994
If you do it, you will be successful in the long run, but the question is always about,
let's worry about this quarter, let's have this fiscal year, we are not thinking about

135
00:08:41,994 --> 00:08:43,275
five years from now, right?

136
00:08:43,275 --> 00:08:46,645
So very few ah people think that way.

137
00:08:46,645 --> 00:08:50,607
But that reminds me actually of something that was very interesting.

138
00:08:50,607 --> 00:08:54,428
This is, was to year 2000.

139
00:08:54,428 --> 00:08:58,549
So voice over IP was a big thing and we were building an IPPBX at 3Com.

140
00:08:58,549 --> 00:09:03,860
ah 3Com is a big company, a competitor of Cisco for.

141
00:09:04,000 --> 00:09:07,438
and networking and enterprise telecommunication space.

142
00:09:07,438 --> 00:09:13,550
So this PBX says for people who don't know like phones on their desk, a lot of times you
don't have them anymore.

143
00:09:13,550 --> 00:09:15,374
What are you talking about?

144
00:09:17,219 --> 00:09:28,247
So we had to test this product and one of our software developers, came and he said, no,
I've added a hidden telnet interface to it because the testing would be, the testers will

145
00:09:28,247 --> 00:09:37,694
pick up the phone, dial numbers, do this transfer, call hold, call record, all those
buttons, right, call forward, hand group, do not disturb, that kind of a thing.

146
00:09:37,694 --> 00:09:43,694
So he put in a telnet interface where you could, through that interface, you could, as if
I have pressed the button nine.

147
00:09:43,694 --> 00:09:46,354
as if I press the hold button kind of a thing.

148
00:09:46,354 --> 00:09:48,654
Then he showed me and he says, you know Anoop, see this?

149
00:09:48,654 --> 00:09:53,574
I'm sitting and all these five phones are on this desk and randomly they are ringing each
other.

150
00:09:53,574 --> 00:09:56,274
One is answering, there's a recording happening.

151
00:09:56,314 --> 00:10:04,614
So that's where it was both the physical machine as well as the software running on it was
automated because I forced the developers to test it out there.

152
00:10:04,614 --> 00:10:07,694
And that, so this thing works for a while, right?

153
00:10:07,694 --> 00:10:13,274
So make sure you invest time in asking your developers to test the product early on in
this.

154
00:10:13,410 --> 00:10:17,414
The earlier you ask them to do it, you will make a more testable product.

155
00:10:17,612 --> 00:10:24,665
Well, you know, so that's the thing, is like I always responded back instead, you know,
when someone asked like how much budget should we allocate for testing, it's like how much

156
00:10:24,665 --> 00:10:28,086
budget should we allocate for having customers trust our brand?

157
00:10:28,086 --> 00:10:38,089
And what is the what is the you know, provide me a a quantitative metric that represents
our brand trust without using something ridiculous like uh NPS score.

158
00:10:38,922 --> 00:10:39,736
Uh

159
00:10:39,736 --> 00:10:41,928
And like that's an impossible problem to answer.

160
00:10:41,928 --> 00:10:47,792
And so like if someone doesn't if someone's challenging you on how much budget to allocate
for software development, it goes the other way.

161
00:10:47,792 --> 00:10:53,098
Like realistically, as you said, look at your product and how do you want your customers
to feel about it?

162
00:10:53,098 --> 00:11:02,596
And I think that can answer a lot of things because I get a lot of experienced senior
engineering leaders, you know, staff plus principal engineers, directors of technology,

163
00:11:02,596 --> 00:11:06,680
uh, that will often say, Yeah, we know we need this particular budget to ensure

164
00:11:06,680 --> 00:11:08,281
That we have this much reliability.

165
00:11:08,281 --> 00:11:12,195
And my question is always how much reliability do your customers expect?

166
00:11:12,195 --> 00:11:13,446
And they can't answer the question.

167
00:11:13,446 --> 00:11:16,779
It's like, what if it's down 90% of the time?

168
00:11:16,779 --> 00:11:18,070
Will they will they care?

169
00:11:18,070 --> 00:11:20,021
Like, yes, they will care so much.

170
00:11:20,021 --> 00:11:22,163
Will they care enough to switch to a competitor?

171
00:11:22,163 --> 00:11:23,677
No, they probably won't do that.

172
00:11:23,677 --> 00:11:28,809
Matter of fact, we've been down, you know, every single week for the whole year and no
one's left.

173
00:11:28,809 --> 00:11:31,231
I'm like, why are you investing in reliability then?

174
00:11:31,231 --> 00:11:33,472
Why who cares if your stuff is broken?

175
00:11:33,853 --> 00:11:35,294
And they don't have an answer.

176
00:11:35,330 --> 00:11:43,154
And I think this is where it really comes back is you can't be making technology decisions
in a vacuum without understanding the impact to customers here.

177
00:11:43,154 --> 00:11:50,619
If you want to have long-term value that you can, I hate to say, extract from your
customers, you have to understand what long-term means for them.

178
00:11:50,619 --> 00:12:00,764
If customers will be with you five years, 10 years, 50 years, potentially longer, then the
long-term implication of the usage of your software is important.

179
00:12:00,764 --> 00:12:02,505
And so it should be easy to justify.

180
00:12:02,505 --> 00:12:04,386
Well, we'll make a change today.

181
00:12:04,386 --> 00:12:13,088
And we may not find out for five years down the road whether or not there's a problem and
who at that point is gonna remember what the features that were supposed to work or or do

182
00:12:13,088 --> 00:12:13,678
at that point.

183
00:12:13,678 --> 00:12:17,339
And so you really do need to focus on testing in that regard.

184
00:12:17,339 --> 00:12:20,730
And the only way you can get that done is if you do it, you know, up front.

185
00:12:20,730 --> 00:12:27,632
We took a step historically in my experiences to push testing all the way to the engineer
that was doing software development.

186
00:12:27,632 --> 00:12:31,493
So if if you do TDD, even without pair programming, you write the test first.

187
00:12:31,493 --> 00:12:33,754
So the developer starts by writing the test.

188
00:12:33,754 --> 00:12:34,378
Now

189
00:12:34,378 --> 00:12:42,392
I can understand why that's uh unpopular opinion because who wants to write the test when
I can just write the code and someone else will write throw it over the wall, someone else

190
00:12:42,392 --> 00:12:43,877
will write the test and test it for me.

191
00:12:43,877 --> 00:12:44,888
Sounds fantastic.

192
00:12:44,888 --> 00:12:45,949
I love this job.

193
00:12:45,949 --> 00:12:47,530
Let me keep doing that.

194
00:12:47,810 --> 00:12:56,353
yeah, so at Citrix when we started to put this thing about the developers and peer
programming and TD model, right?

195
00:12:56,353 --> 00:13:05,835
So lot of opposition from the same things like, ah that means that you're wasting my time
writing test cases while I could have written a new feature and all those things.

196
00:13:05,835 --> 00:13:08,316
So I started, I did a bugathon, right?

197
00:13:08,476 --> 00:13:15,662
And uh I said this week, sorry hackathon, was not a bugathon, I've done bugathons in
different times, but.

198
00:13:15,662 --> 00:13:18,482
This was a hackathon which was only to write test cases.

199
00:13:18,482 --> 00:13:20,842
I said, who writes the most number of test cases?

200
00:13:20,842 --> 00:13:25,962
And it was all open to both the test automation team and the software development team.

201
00:13:25,962 --> 00:13:27,342
They'll win a prize.

202
00:13:27,842 --> 00:13:38,002
And what happened is like at the end of it, the smartest in software development in the
team, he wrote the most test cases and he won the prize.

203
00:13:38,002 --> 00:13:45,958
So after that, became like nobody had an excuse to say, no, I don't want to write test
cases because if the best engineer in the team could

204
00:13:45,996 --> 00:13:48,788
was be able to adopt and write test cases also.

205
00:13:48,788 --> 00:13:55,682
That whole concept, the taboo that they had about, write software code for production, I
don't write for test, kind of disappeared in the organization.

206
00:13:55,682 --> 00:14:01,135
It was so fun to watch that whole thing melt away and now everybody is always writing test
cases.

207
00:14:01,416 --> 00:14:10,762
But the hidden cost of quality that you touched upon, sometimes we don't realize that so
many other places where you have hidden cost of quality, like,

208
00:14:10,762 --> 00:14:14,065
If your product is, obviously it comes from testing and building it, right?

209
00:14:14,065 --> 00:14:17,567
also building has to be done right, the architecture and everything.

210
00:14:17,968 --> 00:14:26,975
But we don't realize that sometimes because of that our sales guys actually are spending
more time with the customers on the stability issue, they're on those calls trying to

211
00:14:26,975 --> 00:14:31,639
pacify the customer, or at least as opposed to going and selling more.

212
00:14:31,639 --> 00:14:40,236
So it's not just about losing revenue, your salespeople, they're off the street now,
internally focused on existing customer as opposed to going and hunting or farming.

213
00:14:40,236 --> 00:14:41,599
a different account out there, right?

214
00:14:41,599 --> 00:14:49,006
So there is another place where you really come in, but it's very hard to really put a
number and say, where should the cost of quality go?

215
00:14:49,006 --> 00:14:57,840
So I think we have to be very careful about the incentives we put in place for uh
engineers or even technology leaders or non-technical people, it doesn't really matter,

216
00:14:57,840 --> 00:15:00,171
that we put in place to drive particular behaviors.

217
00:15:00,171 --> 00:15:03,712
Cause we had something very similar at a previous organization that I was in.

218
00:15:03,712 --> 00:15:12,416
And the result was a bunch of engineers r went around automating the creation of tests for
every single code that we line of code that we had.

219
00:15:12,416 --> 00:15:16,940
So for every project that there was, there was a now a new unit test project that popped
up.

220
00:15:16,940 --> 00:15:19,412
with a method for every single function.

221
00:15:19,412 --> 00:15:22,874
And inside was a single statement, assert, pass.

222
00:15:22,914 --> 00:15:28,318
And and so uh like a hundred percent unit test coverage by method count.

223
00:15:28,318 --> 00:15:32,611
I mean obviously not by line or branch count, but every single function passed.

224
00:15:32,611 --> 00:15:34,643
And he definitely won a prize.

225
00:15:34,643 --> 00:15:37,685
He didn't get the benefit of the consequences of that action though.

226
00:15:37,685 --> 00:15:42,608
He got to r leave that company and go somewhere else and make the same mistakes elsewhere.

227
00:15:42,608 --> 00:15:45,260
But we lost a critical signal there because actually

228
00:15:45,260 --> 00:15:53,312
The lack of the unit test project and and unit test allowed us to identify what parts of
the code were quote unquote legacy and untested.

229
00:15:53,312 --> 00:16:02,222
And by doing this, he won the prize of the most tests written, but he c had created a
detriment to the organization because now we lost this signal that we had in place.

230
00:16:02,222 --> 00:16:04,395
And I just I just find this particularly interesting.

231
00:16:04,395 --> 00:16:10,707
So I I think there has to be a lot of goodwill behind why you're doing a particular thing
and people have to be aligned.

232
00:16:10,707 --> 00:16:13,878
And I I'd be curious to, you know, go back in time now and see.

233
00:16:13,878 --> 00:16:21,018
what sort of tests were actually written by the engineer to to to win the prize because
you know it's like if you know a prize you're gonna win.

234
00:16:21,018 --> 00:16:22,843
If it's a competition.

235
00:16:22,843 --> 00:16:27,786
You know, it's just maybe winning the competition is is the value and not necessarily
having the product

236
00:16:27,796 --> 00:16:29,767
Yeah, you're right about it, right?

237
00:16:29,767 --> 00:16:36,399
And I think you just do some of these things because it's just fun and it creates a new
energy in the organization as well, right?

238
00:16:36,399 --> 00:16:38,209
And you get some positive surprises.

239
00:16:38,209 --> 00:16:47,132
almost every time I've run this kind of whether it's a Bhagathan, whether it's an
Akathana, whether it's an Automaton, I think we call it Automaton or something like that.

240
00:16:47,252 --> 00:16:54,220
But metrics are one thing that I always say that, don't chase metrics because

241
00:16:54,220 --> 00:16:56,701
then what happens is that's what gets optimized for.

242
00:16:56,701 --> 00:16:59,792
The entire organization gets optimized around that metric.

243
00:16:59,792 --> 00:17:05,993
In fact, right now, for example, we are doing more and more using AI in our software
development process.

244
00:17:05,993 --> 00:17:07,874
And everybody keeps asking, what's the metric?

245
00:17:07,874 --> 00:17:09,234
And what metric are you looking at?

246
00:17:09,234 --> 00:17:10,875
I said, don't worry about metric.

247
00:17:10,875 --> 00:17:14,836
Because the moment I tell you which metric I'm looking at, I'm sure that will look the
best.

248
00:17:14,836 --> 00:17:23,678
So I look for multiple signals and I'll ask the questions, but I'm not telling you which
particular metric am I more interested in on this particular front, right?

249
00:17:23,810 --> 00:17:31,557
because gains can be different and people say, are writing five hours, we are, our
engineers are producing 548 % more code now.

250
00:17:31,557 --> 00:17:34,450
uh I'll hear those kind of numbers from my peers as well.

251
00:17:34,450 --> 00:17:42,958
But for me, those are very meaningless numbers because I've been there, I have written the
code, know, I still write code, so I know what you can do to come up with those kind of

252
00:17:42,958 --> 00:17:49,492
numbers because the moment you put a metric in front of you, as you said, right, start
writing all the asserts with true and true and enjoy.

253
00:17:50,170 --> 00:17:58,396
We so the question that I had actually put out and the metric that we try to use if we're
gonna, you know, be specific here was roadmap acceleration.

254
00:17:58,396 --> 00:18:01,078
And I feel like anything else is sort of a mistake.

255
00:18:01,078 --> 00:18:10,094
Things that and why I mean that is things that you wanted to achieve for the benefit of
your organization, company, product, customer, you thought was gonna take a very long time

256
00:18:10,094 --> 00:18:16,448
to do based off of whatever sort of back of the napkin calculation you had done and that
you put in your OKRs or KPIs.

257
00:18:16,628 --> 00:18:19,224
And it turned out after implementing

258
00:18:19,224 --> 00:18:27,216
process changes, adding LLMs or agentics software development strategies, it got moved up
as far as when you were you considered it done.

259
00:18:27,216 --> 00:18:30,097
Not you did less, you did different, but it got done.

260
00:18:30,337 --> 00:18:34,848
And I haven't had any leaders come back and tell me for a long time that this was actually
happening.

261
00:18:34,848 --> 00:18:38,339
Yeah, we were ever all the engineers feel way more productive.

262
00:18:38,339 --> 00:18:43,821
It aligned with the Dora 2025 report that came out last year, which we did a whole episode
on.

263
00:18:43,821 --> 00:18:47,616
And the the TLDR for that is lots of organizations were saying,

264
00:18:47,616 --> 00:18:52,569
Everyone felt they were being more productive, felt that they were delivering more or
faster.

265
00:18:52,569 --> 00:18:56,341
But the only metrics that were actually measurable were quality metrics.

266
00:18:56,341 --> 00:19:00,053
And they said that quality went down, negative impact to your business.

267
00:19:00,053 --> 00:19:04,576
So everyone felt like the right thing was happening, but the outcome was quite the
opposite.

268
00:19:04,576 --> 00:19:14,412
And uh a couple episodes ago, we had the ex-CTO of Grafana on, who actually was the first
person who said, we are seeing roadmap acceleration that we would not have seen

269
00:19:14,412 --> 00:19:15,288
previously.

270
00:19:15,288 --> 00:19:16,588
But I think it's a rarity still.

271
00:19:16,588 --> 00:19:25,512
And I think it's going to com continue to be a rarity, especially when the mindset that we
have doesn't necessarily align with the the long-term vision that we're putting out there.

272
00:19:25,512 --> 00:19:30,123
As you put out, like if you want teams to write tests, you need to encourage tests being
written.

273
00:19:30,123 --> 00:19:33,424
You have to encourage the long term mindset in order to make that happen.

274
00:19:33,424 --> 00:19:38,222
As long as you're focused on the next quarter or whatever the current metric is, you're
going to have a problem.

275
00:19:38,222 --> 00:19:43,562
No, I think I'm sure the Grafana team is seeing a roadmap acceleration.

276
00:19:43,862 --> 00:19:51,982
But one thing you have to always feel done and look into is if they made that as a goal to
measure, because roadmap acceleration come in many ways and I've seen it, right?

277
00:19:51,982 --> 00:19:56,642
Okay, I'm going to declare GA much sooner than like three months before the product is
ready, right?

278
00:19:56,642 --> 00:19:58,482
All right, my feature is GA now.

279
00:19:58,482 --> 00:20:01,302
That's what you're measuring for roadmap acceleration, right?

280
00:20:01,302 --> 00:20:07,082
So that's why what I say is do not make those as public statements as to what you're
measuring.

281
00:20:07,254 --> 00:20:11,796
And the other thing I also keep saying is there is something called output gain versus
outcome gain, right?

282
00:20:11,796 --> 00:20:12,896
And that's where it is.

283
00:20:12,896 --> 00:20:14,576
What is the outcome that you're looking for?

284
00:20:14,576 --> 00:20:19,918
There's definitely an output increment uh that is happening with AI for sure.

285
00:20:19,918 --> 00:20:22,569
But is there an outcome gain that is happening as well?

286
00:20:22,569 --> 00:20:24,520
And outcome gain at the end is there.

287
00:20:24,520 --> 00:20:26,520
You know the top numbers that companies work on, right?

288
00:20:26,520 --> 00:20:27,431
Top line, bottom line.

289
00:20:27,431 --> 00:20:30,122
Is there some impact happening out there or not, right?

290
00:20:30,122 --> 00:20:32,002
Even if I accelerate the roadmap,

291
00:20:32,002 --> 00:20:33,373
Did I build a meaningful product?

292
00:20:33,373 --> 00:20:35,754
Did I create a feature bloat in my product even, right?

293
00:20:35,754 --> 00:20:38,384
As opposed to really doing anything good out there.

294
00:20:38,485 --> 00:20:41,666
So it's a multifaceted problem.

295
00:20:41,666 --> 00:20:45,248
I think what we have to do is make sure you measure properly.

296
00:20:45,248 --> 00:20:50,050
But quality going down, we are seeing this problem and hearing about this problem
everywhere.

297
00:20:50,050 --> 00:20:52,871
In fact, it's not just on the software development.

298
00:20:52,891 --> 00:20:58,573
It's even the content in the software design, the architecture documents that are being
created.

299
00:20:58,573 --> 00:21:00,128
We have a quality problem.

300
00:21:00,128 --> 00:21:04,702
Engineers are coming and complaining already saying, I know there's so much of content to
review now.

301
00:21:04,702 --> 00:21:10,007
Because before, you know how the software engineers are like, they hate writing
documentation.

302
00:21:10,007 --> 00:21:11,578
It's like pulling teeth out of them, right?

303
00:21:11,578 --> 00:21:12,459
In most cases, right?

304
00:21:12,459 --> 00:21:15,011
Very few of them love writing documentation.

305
00:21:15,012 --> 00:21:19,456
Now with the use of LLMs, everybody's writing a 20 page document on even a small thing,
right?

306
00:21:19,456 --> 00:21:21,728
And now, and some of it could be wrong.

307
00:21:21,728 --> 00:21:24,991
Now the reviewers have to go and read each and every line out of it, right?

308
00:21:24,991 --> 00:21:26,902
And so there is a waste of...

309
00:21:27,023 --> 00:21:35,173
So there is a quality problem in almost every phase of software development life cycle,
whether it's architecture documents, whether it's design documents, whether it's actual

310
00:21:35,173 --> 00:21:35,874
code.

311
00:21:35,874 --> 00:21:39,558
And we are as an industry facing a testing problem because of that.

312
00:21:39,702 --> 00:21:42,375
Why why do you say we're facing a testing problem?

313
00:21:42,690 --> 00:21:50,657
Because the amount of content that's coming in, reliable validation of that content, it's
still a human problem to be taken care of.

314
00:21:50,657 --> 00:22:00,524
Till that, till we go ahead and solve that testing problem in a way where you can rely lot
more on non-humans to solve it.

315
00:22:00,524 --> 00:22:05,377
we have actually created this problem much worse than it was before with the drug.

316
00:22:05,377 --> 00:22:08,659
So that's what is the one of the side effects, right?

317
00:22:08,659 --> 00:22:12,312
Like how you take drugs and they will solve a problem and you get some side effects.

318
00:22:12,312 --> 00:22:14,553
is one of effects.

319
00:22:14,553 --> 00:22:18,906
Use of AI in the entire process is we have created a testing problem.

320
00:22:19,360 --> 00:22:29,173
I I agree, you know, and I I like the analogy because I I think uh what we've applied to
the LLM provider companies have created products that are addictive, similar to TikTok and

321
00:22:29,173 --> 00:22:32,754
literally I I think a the analogy of drugs is very, very spot on.

322
00:22:33,355 --> 00:22:43,438
There is there is this this quote that uh I often feel like I'm repeating, at least back
to myself, which is uh if you have i if you think that technology will solve your human

323
00:22:43,438 --> 00:22:44,418
problem.

324
00:22:44,624 --> 00:22:47,616
you don't understand technology and you don't understand your problem.

325
00:22:47,616 --> 00:22:57,833
Uh and I I think that for sure is what a lot of people are stumbling on today, which uh,
you know, historically there was no no reason to understand your problem uh fully, right?

326
00:22:57,833 --> 00:23:05,538
You you see something that looks like it's I hate this analogy, looks like a nail and you
have a hammer and and you so and and everything looks like a nail.

327
00:23:05,538 --> 00:23:07,909
So why not why not go to town?

328
00:23:07,909 --> 00:23:09,430
But I I I think

329
00:23:09,462 --> 00:23:19,540
Realistically, from a metrics standpoint, we still need to find an opportunity to share
something public that allows us to align on whether or not an experiment that we're making

330
00:23:19,540 --> 00:23:28,357
or a change we're making to our organizational structure or software development process,
SDLC or technology or et cetera, actually turns out better in the end.

331
00:23:28,357 --> 00:23:35,984
And I know people hate measuring stuff and I know people actually hate running an
experiment, but some some organizations still want to do that.

332
00:23:35,984 --> 00:23:39,160
And I think at this point it's still very difficult to identify.

333
00:23:39,160 --> 00:23:40,260
how you measure it.

334
00:23:40,260 --> 00:23:46,797
I I think you as you rightly pointed out, roadmap acceleration is even difficult to
identify if that's the right metric.

335
00:23:46,797 --> 00:23:54,214
Obviously, if you create a a metric that you make as your target, any metric that becomes
a a target ceases to be a good metric, right?

336
00:23:54,214 --> 00:24:00,846
And I I think we have that wisdom, but it still becomes incredibly problematic to deal
with in practice.

337
00:24:00,846 --> 00:24:09,546
I think what you have to do is constantly keep working on, I would say, a combination of
signals to look for things.

338
00:24:09,746 --> 00:24:11,626
For example, I'll give you a quick.

339
00:24:11,626 --> 00:24:17,306
We had a code in a certain area of our actual deployment process.

340
00:24:17,306 --> 00:24:21,386
The SRE team had some code that was a tech debt for them.

341
00:24:21,386 --> 00:24:24,826
And it was so complex, they were too scared to touch it.

342
00:24:24,826 --> 00:24:28,046
And what I've been told in my company is that I've been only here for a year.

343
00:24:28,046 --> 00:24:32,986
They've been thinking about changing that code for the last five years, but they're so
scared to touch it, right?

344
00:24:33,146 --> 00:24:42,446
And then in one of the, so I do a hackathon, like we do four hackathon weeks, but two of
them are actually purely innovation-focused hackathons twice a year.

345
00:24:42,446 --> 00:24:48,726
So in that one week, they went ahead as a team and solved it using the help of AI.

346
00:24:48,726 --> 00:24:53,846
Now, how do you put that into your, was it faster to release a feature?

347
00:24:53,846 --> 00:24:58,106
Like the kind of tech that you are able to offload, you don't need those

348
00:24:58,946 --> 00:25:09,566
I would say, uh unique arm, ingenious to be so much expert in that dated way of doing
stuff that can only support that system, that problem, that the risk that you reduce on

349
00:25:09,566 --> 00:25:11,018
the organization, right?

350
00:25:11,018 --> 00:25:13,290
So there is benefits where you can gain.

351
00:25:13,290 --> 00:25:17,944
So that's why you have to look at multiple signals when you're looking for what am I
gaining out of using this?

352
00:25:17,944 --> 00:25:20,184
And then one signal alone won't help you.

353
00:25:20,184 --> 00:25:25,606
So I mean, obviously we could talk all day about the AI stuff, but I'm much more
interested from the testing standpoint.

354
00:25:25,606 --> 00:25:33,119
Uh I think one of the challenges that we brought up here is realistically having the right
tests is important and the incentives to have the right tests.

355
00:25:33,119 --> 00:25:37,791
But at the end of the day, how do you know that you have the right tests and what do those
actually look like?

356
00:25:37,791 --> 00:25:41,332
Because I know if anyone's using an LLM or even if they're not,

357
00:25:41,332 --> 00:25:46,326
We we still see lots of tests being generated that are very technical in nature, right?

358
00:25:46,326 --> 00:25:50,949
I called this function and then this function, and these were the parameters passed from
one to the other.

359
00:25:50,949 --> 00:25:56,133
They're very internal behavior focused rather than input and output focused.

360
00:25:56,133 --> 00:26:00,506
And even when they are input and output focused, they don't feel like they're very product
driven.

361
00:26:00,506 --> 00:26:05,190
They're very, yeah, this method that adds two numbers correctly added the two numbers
together.

362
00:26:05,190 --> 00:26:08,034
But we don't say why it's a sum function or

363
00:26:08,034 --> 00:26:09,618
how that function is supposed to operate.

364
00:26:09,618 --> 00:26:11,322
And I I I've definitely struggled.

365
00:26:11,322 --> 00:26:19,338
How do you convey that wisdom in writing the tests or making sure that future tests are
written with this mindset in place?

366
00:26:19,722 --> 00:26:25,124
Well, that's an interesting problem and the right problem to focus on.

367
00:26:25,505 --> 00:26:29,586
You have to start looking at actually your customer intent.

368
00:26:29,586 --> 00:26:37,750
And sometimes it's easy because if you have an existing customer base, lot of companies,
actually most of the companies that we work with, they have over time built an existing

369
00:26:37,750 --> 00:26:38,260
customer base.

370
00:26:38,260 --> 00:26:46,798
So knowing your customer behavior, being able to analyze that and be able to create intent
out of it is where like...

371
00:26:46,798 --> 00:26:52,818
going to the original topic of RPA, it can help you as long as you can think of it as how
is the customer using my product?

372
00:26:52,818 --> 00:26:57,738
Can I go ahead and record that behavior and mimic my test environments to be closer to it?

373
00:26:57,738 --> 00:27:01,978
Because at the end of the day, let's say I have 1,000 lines of code, just for example,
right?

374
00:27:02,058 --> 00:27:08,678
And do I need to test all 1,000 lines of code if the path that all my customers use only
exercises 250 lines of the code?

375
00:27:08,678 --> 00:27:09,198
No, right?

376
00:27:09,198 --> 00:27:12,158
I need to make sure that 250 lines of the code is well tested.

377
00:27:12,158 --> 00:27:14,700
It's great to test those others, maybe someday.

378
00:27:14,700 --> 00:27:15,791
But what's my priority?

379
00:27:15,791 --> 00:27:18,836
Because at the end, always run the prioritization of stuff.

380
00:27:18,836 --> 00:27:23,704
So knowing how your customer uses the product and getting that closed loop is very
important, right?

381
00:27:23,704 --> 00:27:26,695
You it's interesting you bring up the the benefits here.

382
00:27:26,695 --> 00:27:29,677
And I I think this is maybe something that we didn't really get into.

383
00:27:29,677 --> 00:27:38,120
Historically, I think incorrectly a lot of organizations have applied the idea of the
quality, the like the iron triangle to software development.

384
00:27:38,120 --> 00:27:49,646
And I I have this I have this link to this article, which I think is really good and jumps
into, basically explaining how quality is not a trade-off with speed or or cost or

385
00:27:49,646 --> 00:27:50,406
features.

386
00:27:50,406 --> 00:27:51,206
It's

387
00:27:51,252 --> 00:27:55,175
fundamentally you always get quality and quality enables these things.

388
00:27:55,175 --> 00:27:58,687
If you are tests, validate your code to do the right thing.

389
00:27:58,687 --> 00:28:04,712
When you make new code, you can ensure you aren't spending time on rework because there's
a bug that was introduced.

390
00:28:04,712 --> 00:28:13,438
If you have tests, you we talked about this earlier, you're writing your architecture of
your services or your platform in a way which is more maintainable.

391
00:28:13,438 --> 00:28:16,590
So future tests can be written or new code can be written easier.

392
00:28:16,590 --> 00:28:19,167
And I I find that these are just a

393
00:28:19,167 --> 00:28:31,845
often misunderstood and organizations focus on, well maybe we don't need to test this as a
extra effort with low ROI rather than understanding the implications of doing the test in

394
00:28:31,845 --> 00:28:32,926
the first place.

395
00:28:33,726 --> 00:28:35,287
It makes a lot of sense.

396
00:28:35,287 --> 00:28:43,482
In fact, people don't realize that the hidden cost of quality burns up time all the way up
to the top management in organizations.

397
00:28:43,482 --> 00:28:45,813
And the cost keeps going up and up, right?

398
00:28:45,813 --> 00:28:57,860
And so over time, if you don't focus on quality, you're really, really causing so much of,
uh I would say, wastage in the organization of time and talent and opportunity cost.

399
00:28:57,860 --> 00:28:58,630
You name it, right?

400
00:28:58,630 --> 00:28:59,280
So many ways.

401
00:28:59,280 --> 00:29:02,868
So quality is spending anything on quality.

402
00:29:02,868 --> 00:29:04,839
is never a wasted money, right?

403
00:29:04,839 --> 00:29:11,761
It's just that you have to find the right balance to do it in an organization, but do not
make it as an afterthought.

404
00:29:11,761 --> 00:29:16,652
Make sure it's thought about like quality first kind of a thing that has been said before,
right?

405
00:29:16,652 --> 00:29:18,923
Quality first has to be thought about it.

406
00:29:18,923 --> 00:29:20,473
But think about that, right?

407
00:29:20,473 --> 00:29:24,905
I tell people like when you write software, you use software in your day-to-day world too.

408
00:29:24,905 --> 00:29:30,082
Would you expect that when I go into an ATM machine and swipe my card, occasionally it
doesn't work?

409
00:29:30,082 --> 00:29:32,027
because somebody wrote a poor software out there?

410
00:29:32,027 --> 00:29:32,478
No, right?

411
00:29:32,478 --> 00:29:39,266
So just make sure even you write software, you also think that there are users out there
who can be so frustrated or who could have problems with it, right?

412
00:29:39,266 --> 00:29:46,888
I mean, having traveled to many countries, I definitely now do expect that there are some
number of ATM failures when I'm out in the field for real.

413
00:29:47,788 --> 00:29:55,359
when I started my software engineering career uh a long time ago, I as soon as I started,
I got so disillusioned and defeated.

414
00:29:55,359 --> 00:30:04,303
Like, wow, if software for this organization that is doing something critical is this bad,
I don't trust it running anywhere, like doing anything.

415
00:30:04,303 --> 00:30:09,290
And it always amazes me when someone's like, yeah, we managed to put software on

416
00:30:09,290 --> 00:30:17,806
I I think sa like vehicles that go into space like satellites are that have to live with
the same code for a very long time that n never changes like the fundamental layers.

417
00:30:17,806 --> 00:30:24,191
Although a lot of organizations have figured out ways to to roll firmware updates and
whatnot, but have to live without changes.

418
00:30:24,191 --> 00:30:32,757
Uh when you deploy a device or like a consumer electronic that has a microcontroller that
does something very simple, or like your oven or stove if there's a counter or some visual

419
00:30:32,757 --> 00:30:33,687
display in there.

420
00:30:33,687 --> 00:30:35,449
No one comes and services that code.

421
00:30:35,449 --> 00:30:37,568
It it literally works indefinitely.

422
00:30:37,568 --> 00:30:43,120
ATMs are a little bit weird because often there's over-the-wire updates and whatnot for
security protocols.

423
00:30:43,120 --> 00:30:49,952
But if we're really talking about these, I really hope my oven and dishwasher are not
attached to the internet at this point.

424
00:30:49,952 --> 00:30:53,133
I wouldn't know if they're using some sort, maybe they have a 5G chip.

425
00:30:53,133 --> 00:30:55,763
I maybe I should be I should be checking that.

426
00:30:55,824 --> 00:31:03,596
of course I'm one of those people that's paranoid and my television's not connected to the
internet, but uh I I I do I do worry about that in that particular world.

427
00:31:03,596 --> 00:31:08,256
And with that, now is potentially a good time to switch over to picks for the episode.

428
00:31:08,256 --> 00:31:10,803
So Anu, what did you what did you bring for us today?

429
00:31:10,803 --> 00:31:11,182
I

430
00:31:11,182 --> 00:31:15,363
At this moment, I'm really enjoying, like, it's just book that's been out for a while.

431
00:31:15,363 --> 00:31:17,544
I think people might have known about it.

432
00:31:17,544 --> 00:31:19,223
The book is called Genesis, right?

433
00:31:19,223 --> 00:31:20,945
It's about AI, right?

434
00:31:20,945 --> 00:31:25,676
And what's happening with artificial intelligence, hope, and the human spirit.

435
00:31:25,676 --> 00:31:29,427
It's, one of the authors is Dr.

436
00:31:29,427 --> 00:31:32,628
Henrique Singer, Eric Smid, and Craig Mundy.

437
00:31:32,628 --> 00:31:38,269
so that's very interesting because it talks about which are directions can AI take us.

438
00:31:38,269 --> 00:31:41,080
Like, it doesn't go into the technical aspects of it.

439
00:31:41,080 --> 00:31:43,151
But it's more about the social impact.

440
00:31:43,151 --> 00:31:53,527
one of them is very interesting thing that says that if AI becomes too smart, for example,
I don't give it the entire book, but if AI becomes too smart, then maybe we'll start to

441
00:31:53,527 --> 00:31:58,940
think of it as if it's like the old religion times where people just say, it's like
gospel.

442
00:31:58,940 --> 00:32:01,602
If AI said it, must be true kind of a thing.

443
00:32:01,602 --> 00:32:03,893
So as humans, maybe can we go in that direction?

444
00:32:03,893 --> 00:32:05,750
So that was a possibility that brought up.

445
00:32:05,750 --> 00:32:06,745
So I'm just enjoying it.

446
00:32:06,745 --> 00:32:08,430
I really enjoyed it.

447
00:32:09,750 --> 00:32:19,708
You know, on on my perspective on that is just so ridiculous because I I go back and I see
all these like movies, television shows, et cetera, that always got like software hacking

448
00:32:19,708 --> 00:32:20,148
wrong.

449
00:32:20,148 --> 00:32:25,582
Like they don't understand how uh malicious threat actors actually work and practice
today.

450
00:32:25,582 --> 00:32:28,624
And they're just, you know, someone typing on the keyboard and mashing things.

451
00:32:28,845 --> 00:32:35,109
And now I feel like it ha like the world has started to like life imitating art.

452
00:32:35,109 --> 00:32:36,034
Now

453
00:32:36,034 --> 00:32:43,257
You want to hack something, you literally are like typing, you know, mad in a prompt to an
LLM to say like how to hack the thing without actually doing the work.

454
00:32:43,257 --> 00:32:50,080
And I think now is more reminiscent of what the movies had uh sort of promised us is the
experience.

455
00:32:50,080 --> 00:32:57,103
So I think we can just look to past popular culture of what happens in society when when
the AI takes over.

456
00:32:57,103 --> 00:33:01,005
Obviously, there's the futuristic Dune perspective, right?

457
00:33:01,005 --> 00:33:02,860
Where all AI will have to be banned.

458
00:33:02,860 --> 00:33:03,780
Uh

459
00:33:03,800 --> 00:33:04,910
For one reason or another.

460
00:33:04,910 --> 00:33:14,940
Uh or the lifeless automatons will take it as a goal to eliminate all all humans to solve
a particular prompt gone wrong.

461
00:33:14,940 --> 00:33:18,463
No intelligence required, just a particular aspect.

462
00:33:18,463 --> 00:33:19,834
So I don't know.

463
00:33:19,834 --> 00:33:24,568
I think most of them are not uh not fun end of world scenarios.

464
00:33:24,568 --> 00:33:31,064
I think there was like a a book or an article about like the twelve different ways that
humanity will end as a result of

465
00:33:31,064 --> 00:33:35,167
No, this was the one that I showed was more on the negative side, but the book has the
positive side.

466
00:33:35,167 --> 00:33:36,968
So it talks about all the possibilities.

467
00:33:36,968 --> 00:33:38,559
So that's why I love that book.

468
00:33:38,559 --> 00:33:42,482
So you can take a look at that book called Genesis.

469
00:33:42,482 --> 00:33:45,844
But yeah, I'm a believer.

470
00:33:45,844 --> 00:33:51,599
think as human race, with every new tech that has come, we have always progressed forward.

471
00:33:51,599 --> 00:33:55,631
And I think we'll harness the power of this to our benefits always.

472
00:33:55,631 --> 00:33:58,613
So I'm a very optimist on that front.

473
00:33:58,613 --> 00:33:59,666
oh

474
00:33:59,666 --> 00:34:01,107
we could we could fight about this.

475
00:34:01,107 --> 00:34:06,351
There is like, you know, what is progress and who who gets the benefit of those technology
changes.

476
00:34:06,351 --> 00:34:19,480
I think if that's an interesting topic, there is a book uh called Sapiens, which talks
about yeah, which which actually talks about um self-domestication of humans and what what

477
00:34:19,480 --> 00:34:21,561
what ha what has actually improved in humanity.

478
00:34:21,561 --> 00:34:26,124
And I say improved in quotes, because like that's up for you know you to decide.

479
00:34:26,324 --> 00:34:27,425
but that's not gonna be my pick.

480
00:34:27,425 --> 00:34:28,918
My pick is actually

481
00:34:28,918 --> 00:34:36,480
This paper that was that just came out recently, I say recently, but it may be a couple
years old at old at this point, about, well, here's the question.

482
00:34:36,480 --> 00:34:41,581
Is quantum mechanics required to dis to explain human consciousness?

483
00:34:41,581 --> 00:34:46,943
And my I'm I'm uh totally on board with the quantum theory of consciousness for a long
time now.

484
00:34:46,943 --> 00:34:49,483
I think it can only be explained by quantum physics.

485
00:34:49,774 --> 00:34:55,625
I don't think classical physics is enough to explain like why we can have thoughts or why
there's a consciousness or subconsciousness.

486
00:34:55,765 --> 00:34:58,666
And the paper basically, I'd say

487
00:34:58,666 --> 00:35:00,027
almost proved it.

488
00:35:00,027 --> 00:35:10,192
Uh it identified that there's a way to prevent the anesthetic, I think it's called
isoflorane, basically from impacting our conscious brain.

489
00:35:10,192 --> 00:35:18,287
So even if you take the anesthetic, what they give you in a hospital if you're going to go
under general anesthesia and it causes you to lose consciousness and unconsciousness.

490
00:35:18,287 --> 00:35:27,981
Basically the theory was does this block anesthesia isofluorane from being absorbed and
block neuroreceptors in your brain from completing thoughts?

491
00:35:28,002 --> 00:35:28,588
And

492
00:35:28,588 --> 00:35:39,240
The the test that was released or validated in the in the article basically said, uh, no,
we have a way of basically preventing the anesthetic from impacting humans, or in the in

493
00:35:39,240 --> 00:35:41,653
the experiment was mice, of course, or rats.

494
00:35:41,653 --> 00:35:43,338
You know, it's it's always rats, right?

495
00:35:43,338 --> 00:35:44,304
Mm-hmm.

496
00:35:44,304 --> 00:35:46,745
But it doesn't actually block the uptake.

497
00:35:46,745 --> 00:35:48,606
So what's going on there?

498
00:35:48,606 --> 00:35:58,510
And they theorize that it has to do with impacting the quantum states of something called
um, I think it's the MT.

499
00:35:58,632 --> 00:36:00,213
I I I don't remember.

500
00:36:00,743 --> 00:36:02,064
anyway, really interesting.

501
00:36:02,064 --> 00:36:11,348
It's very short, but uh I I I think this is why I can totally say LMs are not thinking
today and will not have a consciousness in the way we've built them, because they only

502
00:36:11,348 --> 00:36:19,622
emulate it and they aren't capable in the technology that we're utilizing to support them
to actual I had to make this about AI, didn't I?

503
00:36:19,732 --> 00:36:24,012
in the way to support them the technology that we're utilizing doesn't support quantum
processes.

504
00:36:24,012 --> 00:36:25,863
We don't give it the capability to do that.

505
00:36:25,863 --> 00:36:31,218
All computational natures that we have currently for computers, et cetera, are based on
classical mechanics.

506
00:36:31,218 --> 00:36:34,890
I mean, of course, outside of quantum mechanics or quantum computers.

507
00:36:35,054 --> 00:36:43,666
That's interesting and I'll just pick on the word currently because it's not but not yet
is the answer or is it not the answer.

508
00:36:43,990 --> 00:36:44,911
Yeah.

509
00:36:44,922 --> 00:36:46,915
well I mean can we really know anything?

510
00:36:46,915 --> 00:36:56,253
I think that's the the th the the theory of knowledge is is another uh particular uh
definitely outside of the realm of relevance for this podcast, but interesting topic to

511
00:36:56,253 --> 00:36:57,016
discuss.

512
00:36:57,016 --> 00:36:57,867
Yes.

513
00:36:58,191 --> 00:37:00,983
Well, it was a pleasure talking to you, Auran, today.

514
00:37:00,983 --> 00:37:03,309
And thank you so much for having me here.

515
00:37:03,372 --> 00:37:05,167
Of of course I've enjoyed the conversation.

516
00:37:05,167 --> 00:37:12,444
Thank Anub for for joining us and thanks for all the listeners for tuning in to this
week's episode and I hope to see everyone back next.

