
Screaming in the Cloud
ยทE659
Avery Pennarun on Tailscale's Evolution: From Mesh VPN to AI Security Gateway
Episode Transcript
1
00:00:00,150 --> 00:00:03,270
What's very strange about tail scale, uh, and very strange in
2
00:00:03,270 --> 00:00:06,540
the security world in general, is that when you use tail scale to
3
00:00:06,540 --> 00:00:10,020
solve that problem, you accidentally make your system more secure.
4
00:00:10,590 --> 00:00:13,830
And also the easiest thing for all of your engineers and people inside
5
00:00:13,830 --> 00:00:18,270
your company to do becomes the secure thing instead of the insecure thing.
6
00:00:24,555 --> 00:00:26,355
Welcome to Screaming in the Cloud.
7
00:00:26,535 --> 00:00:27,645
I'm Cory Quinn.
8
00:00:27,675 --> 00:00:31,485
It's been a while since I've had Avery Pener run on the show.
9
00:00:31,935 --> 00:00:32,894
Thank you for joining me.
10
00:00:32,894 --> 00:00:36,315
Again, you are still the co-founder and CEO of tail scale,
11
00:00:36,525 --> 00:00:40,575
which at this point is getting pretty darn close to.
12
00:00:40,605 --> 00:00:42,585
You've heard of this company when I bring
13
00:00:42,585 --> 00:00:44,775
it up in almost every conversation I'm in.
14
00:00:45,050 --> 00:00:46,040
That is pretty exciting.
15
00:00:46,040 --> 00:00:46,400
I think.
16
00:00:46,400 --> 00:00:49,370
I can't remember when I was on your show last time, but it was at least a
17
00:00:49,370 --> 00:00:52,340
couple years ago and we've been growing really fast in the last couple years.
18
00:00:52,790 --> 00:00:54,920
This episode is sponsored in part by my day
19
00:00:54,920 --> 00:00:55,880
job Duck.
20
00:00:55,880 --> 00:00:59,090
Bill, do you have a horrifying AWS bill?
21
00:00:59,360 --> 00:01:01,250
That can mean a lot of things.
22
00:01:01,460 --> 00:01:04,490
Predicting what it's going to be, determining what it
23
00:01:04,790 --> 00:01:08,355
should be, negotiating your next long-term contract with
24
00:01:08,480 --> 00:01:12,770
AWS, or just figuring out why it increasingly resembles of.
25
00:01:12,885 --> 00:01:16,515
Phone number, but nobody seems to quite know why that is.
26
00:01:16,785 --> 00:01:20,385
To learn more, visit duck bill hq.com.
27
00:01:20,685 --> 00:01:23,535
Remember, you can't duck the duck bill.
28
00:01:23,595 --> 00:01:28,965
Bill, which my CEO reliably informs me is absolutely not our slogan.
29
00:01:29,414 --> 00:01:31,005
I'm seeing you in more and more places.
30
00:01:31,005 --> 00:01:34,095
I've been using you in my personal environment for many years now,
31
00:01:34,095 --> 00:01:36,645
and the stuff that I set up once upon a time is still working.
32
00:01:36,794 --> 00:01:38,744
You're rolling out new stuff that continues
33
00:01:38,744 --> 00:01:41,115
to add, be additive to this at at at work.
34
00:01:41,115 --> 00:01:43,845
I'm paying you now, which was a big problem I had with you
35
00:01:43,845 --> 00:01:46,815
previously of there's no good way for me to give you money.
36
00:01:46,815 --> 00:01:48,104
Could you maybe fix that?
37
00:01:48,255 --> 00:01:48,794
Good job.
38
00:01:48,824 --> 00:01:49,574
You fix that.
39
00:01:49,695 --> 00:01:53,175
So things are all up and to the right, which is kind of amazing.
40
00:01:54,000 --> 00:01:54,840
It is kind of amazing.
41
00:01:54,840 --> 00:01:56,700
It's amazing how long we can keep doing it.
42
00:01:56,760 --> 00:01:59,400
Although I've been informed that if you keep doubling revenue
43
00:01:59,400 --> 00:02:01,560
at a hundred percent year over year, then in 10 years you'll
44
00:02:01,560 --> 00:02:03,720
be a thousand times bigger, and that might not be realistic.
45
00:02:04,215 --> 00:02:04,665
But it might
46
00:02:06,045 --> 00:02:07,935
at some point you hit population limits.
47
00:02:07,995 --> 00:02:13,035
Uh, last year I gave the opening keynote at Nano 91, and the
48
00:02:13,035 --> 00:02:15,795
whole theme of what I was talking about back then was that
49
00:02:16,065 --> 00:02:21,405
there's been a rising tide in the level of what clouds could take.
50
00:02:22,140 --> 00:02:25,679
From, from folks who are working in on-prem environments,
51
00:02:25,890 --> 00:02:28,079
networking is becoming something of a lost art.
52
00:02:28,079 --> 00:02:31,679
When you find someone who works as a network engineer, they're usually
53
00:02:32,070 --> 00:02:35,579
my age and not new grads who are playing around with these things.
54
00:02:36,089 --> 00:02:38,970
Tail scale is in some ways an answer to some of this, where
55
00:02:38,970 --> 00:02:42,329
you, you're taking things away from the traditional network, uh,
56
00:02:42,359 --> 00:02:45,989
switch and router world and into just make a big flat network.
57
00:02:45,989 --> 00:02:50,010
And then we'll wind up handling this through policy files for access control.
58
00:02:50,580 --> 00:02:54,180
Even recently this year, you folks wound up redoing your policy, uh,
59
00:02:54,185 --> 00:02:58,020
policy, uh, format as far as making it a lot easier to do grants with
60
00:02:58,020 --> 00:03:01,620
access grants, as well as now creating a visual builder, which I've not
61
00:03:01,620 --> 00:03:04,560
yet played with because I haven't found a way to make it work in Vim yet.
62
00:03:05,040 --> 00:03:07,650
One of my fixations as a CEO is I insist that
63
00:03:07,650 --> 00:03:09,675
every change to the policy file get run by me.
64
00:03:09,795 --> 00:03:12,480
Almost nothing else in the whole company runs by me, but I'm like, no,
65
00:03:12,480 --> 00:03:14,700
if you're changing the policy, policy and tax, I wanna see it first.
66
00:03:15,015 --> 00:03:18,555
So we went through a lot of iterations of the ACL grant syntax, uh,
67
00:03:18,585 --> 00:03:21,465
before we finalized it, and I'm really excited about what we came up with.
68
00:03:21,465 --> 00:03:24,465
I realize it's a little strange to be really excited about a
69
00:03:24,465 --> 00:03:27,615
file syntax, but I actually am really excited and I think in a
70
00:03:27,615 --> 00:03:31,815
little understood feature of ACL grants is it's really extensible.
71
00:03:31,845 --> 00:03:36,285
Like you can grant stuff to applications that are provided by people
72
00:03:36,285 --> 00:03:38,590
that are not tail scale, that are running on your tail scale network.
73
00:03:39,045 --> 00:03:42,735
And when you connect to that application over tail scale, it has visibility.
74
00:03:43,245 --> 00:03:46,695
Into the grants that you gave it based on your groups, the tags, the
75
00:03:46,700 --> 00:03:48,674
blah, blah, blah, whatever's going on in your tail, tail net routing.
76
00:03:48,674 --> 00:03:52,005
And it doesn't have to know about, it doesn't have to know what group you're in.
77
00:03:52,005 --> 00:03:54,975
It doesn't have to have its own business logic about what group you're in.
78
00:03:54,975 --> 00:03:57,825
It can just say like, tail scale says this connection
79
00:03:57,825 --> 00:03:59,984
should be allowed to do this thing on this thing.
80
00:04:00,555 --> 00:04:02,415
And you can change all that in a central place.
81
00:04:02,445 --> 00:04:05,415
And so easy example is like Grafana, you can say.
82
00:04:05,490 --> 00:04:10,380
Today, everybody in the production group should have admin access to Grafana.
83
00:04:10,560 --> 00:04:12,960
So when they connect to Grafana, they get admin access.
84
00:04:12,990 --> 00:04:14,430
You don't have to set anything up in Grafana.
85
00:04:14,430 --> 00:04:16,320
If you change your mind later or you change who's in that
86
00:04:16,320 --> 00:04:20,280
group, then next time they connect to Grafana, even if it's
87
00:04:20,280 --> 00:04:22,620
like three seconds later, their permissions are gonna change.
88
00:04:23,010 --> 00:04:23,220
Right?
89
00:04:23,220 --> 00:04:25,200
And that was not possible before we had this ability
90
00:04:25,200 --> 00:04:26,670
to just sort of like pass these things through.
91
00:04:26,790 --> 00:04:29,040
And so it gives you this ability to just build on top of
92
00:04:29,040 --> 00:04:31,200
tail scale and just stop worrying about all that stuff.
93
00:04:31,920 --> 00:04:34,710
You also have a great feature where you can effectively disallow people
94
00:04:34,710 --> 00:04:37,140
from modifying things in the console without going through a whole bunch
95
00:04:37,140 --> 00:04:40,650
of very scary warnings, uh, and mandating effectively a GI ops flow, which
96
00:04:40,650 --> 00:04:44,190
is fantastic, especially combined with the fact that you have test cases
97
00:04:44,190 --> 00:04:45,660
built into your policy files.
98
00:04:46,140 --> 00:04:46,650
Exactly.
99
00:04:46,650 --> 00:04:47,460
Well, it's super fun, right?
100
00:04:47,460 --> 00:04:50,250
Because you, you just said like, you can't use our ACL editor because you like
101
00:04:50,250 --> 00:04:53,790
Vim, and like, that's not actually true because we did this another, another.
102
00:04:54,615 --> 00:04:58,485
Very nerdy thing I'm super excited about is that you can round trip the
103
00:04:58,485 --> 00:05:02,325
JSON of the policy file to the GUI editor and back with no loss of anything.
104
00:05:02,325 --> 00:05:04,305
And it's not just regular JSON, it's our special
105
00:05:04,305 --> 00:05:08,175
weird hug, JSO, that has comments and extra commas.
106
00:05:08,445 --> 00:05:09,675
Uh, which means you can actually have
107
00:05:09,675 --> 00:05:11,565
comments, your js o describing what goes on.
108
00:05:11,865 --> 00:05:14,775
And then when you go to the GUI and then back, the comments don't get lost.
109
00:05:15,015 --> 00:05:19,185
And so you, and also GI Ops can like take this text, store it in GitHub.
110
00:05:19,545 --> 00:05:20,025
Um.
111
00:05:20,090 --> 00:05:21,500
And then push it back when you're done.
112
00:05:21,530 --> 00:05:23,119
And then of course it's not very good to have the
113
00:05:23,119 --> 00:05:25,520
GUI edit it in tail scale if you're using GI Ops.
114
00:05:25,520 --> 00:05:28,520
But you can go to the gui, come up with a rule that you want,
115
00:05:28,549 --> 00:05:31,190
or an edit that you want, and it'll tell you what Text to paste
116
00:05:31,190 --> 00:05:33,830
back into your Git repository to get the results you want.
117
00:05:34,280 --> 00:05:36,650
So it's this very nice flow where like everybody
118
00:05:36,679 --> 00:05:38,390
who likes everything gets to have what they want.
119
00:05:38,724 --> 00:05:39,085
It's weird.
120
00:05:39,085 --> 00:05:40,015
It's just thinking back.
121
00:05:40,015 --> 00:05:44,094
It's been a bunch of small releases, but they add up to almost a, that
122
00:05:44,094 --> 00:05:47,305
completely different product that still does the underlying baseline thing.
123
00:05:47,305 --> 00:05:49,794
It always did, which is flattening the network to make it work
124
00:05:49,794 --> 00:05:52,195
like we all used to think networks did until we knew better.
125
00:05:52,465 --> 00:05:54,294
It's a, it's been a very.
126
00:05:54,854 --> 00:05:55,905
Fun evolution.
127
00:05:56,085 --> 00:05:59,174
I think it was last year you did that partnership with Mulva, where I think
128
00:05:59,174 --> 00:06:03,044
for five bucks a month now I can get, I get access to the Mulva VPN stuff.
129
00:06:03,224 --> 00:06:06,255
It's a couple of clicks of a house and I'm suddenly emerging from anywhere else
130
00:06:06,255 --> 00:06:10,304
I want to be, which is super handy for me and my brother who lives in Brussels.
131
00:06:10,304 --> 00:06:13,815
We are at poe, both have EU and US citizenship, so, but there's an
132
00:06:13,815 --> 00:06:16,969
awful lot of government sites that, oh, you're not physically here.
133
00:06:16,969 --> 00:06:21,015
Clearly you could never wanna access these things for no apparent reason.
134
00:06:21,525 --> 00:06:21,945
Trivial,
135
00:06:21,945 --> 00:06:22,185
easy.
136
00:06:22,185 --> 00:06:22,245
Al
137
00:06:22,905 --> 00:06:26,355
Also my bank, uh, in Canada, whenever I go traveling anywhere
138
00:06:26,355 --> 00:06:28,755
that is not Canada, they're like, oh my God, nobody outside
139
00:06:28,755 --> 00:06:30,885
Canada could possibly need to access a Canadian bank.
140
00:06:30,885 --> 00:06:31,575
And they kick me out.
141
00:06:31,995 --> 00:06:35,385
Uh, but I, you know, I could do that by, what I do is I use a, an exit
142
00:06:35,385 --> 00:06:38,625
node on my Apple TV at home and I just bounce through my Apple tv.
143
00:06:39,045 --> 00:06:41,235
Uh, but I also use Malva for experimenting and stuff.
144
00:06:41,955 --> 00:06:44,625
I did that originally and then the raspberry pie I
145
00:06:44,625 --> 00:06:46,815
sent with my brother to his place wound up dying.
146
00:06:46,815 --> 00:06:49,725
And that is not, he is a government functionary there.
147
00:06:49,725 --> 00:06:53,115
He's not really the, uh, the technical type as far as, Hey,
148
00:06:53,115 --> 00:06:55,785
now log into the Linux console and tell me what you see.
149
00:06:55,935 --> 00:06:58,545
Y no, that's why I went with the Apple tv.
150
00:06:58,725 --> 00:07:01,395
'cause they, you know, they're five times as expensive, at least
151
00:07:01,395 --> 00:07:04,185
as a raspberry pie, but they have five times at least as much
152
00:07:04,185 --> 00:07:07,185
quality control as a raspberry pie in the manufacturing process.
153
00:07:07,185 --> 00:07:07,455
So,
154
00:07:07,665 --> 00:07:10,050
and a warranty service that is comprehensible to humans.
155
00:07:10,870 --> 00:07:11,080
Yeah.
156
00:07:11,140 --> 00:07:13,390
And a gui where you can just tell your brother like, Hey, can you go to
157
00:07:13,390 --> 00:07:16,210
the app store and pig tail scale, uh, as opposed to going to the console.
158
00:07:16,570 --> 00:07:16,750
Yeah.
159
00:07:17,065 --> 00:07:21,490
I, I also like things that are, that have changed or some things have not
160
00:07:21,490 --> 00:07:24,610
changed in tail scale that are still somewhat annoying and I understand why.
161
00:07:24,790 --> 00:07:28,360
Uh, I'd love to be available to connect at two tail nets at the same time.
162
00:07:28,510 --> 00:07:31,960
Now you can be logged in and toggle between them, but yeah, a device
163
00:07:31,960 --> 00:07:34,270
that talks between two networks is generally considered a bridge
164
00:07:34,270 --> 00:07:37,480
and corporate security would like a word if you start doing that.
165
00:07:38,409 --> 00:07:41,169
There are ways now to share nodes between tail nets
166
00:07:41,169 --> 00:07:44,109
that start making that a lot more straightforward.
167
00:07:44,530 --> 00:07:47,890
I would still love, on some level, the ability to set a custom
168
00:07:47,890 --> 00:07:51,669
domain for the tail net domain that I can control the certs for.
169
00:07:51,700 --> 00:07:54,099
I, I get that that is a hard thing to do.
170
00:07:54,370 --> 00:07:57,104
I'm sure some big customer somewhere has it, but Yeah.
171
00:07:57,109 --> 00:07:57,310
Yeah.
172
00:07:57,370 --> 00:08:00,424
It's, it's surprising how it's, well, that particular feature.
173
00:08:01,140 --> 00:08:02,130
It's a little hard to do.
174
00:08:02,130 --> 00:08:05,070
I would say the difficulty of doing it is not actually the thing holding us up.
175
00:08:05,099 --> 00:08:07,500
What's the, the thing holding us up is the
176
00:08:07,500 --> 00:08:09,179
phishing potential when you start doing it.
177
00:08:09,750 --> 00:08:12,659
Uh, 'cause you combine that with tail scale funnel and people register
178
00:08:12,659 --> 00:08:15,659
some arbitrary domain that looks suspiciously like, but is not quite.
179
00:08:15,659 --> 00:08:18,929
google.com and next thing you know, you're hosting phishing sites for Google.
180
00:08:18,934 --> 00:08:19,065
Do.
181
00:08:19,670 --> 00:08:20,000
Right.
182
00:08:20,000 --> 00:08:23,180
If everything ends in blurb, butty blurb ts.net, then you don't have
183
00:08:23,180 --> 00:08:26,030
that problem and it's like remarkable how much trouble that saves us.
184
00:08:26,300 --> 00:08:28,400
So we really want to get to the custom domain thing.
185
00:08:28,400 --> 00:08:31,250
We just need to like very carefully control who gets to
186
00:08:31,250 --> 00:08:34,400
have custom domains and minimize the abuse potential.
187
00:08:34,400 --> 00:08:36,710
One way is like to attach it to not easy.
188
00:08:36,919 --> 00:08:37,515
All the people that pay you.
189
00:08:38,520 --> 00:08:40,020
That was one of the things we've been thinking.
190
00:08:40,020 --> 00:08:42,270
It's actually, it's, I mean, it's, it's a pretty good start.
191
00:08:42,270 --> 00:08:43,140
We should probably do that.
192
00:08:43,289 --> 00:08:45,540
Exchanging money for goods and services.
193
00:08:45,540 --> 00:08:46,500
That's wild.
194
00:08:46,590 --> 00:08:46,800
Yeah.
195
00:08:46,890 --> 00:08:47,280
Yeah.
196
00:08:47,370 --> 00:08:49,980
It's just the, do we need to limit it to only those people?
197
00:08:49,980 --> 00:08:51,180
It's kind of sad to have to do that.
198
00:08:51,180 --> 00:08:53,100
I wish we had a better idea, but like, you know,
199
00:08:53,160 --> 00:08:55,199
nevertheless, uh, yeah, it's definitely on the list.
200
00:08:55,199 --> 00:08:57,000
Similarly with sharing, we've been in the same
201
00:08:57,000 --> 00:08:59,370
state with node sharing since I think like 2021.
202
00:08:59,580 --> 00:09:03,150
And a bunch of internal changes have been going on, uh, architecturally
203
00:09:03,150 --> 00:09:06,000
to finally like enable way more kinds of interesting sharing.
204
00:09:06,589 --> 00:09:08,630
Um, but I really see like there's so much
205
00:09:08,630 --> 00:09:12,560
potential to newer kinds of, uh, node sharing.
206
00:09:12,589 --> 00:09:15,079
I don't think you ever want to be in two tail nets at the same time.
207
00:09:15,079 --> 00:09:17,209
I realize that everybody at first thinks you would
208
00:09:17,209 --> 00:09:19,010
want to do that 'cause it would be really tempting.
209
00:09:19,010 --> 00:09:21,859
But it is this bridge between tail nets and it like really confuses.
210
00:09:22,380 --> 00:09:24,930
Like, as an example, I would like to be in two tail nets at the same time.
211
00:09:24,959 --> 00:09:27,030
I have a personal account with my family stuff on it,
212
00:09:27,359 --> 00:09:29,310
and I have a work account with all my work stuff on it.
213
00:09:29,310 --> 00:09:32,609
And where I'm the CEO, uh, that has access to a bunch of sensitive things.
214
00:09:32,849 --> 00:09:35,550
Right now, if I'm at a computer with my, and maybe your
215
00:09:35,550 --> 00:09:37,319
children should not have access to those same things.
216
00:09:37,650 --> 00:09:37,829
Yeah.
217
00:09:37,829 --> 00:09:38,520
Maybe they shouldn't.
218
00:09:38,520 --> 00:09:38,760
Right?
219
00:09:38,760 --> 00:09:41,219
So if I have a device that my children borrow, right?
220
00:09:41,219 --> 00:09:42,219
An iPad or something like that.
221
00:09:42,530 --> 00:09:45,410
I really should not be logged into that device using my tail scale account.
222
00:09:45,530 --> 00:09:47,959
But if I'm on my corporate device, I really
223
00:09:47,959 --> 00:09:49,640
would like to have access to my private stuff.
224
00:09:49,670 --> 00:09:50,240
'cause why not?
225
00:09:50,510 --> 00:09:50,780
Right?
226
00:09:50,780 --> 00:09:53,060
But if I'm logged into both tail nets at the same time, now I'm
227
00:09:53,060 --> 00:09:55,550
inadvertently creating a bridge between my corporate account.
228
00:09:55,550 --> 00:09:57,110
So the security team should lock me out.
229
00:09:57,470 --> 00:09:59,329
Uh, and my personal account, right?
230
00:09:59,510 --> 00:10:01,520
And the security team incidentally almost locked me
231
00:10:01,550 --> 00:10:03,650
out a few days ago because I wasn't on the MDM yet.
232
00:10:03,650 --> 00:10:07,100
So I like forcibly enrolled into the MDM, which forced me to upgrade my Mac Os.
233
00:10:07,160 --> 00:10:09,230
And there's a bunch of new features in Mac OS that I was missing.
234
00:10:09,230 --> 00:10:10,100
So I guess that's good.
235
00:10:10,785 --> 00:10:12,405
And that yak is getting nicely shaved.
236
00:10:13,245 --> 00:10:13,875
Yep, exactly.
237
00:10:13,875 --> 00:10:17,985
So I, I'm pretty far down this path, but, you know, um, anyway,
238
00:10:18,135 --> 00:10:21,105
what I think people want, and what I want to give people is
239
00:10:21,105 --> 00:10:24,135
the ability to log into each device using exactly one account.
240
00:10:24,915 --> 00:10:29,925
And for you to be able to share many or all, or a good
241
00:10:29,925 --> 00:10:32,535
subset of the nodes from another account into your account.
242
00:10:32,955 --> 00:10:34,575
You are almost taking the GitHub identity model.
243
00:10:35,025 --> 00:10:35,565
Yeah, I guess so.
244
00:10:36,540 --> 00:10:38,100
Yeah, I, I have a GitHub account, but I can, I be
245
00:10:38,100 --> 00:10:39,990
outta different organizations that do different things.
246
00:10:39,990 --> 00:10:41,820
My personal account is also what I use for
247
00:10:41,820 --> 00:10:44,130
work, but you can gate access to things and
248
00:10:44,190 --> 00:10:45,450
yeah, that part makes me nervous.
249
00:10:45,450 --> 00:10:47,130
Like when I log into GitHub, I have access to
250
00:10:47,130 --> 00:10:48,780
all my corporate stuff and my personal stuff.
251
00:10:48,780 --> 00:10:50,970
So if I log into my personal GitHub account.
252
00:10:51,605 --> 00:10:54,845
When I'm not on a work computer, I'm like putting work at risk, which is scary.
253
00:10:55,055 --> 00:10:57,485
So what I think we should do is still have the two accounts, but on my
254
00:10:57,485 --> 00:11:00,335
personal devices, I log into my personal account that doesn't have access
255
00:11:00,335 --> 00:11:04,325
to my corp stuff, but I log in my work computer, I've access to all my corp
256
00:11:04,325 --> 00:11:09,155
stuff, and my corporate user also has outgoing access to my personal stuff.
257
00:11:09,215 --> 00:11:09,515
Yeah.
258
00:11:09,515 --> 00:11:11,130
For the last eight years, I haven't really had anything
259
00:11:11,135 --> 00:11:13,805
personal because my entire life has become work Right around
260
00:11:13,805 --> 00:11:16,805
the time that, uh, shit posting on social media became a job.
261
00:11:17,015 --> 00:11:17,855
Yeah, I guess that makes sense.
262
00:11:17,855 --> 00:11:20,075
So yeah, I mean, yeah, I, I'm really talking
263
00:11:20,075 --> 00:11:21,695
about the experience for other people.
264
00:11:21,755 --> 00:11:23,915
Um, but yes, I mean, I have an Apple tv.
265
00:11:23,945 --> 00:11:26,675
Does the corporation want my Apple TV on the corporate network?
266
00:11:26,675 --> 00:11:27,425
Like not really.
267
00:11:27,515 --> 00:11:29,525
So little things like that, and I think we can do it.
268
00:11:29,525 --> 00:11:31,085
We're getting very close to being able to do it.
269
00:11:31,085 --> 00:11:33,815
We just keep, like doubling in size a lot.
270
00:11:33,995 --> 00:11:36,965
Uh, and so most of the engineering that we do, uh, actually ends
271
00:11:36,965 --> 00:11:40,415
up being just like, Hey, you, you now have like a tail net with
272
00:11:40,415 --> 00:11:43,205
hundreds of thousands of nodes on it with like thousands of nodes
273
00:11:43,205 --> 00:11:46,115
churning per minute because someone is using it in a gigantic.
274
00:11:46,555 --> 00:11:47,755
CICD cluster.
275
00:11:48,175 --> 00:11:50,185
Uh, did you know that's an n squared algorithm?
276
00:11:50,275 --> 00:11:52,495
Uh, did you know that the whole system is gonna crash?
277
00:11:52,495 --> 00:11:53,005
'cause you did that?
278
00:11:53,035 --> 00:11:54,895
I'm like, oh, I didn't know that.
279
00:11:55,255 --> 00:11:56,635
Um, but then we had to fix it.
280
00:11:56,785 --> 00:11:59,064
We learn exciting things, uh, through other people's use cases.
281
00:12:00,025 --> 00:12:00,505
Exactly.
282
00:12:00,505 --> 00:12:02,814
So some of this stuff keeps getting delayed, uh, but
283
00:12:02,814 --> 00:12:04,135
it's gonna be really good when it finally comes out.
284
00:12:04,344 --> 00:12:04,495
Yeah.
285
00:12:04,495 --> 00:12:07,255
And, and you're, you have a great list of customer references that are, that
286
00:12:07,255 --> 00:12:10,765
are doing all sorts of fascinating stuff, some of whom I know reasonably well.
287
00:12:10,815 --> 00:12:14,385
And what I've, I also like the fact that there are options if
288
00:12:14,385 --> 00:12:16,815
tail skill isn't right for people, if you want one that is a
289
00:12:16,815 --> 00:12:20,235
lot more confusing, a lot less capable, and much more expensive.
290
00:12:20,235 --> 00:12:23,325
I mean, AWS has launched VPC Lattice and then they've marketed
291
00:12:23,325 --> 00:12:25,815
it so poorly that people don't know if I'm making that up or not.
292
00:12:26,145 --> 00:12:27,705
Yes, I actually had not heard of them.
293
00:12:27,765 --> 00:12:29,415
Uh, that is maybe embarrassing.
294
00:12:29,970 --> 00:12:32,250
Now, Nope, this is par for the course.
295
00:12:32,250 --> 00:12:35,880
I thought it was great when it came out and then I forgot it existed, and
296
00:12:35,880 --> 00:12:39,990
then it just goes years without being mentioned by anyone until I encounter.
297
00:12:39,990 --> 00:12:41,940
It's like, oh, right, that, that, that exists.
298
00:12:41,940 --> 00:12:42,630
That's kind of neat.
299
00:12:42,630 --> 00:12:45,990
I should look into it and every time I do, I come away with, or
300
00:12:45,990 --> 00:12:48,720
I could just use tail scale and save myself a lot of heartache.
301
00:12:48,780 --> 00:12:51,330
So I do honestly, on some level, your next Go to Market for
302
00:12:51,330 --> 00:12:53,970
enterprise, you'd just be offering people a free month of VPC latice.
303
00:12:56,714 --> 00:12:59,084
We've actually had that a few times and there's like a comparison
304
00:12:59,084 --> 00:13:02,204
and we're like, can we please be like first while you're doing the
305
00:13:02,204 --> 00:13:05,084
comparison and then you can, you know, install the other ones later.
306
00:13:05,204 --> 00:13:07,305
And they do, you know, they're done with tail scale in like 15
307
00:13:07,305 --> 00:13:09,765
minutes and then they go off and they try to install the next one.
308
00:13:10,275 --> 00:13:11,714
But it, but if they try to install the next
309
00:13:11,714 --> 00:13:13,574
one first, they might never get to tail scale.
310
00:13:13,574 --> 00:13:13,724
Right.
311
00:13:13,755 --> 00:13:14,505
'cause they don't finish.
312
00:13:15,255 --> 00:13:17,235
That's the, that's the dark secret of POCs.
313
00:13:19,035 --> 00:13:19,305
Yep.
314
00:13:19,845 --> 00:13:22,995
You've done a fair number of things that are,
315
00:13:23,055 --> 00:13:25,155
it's hard to even describe what tail scale is.
316
00:13:25,155 --> 00:13:27,045
You, you have tail drop, which is effectively
317
00:13:27,045 --> 00:13:29,295
an end-to-end, uh, file sharing option.
318
00:13:29,625 --> 00:13:32,385
Uh, it feels like you are flirting with
319
00:13:32,385 --> 00:13:35,355
becoming almost a service discovery tool.
320
00:13:35,355 --> 00:13:37,665
Uh, one of the, we have enough service niche in the world, but
321
00:13:37,665 --> 00:13:40,185
it feels like this one makes a strong contention for being one.
322
00:13:40,729 --> 00:13:44,270
Well, I have, uh, we're, we're trying out new versions of the
323
00:13:44,270 --> 00:13:46,370
mission statement 'cause previous ones were too complicated.
324
00:13:46,370 --> 00:13:50,239
I will, I will present a preliminary version, uh, that we've been trying out.
325
00:13:50,660 --> 00:13:54,170
Uh, it is a new layer three for every device everywhere.
326
00:13:55,099 --> 00:13:56,329
It's like maybe too simple.
327
00:13:56,329 --> 00:13:58,280
You have to be a network person to know even what I'm talking
328
00:13:58,280 --> 00:14:01,219
about by layer three, I tried like new internet protocol.
329
00:14:01,280 --> 00:14:04,339
Sometimes people are afraid of that 'cause it's not like IPV seven, but it
330
00:14:04,339 --> 00:14:08,180
does the job of what layer three, the internet protocol was supposed to do.
331
00:14:08,765 --> 00:14:10,505
And let me, let me try to explain what that means.
332
00:14:10,505 --> 00:14:14,585
So like way back in the day when I logged into the internet, I could connect
333
00:14:14,585 --> 00:14:18,605
to any device anywhere that was on the internet by using its IP address.
334
00:14:18,785 --> 00:14:22,265
That is, has not been the case for now decades, right?
335
00:14:22,265 --> 00:14:24,395
It's now gotten to the point where, in fact, the only
336
00:14:24,395 --> 00:14:26,460
things I can really connect to by IP address are.
337
00:14:27,390 --> 00:14:29,820
Maybe my wifi router, if I can remember what the IP
338
00:14:29,820 --> 00:14:33,000
address is, and I'm in my house, uh, or cloud providers
339
00:14:33,000 --> 00:14:35,340
who own like most of the public IP space at this point.
340
00:14:35,700 --> 00:14:36,840
And that's like kind of weird.
341
00:14:36,840 --> 00:14:38,880
That defeats a lot of the purpose of the internet.
342
00:14:38,880 --> 00:14:42,390
Another thing that happened is if you have a, even if you
343
00:14:42,390 --> 00:14:44,760
had that connectivity, imagine you had IPV six rolled out
344
00:14:44,760 --> 00:14:48,330
everywhere, which requires a bit of a big imagination.
345
00:14:48,330 --> 00:14:49,260
But let us imagine.
346
00:14:49,260 --> 00:14:50,310
IPV six was everywhere.
347
00:14:50,580 --> 00:14:53,040
If I switch to a different network, like between
348
00:14:53,040 --> 00:14:55,355
wifi and cellular, my IP address changes.
349
00:14:56,550 --> 00:14:58,290
And now the connection breaks.
350
00:14:58,740 --> 00:15:01,650
Uh, and I actually can't find that device unless I use DNS.
351
00:15:01,650 --> 00:15:04,140
Everyone's best friend, DNS, the thing that is not anywhere
352
00:15:04,140 --> 00:15:07,410
in the OSI stack, but is somehow playing some job, making
353
00:15:07,410 --> 00:15:09,720
some of the layers of the OSI stack work together, right?
354
00:15:09,720 --> 00:15:12,450
So now I'm like dynamic DNS, I'll just update it every time my
355
00:15:12,450 --> 00:15:15,660
phone jumps between wifi and cellular, like not likely, right?
356
00:15:15,660 --> 00:15:18,150
And so like the actual interneting part of
357
00:15:18,150 --> 00:15:20,400
the internet stack does not work anymore.
358
00:15:20,430 --> 00:15:21,930
It's not location independent.
359
00:15:22,220 --> 00:15:25,550
And it doesn't make everything in the world addressable to me, right?
360
00:15:25,580 --> 00:15:26,720
It's actually layer two.
361
00:15:27,380 --> 00:15:29,630
It's just a replacement for ethernet addresses
362
00:15:29,780 --> 00:15:30,920
because every time my interface changes.
363
00:15:32,084 --> 00:15:33,525
The address is a different thing.
364
00:15:33,525 --> 00:15:35,594
It might as well be an ethernet port, right?
365
00:15:35,594 --> 00:15:38,655
And it hasn't done this job that's like missing from the stack.
366
00:15:38,655 --> 00:15:41,235
And so tail scale jumps in there and it's a tunnel, but
367
00:15:41,235 --> 00:15:43,155
it's like, hey, it works the way it's supposed to work.
368
00:15:43,155 --> 00:15:44,505
Like obviously the world has changed.
369
00:15:44,505 --> 00:15:46,425
You don't want everyone in the world to be able to access
370
00:15:46,425 --> 00:15:49,074
you, but everyone I want to be able to access me gets an it.
371
00:15:49,334 --> 00:15:51,525
It know it can find out my name, and I get a fixed ip.
372
00:15:52,115 --> 00:15:53,675
And I'll make it work everywhere.
373
00:15:53,675 --> 00:15:55,415
And it doesn't change when my device moves around.
374
00:15:55,714 --> 00:15:58,115
So tail scale, all the stuff you can talk about.
375
00:15:58,385 --> 00:16:01,235
But the thing that it does is it actually pro produces
376
00:16:01,235 --> 00:16:04,175
layer three of the OOSI stack for the first time in decades.
377
00:16:04,775 --> 00:16:05,670
That's nothing short of magical.
378
00:16:05,730 --> 00:16:05,949
Uh.
379
00:16:06,495 --> 00:16:10,275
It's, it's weird in that because this gets highly technical, highly,
380
00:16:10,275 --> 00:16:14,235
quickly, and goes very deep, but it is stupid simple to get set up.
381
00:16:14,355 --> 00:16:16,455
We were just traveling in France, my wife and I, and she
382
00:16:16,455 --> 00:16:20,115
wanted to access something that was only available from home.
383
00:16:20,145 --> 00:16:20,505
Great.
384
00:16:21,075 --> 00:16:22,275
Hand me your iPad a second.
385
00:16:22,275 --> 00:16:23,925
I didn't even bother to have her set up an account.
386
00:16:23,925 --> 00:16:26,115
I just logged it into my tail net so now she can get
387
00:16:26,115 --> 00:16:28,485
access to my shit posting nonsense if she really wants it.
388
00:16:28,785 --> 00:16:31,425
And suddenly it worked when I turned it on as an exit node.
389
00:16:31,725 --> 00:16:31,995
Uh.
390
00:16:32,190 --> 00:16:34,500
I've also found, and this is what really sparked the
391
00:16:34,500 --> 00:16:38,190
idea of having this conversation now, is with now
392
00:16:38,190 --> 00:16:40,200
I have a test Kubernetes cluster that mostly works.
393
00:16:40,200 --> 00:16:42,630
I have your provisioner that auto the operator that
394
00:16:42,630 --> 00:16:45,390
automatically gives access to any service I put on the thing.
395
00:16:45,660 --> 00:16:47,940
It's got some drama when the nodes themselves
396
00:16:47,940 --> 00:16:50,340
are on the tail net and that becomes their magic.
397
00:16:50,340 --> 00:16:52,290
DNS becomes their resolver.
398
00:16:52,560 --> 00:16:55,290
It tries to pass those out to containers and that becomes a little
399
00:16:55,290 --> 00:16:57,990
bit of a, uh, let's patch cord and s to make it not do that.
400
00:16:58,380 --> 00:17:01,260
But once I do, I can spin up arbitrary containers.
401
00:17:01,915 --> 00:17:05,635
Not have to worry about security, which sounds like a wild thing to say,
402
00:17:05,754 --> 00:17:09,415
but the only place that those things are available is on the tail net.
403
00:17:09,954 --> 00:17:13,435
I'm the only person except for my wife's iPad on the tail net,
404
00:17:13,944 --> 00:17:17,425
and even then I could restrict it down further via ACL grants.
405
00:17:17,575 --> 00:17:21,024
Suddenly I'm doing the thing that a lot of people used to do on the open
406
00:17:21,024 --> 00:17:24,655
internet of, oh, I'm not big enough to find, no one will find this weird port.
407
00:17:24,655 --> 00:17:27,504
I've bound it to only there is security.
408
00:17:27,504 --> 00:17:29,125
It's not just pretend security.
409
00:17:29,895 --> 00:17:30,195
Right.
410
00:17:30,254 --> 00:17:32,534
And that's another thing that like, you know, if it's, again,
411
00:17:32,564 --> 00:17:35,504
IPV six, if it had been fully rolled out today, still wouldn't
412
00:17:35,504 --> 00:17:38,294
solve that problem because it was invented 30 years ago and
413
00:17:38,294 --> 00:17:40,245
there's been 30 years of new problems since then, right?
414
00:17:40,245 --> 00:17:42,405
So it's like time for like a thing past IPV six,
415
00:17:42,405 --> 00:17:44,955
if we could move past it ourselves psychologically.
416
00:17:45,105 --> 00:17:47,024
But like there has to be identity, there has to be
417
00:17:47,024 --> 00:17:49,514
security, there has to be a concept of like, which things.
418
00:17:49,785 --> 00:17:53,805
Are allowed to connect to which other things, not just the dream of the late
419
00:17:53,805 --> 00:17:57,195
1990s of like, you know what, if everybody could just talk to everybody,
420
00:17:57,405 --> 00:18:00,075
the whole world would be happier and we'd have world peace and stuff.
421
00:18:00,345 --> 00:18:03,135
And we sort of learned from the internet that like world peace doesn't happen
422
00:18:03,135 --> 00:18:06,615
when everybody can like chase you around and harass you all day, right?
423
00:18:06,615 --> 00:18:08,865
And so you just need that level of security.
424
00:18:08,865 --> 00:18:11,415
But you want the feeling that we had on the small internet
425
00:18:11,805 --> 00:18:15,405
before, you know, most of the really bad people showed up.
426
00:18:15,645 --> 00:18:16,725
I think that's the right path.
427
00:18:16,725 --> 00:18:17,295
It's.
428
00:18:18,345 --> 00:18:22,425
You have, I keep forgetting this because of course in your case you
429
00:18:22,425 --> 00:18:25,514
have to deal with a, especially with a free way to get started here,
430
00:18:25,814 --> 00:18:30,284
you have to deal with a tremendous amount of abuse concerns on this.
431
00:18:30,645 --> 00:18:33,735
But it's, it's also not traffic necessarily passing through you.
432
00:18:33,885 --> 00:18:35,655
One of the smarter things you've done from pure cloud
433
00:18:35,655 --> 00:18:38,595
economics perspective is you're the coordination central
434
00:18:38,595 --> 00:18:41,385
point, but the actual heavy duty traffic is point to point.
435
00:18:42,375 --> 00:18:42,645
Yep.
436
00:18:42,705 --> 00:18:43,185
Exactly.
437
00:18:43,185 --> 00:18:45,225
So tail scale splits in, in network terms.
438
00:18:45,225 --> 00:18:48,045
We call it the control control plane and the data plane, right?
439
00:18:48,045 --> 00:18:50,805
The control plane is decides like how to distribute the keys, how do
440
00:18:50,805 --> 00:18:54,525
you log in, um, who should be allowed to talk to which other people.
441
00:18:54,525 --> 00:18:57,045
And then it like sends those instructions to every device in your tail net.
442
00:18:57,045 --> 00:19:00,075
And then the devices themselves, uh, handle the data plane, which is
443
00:19:00,075 --> 00:19:03,225
sending the data direct whenever possible directly point to point between.
444
00:19:03,305 --> 00:19:04,025
Between themselves.
445
00:19:04,175 --> 00:19:06,725
So it doesn't cost us anything to transport your data.
446
00:19:06,815 --> 00:19:08,405
And it costs us very little to be the
447
00:19:08,405 --> 00:19:10,085
simple coordination point between the notes.
448
00:19:10,085 --> 00:19:11,885
And this is what makes it extremely scalable.
449
00:19:11,975 --> 00:19:13,595
And a lot of this stuff is based on some of
450
00:19:13,595 --> 00:19:15,125
the original concepts of the internet, right?
451
00:19:15,125 --> 00:19:16,715
It's like, look, it should be extremely scalable.
452
00:19:16,715 --> 00:19:20,105
You can't have like one company that is routing all the traffic for
453
00:19:20,105 --> 00:19:23,405
you, such as at and t back in the day with the telephone network, right?
454
00:19:23,645 --> 00:19:26,225
Like you just, you know, it works, but you shouldn't have that.
455
00:19:26,225 --> 00:19:28,025
You should build a system where that doesn't happen.
456
00:19:28,325 --> 00:19:29,945
Uh, and tail scale is very much, uh.
457
00:19:30,395 --> 00:19:33,695
Moving along those lines and it, it, it is kind of magical, especially
458
00:19:33,695 --> 00:19:37,025
because if you get two devices sitting right next to each other on your local
459
00:19:37,025 --> 00:19:40,775
network, they get direct connections to each other on your local network.
460
00:19:41,045 --> 00:19:41,345
Right.
461
00:19:41,345 --> 00:19:44,165
Almost any other thing will try to beam it up to the internet and
462
00:19:44,165 --> 00:19:48,635
back, which is pointless in situations where they're side by side.
463
00:19:48,755 --> 00:19:51,935
And so if you've got a data center or a VPC filled with containers and they
464
00:19:51,935 --> 00:19:54,815
want to talk to each other, it's really silly to send all those things to
465
00:19:54,815 --> 00:19:57,935
the internet and back to say nothing of like the egress fees you'll incur.
466
00:19:59,130 --> 00:19:59,250
It.
467
00:19:59,310 --> 00:20:01,230
What, what's weird to me is also how effective you are at
468
00:20:01,260 --> 00:20:04,500
routing money to other companies, uh, uh, through tail scale.
469
00:20:04,500 --> 00:20:06,450
I use moad, uh, as we've discussed.
470
00:20:06,450 --> 00:20:09,240
I also pay for next DNS because that's where I do most of
471
00:20:09,240 --> 00:20:11,550
my ad blocking, which makes it super handy when I try and
472
00:20:11,550 --> 00:20:13,830
hit something like a link in an email that gets blocked.
473
00:20:14,070 --> 00:20:14,340
Great.
474
00:20:14,340 --> 00:20:15,240
I could special case it.
475
00:20:15,240 --> 00:20:16,200
Why would I do that?
476
00:20:16,350 --> 00:20:19,920
I'll just toggle off tail scale, hit the thing I need to and turn it back on.
477
00:20:20,070 --> 00:20:21,360
I do that multiple times.
478
00:20:21,360 --> 00:20:24,180
Every day you have become, uh, something I use
479
00:20:24,180 --> 00:20:25,950
constantly, but also almost never think about.
480
00:20:26,490 --> 00:20:30,120
Which is the, honestly, the, the Val Howa of infrastructure.
481
00:20:30,420 --> 00:20:30,660
Yep.
482
00:20:30,750 --> 00:20:32,160
Infrastructure is really tricky.
483
00:20:32,160 --> 00:20:35,280
'cause we have, you know, we're trying to balance, uh, word of mouth.
484
00:20:35,280 --> 00:20:37,260
'cause you want everyone to brag about how they use tail scale.
485
00:20:37,260 --> 00:20:39,900
And simultaneously the best infrastructure
486
00:20:39,900 --> 00:20:41,340
is the infrastructure you never think about.
487
00:20:41,520 --> 00:20:45,000
So it reminds me, I forget the name of this, that this trendy workout.
488
00:20:45,640 --> 00:20:48,340
Uh, campaign from like 10 years ago where like the, the
489
00:20:48,340 --> 00:20:50,440
joke was like, how do you know someone's on this trendy
490
00:20:50,440 --> 00:20:52,540
workout campaign is like, they won't stop talking about it.
491
00:20:52,990 --> 00:20:57,160
Uh, so tail scale people, people love their infrastructure so much that they
492
00:20:57,160 --> 00:21:00,460
will not stop talking about it, which is a very strange situation to be in.
493
00:21:00,520 --> 00:21:02,590
Uh, I did not see that coming when we started the company,
494
00:21:02,740 --> 00:21:05,080
but it's more or less what like drives the adoption of tail.
495
00:21:05,870 --> 00:21:06,140
This
496
00:21:06,140 --> 00:21:07,130
episode is sponsored
497
00:21:07,130 --> 00:21:08,990
in part by my day job Duck.
498
00:21:08,990 --> 00:21:12,170
Bill, do you have a horrifying AWS bill?
499
00:21:12,440 --> 00:21:14,330
That can mean a lot of things.
500
00:21:14,540 --> 00:21:17,630
Predicting what it's going to be, determining what it
501
00:21:17,630 --> 00:21:21,440
should be, negotiating your next long-term contract with
502
00:21:21,440 --> 00:21:25,700
AWS, or just figuring out why it increasingly resembles.
503
00:21:25,920 --> 00:21:29,580
Phone number, but nobody seems to quite know why that is.
504
00:21:29,880 --> 00:21:33,450
To learn more, visit duck bill hq.com.
505
00:21:33,780 --> 00:21:36,630
Remember, you can't duck the duck bill.
506
00:21:36,660 --> 00:21:42,000
Bill, which my CEO reliably informs me is absolutely not our slogan.
507
00:21:42,530 --> 00:21:46,699
Yeah, every time I see weird questions on come through on the AWS subreddit,
508
00:21:46,699 --> 00:21:50,000
which I keep a loose eye on, it's like, oh, that sounds like a tail scale usage.
509
00:21:50,000 --> 00:21:52,070
And sure enough, it's always the first comment someone has there.
510
00:21:52,070 --> 00:21:55,310
Have you considered using tail scale for this, like a sensible person, which
511
00:21:55,730 --> 00:21:56,870
Yeah, exactly.
512
00:21:57,139 --> 00:21:57,320
Yeah.
513
00:21:57,320 --> 00:21:59,629
And you mentioned like these partners that we work with and roading
514
00:21:59,629 --> 00:22:04,580
money to them, like tail scale is increasingly, uh, it's a little among.
515
00:22:04,635 --> 00:22:07,004
You know, in the, in the entrepreneur world, you have to be
516
00:22:07,004 --> 00:22:09,495
really careful with this word, but we are increasingly a platform.
517
00:22:09,885 --> 00:22:10,845
And what is a platform?
518
00:22:10,845 --> 00:22:14,625
It's like the base layer of something that people build on top of, right?
519
00:22:14,625 --> 00:22:18,315
And uh, I was talking to our investors the other day and someone said like,
520
00:22:18,615 --> 00:22:22,695
look, the, the advice, uh, or the most important thing to know about building
521
00:22:22,695 --> 00:22:26,715
a platform, and the biggest mistake almost everybody makes is trying to do it.
522
00:22:27,405 --> 00:22:30,014
Uh, and especially doing it too soon, like almost
523
00:22:30,014 --> 00:22:32,955
no company ever actually builds a platform.
524
00:22:33,225 --> 00:22:35,625
And if you are wrong and you go and build one
525
00:22:35,625 --> 00:22:37,544
anyway, you waste a ton of time and energy.
526
00:22:37,875 --> 00:22:40,665
And so we've been a little bit dragged into building a platform.
527
00:22:40,665 --> 00:22:42,764
I've started talking about last year how maybe
528
00:22:42,764 --> 00:22:44,405
someday tail scale's gonna evolve into a platform.
529
00:22:44,645 --> 00:22:44,915
Form.
530
00:22:45,365 --> 00:22:49,985
Uh, and then this year we made a feature that's called the Tail Net's, API.
531
00:22:50,225 --> 00:22:53,375
So a completely automated way to create a new tail net,
532
00:22:53,405 --> 00:22:55,745
add devices to it, and then spin down the tail net, share
533
00:22:55,745 --> 00:22:57,935
it with other people and stuff, just entirely API based.
534
00:22:58,055 --> 00:23:00,245
And so now we have big cloud providers that are
535
00:23:00,245 --> 00:23:01,685
like, you know what, I'm gonna make my InterCloud.
536
00:23:02,295 --> 00:23:04,905
Connections, just use tail scale in the background
537
00:23:04,905 --> 00:23:06,465
and our customers don't even have to know about it.
538
00:23:06,465 --> 00:23:09,615
And I'm gonna do it all using the tail nets A API, right?
539
00:23:09,615 --> 00:23:12,285
So we're kind of like, well, this is way ahead of schedule now we're a platform.
540
00:23:12,405 --> 00:23:13,695
Um, and I don't even,
541
00:23:13,875 --> 00:23:16,485
can you cheat it under the hood to take specific decisions
542
00:23:16,485 --> 00:23:18,735
on the path traffic takes to get from point A to point B?
543
00:23:19,095 --> 00:23:19,725
Yeah, exactly.
544
00:23:19,725 --> 00:23:25,065
I mean, they're basically, um, well the big, the, the problem space that these
545
00:23:25,065 --> 00:23:28,785
people are mostly in is like they're, you know, lower tier cloud providers.
546
00:23:28,935 --> 00:23:32,175
They provide, you know, the biggest thing is usually GPUs, right?
547
00:23:32,265 --> 00:23:34,905
Uh, at better prices than the big cloud providers have.
548
00:23:34,905 --> 00:23:37,065
And then customers like ignore the prices.
549
00:23:37,065 --> 00:23:38,535
They actually have them for rent.
550
00:23:38,945 --> 00:23:39,245
Yeah.
551
00:23:39,245 --> 00:23:40,865
Or more availability, et cetera, right?
552
00:23:40,865 --> 00:23:42,965
Or the right ones at all kinds of things.
553
00:23:43,205 --> 00:23:45,755
Uh, but then the same customers wanna run the rest
554
00:23:45,755 --> 00:23:47,465
of their stuff in a more mature cloud provider.
555
00:23:47,525 --> 00:23:50,165
Now you've got a connection problem between like, kind of
556
00:23:50,165 --> 00:23:53,945
weird GPU Cloud provider and the top tier provider, right?
557
00:23:54,095 --> 00:23:56,405
And so how do you connect between cloud providers?
558
00:23:56,405 --> 00:23:57,185
Well, it's actually hard.
559
00:23:57,185 --> 00:23:58,570
Almost nobody makes a product for that at all.
560
00:23:59,440 --> 00:24:01,960
Um, these cloud providers, they could tell you like,
561
00:24:01,960 --> 00:24:03,670
go use tail scale, but then you have to go figure out a
562
00:24:03,670 --> 00:24:05,590
third product that kind of slows down their marketing.
563
00:24:05,800 --> 00:24:06,670
So they're just like, you know what?
564
00:24:06,730 --> 00:24:09,490
We will provide the service of connecting you to anything.
565
00:24:09,910 --> 00:24:10,870
Uh, don't even worry about it.
566
00:24:11,830 --> 00:24:14,860
Uh, and they just like set up a tail net and suddenly their VPC
567
00:24:14,860 --> 00:24:17,920
on that cloud is actually connected to the VPC on the other cloud
568
00:24:18,100 --> 00:24:18,970
and it's the right path.
569
00:24:19,240 --> 00:24:22,720
But what I have found that is, so I guess.
570
00:24:23,159 --> 00:24:27,210
Compelling about all of this has just been that over the years it
571
00:24:27,210 --> 00:24:30,629
has, it has solved so many weird problems and I continue to watch
572
00:24:30,629 --> 00:24:35,129
the logos on your site continue to expand, uh, to going from small
573
00:24:35,129 --> 00:24:38,159
companies to mid-size companies like, I don't know, Microsoft.
574
00:24:39,080 --> 00:24:41,690
Yeah, Microsoft, uh, recently got added to our logo list.
575
00:24:41,750 --> 00:24:44,120
Uh, there's a bunch of other, you know, there's subsidiaries
576
00:24:44,120 --> 00:24:45,830
of Microsoft, there's a bunch of other big names.
577
00:24:45,980 --> 00:24:48,260
Most of our biggest names are still not actually in our
578
00:24:48,260 --> 00:24:50,300
logo list because we didn't get logo rights for them.
579
00:24:50,360 --> 00:24:53,930
Uh, people often that is always the way that it works.
580
00:24:53,990 --> 00:24:56,630
It's especially true in the security world, uh,
581
00:24:56,660 --> 00:24:58,400
because security people are like, wait, I don't want to
582
00:24:58,400 --> 00:25:01,220
advertise what our infrastructure is using for security.
583
00:25:01,220 --> 00:25:02,810
That's just like painting a sign on our back.
584
00:25:02,850 --> 00:25:02,909
Yeah.
585
00:25:03,510 --> 00:25:06,300
Do you view yourself as a security product tradition?
586
00:25:06,300 --> 00:25:10,139
Well, so I've ex uh, well, I'm, I'm stumbling on this
587
00:25:10,139 --> 00:25:13,350
because the correct answer is sort of, uh, or Yes.
588
00:25:13,409 --> 00:25:16,169
Well, who, whose cost center is our purchase, this contract coming out of?
589
00:25:16,169 --> 00:25:16,590
Sure.
590
00:25:16,590 --> 00:25:17,550
We're a security platform.
591
00:25:17,550 --> 00:25:17,939
I get it.
592
00:25:17,939 --> 00:25:19,260
Go, go where the money is.
593
00:25:19,260 --> 00:25:19,679
I hear you.
594
00:25:19,830 --> 00:25:20,610
Are you an analyst?
595
00:25:20,610 --> 00:25:20,969
No.
596
00:25:20,969 --> 00:25:22,290
Unless you have analyst budget, then?
597
00:25:22,290 --> 00:25:22,530
Yes.
598
00:25:22,800 --> 00:25:22,980
Yeah.
599
00:25:22,980 --> 00:25:23,790
So tail scale.
600
00:25:23,790 --> 00:25:27,570
I think the best term I heard for it is a mesh VPN firewall.
601
00:25:28,350 --> 00:25:28,710
Right.
602
00:25:29,040 --> 00:25:33,690
Um, and the reason for that is most people who end up adopting tail scale,
603
00:25:33,690 --> 00:25:36,540
adopt tail scale 'cause it solves a connectivity problem that they have
604
00:25:36,540 --> 00:25:39,450
right now, and they just, it becomes the easiest way to connect things.
605
00:25:39,810 --> 00:25:42,960
What's very strange about tail scale, uh, and very strange in
606
00:25:42,960 --> 00:25:46,200
the security world in general, is that when you use tail scale to
607
00:25:46,200 --> 00:25:49,710
solve that problem, you accidentally make your system more secure.
608
00:25:50,250 --> 00:25:52,890
And also the easiest thing for all of your engineers
609
00:25:52,890 --> 00:25:55,710
and people inside your company to do becomes.
610
00:25:56,129 --> 00:25:57,720
The secure thing instead of the insecure
611
00:25:57,720 --> 00:25:59,430
thing, and nobody really sees that coming.
612
00:25:59,580 --> 00:26:01,500
Um, but then once it gets there, the security
613
00:26:01,500 --> 00:26:03,840
people are like, wow, how come I'm not the bad guy?
614
00:26:03,900 --> 00:26:04,860
I'm always the bad guy.
615
00:26:04,920 --> 00:26:05,910
I don't wanna be the bad guy.
616
00:26:06,090 --> 00:26:07,230
Uh, we love tail scale.
617
00:26:07,590 --> 00:26:10,200
Uh, most of the time today, tail scale is not adopted
618
00:26:10,200 --> 00:26:12,420
through the security team because the burning problem
619
00:26:12,420 --> 00:26:14,490
is not like blocking people from connecting to things.
620
00:26:14,670 --> 00:26:17,070
The burning problem is usually connecting to things.
621
00:26:17,310 --> 00:26:18,870
Uh, but you get both at the same time.
622
00:26:19,260 --> 00:26:20,639
And that was like from the very beginning.
623
00:26:20,639 --> 00:26:22,350
At tail scale, usually you have to buy like a
624
00:26:22,350 --> 00:26:25,590
connectivity thing, like a router or a VPN and a firewall.
625
00:26:25,949 --> 00:26:28,104
And they're run by different teams and they fight with each other all day.
626
00:26:28,770 --> 00:26:32,190
Honestly, I found that the most people I talk to the most
627
00:26:32,190 --> 00:26:34,080
who are the biggest champions of tail scale are the ones
628
00:26:34,080 --> 00:26:35,910
that are empowered to do the thing that they wanna do.
629
00:26:36,210 --> 00:26:39,870
It's, oh, uh, the policy is because I said so the end, this
630
00:26:39,870 --> 00:26:41,460
doesn't feel like it's something that's gonna be instituted
631
00:26:41,460 --> 00:26:43,660
top down just because it's not painful enough for the user.
632
00:26:44,355 --> 00:26:47,175
We have a new experimental thing that we're working on, uh, that I
633
00:26:47,175 --> 00:26:50,715
think is really gonna appeal to security teams specifically as buyers.
634
00:26:50,985 --> 00:26:52,995
And I wanna run it by you and like hopefully get
635
00:26:52,995 --> 00:26:54,615
feedback from everybody else who's listening.
636
00:26:54,675 --> 00:26:58,395
Um, you can, you can post my email address or my blue sky or whatever you want.
637
00:26:58,575 --> 00:27:01,005
Um, so people get you say that and yet.
638
00:27:01,975 --> 00:27:03,595
You know, I, I get hate mail.
639
00:27:03,595 --> 00:27:05,545
I've, I've received hate mail at this point.
640
00:27:05,545 --> 00:27:07,975
I, as a CI get hate mail from my own employees.
641
00:27:08,035 --> 00:27:12,055
Um, and so, you know, the, the skin gets thicker over time.
642
00:27:12,355 --> 00:27:13,255
Um, but yeah.
643
00:27:13,255 --> 00:27:14,245
So here's the thing.
644
00:27:14,545 --> 00:27:16,825
Uh, ai, I think we've all heard about it.
645
00:27:16,945 --> 00:27:17,785
People are deploying it.
646
00:27:18,355 --> 00:27:23,004
Um, in their companies, uh, and often carelessly, believe it or not, uh,
647
00:27:23,004 --> 00:27:26,544
they don't always think about all the consequences before roll out ai.
648
00:27:26,784 --> 00:27:29,305
Uh, and yet, some many companies, and some of them we've
649
00:27:29,305 --> 00:27:31,794
heard about, uh, more than others, but many companies
650
00:27:31,794 --> 00:27:34,195
have directives from the top down to roll out more ai.
651
00:27:34,555 --> 00:27:36,534
So the CISO is sitting here and it's like, wow.
652
00:27:37,330 --> 00:27:39,310
Everything you guys are doing is horrible.
653
00:27:39,699 --> 00:27:41,350
Uh, and this is a ticking time bomb.
654
00:27:41,469 --> 00:27:44,199
And I can't believe that I have to say yes to this because
655
00:27:44,199 --> 00:27:46,780
my job is not just to block progress in the company.
656
00:27:46,780 --> 00:27:50,500
My job is to like ensure success or ensure security as much as we can.
657
00:27:50,830 --> 00:27:53,379
But if they say yes, it's like there's gonna be a breach.
658
00:27:53,709 --> 00:27:55,239
And if they say no, they're probably gonna
659
00:27:55,239 --> 00:27:56,739
get fired because they're blocking progress.
660
00:27:57,129 --> 00:27:57,610
Right.
661
00:27:58,090 --> 00:28:02,260
I think a solution to this is when you want to, and, oh,
662
00:28:02,260 --> 00:28:04,159
sorry, I forgot another part of the story, which is that.
663
00:28:04,794 --> 00:28:06,835
When you're bringing AI into the company, that's one thing.
664
00:28:06,955 --> 00:28:11,605
The new trend in AI is this MCP protocol model, context protocol that you
665
00:28:11,605 --> 00:28:15,804
can use to connect your favorite AI agent to your favorite data source,
666
00:28:15,804 --> 00:28:19,284
no matter what it might be, or all of your favorite data sources, right?
667
00:28:19,725 --> 00:28:22,485
When you do that, all kinds of terrible and exciting things can happen.
668
00:28:22,485 --> 00:28:25,965
And if you Google around a bit, you can find examples of like someone hooking.
669
00:28:25,980 --> 00:28:26,460
Hooking.
670
00:28:26,865 --> 00:28:27,165
Oh yeah.
671
00:28:27,165 --> 00:28:29,055
The, the attack vector now is quite literally
672
00:28:29,055 --> 00:28:31,965
telling the computer, trust me bro, in those words,
673
00:28:32,505 --> 00:28:35,415
and it's so, so exciting, the kinds of problems you can have, like
674
00:28:35,595 --> 00:28:38,955
some people hooked at GitHub up to this and like the repo that it
675
00:28:38,955 --> 00:28:43,155
looked at contained instructions to the LLM that then convinced it
676
00:28:43,155 --> 00:28:46,245
to take the rest of the data in GitHub and send it to somebody else.
677
00:28:46,245 --> 00:28:48,970
And he is like, wow, that's, that's a super neat attack.
678
00:28:49,030 --> 00:28:52,540
Uh, as a security person, I can appreciate super neat attacks, but also
679
00:28:52,540 --> 00:28:54,880
like, wow, what are you gonna do to defend against this kind of thing?
680
00:28:55,150 --> 00:28:55,390
Right?
681
00:28:55,390 --> 00:28:59,530
And I think the answer is the LLM has gotta be supervised, just like any, uh.
682
00:29:00,090 --> 00:29:03,300
Any person or any weird thing that you put into your network,
683
00:29:03,300 --> 00:29:08,910
you've gotta have auditability control, acls, identity encryption,
684
00:29:09,300 --> 00:29:11,280
uh, all that stuff that you should always have, that you
685
00:29:11,280 --> 00:29:13,950
actually don't have today when you hook an AI ops to stuff.
686
00:29:14,430 --> 00:29:14,730
Right?
687
00:29:14,730 --> 00:29:16,890
The way to do that is to funnel your AI traffic
688
00:29:16,890 --> 00:29:20,070
into a thing that has the ability to audit log.
689
00:29:20,190 --> 00:29:20,700
Um.
690
00:29:20,765 --> 00:29:23,435
And and control and filter and decide what can connect
691
00:29:23,435 --> 00:29:24,935
to which other things and then forward it on through.
692
00:29:25,625 --> 00:29:26,015
Right?
693
00:29:26,315 --> 00:29:28,775
And of course, tail scale is a connectivity and security layer
694
00:29:28,775 --> 00:29:31,745
that makes it easy to build such a thing and deploy such a thing.
695
00:29:32,285 --> 00:29:34,295
But then you have a really interesting other problem.
696
00:29:34,295 --> 00:29:36,965
And I apologize if this is getting like weirdly deep, but I hope
697
00:29:36,965 --> 00:29:40,235
your audience loves weirdly deep things once you've got a proxy.
698
00:29:40,990 --> 00:29:44,440
Once you've got a proxy that is forwarding traffic from, like it's
699
00:29:44,470 --> 00:29:47,919
acting on behalf of Avery, say, on its way to Salesforce, right?
700
00:29:48,310 --> 00:29:49,720
Avery goes into the proxy.
701
00:29:49,720 --> 00:29:51,370
The proxy then wants to go to Salesforce.
702
00:29:51,370 --> 00:29:53,230
The Salesforce says is like, okay, you're a proxy.
703
00:29:53,230 --> 00:29:54,340
You have a like service account.
704
00:29:54,850 --> 00:29:55,389
What did we do?
705
00:29:55,389 --> 00:29:57,490
Do we set up the service account to have, we have global
706
00:29:57,490 --> 00:30:01,690
access to Salesforce, and then the proxy needs to be trusted
707
00:30:01,690 --> 00:30:03,879
to only give Avery the stuff Avery should have access to.
708
00:30:04,120 --> 00:30:06,040
Well, that sounds like a terrible idea, but.
709
00:30:06,945 --> 00:30:09,705
It can't act as Avery by default, because it's not Avery.
710
00:30:09,705 --> 00:30:12,525
It's running as proxy and it had incoming connection
711
00:30:12,525 --> 00:30:14,535
from Avery that doesn't give it rights to Salesforce.
712
00:30:14,835 --> 00:30:16,785
So you have to have this little interchange
713
00:30:16,995 --> 00:30:18,225
to avoid confused deputy that way.
714
00:30:18,885 --> 00:30:19,485
Yeah, exactly.
715
00:30:19,485 --> 00:30:23,205
So you have to have this interesting interchange where Avery makes
716
00:30:23,205 --> 00:30:26,895
a connection to this proxy, and the proxy has the right to exchange
717
00:30:26,895 --> 00:30:30,945
that, that identity for a token that allows it to access Salesforce.
718
00:30:31,290 --> 00:30:33,960
As Avery with a little note on it that says, by the
719
00:30:33,960 --> 00:30:36,150
way, it's Avery's ai, don't give it too much stuff.
720
00:30:36,240 --> 00:30:37,740
So it's like Avery minus minus.
721
00:30:38,190 --> 00:30:42,630
To do that you can use an OAuth protocol that I won't go into, but
722
00:30:42,630 --> 00:30:45,000
it's like there's, you know, originally when the MCP standard came
723
00:30:45,000 --> 00:30:48,750
out 10 months ago, I think, uh, there was like almost literally
724
00:30:48,750 --> 00:30:51,330
at this page intentionally left blank in the security section.
725
00:30:51,690 --> 00:30:53,910
Uh, since then there has been an improvement where they,
726
00:30:54,030 --> 00:30:56,010
they said actually OAuth should be the way you do this.
727
00:30:56,130 --> 00:30:57,570
And then people started implementing that
728
00:30:57,570 --> 00:30:58,865
and now they're at the stage where like.
729
00:30:59,435 --> 00:31:02,255
It tries to o off to like 10 different things and each of those things
730
00:31:02,255 --> 00:31:05,225
leads you to a click through, uh, to granted permission to do some stuff.
731
00:31:05,705 --> 00:31:09,365
So with tail scale, we have this neat feature where like every connection
732
00:31:09,365 --> 00:31:12,965
that happens on the tail scale network has your identity already attached.
733
00:31:13,055 --> 00:31:14,255
You don't have to click through anything.
734
00:31:14,255 --> 00:31:16,595
It's just like inside your tail net, everything knows who you are.
735
00:31:16,925 --> 00:31:18,575
Every request inherently becomes authenticated.
736
00:31:19,265 --> 00:31:19,745
Exactly.
737
00:31:20,490 --> 00:31:23,640
So the trick we did is we wrote this new tool on top of tail
738
00:31:23,640 --> 00:31:27,150
scale called T-S-I-D-P, uh, the tail scale identity provider.
739
00:31:27,600 --> 00:31:28,590
Uh, it's open source, by the way.
740
00:31:28,590 --> 00:31:31,530
You can look at the, uh, GitHub repository and fork can do whatever you want.
741
00:31:31,530 --> 00:31:34,590
It's only a few hundred lines, and what it does is it's a complete OAuth
742
00:31:34,590 --> 00:31:38,790
server, but the user side is just, I already know who you are, right?
743
00:31:39,000 --> 00:31:42,390
So when you try to access a service, the service redirects you to your
744
00:31:42,390 --> 00:31:45,420
IDP, which says, I already know who you are, and then redirects it back.
745
00:31:45,660 --> 00:31:46,470
No click throughs.
746
00:31:47,040 --> 00:31:50,250
But it's controlled by the ACL grant policy we talked about earlier.
747
00:31:50,280 --> 00:31:51,930
'cause it's just a tool on top of tail scale.
748
00:31:51,930 --> 00:31:53,970
We didn't have to modify tail scale to make any of this work.
749
00:31:54,240 --> 00:31:57,300
It decides which kinds of tokens it's willing to exchange
750
00:31:57,540 --> 00:32:00,420
on behalf of this proxy running inside your tail net.
751
00:32:01,260 --> 00:32:05,100
Right, but this proxy, the TS IDP server, can be
752
00:32:05,100 --> 00:32:08,100
accessible over tail scale funnel to the outside world.
753
00:32:08,340 --> 00:32:11,580
So you can even use TS IDP with any service on the
754
00:32:11,580 --> 00:32:14,580
internet that supports custom IDP or custom OIDC.
755
00:32:15,240 --> 00:32:17,550
So you have this really interesting situation where.
756
00:32:17,895 --> 00:32:20,685
From the very beginning, tail scale is like, I'm not gonna be an IDP.
757
00:32:21,015 --> 00:32:22,545
We're not doing usernames and passwords.
758
00:32:22,575 --> 00:32:23,415
Get outta my way.
759
00:32:23,655 --> 00:32:24,435
That's the past.
760
00:32:24,465 --> 00:32:25,575
Let's live in the future.
761
00:32:25,755 --> 00:32:26,565
Use a real IDP.
762
00:32:27,045 --> 00:32:31,515
You should still do that, but you can use that to get into tail scale.
763
00:32:32,115 --> 00:32:35,925
And after that, you can use TS IDP to connect to everything else.
764
00:32:36,675 --> 00:32:39,525
And this MCP thing means your AI can do the same thing.
765
00:32:39,945 --> 00:32:40,275
Right.
766
00:32:40,275 --> 00:32:42,105
And all of it can be zero click because you
767
00:32:42,105 --> 00:32:44,745
can set a policy on your administrator for you.
768
00:32:44,745 --> 00:32:48,315
Company can set a policy on T-S-I-D-P to decide which things can be zero click.
769
00:32:49,035 --> 00:32:49,335
Right?
770
00:32:49,395 --> 00:32:52,935
And if you're worried about sort of privacy, I know a lot
771
00:32:52,935 --> 00:32:55,695
of people who like use Google, uh, log in with Google are
772
00:32:55,695 --> 00:32:57,855
like, ah, Google's tracking me all over the internet now.
773
00:32:57,855 --> 00:33:00,105
'cause I use login with Google every time I log into a service.
774
00:33:00,105 --> 00:33:01,155
They know every service I use.
775
00:33:01,665 --> 00:33:04,425
Now Google only knows that you use tail scale, right?
776
00:33:04,425 --> 00:33:07,514
Because your instance of TS IDP that you ran, that is open
777
00:33:07,514 --> 00:33:09,975
source, is the one doing all the rest of your authentication.
778
00:33:10,365 --> 00:33:11,745
And so you have access to all.
779
00:33:11,774 --> 00:33:14,145
You're the only one that has access to all that private information.
780
00:33:14,175 --> 00:33:14,835
Even we don't.
781
00:33:14,865 --> 00:33:16,395
'cause it's just a tool, right?
782
00:33:16,395 --> 00:33:17,445
Built on top of tail scale.
783
00:33:18,585 --> 00:33:21,254
And so the combination of all that stuff allows you to like
784
00:33:21,254 --> 00:33:23,985
control your AI access, but it also lets you have zero click
785
00:33:23,985 --> 00:33:26,445
authentication to like everything on the internet if you want.
786
00:33:26,985 --> 00:33:29,595
And it also lets you have zero click authentication to
787
00:33:29,595 --> 00:33:31,815
things on your tail net that don't understand tail scale.
788
00:33:31,875 --> 00:33:33,764
All they need to understand is custom OAuth.
789
00:33:34,215 --> 00:33:36,315
So I think Home Assistant is a really popular one.
790
00:33:36,345 --> 00:33:37,605
Grafana is another one, et cetera.
791
00:33:38,835 --> 00:33:40,335
So I apologize for that monologue.
792
00:33:40,635 --> 00:33:42,075
I'm still working on the short version.
793
00:33:42,435 --> 00:33:43,215
No, please.
794
00:33:43,215 --> 00:33:45,795
It's, it, it's a, it's a fascinating approach
795
00:33:45,795 --> 00:33:47,685
because we are definitely in a post network world.
796
00:33:47,685 --> 00:33:49,905
It used to be that once upon a time you had breaches where
797
00:33:49,905 --> 00:33:52,665
I'm gonna go and I'm going to go and take things out of your
798
00:33:52,665 --> 00:33:55,395
system and then send it to a different system somewhere else.
799
00:33:55,545 --> 00:33:59,025
Now you can do all of that just by hitting the same single endpoint.
800
00:33:59,025 --> 00:34:00,855
That's just the AWS Control plane.
801
00:34:01,190 --> 00:34:04,070
And it just a question of what the content of those requests are.
802
00:34:04,220 --> 00:34:07,010
So you effectively have to, I don't think we call it this anymore, but
803
00:34:07,010 --> 00:34:10,670
you need to, uh, man in the middle, everything that is being passed
804
00:34:10,670 --> 00:34:13,670
through for deep packet inspection, which in turn then becomes, if you
805
00:34:13,670 --> 00:34:16,610
can see all the payloads, well, you now have a central point of attack
806
00:34:16,610 --> 00:34:21,170
for that, but people have already accepted you in a security facing role.
807
00:34:21,409 --> 00:34:22,220
I think that.
808
00:34:22,620 --> 00:34:25,980
It is a more novel approach that is likely to get further
809
00:34:26,130 --> 00:34:29,490
than the current security posture, which is putting the
810
00:34:29,550 --> 00:34:32,670
No seriously, bro, be secure in all caps in the system.
811
00:34:32,670 --> 00:34:32,970
Prompt.
812
00:34:33,930 --> 00:34:34,140
Yeah.
813
00:34:34,140 --> 00:34:34,800
Well, exactly.
814
00:34:34,800 --> 00:34:36,990
And the best thing about this MCP proxy
815
00:34:36,990 --> 00:34:38,790
thing, first of all, you can have it right.
816
00:34:38,790 --> 00:34:40,380
We have a little default one.
817
00:34:40,380 --> 00:34:40,980
It's open source.
818
00:34:40,980 --> 00:34:44,190
Again, you can like build your own if you want, and it can run on your
819
00:34:44,190 --> 00:34:47,640
private tail net and it can access stuff that's on your private tail net.
820
00:34:47,915 --> 00:34:50,585
It can be accessed by your favorite LLM that might
821
00:34:50,585 --> 00:34:52,295
or may not be running on your private tail net.
822
00:34:52,324 --> 00:34:55,264
And also it can access things outside your private tail net so there's no people
823
00:34:55,264 --> 00:34:59,375
coming in trying to beat on your MCP server to find the security holes, right?
824
00:34:59,375 --> 00:35:01,715
It's only the content that matters.
825
00:35:01,715 --> 00:35:04,205
And for that, you can have something filtering the content and watching
826
00:35:04,205 --> 00:35:06,545
what's going on to make sure the AI doesn't go wildly off track.
827
00:35:07,145 --> 00:35:08,045
Yeah, I think that's
828
00:35:08,045 --> 00:35:09,335
the, that is the right path.
829
00:35:09,785 --> 00:35:11,855
It's, it's part of a defense in depth approach.
830
00:35:12,450 --> 00:35:12,840
Exactly.
831
00:35:12,840 --> 00:35:15,000
We're aiming for this like, again, convenience, where like the
832
00:35:15,000 --> 00:35:18,240
easiest way to roll out AI in your company is the tail scale
833
00:35:18,240 --> 00:35:20,730
way, and also coincidentally, it's gonna be way more secure.
834
00:35:20,790 --> 00:35:24,090
If we can get that, then I think we'll really, like, we'll be on the.
835
00:35:24,810 --> 00:35:27,480
The two problems I can see, you're gonna have one, you use
836
00:35:27,480 --> 00:35:29,280
the Salesforce, Salesforce example, but everything has to
837
00:35:29,280 --> 00:35:32,400
start supporting this on some level at an application level.
838
00:35:32,640 --> 00:35:34,710
So they need to support OAuth.
839
00:35:34,980 --> 00:35:37,170
They don't need to support any of the rest of the stuff.
840
00:35:37,200 --> 00:35:40,710
And that's what's really neat because everybody who makes an MCP server has
841
00:35:40,710 --> 00:35:44,460
to support OAuth now as part of the standard and like where APIs were kind
842
00:35:44,460 --> 00:35:47,850
of hard to get access to before the trend is that, look, everyone's gonna
843
00:35:47,850 --> 00:35:51,240
be mad at us as a vendor if we don't support OAuth for getting API keys.
844
00:35:51,635 --> 00:35:52,025
Right.
845
00:35:52,055 --> 00:35:54,905
As long as you have that, all of the rest of this magic is
846
00:35:54,905 --> 00:35:57,815
happening behind the scenes, the gateway has to understand all this.
847
00:35:57,815 --> 00:36:00,725
Tis IDP and everything, everybody else just sees to know server.
848
00:36:01,175 --> 00:36:01,505
Yeah.
849
00:36:01,505 --> 00:36:04,835
The, the other challenge that you're gonna have, and this is trivial of
850
00:36:04,835 --> 00:36:08,255
course, is you have to come up with a few, uh, reference implementations of
851
00:36:08,255 --> 00:36:11,975
this that are basically click, click done, and to show folks how it works.
852
00:36:12,155 --> 00:36:13,865
They can modify to their own approach.
853
00:36:13,865 --> 00:36:15,995
But historically, my big problem with.
854
00:36:16,015 --> 00:36:20,004
Uh, early with early stage products is the documentation's not there.
855
00:36:20,004 --> 00:36:22,975
You've gotta basically read the code, come up from first principles,
856
00:36:22,975 --> 00:36:25,194
how you want to tell it to actually do the thing that you do.
857
00:36:25,435 --> 00:36:28,404
A little bit of documentation goes a long way and not for nothing.
858
00:36:28,404 --> 00:36:31,825
Increasingly, that documentation is being written for LLMs
859
00:36:31,825 --> 00:36:34,254
so that they can then explain how to do this to folks.
860
00:36:34,345 --> 00:36:36,295
So there's a, there's a bit of a lead time.
861
00:36:36,295 --> 00:36:39,085
It has to be absorbed into the models before it starts spinning out.
862
00:36:39,735 --> 00:36:39,975
Yep.
863
00:36:40,005 --> 00:36:40,185
Yeah.
864
00:36:40,185 --> 00:36:43,245
The best we have right now, uh, we have a, a YouTube personality
865
00:36:43,245 --> 00:36:45,555
that works for us that runs the tail scale YouTube channel,
866
00:36:45,555 --> 00:36:48,585
Alex, and he's got at least one video about T-S-I-D-P.
867
00:36:48,975 --> 00:36:51,735
Uh, that's from before we added this MCP layer, but it's actually
868
00:36:51,735 --> 00:36:54,225
pretty well done, was like many, many people in their personal
869
00:36:54,225 --> 00:36:56,415
tail nets are already using T-S-I-D-P for their own stuff.
870
00:36:57,310 --> 00:36:59,380
Uh, so I think there's gonna be some, some growth there.
871
00:36:59,380 --> 00:37:00,580
But yeah, we're gonna have to document it.
872
00:37:00,580 --> 00:37:01,600
We're gonna have to do all that work.
873
00:37:01,630 --> 00:37:05,200
This is all pretty, pretty early stage, but we're really, we're interested
874
00:37:05,200 --> 00:37:07,750
in like talking to people who think this is gonna be interesting to
875
00:37:07,750 --> 00:37:10,270
them and like kind of working with them on making the product better
876
00:37:10,540 --> 00:37:13,900
and also integrating into the open source world, uh, because tail scale.
877
00:37:14,115 --> 00:37:15,285
Personal plan is free.
878
00:37:15,705 --> 00:37:19,665
Um, and it's unlimited, essentially, uh, unlimited time.
879
00:37:19,725 --> 00:37:20,714
Lots and lots of devices.
880
00:37:20,714 --> 00:37:21,915
You can do all kinds of stuff with it.
881
00:37:21,915 --> 00:37:25,814
And it would be nice to make people, or have people who are using this
882
00:37:25,814 --> 00:37:28,424
in their home lab already, they can take advantage of this thing as well.
883
00:37:29,055 --> 00:37:29,325
Oh yeah.
884
00:37:29,405 --> 00:37:32,345
I, I do a lot of testing in my home lab for this exact sort of thing.
885
00:37:32,610 --> 00:37:36,275
I, I still haven't gotten quite to a level of comfort where I'm
886
00:37:36,275 --> 00:37:39,575
putting production nodes independently On the tail net, I tend
887
00:37:39,575 --> 00:37:43,325
to use subnet routers, and, and that is for now the way that I
888
00:37:43,325 --> 00:37:46,295
approach it, just because it, it feels like taking anything into
889
00:37:46,295 --> 00:37:49,445
a critical path, past a certain point, has risk attached to it.
890
00:37:50,915 --> 00:37:52,685
That's how we built it and that's how it works for now.
891
00:37:52,685 --> 00:37:54,665
If I were doing it today, I don't know that I would be
892
00:37:54,665 --> 00:37:58,055
as cautious given the conversations I've had since then
893
00:37:58,055 --> 00:38:00,130
with customers who are working with it in that way.
894
00:38:01,169 --> 00:38:01,379
Yep.
895
00:38:01,410 --> 00:38:04,470
There are, there are some very big name customers, some of which I can name
896
00:38:04,470 --> 00:38:08,040
and some of which I can't, that are like all in on, like, we're gonna run
897
00:38:08,040 --> 00:38:11,460
Kubernetes in every single pod, in every single cluster, in every single store.
898
00:38:11,879 --> 00:38:14,730
Um, and turning 'em like crazy 'cause that's what Kubernetes does.
899
00:38:14,759 --> 00:38:16,410
Uh, and they, they seem to be pretty happy.
900
00:38:16,680 --> 00:38:19,200
It means we have to have like pretty high up tam on our control server
901
00:38:19,740 --> 00:38:22,620
Tail scale is designed so that even if the control server went down for.
902
00:38:22,870 --> 00:38:25,540
A while in, in fact, it could go down for hours.
903
00:38:25,540 --> 00:38:27,009
The data plane keeps on working.
904
00:38:27,310 --> 00:38:29,109
So there's only certain things that stop working if
905
00:38:29,109 --> 00:38:30,640
the control plane is like out of touch for a while.
906
00:38:30,640 --> 00:38:33,160
So you have this like, pretty high level of resilience that people
907
00:38:33,160 --> 00:38:35,620
don't expect, and it comes from us not routing your traffic.
908
00:38:36,040 --> 00:38:37,029
Uh, for the most part,
909
00:38:37,299 --> 00:38:38,830
that that is the bridged cross.
910
00:38:38,830 --> 00:38:40,900
And you've, you've hit a point now where
911
00:38:41,859 --> 00:38:43,750
there's enough of a community around tail scale.
912
00:38:44,384 --> 00:38:47,595
That if someone's trying to do something that no one else has really done
913
00:38:47,595 --> 00:38:52,515
before, it is no longer likely that they're doing something correctly.
914
00:38:52,815 --> 00:38:55,035
I, I don't mean to be unkind, but in the early days, I
915
00:38:55,035 --> 00:38:58,484
would, I was talking to your team near constantly with what?
916
00:38:58,484 --> 00:38:59,504
How do I do this thing?
917
00:38:59,504 --> 00:39:00,825
Oh, we hadn't considered that.
918
00:39:01,065 --> 00:39:04,424
Now, whenever I ask any of those questions that come up like, oh, here's a
919
00:39:04,424 --> 00:39:07,785
giant blog post on how to do that, or, here's the GitHub issue where we explain
920
00:39:07,785 --> 00:39:12,105
exactly how you're holding it wrong, and so on and so forth, which is just.
921
00:39:12,170 --> 00:39:14,720
It's, it's a, it's a maturing of the product.
922
00:39:15,530 --> 00:39:15,800
Yep.
923
00:39:15,830 --> 00:39:15,980
Yeah.
924
00:39:15,980 --> 00:39:17,750
We've been putting a lot of work into maturing it.
925
00:39:17,750 --> 00:39:21,800
I think one of the hardest things as CEO, uh, is just con
926
00:39:21,860 --> 00:39:25,919
convincing everybody to not build a. Everything they want.
927
00:39:25,919 --> 00:39:28,410
And just like, let's focus on refining the core.
928
00:39:28,589 --> 00:39:30,600
Let's do everything we can to run this business so
929
00:39:30,600 --> 00:39:32,339
that the core gets better and better and better.
930
00:39:32,339 --> 00:39:33,419
And that's how we're gonna make money.
931
00:39:33,629 --> 00:39:35,939
Not like building tons of stuff on top.
932
00:39:36,480 --> 00:39:38,669
Uh, which I know is a pretty unusual, especially in
933
00:39:38,669 --> 00:39:40,649
the security world, is not the normal way to do it.
934
00:39:40,649 --> 00:39:41,970
The normal way to do it's collect.
935
00:39:42,450 --> 00:39:44,634
Uh, I know it's like collecting Pokemon cards or whatever.
936
00:39:44,714 --> 00:39:46,799
Well, I need a DLP and I need this and I need this
937
00:39:46,799 --> 00:39:48,390
and I need this and I need this and I need this.
938
00:39:48,390 --> 00:39:50,220
And now you can buy it from one vendor and
939
00:39:50,220 --> 00:39:51,720
it's gonna be a collection of like sort of.
940
00:39:52,365 --> 00:39:54,075
Halfheartedly integrated tools, right?
941
00:39:54,075 --> 00:39:55,605
And tail scale is like, look, we're not that.
942
00:39:55,904 --> 00:39:58,845
We have this one thing, it works super well and it's gonna work
943
00:39:58,845 --> 00:40:01,904
with all the other stuff you buy from other people, but it means
944
00:40:01,904 --> 00:40:04,275
we spend all our time just like, you know, writing docs like
945
00:40:04,275 --> 00:40:07,035
those or fixing the bugs that led to the need for docs like those.
946
00:40:07,694 --> 00:40:08,535
It's really neat.
947
00:40:08,924 --> 00:40:11,920
Any, any last words on what we can expect in the somewhat near future?
948
00:40:12,240 --> 00:40:13,275
Anything fun and exciting?
949
00:40:13,395 --> 00:40:16,095
Uh, coming down the pike, which I know is a weird thing
950
00:40:16,095 --> 00:40:18,310
to say about a networking infrastructure tool, and yet.
951
00:40:20,130 --> 00:40:23,160
Um, I think the, the two most interesting
952
00:40:23,160 --> 00:40:24,810
things are happen are that are gonna happen.
953
00:40:24,810 --> 00:40:27,960
One of them is more and more stuff is gonna be buildable on
954
00:40:27,960 --> 00:40:31,410
top of tail scale or include tail scale as an option in it.
955
00:40:31,410 --> 00:40:33,450
So we're starting to see more and more things like, Hey, if
956
00:40:33,450 --> 00:40:35,970
you run my program, it's linked with the tail scale library.
957
00:40:35,970 --> 00:40:38,730
Just paste your offkey here and that thing is just going to work.
958
00:40:39,180 --> 00:40:41,310
Um, a similar one is, I think.
959
00:40:41,310 --> 00:40:43,650
Think, I don't know if we've announced it or not, we're gonna announce it.
960
00:40:43,650 --> 00:40:44,610
If not, this is the announcement.
961
00:40:44,640 --> 00:40:48,210
Um, the workload identity feature that allows, if you're
962
00:40:48,210 --> 00:40:50,430
using tail scale with GitHub actions, for example, to
963
00:40:50,430 --> 00:40:52,620
just like, not even use Offkey because you can set it up.
964
00:40:52,620 --> 00:40:54,810
It's like, oh, this is your account on GitHub.
965
00:40:54,960 --> 00:40:57,180
I believe GitHub when it says it's running under this account.
966
00:40:57,180 --> 00:40:59,610
So now everything just has access to your tail net automatically.
967
00:40:59,910 --> 00:41:01,200
That's super slick way to do it.
968
00:41:01,200 --> 00:41:03,450
You don't have to manage rotating off keys and stuff like that.
969
00:41:03,750 --> 00:41:06,509
And I guess the third one is, uh, for direct connectivity.
970
00:41:07,049 --> 00:41:08,370
You know, life is not always perfect.
971
00:41:08,370 --> 00:41:09,660
Sometimes firewalls are weird.
972
00:41:09,839 --> 00:41:13,020
Um, so we have this new thing called, you
973
00:41:13,020 --> 00:41:13,924
mean there are times where they're not.
974
00:41:15,195 --> 00:41:16,545
Well, some are weirder than others.
975
00:41:16,545 --> 00:41:18,675
So we, we get through almost all the weird firewalls,
976
00:41:18,675 --> 00:41:20,805
but there's some extremely weird firewalls out there.
977
00:41:21,134 --> 00:41:24,404
Uh, we have this new thing called the peer relay also in alpha, but if
978
00:41:24,404 --> 00:41:27,795
you're interested in choir within, uh, should be in beta sometime soon.
979
00:41:27,795 --> 00:41:30,975
But you can have really access if you, if somebody asks, um.
980
00:41:31,294 --> 00:41:34,174
It allows, basically, if you remember the old days of Skype and
981
00:41:34,174 --> 00:41:37,294
Supernodes, it allows you to build supernodes that will route the
982
00:41:37,294 --> 00:41:40,294
traffic in situations where direct connections are not possible.
983
00:41:40,294 --> 00:41:42,845
So you can still get full speed if you put your Supernodes in
984
00:41:42,845 --> 00:41:46,444
the right places, um, including behind a firewall if you want.
985
00:41:46,444 --> 00:41:46,520
So then.
986
00:41:47,085 --> 00:41:49,395
Even when things can't manage to get direct connections, 'cause
987
00:41:49,395 --> 00:41:52,125
your internal firewalls are too weird, if they can connect to the
988
00:41:52,125 --> 00:41:55,065
supernode behind your firewall, you can still avoid the egress traffic.
989
00:41:55,575 --> 00:41:57,765
Uh, so this is something that our biggest customers
990
00:41:57,765 --> 00:41:59,835
with, of course, the weirdest firewalls and the most
991
00:41:59,835 --> 00:42:02,175
firewalls, uh, are gonna benefit from humongously.
992
00:42:02,895 --> 00:42:04,995
I would love to hear the story about which firewalls
993
00:42:05,355 --> 00:42:07,214
that are doing this and how they're configured.
994
00:42:07,245 --> 00:42:09,825
'cause that is such a rare occurrence in the modern era, but,
995
00:42:10,365 --> 00:42:15,495
oh yeah, we actually, we actually sponsored a patch to free BSD to finally
996
00:42:15,495 --> 00:42:19,004
fix this problem, 'cause for a while, free BSD, any free BSD based firewall.
997
00:42:19,274 --> 00:42:20,265
Of course, it's pf.
998
00:42:21,275 --> 00:42:23,585
Yeah, well, uh, it was, it's in, it's intended
999
00:42:23,585 --> 00:42:25,384
as a security feature to be blocking this stuff.
1000
00:42:25,384 --> 00:42:28,625
It just turns out when you do the whole like, decision tree, it turned
1001
00:42:28,625 --> 00:42:30,875
out that didn't increase security at all and just made everyone's
1002
00:42:30,875 --> 00:42:34,355
life miserable and in instead of, uh, of, 'cause it makes it so
1003
00:42:34,355 --> 00:42:37,775
secure that to get anything done, people start rolling out UPNP.
1004
00:42:38,345 --> 00:42:40,714
Uh, and UPMP is never a good choice.
1005
00:42:40,845 --> 00:42:44,355
Uh, security wise, and yet it's the only workaround to this problem.
1006
00:42:44,355 --> 00:42:45,705
So we finally convinced them of this.
1007
00:42:45,705 --> 00:42:48,735
We sponsored free BSD to like, Hey, can you at least make it a flag?
1008
00:42:48,944 --> 00:42:51,464
Why does AHI take up all my CPU core?
1009
00:42:51,525 --> 00:42:53,055
Yeah, yeah, yeah.
1010
00:42:53,085 --> 00:42:56,535
So we made, uh, now there's a flag and I think the flag is now the default.
1011
00:42:56,535 --> 00:42:59,085
Just not be silly, but there are a few other firewall vendors that
1012
00:42:59,085 --> 00:43:01,815
are doing the same thing, but I'm hoping we can talk them out of it.
1013
00:43:01,815 --> 00:43:03,765
'cause it's actually a relatively simple, it's called a
1014
00:43:03,765 --> 00:43:07,069
hard net versus a. Easy net in tail scale terminology,
1015
00:43:07,160 --> 00:43:09,650
and they make their hard net hard for like it turns out.
1016
00:43:09,920 --> 00:43:12,859
No good reason, and it's avoidable if you change your code just a little bit.
1017
00:43:13,310 --> 00:43:15,560
But unfortunately, sometimes it's our competitors making
1018
00:43:15,560 --> 00:43:17,810
the firewall so they don't always super eager to do that.
1019
00:43:18,060 --> 00:43:18,120
Yeah,
1020
00:43:19,595 --> 00:43:22,230
I, I really wanna thank you for taking the time to speak with me.
1021
00:43:22,410 --> 00:43:24,480
If people wanna find out more, where should they go?
1022
00:43:24,720 --> 00:43:26,190
Uh, well, there's tail scale.com.
1023
00:43:26,279 --> 00:43:27,090
We have a blog.
1024
00:43:27,210 --> 00:43:28,710
Uh, sometimes I post in the blog.
1025
00:43:28,710 --> 00:43:30,540
I also have an account on Blue Sky.
1026
00:43:30,840 --> 00:43:33,870
I have a little used account on the system for Blue known as Twitter.
1027
00:43:34,415 --> 00:43:36,875
Uh, and I have my own blog on appware.ca,
1028
00:43:37,115 --> 00:43:39,634
which has been a recurring presence on the newsletter.
1029
00:43:39,634 --> 00:43:41,285
Whenever you put something interesting out there,
1030
00:43:41,375 --> 00:43:43,475
and we'll put links to all of this in the show notes.
1031
00:43:43,745 --> 00:43:45,875
Thank you so much for taking the time to speak with me.
1032
00:43:45,935 --> 00:43:46,654
I appreciate it.
1033
00:43:47,165 --> 00:43:48,065
Thank you very much.
1034
00:43:48,065 --> 00:43:48,185
It's
1035
00:43:48,185 --> 00:43:48,875
always a pleasure.
1036
00:43:49,355 --> 00:43:52,235
Avery Penan, CEO, and co-founder of Tail Scale.
1037
00:43:52,415 --> 00:43:55,595
I'm cloud economist Cory Quinn, and this is Screaming In the Cloud.
1038
00:43:55,775 --> 00:43:58,415
If you've enjoyed this podcast, please, we have a five star review
1039
00:43:58,415 --> 00:44:01,175
on your podcast platform of choice, whereas if you've hated this
1040
00:44:01,175 --> 00:44:04,295
podcast, please, we have a five star review on your podcast platform
1041
00:44:04,295 --> 00:44:08,015
of choice along with an angry comment that isn't going to post properly
1042
00:44:08,135 --> 00:44:10,775
because you once again have misconfigured your crappy firewall.