1
00:00:00,192 --> 00:00:01,560
Governor and a member of

2
00:00:01,560 --> 00:00:03,195
congress last month and in some

3
00:00:03,195 --> 00:00:05,197
cases left voicemails and sent

4
00:00:05,197 --> 00:00:07,399
texts on the messaging app

5
00:00:07,399 --> 00:00:09,234
signal. Hamza Choudhury is an

6
00:00:09,234 --> 00:00:10,302
A.I. And national security lead

7
00:00:10,302 --> 00:00:12,204
at the future of life institute.

8
00:00:12,204 --> 00:00:13,872
He's with us this hour fr

9
00:00:13,872 --> 00:00:15,207
Washington. Thank you for being

10
00:00:15,207 --> 00:00:18,210
taking time to speak with us.

11
00:00:18,210 --> 00:00:19,211
It's appreciated.

12
00:00:19,211 --> 00:00:20,212
>> Thank you for.

13
00:00:20,212 --> 00:00:21,213
>> Having.

14
00:00:21,213 --> 00:00:22,214
>> Me, John.

15
00:00:22,214 --> 00:00:23,281
>> Okay, so not much seems to be

16
00:00:23,281 --> 00:00:24,349
known about who is actually

17
00:00:24,349 --> 00:00:25,417
impersonating Marco Rubio, but

18
00:00:25,417 --> 00:00:26,484
there seems to be some insight

19
00:00:26,484 --> 00:00:27,686
into why, according to a

20
00:00:27,686 --> 00:00:28,753
diplomatic cable sent from the

21
00:00:28,753 --> 00:00:29,821
state department, which reads in

22
00:00:29,821 --> 00:00:31,223
part, the actor likely aimed to

23
00:00:31,223 --> 00:00:33,525
manipulate targeted individuals

24
00:00:33,525 --> 00:00:34,693
using A.I. Generated text and

25
00:00:34,693 --> 00:00:36,461
voice messages, with the goal of

26
00:00:36,461 --> 00:00:38,630
gaining access to information or

27
00:00:38,630 --> 00:00:40,232
accounts. We don't know if they

28
00:00:40,232 --> 00:00:42,234
succeeded, but how big of a

29
00:00:42,234 --> 00:00:43,702
security risk is this,

30
00:00:43,702 --> 00:00:44,769
especially given that a weak

31
00:00:44,769 --> 00:00:46,638
link within the system is being

32
00:00:46,638 --> 00:00:49,708
targeted here and that weak spot

33
00:00:49,708 --> 00:00:51,243
is actual people?

34
00:00:51,243 --> 00:00:52,644
>> Well, it's a.

35
00:00:52,644 --> 00:00:53,712
>> Concerning development, but

36
00:00:53,712 --> 00:00:54,913
it's not shocking. The

37
00:00:54,913 --> 00:00:56,314
surprising thing here isn't that

38
00:00:56,314 --> 00:00:58,250
this has happened. It's

39
00:00:58,250 --> 00:00:59,484
surprising that it's taken this

40
00:00:59,484 --> 00:01:01,253
long. We've seen generative A.I.

41
00:01:01,253 --> 00:01:02,721
Products being used to create

42
00:01:02,721 --> 00:01:05,924
fairly compelling deepfakes for

43
00:01:05,924 --> 00:01:07,459
financial scams to perpetuate

44
00:01:07,459 --> 00:01:09,661
sexual abuse, to launch

45
00:01:09,661 --> 00:01:11,496
manipulation campaigns for the

46
00:01:11,496 --> 00:01:12,564
past two years now. And it was

47
00:01:12,564 --> 00:01:14,266
only a matter of time before

48
00:01:14,266 --> 00:01:15,433
state and non-state actors,

49
00:01:15,433 --> 00:01:17,269
which wish to harm the united

50
00:01:17,269 --> 00:01:19,504
States woke up and said, this is

51
00:01:19,504 --> 00:01:21,406
a key tool in our arsenal, a

52
00:01:21,406 --> 00:01:24,276
tool that can be used to

53
00:01:24,276 --> 00:01:26,278
perpetuate disinformation in any

54
00:01:26,278 --> 00:01:28,280
language with an input of any

55
00:01:28,280 --> 00:01:29,681
language. A system that can be

56
00:01:29,681 --> 00:01:31,783
used to create vast tracts of

57
00:01:31,783 --> 00:01:33,285
disinformation at superhuman

58
00:01:33,285 --> 00:01:34,619
speeds, and also systems which

59
00:01:34,619 --> 00:01:36,588
can be used to create this

60
00:01:36,588 --> 00:01:38,290
manipulative content in audio,

61
00:01:38,290 --> 00:01:41,293
video and text. So what we're

62
00:01:41,293 --> 00:01:43,561
seeing now is the logical

63
00:01:43,561 --> 00:01:44,629
conclusion of what we've been

64
00:01:44,629 --> 00:01:45,697
seeing for the past two years.

65
00:01:45,697 --> 00:01:47,365
>> That diplomatic cable also

66
00:01:47,365 --> 00:01:51,436
points out that the campaign was

67
00:01:51,436 --> 00:01:52,504
highly detailed, and the actor

68
00:01:52,504 --> 00:01:53,571
demonstrated extensive knowledge

69
00:01:53,571 --> 00:01:54,639
of the department's naming

70
00:01:54,639 --> 00:01:55,707
conventions and internal

71
00:01:55,707 --> 00:01:56,775
documentation. So, you know,

72
00:01:56,775 --> 00:01:57,976
just playing detective here, it

73
00:01:57,976 --> 00:01:59,311
sounds like this could be a

74
00:01:59,311 --> 00:02:00,312
former maybe a current employee

75
00:02:00,312 --> 00:02:01,313
at the state department, or at

76
00:02:01,313 --> 00:02:03,581
least a government employee. It

77
00:02:03,581 --> 00:02:04,949
seems that inside knowledge is

78
00:02:04,949 --> 00:02:06,318
that what makes this such a

79
00:02:06,318 --> 00:02:07,319
credible threat? Much more so

80
00:02:07,319 --> 00:02:09,921
than the A.I. Technology which

81
00:02:09,921 --> 00:02:11,323
is being used?

82
00:02:11,323 --> 00:02:12,624
>> Well, it depends by what we

83
00:02:12,624 --> 00:02:13,725
mean here, by insider knowledge

84
00:02:13,725 --> 00:02:15,527
and the public reporting on this

85
00:02:15,527 --> 00:02:18,596
is fairly has been pretty sparse

86
00:02:18,596 --> 00:02:19,664
so far. There's a fair amount

87
00:02:19,664 --> 00:02:21,333
of information that you can find

88
00:02:21,333 --> 00:02:23,335
out about designations for s1,

89
00:02:23,335 --> 00:02:25,603
s2 and s3, how the state

90
00:02:25,603 --> 00:02:26,838
department is divvied up into

91
00:02:26,838 --> 00:02:28,873
different secretariats, what the

92
00:02:28,873 --> 00:02:31,343
standard operating procedure is,

93
00:02:31,343 --> 00:02:32,344
and each secretariat just based

94
00:02:32,344 --> 00:02:33,345
off of what's on the internet

95
00:02:33,345 --> 00:02:34,746
and spending a couple of years

96
00:02:34,746 --> 00:02:37,816
in DC. So the most concerning

97
00:02:37,816 --> 00:02:39,351
development here, alongside the

98
00:02:39,351 --> 00:02:41,353
potential threat of an insider

99
00:02:41,353 --> 00:02:42,354
threat, is also that there are

100
00:02:42,354 --> 00:02:43,355
folks who are far away from the

101
00:02:43,355 --> 00:02:44,356
state department outside these

102
00:02:44,356 --> 00:02:46,691
borders who could commit an

103
00:02:46,691 --> 00:02:47,759
equally sophisticated attack in

104
00:02:47,759 --> 00:02:48,827
the future.

105
00:02:48,827 --> 00:02:49,894
>> Yeah, you mentioned this

106
00:02:49,894 --> 00:02:50,962
isn't the first time something

107
00:02:50,962 --> 00:02:52,030
like this has happened. Back in

108
00:02:52,030 --> 00:02:54,466
may, I was used to impersonate

109
00:02:54,466 --> 00:02:55,533
the white house chief of staff

110
00:02:55,533 --> 00:02:57,702
in a bid to contact lawmakers.

111
00:02:57,702 --> 00:02:59,704
We also know that the former

112
00:02:59,704 --> 00:03:00,772
Obama chief of staff, David

113
00:03:00,772 --> 00:03:02,874
Axelrod, warned on Tuesday, this

114
00:03:02,874 --> 00:03:04,376
is the new world in which we

115
00:03:04,376 --> 00:03:05,377
live, and we better figure out

116
00:03:05,377 --> 00:03:06,778
how to defend against it because

117
00:03:06,778 --> 00:03:08,713
of its implications for our

118
00:03:08,713 --> 00:03:10,382
democracy and the world. So I

119
00:03:10,382 --> 00:03:12,784
guess, you know, what are the

120
00:03:12,784 --> 00:03:14,386
implications here? And also, it

121
00:03:14,386 --> 00:03:15,387
seems the guardrails which are

122
00:03:15,387 --> 00:03:16,755
in place, the guidelines, the

123
00:03:16,755 --> 00:03:18,490
security protocols, even the

124
00:03:18,490 --> 00:03:19,557
technology which is being used

125
00:03:19,557 --> 00:03:20,625
to for security reasons is all

126
00:03:20,625 --> 00:03:23,395
of that sort of stuck in the

127
00:03:23,395 --> 00:03:25,997
early 2000, in a world before I.

128
00:03:25,997 --> 00:03:27,565
>> I think that's partly true.

129
00:03:27,565 --> 00:03:28,633
And maybe we can zoom out to

130
00:03:28,633 --> 00:03:29,701
first talk about the united

131
00:03:29,701 --> 00:03:31,569
States and then talk about the

132
00:03:31,569 --> 00:03:33,938
world. So it is still somewhat

133
00:03:33,938 --> 00:03:36,775
hard to do what was committed in

134
00:03:36,775 --> 00:03:37,842
this particular attack, because

135
00:03:37,842 --> 00:03:38,910
you're talking about the

136
00:03:38,910 --> 00:03:39,978
secretary of state. You're also

137
00:03:39,978 --> 00:03:41,413
talking about communication

138
00:03:41,413 --> 00:03:42,514
channels, which have various

139
00:03:42,514 --> 00:03:44,082
verification pathways before you

140
00:03:44,082 --> 00:03:45,417
actually get on the phone with

141
00:03:45,417 --> 00:03:46,418
the foreign minister. That's

142
00:03:46,418 --> 00:03:47,419
less true for your average

143
00:03:47,419 --> 00:03:49,421
national security staffer in the

144
00:03:49,421 --> 00:03:52,056
government, right? You have a

145
00:03:52,056 --> 00:03:53,425
handful of staffers in congress

146
00:03:53,425 --> 00:03:54,426
who decide the trillion dollar

147
00:03:54,426 --> 00:03:55,427
Pentagon budget, and they

148
00:03:55,427 --> 00:03:57,061
definitely don't have the same

149
00:03:57,061 --> 00:03:58,430
security protocol that the

150
00:03:58,430 --> 00:03:59,864
secretary of state does. You

151
00:03:59,864 --> 00:04:01,599
have national security staffers

152
00:04:01,599 --> 00:04:02,667
working in the intelligence

153
00:04:02,667 --> 00:04:03,735
community with highly sensitive

154
00:04:03,735 --> 00:04:04,803
information, which, if released,

155
00:04:04,803 --> 00:04:07,439
could cause severe harm to the

156
00:04:07,439 --> 00:04:08,807
United States. And you also have

157
00:04:08,807 --> 00:04:10,542
folks touching our nuclear

158
00:04:10,542 --> 00:04:12,677
weapons system, relying on a

159
00:04:12,677 --> 00:04:14,879
personal reliability program,

160
00:04:14,879 --> 00:04:16,815
which was last updated more than

161
00:04:16,815 --> 00:04:18,082
30 years ago. Now, the situation

162
00:04:18,082 --> 00:04:20,452
outside the United States is

163
00:04:20,452 --> 00:04:21,453
even more urgent, and the best

164
00:04:21,453 --> 00:04:22,821
way to look at this is through

165
00:04:22,821 --> 00:04:23,888
the social media lens. So you

166
00:04:23,888 --> 00:04:26,558
could go back five years and

167
00:04:26,558 --> 00:04:28,660
look at what happened with

168
00:04:28,660 --> 00:04:29,727
social media and misinformation

169
00:04:29,727 --> 00:04:32,964
in Myanmar. And the simple

170
00:04:32,964 --> 00:04:34,032
reason for that was that the

171
00:04:34,032 --> 00:04:35,500
vast majority of fact checkers

172
00:04:35,500 --> 00:04:37,869
hired by social media companies

173
00:04:37,869 --> 00:04:39,103
were moderating content in

174
00:04:39,103 --> 00:04:40,972
English, whereas most users of

175
00:04:40,972 --> 00:04:42,707
the social media companies were

176
00:04:42,707 --> 00:04:43,775
based outside the United States.

177
00:04:43,775 --> 00:04:44,843
In non-english speaking

178
00:04:44,843 --> 00:04:46,711
communities. The most concerning

179
00:04:46,711 --> 00:04:48,580
threat then globally is the same

180
00:04:48,580 --> 00:04:49,914
kind of attack being committed

181
00:04:49,914 --> 00:04:51,649
against a foreign minister. In

182
00:04:51,649 --> 00:04:56,488
amharic or igbo or arabic or

183
00:04:56,488 --> 00:04:59,491
chechnyan, and those are

184
00:04:59,491 --> 00:05:00,492
countries with institutional

185
00:05:00,492 --> 00:05:01,493
setups which are just less well

186
00:05:01,493 --> 00:05:02,594
developed to respond and are

187
00:05:02,594 --> 00:05:04,162
less robust to the kind of

188
00:05:04,162 --> 00:05:05,497
threats we're talking about

189
00:05:05,497 --> 00:05:08,166
here. So as concerning as it is

190
00:05:08,166 --> 00:05:10,034
here in America, it is a much

191
00:05:10,034 --> 00:05:11,936
more concerning development for

192
00:05:11,936 --> 00:05:13,805
countries elsewhere where those

193
00:05:13,805 --> 00:05:14,906
systems just aren't as robust

194
00:05:14,906 --> 00:05:16,508
as we need them to be to respond

195
00:05:16,508 --> 00:05:17,609
to these A.I. Threats.

196
00:05:17,609 --> 00:05:19,677
>> Add it to the list of just