writing xks fingerprints - electronic frontier foundation€¦ · 6/7/2015 · fingerprints 101...
TRANSCRIPT
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Writing XKS Fingerprints
November 2010
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
• Naming Fingerprints • Simple Keywords • Boolean Logic • Variables • Context-Sensitive
TOP SECRET//COMINT
• • • • • • • •
Fingerprints 101
• Whats in a name?
• The XKS Fingerprint naming convention can help organize fingerprints and make searching easier so its important to make sure you name your fingerprint inline with the existing convention
TOP SECRET//COMINT
what's in a name • For example, fingerprint names look like this:
• encryption/archive/rar • encryption/archive/pkzip • encryption/archive/pkzip
• Notice the directory-like structure so that all encryption fingerprints are within the same "folder" and all encryption/archive fingerprints are within the same "folder"
TOP SECRET//COMINT
• • • • • • • •
what's in a name
• This allows for smarter searching because you could look for all encryption fingerprints by searching for encryption/* or search for all encryption/archive fingerprints by searching for encryption/archive/* and etc.
TOP SECRET//COMINT
what's in a name
• When you want to submit a new fingerprint, look to see if it would fit into any existing fingerprint folders.
• Best way to do this is to use either the "Field Builder" or "Tree Field Builder" next to the AppID+Fingeprints field in the search forms
AppID (+Finaerprints) f fu l l text l :
T I rPopulate with Field Builder] I "¿1 rPopulate with Tree Field Builder"!
TOP SECRET//COMINT
CRET STRAPi
fr-r- ^^BL. , , « -« « « » » « i . . . . . . ' . il I
What's in a name
• The field builders allow you to browse existing fingerprint directories to see if one already exists for your new fingerprint
Field Bui lder
J ^ A p p I D (+ Fingerprints)
> j A p p l i c a t i o n s
0 r C T _ M O
!> Q G P S j t u c f c m g
H T M L 5 _ G e o l o c a t i o n
i> C J I C B M
0 L Q U A N T U M B O T
l> a T A O
> L adver t isement
;> a f g h a n _ m o i
> analyt ics
!> ano
i> a n o n y m i z e r
> _ antivirus
i> L I app
i> i app l ica t ion
l> app l ica t ions
associate d_sessi on
> L backdoor
!> L backdoors
> t z
D- L _ botnet
A
TOP SECRET//COMINT
TOP SECRET STRAPi
ingerprint directories Field Builder
Field Builder
AppID (+Fingerprints)
top ic/w md/iran/ir is I
top ic/w md/iran/ir isl/ed il/chat_body
top ic/w md/iran/
top ic/w md/iran/
top ic/w md/iran/
top ic/w md/iran/
top ic/w md/iran/
top ic/w md/iran/ir is l/ed 13
AppID (+Fingerprints)
r isl/ed il/docurnent_body
risl/edil/email_body
r isl/ed il/filename
r isl/ed il/ur I _path
r isl/ed ¡2
mojahe V
encryption/mo]aheden2
encryption/mo jaheden2/encodedheader
encryption/mo jaheden2/hidden
encryption/mo jaheden2/hidden2
encryptton/mojaheden2/hidden44
encryption/mo jaheden2/secure_file_encoded
encryption/mo jaheden2/securefile
Field Builder
AppID (+Fingerprints)
botnet/black| v
botnet/blackenergybot/command/die
bolnet/blackenergybot/command/flood
bo tne t/b lackenergybo t/co m mand/ic mp
bolnet/blaokenergybot/command/stop
botnet/blackenergybot/command/syn
botriet./blackenergybot/comrnarid/wait
TOP SECRET//COMINT
M
WK———
what's in a name
• If no existing directory makes sense for your fingerprint, you can always create a new one.
TOP SECRET//COMINT
Fingerprints 101: Getting Started
• The first step is to define the name of the fingerprint.
• To do that, follow the syntax below:
fingerprint( encryption/archive/test_new') =
TOP SECRET//COMINT
TOP SECRET STRAPi M
m
Steir-s^S^.vjb U iV VVVW aBB9 ' v . ^ f — ~TFEr
Fingerprints 101: Getting Started
• Note that fingerprint names can not have spaces or any other punctuation other than / which denote directories and _ which can be used in the place of spaces to make fingerprint names easier to read
fingerprint (encryption/archive/test_new?)
TOP SECRET//COMINT
Fingerprints 101 As an example, let s say we want to fingerprint traffic like this:
Met 20D7 : ij fi 1,19?
. J-jiyJ \\iA-s
3M 08:04 ,2003-18-02
g tawab
a
### Begin ASRAR El Mojahedeen v2.0 Encrypted Message ### r/RgTzT/ATRhN2E1 Zjg1 D WQyN WRj M mE2ZTd IN zZmZD h IO D UxZWZh M D Q1 Mj YwMjViZGUO
ZG YwMj d k M m J m NTA4ZD Y2Yj kOM G U2N G N i Yjg5 M z NjZTcS MThjY2 Y1 Zm Y5MTg zZD I kYj hj MTE x0GYzYjc12DdiMDAxNTQzZmVINDVIY2YyMGJjYjU20DkyYjdmYjFjYjAzMWM5ZDQ20WFIMzg
4NThhM2l1 Mjc50DkzZGNhOGRmNWJmNjVIZjQOMjMxNDM4MDIyO Tq1 MmRjM G J iN GN k YTN kYTQ4 Mz MxZj Ri N2 F i Nj 13 Mj E1N GI3 MTA3ZDQ4 N WRm YzMyOTU zZ jZl Mj g3Nj Q10 GQ4 MTA3 N
TU2 N2Zk N2Zj Yz U z Yz Yy Mj FIOD AwN2 VkM2 U6 MTZi N D Y2 M m M2ZTV IYj Q2 YzIO Q G Q2 OD U x W W VkMJI2M WVIND Ay O GlOMThkMTdhNTY1 YzlxMD gy O GZIM2I wZWZJ MDgwM2U4MzNINDg10D U xZTc4 OD c 1MTY2 Fv/t2 IONjU5Zj BhZj Vh Nj kOOTl h N GE x OTh m YWVI N m F IZjly N mM wZD A3 M DM0
NjJkZDhhMm!4ZmRhYjc3NmZINDFk0DkyYjBhY] Y3MD010GVIMj dhYmUv/ZTlyNGlx YmQyZDIz ZjliM2E6ZGQ5NmNhZDQxQTM4NTIQMjc3MzBIOWE\vZWE1 Njk3Yj gxY2ViNTQ1 OWULnoiA/D
U LljTE uD J qn $ OG M RH e s i/8PTnZj Q2y qb mK b F k I Pj wM h $7 FU hF AO w74 S+i+ Po k OREcS X h d P+ y9 /
G ul3ju YTv ri EO x G x2 Os SfN S5 kfR X X H1 DaTn b7 Oyufe9 rfi mM IQ6 e6E0SRUIdU6YVupz0hhgd4Dof
SBbFR30vgOS+pUxDYgmEOr/RA+fYi47tuHQMh+-dynZciQspNdmRUmkiEpFqF03sPHS/1 Oinjqo e 1G sf8+xn52 XE2 q JWd nU+4 XJ W nUI s VN AJ v2n s L+S2TG1 IHbgocmpQoxyOBOSX PcRv J+2 J ek V37 k 1 XyO NZk9YH+• V3 aWYP Xt+y m+wGO X NTqP HI U1J WAZq I2N K/c S Xt9 DMtCtc b8 c z Rj6 G9IX v J9
Eny7t06x P d9B G i o9 M+3 Q ull kZH L E mJ i A v g v B6 R/X/3 wh B q k5zM H QLfo+VJcXS um W5m Rtg Cj z S P W6IZZFCGUB4SK4P XT52ZC0B2 kWD8 VMyNffr lsTG4 X U e s gx47 Hd6 x M L8 wfc pjffZwKN K+EfKI P
==Z1 o w2S A9N3 uL IX B X62 L hOyj/1 i qfJ2 F N R7 Al ON SEj wKo g gVm k x D i uG aQ i+Tu rpxBgatlg
m End ASRAR El Mojahedeen v2.0 Encrypted Message m
Jl iiijioJi^ wU^yi
)c o. I a i iLdblI -•-'•L-JI
I ^•^••J, 1 LklfrJI .
uluJdl ,
^UJI JiU, JI . Itl ..nil cr.fft
.s % i % > c 11 ..,, | .. n [
• IT I . nil
-LlafrJI , -'J'JL1
I4L» JX-OLJL ^JybVpj
_ L-ItxoJI .
JI P- lcfrJI
•¡¿koJj r- lccJI
I - JI.V.II •-oJOJ I
, Lub&JI / „cli3
<U£uJI 0UL0JI
1 teibving 1 Hirlrtei fifths
Fingerprints 101 One thing that could be used to find data like this is the String ASRAR El Mojahdeen V 2 . 0 Encrypted Message
Met 2037 : ij fi 1,19?
. Jsiiyj
3M 03; 04 ,2003-18-02
g t t a w a b
Begii jASRAR El Mojahedeen v2.0 Encrypted MessagfliOT r/RgTzT/ATRhN2E1 Zj g 1 D W Q y N W R j W . - " ^ „ M . v ^ n u m n , tiUtttifilGUO
J«» Jbuo
ZG YwMj d k M m J m NTA4ZD Y2Yj kOM G U2N G N i Yjg5 M z NjZTcS MThjY2 Y1 Zm Y5MTg zZD I kYj hj MTE x OG YzYj c1 ZDd i M D Ax NTQzZmVIN • VIY2 Yy M G Jj Yj U2 OD kyYj dm Y j FjYj Az M WM5ZD Q2 O WFIM z g
4NThhM2l1 Mjc50DkzZGNh0GRmNWJmNjVIZjQQMjMxNDM4MDIy0 Tg1 MmRjM G J iN 6 N k YTN kYTQ4 Mz MxZj Ri N2 F i Nj 13 Mj E1N GI3 MTA3ZDQ4 N YVRm YzMyOTU zZ jZl Mj g3Nj Q10 GQ4 MTA3 N
TU2 N2Zk N2Zj Yz U z Yz Yy Mj FI OD AwN2 VkM2 U6 MTZi N D Y2 M m M2ZTV lYj Q2 YzIO 0 G Q2 OD U x N W VkMJ I2MWVINDAyO G lOMTh k MTd hNTY1 Yzl x M D gyO GZIM2IwZWZJ M D gwM2 U4 Mz NIN D g l O D U xZTc4 OD c 1MTY2 M2 IONjU5^ BhZj Vh Nj kOOTl h N GE x OTh mYWVI N m F IZjlyN mM wZD A3 M DM0
NjJkZDhhMm!4ZmRhYjc3NmZINDFk0DkyYjBhY] Y3MD010GVIMj dhYmUwZTlyNGlx YmQyZDIz Zjl ¡M2E6ZG Q6N m HhZDQ x QTM4 NTIQMjc3 M z BIO WE wZWE 1 Nj k3Yj g x Y2 Vi NTQ10WUL n oi A/D
U LljTE uD J qn e OG M RH e s i/SPTnZj 02y qb mK b F k I Pj wM h e7 FU hF AO w74 S+i+Po k OREo5 X h d P+ y9 /
G ul3ju YTv ri EO x G x2 Os SfN S5 kfR X X H1 DaTn b7 Oyufe9 rS mM IQ6 e6E0SRUIdU6YVupz0hhgd4Dof
S Bb F R3 O v g OS+pU x • Yg m EO r/RA+fY147tu H QM h+dynZq Q sp N d mRUmkj Ep F qF03 s P H S/1 Oi nj q o e 1G stB+xn52 XE2 q/Wd nU+4 XJ WnlJI s VN AJ v2n s L+S2TG11H b go c m p Qo xyOBO SX PcR v J+2 J ek V37 k 1 XyO NZk9YH+• V3 aWYP Xt+y m+wG0 X NTqP HI U1J WAZq I2N K/c S Xt9 DMtCtc b8 c z R]6 G9IX v J9
Eny7t06x P d9B G i o9 M+3 Q ull kZH L E mJ i A v g v B6 R/X/3 wh B q k5zM H QLfo+VJcXS um W5m Rtg Cj z S P W6IZZFCGUB4SK4P XT52ZC0B2 kWD8 VMyNffr lsTG4 X U e s gx47 Hd6 x M L8 wfc pj/fZwKN K+EfKI P
==Z1ow2SA9N3uLIXBX62LhOyj/1iqfJ2FNR7AIONSEjwKoggVmkxDiuGaQi+TurpxBgat1g
m End ASRAR El Mojahedeen v2.0 Encrypted Message m
vil LoJI wLl^yi
)c o. I a i iLdblI -•-'•L-JI
I ^•^••J, I LklfrJI .
uluJdl ,
^UJI JiU, JI . |tl ..nil cr.tfl
.s % i % > c M .. i, I .. n I
• IT I . nil
-LlafrJI , -'J'JL1
I4L» JX-OLJL ILyitbyj
_ L-ItxoJI .
JI P- blccJ I
•¡¿koJj I
I - JI.V.II OJ- LL .oJ'jJ I
, LuL&JI / „cli3
<U£uJI oUioJj
Iteilaving I Hirlrtei fifths
Fingerprints 101: Keywords
• So let s create a fingerprint to tag any data that contains that string
ASRAR El Mojahdeen V2.0 Encrypted Message
TOP SECRET//COMINT
Fingerprints 101: Keywords
First we'd define the fingerprint with a name:
fingerprint( encryption/mojahdeen2?)
TOP SECRET//COMINT
Fingerprints 101: Keywords
• Then, simply put the string in single quotes to denote that XKS needs to look for it as a keyword:
fingerprint(encryption/mojahdeen2?) = 'ASRAR El Mojahdeen V2.0 Encrypted Message'
TOP SECRET//COMINT
Fingerprints 101: Keywords
Finally, all fingerprint definitions need to end with a semi colon to tell XKS that the definition is finished
fingerprint(encryption/mojahdeen2?) = 'ASRAR El Mojahdeen V2.0 Encrypted Message';
TOP SECRET//COMINT
• H * —
Fingerprints 101: Keywords Using the fingerprint GUI on XKS Central, we test to see if this compiles:
Fingerprint Validation / Submittal
Step #1 Step #2 Step #3
y ICompile; 1 Test Against Session Data I d Save tfi Help
Signature
fingerprint (T encryption/moj a h d e e n 2 T ) = T Â S R A R El Mojahdeen v2.0 Encrypted Message7;
S u ccess!
Results
SUCCESS!
Congratulat ions, your f ingerpr int was successfully compiled!
Now use the Test button to run it against the designated session data,
Fingerprints 101 Once checked in, the fingerprint will hit on data like this:
Met 2037 : ij fi 1,19?
. J-jiyJ \\iA-s
3M 08:04 ,2003-18-02
g tawab
### Begin ASRAR El Mojahedeen v2.0 Encrypted Messag r/RgTzT/ATRh N2E1 Zj g 1 O W Q y N W R j t & ^ f l U J t t A t i U t f f i t t
J«» Jbuo uo
ZG YwMj d k M m J m NTA4ZD Y2Yj kOM G U2N G N i Yjg5 M z NjZTcS MThjY2 Y1 Zm Y5MTg zZD I kYj hj MTE x0GYzYjc12DdiMDAxNTQzZmVINDVIY2YyMGJjYjU20DkyYjdmYjFjYjAzMWM5ZDQ20WFIMzg
4NThhM2l1 Mjc50DkzZGNhOGRmNWJmNjVIZjQOMjMxNDM4MDIyO Tq1 MmRjM G J iN GN K YTN kYTQ4 Mz MxZj Ri N2 F i Nj 13 Mj E1N GI3 MTA3ZDQ4 N WRm YzMyOTU zZ jZl Mj g3Nj Q10 GQ4 MTA3 N
TU2 N2Zk N2Zj Yz U z Yz Yy Mj FIOD AwN2 VkM2 U6 MTZi N D Y2 M m M2ZTV iYj Q2 YzIO Q G Q2 OD U x W W VkMJI2M WVIND Ay O GlOMThkMTdhNTY1 YzlxMD gy O GZIM2I wZWZJ MDgwM2U4MzNINDg10D U xZTc4 OD c 1MTY2 M2 IONjU5Zj BhZj Vh Nj kOOTl h N GE x OTh mYWVI N m F IZjly N mM wZD A3 M DM0
NjJkZDhhMm!4ZmRhYjc3NmZINDFk0DkyYjBhY] Y3MD010GVIMj dhYmUv/ZTlyNGlx YmQyZDIz ZjliM2E6ZGQ5NmNhZDQxOTM4NTIQMjc3MzBIOWE\vZWE1 Njk3Yj gxY2ViNTQ1 OWULnoiA/D
U LljTE uD J qn $ OG M RH e s i/8PTnZj 02y qb mK b F k I Pj wM h $7 FU hF AO w74 S+i+ Po k OREcS X h d P+ y9 /
G ul3ju YTv ri EO x G x2 Os SfN S5 kfR X X H1 DaTn b7 Oyufe9 rS mM IQ6 e6E0SRUIdU6YVupz0hhgd4Dof
SBbFR30vgOS+pUxDYgmEOr/RA+fYi47tuHQMh+-dynZciQspNdmRUmkiEpFqF03sPHS/1 Oinjqo e 1G sf8+xn52 XE2 q JWd nU+4 XJ W nUI s VN AJ v2n s L+S2TG1 IHbgocmpQoxyOBOSX PcRv J+2 J ek V37 k 1 XyO NZk9YH+• V3 aWYP Xt+y m+wGO X NTqP HI U1J WAZq I2N K/c S Xt9 DMtCtc b8 c z R]6 G9IX v J9
Eny7t06x P d9B G i o9 M+3 Q ull kZH L E mJ i A v g v B6 R/X/3wh B q k5zM H QLfo+VJcX9 um W5m Rtg Cj z S P W6IZZFCGUB4SK4P XT52ZC0B2 kWD8 VMyNffr lsTG4 X U e s gx47 Hd6 x M L8 wfc pj/fZwKN K+EfKI P
==Z1 o w2S A9N3 uL IX B X62 L hOyj/1 i qfJ2 F N R7 Al ON SEj wKo g gVm k x D i uG aQ i+Tu r p x B g at1 g
m End AS RAR El Mojahedeen v2.0 Encrypted Message m
Jl vii ioji wU^yi
^ o. I i iLdblI .•-'•l-J I
i • ••J, 1 JI . «_»l i ->Jd I ,
^UJI JiU, JI
. |tl ..nil cr.tfl .S % i % > c 11 ..,, I .. n [
. irI. nil
-LlafrJI , -'J'JL1
I4L» JX-OLJL gLy llfV
_ L-ItxoJI . .»¿.¿I
I - JI.V.II •-oJOJ I
, Lub&JI / „cli3
<U£uJI 0UL0JI
1 teibving I t*n< Hirlrtei fifths
TOP SECRET STRAPi
101
As a second example, let s say we want to find data like this:
Us ing T X T f o r m a t t e r
Ref: June 07, 201000803/Q-02135 Islamabad: National Deve 1 o pme nt Co iti pi ex Plot Wo : ^ ^ H Street Mo:
A.
Sector: I s l a m a b a d . Attn:| | AH Purchase SUBJECT : QUOTATION AGAINST YOUR ENQUIRY REF:Purchase of RTV Silicon DATED: 18/05/2010 Dear Sir., With reference to your subject enquiry, we are pleased to enclose our Quotation No: Q-02135-05-567 dated: 07/06/2010/ for your perusal. Please see the 1 Terms of Sale1 attached with our quote for any further details. We hope our offer suits your requirements and we look forward to your valuable purchase order in clue
•
TOP SECRET//COMINT
Fingerprints 101
Look for keywords that could be used to find traffic like th is in the future.
Using T X T f o r m a t t e r
Ref: June 07, 201000803/Q-02135 Islamabad: A.
Deve lo pment C omp 1 e>: Street Mo: Plot Wo:
Sector: Islamabad Attn: AH Purchase SUBJECT : QUOTATION AGAINST YOUR ENQUIRY REF:Purchase of RTV Silicon DATED: 18/05/2010 Dear Sir., With reference to your subject enquiry, we are pleased to enclose our Quotation No: Q-02135-05-567 dated: 07/06/2010/ for your perusal. Please see the 'Terms of Sale1 attached with our quote for any further details. We hope our offer suits your requirements and we look forward to your valuable purchase order in clue
TOP SECRET//COMINT
TOP SECRET STRAPi
101
What if we looked for "National Development Complex" and "Quotation"
T X T f o r m a t t e r
I s l a m a b a d | National D eye 1 o pme n t Co tti pi ex
Sector: Islamabad Attn: AH Pure ha; SUBJECT :^QUOTATION JGAINST YOUR ENQUIRY REF:Purchase of RTV Silicon DATED: 18/05/201* Dear Sit., With reference to your subject enquiry, we are pleased to enclose our Quotation No: Q-02135-05-567 dated: 07/06/2010/ for your perusal. Please see the 'Terms of Sale1 attached with our quote for any further details. Lie hope our offer suits your requirements and we look forward to your valuable purchase order in due
A.
M
WK———
Fingerprints 101: Boolean Logic
• Starting with these two keywords, we'd like to use Boolean Logic to create our new fingerprint
• national development complex • quotation
TOP SECRET//COMINT
TOP SECRET STRAPi
"
Fingerprints 101: Boolean Logic
Again, step one think of a name:
fingerprint( cp/pakistan/agencies/ndc')
TOP SECRET//COMINT
M
WK———
Fingerprints 101: Boolean Logic
• Step two, put single quotes around all keywords:
fingerprint( cp/pakistan/agencies/ndc') = 'National Development Complex' quotation
TOP SECRET//COMINT
TOP SECRET STRAPi
"
Fingerprints 101: Boolean Logic
Use the Boolean operator and
fingerprint( cp/paldstan/agencies/ndc') 'National Development Complex' and quotation
TOP SECRET//COMINT
• Finish the expression with the semi-colon.
fingerprint( cp/paldstan/agencies/ndc') = 'National Development Complex' and quotation ;
TOP SECRET//COMINT
Fingerprints 101: Boolean Logic Use the fingerprint GUI to confirm the fingerprint definition compiles
Fingerprint Validation / Submittal Fingerprint Validation / Submittal
S t e p r f M S t e p # 2 S t e p # 3
j C o m p i l e l 0 T&si A g a i n s t S e s s i o n D a t a [ y j S a v e f l H e l p
Signature
fingerprint(T cp/pakistan/agencies/ndcT) 'national development complex7 and fquotation';
II
ÍU(ü(E
S u c o e s s !
Results
SUCCESS! Congratulations, your f ingerprint was successfully compiled!
Now use the Test button to run it against the designated session data. 1 y j r 1 / / 1
TOP SECRET STRAPi
101
This fingerprint will now successfully find all sessions like this in the future!
Using T X T formatter
Pef: June 07, 201000803/Q-02135 Islamabad: National Development- Complex Plot Mo: Street. Ho: Sector Is lame Attn: AM Purchase SUBJECT : QUOTATION AGAINST YOUR ENQUIRY REF:Purchase of RTV Silicon DATED: 13/0.5/2 010 Dear Sir, With reference to your subject enquiry, we are pleased to enclose our Quotation No: Q-02135-05-567 dated: 07/06/2010, for your perusal. Please see the 'Terms of Sale1 attached with our quote for any further details. tte hope our offer suits your requirements and we look forward to your valuable purchase order in due
TOP SECRET//COMINT
Fingerprints 101 However, how can we account for variations of how the traffic might be seen? Maybe "National Development Complex" will be listed as "NDC". Or maybe instead of a "Quotation" it will be a "Invoice" and etc.
Using T X T formatter
Ref: June 07, 201000803/Q-02135 Islamabad: National Development- Complex Plot Mo Mo : | | Sector: Q | Is 1 arriab ad. Attn:| | AM Purchase SUBJECT : QUOTATION AGAINST YOUR ENQUIRY REF:Purchase of RTV Silicon DATED: 19/ 0.5/ 2 010 Dear Sir, With reference to your subject enquiry, we are pleased to enclose our Quotation Wo: Q-0213 5-05-5 67 dated: 07/06/2010, for your perusal. Please see the 'Terms of Sale1 attached with our quote for any further details. tte hope our offer suits your requirements and we look forward to your valuable purchase order in due
TOP SECRET//COMINT
M
WK———
Fingerprints 101: Boolean Logic
• Keywords can also be grouped together by parentheses to form more complex Boolean logic:
TOP SECRET//COMINT
M
WK———
Fingerprints 101: Boolean Logic
• For example, we can expand on our previous fingerprint like so
fingerprint( cp/pakistan/agencies/ndc') = ('National Development Complex or 'NDC')
and (quotation or 'invoice') ;
TOP SECRET//COMINT
TOP SECRET STRAPi
"
Quick Aside 1: Context Sensitivity
All keywords in X-KEYSCORE are case-insensitive by default.
So in the previous fingerprint 'NDC will match on ride, NdC, nDC etc.
TOP SECRET//COMINT
Quick Aside 1: Context Sensitivity
• If you want to force a keyword to be case sensitive, simply append a c after the single quotes.
• Ex: 'NDCc will only hit when NDC is found in all caps, or 'ndc c will hit only when ndc is found in all lower case and etc.
TOP SECRET//COMINT
Quick Aside 2: Keyword Scanning
• By default keywords in fingerprints can hit in substrings since for example 'ndc' is found within grandchildren.
• So this fingerprint fingerprint( cp/pakistan/agencies/ndc') =
'NDC; Will hit on terms like:
• grandchildren • handcard • handcuffs • etc.
TOP SECRET//COMINT
• In specific cases to avoid false hits you can use the word7
context. • Or force there to be a space on either or both ends of the term
by including them inside the single quotes • So this fingerprint becomes:
' NDC
OR:
fingerprint( cp/pakistan/agencies/ndc) = woid('NDC');
TOP SECRET//COMINT
Let s say that this fingerprint is producing good hits, but it also hitting on spam E~ mails.
fingerprint(cp/pakistan/agencies/ndc?) = ('National Development Complex' or 'NDC')
and (quotation or 'invoice') ;
TOP SECRET//COMINT
•We can use the Boolean and not to defeat unwanted traffic like below:
fingerprint( cp/pakistan/agencies/ndc') = (('National Development Complex' or 'NDC')
and (quotation or 'invoice')) and not (Viagra or 'herbal supplement7);
TOP SECRET//COMINT
TOP SECRET STRAPi M
B-.v
Steir-sS.vjb U iVVVVW aBB9 ' v . ^ f — -T ggF
_ I | , — P
Fingerprints 101: Variables
• Variables allow you to link to a list of keywords. For example, working with this fingerprint, we could create variables to each grouping of terms.
fingerprint( cp/pakistan/agencies/ndc?) = (('National Development Complex or 'NDC) and
(quotation or 'invoice')) and not (Viagra or 'herbal supplement');
TOP SECRET//COMINT
TOP SECRET STRAPi M
m
Steir-sS.vjb U iVVVVW aBB9 ' v . ^ f — ~TFEr
_ — , • — - f f ™ ^ -
Fingerprints 101: Variables
Variables use the same syntax as fingerprints
$NDC_terms = 'National Development Complex' or 'NDC;
$procurement_terms = quotation' or 'invoice'; $spam_defeats = Viagra' or 'herbal supplement';
fingerprint( cp/pakistan/agencies/ndc') = ($NDC_terms and $procurement_terms) and not
$spam_defeats; TOP SECRET//COMINT
TOP SECRET STRAPi M
m
Steir-sS.vjb U • .'VWVVVV aBB9 ' v . ^ f — -T ggF
_ — , • — - f f ™ ^ -
Fingerprints 101: Variables
• Variables can be re-used in multiple fingerprints For example, we could have:
fingerprint (cp/pakistan/agencies/ndc?) = ($NDC_terms and $procurement_terms) and not
$spam_defeats; fingerprint( cp/pakistan/angencies/ndc/testingO = $NDC_terms and (missile launch' or 'tactical radio');
TOP SECRET//COMINT
• • • • • • • •
Fingerprints 101: Variables
• In the future, you can modify the variable $NDC_terms and it will automatically affect both fingerprints since they use that variable in their definition.
TOP SECRET//COMINT
• For example, take the first scenario: "I want to look for documents from Iran that mention a banned item"
• Just using keywords with Boolean equations, how could we restrict the term to only a document body and only coming from Iran?
TOP SECRET//COMINT
Context Sensitive Scanning • X-KEYSCORE s context sensitive scanning engine
allows you to explicitly say where you want a term to hit.
• As an early example, the Tech Strings in Documents capability allowed analysts to restrict terms to only Email, Chat or Documents Bodies
• The full XKS Context Sensitive Scanning engine allows for over 70 unique contexts to be used as part of an fingerprint
TOP SECRET//COMINT
Context Sensitive Scanning • For example, take the first scenario: "I want to look for documents from Iran that mention a banned item"
• Using the XKS context for Country Code (based on NKB information) and the XKS context for Document Bodies, this easily becomes:
fingerprintCdemo/scenarioi') = ccCir') and doc_body('banned item')
TOP SECRET//COMINT
Context Sensitive Scanning As another example, let s say we want to tag all Iphone usage
Using the XKS context for User Agent this easily becomes
fingerprint(<demo/scenario2') = user_agent('iphone');
TOP SECRET//COMINT
TOP SECRET STRAPi M
Sfcir-swCS.vjb AV iVVVVW »»»9 ^ f — i i —
USSID18/HRA Considerations
XKS Fingerprints may not be USSID18 or HRA compliant if they are queried on by themselves
For example, we may want to fingerprint the use of mobile web devices like the I Phone, so that attribute could be used as part of a more complex query.
But querying for the I Phone fingerprint itself would be a USSID18 and HRA violation.
TOP SECRET//COMINT
TOP SECRET STRAPi
• —-— m B i n r ^
USSID18/HRA Considerations
• But if you want to look for an IPhone user from an Iranian Proxy accessing his MaiLru account:
IP Address :
AppID (+Fingerprints) [fülltest] :
Field Builder
AppID (-1-Fingerprints)
browser/ce I Iphorie/iphoriel V
Add to Field C l o s e
Field Builder
AppID (+Fingerprints)
ma i I/web mai I/ma i Ir u ni)))))))))))))))))))))))))))))))))) ni))))))))))))))))))))))))))))))))))
ma i I/web mai I/ma i Ir u
ma i I/web mai I/ma i Ir u/attach ment
ma i l/web ma i I/ma i Ir u/post
V
TOP SECRET//COMINT
Context Sensitive Scanning
What contexts are available for use in XKS Fingerprints?
TOP SECRET//COMINT
HTTP Activity Contexts (1 of 2) html_title(expr) The normalized extracted text web page titles
html titlefhow to' and 'bomb')
http_host(expr) The "Host:" name given in the http header. http_host( yahoo.com')
http_url(expr) Every URL from HTTP GET and POST commands. http_url(7mail/inbox?action=delete')
http_url_a rgs ( expr) All arguments given as part of a URL (ie. all text following the T i n a URL string) http u r 1 ('acti on=de 1 ete')
http_referer(expr) The "Referer:" URL given in the HTTP header http referer(1http://badwebsite/cp?action=show')
http_language(expr) The normalized two letter iso-6393 language code as inferred from any http and or html header info http language('fa or 'de')
TOP SECRET//COMINT
HTTP Activity Contexts (2 of 2)
http_cookie(expr) The "Cookie:" field given in the http header. httpcooki e(/ P R E F=\d\d [a-z] /)
http_server(expr) The "Server:" type name in the http header.
http_server('GWS/2.r or 'Apache')
http_user_agent(expr) The "User-Agent:" field given in the http header.
http_user_agent(/MozillaV[45]/ or 'Chrome')
web_search(expr) The normalized extracted text from web searches
web_search(<ricin' or 'plague')
x_fo rwa rd e d_for (exp r ) The X-Forwarded For IP address from the HTTP Header
x_forwarded_for('i.2.3.4')
TOP SECRET//COMINT
Protocol Contexts 1 of 2 ip(expr) The source or destination IP address of the session
ipf 127.0.0.1')
from_ip(expr) The source IP address of the session fromipC 127.0.0.1')
to_ip(expr) Every URL from HTTP GET and POST commands. to_ip(£ 127.0.0.1')
ip_subnet(expr) IP subnet in CI DR notation. ip_subnet('7.211.143.148/24')
port(expr) The source or destination TCP or UDP port number. port('22')
from_port(expr) The source TCP or UDP port number. from_port('22')
to_port(expr) The destination TCP or UDP port number. to_port('22')
TOP SECRET//COMINT
Protocol Contexts 1 of 2 cc(expr) The country (either to OR from) based on IP address
cc('ir' or 'pk')
from_cc(expr) The source country based on TP address from_cc('ir' or 'pk')
to_cc(expr) The destination country based on IP address to_cc('ir' or 'pk')
protocol(expr) The textual form of the IP next protocol, protocol ('TCP')
next_protocol(expr) The textual form of the IP next protocol. ip_nextjprotocol(' 17')
mac_address(expr) The MAC address of the target network device. macad dress ('00:16:3E:3F:BD:EF')
TOP SECRET//COMINT
TOP SECRET STRAPi
ommunication Based Contexts email_body(expr) The UTF-8 normalized text of all email bodies.
email bodyfhow to' and 'build' and ('bomb' or 'weapon')) chat_body(expr) The UTF-8 normalized text of all chat bodies.
chat_body('how to' and 'build' and ('bomb' or 'weapon'))
document_body(expr) The UTF-8 normalized text of the Office document. -Office documents include (but are not limited to) Microsoft Office, Open Office, Google Docs and Spreadsheets.
document_body('how to' and 'build' and ('bomb' or 'weapon'))
calendar_body(expr) The UTF-8 normalized text of all calendars. An example is Google Calendar. calendar_body(' wedding')
archive_files(expr) Matches a list of files from within an archive. For example is a ZIP file is transmitted, all names of files within are passed to this context. archive_files('bad.dir or 'virus.doc')
http_post_body(expr) The UTF-8 normalized text HTTP url-encoded POSTs. http post body('action=send' and 'badguy@yahoo')
Communication Based Contexts Aliases
doc_email_body(expr) This covers the email body and document_body contexts doc_email_body('how to' and 'build' and ('bomb' or 'weapon'))
communication_body(expr) This covers the email_body, documentbody arid chat_body contexts chat_body('how to' and 'build' and ('bomb' or 'weapon'))
A guide to XKS contexts can be found
TOP SECRET//COMINT
Context sensitivity Why use context-sensitive scanning?
• More intuitive - you can say what you mean • More accurate - if rmaps.google.com' is mentioned in a
blog post, you don't want to try processing it as a Google Maps session
• Better performance for XKEYSCORE
TOP SECRET//COMINT
Examples • "I want to look for people doing web searches on Jihad from
Kabul"
• Using the from_city() and web_search() context this becomes
fingerprint( clemo/scenar^') = from_city(ckabur) and web_search('jihad');
TOP SECRET//COMINT
Examples
• "I want to look for people using Mojahedeen Secrets encryption from an I Phone"
You can even use existing fingerprints in a fingerprint definition! So this becomes:
fingerprint('demo/scenario4') = fingerprint( encryption/mojahdeen2' and fingerprmt('browser/cellphone/iphone')
TOP SECRET//COMINT
4 • "I want to look for E-mails that mention words from various
categories of interest to CP"
• You can use multiple variables in an equation like this:
email_body($acwitems and $acwpositions and ($acwcountries or $acwbrokers or $aewports));
TOP SECRET//COMINT
• $acwitems = 'machine gun' or grenade or 'AK 47' • $acwpositions = 'minister of defence' or 'defense minister' • $acwcountries = 'somalia or liberia or sudan • $acwbrokers = south africa' or Serbia' or Bulgaria' • $acwports = 'rangood' or albasra' or 'dar es salam'
email_body($acwitems and $acwpositions and ($acwcountries or $acwbrokers or $acwports));
TOP SECRET//COMINT
New Fingerprint GUI New XKS Fingerprint GUI allows analysts to directly test, submit and manage fingerprints through the web
Navigation Menu
d F ingerpr in ts
£] Validate /Submit Approved
¡5] Pending f ^ l My Signatures
Uç Fingerprint Validation J Submittal
Step #1 Step #2 Step # 3 I
s o ? Compile Test Against Sess ion Data t d S a v e « Help
* M
Global Variable Declarations
Type or paste any global VARIABLE DECLARATIONS here
Signature
Type or paste a FINGERPRINT definition here.
P r e s s C o m p i l e w h e n d o n e ed i t i ng
J. ) t . 1 A
New Fingerprint GUI New XKS Fingerprint GUI allows analysts to directly test, submit and manage fingerprints through the web
unqerprint validation / submittal
SlfcfJ #1
jCcmpilej
Step ¿2
> Test Against Cession Ca:a
Step #3
Inl Step ¿2
> Test Against Cession Ca:a
Step #3
Inl uouai vanauie ueciaidiioiis
i r e s t = ' b o m b 1 or ' m i s s l e o r T ±ed';
Signature
fingerprint(Ttest/te3tlT) = email body(Stest)r
© Siucbss!
Results
SUCCESS! congratulat ions, your finge-prin: was successfully compilad!
Now use the Test but ton ta run it against the cesignated sassicn deta.
Questions?
TOP SECRET//COMINT
Syntax Rules
• The definition of the fingerprint will look like this:
fingerprintCtest/blah/something', owner =
Note the single quotes needed for the fingerprint name and owner
TOP SECRET//COMINT
Syntax Rules
Secondly every fingerprint definition must be completed by a semi-colon.
fingerprintCtest/blah/something?, owner = 'badguy ;
TOP SECRET//COMINT
Syntax Rules
Variables also must be completed by a semi-
$badguy = 'bomb' or gun or weapon' ;
fingerprintOtest/blah/something', owner = $badguy;
TOP SECRET//COMINT
Syntax Rules
• Definitions and Variables can span multiple lines
$badguy = 'bomb' or gun or weapon ;
fingerprintOtest/blah/something', owner = $badguy;
TOP SECRET//COMINT