1

Web SpamDiscussion paper

Adam Kilgarriff,* Vit Suchomel*^*Lexical Computing Ltd, UK^NLP Laboratory, Masaryk Univ, Brno, Cz

2

The particular Moroccan oil could very well moisturize dry skin handing it out an even makeup including easier different textures.

Now on the web stores are very aggressive price smart so there genuinely isn’t any very good cause to go way out of your way to get the presents (unless of course of program you procrastinated).

Hemorrhoids sickliness is incorrect to be considered as a lethiferous malaise even though shut-ins are struck with calamitous tantrums of agonizing hazards, bulging soreness and irritating psoriasis.

3

Problem

We don’t want it OUP▪ We can’t take examples from your corpus

without checking every one because they might be web spam

More (or, cleverer) than it used to be enTenTen12 vs enTenTen08

Hard to filter (by design) Moving target

4

Definition

Web spamming : “actions intended to mislead search

engines into ranking some pages higher than they deserve”▪ Gyongyi and Garcia-Molina, 2005

5

Moroccan Oil is alcohol-free and has a patented

weightless formula with no build up. Softensthick unmanageable hair and restores shine

andsoftness to dull lifeless hair. Instantly

absorbedinto the hair. Moroccan Oil will help

eliminatefrizz, speeds up styling time by 40%, and

provideslong-term conditioning to all hair types.Are $20 shampoos and conditioners worth

it?Can good hair-care products be found at thedrugstore, or are the expensive salon

products reallysuperior?

6

Content farms

7

Prior work

Taxonomy of techniques Gy¨ongyi and Garcia-Molina, 2005

AIRWEB (Adversarial IR on the web) Five workshops 2005-09▪ Held at WWW conferences

Since2009▪ attention shifted to social-network spam

8

AIRWEB Web Spam Challenge

Best dataset: UK2007 Second challenge Collecting labelling effort▪ 6479 hosts manually classified

6% of data is spam▪ Uniform random sampling of web crawl▪ By domain, not by page

Six participants in challenge▪ All used supervised learning▪ Top score: 85% area under curve

9

Search engines

No search engines, no web spam Lots of data, expertise, algorithms All private

AIRWEB included representatives

BootCaT Piggybacks on search engine anti-spam

methods

10

Linguistics of spam

Level of web page, not host, domain Contra AirWEB

Lacks coherence Text with injections or target terms Anomalous words

11

enTenTen08, enTenTen12

Hypothesis Biggest difference is web spam

Keywords method Frequencies in each corpus Normalise to per million Add smoothing parameter 0.001 to each Divide higher number by lower Two keyword lists Dictionary filter ▪ hunspell

Study top 100 of each

12

Top items

enTenTen12 enTenTen08

lc Freq Freq/mill Freq Freq/mill Score Rank

tweeted 28711 2.2 11 0.0 507.41 1

jewelries 18012 1.4 35 0.0 118.72 2

tweeting 26024 2.0 67 0.0 93.40 3

colorway 6395 0.5 17 0.0 79.69 4

hemorrhoid 57951 4.5 181 0.1 79.29 5

straighteners 28206 2.2 133 0.0 52.20 6

courageousness 8717 0.7 40 0.0 50.86 7

twitter 712447 54.9 3602 1.1 49.81 8

straightener 23324 1.8 137 0.0 41.94 9

colorways 4242 0.3 23 0.0 40.83 10

anticlimaxes 2584 0.2 14 0.0 37.91 11

13

Classification (part 1)NEW THINGS tweeting tweeted twitter (photo) voltaic (cells) atomizer (as part of apparatus for giving up smoking) jailbreak (verb: remove limitations on an Apple device)

NEW WORDS colorway colorways aftereffect (increasingly spelt as one word)

SHOPPING footwear espadrille sneaker slingback huarache handbags holdalls chronograph chronographs timepiece timepieces watchstrap watchmaking birthstone birthstones foodstuff headpins (jewelry making) pantyliner jerseys SERVICES locksmith locksmiths refacing (for kitchen cabinets)

MONEY refinance refinancing remortgages defrayal cosigner loaners

WEDDINGS bridesmaid boutonnieres honeymoons groomsmen

14

Classification (part 2)HEALTH AND BEAUTY periodontist whitening veneers aligners (both mainly for teeth) hemorrhoid hemorrhoids hairstyles straightener straighteners slimming physique cellulite liposuction stretchmarks suntanning moisturize moisturizes moisturized dehydrators detoxing pimples whiteheads blackhead blackheads breakouts (of acne etc) concealer concealers (of acne etc) tinnitus

RARE DICTIONARY WORDS accouter osculate

MORPHOLOGY humorousness severeness sturdiness impecuniousness comfortableness anxiousness adorableness courageousness neglectfulness moldiness safeness anticlimaxes chitchats attires apparels jewelries jackpots wagerer vacationer dandier acquirable conveyable dejecting unexceptionally

NAMES (incorrectly included - most were filtered out) spellbinders (a name) circuital (album) android (operating system)

OTHER frontward proficiently

15

The converse: 2008 keywordsenTenTen08 enTenTen12

lc Freq Freq/mill Freq Freq/mill Score Rank

twelfths 2070 0.6 98 0.0 74.1 1

holograph 3542 1.1 627 0.0 22.0 2

fuehrer 6581 2.0 1220 0.1 21.2 3

declassification 5332 1.6 989 0.1 21.1 4

subtenant 3283 1.0 619 0.0 20.6 5

indemnifying 3654 1.1 772 0.1 18.5 6

libeler 160 0.0 29 0.0 15.4 7

videocassette 3898 1.2 1033 0.1 14.8 8

maunders 188 0.1 39 0.0 14.6 9

palatinates 86 0.0 13 0.0 13.6 10

wardresses 65 0.0 7 0.0 13.6 11

reexports 290 0.1 73 0.0 13.5 12

videodisc 1278 0.4 363 0.0 13.5 13

16

The converse: 2008 keywordsenTenTen08 enTenTen12

lc Freq Freq/mill Freq Freq/mill Score Rank

twelfths 2070 0.6 98 0.0 74.1 1

holograph 3542 1.1 627 0.0 22.0 2

fuehrer 6581 2.0 1220 0.1 21.2 3

declassification 5332 1.6 989 0.1 21.1 4

subtenant 3283 1.0 619 0.0 20.6 5

indemnifying 3654 1.1 772 0.1 18.5 6

libeler 160 0.0 29 0.0 15.4 7

videocassette 3898 1.2 1033 0.1 14.8 8

maunders 188 0.1 39 0.0 14.6 9

palatinates 86 0.0 13 0.0 13.6 10

wardresses 65 0.0 7 0.0 13.6 11

reexports 290 0.1 73 0.0 13.5 12

videodisc 1278 0.4 363 0.0 13.5 13

enTenTen12 enTenTen08

lc Freq Freq/mill Freq Freq/mill Score Rank

tweeted 28711 2.2 11 0.0 507.41 1

jewelries 18012 1.4 35 0.0 118.72 2

tweeting 26024 2.0 67 0.0 93.40 3

colorway 6395 0.5 17 0.0 79.69 4

hemorrhoid 57951 4.5 181 0.1 79.29 5

straighteners 28206 2.2 133 0.0 52.20 6

courageousness 8717 0.7 40 0.0 50.86 7

twitter 712447 54.9 3602 1.1 49.81 8

straightener 23324 1.8 137 0.0 41.94 9

colorways 4242 0.3 23 0.0 40.83 10

anticlimaxes 2584 0.2 14 0.0 37.91 11

17

New things have arrived on the web, more dramatically than things have left it

18

Observations

All except twitter words, voltaic, jailbreak Almost all of sample of concordances▪ spam ▪ Marketing▪ In between

Lethiferous words Osculate (kiss) – porn spam Accouter (dress) – clothing spam

19

Morphology

11 –ness nouns 6 pre-empted▪ comfortableness / comfort

6 plurals 3 emphatically mass nouns▪ Attires apparels jewelries

2 –er nouns▪ Wagerer (exclusively in pure spam) and

vacationer 2 –able adjectives

20

Morphology: hypotheses

Computer is generating these forms Non-native speakers Dialect: Kachru’s outer circle

India Pakisatan Malaysia Kenya Nigeria

21

Looks computer-generated

This, in addendum to modern sedate safeness concerns, numberless increases in data sum total, and rising cost pressures, closest these organizations with some uncommonly outstanding topic challenges.

22

Looks non-native

A lot of individuals choose for numerous floor bunny rabbit cages with brings joining the levels. This grants the bunny rabbit a lot extra room without borrowing more room inside your haven. Owning a line flooring inside your bunny rabbit Cage isn’t a good plan if you wouldlike to give comfortableness for your bunny rabbit. While having a wire bed with a pull out and makes for simpler maintaining, it’s not all of the time necessaryas bunnies are easily litter box trained.

23

Looks Indian

It is dream of every woman to have a perfect wardrobe. The thing that tops the list to make the wardrobe a complete one is a black shoe. Ladies black shoes add style and versatility to the attires. From casuals to formal black is the colour that makes the feet stand out from the crowd.

24

In sum

The web isn’t what it used to be Search engines: friend or foe Filtering it out

AIRWEB challenges Hard, and a moving target▪ if we publish, spammers will smile

The cline From marketing to gibberish From language to noise▪ Content farms▪ Full of sound and fury / signifying nothing