archiving the mobile web frank mccown, monica yarbrough, & keith enlow computer science dept...

Archiving the Mobile Web

Frank McCown, Monica Yarbrough, & Keith EnlowComputer Science Dept

Harding University

WADL 2013Indianapolis, IN

July 25, 2013

Mobile vs. Stationary Web

Mobile Web-Related Markup Languages

http://en.wikipedia.org/wiki/File:Mobile_Web_Standards_Evolution_Vector.svg

Smartphone era

Two Types of Mobile Web

Feature Phone Web Smartphone Web

cHTML (iMode), WML, WAP, etc.

XHTML, HTML5, etc.

Serving Up Mobile Sites

1. Responsive web design• Same HTML content to desktop and mobile• CSS media queries alter appearance

<!-- CSS media query on a link element --><link rel="stylesheet" media="(max-width: 800px)" href="example.css" />

<!-- CSS media query within a style sheet --><style>@media (max-width: 600px) { .sidebar { display: none; }}</style>

Example of Responsive Web Design

Serving Up Mobile Sites

1. Responsive web design• Same HTML content to desktop and mobile• CSS media queries alter appearance

2. Redirect mobile user agent to mobile site• Client-side redirection• Server-side redirection

Client-Side Redirection• JavaScript detects mobile user agent

// From www.harding.eduvar ua = navigator.userAgent.toLowerCase(); if (queryString.match('version=mobile') || ua.match(/IEMobile|Windows CE|NetFront|PlayStation|like Mac OS Z|MIDP|UP\.Browser|Symbian|

Nintendo|BlackBerry|mobile/i)) { if (!ua.match('ipad')) { if (window.location.pathname.match('.html')) window.location = window.location.pathname.replace('.html', '.m.html'); else window.location = window.location.pathname + 'index.m.html'; }}

Client-Side Redirection

Server-Side Redirection• Server routes mobile user agent to different page

Apache Example:RewriteEngine OnRewriteBase /

RewriteCond %{HTTP_USER_AGENT} (android|bb\d+|meego).+mobile|avantgo|badda\/|blackberry|blazer|etc…|zte\-) [NC]RewriteRule ^$ http://detectmobilebrowser.com/mobile [R,L]

https://developers.google.com/webmasters/smartphone-sites/details

Server-Side Redirection

Serving Up Mobile Sites

1. Responsive web design• Same HTML content to desktop and mobile• CSS media queries alter appearance

2. Redirect mobile user agent to mobile site• Client-side redirection• Server-side redirection

3. User-agent content negotiation• Dynamically serving different HTML for the same URL

User-Agent Content Negotiation

• Server serves up different content for same URL• Use Vary: User-Agent

header in response• Best method for serving content

quickly

Archiving Mobile Sites

1. Responsive web design• Easy: Crawl like normal• Use client tools to view page formatted for mobile

2. Redirect mobile user agent to mobile site• Need to crawl with mobile user agent• Need JavaScript-enabled crawler to handle client-side

redirection

3. User-agent content negotiation• Need to crawl with mobile user agent• Need to distinguish mobile vs. desktop for same URL

How are we doing archiving mobile sites

so far?

Earliest archived

page

Earliest 2007 archived page: WML

Finally some news!

Really???

Great…

Only desktop version is archived!