blogging, googling, syndication: what’s the web coming to?

2005 Professional Excellence Symposium

blogging, googling, syndication: what’s the web coming to?

ian graham

Enterprise IT Strategy & BMO Connect

T: 416.513.5656

E: [email protected]

W: http://www.utoronto.ca/ian/talks/

With thanks to Sherri Bondy for some example weblog sites.

“The race for integration”


what this talk is about

• The culture of web applications– What are the new applications about?– How are they ‘designed’ and put together – Where are they taking the ‘public’ Web (and us!)

• Examples (Lots) – …that help to illustrate the above

• Some conclusions

What the talk is not about– Banking/financial services, software development,

Web Services (tech details), lobsters, rodents.


web culture

• Location

• Connectivity

• Discovery

• Communication & Presence

• Community

• Blogging

• Syndication RSS

• Google

• Jabber, ICQ

• Ebay, Yahoo, Slashdot, Amazon, Blogspots, …

The right side lists just a few example tools & technologies & platforms there are many others …

Concepts Tools / Technologies


a) having a home: blogging

• Short for ‘Weblogging’– A ‘log’ authored on and accessible over the Web

• Simple Web-site generation– Personal (or group) publishing & communications:

• Users often publish regularly (daily, or weekly), and link to friends / colleagues / coworkers

• Links establish virtual ‘communities’ with shared interests• Categorization for topics, etc.

– Software is often free to download (and play with)• Free (or near) hosting available (see list at end)

– Software often supports user-level scripting to building custom functions, behaviors


some example blogs

• http://www.sifry.com/alerts/archives/000298.html [1]– Blog site about blog sites (news, information)

• http://dear_raed.blogspot.com/2003_02_01_dear_raed_archive.html [2]– Reports from inside Iraq before and during 2003 war

• http://blogs.msdn.com/dmassy/ (later)– Internet Explorer development team member

• http://www.nerdgirl.com – A ‘normal’ person (purely social site)

• http://weblogs.mozillazine.org/roadmap/– Inventor of JavaScript – roadmap for development team

• http://www.joelonsoftware.com/– Articles on software development (professional)

• http://www.richardcleaver.com/– Not what you were expecting !!


uses of weblogs

• Social networking– Post musings and share them with friends

• (Semi-) professional information sharing– Writing for richer communication (e.g. news, current

affairs, professional commentary)

• Work-related networking– Write status reports, decisions and analyses, etc, for

sharing / cross-linking with others.

• Popularity?– There are circa 10 million blogsites

– (of course, most are started, and then abandoned!)


“blogspace”

• The collection of blog sites all across the Internet• Unconnected except for hand-created links in the pages


b) two blogspace issues:

1. Letting people know what you have– Publishing information ‘about’ your blog postings

• An advertisement, or summary, about your daily postings.

• RSS -- An XML language for publishing summaries

• RSS stands for Resource Site Summary, or RDF Site Summary, or Really Simple Syndication, or …) [some examples later ]

– RSS is also used by regular web sites, news sites, etc.

2. Finding interesting/relevant stuff– Aggregation sites and desktop clients

• Aggregate RSS info ‘about’ published blogs / other sites

• Headliner clients, topic-specific Web sites

– Categorization and rating systems

• Categorize blog items, Web page/sites

• separate wheat from chaff


b.1) rss feeds from a blogsite

Time-stamped Time-stamped ‘summary’ about ‘summary’ about a blog itema blog item

Metadata (for Metadata (for cataloging, cataloging, indexing)indexing)

Headline textHeadline text A link back to A link back to

the originalthe original

… … An ‘ad’ for the An ‘ad’ for the original content.original content.

<item><dc:creator>Dave Massy [MSFT] </dc:creator><title>Security, Security, Security</title><link>http://blogs.msdn.com/dmassy/archive/2004/06/20/160649.aspx</link><pubDate>Sun, 20 Jun 2004 12:56:00 GMT</pubDate><description><P>This topic seems to be coming up <P> </P><img src ="http://blogs.msdn.com/dmassy/aggbug/160649.aspx" width = "1" height = "1" /> …more text …</description></item>

E.g.: http://blogs.msdn.com/dmassy/ RSS link [3]


b.2) aggregators & syndication

RSS

RSS

RSSRSS

RSS or HTML

Retrieves and categorizes RSS from blog (and other) sitesAnd provides a categorized feed to users, or a Web listing:

http://www.oreillynet.com/meerkat/ an RSS feed aggregator [4] http://www.newzcrawler.com an RSS desktop client [5] http://aggregator.weblogs.co.uk/ a blog-RSS feed aggregator


rss: not just for bloggers

• RSS has been adopted by major media sites as a tool for “syndicating” headlines– http://www.cnn.com/services/rss/ – http://news.bbc.co.uk/

• Blogging has moved beyond ‘cool’– http://news.bbc.co.uk/2/hi/programmes/click_online/

4398243.stm (political and cultural impacts)


api’s for accessing aggregators….

• Programming interfaces to aggregation sites– Simple, easy to use, and URL-based– E.g. Meerkat API --

• http://www.oreillynet.com/meerkat/ [6]• http://www.oreillynet.com/pub/a/rss/2000/05/09/meerkat_api.html

(documentation)

– API based on arguments in the URL (above explains how this works). For example:

• http://www.oreillynet.com/meerkat/?p=1065&_fl=rss10 [6.1]• http://www.oreillynet.com/meerkat/?p=1065&_fl=rss

RSS feed of items in profilecategory 1065

Same, but use different RSS version for feed


and embeddable in web pages:

( . . . demo.html) [6.2] <html>

<head>

<title> Example of RSS feed inclusion using JavaScript </title>

</head>

<body>

<h1> Heading For the Example </h1>

<p> Here is some regular HTML -- ok,

i didn't have time to make it pretty </p>

<p> And here is the JavaScript example code:

<script language="JavaScript" src="http://meerkat.oreillynet.com/?_fl=js">

</script>

</body> </html> Includes script sent by Meerkat; script generates HTML summaries and writes them out as HTML document


summary

• Blogging is for individuals or groups. • RSS (and syndication) help build connectivity, and

communities• Aggregators support simple APIs so it’s easy to build functions

that integrate these tools with others (blogsites, portals, other..) – Richer communities via richer functionality– No conscious high-level architecture: people just try new

ideas, and adopt what works (and drop what doesn’t)– A heterogeneous collection of apps, connected using

simple protocols, APIs.• Each application and protocol is simple (limited scope, easy to

use) and open (encourages reuse, extension)

Key idea: URL as the reference for everything


c. googling

• Google -- A web search engine– http://www.google.com

[7]– Other stuff:

• blogger.com; google groups; news, special search services; maps; gmail (email) …

• Some numbers:– Estimated space > 3,000,000 GB– Number of servers > 100,000 (Linux)

• Access to the index:– Via web pages, and several APIs -- some simple,

some complex


google as a service

• Via APIs that let you reuse the google engine as part of your own application:– A URL-based interface (like Meerkat)

• URL says where to go• Stuff in URL says what you want back

– A full-blown Web services interface• (yes, I know I promised not to mention web

services!)– Both let you, essentially, use Google as the search

engine / database within your own applications (subject to restrictions)


Google’s URL-based API

( . . . demo-google.html) [7.1]

<form id="search" method="get" action="http://www.google.com/custom">

<input type="hidden" name="cof" value="LW:600;LH:58;L:http://www.bmo.com/images/

...banner.gif;GIMP:#cc0000;T:black; ...;AWFID:9262c37cefe23a86;"> <input type="hidden" name="domains" value=“bmo.com"> <input type="hidden" name="sitesearch" value=“bmo.com"> <input type="text" id="q" name="q"

accesskey="s" size="30"> <input type="submit" id="submit" value="go"></form>

----- Info – How it Works -----------------------http://www.google.com/faq_freewebsearch.htmlhttp://cyberzoide.developpez.com/html/google/


google’s web services api

• Experimental interface, but widely used– Requires registration for access key (simple, and

free)– Provides library and tools for building simple

applications, using SOAP-based web services. • Java and .NET examples provided

• API covers– Ways to query the google database– Response formats / error responses


Google Example[7.2]public class GoogleAPIDemo {

public static void main(String[] args) { …

// Create a Google Search object, set our authorization key

GoogleSearch s = new GoogleSearch(); s.setKey(clientKey);

// do search, then print out result

try {

if (directive.equalsIgnoreCase("search")) {

s.setQueryString(directiveArg);

GoogleSearchResult r = s.doSearch();

System.out.println("Google Search Results:");

System.out.println(r.toString());

} else if (directive.equalsIgnoreCase("cached")) {

….

} else if (directive.equalsIgnoreCase("spell")) {

…

} else {

printUsageAndExit(); }

} catch (GoogleSearchFault f) {

System.out.println("The call to the Google Web APIs failed:");

System.out.println(f.toString()); } }

private static void printUsageAndExit() { … } }}


Google Summary

• Simple API focused on single function– Querying the google database.

• Loose coupling – Google doesn’t care what you’re doing.

• Coarse-grained– Here’s a query, and there’s your answer. No

details beyond those needed for business function.

• These two characteristics are core to almost all ‘Web’ development models.


d. jabber: messaging & presence

Jabberserver

Jabberserver

Jabber clients

• Presence• User directory• Proxys to Yahoo, ICQ• Other services

Jabber servers


Jabber Example

Jabberserver

Jabberserver

Jabber client

• Connect register presence

• Lookup user contact database

• Send text message contact database

Jabber client

• Requests and responses all sent in XML

• Generic XML protocol for exchanging messages, plus some services.

• Can be extended to non-text messaging applications

• Now used for virtual presence: lets you know who else is at a blog or other web site

• Requests and responses all sent in XML

• Generic XML protocol for exchanging messages, plus some services.

• Can be extended to non-text messaging applications

• Now used for virtual presence: lets you know who else is at a blog or other web site

Jabber client [8.1]


Virtual Presence(Jabbber extension: http://www.lluna.de/ [8.2])

1. Web Server registers with virtual presence (VP) server, using a URL

2. VP client gets name of VP server (URL) from Web server.

3. VP client registers with VP server.

4. VP server tells VP client who else is there.

5. VP client provides chat service to other users of the site.

VPserverWS 11

VPclient

22 4433

55


Key features

• Simple, single-purpose• URL-based referencing scheme

– Underlying protocol uses XML • Loose coupling between components

– servers, clients – failure is expected• Less-simple protocol than RSS …

– May explain why jabber (and family) isn’t as popular as other technologies.


And many other examples

• Bittorrent - file sharing / transfers– http://www.bittorrent.com

[9]

• 30% of all Internet traffic is generated by BitTorrent

• Wikis -- collaborative web authoring– http://en.wikipedia.org/wiki/Main_Page -- a

collaboratively edited ‘encyclopedia’• Over 500,000 entries (English)

– http://en.wikipedia.org/wiki/Wiki[10]


An example application

• Web page (from web server) that includes data from an aggregator,– uses google to provide searching– publishes new content info using RSS– Uses jabber to provide presence functionality to

visitors– Provides bittorrent links to downloadable data

• All integrated using URLs, and simple web protocols.


Conclusions

• Social focus on community, networking, connection, sharing

• Technical Software design focused on– Simplicity of interfaces, and applications – URL as the key reference for an application,

function, method– Loose coupling between applications

• Construction of Web applications by integrating these different ‘application services’ together.

• Nice essay on this at:– http://www.smallpieces.com/ -- “Small pieces

loosely Joined”


Conclusion IIConclusion II“A large part of how we think about music is

influenced by the methods by with which it has conventionally been distributed. We think of pop songs as being three or four minutes long because 40 years ago that was all that could fit on one side of a vinyl single.”

Moby

We often think of Internet-based computing is the same way -- in terms of what we know -- not what it is becoming. It is different, and challenges our thinking!

Couldn’t be a more fun time to be doing software!


blogging, googling, syndication: what’s the web coming to?

ian graham

Enterprise IT Strategy & BMO Connect

T: 416.513.5656

E: [email protected]

W: http://www.utoronto.ca/ian/talks/

“The race for integration”

--- The End ---


Some Tools and Sites

• Blogging software– http://www.moveabletype.com– http://www.livejournal.org/– http://manila.userland.com/– http://www.slashcode.com/

• Free blogging sites – http://www.blogger.com http://newrandom.blogspot.com (Ian’s site)– http://www.livejournal.com

• RSS / Atom Aggregation sites– http://aggregator.weblogs.co.uk/– http://www.bloglines.com/– http://www.syndic8.com/– http://meerkat.oreilly.com (Open Wire Service)

• API Documentation: http://www.oreillynet.com/pub/a/rss/2000/05/09/meerkat_api.html • RSS / Atom Desktop clients

– http://www.newzcrawler.com– http://www.headlineviewer.com


Some Tools and Sites

• Google tools and APIs – Google Hacks, Calishain and Dornfest, O’Reilly and Associates, 2003

• http://www.oreilly.com/catalog/googlehks/index.html– http://searchenginewatch.com/searchday/article.php/2161301– http://www.google.com/faq_freewebsearch.html (Free websearch)– http://www.google.com/services/custom.html (custom search)– http://cyberzoide.developpez.com/html/google/– http://www.google.ca/apis/ (Google Web service APIs)

• Jabber– http://www.jabber.org– http://www.lluna.de (virtual presence extension)

• Bittorrent– http://bitconjurer.org/BitTorrent/

• Wikis– http://en.wikipedia.org/wiki/Wiki (all about Wikis)

• Java Development Kit– http://java.sun.com/j2se/1.4.2/download.html

blogging, googling, syndication: what’s the web coming to?

Documents

regular web sites

blog sites news

news sites

public web

collection of blog sites

information http

social site http

example weblog sites