saving the world through the wonder that is >>> cql

Saving the world throughthe wonder that is

>>> CQL <<<Mike Taylor <[email protected]>

Index Data – http://www.indexdata.com

With a little help from BMNH R5937 “The Archbishop”

mailto:[email protected]

http://www.indexdata.com/

Saving the world through CQL Mike Taylor, Index Data <[email protected]>

Overview

Where we're headed:

Why OpenSearch rules so much


Overview

Where we're headed:

Why OpenSearch rules so muchWhy OpenSearch sucks so hard


Overview

Where we're headed:

Why OpenSearch rules so muchWhy OpenSearch sucks so hardCQL saves the world!


Overview

Where we're headed:

Why OpenSearch rules so muchWhy OpenSearch sucks so hardCQL saves the world!Why CQL is hard to implement


Overview

Where we're headed:

Why OpenSearch rules so muchWhy OpenSearch sucks so hardCQL saves the world!Why CQL is hard to implementWhy CQL is easy to implement


Overview

Where we're headed:

Why OpenSearch rules so muchWhy OpenSearch sucks so hardCQL saves the world!Why CQL is hard to implementWhy CQL is easy to implementNow what?



OpenSearch is great because:It gives clients a well-defined way to request searchesIt returns data in a well-known format

OpenSearchMetasearcher

OpenSearchServer 1

Nice, coherent XML





OpenSearchServer 1

OpenSearchServer 2

Nice, coherent XML Same XML





OpenSearchServer 1

OpenSearchServer 3

OpenSearchServer 2

Nice, coherent XML Same XML Same XML again!


Why OpenSearch sucks so hard


OpenSearchServer 1

OpenSearchServer 3

OpenSearchServer 2

Ghasty proprietaryquery language #1




OpenSearchServer 1

OpenSearchServer 3

OpenSearchServer 2


Revolting proprietaryquery language #2




OpenSearchServer 1

OpenSearchServer 3

OpenSearchServer 2



Ethically horrifying,foul and totallyincomprehensiblequery language #3




OpenSearchServer 1

OpenSearchServer 3

OpenSearchServer 2



Ethically horrifying,foul and totallyincomprehensiblequery language #3

!!!!!!

Aaaargh!Aaaargh!

!!!!!!

Mike Taylor, Index Data <[email protected]>

CQL saves the world!

Saving the world through CQL

OpenSearchSRUMetasearcher

SRUServer 1

SRUServer 3

SRUServer 2

Lovely CQL





SRUServer 1

SRUServer 3

SRUServer 2

Lovely CQLLovely CQL





SRUServer 1

SRUServer 3

SRUServer 2

Lovely CQL Lovely CQLLovely CQL





SRUServer 1

SRUServer 3

SRUServer 2


: –)


CQL saves the world: see for yourself


EuropeanLibrary

SRUServer 1

SRUServer 3

SRUServer 2


Talk to Theo about thishttp://www.###


What if ...



OpenSearchServer 1

OpenSearchServer 3

OpenSearchServer 2

Lovely CQLEven morelovely CQL!Yum!

Lovely CQL


OK, what actually is CQL?


dinosaur




dinosaurdinosaur and vertebra




dinosaurdinosaur and vertebradinosaur or pterosaur




dinosaurdinosaur and vertebradinosaur or pterosaurvertebra not dorsal




dinosaurdinosaur and vertebradinosaur or pterosaurvertebra not dorsal(dinosaur or pterosaur) and (vertebra not dorsal)




dinosaurdinosaur and vertebradinosaur or pterosaurvertebra not dorsal(dinosaur or pterosaur) and (vertebra not dorsal)title=churchill




dinosaurdinosaur and vertebradinosaur or pterosaurvertebra not dorsal(dinosaur or pterosaur) and (vertebra not dorsal)title=churchillcreator=churchill




dinosaurdinosaur and vertebradinosaur or pterosaurvertebra not dorsal(dinosaur or pterosaur) and (vertebra not dorsal)title=churchillcreator=churchillsubject = (dinosaurs and biomechanics)




dinosaurdinosaur and vertebradinosaur or pterosaurvertebra not dorsal(dinosaur or pterosaur) and (vertebra not dorsal)title=churchillcreator=churchillsubject = (dinosaurs and biomechanics)

Hey! What's the big deal? This is easy!




People of a faint disposition might like to look away now.





dc.author=(kern* or ritchie) and (bath.title exact "the c programming language" or dc.title=elements prox/distance=4 dc.title=programming) and subject any/relevant "style design analysis"

(Yes, that's all one query.)





dc.author=(kern* or ritchie) and (bath.title exact "the c programming language" or dc.title=elements prox/distance=4 dc.title=programming) and subject any/relevant "style design analysis"

(Yes, that's all one query.)

Don't try this at home, kids. Get JavaScript to build it for you.


The CQL learning curve


Power of query that can be expressed

Eff

ort

in learn

ing q

uery

lan

guag

e

SQL





Eff

ort

in learn

ing q

uery

lan

guag

e

SQL

Google





Eff

ort

in learn

ing q

uery

lan

guag

e

SQL

Google

CQL


Why CQL is hard to implement


Are you kidding? Just look at all that power:

multi-word terms with wildcards and anchoringindexes (creator, title, subject ...)index prefixes (dc for Dublin Core, rec for Record Metadata)relations (< <= => > = <> exact all any)relation modifiersbooleans (and, or, not, prox)boolean modifiers (ouch, ouch, ouch)prefix mapping ... and much, much more!


Why CQL is easy to implement


How do you decode CGI parameters?– use a library!

How do you parse XML?– use a library!

How do you send an HTTP request and get the response?– use a library!


Why CQL is easy to implement


How do you decode CGI parameters?– use a library!

How do you parse XML?– use a library!

How do you send an HTTP request and get the response?– use a library!

Now, who can guess how to parse CQL?


Why CQL is easy to implement (Java)


import org.z3950.zing.cql;

CQLParser parser = new CQLParser();CQLNode root;try {

root = parser.parse(cql);} catch (CQLParseException ex) {

System.err.println("Syntax error: " + ex.getMessage());}


Why CQL is easy to implement (Java)


import org.z3950.zing.cql;

CQLParser parser = new CQLParser();CQLNode root;try {

root = parser.parse(cql);} catch (CQLParseException ex) {

System.err.println("Syntax error: " + ex.getMessage());}

Super-easy!Super-easy!


Why CQL is easy to implement (Perl)


use CQL::Parser;

my $parser = new CQL::Parser();my $node;eval {

$node = $parser->parse($query);}; if ($@) {

die "CQL syntax error: $@";}


Why CQL is easy to implement (Perl)


use CQL::Parser;

my $parser = new CQL::Parser();my $node;eval {

$node = $parser->parse($query);}; if ($@) {

die "CQL syntax error: $@";}

EmbarrassinglyEmbarrassingly

Trivial!Trivial!


Why CQL is easy to implement (C)


#include <yaz/cql.h>

CQL_parser cp = cql_parser_create();struct cql_node *root;r = cql_parser_string(cp, query);if (r) {

fprintf(stderr, "Syntax error\n");} else {

root = cql_parser_result(cp);}


Why CQL is easy to implement (C)


#include <yaz/cql.h>

CQL_parser cp = cql_parser_create();struct cql_node *root;r = cql_parser_string(cp, query);if (r) {

fprintf(stderr, "Syntax error\n");} else {

root = cql_parser_result(cp);}

A child of threeA child of three

could do it!could do it!


Why CQL is easy to implement (Intercal)


DO (5) NEXT (5) DO FORGET #1 PLEASE WRITE IN :1 DO .1 <- 'V-":1~'#32768c/#0'"c/#1'~#3 DO (1) NEXT DO :1 <- "'V-":1~'#65535c/#0'"c/#65535' ~'#0c/#65535'"c/"'V-":1~'#0c/#65535'" c/#65535'~'#0c/#65535'" DO :2 <- #1 PLEASE DO (4) NEXT (4) DO FORGET #1 DO .1 <- "V-':1~:2'c/#1"~#3 DO :1 <- "'V-":1~'#65535c/#0'"c/":2~'#65535 c/#0'"'~'#0c/#65535'"c/"'V-":1~'#0


Why CQL is easy to implement (Intercal)


DO (5) NEXT (5) DO FORGET #1 PLEASE WRITE IN :1 DO .1 <- 'V-":1~'#32768c/#0'"c/#1'~#3 DO (1) NEXT DO :1 <- "'V-":1~'#65535c/#0'"c/#65535' ~'#0c/#65535'"c/"'V-":1~'#0c/#65535'" c/#65535'~'#0c/#65535'" DO :2 <- #1 PLEASE DO (4) NEXT (4) DO FORGET #1 DO .1 <- "V-':1~:2'c/#1"~#3 DO :1 <- "'V-":1~'#65535c/#0'"c/":2~'#65535 c/#0'"'~'#0c/#65535'"c/"'V-":1~'#0

So easy,So easy,

it hurts!it hurts!


Now what?


Once you've got your CQL parse tree ...

Walk the treeConvert it to your database's local query languageIf a CQL construct is unsupported, throw a diagnostic


Now what?


Once you've got your CQL parse tree ...

Walk the treeConv ert it to your database's local query languageIf a CQL construct is unsupported, throw a diagnostic

It doesn't matter if you don't implement the whole language(No-one does that.) The goal is:

“if you can say it in your DB, you can say it in CQL.”


Boldly facing a bright new dawn


It really and truly is not that big a deal to implement CQL.





Winners:Content providers: their databases get used





Winners:Content providers: their databases get usedVendors: they can concentrate on the interesting problems





Winners:Content providers: their databases get usedVendors: they can concentrate on the interesting problemsLibraries: they don't need expensive metasearch solutions





Winners:Content providers: their databases get usedVendors: they can concentrate on the interesting problemsLibraries: they don't need expensive metasearch solutionsUsers: they find more good stuff!





Winners:Content providers: their databases get usedVendors: they can concentrate on the interesting problemsLibraries: they don't need expensive metasearch solutionsUsers: they find more good stuff!

Losers:No-one!

Saving the world throughthe wonder that is

>>> CQL <<<Mike Taylor <[email protected]>

Index Data – http://www.indexdata.com

With a little help from BMNH R5937 “The Archbishop”, an as-yet undescribed brachiosaurid sauropod from the Upper Jurassic of the Tendaguru formation, Tanzania, Africa.

mailto:[email protected]

http://www.indexdata.com/

saving the world through the wonder that is >>> cql

Documents

index data cql

opensearch rules

index data overviewwhere

world throughthe

muchwhy opensearch sucks

lovely cqlmike taylor

implementwhy cql

welldefined way