elasticsearch query dsl - not just for wizards
Post on 15-Jan-2015
3.948 Views
Preview:
DESCRIPTION
TRANSCRIPT
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Clinton Gormley @clintongormley
Elasticsearch Query DSL … not just for wizards
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
elasticsearch.org/guide
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
elasticsearch
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
elasticsearch• real-time
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
elasticsearch• real-time • distributed
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
elasticsearch• real-time • distributed • search
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
elasticsearch• real-time • distributed • search • analytics
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
mapping
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
mapping
analysis
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
mapping
analysis query dsl
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
mapping
analysis query dsl
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
query dsl
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
query dslflexible, powerful query language
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
queries
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
queries filters
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
queries• relevance
filters• boolean yes/no
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
queries• relevance• full text
filters• boolean yes/no• exact values
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
queries• relevance• full text• not cached
filters• boolean yes/no• exact values• cached
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
queries• relevance• full text• not cached• slower
filters• boolean yes/no• exact values• cached• faster
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
queries• relevance• full text• not cached• slower
filters• boolean yes/no• exact values• cached• faster
Filter first, then query remaining docs
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET /_search
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET /_search{ "query": {...} }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET /_search{ "query": { "match": { "title": "search" }} }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET /_search{ "query": { match_all: {}} }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET /_search{ "query": { match_all: {}} }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET /_search{ "query": { "filtered": { "query": {...}, "filter": {...} } } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET /_search{ "query": { "filtered": { "query": { "match": { "title": "search" }}, "filter": {...} } } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET /_search{ "query": { "filtered": { "query": { "match": { "title": "search" }}, "filter": { "term": { "status": "active" }} } } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET /_search{ "query": { "filtered": { "query": { "match_all": {}}, "filter": { "term": { "status": "active" }} } } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET /_search{ "query": { "filtered": { "query": { "match_all": {}}, "filter": { "term": { "status": "active" }} } } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
how data is indexed
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } !
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } !{ "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } !{ "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" }
where content like “%brown%fox%”
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } !{ "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" }
slow & inflexible
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } !{ "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" }
“analysis”
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } !{ "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "title": [quick,brown,rabbits], "content": [brown,rabbits,are,commonly,seen] } !{ "title": [keeping,pets,healthy], "content": [my,quick,brown,fox,eats,rabbits,on,a, regular,basis] }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term Doc 1 Doc 2aarebasisbrowncommonlyeatsfoxmyonquickrabbitsregularseen
field: content
sorted list of unique terms
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term Doc 1 Doc 2aarebasisbrowncommonlyeatsfoxmyonquickrabbitsregularseen
field: content
where they occur
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term Doc 1 Doc 2aarebasisbrowncommonlyeatsfoxmyonquickrabbitsregularseen
field: content
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term Doc 1 Doc 2aarebasisbrowncommonlyeatsfoxmyonquickrabbitsregularseen
field: content
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
inverted index
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
inverted indexnot just for text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
inverted indexnumbers, dates, bools, enums
geopoints, geoshapes, etc
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE field = "value"
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE field CONTAINS "value"
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE field CONTAINS "value"
term filter
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"term": { "title": "brown" }
WHERE field CONTAINS "value"
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET /_search{ "query": { !!!! } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET /_search{ "query": { "filtered": { !! } } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET /_search{ "query": { "filtered": { "query": { ... }, "filter": { ... } } } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET /_search{ "query": { "filtered": { "query": { "match_all": {} }, "filter": { ... } } } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET /_search{ "query": { "filtered": { "query": { "match_all": {} }, "filter": { ... } } } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET /_search{ "query": { "filtered": { "query": { "match_all": {} }, "filter": { "term": { "title": "brown" }} } } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term Doc 1 Doc 2brownhealthykeepingpetsquickrabbits
field: title
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term Doc 1 Doc 2brownhealthykeepingpetsquickrabbits
field: title
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"term": { "title": "brown" }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"term": { "title": "brown" }
➔ result: bitset[ 1, 0 ]
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"term": { "title": "brown" }
➔ result: bitset[ 1, 0 ]
➔ cache as: "title:brown"
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE field IN ["val",…]
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE field IN ["val",…]
terms filter
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"terms": { "title": ["quick", "pets"] }
WHERE field IN ["val",…]
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term Doc 1 Doc 2brownhealthykeepingpetsquickrabbits
field: title
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"terms": { "title": ["quick","pets"] }
➔ result: bitset[ 1, 1 ]
➔ cache as: "title:quick title:pets"
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE field >= "val1" AND field < "val2"
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE field >= "val1" AND field < "val2"
range filter
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"range": { "content":{ "gte": "a", "lt": "m" } }
WHERE field >= "val1" AND field < "val2"
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term Doc 1 Doc 2aarebasisbrowncommonlyeatsfoxmyonquickrabbitsregularseen
field: content
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
➔ result: bitset[ 1, 1 ]
➔ cache as: "content:[a TO m}"
"range": { "content":{ "gte": "a", "lt": "m" } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"range": { "date":{ "gte": "2014-‐01-‐01", "lt": "2041-‐02-‐01" } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"range": { "date":{ "gte": "2014-‐01-‐01", "lt": "2041-‐02-‐01" } }
numeric/date fields optimised
for range filters
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"range": { "date":{ "gte": "now -‐ 1h" } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"range": { "date":{ "gte": "now -‐ 1h" } }
not cached
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"range": { "date":{ "gte": "now -‐ 1h / h" } }
cached
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE field IS NOT NULL
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE field has any term
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE field has any term
exists filter
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"exists": { "field": "title" }
WHERE field has any term
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
missing filter
WHERE field has no term
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"missing": { "field": "title" }
WHERE field has no term
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
AND … OR … NOT
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
AND … OR … NOT
bool filter
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": [ <filters> ], "should": [ <filters> ], "must_not": [ <filters> ] }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": [ <filters> ], # AND "should": [ <filters> ], "must_not": [ <filters> ] }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": [ <filters> ], "should": [ <filters> ], # OR "must_not": [ <filters> ] }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": [ <filters> ], "should": [ <filters> ], "must_not": [ <filters> ] # NOT }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": { "term": { "title": "rabbits" }}, !!!!!}
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": { "term": { "title": "rabbits" }}, "should": [ { "term": { "title": "quick" }}, { "term": { "content": "quick" }} ], !}
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": { "term": { "title": "rabbits" }}, "should": [ { "term": { "title": "quick" }}, { "term": { "content": "quick" }} ], "must_not": { "term": { "content": "fox" }} }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": [ { "bool": { "should": [ { "term": { "title": "rabbits" }}, { "term": { "content": "rabbits" }} ]}}, { "bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "content": "quick" }} ]}} ], "must_not": { "term": { "content": "fox" }} }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": [ { "bool": { "should": [ { "term": { "title": "rabbits" }}, { "term": { "content": "rabbits" }} ]}}, { "bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "content": "quick" }} ]}} ], "must_not": { "term": { "content": "fox" }} }
not cached
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": [ { "bool": { "should": [ { "term": { "title": "rabbits" }}, { "term": { "content": "rabbits" }} ]}}, { "bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "content": "quick" }} ]}} ], "must_not": { "term": { "content": "fox" }} }
cached
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
result bitset = ! (title:rabbits OR content:rabbits) AND (title:quick OR content:quick) AND NOT content:fox
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
filters• boolean yes/no • exact values • cached • faster
Filter first, then query
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
how relevant is this term?
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
how relevant is this term?term query
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
≈ term filter + relevance
how relevant is this term?term query
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"term": { "title": "brown" }
how relevant is this term?
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET /_search{ "query": { !!! } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET /_search{ "query": { "term": { "title": "brown" } } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "_index": "myindex", "_type": "mytype", "_id": "1", "_score": 0.5, "_source": { "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } }
how relevant is this doc?
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
relevance score
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
relevance scoreHow common is the term in this doc? ➔ more is better
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
relevance scoreHow common is the term in this doc? ➔ more is better How common is the term in ALL docs? ➔ less is better
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
relevance scoreHow common is the term in this doc? ➔ more is better How common is the term in ALL docs? ➔ less is better How long is this doc? ➔ shorter is better
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
lucene similarityHow common is the term in this doc? ➔ more is better How common is the term in ALL docs? ➔ less is better How long is this doc? ➔ shorter is better
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
lucene similarityHow common is the term in this doc? ➔ more is better How common is the term in ALL docs? ➔ less is better How long is this doc? ➔ shorter is better
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term frequency !
How common is the term in ALL docs? ➔ less is better How long is this doc? ➔ shorter is better
lucene similarity
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term frequency !
How common is the term in ALL docs? ➔ less is better How long is this doc? ➔ shorter is better
lucene similarity
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term frequency !
Inverse document frequency !
How long is this doc? ➔ shorter is better
lucene similarity
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term frequency !
Inverse document frequency !
How long is this doc? ➔ shorter is better
lucene similarity
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term frequency !
Inverse document frequency !
Length norm
lucene similarity
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
AND … OR … NOT
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
AND … OR … NOT
bool query
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
like bool filter, but different...
AND … OR … NOT
bool query
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": [ <queries> ], # AND "should": [ <queries> ], "must_not": [ <queries> ] # NOT }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": [ <queries> ], # AND "should": [ <queries> ], # Hmmm "must_not": [ <queries> ] # NOT }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": [ <queries> ], # AND "should": [ <queries> ], # Hmmm "must_not": [ <queries> ], # NOT "minimum_should_match": ? # Hmmm }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
no "must" queries"bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "title": "brown" }}, { "term": { "title": "rabbits" }} ] }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
no "must" queries
at least one must match
"bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "title": "brown" }}, { "term": { "title": "rabbits" }} ] }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
no "must" queries
minimum_should_match = 1
"bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "title": "brown" }}, { "term": { "title": "rabbits" }} ] }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": { "term": { "title": "quick"}}, "should": [ { "term": { "title": "brown" }}, { "term": { "title": "rabbits" }} ] }
with "must" queries
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": { "term": { "title": "quick"}}, "should": [ { "term": { "title": "brown" }}, { "term": { "title": "rabbits" }} ] }
with "must" queries
all are optional!
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": { "term": { "title": "quick"}}, "should": [ { "term": { "title": "brown" }}, { "term": { "title": "rabbits" }} ] }
with "must" queries
minimum_should_match = 0
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
bool filter ➔ T/F !
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
bool filter ➔ T/F !
bool query ➔ _score
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
_score of bool query =
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
_score of bool query = sum( _score of each query)
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
_score of bool query = sum( _score of each query) * num of matching queries
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
_score of bool query = sum( _score of each query) * num of matching queries / num of queries
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
more matching should queries
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
more matching should queries ==
better relevance score
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
trim the long tail"bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "title": "brown" }}, { "term": { "title": "rabbits" }} ] }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
trim the long tail"bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "title": "brown" }}, { "term": { "title": "rabbits" }} ], "minimum_should_match": "75%" }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
trim the long tail"bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "title": "brown" }}, { "term": { "title": "rabbits" }} ], "minimum_should_match": "75%" # 2 of 3 }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
match query
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
match queryhigh level query
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
match queryhigh level query
understands mapping & analysis
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
match query➔ analyze query string ➔ rewrite query
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
one word query
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "match": { "title": "QUICK!" }}
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "match": { "title": "QUICK!" }}
title:quick
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "match": { "title": "QUICK!" }}
title:quick
{ "term": { "title": "quick" }}
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
multi word query
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "match": { "title": "QUICK FOX!" }}
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "match": { "title": "QUICK FOX!" }}
title:quick OR title:fox
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "match": { "title": "QUICK FOX!" }}
title:quick OR title:fox
{ "bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "title": "fox" }} ] }}
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
all words must match
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "match": { "title": "QUICK FOX!" } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "match": { "title": { "query": "QUICK FOX!", ! } } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "match": { "title": { "query": "QUICK FOX!", "operator": "and" } } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "title": "fox" }} ] } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "bool": { "must": [ { "term": { "title": "quick" }}, { "term": { "title": "fox" }} ] } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
trim long tail
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "match": { "title": { "query": "QUICK BROWN FOX!", "minimum_should_match": "75%" } } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "title": "brown" }}, { "term": { "title": "fox" }} ], "minimum_should_match": 2 } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
fuzzy queries
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
fuzzy querieslevenshtein edit distance
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
bron ➔ brown
insertion
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
bron ➔ brown
foxs ➔ fox
deletion
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
bron ➔ brown
foxs ➔ fox
kiuck ➔ qiuck
substitution
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
bron ➔ brown
foxs ➔ fox
kiuck ➔ qiuck ➔ quick
transposition
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "match": { "title": { "query": "KIUCK BRON FOXS!", "fuzziness": "AUTO" } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
phrase / proximity
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "match_phrase": { "title":"QUICK BROWN FOX!" } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "match_phrase": { "title": { "query": "BROWN QUICK FOX!", "slop": "10" } } }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
combine queries
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": { <min_should_match> }, "should": [ { <fuzzy> }, { <proximity> } ] }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": { <min_should_match> }, "should": [ { <fuzzy> }, { <proximity> } ] }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
! "match": { "title": { "query": "<words>", "minimum_should_match": "75%" } } !
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": { <min_should_match> }, "should": [ { <fuzzy> }, { <proximity> } ] }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
! "match": { "title": { "query": "<words>", "fuzziness": "AUTO" } } !
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": { <min_should_match> }, "should": [ { <fuzzy> }, { <proximity> } ] }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
! "match_phrase": { "title": { "query": "<words>", "slop": "10" } } !
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "must": { <min_should_match> }, "should": [ { <fuzzy> }, { <proximity> } ] }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
multi-field queries
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
easy!
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "should": [ { "match": {"first": "Reginald" }}, { "match": {"middle": "Kenneth" }}, { "match": {"last": "Dwight" }} ], "minimum_should_match": "75%" }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
hard!
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "should": [ { "match": { "title": "quick brown fox" }}, { "match": { "content": "quick brown fox" }} ] }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" }
{ "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" }
{ "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" }
better match
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" }
{ "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" }
But 2 matches wins
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
dis_max query
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
all docs which match any query
dis_max query
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
all docs which match any query
dis_max query
_score = best matching query
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool": { "should": [ { "match": { "title": "quick brown fox" }}, { "match": { "content": "quick brown fox" }} ] }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"dis_max": { "queries": [ { "match": { "title": "quick brown fox" }}, { "match": { "content": "quick brown fox" }} ] }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" }
{ "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" }
{ "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"dis_max": { "queries": [ { "match": { "title": "quick brown fox" }}, { "match": { "content": "quick brown fox" }} ], "tie_breaker": 0.2 }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
all docs which match any query
dis_max query
_score = best matching query + tie_breaker * others
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
multi_match query
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
match query on multiple fields
multi_match query
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"dis_max": { "queries": [ { "match": { "title": "quick brown fox" }}, { "match": { "content": "quick brown fox" }} ], "tie_breaker": 0.2 }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"multi_match": { "query": "quick brown fox", "fields": [ "title", "content" ] "tie_breaker": 0.2 }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"multi_match": { "query": "quick brown fox", "fields": [ "title", "content" ] "tie_breaker": 0.2, # "type": "best_fields" }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
best_fields
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
find whole "concept" in one field
best_fields
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"quick brown fox" in title or content
best_fields
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
best_fields
dis_max"quick brown fox" in title or content
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"title": { "type": "string" }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"title": { "type": "string", "fields": { !!!!!!!}}
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"title": { "type": "string", "fields": { "stemmed": { "type": "string", "analyzer": "english" } !!!}}
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"title": { "type": "string", "fields": { "stemmed": { "type": "string", "analyzer": "english" }, "autocomplete": { "type": "string", "analyzer": "edge_ngrams" }}}
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
title: [ brown, fox, jumped ] !
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
title: [ brown, fox, jumped ] !title.stemmed: [ brown, fox, jump ] !
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
title: [ brown, fox, jumped ] !title.stemmed: [ brown, fox, jump ] !title.autocomplete [ b, br, bro, brow, brown, f, fo, fox, j, ju, jum, jump, jumpe, jumped ]
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"multi_match": { "query": "quick brown fox", "fields": [ "title", "title.stemmed", "title.autocomplete" ] "type": "most_fields" }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
most_fields
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
most_fieldsmatch same text
analyzed in different ways
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
most_fieldsmore matching fields = better
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
most_fields
boolmore matching fields = better
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{ "first": "Reginald", "middle": "Kenneth" "last": "Dwight" }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"multi_match": { "query": "Reginald Kenneth Dwight", "fields": [ "first", "middle", "last" ] !}
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"multi_match": { "query": "Reginald Kenneth Dwight", "fields": [ "first", "middle", "last" ] "type": "????" }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"multi_match": { "query": "Reginald Kenneth Dwight", "fields": [ "first", "middle", "last" ] "type": "most_fields" }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"multi_match": { "query": "Reginald Kenneth Dwight", "fields": [ "first", "middle", "last" ] "type": "most_fields" }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
problem
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
problemfield centric
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
( first: Reginald OR first: Kenneth OR first: Dwight ) OR ( middle:Reginald OR middle:Kenneth OR middle:Dwight ) OR ( last: Reginald OR last: Kenneth OR last: Dwight )
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
( first: Reginald OR first: Kenneth OR first: Dwight ) OR ( middle:Reginald OR middle:Kenneth OR middle:Dwight ) OR ( last: Reginald OR last: Kenneth OR last: Dwight )
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
problem
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
problemoperator: and
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
( first: Reginald AND first: Kenneth AND first: Dwight ) OR ( middle:Reginald AND middle:Kenneth AND middle:Dwight ) OR ( last: Reginald AND last: Kenneth AND last: Dwight )
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
( first: Reginald AND first: Kenneth AND first: Dwight ) OR ( middle:Reginald AND middle:Kenneth AND middle:Dwight ) OR ( last: Reginald AND last: Kenneth AND last: Dwight )
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
problem
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
problemterm frequencies
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
problemterm frequencies
first:dwight ➔ common
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
problemterm frequencies
first:dwight ➔ commonlast: dwight ➔ uncommon
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
solution
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
index time solution
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
index time solutionsingle "fullname" field
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"first": { "type": "string" }, "middle": { "type": "string" }, "last": { "type": "string" } !
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"first": { "type": "string" }, "middle": { "type": "string" }, "last": { "type": "string" }, "full": { "type": "string" } !
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"first": { "type": "string", "copy_to": "full" }, "middle": { "type": "string", "copy_to": "full" }, "last": { "type": "string", "copy_to": "full" }, "full": { "type": "string" } !
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"match": { "full": "Reginald Kenneth Dwight", "minimum_should_match": "75%" }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
query time solution
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
query time solutionterm-centric query
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
( first: Reginald AND first: Kenneth AND first: Dwight ) OR ( middle:Reginald AND middle:Kenneth AND middle:Dwight ) OR ( last: Reginald AND last: Kenneth AND last: Dwight )
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
( first: Reginald OR middle:Reginald OR last: Reginald ) AND ( first: Kenneth OR middle:Kenneth OR last: Kenneth ) AND ( first: Dwight OR middle:Dwight OR last: Dwight )
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
blend(first,middle,last):Reginald AND blend(first,middle,last):Kenneth AND blend(first,middle,last):Dwight
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
blend(first,middle,last):Reginald AND blend(first,middle,last):Kenneth AND blend(first,middle,last):Dwight
blends term frequencies
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
cross_fields
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
cross_fieldsquery multiple fields as if they were one
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"multi_match": { "query": "Reginald Kenneth Dwight", "fields": [ "first", "middle", "last" ] "type": "cross_fields" }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"multi_match": { "query": "Reginald Kenneth Dwight", "fields": [ "first", "middle", "last" ] "type": "cross_fields", "minimum_should_match": "75%" }
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
best_fields: whole concept in single field
most_fields: same text, different analyzers
cross_fields: treat multiple fields as one
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
understand the building blocks
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
the rest is details
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
thank you@clintongormley
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
thank you@clintongormley
elasticsearch.org/downloads
elasticsearch.com/support
elasticsearch.com/jobs
top related