search intelligently - liferay symposium north america 2016, chicago, usa
TRANSCRIPT
![Page 1: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/1.jpg)
Search IntelligentlyBuilding the Digital Experience with
Liferay DXP, new queries and Elasticsearch
André Ricardo Barreto de Oliveira ("Arbo")Search Infrastructure Lead - Liferay, Inc.
Chicago, USASeptember 27, 2016
![Page 2: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/2.jpg)
Searchin the age of
Digital Transformation
![Page 3: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/3.jpg)
The classic approach to Search
● A tool for content retrieval, and little else
● Analyze - Index - Query - Display - Repeat
● Website finished: "add a Search Bar" as an afterthought
● No real correlation between Search and Business Case
@arbocombr
![Page 4: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/4.jpg)
Meanwhile, in the Search landscape...
● Big Data
● Analytics and Statistics
● Post-text content
● Maps and Geolocation
@arbocombr
● Natural Language Processing
● Bots
● Machine Learning
● Artificial Intelligence
![Page 5: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/5.jpg)
Search meets Digital Transformation
● Well-designed Search, now central to digital business
● A search is often the starting point for the User Journey
● Results with nonlinear interaction- and action
● Impossible to tell apart: Search and the Digital Experience
@arbocombr
![Page 6: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/6.jpg)
TheSearch Centric
Digital Experience
@arbocombr
![Page 7: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/7.jpg)
Business assets designed for Search
@arbocombr
![Page 8: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/8.jpg)
![Page 9: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/9.jpg)
![Page 10: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/10.jpg)
![Page 11: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/11.jpg)
![Page 12: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/12.jpg)
Turn chaotic into manageable
@arbocombr
![Page 13: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/13.jpg)
![Page 14: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/14.jpg)
![Page 15: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/15.jpg)
![Page 16: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/16.jpg)
![Page 17: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/17.jpg)
Find the needle in the haystack
@arbocombr
![Page 18: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/18.jpg)
![Page 19: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/19.jpg)
![Page 20: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/20.jpg)
![Page 21: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/21.jpg)
![Page 22: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/22.jpg)
Predict user intention
@arbocombr
![Page 23: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/23.jpg)
![Page 24: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/24.jpg)
![Page 25: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/25.jpg)
![Page 26: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/26.jpg)
![Page 27: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/27.jpg)
Show what they need to see
@arbocombr
![Page 28: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/28.jpg)
![Page 29: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/29.jpg)
![Page 30: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/30.jpg)
![Page 31: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/31.jpg)
![Page 32: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/32.jpg)
Liferay DXP Demo:What's new in Search
@arbocombr
![Page 33: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/33.jpg)
![Page 34: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/34.jpg)
![Page 35: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/35.jpg)
![Page 36: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/36.jpg)
![Page 37: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/37.jpg)
![Page 38: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/38.jpg)
![Page 39: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/39.jpg)
![Page 40: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/40.jpg)
![Page 41: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/41.jpg)
![Page 42: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/42.jpg)
![Page 43: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/43.jpg)
Liferay DXP Search: Demo Time
New and upcoming features:
building your own Search experience
@arbocombr
![Page 44: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/44.jpg)
Buildingintelligent
queries
@arbocombr
![Page 45: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/45.jpg)
The classic approach to queries
● Full text "bag of words" → all results with same relevance
● Substring (*wildcards*) → performance hit in large indices
● Limited flexibility for special parsing cases (e.g. emails)
● Manual configuration for fields with custom analyzers
@arbocombr
![Page 46: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/46.jpg)
Intelligent queries in Liferay DXP
● Modular OSGi extension points for query builders
● Construct sophisticated, compound queries per field
● Add custom analyzers and type mappings programmatically
● Design searchable assets and fine tune relevance
@arbocombr
![Page 47: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/47.jpg)
Rich data requires tailored search
@arbocombr
TITLE(Short, autocomplete aware)
Description(Lengthy, full text)
Email address(Special formats, symbols)
Geolocation(Coordinates)
![Page 48: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/48.jpg)
Description field style - ready to use
● Classic full text search
● Match by any number of words (or phrases in quotes)
● Proximity: words near each other become top results
@arbocombr
![Page 49: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/49.jpg)
Title field style - ready to use
● Full text search like classic Description- with tweaks
● Autocomplete ready: match to start will boost relevance
● A perfect match to the exact title becomes a top result
● Ignore proximity, since titles are short- best performance
@arbocombr
![Page 50: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/50.jpg)
Substring field style (discouraged)
● Find anywhere in "keyword" field: "[email protected]"
● Not actually analyzed- full scan- kills performance
● Kept for backward compatibility only ("wildcard", "like")
● Use intelligent mappings, analyzers and queries instead
@arbocombr
![Page 51: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/51.jpg)
![Page 52: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/52.jpg)
An intelligent platform
● A well designed User Journey- an increase in Search volume
● Content suggestion- incremental filters- advanced queries
● Liferay DXP: focus on application and content management
● Search engine: external scalability, dynamic advantages
@arbocombr
![Page 53: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/53.jpg)
Elasticsearch:the Liferay DXP
official search engine
@arbocombr
![Page 54: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/54.jpg)
● Match
● Multi Match
● Match All
● Query String
● Term / Terms
Elasticsearch queries for any use case
@arbocombr
● Regexp
● Fuzzy
● Type
● Ids
● DisMax
● REST API for compound queries- Lucene with better syntax
● Range
● Exists
● Missing
● Prefix
● Wildcard
● Geo Distance
● Geo Distance Range
● Geo Bounding Box
● Geo Polygon
● More Like This
![Page 55: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/55.jpg)
Fine tune relevance rapidly...
… then bring it back into your Liferay search
@arbocombr
GET /cars/transactions/_search{ "query" : { "constant_score": { "filter": { "range": { "price": { "gte": 10000 } } } } }, "aggs" : { "single_avg_price": { "avg" : { "field" : "price" } } }}
![Page 56: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/56.jpg)
Similarity suggestion: More Like This
● Liferay DXP: MoreLikeThisQuery
● User viewing blogs, documents, your own custom entities
● Automatically suggest related assets, based on content
● Full text and specific fields
@arbocombr
![Page 57: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/57.jpg)
Massive queries need great scalability
@arbocombr
![Page 58: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/58.jpg)
A seamless transition
@arbocombr
Production modeDev mode
![Page 59: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/59.jpg)
Enterprise-grade security: Shield
@arbocombr
![Page 60: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/60.jpg)
Cluster monitoring: Marvel
@arbocombr
![Page 61: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/61.jpg)
Visualization: Kibana
@arbocombr
![Page 62: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/62.jpg)
Takeaways
@arbocombr
![Page 63: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/63.jpg)
The Digital Experience and Search
● Digital Transformation: not just index-and-find anymore
● Your User Journey will often start with a search
● Effective matches and refinements generate business
● With more user searches, underlying platform must scale
@arbocombr
![Page 64: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/64.jpg)
Liferay DXP: innovations in Search
● Elasticsearch: Lucene at core, improvements at every level
● Enterprise-grade Search with Shield, Marvel and Kibana
● Maximum scalability, decoupled from the DXP footprint
● Flexibility with modular API and OSGi extension points
@arbocombr
![Page 65: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/65.jpg)
Intelligent queries for all use cases
● DXP: many new filters and queries to mix and match
● Ultimate relevance with per-field analyzers, queries, boosts
● Geolocation and more native field type mappings
@arbocombr
![Page 66: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/66.jpg)
Intelligent Search for your User Journey
● Leverage modularity for fast, extensible development
● Small components that modify queries and share results
● Aggregations, filters, boxes, lists, maps, custom UI portlets
● Search Pages tailored to the needs of your business
@arbocombr
![Page 67: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA](https://reader031.vdocuments.site/reader031/viewer/2022020213/5877533c1a28ab84388b6b2f/html5/thumbnails/67.jpg)
Thank you- The Liferay Search Infrastructure Team -
@arbocombr
André de Oliveira ➤ Lead, Engineering (USA)
Tibor Lipusz ➤ SME, Support (Hungary)
Felipe Pires, Vitor Fernandes ➤ Design (Brazil)
Rodrigo Paulino ➤ Back-end (Brazil)
Jonathan Mak, Kevin Tan ➤ Front-end (USA)
Albert Lee, Brian Lee ➤ QA (USA)
Russell Bohl, Rich Sezov ➤ Tech writing (USA)
David Truong, Michael Han ➤ Product Management (USA)
http://j.mp/SearchLiferayNorthAmerica2016
github.com/arboliveira