Shaping diamonds with ElasticRostislav Aronov, System Architect, Sarine

• Rostislav Aronov• System Architect @ Sarine• More then 25 years in SW industry• From coder via DBA, IT, DevOps to architect• Currently leading key projects at Sarine -

• Cloud connected planning application• Sarine Profile • Others

About Me

On Time, On quality, On budget!

• Public company – Singapore exchange• Market share at manufacturing – more than 80%• Millions of the world’s diamonds are touched by Sarine

technology• CUT grading – ALL major labs – NGTC, GIA, AGS, HRD,

IGI…• Number of employees – 600 World Wide

• Video - https://youtu.be/0PL9BtO-xl4

About Sarine Technologies

About Sarine Technologies

About Sarine Technologies

Mining & Rough Trade Manufacturing Gemological

Reporting Polish Trade Retail

About Sarine Technologies

Perfect StoneHigh Clarity, Colour and Cut

grades Low Clarity Low Colour Low Cut

Sarine ProfileTM

About Sarine Technologies

Sarine Planning application

• Stand-alone (disconnected) desktop application• No usage statistics• No customer feedback• Support done by screen sharing• Application/Client Security only• Applanders - Escaping Appland!

About Sarine Technologies

Sarine – Cloud connected application

Work station (client)

Sarine Application

Back End Service for Data processing

Data StoreSQL-RDS/

DynamoDB/S3

Elasticsearch

LogstashKibana

Lambda Statistic collector

• Application centric logs – requests and results statistic data• .Net Services activity logs – log4net info and error level logs• Client sense logs – requests round trip statistics• Lambda activity logs – info and errors

Sarine – Cloud connected application

• CEO – Now this is 21st century!• PM – We have valuable feedback• CC – It is easy to help our customers• DevOps – We know how to monitor and maintain the production

system• Dev – We have cool technology• And more…

Elastic and Sarine - Business Value

• Elasticsearch cluster with 3 nodes• 125 indices (daily, weekly, monthly)• 268 shards• 400 millions docs• 1 terabyte data• And counting…

Elastic – How we use it?

• Shield• Marvel• Watcher• Head• Kopf• CB – XDCR• Cloud – AWS for S3 backups• Curator

Elastic – How we use it?

• DataZone• POC• Architecture and Sizing• First implementation• Team education

• Elastic subscription• Fine tuning and best practices recommendation• Production monitoring, 24/7 support• Team education

Partners for our Big Data Journey

• More Sarine projects on elastic• Sarine Profile

• Main data store – Couchbase• Main log/history system• Main search system• Main Data visualization system

• Automatic tests• Results collection• Data visualization, reports

• Main monitoring tool• Alerts based on watchers• Main dashboard

Elastic – Is a Strategic Partner

• Use elastic for real-time data collection, analysis and visualization• Use elastic for super fast search as a part of web and desktop

application• Plan index structure (shards, periodic, size)• Plan the cluster size, use at least 3 nodes to avoid split brain• Design index mapping carefully. Use templates, analyzers

according to your needs. Default setting are good enough for general case only!• Educate you team, it is new and not intuitive for most people

technology• Use partners and consulting at least at the begging• Re-indexing is expensive. Avoid it.

Elastic – Recommendations

Q & A

Sarine - Shaping diamonds with Elastic

• Careers at Sarine• Client Ninja / Developer for Presentation Team• Full stack Developers for Cloud Services & Presentation

Teams• Test Engineer to lead QA in Cloud Services team

Sarine - Shaping diamonds with Elastic

Thank You!