more kibana
DESCRIPTION
Introduction about Kibana version3. And something more about my own panels and kibana-auth for it.TRANSCRIPT
![Page 1: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/1.jpg)
{{More}} Kibana
@argv
![Page 2: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/2.jpg)
Who am I
Perl Monger
Author of《网站运维技术与实践》
SRE Architect @sina.com
weibo: @ARGV
github: https://github.com/chenryn
blog: http://chenlinux.com
![Page 3: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/3.jpg)
ELK and IUsing ELK from 2012
<Logstash> at slideshare.net had 18232 views
3.3 & 4.2 chapters in my Book
2 ebook at gitbook.io:
[logstash best practice](https://www.gitbook.io/book/chenryn/logstash-best-practice)
[kibana Chinese Guide](http://kibana.logstash.es/)
kibana fork: <https://github.com/chenryn/kibana>
![Page 4: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/4.jpg)
Kibana Introangular.js(framework) + jquery.flot(visualize) + elastic.js(search)
time-based comparisons
make sense of your data
empower more team members
flexible interface, easy to share
powerful search syntax
easy set up
![Page 5: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/5.jpg)
![Page 6: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/6.jpg)
Kibana layout
dashboard
row != line
panel
timepicker/query/filtering
charts/table/text…
![Page 7: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/7.jpg)
histogram@timestamp basedcount/mean/total
bar/lines/stack/percentselected queries
![Page 8: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/8.jpg)
tablepaging
fields.listhighlightsortable
micro analysis
![Page 9: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/9.jpg)
termsbar/pie/tablemissing/other
donut/legend/lable
![Page 10: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/10.jpg)
TopN query
query termfacets before panel rendering
multi-facet with filtering each term
![Page 11: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/11.jpg)
Dynamic Kibana
logstash.json
http://yourserver/index.html#/dashboard/file/logstash.json?query=status:200&from=7d
logstash.js
http://yourserver/index.html#/dashboard/script/logstash.js?query=status:403,status:404&from=7d
![Page 12: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/12.jpg)
Game Over?
![Page 13: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/13.jpg)
No, We need more!
![Page 14: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/14.jpg)
Range panel
No range panel in kibana3
well, it’s in kibana4 now~
DIY beginning
![Page 15: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/15.jpg)
findsearch range facets in ES doc. √
find pie charts code in Kibana. √
copy terms/, paste to range/. √
change request in module.js. √
change ng-model in editor.html. √
it work. √
![Page 16: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/16.jpg)
module.jsscope.ejs.RangeFacet()
rangefacet.addRange()
rangefacet.field()
request.facet()
scope.ejs.doSearch()
results.then()
![Page 17: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/17.jpg)
editor.html<tr ng-repeat=“value in panel.values”>
<td><input ng-model=“value.from”></td>
<td><input ng-model=“value.to”></td>
</tr>
![Page 18: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/18.jpg)
Range PanelDIY Result
![Page 19: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/19.jpg)
More DIY panelspercentile panel
selectable bettermap providers
queries generate helper
histogram thereshold notification
china map panel
term_stats map panel
statisticstrend panel
multifieldhistogram panel
valuehistogram panel
force panel
export table to CSV
uniq mode for histogram
![Page 20: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/20.jpg)
Percentile Panel
![Page 21: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/21.jpg)
Gaode(高德地图 )
![Page 22: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/22.jpg)
queries generate helper
![Page 23: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/23.jpg)
histogram threshold notification
threshold/anomaly detectionHTML5 notification API
![Page 24: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/24.jpg)
china map
![Page 25: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/25.jpg)
term_stats map
![Page 26: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/26.jpg)
statisticstrend panel
term_stats for trend panel
![Page 27: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/27.jpg)
multifieldhistogram
different histogram setting for each query
for example: A : (B*1000)
![Page 28: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/28.jpg)
valuehistogramdetect the probability distribution of
responsetime
![Page 29: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/29.jpg)
forcemerge from packet beat
![Page 30: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/30.jpg)
uniq histogramuse CardinalityAggregation API
![Page 31: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/31.jpg)
exportAsCsvuse HTML5 Blob feature and file_saver.js
![Page 32: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/32.jpg)
embedded dashboard
show only panel on dashboard, useful to embedded kibana panel into other
systems
![Page 33: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/33.jpg)
snapshot for reportvar page = require('webpage').create();var address = 'http://kibana.dip.sina.com.cn/#/dashboard/elasticsearch/h5_view';var output = 'kibana.png';page.viewportSize = { width: 1366, height: 600 };page.open(address, function (status) { if (status !== 'success') { console.log('Unable to load the address!'); phantom.exit(); } else { window.setTimeout(function () { page.render(output); phantom.exit(); }, 20000); }});
![Page 34: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/34.jpg)
Kibana-AuthExists solutions:
nginx + htpasswd(Kibana3 recommanded)
nodejs + CAS(Community recommanded)
sinatra(Kibana4 used)
authentication VS authorization
![Page 35: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/35.jpg)
my solutionasync transparent proxy
fake `/_nodes` JSON
`kibana-auth` index for cluster and indices authorization
`kibana-int-$user` index for dashboards authorization
Authen::Simple framework
LDAP
kibana-int-a
logstash-abc-
2014.10.25logstash-
opq-2014.10.2
5
kibana-int-b
logstash-xyz-
2014.10.25logstash-
dev-2014.10.2
5
kibana-auth
a:{“server”:”a
”,prefix:[“logstash-
abc”]}b:
{“server”:”b”,prefix:
[“logstash-dev”]}
KbnAuth
3
21
![Page 36: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/36.jpg)
kibana-auth$ curl -XPOST http://127.0.0.1:9200/kibana-auth/indices/sri -
d '{ "prefix":["logstash-sri","logstash-ops"], "route": "/file/dashboard/logstash.json", "server":"192.168.0.2:9200"}'
User “sri“ now can and **ONLY** can access `logstash-sri-yyyy.MM.dd` and `logstash-ops-yyyy.MM.dd` etc stored in `192.168.0.2:9200`
![Page 37: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/37.jpg)
kibana-int-$user
./script/kbnauth migratint sri logstash accesslog php-error
read logstash/accesslog/php-error dashboards’ schema from your original kibana-int index, and write into `kibana-int-sri`
![Page 38: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/38.jpg)
Authen::SimpleAuthen::Simple::ActiveDirectory
Authen::Simple::CDBI
Authen::Simple::DBI
Authen::Simple::FTP
Authen::Simple::HTTP
Authen::Simple::Kerberos
Authen::Simple::LDAP
Authen::Simple::NIS
Authen::Simple::PAM
Authen::Simple::Passwd
Authen::Simple::POP3
Authen::Simple::RADIUS
Authen::Simple::SMB
Authen::Simple::SMTP
Authen::Simple::SSH
![Page 39: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/39.jpg)
Overview{ eshost => 'http://127.0.0.1:9200', hypnotoad => { listen => [‘http://*:80'] }, secret => 'kibana_auth_secret', authen => { LDAP => { host => 'ad.company.com', binddn => '[email protected]', bindpw => 'secret', basedn => 'cn=users,dc=company,dc=com', filter => '(&(objectClass=organizationalPerson)(objectClass=user)(sAMAccountName=%s))' }, Passwd => { path => ‘.htpasswd' } }}
![Page 40: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/40.jpg)
The Last But Not Latest
give a star(34 star now)
give a try(kibana4 still beta now)
give a feedback
If a new user has a bad time, it's a bug in logstash.
![Page 41: More kibana](https://reader033.vdocuments.site/reader033/viewer/2022050818/558ce35ed8b42a50418b45f0/html5/thumbnails/41.jpg)
Thank You!