logstash, elasticsearch, kibana
TRANSCRIPT
ELK Cecil
EasticSearch: Logstash: Kibana: ElasticSearch Viszualazation Tool
ServerLogs
ELK? ( )
Logstash (input -> filter -> output)
input: filter: (, )output:
Logstash ./logstash -f configfile : input, filter, output input { // Input pluginfile {path => ../apiTrans.log}}filter { // filter plugingrok {match => {message => %{COMBINEDAPACHELOG}}}geoip {source => [clientip]"}}output { // output pluginelasticsearch {hosts => ["http://localhost:9200" ]}}
ElasticSearch (full-text) / ( , , )
, , , , ,JSON , Restful API Features Elasticsearch
ELK curl -XPUT 'localhost:9200/customer/external/1?pretty' -d '{ "name": "John Doe"}'curl -XGET 'localhost:9200/customer/external/1?pretty'{ "_index" : "customer", "_type" : "external", "_id" : "1", "_version" : 1, "found" : true, "_source" : { "name": "John Doe" }}
RDBDatabaseTableRowColumnSchemaElastic SearchIndexTypeDocumentFieldMapping
RDB : : Mapping
Kibana (with ElasticSearch)(Elastic Search )
Discover: Elasticsearch Visualize: Dashboard: 2
VisualizeElasticsearch aggregation Bucket: metric:
MetricBucket{ "query": {} , "aggs" : { "grades_stats" : { "stats" : { "field" : "grade" } } }}{ "query": {} , "aggs" : { "genders" : { "terms" : { "field" : "gender" } } }}{ ... "aggregations": { "grades_stats": { "count": 6, "min": 60, "max": 98, "avg": 78.5, "sum": 471 } }}{ ... "aggregations" : { "genders" : { ... "buckets" : [{ "key" : male", "doc_count" : 10 },{ "key" : female", "doc_count" : 10 }, ] } }}
Production Broker Elastic Search / .ex) syslog
Redis Production Environment
Referenceselastic, https://www.elastic.co