Technologies for Logging via the Network
Microservices can support central logging by sending log data directly via the network. Most log libraries support such an approach. Special protocols like GELF (Graylog Extended Log Format)1 can be used for this or long-established protocols like syslog, which is the basis for logging in UNIX systems. Tools like the logstash- forwarder,  Beaver, or Woodchuck are meant to send local files via the network to a central log server. They are sensible in cases where the log data is supposed to be also locally stored in files.
ELK for Centralized Logging
Logstash, Elasticsearch, and Kibana can serve as tools for the collection and processing of logs on a central server (see Figure 11.1). These tools form the ELK stack (Elasticsearch, Logstash, Kibana).
- • With the aid of Logstash log files can be parsed and collected by servers in the network. Logstash is a very powerful tool. It can read data from a source, modify or filter data, and finally write it into a sink. Apart from importing logs from the network and storage in Elasticsearch, Logstash supports many other data sources and data sinks. For example, data can be read from message queues or databases or written into them. Finally, Logstash can also parse data and supplement it—for example, time stamps can be added to each log entry, or individual fields can be cut out and further processed.
- • Elasticsearch stores log data and makes it available for analyses. Elasticsearch cannot only search the data with full text search, but it can also search in individual fields of structured data and permanently store the data like a database. Finally, Elasticsearch offers statistical functions and can use those to analyze
Figure 11.1 ELK Infrastructure for Log Analysis
data. As a search engine Elasticsearch is optimized for fast response times so that the data can be analyzed quasi-interactively.
• Kibana is a web user interface that enables analysis of data from Elastic- search. In addition to simple queries, statistical evaluations, visualizations and diagrams can be created.
All three tools are open source projects and are available under the Apache 2.0 license.
-  https://www.graylog.org/
-  https://github.com/elastic/logstash-forwarder
-  https://github.com/python-beaver/python-beaver
-  https://github.com/danryan/woodchuck
-  https://www.elastic.co/products/logstash
-  https://www.elastic.co/products/elasticsearch
-  https://www.elastic.co/products/kibana