Data Collection

From DreamFactory
Jump to: navigation, search
DFEData Collection


DFE's Data Collection System (DCS) autonomously listens for, and logs, information from any instances deployed in your enterprise. This data is collated and then made available under the Reports tab of the Console.

Data-collection-components.png

The Mighty Mighty ELK

The data storage and visualization engine underneath this all is the combination of three separate pieces of software: Elasticsearch, Logstash, and Kibana. These three together are commonly referred to as the "ELK" stack (first letter of each component). The system listens for data of various types from various sources. This all depends on configuration.

Installing the ELK Stack

The procedure is simple and well documented elsewhere:

Configuration

You must configure the Logstash component to listen for incoming audit packets from the DFE system. These are transmitted as UDP packets from each deployed instance. To tell logstash to listen for these packets,