Install ElastiFlow in Docker

We all know it’s a must to have a fancy Dashboard to monitor your network traffic and stuff like that. I’m going to install ElastiFlow to monitor my network.

It’s pretty easy to get ElastiFlow up and running you can find the repo over at Github robcowart/elastiflow.

There is a docker-compose.yml file already prepared, I changed only a few lines to match my network

elastiflow-elastisearch:
  ...
  restart: 'always'
  volumes:
    - /srv/elastiflow/es:/usr/share/elasticsearch/data
...
elastiflow-kibana:
  ...
  restart: 'always'
  environment:
    SERVER_HOST: 192.168.1.17
elastiflow-logstash:
  ...
  restart: 'always'
  environment:
    ELASTIFLOW_RESOLVE_IP2HOST: 'true' # True for reverse DNS lookup
    ELASTIFLOW_NAMESERVER: '8.8.8.8' # DNS Server for reverse DNS lookup

I only had to chage the persistent storage path for the elastisearch service, and then I changed the SERVER_HOST variable to match my server IP. Then it is optional to set the ELASTIFLOW_RESOLVE_IP2HOST to true, this will enable reverse DNS lookups for all IP addresses. ELASTIFLOW_NAMESERVER is the DNS server logstash will use.

To start the services docker-compose takes care of everything else

docker-compose up -d

Wait a while to let the containers start up.

When everything is up and running you will find the dashbord at http://SERVER_HOST:SERVER_PORT in my case it is http://192.168.1.17:5601

Netflow data will be collected at port 2055.

Disable host-mode networking

I don’t like running docker services in host mode, it does not feel right.

This is my changes to get ElastiFlow up and running using dockers internal networking and port expose.

changes to docker-compose.yml

elastiflow-elastisearch:
  ...
  restart: 'always'
  #network_mode:host
  networks:
    - elastiflow-net
  ports:
    - 9200:9200
  volumes:
    - /srv/elastiflow/es:/usr/share/elasticsearch/data
...
elastiflow-kibana:
  ...
  restart: 'always'
  #network_mode:host
  networks:
    - elastiflow-net
  ports:
    - 5601:5601 # Kibana Dashboard port
  environment:
    SERVER_HOST: 0.0.0.0 # Listen to 0.0.0.0 because we don't know the IP
    ELASTICSEARCH_HOSTS: "http://elastiflow-elasticsearch:9200"
...
elastiflow-logstash:
  ...
  restart: 'always'
  #network_mode:host
  networks:
    - elastiflow-net
  ports:
    - 2055:2055  # Netflow
    - 4739:4739  # IPFIX
    - 6343:6343  # SFlow
  environment:
    ELASTIFLOW_RESOLVE_IP2HOST: 'true' # True for reverse DNS lookup
    ELASTIFLOW_NAMESERVER: '8.8.8.8' # DNS Server for reverse DNS lookup
    ELASTIFLOW_ES_HOST: 'elastiflow-elasticsearch:9200'

Setup Kibana dashboards

First we need to download the ndjson configuration file from the ElastiFlow github. In the subfolder, Kibana, there is a file called elastiflow.kibana.7.8.x.ndjson. Download this file and save it somewhere, then open the kibana settings in a browser, http://<server-ip>:5601/

Installing elastiflows dashboards

When you have opened Kibana, browse over to Stack Management-> Kibana/Saved objects. Click on Import in the top right and select the elastiflow.kibana.7.8.x.ndjson we downloaded. This will load all the pre-built Elastiflow dashboards into Kibana.

Create Index lifcycle policy

When I have done all the above, the elastiflow- index under Index management is missing a life cycle policy. This will just say how much data to save, by size and age. Under the Index Lifcycle Policies we can create a new lifecycle policy called elastiflow. If you call the policy something else, you need to change the policy for the elastiflow- index.