This sample demonstrates using Packetbeat with Elasticsearch and Alerting to detect DNS tunnels using the number of unique FQDNs per domain as an indicator of compromise.

For a detailed walk-through of the watch and the aggregations used here, see the Elastic blog post titled Detecting DNS Tunnels with Packetbeat and Watcher.

The above blog utilises an earlier version of the Elastic Stack. This sample has been updated to reflect Elastic 5.0 and thus uses X-Pack with Alerting, rather than Watcher. Principles are configuration remain largely the same. Key Changes include:

• Use of Painless scripting instead of Groovy
• Installation of X-Pack plugin rather than Watcher
• Update of configuration options and API endpoints to reflect 5.0

1. Download and extract Packetbeat.

   # Use the appropriate download link for your OS and architecture.  Assumes use of 5.x.
   $ curl -O https://download.elastic.co/beats/packetbeat/packetbeat-<version>-<arch>.tgz
   $ tar xf packetbeat-*.tgz
   $ mv packetbeat-<version>-<arch> packetbeat

2. Download and install Elasticsearch.

   Follow the Installation & Setup Guide to install and test the elastic stack (you can skip this step if you already have a working installation of the Elastic Stack) Kibana and Logstash are not required for this example.

3. Install the Elastic Stack X-Pack containing Alerting (you can try them for 30 days).

   $ elasticsearch/bin/elasticsearch-plugin install x-pack

4. In addition to Alerting, this X-Pack will install shield thus requiring security configuration. For simplicity, this should be disabled for this demo via:

   $ echo 'xpack.security.enabled: false' >> elasticsearch/config/elasticsearch.yml

   The example additionally uses painless scripts which require regex support, which should be enabled through:

   $ echo 'script.painless.regex.enabled: true' >> elasticsearch/config/elasticsearch.yml

5. Install the Painless scripts

   $ cp *.painless elasticsearch/config/scripts

   The watch uses both and inline and file based script for purposes of example. The scripts will differ from the above blog post due to the use of the 5.0 painless scripting language.

6. Start Elasticsearch

   $ elasticsearch/bin/elasticsearch

7. From a new terminal install the copy the customized template packetbeat-dns.template.json for Packetbeat. This enhances the default template by using a custom analyzer for the dns.question.name field and will be installed by Packetbeat at run time.

   $ cp packetbeat-dns.template.json <packetbeat base directory>/packetbeat.template.json

8. Index the DNS tunnel data from the PCAP file.

   # Set the timestamps in the PCAP to the current time. The timestamp of the
   # last packet is 1282356664 seconds since epoch.
   $ offset=$(($(date +"%s") - 1282356664))
   $ editcap -t +${offset} dns-tunnel-iodine.pcap dns-tunnel-iodine-timeshifted.pcap
   $ ./packetbeat/packetbeat -e -v -waitstop 10 -t -I dns-tunnel-iodine-timeshifted.pcap
   # Verify that data was indexed:
   $ curl http://localhost:9200/packetbeat-*/_count?pretty

9. Copy the provided config file to to the base directory. Be sure to backup any existing configuration files. You may also need to change the interface monitored based on your environment using the parameter 'packetbeat.interfaces.device'

   $ cp packetbeat.yml <packetbeat base directory>/packetbeat.yml

10. Index DNS traffic from your own machine.

    # Set the interface that you wish to monitor in packetbeat.yml
    $ ./packetbeat/packetbeat -e -v -d "dns"

11. From a new terminal make some DNS requests

    $ nslookup www.google.com
    $ nslookup www.yahoo.com

12. Execute the watch. This does not install the watch, it only executes it. This allows you to make changes to the watch and easily retest.

    $ curl -XPUT http://localhost:9200/_watcher/watch/_execute?pretty -d@unique_hostnames_watch.json

13. Verify the output

    ...
    "condition" : {
      "type" : "script",
      "status" : "success",
      "met" : true
    },
    "transform" : {
      "type" : "script",
      "status" : "success",
      "payload" : {
        "alerts" : {
          "pirate.sea." : {
            "total_requests" : 212,
            "unique_hostnames" : 211,
            "total_bytes_in" : 14235.0,
            "total_bytes_out" : 35212.0,
            "total_bytes" : 49447.0
          }
        }
      }
    },
    ...