Get metrics from Fluentd to:

• Visualize Fluentd performance.
• Correlate the performance of Fluentd with the rest of your applications.

The Fluentd check is included in the Datadog Agent package, so you don't need to install anything else on your Fluentd servers.

Edit the fluentd.d/conf.yaml file, in the conf.d/ folder at the root of your Agent's configuration directory to start collecting your FluentD metrics and logs. See the sample fluentd.d/conf.yaml for all available configuration options.

In your fluentd configuration file, add a monitor_agent source:

<source>
  @type monitor_agent
  bind 0.0.0.0
  port 24220
</source>

1. Add this configuration block to your fluentd.d/conf.yaml file to start gathering your Fluentd metrics:

     init_config:
   
     instances:
       - monitor_agent_url: http://localhost:24220/api/plugins.json
         #tag_by: "type" # defaults to 'plugin_id'
         #plugin_ids:    # collect metrics only on your chosen plugin_ids (optional)
         #  - plg1
         #  - plg2

   See the sample fluentd.d/conf.yaml for all available configuration options.

2. Restart the Agent to begin sending Fluentd metrics to Datadog.

As long as you can forward your FluentD logs over tcp/udp to a specific port, you can use that approach to forward your FluentD logs to your Datadog agent. But another option is to use the Datadog FluentD plugin to forward the logs directly from FluentD to your Datadog account.

Proper metadata (including hostname and source) is the key to unlocking the full potential of your logs in Datadog. By default, the hostname and timestamp fields should be properly remapped via the remapping for reserved attributes.

Add the ddsource attribute with the name of the log integration in your logs in order to trigger the integration automatic setup in Datadog. Host tags are automatically set on your logs if there is a matching hostname in your infrastructure list. Use the ddtags attribute to add custom tags to your logs:

Setup Example:

  # Match events tagged with "datadog.**" and
  # send them to Datadog

<match datadog.**>

  @type datadog
  @id awesome_agent
  api_key <your_api_key>

  # Optional
  include_tag_key true
  tag_key 'tag'

  # Optional tags
  dd_source '<INTEGRATION_NAME>'
  dd_tags '<KEY1:VALUE1>,<KEY2:VALUE2>'
  dd_sourcecategory '<SOURCE_CATEGORY>'

</match>

Additional parameters can be used to change the endpoint used in order to go through a proxy:

• host: Proxy endpoint when logs are not directly forwarded to Datadog (default value is intake.logs.datadoghq.com)
• port: Proxy port when logs are not directly forwarded to Datadog (default value is 10514)
• ssl_port: Port used when logs are forwarded in a secure TCP/SSL connection to Datadog (default is 10516)
• use_ssl: If true, the Agent initializes a secure TCP/SSL connection to Datadog. (default value is true)

This also can be used to send logs to Datadog EU by setting:

<match datadog.**>

  ...
  host 'tcp-intake.logs.datadoghq.eu'
  ssl_port '443'

</match>

Datadog tags are critical to be able to jump from one part of the product to another. Having the right metadata associated with your logs is therefore important in jumping from a container view or any container metrics to the most related logs.

If your logs contain any of the following attributes, these attributes are automatically added as Datadog tags on your logs:

• kubernetes.container_image
• kubernetes.container_name
• kubernetes.namespace_name
• kubernetes.pod_name
• docker.container_id

While the Datadog Agent collects Docker and Kubernetes metadata automatically, FluentD requires a plugin for this. We recommend using fluent-plugin-kubernetes_metadata_filter to collect this metadata.

Configuration example:

# Collect metadata for logs tagged with "kubernetes.**"
 <filter kubernetes.*>
   type kubernetes_metadata
 </filter>

Run the Agent's status subcommand and look for fluentd under the Checks section.

See metadata.csv for a list of metrics provided by this integration.

The FluentD check does not include any events.

fluentd.is_ok:

Returns 'Critical' if the Agent cannot connect to Fluentd to collect metrics. This is the check which most other integrations would call can_connect.

Need help? Contact Datadog support.

• How to monitor Fluentd with Datadog