Skip to content

Commit

Permalink
(DOCS-6845) Add enrichment options and screenshots (#20997)
Browse files Browse the repository at this point in the history
* (DOCS-6845) Add enrichment options and screenshots

* Apply suggestions from code review

Co-authored-by: Brett Blue <[email protected]>

* (DOCS-6845) Incorporate editorial feedback

---------

Co-authored-by: Brett Blue <[email protected]>
  • Loading branch information
buraizu and brett0000FF authored Dec 20, 2023
1 parent bfa0dd2 commit 6b7f664
Show file tree
Hide file tree
Showing 4 changed files with 26 additions and 8 deletions.
34 changes: 26 additions & 8 deletions content/en/network_monitoring/devices/netflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,34 +53,52 @@ After saving your changes, [restart the Agent][4].
## Aggregation
The Datadog Agent automatically aggregates the received NetFlow data in order to limit the number of records sent to the platform while maintaining most of the information. By default there is a 5 minute aggregation interval, during which flow recordings which share the same identifying information (source and destination address and port, protocol, and so forth) will be aggregated together. Additionally, the Datadog Agent can detect ephemeral ports and remove them. As a result, you may see Flows with `port:*`.
The Datadog Agent automatically aggregates the received NetFlow data in order to limit the number of records sent to the platform while maintaining most of the information. By default there is a five-minute aggregation interval, during which flow recordings which share the same identifying information (source and destination address and port, protocol, and so forth) are aggregated together. Additionally, the Datadog Agent can detect ephemeral ports and remove them. As a result, you may see Flows with `port:*`.

## Enrichment

Your NetFlow data is processed by the Datadog backend and are enriched with the available metadata from your devices and interfaces. Enrichment is based on the NetFlow exporter IP and the interface indexes. To disambiguate possible collisions between reused private IPs, you can configure a different `namespace` for each Agent configuration file (with the setting `network_devices.namespace`)
Your NetFlow data is processed by the Datadog backend and enriched with the available metadata from your devices and interfaces. Enrichment is based on the NetFlow exporter IP and the interface indexes. To disambiguate possible collisions between reused private IPs, you can configure a different `namespace` for each Agent configuration file (with the setting `network_devices.namespace`).

If the NetFlow exporter IP is one of the device IPs, but not the one configured on the SNMP integration, Datadog will attempt to locate the device that the exporter IP belongs to, and will enrich your NetFlow data with it is as long as the match is unique.
If the NetFlow exporter IP is one of the device IPs, but not the one configured on the SNMP integration, Datadog attempts to locate the device that the exporter IP belongs to, and enriches your NetFlow data with it is as long as the match is unique.

### Cloud provider IP enrichment

Datadog enriches IPs with public cloud provider service and region for IPv4 addresses, so you can filter for flow records from a specific service and region.

{{< img src="network_device_monitoring/netflow/netflow_cloud_provider_ip_enrichment.png" alt="Netflow IPs enriched with cloud provider name, region, and service" width="80%" >}}

### Port enrichment

Datadog enriches ports in NetFlow with IANA (Internet Assigned Numbers Authority) data to resolve well known port mappings (such as Postgres on 5432 and HTTPS on 443). This can be seen when searching for source or destination application names on NetFlow.

{{< img src="network_device_monitoring/netflow/netflow_iana_port_mappings.png" alt="The NetFlow page filtered by @destination.application_name and displaying names for ports such as HTTPS" width="80%" >}}

#### Custom port enrichment

You can also add your own custom enrichments to map ports and protocols to specific applications (for example, if a custom service runs on a specific port). This makes it easier for network engineers and their teams to interpret and query NetFlow data with human-readable names.

From the **Configuration** tab in NetFlow, click **Add Enrichment** to upload the CSV file containing your custom enrichments.

{{< img src="network_device_monitoring/netflow/new_enrichment.png" alt="The New Enrichment Mapping modal in the Netflow configuration tab" width="80%" >}}

## Visualization

You can find the NetFlow page on the [Network Devices page][5].
{{< img src="network_device_monitoring/netflow/netflow_page.png" alt="NetFlow Page" >}}
{{< img src="network_device_monitoring/netflow/netflow_page.png" alt="NetFlow Page" width="80%" >}}

This data is also available in Dashboards, Notebooks, and more for more precise queries and for correlating with other sources of data.
{{< img src="network_device_monitoring/netflow/notebook.png" alt="Notebook" >}}
{{< img src="network_device_monitoring/netflow/notebook.png" alt="Notebook" width="80%" >}}

## Retention

NetFlow data is retained for 30 days by default.


## Further Reading

{{< partial name="whats-next/whats-next.html" >}}


[1]: https://app.datadoghq.com/account/settings/agent/latest
[2]: /network_monitoring/devices/snmp_metrics/
[3]: /agent/configuration/agent-configuration-files/?tab=agentv6v7#agent-main-configuration-file
[4]: /agent/configuration/agent-commands/?tab=agentv6v7#start-stop-and-restart-the-agent
[5]: https://app.datadoghq.com/infrastructure/devices?facets=&viewTab=netflow
[5]: https://app.datadoghq.com/infrastructure/devices?facets=&viewTab=netflow
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 6b7f664

Please sign in to comment.