Skip to content
DataMiner DoJo

More results...

Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Search in posts
Search in pages
Search in posts
Search in pages
Log in
Menu
  • Updates & Insights
  • Questions
  • Learning
    • E-learning Courses
    • Empower Replay: Limited Edition
    • Tutorials
    • Open Classroom Training
    • Certification
      • DataMiner Fundamentals
      • DataMiner Configurator
      • DataMiner Automation
      • Scripts & Connectors Developer: HTTP Basics
      • Scripts & Connectors Developer: SNMP Basics
      • Visual Overview – Level 1
      • Verify a certificate
    • Video Library
    • Books We Like
    • >> Go to DataMiner Docs
  • Expert Center
    • Solutions & Use Cases
      • Solutions
      • Use Case Library
    • Markets & Industries
      • Media production
      • Government & defense
      • Content distribution
      • Service providers
      • Partners
      • OSS/BSS
    • Agile
      • Agile Webspace
      • Everything Agile
        • The Agile Manifesto
        • Best Practices
        • Retro Recipes
      • Methodologies
        • The Scrum Framework
        • Kanban
        • Extreme Programming
      • Roles
        • The Product Owner
        • The Agile Coach
        • The Quality & UX Coach (QX)
    • DataMiner DevOps Professional Program
      • About the DevOps Program
      • DataMiner DevOps Support
  • Downloads
  • More
    • DataMiner Releases & Updates
    • Feature Suggestions
    • Climb the leaderboard!
    • Swag Shop
    • Contact
    • Global Feedback Survey
  • PARTNERS
    • All Partners
    • Technology Partners
    • Strategic Partner Program
    • Deal Registration
  • >> Go to dataminer.services

Does DataMiner support NetFlow?

Solved1.52K views10th July 2020NetFlow
4
Thomas Gunkel [SLC] [DevOps Advocate]1.22K 7th July 2020 0 Comments

I am also interested in possibilities DataMiner has retrieving & storing NetFlow-Data. Also keen to learn about the performance (number of netflow records/s) and how NetFlow data gets stored in DataMiner.

Thomas Gunkel [SLC] [DevOps Advocate] Selected answer as best 10th July 2020

1 Answer

  • Active
  • Voted
  • Newest
  • Oldest
12
Jeroen Nietvelt [SLC] [DevOps Advocate]1.34K Posted 7th July 2020 1 Comment

I can confirm that we currently have the capability of capturing, storing and presenting informative dashboards on the recorded netflow traffic. We capture the data by having a Netflow collector element installed and operating on a DataMiner agent. This element will parse the incoming netflow traffic and complete it with extra metadata such as a hostname to IP mapping and Application name to IP port mapping information.

It also features the capabilities of optionally doing Top flow filtering (basically filter out the many tiny data communication packets) and to perform data aggregation to allow high resolution data captures to be stored for a small timeframe whilst storing lower resolution data captures for a longer timeperiod.

We store this processed netflow traffic in a document store database called ElasticSearch. This will allow us to be very flexible on the reporting side of the captured netflow traffic.

Typically a single DMA can process up to 1500 - 2000 incoming netflow packets per second which translate into approx. 35000 - 45000 netflow flows per second. We can horizontally scale this processing by installing more DataMiner agents each hosting a netflow collector element. At the same time, the Elasticsearch cluster where the netflow traffic will be stored needs to be scaled accordingly as well.

Once the data is available in the Elasticsearch cluster, we offer a dedicated easy to configure dashboard component which allows you to:

  1. Select which data you want to see.
  2. How this data needs to be grouped and aggregated.
  3. How this data needs to be presented.

Ben Vandenberghe [SLC] [DevOps Enabler] Posted new comment 31st July 2020
Ben Vandenberghe [SLC] [DevOps Enabler] commented 31st July 2020

Great info Jeroen. And we have more coming… I’m looking forward to see some of this Netflow data being leveraged in the new node-edge component, and the real kicker will come when this is combined / overlayed with data from other sources (e.g. via the Generic Query Interface). Quick follow-up question, as the data is stored in a LoggerTable (I presume, because you mention the records going into ElasticSearch), are those records then also available in the Search page in the DataDisplay of the collectors (where you can do key word search queries)?

You are viewing 1 out of 1 answers, click here to view all answers.
Please login to be able to comment or post an answer.

My DevOps rank

DevOps Members get more insights on their profile page.

My user earnings

0 Dojo credits

Spend your credits in our swag shop.

0 Reputation points

Boost your reputation, climb the leaderboard.

Promo banner DataMiner DevOps Professiona Program
DataMiner Integration Studio (DIS)
Empower Katas
Privacy Policy • Terms & Conditions • Contact

© 2025 Skyline Communications. All rights reserved.

DOJO Q&A widget

Can't find what you need?

? Explore the Q&A DataMiner Docs

[ Placeholder content for popup link ] WordPress Download Manager - Best Download Management Plugin