solution Use Case
Media flow tracking & analytics
With the ongoing migration in the media and broadband industry towards an IP infrastructure, media companies, service providers or cable operators, just to name a few, are all facing plenty of new challenges. The move to IP is not just another shift in technology. This time, operational procedures need to change as well and new tools need to be developed to support those.
Without any doubt, IP offers plenty of advantages and opportunities for the media industry. However, the nature of an IP architecture makes it more difficult to isolate problems when they arise. Proactively monitoring, visualizing and troubleshooting flows in a network end to end in an efficient way has become a must.
Different people in an organization have different requirements for a media flow monitoring & analytics solution. An IT engineer will look at flow traffic from a different angle than somebody from the media operations team. Before flow information can be presented to different teams in your organization in the right way, the right data needs to be collected and aggregated from different data sources.
Data acquisition
Switch fabric
A real-time media flow analytics solution always requires an integration with the technologies and interfaces provided by the switch manufacturers to get a first overview on the unicast and multicast flows which are present in the IP fabric. Mainly those are:
- Flow technologies such as NetFlow, SFlow, Jflow, etc., to retrieve metrics like source IP address, source port, destination IP address, destination port and protocol (also called the 5tuple).
- Streaming telemetry protocols, for example OpenConfig, to get real-time data on switch port status information, port utilization, port error rates, not to forget about DOM (Digital Optical Monitoring) metrics to monitor your SFPs metrics (e.g. RX, TX power, etc.) in optical networks.
- Vendor-specific API integrations to read ACLs, multicast routing & IGMP tables to keep track all routing information.
- Think of host and flow policies to check the maximum flow bandwidth allowed per ingress interface and to retrieve details about which hosts are allowed to join a stream.
- Some vendors also provide very specific flow functionality, e.g. RTP flow monitoring capabilities to retrieve the actual bitrate of a specific flow in the network.
Direct integration with every switch is the preferred way to retrieve all the above info. Nevertheless, there are specific vendor tools available, for example Cisco DCNM or Arista CloudVision, that can also be used as a data source. This could be an option when direct switch integration is prohibited by IT security rules. Every deployment is different, and with DataMiner, you can rest assured to have access to the data you need.
Edge devices
Media flow tracking also requires interfacing with the network edge devices such as unicast or multicast senders to constantly check if the sources are streaming on the correct addresses, but also to see if the incoming signal (RF, SDI, ASI, IP) is okay before they enter the IP fabric you want to monitor. The same goes for the destinations. For example, it is essential to check stream presence for redundant inputs (e.g. ST2022-7).
Supporting data sources
360° flow tracking does not stop there. It is not only important to track which flows are present, the active flows also need to be compared against systems that know which streams should be present at what time. Those could be a service providers’ work order system to manage contribution links, a third-party SDN controller in a broadcast MCR environment, or a proprietary database scheduling the streams for a video headend.
And it is not only about monitoring the expected streams. It is even more important to also identify those flows in the network which are present but should not be there, often referred to as ghost streams.
Whereas for IT teams, it is often sufficient to work with the actual multicast IP addresses, media operation teams rely more on descriptive labels that are typically defined in the broadcast controller to identify (virtual) sources and destinations. A real-time media flow analytics solution also needs to interface with those products and map the IP addresses with the labels. A label can represent a single multicast stream but also plenty of media flows, for example, in a ST2110 environment where a label “Camera1” could consist of a redundant UHD and HD video flow, 16 audio flows plus ANC data flows, all represented by one single label.
Test and measurement equipment
Media flow monitoring also requires the integration of dedicated online or offline network or media flow analyzers. Sometimes, you need to go down to packet level to identify a flow problem, e.g. to record the network traffic and to do further offline analysis of IP packets. DataMiner integrates with standard IT tools like Wireshark but also has integrations with specific flow analytics tools like the EBU LIST (Live IP Software Toolkit) tool to inspect uncompressed SMPTE-2110-x traffic or online analyzers like Tektronix Prism or BridgeTech VB440, just to mention a few examples.
Data consumption and visualization
Real-time media flow tracking & analytics can be treated as a 2-step process.
24/7 flow monitoring in the network gives you an immediate notification when a stream is missing at a destination. Cross-checking the control/scheduling system against stream presence at each destination can be done for thousands of streams concurrently.
Monitoring is not only a reactive action anymore:
- DataMiner forecasting, anomaly or pattern detection, just to name a few machine learning supported features in DataMiner, turns your operation into a proactive one to identify issues before they actually have an impact.
- DataMiner Application and Services Verification (ASV) supports proactive networking checks. Those can be simple things like pinging your edge devices or automatically perform a traceroute to track the pathway of IP packets.
Incident management: In case there is an issue with any of the streams at a destination, a media flow analytics solution needs to provide an easy-to-use-tool to troubleshoot and identify the root cause of the issue. This is only possible when you “crawl” through the network from a signal source to the destination and interface with all supporting data sources as described above.
Summary
The DataMiner real-time media flow monitoring & analytics solution proactively monitors your media flows, provides a true 360° in-depth view, from your flow source to each destination, and offers an easy-to-use UI to deep dive into the root cause of any flow issue, be it the network, the source or the destination device which is causing your flow issue.
- Customizable UIs/data visualizations targeted to different operational teams are provided to fulfill everybody’s needs: DataMiner really bridges the two worlds of IT and media operations within a single solution.
- At the same time, there is a guarantee that DataMiner can interface with all your equipment, independent of the vendor or protocol, now and in the future. And whenever your operational workflows are changing, DataMiner will adapt to them, instead of the other way round.
An outlook
The solution does not stand on its own but is tightly integrated with other DataMiner IT-oriented solutions, such as DataMIner Infrastructure Discovery and Provisioning (IDP). IDP automatically detects your network infrastructure and your media edge devices as well as the connectivity (e.g. via LLDP protocol) between switches and the edge devices. With that, IDP automates all the steps to setting up and provisioning your media flow tracking & analytics solution, assuring that all network infrastructure data is always up to date.
For more information, also have a look at our recent blog post.