Network data quality and authority have been problems for a very long time, and IT organizations can no longer afford to ignore the issue. Enterprise Management Associates (EMA) recommends that CIOs establish a network data architecture team and give it the budget and authority needed to improve the data that IT organizations rely on for day-to-day network operations.
When managing a network, an IT organization typically deals with countless classes of network data. This data can be organized into two general categories: intent data and state data. Intent data is a record of how a network should operate, and state data is proof of how the network is operating. So much of modern network operations is about discovering disagreements between intent and state. Answering that question is difficult when the data is scattered and its quality and authority are questionable.
As an analyst who researches how IT organizations design, build, and operate their networks, I find that network data is a constant source of pain. Network teams struggle with data quality, fragmentation, authority, access, and trust. And these issues undermine everything they try to do.
Here are the numbers:
Additionally, the issue of network data quality stretches beyond network operations. Many other parts of a company rely on network data for critical operations. Cybersecurity teams require network data for incident detection and investigations. DevOps and cloud teams consume network data when making network-related changes to their environments or troubleshooting application issues. Compliance teams have rigorous data requirements when they conduct network audits.
The fact that only 45% of enterprises are completely confident in the accuracy of their network source of truth points to a major gap in network intent data quality.
Network intent data is critical to all aspects of operations, from designing and deploying networks and implementing changes to responding to network incidents. Examples of intent data include configuration standards, IP address space, network inventory information, connectivity and topology, and security policies.
Engineers reference intent data for a variety of operations. When they make a change, they need to ensure that the change will comply with intent. When they troubleshoot a problem, they need to gather intent data for the problem domain, which helps them identify whether any drift from intent is the root cause of a fault or performance issue.
Good network intent data streamlines manual operations, but it also enables network automation. For example, automation tools with programmatic access to a network source of truth can instantly pull all the data they need to create a new configuration. Additionally, intent data is the foundation of agentic network operations. AI needs to understand how a network should operate before it does anything else.
Network engineers often tell me that intent data is scattered across multiple siloed systems of record, and the quality and authority of those systems are often unclear. For instance, network inventory data might live in two or three network controllers. IP address space often resides in a spreadsheet. And an organization might have two systems of record for config standards. A tier 1 engineer who is working a trouble ticket will struggle to gather info across such an environment.
Lack of trust in network observability data points to a variety of tool problems that are undermining how network teams monitor and troubleshoot their networks. Network observability data is usually siloed and the quality is uneven. Data silos and quality issues lead to visibility gaps, where network operators are blind to emerging problems until disaster strikes.
A company might have one tool for collecting device metrics, another for monitoring flows, a third for collecting and analyzing packets, a fourth for synthetic traffic analysis, and a fifth for detecting config changes. Many tool vendors have proprietary methods for managing data, which makes it difficult for IT organizations to integrate and correlate data across tools. This forces operators to context-switch from one tool to another as they work a problem, and this kills efficiency.
Beyond silos, EMA’s recent research found that network operators primarily face four data quality issues with their observability tools. First, security risk is a challenge, with policies often preventing network teams from collecting data from certain parts of the network or limiting what data they can send to cloud-based tools. Second, they are dealing with architectural issues such as scalability limits, data collection errors, and insufficient support for modern architecture like hybrid and multi-cloud. Third, they encounter skills gaps on network teams which prevent them from instrumenting the network completely. This leads to blind spots. Finally, cost and licensing issues place ceilings on how much data operators can collect with their tools. This forces them to make hard choices about what aspects of their environment they can and can’t monitor.
The stakes for network data quality are about to increase. The status quo is already untenable, but AI investments will push network data to its breaking point. EMA research finds that most IT organizations expect to embrace agentic network operations over the next two years. Given the low number of IT professionals who believe their network data quality can enable this AI-powered future, IT leaders need to tackle the issue of network data now.
EMA proposes that CIOs introduce a network data architecture practice or at least appoint a network data architect who thinks about things in terms of data, no just tools. This architecture practice should have the authority and budget needed to remediate the data quality issues that most network infrastructure and operations teams are struggling against. Last year a network architect told me that his organization spends $2 million a year with its network infrastructure vendor, but his CIO declined to approve a $60,000 request for a source of truth tool that would improve how intent data was managed. This points to blind spot that many IT organizations have about the criticality of this data problem.
Most large IT organizations already have people capable of taking on a network data architecture role. They are often serving as network observability or network automation architects, NetDevOps engineers, or data architects. They spend a lot of time implementing the tools that drive network operations, but they also have first-hand experience with the data quality issues that challenge those tools.
The network data architecture team should work across groups to establish standards for data quality and management. It should conduct a thorough review of all systems of record for network intent and all network observability tools. The team should establish tools and processes for network data management across systems, and it should recommend ways to improve data quality, eliminate data silos, and optimize telemetry collection.
Here are some examples of what a network data architecture practice might do:
EMA believes IT leaders can dramatically improve network resilience, security, compliance, and operational efficiency by establishing a network data architecture practice and empowering its people with budget resources and the authority to make decisions across silos. EMA’s recent market research has affirmed that this should be a priority for CIOs, and we will conduct dedicated multi-sponsor market research on the concept of network data architecture in the coming months. If you would like to learn more, please contact me at smcgillicuddy@enterprisemanagement.com.