To assess the likely winners in the race for the unified analytics warehouse, it is important to understand the various requirements of modern analytics programs and the unified analytics warehouse.
Data Requirements
The modern enterprise produces and utilizes a broad range of data types. For example, a customer-facing application may capture semi-structured web stream data, mobile app data, raw text from email, location data, and structured data from both marketing and sales automation systems. Customer engagement requires a single platform to easily capture, combine, and analyze these different streams and sets of data. The support of multi-structured data is vital to the success of the unified analytics warehouse.
Today’s digital enterprise also needs to respond intelligently to real-time business and customer events. Therefore, the UAW must enable organizations to ask questions of data while it is in motion and while it is at rest, or to enable queries across both states of data.
Enterprise Requirements
Both longstanding companies and digital-first companies require specific enterprise capabilities to meet corporate, regulatory, and competitive requirements. These include, but are not limited to, management, orchestration, security, privacy, performance, scalability, agility, resiliency, and affordability. A successful unified analytics warehouse must be enterprise-ready.
Because of the size and complexity of modern data warehouses and data lakes, plus the modernization of data platforms, AI-enabled automation also becomes a key requirement. Automation can be broken down into recommendations and process automation. The most advanced UAW technology should make recommendations for at least structure, schema, data relationships, and performance tuning. These platforms should also automate actions like metadata generation, tuning, maintenance, elasticity, query routing, data tiering, change management, and code generation.
Infrastructure Requirements
Historically, businesses moved their data from databases to file systems to save money. Now, they are moving from file systems to object storage. In the world of analytics, it is important to remember that cheap storage is limited. If it is not accessible for analysis, cheap is not enough. For this reason, the unified analytics warehouse must be able to provide a rich and consistent set of analytical capabilities across all storage tiers.
More advanced UAWs will automate the movement of data in and out of file systems and object storage when needed. In order to make the UAW cost-effective, it should not be tied too tightly to the hardware. Cloud and appliance vendors with tight ties to their underlying infrastructure have little motivation to optimize their software and provide the most cost-effective technology offering.
Hybrid Requirements
Recent EMA research shows that 53% of all data is now in the cloud. Because of this massive shift of data, both multi-cloud and hybrid support is essential for the unified analytics warehouse. A legitimate UAW will enable management of these different systems in a single pane of glass for all environments in the cloud and on-premises.
Modern requirements dictate that certain workloads may need to move from one cloud provider to another, from on-premises to cloud, or from cloud to on-premises. In the best-case scenario, the unified analytics warehouse software should live and act in the same way across different cloud providers and on-premises.
Analytical Processing Requirements
Analytics are the primary use case of the unified analytics warehouse. Therefore, at a minimum, the unified analytics warehouse must include a common set of prebuilt analytical algorithms and functions to address every step of the machine learning process. Those algorithms should be embedded in the platform for ease of use and high performance on the largest data volumes, without any down-sampling of data. Additionally, it must support the rapid development and execution of ad hoc analytics.
Ideally, the UAW should perform well for different analytical processing types of workloads. The performance spectrum should cover data intensive with low compute, compute intensive with low data, data intensive with high compute, and a high number of concurrent users or queries.
User Expectations
At the highest level, users expect the unified analytics warehouse to provide seamless unification of all interactions with data and analytics. Users do not anticipate having to move to different environments to access data or having to use different interfaces to manage diverse data. Users also expect a platform that allows them to put aside religious beliefs about how analytics should be done. With the UAW, data engineers, data scientists, and data analysts no longer need to fight about who is right and who is wrong. They have a single environment where they can collaborate for the greater good of the enterprise.
To support a unified data workforce, the UAW must also support a broad set of different approaches to analytics. The data scientist must be able to use R, Python, and notebooks to execute discovery analytics or advanced analytics like machine learning on multi-structured data. The data analyst must have ready access to multi-structured data using SQL, the lingua franca of analytics. Business users must be able to simply combine data, build reports, or construct dashboards for use in their everyday work. Finally, the platform must enable easy-to-access to high-performance analytics via pre-built embedded algorithms. It must be straightforward to combine all analytics for greater insight and ask questions of data in near-real time.
For the rest of the story, read the full white paper, The Emergence of the Unified Analytics Warehouse – Data Lakes and Data Warehouses Merge.
