Cloud is still a rapidly evolving discipline, with currently many organizations thinking about how to get started or how to take their limited cloud environment to the next level. These organizations are faced with two general challenges. They have to a) get their own house - data center – in order and b) identify a shortlist of vendors that best fit their individual requirements.
To aid organizations in transitioning to cloud architectures, EMA will be conducting two major research projects this fall – “The Journey to the Cloud: Challenges & Opportunities” and the “Cloud Management Radar Report” – that are both aimed at identifying the most significant cloud challenges and opportunities, as they are perceived and experienced by end users. Before introducing these projects in more detail, I will provide the context required to fully understand the importance of this research.
Flashback: 10 Years Ago – The Dawn of Virtualization
About a decade ago, organizations started planning their first server virtualization projects. At the time, virtualization appeared to be the “Promised Land”, where server utilization rates would skyrocket and deployment times would drop to just minutes – at least, in theory. In reality, end users are often still waiting for multiple weeks for the resources they ordered, and resource utilization rates today are not significantly higher than the average 18%-20% they were ten years ago.
Today, we know that server virtualization was only the beginning of the road toward the efficient data center. Virtualization projects were initially aimed at the low hanging fruit, providing resources to developers for non-mission critical applications. The easy availability of these resources led to an explosion of the number of operating systems to manage. This proliferation of operating systems often resulted in unanticipated resource bottlenecks. Storage, server, and network vendors tremendously benefited from these increased infrastructure requirements, while the organization was often left with a more complex environment made up of inconsistently provisioned virtual machines. The administration and maintenance of this increased number of virtual servers became more and more challenging, leading to cost, compliance, and security issues.
While we do not want to paint virtualization as a failed discipline, we think it is essential to point out the challenges and growing pains that were caused by virtualization. Many organizations still do not have the management solutions - processes and software - in place to provide the governance required to virtualize at least most of their data center infrastructure. This management deficit has led to virtualization stall, where the complexity caused by virtualized infrastructure required too much manual management for the organization to consider further virtualization projects. Therefore, even today, many data centers have only virtualized the low-hanging fruit, still hosting more complex applications on physical infrastructure.
Virtualization Today – A Central Pre-condition for Cloud
The resolution of the above described automation and orchestration challenges is a central pre-condition for any successful cloud strategy. Installing a cloud platform by itself will not resolve data center automation deficits. Automating the provisioning and lifecycle management of data center resources - compute, network, storage, and application stacks - ensures consistency, scalability, and cost efficiency. It also leads to more secure and compliant application environments and enables a SLA-centric approach to data center management. Only a well automated and highly standardized data center environment is ready for the cloud. The cloud technology platform is merely there to integrate existing IT management systems and provide physical, virtual, and application resources in a consistent, compliant, and self-service fashion.
The Economic Argument behind Cloud
Based on EMA's recent research report, organizations move to the cloud for the following key reasons:
Agility: The ability to quickly provision server, network, storage, and software resources for the end user enables organizations to quickly adjust to market requirements. Without this agility, end users often have to wait for weeks or months until they receive the requested resources.
CAPEX and OPEX: Automation and resource pooling leads to lower hardware and maintenance cost, as fewer maintenance personnel is required per server and as fewer physical servers are required for each business service.
Resiliency: Built-in high availability and rapid disaster recovery were named as central reasons for adopting a cloud platform.
Agility, cost savings, and added resiliency all are factors that help aligning enterprise IT with the organization's business processes. Being able to respond quicker and at a lower cost than the competition to today's market challenges, constitutes a vital strategic advantage that is enabled by the cloud.
What will be Hosted in the Cloud
When IT executives were asked about what applications will be moved to the cloud, there were hardly any taboos. From simple messaging servers to mainframe-based solutions, respondents were planning to eventually move the vast majority of applications to the cloud. While the path to the cloud will prove challenging for many legacy applications, these research results demonstrate the tremendous expectations placed in the cloud. Cloud is widely regarded as a transformative IT paradigm that will truly harness the IT organization to help the business compete more successfully in the market place.
Core Definition of Cloud
Now that we have illustrated the business impact of cloud, let's come up with a preliminary definition of what cloud really is. Generally, cloud should be seen as an abstraction layer that provides physical and virtual resources to the end user in a self-service manner. Within this context, it is essential to stress that a cloud platform must integrate with the existing data center infrastructure. Most organizations have gradually built out their data center over years or even decades. This means that leveraging the tremendous knowledge that is inherent in dozens or even hundreds of existing systems management solutions is key to a successful cloud deployment. The core components of a cloud solution are as follows:
- Self-service portal: End user provisioning of application stacks, raw operating systems, or physical hardware.
- Orchestration & automation: Task-based provisioning and management of existing infrastructure and application elements.
- Monitoring & lifecycle management: Monitoring resource and application health, as well as managing the lifecycle of operating system images and application stack.
- Virtualization and management: Virtualization management is an essential precondition for a successful cloud. Only when existing hardware and application resources are automated and efficiently managed, does the move to the cloud make sense.
- Public cloud integration: Integration with public cloud services allows the organization to realize flexibility and benefit from economies of scale.
- Physical management: The provisioning and lifecycle management of physical storage, networking, and compute resources has to be automated, based on SLA requirements.
- Chargeback/Showback: End users have to be charged for their resource utilization, to incentivize them to be economical.
- API: Cloud platforms must provide APIs to programmatically provision and manage resources.
The Marketplace for Cloud Platforms
The market place for cloud platforms fulfilling the above requirements can be subdivided into five main clusters:
- Open source platforms such as Eucalyptus, Red Had, Convirture, Nimbula
- Systems management vendors such as CA Technologies, BMC, IBM, ASG, Cisco IA (NewScale), Microsoft
- Pure-play cloud vendors such as Abiquo, Enomaly, Hexagrid, Egenera, Tibco, DynamicOps/VMware, Embotics, Right Scale
- Converged infrastructure vendors such as IBM, HP, VCE, Flexpod, Pivot3, Dell
- Virtualization platform vendors such as Citrix, VMware, Paralls, Microsoft, Virtustream
This diversity of cloud vendors demonstrates that identifying the most suitable vendor for the organization’s individual requirements is a non-trivial task.
Upcoming EMA Cloud Research for Fall 2012
EMA is preparing two major research projects this fall, designed to cast more light on what is most important when planning a cloud strategy and making a cloud platform selection. Based on these empirically determined customer priorities, the research will compare and evaluate the individual vendors’ cloud platforms. This research will provide guidance for end customers and vendors regarding how to leverage the cloud to create maximum value:
The Journey to the Cloud: Challenges and Opportunities
This research project will focus on determining real-life challenges and pain-points as they are experienced by customers when getting started with or expanding on their own private and hybrid cloud deployment. As a baseline, EMA will identify the current customer status quo in terms of the main preconditions for cloud regarding virtualization management, infrastructure consolidation, standardization, automation, and orchestration. Within this context, EMA will collect detailed information on the adoption rate of innovative solutions such as storage virtualization and automation, I/O consolidation, VM auto-alignment, quality of service management, advanced analytics, cloud bursting, high availability, multi tenancy, enterprise software integration, and more.
Pinpointing the challenges of organizations of different sizes and verticals when integrating private and public cloud solutions with their current data center will provide a central foundation for EMA’s Radar Report for cloud automation and orchestration platforms.
Cloud Orchestration and Automation Radar Report
Private cloud platforms consist of three core elements: resource orchestration, process automation, and self-service management. Resource orchestration aims at the optimal alignment of all existing physical and virtual resources within the datacenter, to achieve a maximum degree of scalability and flexibility. Process automation includes all processes and workflows required to provision, manage, and retire IT resources. Service catalog, access control, and chargeback are some of the critical elements of self-service management.
The current maze of cloud management products is staggering and causes considerable uncertainty within the IT organization. The EMA Cloud Orchestration and Automation Radar will evaluate software platforms for creating and configuring private and hybrid clouds. The study will place a special focus on ease-of-deployment and the platforms’ ability to support legacy software and hardware elements. Further evaluation criteria will be derived from the research results of the “Journey to the Cloud: Challenges and Opportunities” study. Like all EMA Radar Reports, this research will provide deep customer-centric insights into the industry, providing end users with the guidance they require to make an informed vendor selection.
We very much look forward to both research reports and to an ongoing dialog with end users and cloud platform vendors. I will provide updates regarding both research efforts via Twitter: @torstenvolk
