EMA Blog: Business Intelligence and IT Management

Enterprise Management Associates (EMA) has recognized that big data implementers and consumers rely on a variety of platforms to meet their big data requirements. These platforms include new data management technologies such as Hadoop, MongoDB, and Cassandra, but the collection also includes traditional SQL-based data management technologies supporting data warehouses and data marts; operational support systems such as customer relationship management (CRM) and enterprise resource planning (ERP); and cloud-based platforms. EMA refers to this collection of platforms as the Hybrid Data Ecosystem (HDE):

As Big Data initiatives mature into enterprise data sources supported by NoSQL products for analytics and operational systems, a clash of cultures is on the horizon (if not here already). Traditional IT implementations teams and their top-down programs rarely see eye to eye with the grass roots culture of NoSQL platform operators. But this divide is not merely between the camps of Big Data/NoSQL and traditional IT implementation teams. This is just the tip of the iceberg…. The divide becomes much more pronounced when you take the discussion to the executive suite. CMOs and CFOs, who “own” results of analytical and operational systems, are less concerned with data center standards and development methodologies as they time to value. CIOs and CTOs, responsible for implementing the connectivity and integration between NoSQL platforms and the rest of the traditional IT environment, are facing pressures to avoid chasing the latest technology fad(s).

If you look at the history of Big Data requirements (volume, velocity and variety), and the NoSQL platforms supporting those requirements, you see a history of organizations and development teams breaking the mold of traditional information technology (IT) programs. Instead of following the traditional IT methodologies to solve the Big Data issues, these teams pushed the envelope and invented new technologies to solve those “volume, velocity and variety” problems. More often than not, these efforts were accomplished using collaborative, bottom-up methodologies, such as Open Source, rather than rigid, top-down approaches found in traditional product development methodologies. Specifically, if you look at the history of the Hadoop development at Yahoo, you see an approach that sought the input and wide spread resources of the Open Source movement rather than a more rigid proprietary approach.