In order to be able to adapt intelligently and at high speed to new competitive challenges, business users need access to information that remains consistent however much their organization is changing.
As demand for responsive business intelligence and business performance management grows, global enterprises are still turning to data warehouses as their preferred source of data for analysis. The principle of gathering corporate data into a single, consistent store remains perfectly valid, but as businesses are constantly changing, the practice of traditional data warehousing can prove complex, costly and prone to failure.
The fundamental problem is that traditional data warehousing methodology promotes stasis of the business model, but businesses thrive on change. The difficulty of reconciling these opposites is a major contributor to why four in every ten data warehouse implementations are expected to fail.
Conventional data warehousing wisdom says that you should plan for a lengthy and expensive implementation, that you will need an army of skilled project managers and technicians, and that you can forget about trying to reflect the changing state of your business: A data warehouse is static data in a static model, custom-built to meet fixed user requirements.
However, in order to be able to adapt intelligently and at high speed to new competitive challenges, business users need access to information that remains consistent however much their organization is changing. The cost and time overheads of recoding a conventional data warehouse to track every change in the business are prohibitive, so reporting in such an environment will always be delayed or inaccurate, and business intelligence initiatives will not deliver actionable conclusions.
Leaders of responsive, ROI-conscious enterprises rightly observe that this is no way to support a business. Rather than molding their business models to fit in with what data warehousing convention says is possible, major companies such as Royal Dutch/Shell Group, HBOS plc, and Unilever are breaking the rules, using next-generation tools and methodologies that make data warehousing responsive to their businesses, and highly cost-effective.
Next-generation data warehousing assumes that both the business model and reporting requirements are ever-changing. This enables businesses not only to obtain up-to-date business intelligence, but also to compare present, past and predicted performance, no matter what the business structure is at any given time. This enables business leaders to run truly adaptive enterprises, capitalizing on opportunities and reacting to global events faster than the competition.
The conventional rules–and how to break them
— Build, don’t buy. Your enterprise is unique, so your data warehouse will need to be highly customized, tailored and coded to suit your individual business model. By using a data warehousing application with a generic data structure, users can create customized data warehouses without the usual cost or time overheads.
— The enterprise must clearly define an end-point for the data warehouse before starting any development work; the source systems to be used, and the queries and reporting formats needed, must be defined in advance With next-generation data warehousing, defining an end-point is no longer necessary, giving business intelligence and performance management tools the ability to be adapted to changing user requirements. The latest data warehousing techniques make it easier to define new data feeds and alter existing ones, as new star schemas can be automatically created. Adding a new transaction data set, or modifying an existing one and then regenerating the star schema, is a point-and-click operation. Business users can also alter their own reporting and querying requirements through defining and managing their own data marts.
— Freeze your business, and build the data warehouse to reflect it. Redesign is complex and expensive, therefore model the business as it is, and build your data warehouse to those specifications. Global enterprises may introduce new brands, acquire competitors or sell off under-performing business units on a daily basis, so freezing the business is an impractical proposition. By separating data from the business model, and allowing multiple models to co-exist, next generation data warehousing enables the data warehouse to evolve at the same speed as the business even during implementation.
— Time variance is expensive and difficult to manage, so you must apply ongoing changes to the business model indiscriminately to all data, whether current, historical or future. Next generation data warehouses provide a generic data structure that separates transaction and reference (business context) data from the current business model, and stores them all as separate entities. This makes it possible to view all of the organization’s collected data according to past, current, or future business models. A clear view of data in current and future business models is particularly important during merger and acquisition activity, where it enables decision-makers to compare pre- and post-merger performance at high speed and low cost.
— Federations of data warehouses are too complex and costly to build and synchronize. Handling multiple business models around the world is a sure-fire way to destroy the integrity of data. By storing data separately from its model, enterprises can support multiple business models across a federation with greater ease. Synchronization can be handled automatically, with new business models distributed over the internet, and reporting controlled from a central point for maximal cost-effectiveness.
— A major data warehousing project requires significant investments in programming skills, as well as in project management, system architecture, business reporting, Online Analytical Processing (OLAP), and database architecture skills By using a pre-built data warehousing application that can quickly be adapted to suit the business, then managed by business users via a simple interface, enterprises can create and run data warehouses without the investment in programming skills normally required–and without needing a skilled database administrator for every local instance.
— Building a data warehouse could cost in the millions and take many months, if not years. Enterprises that use data warehousing applications rather than building from scratch can expect much faster implementation at significantly reduced cost. Next-generation data warehousing software also gives enterprises the opportunity to change the structure and purpose of the data warehouse during the implementation cycle, reducing the need for exhaustive pre-planning and dramatically cutting the risk of project failure.
The next generation goes live
Next-generation data warehousing is not merely a blueprint for the future, but a reality in major enterprises around the world, where it is saving time and money, and delivering a clearer and more accurate view of performance throughout change.
Take, for instance, global FMCG giant Unilever. It regularly undertakes mergers and acquisitions, so it needed a data warehouse that would not require its multiple business models to remain static. The company also needed to be able to view historical brand performance, in order to measure the effects of restructuring initiatives. Unilever successfully broke through the constraints of conventional data warehousing, building a flexible and cost-effective solution that has delivered rapid results.
Using next-generation data warehousing technology, Unilever has succeeded in bringing together complex, time-variant data from numerous systems, and is using this data to deliver relevant and timely management information directly to business users. The company now has commonality across supply-chain, brand, customer and financial data, all cross-referenced by the same master reference data warehouse, ensuring greater consistency and accuracy of information.
The solution has made a substantial contribution to savings in procurement, and expanded Unilever’s ability to view the historic and projected performance of global brands across financial and non-financial measures.
Breaking free from constraints
Enterprise leaders seeking to improve the ROI of their management information initiatives no longer need to feel that data warehousing technology holds them back. As the above examples demonstrate, new software and methodologies make it possible to create highly responsive data warehouses that can be managed at low cost in rapidly-changing business environments. These data warehouses can deliver a consistent view of the past and the present without requiring any costly changes to source systems, and automatically adapt to business change.
By challenging restrictive assumptions about data warehousing, enterprises can develop the flexibility they need, but without having to make unsustainable investments in technology. In a climate of cost-cutting, can any enterprise afford to ignore next-generation data warehousing?
Cliff Longman is CTO of the Kalido Group, a technology company that was recently spun out of the Royal Dutch/Shell Group.