Agile Business Intelligence (ABI): WTF or WTFN?
Key Points in Defining Agility
I define agility as the ability of an organization to handle events or implement strategies that change the functioning of key organizational processes. Agility can be further categorized as:
- proactive and reactive;
- anticipated and unanticipated;
- internally or externally caused;
- new-product-, operational-, and disaster-centric.
That is, improved agility enhances one or all of these areas.
Initial data suggest that improvements in new-product development (proactive, unanticipated, externally caused) have the greatest impact, since they have spill-over effects on the other categories (anticipated, internally-caused, operational, and disaster). However, improvements in operational and disaster agility can also deliver significant bottom-line long-term benefits. Improved agility can be measured and detected from its effects on organizational speed, effectiveness, and "follow-on" metrics (TCO, ROI, customer satisfaction, business risk).
The implications for Agile BI are:
- Unless improved BI agility helps new-product development, its business impact is small.
- Increased speed (faster reporting of results) without increased effectiveness (i.e., a more agile business decision-making process) has minimal impact on overall agility.
- Improvements to "reactive" decision-making deliver good immediate results, but fewer long-term impacts than improvements to "proactive" decision-making which anticipates rather than reacts to key environmental changes.
In summary, agile BI that is part of an overall agile decision-making and new-product-strategy-driving business process, and that emphasizes proactive search for extraorganizational data sources, should produce much better long-term bottom-line results than reactive BI that depends on relatively static and intra-organizational data sources.
The Fundamental Limit to Today's Agile Decision-Making via BI
Let's consider some questions about BI with answers based on Aberdeen Group data usefulness study, (used by permission of Aberdeen Group):
1. Question: Where do the greatest threats to the success of the organization lie, in its internal business processes or in external changes to its environment and markets? Answer: In most cases, external.
2. Question: Which does better at allowing the business person to react fast to, and even anticipate, external changes - internally gathered data alone, or internal data plus external data that appears ahead of or gives context to internal data? Answer: Typically, external.
3. Question: What percentage of BI data is external data imported immediately, directly to the data store? Answer: Usually, less than 0.1 %.
4. Question: What is the average time for the average organization from when a significant new data source shows up on the Web to when it begins to be imported into internal databases, much less BI? Answer: more than half a year.
The fundamental limit to the agility and effectiveness of BI therefore lies not in any inability to speed up analysis, but in the fact that today's BI and the business processes associated with it are designed to focus on internal data. Increasingly, your customers are moving to the Web; your regulatory environment is moving to the Web; mobile devices are streaming data across the Web; new communications media like Facebook and Twitter are popping up; and businesses are capturing a very small fraction of this data, primarily from sources (long-time customers) that are changing the least.
As a result, the time lost from deducing a shift in customer behavior from weekly or monthly per-store buying instead of social-network movement from one fad to another dwarfs the time saved when BI detects the per-store shift in a day instead of a weekend; and a correct reaction to the shift is far less likely without external contextual data.
This is an area where agile new product development is far ahead of BI. Where is the BI equivalent of reaching out to external open-source and collaborative communities? Of holding "idea jams" across organizations? Of features/information as a Web collaboration between external users and code/query creators? Of "spiraling in on" a solution? Of measuring effect by "time to customer value" instead of "time to complete" or "time to decide"?
A simple but major improvement in handling external data in BI is pretty much doable today. It might involve integrating RSS feeds as pop-ups and Google searches as complements to existing BI querying. But if any major BI vendor currently features this capability on the front page of its Web site, I have yet to find it.
