BI For The Masses: 3 Solutions That Will Never Happen
Idea Number 1: Google Exploratory Data Analysis
I'm reading through someone's blog when they mention "graphical analysis." What the hey? There's a pointer to another blog, where they make a lot of unproven assertions about graphical analysis. Time for Google: a search on graphical analysis results in a lot of extraneous stuff, some of it very interesting, plus Wikipedia and a vendor who is selling this stuff. Wikipedia is off-topic, but carefully reading the article shows that there are a couple of articles that might be on point. One of them gives me some of the social-networking theory behind graphical analysis, but not the products or the market. Back to Google, forward to a couple of analyst market figures. They sound iffy, so I go to a vendor site and get their financials to cross-check. Not much in there, but enough that I can guesstimate. Back to Google, change the search to "graphical BI." Bingo, another vendor with much more market information and ways to cross-check the first vendor's claims. Which products have been left out? An analyst report lists the two vendors, but in a different market, and also lists their competitors. Let's take a sample competitor: what's their response to "graphical analysis" or graphical BI? Nothing, but they seem to feel that statistical analysis is their best competitive weapon. Does statistical analysis cover graphical analysis? The names SAS and SPSS keep coming up in my Google searches. It doesn't seem as if their user manuals even mention the word "graph". What are the potential use cases? Computation of shortest path. Well, only if you're driving somewhere. Still, if it's made easy for me … Is this really easier than Mapquest? Let's try a multi-step trip. Oog. It definitely could be easier than Mapquest. Can I try out this product? All right, I've got the free trial version loaded, let's try the multi-step trip. You know, this could do better for a sales trip than my company's path optimization stuff, because I can tweak it for my personal needs. Combine with Google Maps, stir … wouldn't it be nice if there was a Wikimaps, so that people could warn us about all these little construction obstructions and missing signs? Anyway, I've just given myself an extra half-hour on the trip to spend on one more call, without having to clear it.
Two points about this. First, Google is superb at free-association exploratory analysis of documents. You search for something, you alter the search because of facts you've found, you use the results to find other useful facts about it, you change the topic of the search to cross-check, you dig down into specific examples to verify, you even go completely off-topic and then come back. The result is far richer, far more useful to the "common end user" and his or her organization, and far more fun than just doing a query on graphical data in the company data warehouse.
Second, Google is lousy at exploratory data analysis, because it is "data dumb": It can find metadata and individual pieces of data, but it can't detect patterns in the data, so you have to do it yourself. If you are searching for "graphical analysis" across vendor web sites, Google can't figure out that it would be nice to know that 9 of 10 vendors in the market don't mention "graph" on their web sites, or that no vendors offer free trial downloads.
The answer to this seems straightforward enough: add "guess-type" data analysis capabilities to Google. And, by the way, if you're at work, make the first port of call your company's data-warehouse data store, full of data you can't get anywhere else. You're looking for the low-priced product for graphical analysis? Hmm, your company offers three types through a deal with the vendor, but none is the low-cost one. I wonder what effect that has had on sales? Your company did a recent price cut; sure enough, it hasn't had a big effect. Except in China: does that have to do with the recent exchange rate manipulations, and the fact that you sell via a Chinese firm instead of on your own? It might indeed, since Google tells you the manipulations started 3 weeks ago, just when the price cut happened.
You get the idea? Note that the search/analysis engine guessed that you wanted your company's data called out, and that you wanted sales broken down by geography and in a monthly time series. Moreover, this is exploratory data analysis, which means that you get to see both the summary report/statistics and individual pieces of raw data - to see if your theories about what's going on make sense.
In Google exploratory data analysis, the search engine and your exploration drive the data analysis; the tools available don't. It's a fundamental mind shift, and one that explains why Excel became popular and in-house on-demand reporting schemes didn't, or why Google search was accepted and SQL wasn't. One's about the features; the other's about the consumer's needs.
Oh, by the way, once this takes off, you can start using information about user searches to drive adding really useful data to the data warehouse.
Idea Number 2: The Do The Right Thing Key
Back in 1986, I loved the idea behind the Spike Lee movie title so much that I designed an email system around it. Here's how it works:
You know how when you are doing a "replace all" in Word, you have to specify an exact character string, and then Word mindlessly replaces all occurrences, even if some should be capitalized and some not, or even if you just want whole words to be changed and not character strings within words? Well, think about it. If you type a whole word, 90% of the time you want only words to be replaced, and capitals to be added at the start of sentences. If you type a string that is only part of a word, 90% of the time you want all occurrences of that string replaced, and capitals when and only when that string occurs at the start of a sentence. So take that Word "replace" window, and add a Do the Right Thing key (really, a point and click option) at the end. If it's not right, the user can just Undo and take the long route.
The Do The Right Thing key is a macro; but it's a smart macro. You don't need to create it, and it makes some reasonable guesses about what you want to do, rather than you having to specify what it should do exactly. I found when I designed my email system that every menu, and every submenu or screen, would benefit from having a Do The Right Thing key. It's that powerful an idea.
How does that apply to BI? Suppose you are trying to track down a sudden drop in sales one week in North America. You could dive down, layer by layer, until you found that stores in Manitoba all saw a big drop that week. Or, you could press the Break in the Pattern key, which would round up all breaks in patterns of sales, and dig down not only to Manitoba but also to big offsetting changes in sales in Vancouver and Toronto, with appropriate highlighting. 9 times out of ten, that will be the right information, and the other time, you'll find out some other information that may prove to be just as valuable. Now do the same type of thing for every querying or reporting screen …
The idea behind the Do The Right Thing key is actually very similar to that behind Google Exploratory Data Analysis. In both cases, you are really considering what the end user would probably want to do first, and only then finding a BI tool that will do that. The Do The Right Thing key is a bit more buttoned-up: you're probably carrying out a task that the business wants you to do. Still, it's way better than "do it this way or else."
Idea Number 3: Build Your Own Data Store
Back in the days before Microsoft Access, there was a funny little database company called FileMaker. It had the odd idea that people who wanted to create their own contact lists, their own lists of the stocks they owned and their values, their own grades or assets and expenses, should be able to do so, in just the format they wanted. As Oracle steadily cut away at other competitors in the top end of the database market, FileMaker kept gaining individual customers who would bring FileMaker into their local offices and use it for little projects. To this day, it is still pretty much unique in its ability to let users quickly whip up small-sized, custom data stores to drive, say, class registrations at a college.
To my mind, FileMaker never quite took the idea far enough. You see, FileMaker was competing against folks like Borland in the days when the cutting edge was allowing two-way links between, let's say, students and teachers (a student has multiple teachers, and teachers have multiple students). But what people really want, often, is "serial hierarchy". You start out with a list of all your teachers; the student is the top level, the teachers and class location/time/topic the next level. But you next want to see if there's an alternate class; now the topic is the top level, the time at the next level, the students (you, and if the class is full) at a third level. If the number of data items is too small to require aggregation, statistics, etc.; you can eyeball the raw data to get your answers. And you don't need to learn a new application (Outlook, Microsoft Money, Excel) for each new personal database need.
The reason this fits BI is that, often, the next step after getting your personal answers is to merge them with company data. You've figured out your budget, now do "what if": does this fit with the company budget? You've identified your own sales targets, so how do these match up against those supplied by the company? You download company data into your own personal workspace, and use your own simple analysis tools to see how your plans mesh with the company's. You only get as complex a user interface as you need.
