Business Intelligence for the Enterprise

Preface

Let me begin by stating that this book is not intended to be used as a technical reference for your validation or argument in the evaluation or assessment of Business Intelligence solutions. I have avoided the use of products and acronyms whenever possible. This is not because I do not know the products or because I don't have an opinion about them. It is because I want you to think about what you are doing for Business Intelligence and why.

A "best-of-breed" BI tool can be implemented so poorly that it provides no value at all. A dated but well employed BI tool may bring spectacular business results. I probably will repeat myself a bit, including using the phrase "Activity does not equate to success." BI success does not lie in the volume of usage nor the volume of paper generated. BI success is measured in business impact and by the improvements in critical areas that can be attributed to its implementation.

I began working in the end-user computing arena in 1981 and have worked with nearly every tool and database on the market since then. Behind all these efforts lies a common theme--the need to produce key business information from existing data.

I do not pass judgment on any specific solution or vendor. I do, however, believe that too many organizations take the easy way out by selecting some tools, hoping the end users will "magically" emerge with what they want. Meanwhile, the rest of the "important" work in data processing goes on...and on...and on. If an organization decides to provide some data and analytics tools to end users to get them off the Information Technology (IT) Department's back, it will fail and the organization will end up spending much more in the long run because ultimately they'll have to attempt to get it right.

Your view and definition of BI will dramatically vary from others depending upon your role within the enterprise. If your position requires you to work with getting the corporate data into shape for BI, the issues you face will be drastically different than those of individuals trying to use what you have created.

If you are a non-technical end user, your definition probably will be filled with noble business initiatives and some usage and functional assumptions about the tools you will use.

An enterprise view of BI would combine definitions from all users and roles into a singular view with goals that span the enterprise--oops, I just gave away the rest of the material in this book. Please do not stop here.

Years ago, I was sitting in an introductory class to a new, exciting query and analysis tool. The instructor had been constantly hammering into us that the tool we were trying to learn was "intuitively obvious to use." One of my fellow students finally said, "I am sorry, but I don't seem to be getting it. I thought you said this was going to be intuitive!?" The instructor said, "Well, the more you use it, the more intuitive it becomes!"

We have all heard the adage that a million apes with a million typewriters given a million years would eventually write War and Peace. Sometimes, a BI solution can seem that way. If the users have to constantly flail at data with obtuse options in an attempt to produce a result, then something is dramatically wrong.

Business Intelligence solutions are anything but intuitive. Regardless of how many names and technologies have been applied to the discipline we now call BI, it is a difficult yet incredibly rewarding area in which to work. BI can change the course of how an enterprise operates and can keep an organization afloat in difficult times. BI elements can be used to discover trends and information that "normal" thinking would not have disclosed.

How does one get from "flailing" to spectacular success? Needless to say, your business has to be viable and in a position to increase profitability and implement changes. The heart of any BI success story begins with the corporate "intent" in implementing BI. Sometimes, a firm just gets lucky and discovers that increasing the level of information flow has a positive effect on the business. In most cases, many end users feel warmer and fuzzier because they are getting more information but could not begin to tell you whether it has changed the bottom line.

Business Intelligence is mostly query and reporting. In Chapter 2, I will apply a more formal definition of BI, but for now I will limit my definition to query and reporting. Of course, you may be asking, "Isn't there much more involved in today's BI solutions? What about data extraction, cleansing, and all the data preparation required?" Yes, there are many ancillary and necessary steps involved in delivering BI solutions. But at the heart of every solution is the need to access the data we've captured and to perform some analysis that we will use in our business.

Let's look at an Executive Information System (EIS). There is so much talk about "executive dashboards" today. Somewhere beneath the clever graphical interface and presentation lies some data and a corresponding set of values that were produced with a query tool. The trappings of the presentation style do not negate the need for a core technology to deliver the data.

The marketplace today is flooded with BI tools. There are numerous query and reporting solutions available, and most large organizations have installed several solutions that are being used for any number of business purposes. Are they all necessary? Are there significant, unique differences among them that justify having several "power tools" installed at once? The answers to these questions are usually "no" to the first and "maybe" to the second. Is there any harm in having sets of tools from different vendors? I submit that the answer to this question is a definitive "yes!"

The title of this book is Business Intelligence for the Enterprise. I have found that a very small number of organizations have attempted to establish a global view and effort in implementing BI. The majority of firms that practice BI tend to perform random acts of analysis on data.

The roots of BI date back to early mainframe query tools when data processing departments began to open corporate data stores to end users with 4GLs (fourth generation languages). These early BI implementations via 4GLs managed to excite the users as well as raise the frustration level on both sides.

IT was unhappy at the unexpected impact on existing systems, as well as the inevitable hand-holding required to get the tools working. IT also had to spend time it simply did not have trying to resolve many technical issues such as getting a tool to work properly in accessing a data source. The end users were unhappy because in every case there was far more work involved in extracting results from the data. There were far too many techno-geek aspects to getting the job done.

The worst impact was the rift between corporate IT and the end users. IT was perceived as keeping the keys to the castle and creating massive roadblocks to prevent the users from accessing their data. The end users were perceived as being technically inept and a bunch of "whiners" who always had to run to IT when the going got rough--which typically didn't take very long.

In every case, there were a few individuals who emerged as experts in the tools and usage. Some were from IT, and some were end users with a more technical aptitude. The number of individuals outside of IT that could actually do very much with an analytics tool was typically small. In many cases, these savvy end users went on to succeed at higher levels within the organization. Many of them are the drivers of point solutions today.

Let me present an example of an early BI implementer. Customer XYZ has a mainframe with lots of data stored in a variety of formats including VSAM, sequential datasets, and even tape that the users require for historical analysis. The data has been stored in efficient formats, such as packed-decimal to take up as little disk space as possible. Many of the text fields have intelligent keys where a couple of characters are used to describe longer information items (e.g., two positions in a longer field used to signify districts or other lengthy information).

This company has discovered a vendor tool that allows it to access and query this legacy information. However, setting up the access and performing actions, such as stripping out the intelligent key positions and translating them into understandable text, is not trivial. The vendor's suggestion is to extract the data in its raw form, transform the encoded values to understandable ones, and store the data in the vendor's proprietary format.

Even when the data has been molded into a source that is better for common business use, it is still not structured for the non-technical end users to do much with it. The processing requirements and calculations are more challenging than many end users can handle, so we now have an enriched set of data held in a proprietary source. However, the era of end users performing their own query and reporting functions began, and it has been in a state of continuous change ever since.

Today, there are more choices for analytics tools than years ago. Instead of having the data, the tools, and the analysis processes on the same platform (e.g., S/390), one can now remotely analyze a server from afar using tools on workstation platforms. Many corporations allow users to perform these analytics from a variety of tools with varying results.

The difference between traditional BI and BI at the enterprise level is significant. I am an avid fan of using fewer tools and centralizing BI at the server level. I use several keywords in discussing BI with customers. One of the first ones I toss out is math. BI is all about "the math."

BI users often want to perform very specific and difficult calculations. Regardless of the platform, the tool, or the technologies, few BI solutions allow users to drag some columns onto a report and select the Run option. Even the most elegant data warehouse structure assumes the users will need to perform some calculations beyond the basic storage structure of the data. Where and how one decides to perform the math can have an enormous impact on BI architecture

The other word I use in discussing enterprise level BI is awareness. Do different user departments, groups, and so on have effective intercommunication with each other such that they can share their BI efforts? In the overwhelming majority of cases, the answer is a ringing "no!"

Because BI efforts require data and math, does it not make sense that one would try to condense as much of the effort into as little iteration as possible? We probably have all heard about the scenario where different users walk into a meeting with different results supposedly based upon the same data. Do these events occur because they are using different tools or different data? What if they are using the same tools, but have different skill levels? What if all the results are incorrect?

The objectives for quality BI efforts are many. The guidelines I recommend in establishing a successful BI environment are scattered throughout this book. I have included a series of checklists in the Appendix at the end of the book, and I hope that they will assist in your selection of BI solutions. Now let's define this thing we dub Business Intelligence.