Bringing Search and Business Intelligence Together

There’s been some discussion within various BI and search tech circles recently about the use of search within Business Intelligence. It seems that the big question is how to knit the two together into a complimentary system. Earlier in the decade, Business Objects, Cognos and other BI firms acquired or built search UIs on top of their BI report and meta data repositories. So search on top of BI. More recently, other search technology vendors talk of having an ODBC interface (and support for SQL) so that BI tools can query search indexed data. So, this time, BI on top of search. In both cases, search is added to a preexisting core of classic BI technology.

The challenge is that the notion of BI has mutated a bit from Howard Dresdner’s 1989 definition of Business Intelligence: “concepts and methods to improve business decision making by using fact-based support systems.” How so, you make ask. Over the last decade or so, BI innovation has been mostly about providing technology that allows business analysts and managers to gain access to business metrics – numeric data about various facets of a business. So, mostly clever ways to paint numbers onto a display as a report, view or dashboard, to speed the query and summation of numbers, to enable complex deterministic and even stochastic algorithms and to manage metric data (data warehouses).

Yet, so often the difficult questions that managers and executives are attempting to answer are the ‘why’ questions. By their very nature, numbers can often obscure cause and effect relationships. Yet, these relationships are often the key to determining how to embrace an opportunity or counter a threat. Herein lays the value of unstructured information or, more precisely, descriptive text that gives insights into cause and effect and illuminates the nuance to relationships. Of course descriptive text often lacks that unique identifier that enables an RDB to join it to other data. This is where semantics and natural language processing can add value, by relating the cause and effect relationships embodied in unstructured data with quantified metrics in structured data. But this kind of matching isn’t an add-on to BI. It’s an activity that requires a different data processing approach, both during user queries and during data preparation. So, perhaps, rather than consider the idea of adding search to BI, an alternative approach yielding new, as-yet unrealized insight is to add BI to search. The concept is to define a set of structures that constitute important entities of a business – we call them business items. The quantitative metrics are added as attributes to that core set of entities, rather than vice versa.

This is where we at Exalead see the next generation of business intelligence going. The idea is to identify business entities as the atomic elements of a virtual representation of a business. Then search and semantics technology can be used to describe the state of and relationships between these entities, including the user of quantitative metrics in those descriptions. In this manner, executives can view their data as they view their businesses – as a collection of valuable relationships with cause and effect implicit in the descriptions.