Data Discovery Tools vs Business Intelligence Software: Together is Better
2 August 2018
Over the past several years, the market for data discovery tools has grown at 2.5 times that of the overall market for Business Intelligence Software and analytics platforms (IDC). Likewise, BI and analytics remain the highest priority for Gartner clients worldwide again this year. So it’s an understatement to say that data discovery and BI are very hot market segments.
Reflecting those trends, Gartner even changed the primary focus for its annual Magic Quadrant for Business Intelligence Software and Analytics platforms to concentrate primarily on self-service data discovery and platforms that could be used by business managers without IT involvement.
What are data discovery tools/what is data mining software?
Used for data mining and knowledge discovery, data discovery tools enable users to explore and examine data to uncover trends and insights which may not be visible on the surface. Often such tools are implemented by individual users or teams within an organization to meet departmental needs.
While there are many clear advantages for line of business managers and data scientists to use their own data discovery tools without IT’s involvement, and no one can dispute their rapid growth over the past several years, it’s important that companies don’t lose sight of the value of full enterprise BI and Analytics platforms, the best of which now include self service data discovery as a key component.
To put the issues into proper perspective, consider the following five key factors that need to be evaluated in the process of choosing the right mix of data discovery, BI reporting and analytics functionality:
1. Who’s got the correct numbers?
As siloes of self-service data discovery crop up around a company, there’s going to be a great deal of confusion and faulty decisions from different results as those various departments apply their own customized models, from their own sources of data that have not been validated and are often inconsistent with other data sources being used elsewhere in the company. What looked like incredibly insightful analytical results back in one department’s conference room will suddenly look inaccurate when compared with another department that applied its own data discovery from questionable data.
That scenario then becomes the source of many a headache at the C-level when executives compare and contrast the two sets of results. They’ll likely recall that there were both compelling reasons and solid logic behind the search for a single source of truth in the early days of BI. But if the underlying data is wrong, then the entire exercise is flawed and the results will be equally as wrong.
2. Data Siloes Cost Money and Lead to Bad Decisions
When there are dozens of data analysts around a company relying on their own siloes of data to make crucial decisions, without the valuable input of other departments’ findings or properly governed central consolidation, costly errors are bound to result. Take for example if sales prioritizes closing a million dollar deal with a customer that has not been paying its bills for the past two quarters, or if marketing spends most of its budget pushing an item that is about to be dropped from the product line.
3. What’s good for one department, probably isn’t for the full enterprise
Analyses that benefit the entire company with real long-term value usually require far more sophisticated and complex modeling than departmental data discovery by casual business users allows. And the reverse is true as well, since a single insight from one department will probably be based on models and data that won’t scale to benefit the entire enterprise.
4. The productivity trap
While on the surface it would appear to be a good idea to empower inexperienced business users with self-service data discovery tools to conduct their own analyses, building morale and allowing the occasional “ah-ha!” moments, monitoring their time spent doing so usually reveals an inordinate number of hours wasted on irrelevant analyses. And that time, multiplied by five or six departments, can spell the difference between black and red ink at end of the quarter.
5. Repeat of the Excel nightmare
With the recent proliferation of departmental data discovery, too many companies are creating chaos from thousands of business users running hundreds of their own analyses. They are essentially repeating the nightmare of Excel spreadsheets everywhere, which modern Business Intelligence Software and Analytics platforms are supposed to replace. Haven’t we seen this movie before? And don’t we remember how badly it ends?
So how should businesses approach data analysis?
What’s become increasingly evident in recent years is that data discovery tools need to be an integral part of an enterprise BI system if they are to truly deliver the flexibility and speed required by users, while also safely and productively delivering holistic views of the business at an enterprise level, full trustworthy data consistency, and the correct level of data governance.
And it doesn’t end there. Tools must go beyond pure Business Intelligence reporting to offer the ability to predict future outcomes, deliver comprehensive planning capabilities and support better, more informed decisions business-wide. Only then can the true potential of data be exploited.
This is the essence of a successful Decision-Making Platform, which combines data discovery, business intelligence, analytics and planning, drawing validated data from a single database.