Breaking down data silos

I recently read an article by Dr. Vincent Granville titled “Data silos are unavoidable, and maybe desirable”. I completely understand where Dr. Granville is coming from, and to some degree, agree with him. Finding a data scientist with in-depth knowledge of insider information and understanding of how to connect these independent silos together would be a tremendous asset to any organization. However, as Dr. Granville states, what happens if this person gets sick, or decides to leave the organization? How would this affect that organization? This is where Ikanow comes in. We help eliminate these concerns and needs. While data silos may be desirable, in this situation, the ability to break down data silos can provide new insights and intelligence never before seen or realized.

So why mention this article?

A while back we created an infographic titled “Understanding Big Data”. Here we shared information around data types, tools, and a brief use case from one of our clients. I’d now like to expand upon this use case and provide more details around the problem this client faced and the solution we helped implement.

Our client is a U.S. Government agency and their problem was two-fold.

First, their data was in multiple, disparate databases (including both SQL and Oracle data formats). The databases were only able to be queried individually via very strict Boolean searches using cumbersome search pages. The search pages were cluttered with multi-select drop-down lists containing, in some cases, hundreds of items to pick through. This agency recognized the need for a more robust, free-text search with some latent semantic capability (catching documents with similar terms, even if the verbiage of the query wasn’t an exact match) with the ability to query all of their databases at the individual level or as a complete corpus of data.

Second, individuals within this agency had to deal with the very cumbersome task of matching up every element in their plans (some plans included more than 500 elements) with the appropriate documents in the databases mentioned in the previous problem. To compound the issue, this task is solely for the benefit of management-for-metrics and provides nearly no direct benefit to the end user performing the labor-intensive task. The users were required to query each of the disparate databases mentioned above and save those queries for each element of their plan. For a 500-line plan, this would mean 4,000+ queries. After work-shopping this process, it became apparent that all of the query keywords came directly from the element in the plan itself allowing for the query process to be streamlined or even automated.

We worked with this agency to unify these disparate data sources within one platform, Infinit.e, without changing or moving any of their data. Instead of performing multiple queries (for each data source), they were able to perform one query. This drastically improved their processes; however, to make this even more automated, we were able to transform this query into a 1-button process, which automatically pulls information from the appropriate plan element. This has lead to a tremendous amount of time-savings for the end-user which has helped to ensure that this ill-loved administrative task is considerably less obtrusive.

By implementing our platform, we were able to automate and improve their internal processes. What once took a week or more can now be accomplished in minutes.

What does this mean for your organization? Well, do you have disparate data sources (structured and/or unstructured) in which you would like to unify and gain intelligence from? Would you like to fuse relational databases with CRM systems, social media data, blogs and other data sets? This is the type of power and intelligence you can gain by using our platform. If you are interested in learning more, please feel free to contact me me directly.

Interested in receiving quarterly newsletters from IKANOW?

Learn more from IKANOW: