What is Dark Data?
Simply put, Dark Data is Operational Data you currently do not capture or analyse, but is being generated every second through your CRM, ERP, SCADA, HTTP and WIFI systems. Consulting and Market Research company Gartner Inc. Describes Dark Data as 'Information assets that organisations collect, process and store in the course of the regular business activity, but generally fail to use for other purposes'.
With growing accumulation of structured, unstructured and semi-structured data in organisations - increasingly through the adoption of big data applications - Dark Data has come especially to denote operational data that is left unanalyzed. Such data is seen as an economic opportunity for companies if they can take advantage of it to drive new revenues or reduce external costs.
Some examples of of data that is often left dark includes server log files that can give clues to website visitor behaviour, customer call detail records that can indicate consumer sentiment and mobile geolocation data that can reveal traffic patterns to aid in business planning.
What are the limits of Dark Data?
The truth is, at present, no one actually knows the limits of what we can unlock or how far we can take this new technology. We know that companies are only analysing around 1% of their Data, which means that they are only generating 1% of potential revenue from this data!
When you think about this, the whole realm of unlocking Dark Data becomes very interesting for companies who want to sharpen the way they understand their business, as it allows them to make 'Better Smarter Business Decisions', increase income and decrease costs.
How can you unlock Dark Data?
There are 3 key steps to accomplish the most out of your Dark Data: capture, unlock and gain results through BI. While all 3 are definitely challenging, capturing data is arguably the most critical and difficult part.
Datumize Data Collector DDC1.2 unlocks Dark Data and Operational Data so that we, in collaboration with to our exclusive analytical partner Mind Analytics, can offer our clients an insightful visual analytic dashboard that will help them make the best use of their Dark Data.
Currently the industries who are benefiting the most from our DDC 1.2 are Retail, Leisure, Automotive and Utilities. Our Business development team works hard with clients to understand their current pains, and help turn them into solutions that will generate better revenues, increase productivity and help make those smarter business decisions.
What are the Costs of 'Dark Data'
Typically, Dark Data is complex to analyse and stored in locations where analysis is difficult. The overall process can be costly if not handled by Data Extraction Experts. The DDC 1.2 seamlessly and non intrusively integrates with your current systems without the need to change or adapt protocol, collecting real-time Data.
The most unique feature of the DDC 1.2 is its flexibility, which allows our engineers to pinpoint the exact data pain you are experiencing, capture your Dark Data which is then charged per TB.
What is the future of Dark Data?
While it may be difficult to quantify the extent of 'Dark Data' and dark integration, our clients are clear that it is large and important. I have been mystified as to how some of the enterprises in which i have worked with continue to function given the state of their application and data landscapes.
The only conclusion that i can come to is that there is something else going on in terms of information management that compensates for the very real limitations that exist. Somehow users are getting around the problems, and it can only be through 'Dark Data' . The denial, or ignorance, of enterprise information management (EIM) concerning dark data is not going to survive in the long term.
Data is increasingly being recognised as both valuable and a source of critical business changing solutions for enterprises. The focus of data management is beginning to swing from the logical world of models to the physical world of data values. Responsibility for data is becoming even more personnel with individual responsibilities being emphasised rather than anonymous applications or the enterprise as a whole. Dark Data, will inevitably be challenged and it will be up to EIM to tame it.
