Dark data creates blind spots for organizations

Dark data. It sounds ominous, but it’s not malicious; it’s simply data that’s not getting seen. For many organizations, it’s causing problems but they don’t even know it.

While dark data does have the connotation of being highly secure information or having negative attributes, said Greg Milliken, vice president of marketing at M-Files, the company takes a broader view: it’s data that is important to a business that has become invisible because it can’t be found effectively.

Dark data has always been around to a degree, but as companies get bigger and more people are capturing information, knowledge ends up stored in separate systems and silos. “Important data doesn’t get found,” he said. “It’s getting to be a great problem as the amount of data increases.”

In some cases, said Milliken, users may find themselves recreating data that already exists somewhere else within the organization. And it’s not the data is dark to everyone: for example, an organization’s customer support department might have some information on a customer, while the sales team has other information. Each could benefit from the other’s knowledge; sales staff need to know that a customer is not happy and has made a great deal of support calls.

The democratization of technology has meant many different users are entering data into different discrete systems set up by the line of business, said Milliken. In the past, an enterprise content management (ECM) system would be set up and administered by the IT department, and the more sophisticated ones were difficult to use.

There’s no rule of thumb on what proportion of data in an organization is dark, said Milliken, and in some cases it’s okay for it to be dark because it’s just clutter, but he said the premise of M-Files that is there is a fundamental opportunity for all business and all vertical industries to improve how they handle information.

Greg Milliken, VP of marketing at M-Files
Greg Milliken, vice president of marketing at M-Files

The opportunity is greater depending how a company was started and how it has grown, he said, and it may not be about the volume of data, but rather its criticality. The inability to find a small yet significant piece of information could have a negative impact on a business.

The ECM is largely considered mature, but for M-Files, everything is greenfield, said Milliken. The big ECM players may have all of the functions and features listed as part of their offering, but with the growth of cloud and mobile, the market is shifting. “A window has opened that needs to be revisited,” said Milliken. A lot of systems do what they do pretty good, but it’s often not very easy for the average user to connect the dots between different data, which inhibits effective adoption.

Opening up the world of dark data is reliant on connecting data in structured databases, such as CRM tools, to unstructured content throughout an organization. “The connection of these two is a clue to opening up the world of dark data,” said Milliken. The challenge with unstructured data is how users organize their files is completely arbitrary – their folder structure makes sense to them, but may not may not make sense to a co-worker. How information is organization is highly subjective based on the company and the individuals within it.

Unfortunately, there’s no ready clues to alert companies that they may have a dark data problem, although there are often indirect hints, said Milliken. An audit that took too long because information was difficult to track down or a bottleneck in a workflow. “Usually companies have some symptom — something else is awry in the company,” he said.

“Some processes aren’t working. Customer satisfaction is dropping. Audits are failing.”

Would you recommend this article?


Thanks for taking the time to let us know what you think of this article!
We'd love to hear your opinion about this or any other story you read in our publication.

Jim Love, Chief Content Officer, IT World Canada

Featured Download

Gary Hilson
Gary Hilson
Gary Hilson is a Toronto-based freelance writer who has written thousands of words for print and pixel in publications across North America. His areas of interest and expertise include software, enterprise and networking technology, memory systems, green energy, sustainable transportation, and research and education. His articles have been published by EE Times, SolarEnergy.Net, Network Computing, InformationWeek, Computing Canada, Computer Dealer News, Toronto Business Times and the Ottawa Citizen, among others.

Featured Articles

Empowering the hybrid workforce: how technology can build a better employee experience

Across the country, employees from organizations of all sizes expect flexibility...

What’s behind the best customer experience: How to make it real for your business

The best customer experience – the kind that builds businesses and...

Overcoming the obstacles to optimized operations

Network-driven optimization is a top priority for many Canadian business leaders...

Thriving amid Canada’s tech talent shortage

With today’s tight labour market, rising customer demands, fast-evolving cyber threats...

Staying protected and compliant in an evolving IT landscape

Canadian businesses have changed remarkably and quickly over the last few...

Related Tech News

Tech Jobs

Our experienced team of journalists and bloggers bring you engaging in-depth interviews, videos and content targeted to IT professionals and line-of-business executives.

Tech Companies Hiring Right Now