Unstructured Data: a Challenge for IT Decision-Makers

by Alex KhizhniakJanuary 29, 2008
What's the state of corporate information in today's enterprises?

Terabytes of unstructured data

Found a great post by Paul Weinberg observing a recent survey report by Taneja Group, storage and server industry analysts. They surveyed 238 IT decision-makers in North America and Great Britain around different industries. To give you an idea about the audience, I’d quote that “53 percent of users had 11 terabytes or more of unstructured data in their environments.”

So, 62 percent of the respondents reported that the unstructured data within their companies was growing between 16 and 75 percent per year. Despite the fact that the dispersion in this result is definitely too wide, this seems to be true.

Taneja discovered that the major drivers for unstructured data growth among survey respondents are Microsoft Office (78 percent), e-mail attachments (66 percent), and backup and archival (81 percent combined).

As you can see, enterprises are still suffering from files disintegration. For them, it is still a challenge to manage e-mail attachments and MS Excel spreadsheets as a part of one-view structure or SOA.


Too expensive to solve

Furthermore, Steve Norall, a senior analyst at Taneja Group, is inclined to think that people are not going to move all of the data into a single storage space. Why?.. Due to huge expenses. (Yep, yep, open-source middleware is a Joker here.) So, he predicts that file management and integration companies will benefit from this and “should prosper.”

Finally, the majority of respondents expected their file management and control budgets would grow by up to 20 percent in the next 12 months.

This means the problem is really a headache and the executives are ready to pay for a solution. Besides the adoption of open-source data integration and file management software, I expect that any related services are going to be on the rise, as well. One of the probable solutions is to take an open-source tool set (like Apatar) and allocate a budget for customizing this software to your unique integration needs.

In this case, the unstructured data management costs may stay far below the 20 percent level expected by the executives.