Site icon IT World Canada

Compressed data search tech goes mainstream

After selling its data compression and search technology to the auto industry for six years, WindSpring is eyeing the storage market with a search engine that can find information while the data is still compressed.

The company was just awarded a U.S. patent for its Data Management Tools (DMT) product, which allows direct access to compressed data sets without the need for decompression.

WindSpring CEO Tom Hunt said by the end of this month his company will begin shipping products to storage equipment manufacturers and other firms. Hunt sees WindSpring’s offering as being particularly valuable to the mobile market, where the more data you can store in a smaller space the better. That typically means compression is needed.

Hunt claims most popular compression algorithms, such as PKZIP, haven’t changed much since the 1980s. They provided good methods for reduced data storage requirements, but offered no efficient way to search the data — other than to first restore it to a primary system and then decompress it.

“It’s literally the same codec that’s been round 20 to 30 years,” he said.

At a high-level, WindSpring’s flagship DMT software development kit (SDK) compresses data and indexes it at the same time; when a data search takes place, only the exact data requested is retrieved and decompressed.

“Our encoding mechanism creates a table where we know where everything is always. So when a search engine says ‘find this information,’ we go right to it,” he said. “So it’s instantaneous access, with very little overhead.”

If a company is already using backup or archival software that relies on an off-the-shelf compression SDK such as LZMA or LZJB for backup and database compression, WindSpring’s universal decoding capability can take ir or any industry standard compression algorithm, read it into its system and encode it for later searches.

WindSpring has been delivering its compression technology to the embedded market, including navigation systems in for several automakers including Mercedes Benz, Toyota and Lexus.

Along with the mobile market, WindSpring has its sights on a wide range of potential original equipment manufacturers, such as IBM, Oracle, EMC, HP, and NetApp, as well as disk drive makers. They could include the software in their products, helping corporations with regulatory compliance and legal discovery requests, Hunt said.

“You may have a DB2 or Oracle application and need to optimize your storage, but you still be able to search it, and you can’t keep building bigger and bigger data centres,” he said.

Exit mobile version