Storage market faces open-source revolution

A provider of distributed, open-source grid-based storage plans to take on established proprietary vendors

The storage market has become the latest to face the open-source revolution, with a new product from Cleversafe claiming to offer secure, economical and private storage using a dispersed storage grid technology.

The product combines RAID attributes and the striping of data slices across multiple storage locations. It uses Information Dispersal Algorithms (IDAs) to separate data into 11 encrypted data slices and distribute them, via secure internet connections, to 11 storage locations throughout the world.

Transmission and storage of data is therefore inherently private and secure, the company claims.

Each data slice also contains a coded slice, a sum of the data values in the other data slices (mostly analogous to RAID parity data). Each data slice and its associated coded slice is stored in a different location.

Data on the grid remains private and secure in the face of natural catastrophes, or failures of hardware, connection, facility or IT management, Cleversafe says. No single entire copy of the data is in one location. Up to five of the locations can fail and all the data is still recoverable.

The individual data slices also do not carry enough information for an unauthorised viewer to determine the original content.

IDAs have been used on supercomputers to securely store critical and sensitive information but the technology is proprietary and costly. Cleversafe has open-sourced its IDAs, so commodity servers can be used to accomplish the same functions at a fraction of the cost. Using the freely available Cleverspace software, customers can therefore build their own storage grid.

Businesses are heading towards petabytes of data, even exabytes, and even ordinary citizens are crossing over from gigabytes to terabytes of information. As such, there is a vast potential mainstream market for dispersed storage grid technology and Cleversafe’s product may be the first of many.

Cleversafe has also announced a DSGrid File System (dsgfs), which allows a dispersed storage grid to appear as a mountable file system for Linux-based software applications.

“This new project means that most Linux-based applications will now be able to seamlessly take advantage of the benefits of dispersed storage, which include security, reliability and privacy,” says John Quigley, dsgfs project lead.

“Cleversafe is inviting organisations and individuals to freely participate in the project.”