Storage needs are ballooning. Data volumes will soon overwhelm organisations like a never receding Tsunami if nothing is done about it. There are very few choices. Organisations must:
1. Get more storage space;
2. Archive / take offline data that is no longer relevant; or
3. Compress the data stored.
While falling disk prices and innovative storage systems have made it possible to hire more disk space or archive data effectively, it is data compression that has received a lot of attention in recent years. Compression not only saves disk space, it also saves bandwidth required for transmission of data over the network. Data compression includes data de-duplication and is relevant both for data storage and data archival.
Disk based “de-duplication systems” compress data by removing duplicates of data across the data storage system. Some implementations compress data at a ratio of 20:1 (total data size / physical space used) or even higher. This may be done by reducing the footprint of the versioned data during incremental or differential backup.
Vendors use a variety of algorithms to de-duplicate data. Chunking algorithms break the data into chunks for de-duplication purposes. Chunking can be defined by physical layer constraints, as sliding blocks or single instance storage algorithms. Client backup de-duplication systems use hash calculations to evaluate similarity between files for removal and referencing of duplicates. Primary and secondary storage de-duplication designs also vary. While primary storage de-duplication is directed towards performance optimisation, secondary storage is more tolerant of performance degradation and hence de-duplication algorithms are constructed with more leeway.
Until recently, data de-duplication was only associated with secondary storage. The increasing importance of the cloud as a primary storage has created a lot of interest in de-duplication technologies for primary storage.
We, at Backup Technology, offer an agentless cloud backup service powered by Asigra. Our software de-duplicates data at source and transmits only the compressed, bandwidth efficient incremented or differentiated ‘bits and bytes’ or blocks of the backup set, over the network. Global level de-duplication technology at source is also available for the hybrid versions of the cloud for local backups. The comprehensive solution is non-invasive and ideally suited for small, medium businesses, as well as enterprises. Why don’t you try out our software before you decide to commit yourself?
