Professional backup · Hosting in France

CLURK Backup

Data deduplication

What is data deduplication? This technique reduces repeated data to help limit the volume transferred or stored, depending on the implementation.

Data deduplication

What is data deduplication?

Deduplication, also called data factoring or single-instance storage, is a data-storage technique that identifies repeated data sequences to save space.

In block-based deduplication, files are divided into chunks. Identifiers help detect repeated content and are recorded in an index. L’objectif de la déduplication est de ne stocker qu’une seule fois un même tronçon. Aussi, une nouvelle occurrence d’un tronçon déjà présent n’est pas à nouveau sauvegardée, mais remplacée par un pointeur vers l’identifiant correspondant.

Data deduplication — repeated data

Depending on the implementation, deduplication may run while data is written to storage, in the background after writing, or through a combination of these methods.

On supported CLURK Backup sets, deduplication aims to reduce repeated blocks. Available processing and settings depend on the agent, module and storage architecture.

Deduplication modes depend on the agent, module and destination. They must be checked in the backup-set and platform configuration.

Performance impact depends on the resources and data

CPU, memory, change rate and storage performance influence backup and recovery operations. A representative test helps assess the actual effect.

The benefit when copies are moved depends on the backup format and the transfer or replication method. Moving files to another destination does not automatically preserve the same deduplication ratio.

How deduplication works

Block-based deduplication seeks to avoid storing or transferring the same content repeatedly. Granularity and settings depend on the implementation.

Content fingerprints can be used to identify blocks that may be identical.

As data is processed, the deduplication mechanism compares fingerprints or equivalent metadata to identify repeated blocks.

When a match is found, the system checks, according to its implementation, whether an existing block can be referenced instead of writing another copy:

The backup metadata records the relationships needed to reconstruct the files and versions. Integrity checks and recovery tests remain important to verify that recovery works correctly.

Benefits of CLURK Backup deduplication

Virtual-desktop environments can contain many repeated files and therefore offer useful opportunities for deduplication. The actual benefit should be measured on representative data.

The result for databases also depends on block changes, compression and encryption. A unique database record key does not by itself establish whether blocks can be deduplicated.

Deduplication configuration

Check whether deduplication is available and enabled for the backup set, according to the selected agent and module. It is not assumed to be identical across all volumes or systems.

Configuration should balance transferred volume, stored volume, execution time and recovery needs. No fixed saving or performance gain is guaranteed.

Data deduplication — operation

← All CLURK articles

Your data deserves the right backup.

Let’s discuss your needs and data-protection strategy.

Request a free trial