Validation Overview
Creation Date: August 24, 2006
Revision Date: August 24, 2006
Product: DS‑Client
Summary
The Validation process is designed to verify the restorability of online data.
See Also
Without the customer / DS-Client encryption keys, DS-System is not able to create the digital signature to validate backed up data for some types of corruption. To avoid transmitting large amounts of data from DS‑System to DS‑Client for validation, DS‑Client will send the encryption keys to DS‑System based on your ACCEPTANCE / ACKNOWLEDGEMENT / AGREEMENT to this option.
The decision to perform validation must be made from the DS‑Client side. The customer knows what data is more important (i.e. data that needs to be validated more often). When a Validation process is triggered on DS‑Client side, DS‑Client sends a request to DS‑System to perform the validation.
For each file (generation), DS‑System will first check the file header and delta linking/library linking, etc.
• If all the above parameters are correct, it will try to validate the data by performing a virtual restore. The data will be decrypted and decompressed to generate the original signature.
• If it fails due to a decryption or decompression problem, the validation fails.
Finally, it will compare the generated signature with the original one: if it does not match, the validation fails.
For any validation failure, the error will be reported on both DS‑Client and DS‑System.
• If the validation fails due to reading or network problems, it is unknown whether or not the file is valid. Validation will skip the file and report the corresponding errors on both DS‑System and DS‑Client event logs.
• For other failures where DS‑System can confirm the file is corrupted, the file will be moved to the trash (\del directory). All dependent files will also be moved.
Since the validation process will read all data to check the digital signature, the validation process is Disk I/O intensive on the DS‑System side.
The whole process is almost the same as an actual restore. Instead of writing to a target location, the decrypted/decompressed data is deleted after generating the signature.
The following options are provided to help validate online data in the most flexible and efficient way: validation of all data, selective validation, scheduled / on-demand validation, excluding data deleted from source, and resuming mechanism, etc.
Validation functionality description
The customer enters the encryption keys (from DS-User) and DS‑Client verifies them. A message is displayed with a description of how the encryption key(s) are involved in the Validation process.
The encryption keys are encrypted when being transmitted from DS‑User to DS‑Client, and from DS‑Client to DS‑System. Encryption keys are only used to perform the data Validation: DS‑System will not keep the keys after the Validation has finished.
The Validation process can be triggered on demand or scheduled.
On-demand Validation has two options:
• Validate all online data (all generations of all files).
• Selective - select directories/files as well as generations to validate.
The execution sequence of scheduled activities for a backup set is as follows:
1. Perform Backup
2. Enforce Retention
3. Perform Validation
4. Perform BLM.
When setting a schedule to perform Validation, the customer must enter the encryption key(s) and DS‑Client verifies them. The Validation activity can only be enabled in a schedule if the user provides the correct encryption key(s).
The scheduled Validation has options to:
• Validate all online data (all generations of all files) or the last generation only (snap shot)
• Include or exclude files already deleted from source.
The scheduled Validation also has the option to resume from the point of interruption of the previous scheduled Validation process.
If a scheduled Validation completely processes all selected files, the next scheduled Validation (for the same backup set) will process files from the beginning.
To determine the interruption point, the files are processed in sequence based on the order of share ID, directory ID, file ID and generation.
NOTE: On demand Validations do not have the resume mechanism.
Validation is not available for statistical backup sets, since there is no data on DS‑System to validate.
• If a file corruption is detected (including Digital Signature does not match errors), that file and all files that depend on it (occasionally, a delta may formally depend on a file, but does not actually depend on it) are moved to the \del directory. The corresponding error will be reported on both DS‑System and DS‑Client sides.
• If a file’s restorability status cannot be determined temporarily (networking problems, etc.), DS‑System will skip the validation for this file. The corresponding error will be reported on both DS‑System and DS‑Client sides.
• If a file originally did not have a signature that is needed for Validation, a warning will be reported.
• If any bad files are removed, DS‑System will mark the backup set as out of sync at the end of the Validation process.
Only users with the Administrator Role on DS‑Client are allowed to start a Validation process.