|
Article ID: 243
Last updated: 26 May, 2023
It is a good practice to confirm whether your files are complete and accurate before you transfer the files to or from NAS, and again after the transfer is complete. The easiest way to verify the integrity of file transfers is to use the NAS-developed Shift tool for the transfer, with the --verify option enabled. As part of the transfer, Shift will automatically checksum the data at both the source and destination to detect corruption. If corruption is detected, partial file transfers/checksums will be performed until the corruption is rectified. For example:
In addition to Shift, there are several algorithms and programs you can use to compute a checksum. If the results of the pre-transfer checksum match the results obtained after the transfer, you can be reasonably certain that the data in the transferred files is not corrupted. If data is corrupted during a transfer, a good checksum algorithm will yield different results before and after the transfer, with high probability. The following checksum programs are available on HECC systems:
For example:
To check the integrity of an existing tar file against a directory:
The md5sum utility is more reliable than the sum or cksum commands for detecting accidental file corruption, as the chances of accidentally having two files with identical MD5 checksums are extremely small. The utility is installed by default in most Unix, Linux, and Unix-like operating systems. We recommend that you compute the md5sum of a file before and after the transfer. The following example shows that the file filename is complete and accurate after the transfer, based on its md5sum.
See the sum, cksum, md5sum, and msum man pages for more information on these commands. |