Parchive
|
|
Parchive (or parity volume set archive) is an error-correction system that can be applied to a collection of files to allow recovery when one or more of the files is lost. It was designed to be used on Usenet for transmission of large files such as movies. It is implemented using Reed-Solomon error correction.
Files posted to Usenet newsgroups are broken into small parts and posted as text messages. Transmissions over Usenet are not guaranteed, so the larger the file posted, the more likely it is that some parts will become lost or corrupted along the way.
There are incompatible versions 1 and 2 of the file format specification.
For version 1, given files f1, f2, ..., fn, the Parchive consists of an index file (f.par) and a number of "parity volumes" (f.p01, f.p02, etc.) Given all of the original files except for one (for example, f2) it is possible to create the missing f2 given all of the other original files and any one of the parity volumes. Alternatively, it is possible to recreate two missing files from any two of the parity volumes and so forth.
The index files (*.par in version 1 and *.par2 in version 2) are not needed to recover any data. The indexes consist solely of hashes to quickly identify the target files, and their content is duplicated in every parity volume. Index files are used to quickly check for errors to see if additional parity files are required. They were most useful in version 1 where the parity volumes were much larger than the short index.
While Version 1 provided a great improvement in the way files were transmitted over Usenet, it was lacking in some respects, principally that if even 1% of a file was missing, Version 1 would require a whole recovery file to repair that 1% of a file. If 100 files were each missing only 1%, the missing data would total 1 file but Version 1 required 100 files to repair! Needless to say, this was tremendously inefficient.
To improve on this situation, a second version of Parchive was released which in the worst case would work about as well as Version 1 but in the best case, would provide a great improvement. Version 2 allows files to be divided into blocks and then allows recovery files to be as small as one of these blocks. Thus, if a fraction of a file is missing, you'll only need to download a few extra recovery blocks rather than entire (full-sized) recovery files as was necessary with Version 1.
External links
- Parchive project, with specifications and example implementations. (http://parchive.sourceforge.net)
- Slyck's Guide to the Newsgroups - PAR and PAR2, describes .par's use in Usenet file-sharing (http://www.slyck.com/ng.php?page=6)
- QuickPar: Windows PAR2 CRC Creation & Repair Tool (http://www.quickpar.org.uk/AboutPAR2.htm)