Archiving Data on the CD-R


Data from the 40" CCD observing runs is being archived on the Recordable CDs. These discs hold 650 Mb of data, which is about 340 of the current (1997) CCD frames.

The data must first be moved to a hard disk which can be accessed quickly by astsun. There is a minimum transfer rate which should be maintained when actually writing the disk, so it would be wise to use one of the /home/astro disks. Ask Eric to create a directory for you on one of these disks which has at least a free gigabyte. (It is not necessary to have one block of space; if needed you can read data onto several different /home/astro2-6 disks in order to fit it all in. But obviously it is easier if you can just work in one space on a single disk.)

The data is normally read on from an exabyte tape, using the exabyte drive connected to astsun (/dev/nrst5). Check to see if the drive has been cleaned recently by looking at the date inside the 8mm Cleaning Cartridge case. Insert the Cleaning tape if needed, and it will eject it after it has completed the cleaning (several minutes).

Once the drive is clean, simply load the data tape in the drive, log into astsun, and change to the directory on the /home/astro# disk on which you have lots of free space. Then just type "tar -xvf /dev/nrst5". This will extract the first tar file from the exabyte and write it onto the hard disk. If there is more to extract, type "tar -xvf /dev/nrst5" once to skip over the eof marker, and then again to start reading the next tar file. It would simplify things if one tar files was written for each run, since that is roughly how much will fit on one CD-R. One tar file per night makes it necessary to type "tar -xvf /dev/nrst5" repeatedly. Which means hanging around the computer the whole time.

Once the entire run, or group of runs, that will be written to a CD-R are on disk (remember the CD-R can hold 650 Mb with 5-10 Mb taken up by overhead.), you can eject the exabyte tape by typing "mt -f /dev/nrst5 rewoffl" (or keep reading more tar files onto disk for the next CD-R).

Start the GEAR software on astsun by typing "msgen" (multi-session generator). This should put you into the menu driven program. Type "help" for a list of commands and/or get the GEAR UNIX manual from Eric. I am assuming you will be writing a CD-R all at once. There is multi-session capability with this software: you could write some data, and then write more to the same disk at another time. The one time I did try this, I ruined the disk, but I think this is because I did not "fixate" the disk correctly. To be safe, I just write it all at once.

Type "NEWVOL PBK 74 ISO" to create a new virtual volume named PBK (named after the observing run), to be written in ISO standard on a 74 minute long disk. Virtual volumes allow you to prepare the disk for writing without making another entire copy of your data somewhere else on the hard disk. They are slower when writing the CD-R (according to the manual, I haven't tried to write a physical volume), but they prevent the need for an additional 650 Mb of disk space, and the time it takes to copy all those files.

Type "SELVOL PBK", which selects the virtual volume PBK as the one you will be dealing with. Actually, after you create a volume with the NEWVOL command, it is automatically selected, but I like to do this to be safe. Select track 1.

Type "cp /home/astro6/rjp0i/pbk/pbk1*" to copy all the pbk1* files in the /home/astro6/rjp0i/pbk/ directory into the virtual volume. Note that no files are actually moved, and this is a very quick process. The virtual volume bookkeeping files (PBK.** files located in the directory you started msgen from) can become quite large (several Mb), so make sure you have the space. Do not copy all of the files for a given run at once. When I have tried this, several files are not copied for some reason. That is why I copy one night at a time (pbk1*, pbk2*...). After you have copied all of the files that you wish to write to the CD-R, use the "ls" command to list all of the files which are in the PBK volume. At the end of the listing, it tells you the number of files in the volume. Make sure this is exactly the number which were on the tar tape. It is very useful to write this number down when making the tar tape: files are usually in numerical order, but invariably there will be a few frames which were deleted, leaving things out of sequence and making it difficult to count the total number of frames for a given run.

After you have used the "cp" command to copy all the files into the virtual volume, you are ready to write the CD-R. Load a Recordable CD into the caddy and insert it into the Pinnacle CD Recorder (to the right of astsun). Then type "WRITE PBK". It shows you a listing of the relevant settings, they should be as follows:

Using the virtual image for each track.

Recording of disc enabled.

Number of copies to write = 1.

1x real time speed selected.

Disc fixation enabled.

Multi-session writing enabled.

Change any which are set incorrectly, or just hit return to proceed. The image is verified (if you have moved any of the files on the disk after you created the virtual volume, it should bomb here, since it doesn't know where the files are anymore) and then it checks to see if a blank disc is in the recorder. Then it writes the disc. It records the progress in terms of percentage of the volume which has been written. Seems to take about 1.5 hours to write a disc. It then takes a few minutes to "finish" the disc, after which it ejects the disc.

You can check the disk by loading it into the normal cdrom drive, letting it spin up to speed for a few moments, and then type "cd /cdrom" on astsun. There should be a directory pbk/ there, which should contain all the files you just wrote to the CD-R. Compare the list of files there and the list in /home/astro6/rjp0i/pbk. Examine one of the files for differences: "diff /cdrom/pbk/pbk302.dst /home/astro6/rjp0i/pbk/pbk302.dst". It will take about a minute to compare the files and come back with no error message if they are the same. Type "diff /cdrom/pbk/pbk302.dst /home/astro6/rjp0i/pbk/pbk100.dst" just to see what happens when they really are different.

You can delete all the files in /home/astro6/rjp0i/pbk if you are satisfied with the results. Write the name of the volume on the top (non-green) side of the CD-R using a felt-tip oil based marker and also label the case.

In summary, load the exabyte and read the tar files using tar -xvf /dev/nrst5 until you have loaded all the files onto the hard disk. Then start up the GEAR software by typing "msgen", create a new virtual volume named after the run by typing "NEWVOL PBK 74 ISO". Then select the volume (to be safe) by typing "SELVOL PBK", and copy all of the files from the hard disk into the volume using "cp /home/astro6/rjp0i/pbk/pbk1*" and so on. Don't copy all the files at once, it seems to lose files this way. Type "ls" to see how many files are in the virtual volume, and make sure this is the same number that were on the exabyte. Then simply type "WRITE PBK", hit a carriage return if the defaults look ok, and about 90-100 minutes later the CD should eject.


[ricky@virginia.edu]
Last modified: 30 Jan 1997