Using ZEUS MP at UVA

This document gives a brief outline of how to use ZEUS MP, as we have it set up in the balmer CVS repository. If you have any hangups or questions, please ask Sean, and Kris and Rick are also able (and possibly even willing) to help. I will assume you are working on Hyades.

1. GET A COPY OF THE CODE

   You will get a copy of the code using CVS. (If you've never used CVS before, checkout the CVS homepage at http://www.nongnu.org/cvs/ for documentation.)

   You'll first need to go see Howard to have your username added to the CVS repository list and pick your own CVS password.

   Then, on Hyades, you need to set the following environment variables
   

   $ setenv CVSROOT :pserver:USER@cvs.astro.virginia.edu:/local/cvs/ (Replace the "USER" with your username.)
   $ setenv CVS_RSH ssh
   $ setenv CVSEDITOR pico #Optional (you can pick vi, emacs, ...)

   This syntax is for c shell. It will bee different for bash. You may also want to put these lines in your .cshrc or .bashrc file.

   Now run
   

   $ cvs login

   Enter your password you set up with Howard. Then run
   

   $ cvs checkout ZEUS_MP

   This creates a directory named ZEUS_MP, containing all of the source code. This is created in the current directory (where the cvs command is run), and this is now your personal copy of the code.

   $ cd ZEUS_MP
   $ more README
   

2. SETUP A WORKING DIRECTORY FOR YOUR PROBLEM

   Use the ZEUS_MP/README documentation and look at the setup_zmp script to learn how to create a working directory for you to run/develop a particular problem with ZEUS MP. In this example, we'll setup the 2D hydro Rayleigh-Taylor instability with a single mode perturbation, a problem called RayTay_single. Run

   $ ./setup_zmp -p RayTay_single

   This creates a working directory with all of the code, etc. you need, called run_RayTay_single.

   $ cd run_RayTay_single

3. COMPILE ZEUS MP

   First you'll need to add some modules. Use

   $ module avail

   if you want to see what is available. Then run

   $ module add icc-ifc/8.1
   $ module add mpich/1.2.7-intel81

   You may want to put these lines in your .cshrc/.bashrc file.

   Now you can use the Make_zmp script to create a Makefile and run it automatically. Run

   $ ./Make_zmp -m hyades compile

   This will compile the code. Look at the Make_zmp script to see other options (like optimizing, etc.). After running this once, you can recompile the code using

   $ make clean
   $ make

4. RUN ZEUS MP

   The executable is named zeusmp.x, but on Hyades, you should submit your job using the qsub script called runzmp.sh. First modify this file to your liking. Also, you can modify the input file "zmp_inp" to your liking. Then run the code using

   $ ./runzmp.sh

   This creates a directory called "runmm" (by default; the "mm" is specified in the zmp_inp file), in which your data is dumped. Your data files have names like "hdfmm010200.000" where the first set of numbers corresponds to the processor it was created on, and the last three digits give the output number (starting at 000, and increasing by 1 for each data dump). You will want to patch together all the data files that have the same last three digits. This is done with a postprocessor, described in the next step.

   You will get an email when the run starts and finishes.

5. POSTPROCESS YOUR DATA

   To patch together your data such that each data dump is contained in a single file, you need to run the postprocessor packaged with ZEUS MP. First you need to compile this.

   $ cd ... ZEUS_MP/source/zeusmp2.0.2/pp
   $ make clean
   $ make

   This creates the postprocessor executable zmp_pp.x. Now you can either copy this executable to the directory with your data, put ZEUS_MP/source/zeusmp2.0.2/pp in your path, or create an alias. Once you've done that, move back into your data directory ZEUS_MP/run_RayTay_single/runmm/ and run the postprocessor,

   $ zmp_pp.x

   It will prompt you to give a task. Type

   auto_hsplice

   and hit enter. Then it will ask for beginning and ending dump numbers (corresponding to the last three digits of the data files). Type (e.g.)

   0 15

   (that is zero, space, fifteen) and hit enter. Then type quit, enter. You will now see your patched together data files with names like "hdfmm.000" You are now ready to view/analyze the data.

6. VIEW/ANALYZE YOUR DATA

   Move your data to whatever machine you'll do your analysis on. Use your favorite tool. We'll leave this here for now, since we are still developing the analysis tools in IDL. There are some idl tools already available...

7. DEVELOPING YOUR OWN SETUPS

   Read the documentation in the ZEUS_MP/README file to learn how to create your own problem setup. Eventually, you may want to make your own additions to the CVS repository for all to use. So here's a few more words about CVS:

   Any time someone has made a change in the repository that you want, just run

   $ cvs update

   in your ZEUS_MP directory. This gets you the latest copy of everything in the code, and identifies which files in your own copy that you've changed. It will not overwrite your changes; if there is any conflict between your changes and ones that someone has added to the repository, CVS will let you know, and you can reconcile this by hand.

   If you want to see how your own file is different from the repository copy, use cvs diff. To get a modification history on a file, use cvs log.

   If you think you know what you are doing, and you are sure your changes are ready to submit to the repository, you can use the following commands:

   cvs add
   cvs rm
   cvs commit

   Read the documentation at http://www.nongnu.org/cvs/ for descriptions of these and all other CVS commands.