VERSION: JUNE 2012

A GUIDE TO IDL FOR ASTRONOMERS

R. W. O'Connell



Contents

Translations



I. INTRODUCTION



Section I of this Guide is for those who are unfamiliar with IDL or who are trying to decide whether to adopt it. Sections II and III are for users. Although the examples are taken from applications in astronomy, most of the Guide is general enough to be useful to workers in medical imaging, geophysics, or other areas for which IDL is well suited.

1. Key Features of IDL

[Up to Contents]

The Interactive Data Language (IDL) is a proprietary software system distributed by Exelis Visual Information Solutions, Inc. (http://www.exelisvis.com), originally Research Systems, Inc. IDL grew out of programs written for analysis of data from NASA missions such as Mariner and the International Ultraviolet Explorer. It is therefore oriented toward use by scientists and engineers in the analysis of one-, two-, or three-dimensional data sets. Exelis claims over 150,000 users.

IDL is currently available in LINUX, UNIX/Solaris, Windows, and Macintosh versions. IDL device drivers are available for most standard hardware (terminals, image displays, printers) for interactive display of image or graphics data.

IDL is not simply a package of task-oriented routines in the style of astronomical software systems such as IRAF or CIAO. Instead, it is genuinely a computer language, readily understandable by any computer-literate user. It offers all the power, versatility, and programmability of high level languages like FORTRAN and C. But it incorporates three special capabilities that are essential for modern data analysis:

Users who are conversant with FORTRAN, C, C++, or other high level languages will have little trouble understanding IDL. Its syntax and operation are clear, sensible, and convenient (most similar to FORTRAN's). Because it is interactive, learning IDL through on-line trial-and-error is rapid.

IDL provides the scientist better understanding of and control over computations and data analysis by virtue of a large number of special features:

A functional description of IDL is available at http://www.exelisvis.com/portals/0/pdfs/idl/IDL7_FuncSumm.pdf. More history, background, and context may be found at http://en.wikipedia.org/wiki/IDL_(programming_language). A description of the IDL command-line environment as seen by the user is given below.

Eight versions of IDL have appeared to date. Version 8.1 is the latest.


2. IDL in Context

[Up to Contents]

What role does IDL play in the context of other software readily available to astronomers?

FORTRAN, C, or C++ cannot satisfactorily serve the need of individual users for interactive data analysis because they do not provide a standard interactive environment with supplied graphics device-drivers.

The data reduction and display software with which most astronomers are familiar, including IRAF, STSDAS, AIPS, AIPS++, CIAO, MIDAS, and SUPERMONGO, consists primarily of collections of specialized, task-oriented routines. They are interactive and graphics-oriented, but they operate only in a pre-compiled form that offers the user little opportunity for upgrade or customization. Constituent routines function like "black boxes" and do not provide the user with easily understandable access to their inner workings. These packages can be executed from scripts, but they are not intended as the basis of user-written applications. They are complex enough that professional programmers are required to maintain and enhance them. They adjust only slowly to user requirements. By contrast, not only does IDL offer greater transparency, versatility and efficiency, but also the whole computer-literate user community may be drawn upon to extend and improve IDL applications software.

Comparison with other popular systems:

The intrinsic capabilities of IDL coupled with its extensive user code libraries greatly enhance scientist efficiency.


3. IDL Applications Packages

[Up to Contents]

Because of IDL's versatility, transportability, and ease of use, there is a large, user-written, public library of interactive IDL software, now over 10,000 programs.

Many astronomy-oriented IDL routines and packages are in the public domain.

The IDL source code for applications packages is automatically available, so the user can use the routines as written or, alternatively, readily modify their components (as one would FORTRAN subroutines) to customize them.

IDL is ideally suited for software exchange over the Web. Because no pre-compilation is required, installation of a new package, for instance, is simply a matter of putting the ASCII source files in your IDL path.

IDL's cross-platform design ensures that you can confidently transfer code between environments (e.g. from a Solaris desktop to a Macintosh laptop).

C or FORTRAN programs can be executed from within IDL and data exchanged between the systems.

For examples of IDL computational and graphical applications, run the idldemo demonstration from the LINUX command line.

See below for sources of IDL applications code.


4. THE IDL ENVIRONMENT

[Up to Contents]

Once you start IDL and begin typing at your console, you are communicating with the main level of an arbitrarily large computer program over which you have (nearly) complete control.

People who are familiar with IRAF, CIAO, or AIPS will immediately recognize major new features in the IDL environment.

IDL is oriented toward image analysis and signal processing, but by comparison to IRAF, etc. it also opens a new dimension of capability to make other kinds of important numerical calculations with great efficiency and to create graphical or tabular representations of them. In this regard, it duplicates much of the functionality of MATHEMATICA, MAPLE and MATLAB (though without symbolic manipulation).

Basic IDL functionality has not changed for the last several major releases. Instead, additions have involved areas like GUI support and object-oriented programming, which are directed more at software developers than users interested in data analysis.


5. IDL Limitations

[Up to Contents]

IDL has many virtues, but what are its limitations?

An obvious limitation, and a significant barrier for some people, is that IDL is a proprietary system, which means that each site must purchase an IDL license. Some astronomers object on principle to paying for software.

IDL is an interpreted rather than a compiled language. This means that large IDL programs can execute less rapidly than equivalent compiled programs written in FORTRAN or C.

IDL works best on moderate-sized data sets (say up to 1 GB) and where one does not need to reference individual array elements. Users needing to batch-process large amounts of data with more sophisticated algorithms may find FORTRAN or C routines to be preferable. However, these can be linked into the IDL interactive environment and executed from within IDL.

Because of its optimization for interactive computing, IDL is not the system of choice for large scale numerical computations (e.g. hydrodynamical or N-body simulations). However, it is valuable as a medium to explore new computational approaches in a smaller setting where raw speed is not important, and it is also excellent for visualizing, analyzing, editing, and displaying numerical data sets generated by simulation software.

A problem for novices is that help with IDL may be hard to come by at a new installation. There are, however, consultants available at Exelis, Web-based advice sites, large amounts of on-line or printed documentation, and the examples of thousands of working IDL routines in the public libraries that can help solve many software difficulties.

Data reduction packages are not available in IDL for most of the specific instrumentation available at the major observatories (e.g. CCD mosaic imagery, multi-object spectrographs, echelle spectra, etc.).

Finally, the rapidly proliferating set of IDL applications routines is both a strength and a weakness. While one has access to a wide variety of useful software, this is not always fully tested since the authors typically apply it to problems of limited scope. The hardest part of using IDL often is determining what routines are available for a given application and deciding which is best to use.

Overall, in exchange for its improved versatility and power, IDL requires a higher level of computer skill than do systems like IRAF, AIPS, or CIAO.

On balance, IDL is an invaluable tool for most observational or theoretical astronomers.


6. DEMOS, TUTORIALS, SOFTWARE LIST

[Up to Contents]

Resources: For lists of important astronomy-oriented IDL resources (guides, help files, demos, tutorials, software) other than those packaged with the Exelis distribution, see:

Demonstrations: To get a feel for what IDL can do, try running the package of standard IDL demos supplied by Exelis. Start from the LINUX prompt (within an X-window) and type idldemo.

Tutorials: A set of three introductory IDL exercises which introduce its basic features is available on the UVa ASTR 511 home page. Other sites offering IDL tutorials are linked to the Astronomy User's Library.

Software List: Six different levels of IDL code will be useful to you:

Acknowledgements: In any published work utilizing IDL software written by others, you should acknowledge the author. If you use the Goddard software, the GSFC Astronomy User's Library group and curator Wayne Landsman should be acknowledged. Modified versions of others' software should propagate the authorship list in the header section of each routine.



II. GUIDE TO IMAGE PROCESSING
WITH IDL


[Up to Contents]

This section provides an introduction to intrinsic IDL and user-supplied applications routines frequently used in 2-D image processing and related kinds of data evaluation and computation. Only the most common options for each command are listed.

Assumed:

This guide is oriented toward the UVa/LINUX installation of IDL. However, I have tried to clearly distinguish details which are specific to the local system so that others will be able to use the guide.



1. STARTING & STOPPING IDL

[Up to Contents]

  • To enter IDL: type idl at the LINUX prompt from within an X-window. This starts an IDL session running in "command-line mode" in the terminal window from which it was called. I recommend that IDL novices plan to learn the basics in this environment, although they should certainly explore the alternatives listed next.

  • Troubleshooting at Startup

  • To customize the IDL environment, you can execute any number of special instructions at the start of each session.

  • To give LINUX system commands from within IDL: enter $ as the first character on the command line

  • To interrupt and resume IDL: use the standard ^z and fg LINUX commands.

  • To interrupt an IDL routine: type ^c. To continue the same routine, type .con. To exit the routine after an interrupt and return to the main level, type retall.

  • To repeat or edit and execute an earlier command:

  • To continue a long statement on the following line: end the line with a dollar sign ($). You may do this anywhere in the line where a space would be allowed except within a string variable.

  • To give multiple commands on a single line: separate them with the ampersand (&). E.g.:

  • To leave IDL: type exit or ^d. All windows and data in RAM will be flushed. If you want to save data, use the save command or various other file writing commands before exiting.




    2. HELP

    [Up to Contents]

    Exelis Documentation

    Other IDL Help Resources:

    Determining IDL Version Number

    Learn by Example

    Informational Commands


    3. PROGRAM EXECUTION

    [Up to Contents]

    All intrinsic IDL programs are compiled and ready for execution when you begin your session. Other programs are normally compiled only when you request them. A list of all compiled non-intrinsic routines is presented if you type help,/rou.

    NOTE: all IDL main programs, subroutines ("procedures"), and functions are assumed to be in files with the explicit extension '.pro'.

    Variables

    Mathematical Operators and Functions

    Executive Commands

    execute: The execute command executes any string containing a proper IDL command. The syntax is

    Programs

    Programs are structured text files of IDL commands. They can be stored anywhere in your IDL path. There are three different types of programs, each of which interacts slightly differently with your active IDL session. Coding rules differ slightly in each case.

    Program Examples

    To illustrate some of the differences between main programs, procedures, and functions, here are three corresponding versions of the code for determining the length of the first dimension of a given variable (scalar or array) using the built-in SIZE function. Each version is assumed to be stored in a file named get_dim1.pro in your IDL path.

      [Refer to the definition of the SIZE function in the IDL help files. Note that the form of its output -- the intermediate variable s in the examples -- depends on the number of dimensions in the input variable. Its first element contains the number of dimensions (zero for a scalar); its second element is the length of the first dimension. Note that any text after a semi-colon (;) is optional and is ignored by the compiler.]

    1. Main Program

      
               ; MAIN PROGRAM: get_dim1.pro 
               s = size(a_in)
               if (s[0] eq 0) then d1 = 0 else d1 = s[1]
               end
      
    2. Procedure

      
               PRO get_dim1,a_in,d1
               s = size(a_in)
               if (s[0] eq 0) then d1 = 0 else d1 = s[1]
               return
               end
                  
      
    3. Function

      
               FUNCTION get_dim1,a_in
               s = size(a_in)
               if (s[0] eq 0) then d1 = 0 else d1 = s[1]
               return,d1
               end
      
    To execute these three routines during an interactive session and to print to the terminal command window the resulting value for the first dimension, the commands would be as follows. In all cases, the input variable a_in must have already been defined during the session. Note that you do not have to give a .run command to compile a procedure or function as long as its file name matches its logical name (except for the .pro extension).

    1. Main Program

      
                .run get_dim1
                print,d1
      
    2. Procedure

      
                get_dim1,a_in,d1
                print,d1
      
    3. Function

      
                print,get_dim1(a_in)
      
    Tips

    Importing software


    4. DATA AND IMAGE RETRIEVAL

    [Up to Contents]

    [Down to Data and Image Storage]

    Data transfer to memory

    Directory Routines

    ASCII Files

    readcol is a very handy AstUseLib utility for reading ASCII files consisting of separate columns of numerical data without having to specify a file format. An example of the use of readcol is given in Plotting Example 4 below.

    Binary Files

    FITS Files

    The standard data format for most astronomical data sets today is the
    Flexible Image Transport System (FITS) format. The IDL Astronomy Users Library contains a large package of routines to manipulate FITS files. Intrinsic IDL supports a variety of other common scientific data formats (see the manuals for details) with similar functionality.

    SDAS and IRAF files: These file formats, which store data and header segments separately, are now deprecated, but the AstUseLib has read/write routines for them. See strd and irafrd, respectively.

    Other Image Formats: Endian-Conversions



    5. IMAGE DISPLAY

    [Up to Contents]

    Much of the power of interactive computing comes from your ability to make immediate displays of plots or images on your terminal. Hardware considerations impinge more on making such displays than on other aspects of interactive computing.

    This section describes the underlying mechanics of image graphics and the basic IDL command-line routines for making image displays. If you are only interested in getting a quick display of astronomical images, skip to the description of the ATV display tool in the next section. For making plots, see 10. Plots below.

    Color Graphics Displays

    Displayed Arrays and Color Tables

    Setting a Visual Display Mode on Your Computer

    1. The first step is to determine which "visual display classes" your hardware can support. At the LINUX prompt, type xdpyinfo. (You must be in X-windows to use this command.) This will print a list of the display configurations possible on your system. The entry "class: Pseudocolor, depth: 8 planes" indicates that your system supports 8-bit pseudo-color displays. The entry "class: TrueColor, depth: 24 planes" indicates your system supports 24-bit true-color displays. Some systems will support both of these.

        On Apple MAC OS-X systems, the standard X-11 package can be set to different visual classes. Start X-windows. Click on X11 ===> Preferences ===> Output. The "Colors" menu gives you a choice of the numbers of colors possible. "256 colors" is pseudo-color mode. "Millions of colors" is true-color mode. (Most X-11 releases support both.) Make your selection. You must quit the X-11 program and restart it for the changes to take effect. You can check your selection by giving the xdpyinfo command within an X-window.

    2. Define a visual class for your session. Start IDL. The first command you give that references a window display function or the device utility determines the visual display class that will be used throughout your session. You cannot change classes after that first command. Therefore, it's usually best to put explicit configuration commands using a device call in a startup file that is executed before your interactive session begins. The three relevant forms of the device call are the following:

      • For 8-bit pseudo-color: device,retain=2,pseudo_color=8

      • For 24-bit true-color: device,retain=2,true_color=24,decomposed=1

      • For 24-bit emulated pseudo-color: device,retain=2,true_color=24,decomposed=0

      The retain=2 keyword requests IDL to handle the "backing store" for your display (which holds copies of parts of your display that have been overwritten); this avoids malfunctions that can occur if instead your windows system does the backup.

      These commands must be given before any window is created during your session.

      In the absence of a device command, the default assumed on a 24-bit monitor is true_color=24,decomposed=1.

      To check the current status of your graphics device, type help,/dev. (Note: the emulated pseudo-color mode is indicated by the entry "Graphics pixels: Combined".)

      Although you cannot change the visual class after its first implementation during a given session, in 24-bit mode you can toggle between device,decomposed=1 and device,decomposed=0 as desired.

    3. Reserved Colors on 8-bit Monitors: With an 8-bit monitor, the maximum number of colors available for all applications is 256. When IDL is invoked, it will normally be able to obtain only a fraction of these, say 210, for its use from your windows manager because other applications (Firefox, Acrobat, etc) will have already reserved some colors. The number will vary depending on the number of competing applications. The standard image transfer and display routines like tvscl and tvlct take this into account by using the smaller number of color levels but adjusting them to the full dynamic range possible on your terminal display. For instance, the color vectors may be only 210 elements long, but these will range in index value from 0 to 255. The number of colors available is contained in the system variable !d.n_colors or can be displayed with help,/dev.

      "Color Flashing": If the rest of your 8-bit terminal display blinks out or changes color when you move the cursor into an IDL window, then IDL is using a "private color table" which gets applied to your whole screen when the cursor activates it. This probably means IDL is attempting to use more colors than were free in the shared color table. To ameliorate the problem, try this:
                     Exit IDL
                     Restart IDL
                     As the very first two commands, type:
      	           device,retain=2,pseudo_color=8
                         window,0,col=k
      
      ...where k is the minimum number of color levels you think will be acceptable in your displays.

      Color flashing will not occur on a 24-bit monitor even in emulated pseudo-color mode.

    Transforming Images for Display

      The hardest part of displaying images is selecting the range of image values you want to display and then making them distinct on the screen.

      The array you send to your display must be a 2-D byte array (pseudo-color) or a 3-D byte array (true-color). The descriptions below are for intrinsic or emulated pseudo-color based on 2-D images and involve the use of color tables. There are various utilities for converting your original array (often floating-point) to the proper format and automatically displaying it with pre-defined color tables.

    1. First, you must decide on the size and shape of the display. Depending on the application and the size (in pixels) of your monitor, you may want to compress the original image or extract subarrays from it. In order to inspect the pixel structure of the image, you can expand images so that one original pixel occupies, say, a 10x10 pixel area on your screen. Basic utilities for extraction or enlargement of images are discussed under "Data Inspection & Manipulation" below.

    2. Next, you must determine the range of original image values, imin to imax, you wish to display. The human eye cannot normally distinguish 256 levels of either gray scale or color, and astronomical images often contain much more than a 256:1 intensity range. Only rarely would you want to display the whole range of values present in an image. Normally there is a significant sky background or bias/dark current pedestal which needs to be clipped out. Often, the highest values in images are caused by cosmic rays or other artifacts and lie far above scientifically interesting values.

        To determine the appropriate min/max values you could proceed by trial and error, iterating the image display. More objective methods include using the image value histogram; determining the mode of the sky background and its standard deviation; and making plots of sample slices across the image. Or, you could rely on any of several programs in the user libraries.

    3. Next, you must decide on the transformation, or rescaling, between image values and display values. Most common is a linear transform function in which the zero point is determined by imin and the slope is determined by imax-imin. The original image value range is divided into 256 equal bins for display. Contrast in the display increases with the slope, which is inversely proportional to imax-imin. Most image display routines (such as tvscl) employ a linear transform.

        Depending on the distribution of interesting image values within the chosen range, a non-linear transform may be more useful. For instance, to increase contrast at lower image values and decrease it at higher values, you might use transforms such as alog10(f), f^0.3, or asinh(f). Segmented linear or step-transforms can also be useful. Whatever special transform you apply to the image, you can then use the standard (linear) routines to display the transformed values.

        Alternatively, in pseudo-color you can use a linear transform of image values but manipulate the color table to achieve non-linear discrimination in the displayed image. In some situations, this may be preferable, but it is usually more complicated than simply changing the mathematical transform.

    4. Finally, you must select a color table for the display and apply it. The appearance of images can change dramatically depending on the color table settings. In pseudo-color mode, the color table is stored separately by the hardware and is always ready to be applied to a displayed image. The default color table is a linear gray-scale running from 0 to 255. IDL supplies a total of 41 pre-defined color tables, and there are various tools for modifying these or creating new ones.


    Image Display Commands:

    The basic IDL direct graphics routines for opening, displaying, and adjusting image display windows are window, which creates a new window and sizes it; wset, which defines the "current" window; and wshow, which reveals (or hides) the current window. wdelete deletes a window. These routines are used in conjunction with the two basic image display routines tv and tvscl.

    The routines chan,cdel,ctv,ctvscl described below are MOUSSE versions of the intrinsic IDL windowing/display routines. They offer important convenience features that make work with images faster (e.g. rescaling of window sizes to the image size, combination of the window set and show functions, arbitrary min/max values for the display, etc.). These routines use special common blocks, and to initialize those you must run the Mousse Startup File (see Appendix C). The descriptions below are for this set of substitutes for the intrinsic routines.

    Color Table Commands

    Other Useful Display Commands

    Object Graphics Image Display Software

    References on Image Displays



    6. DATA INSPECTION & MANIPULATION

    [Up to Contents]

    The tools described here are oriented toward analysis of 2-D images, but most are useful for any type of data array.

    Array Manipulation:

    Data Inspection:

    Special Displays and Plots

    The ATV Image Display & Inspection Tool

    Making synthetic images to test software:



    7. DATA AND IMAGE STORAGE

    [Up to Contents]

    [Up to Data and Image Retrieval]

    ASCII Files

    Binary Files: See description of binary files above. The basic IDL routine for writing unformatted binary files is writeu. To create files that are portable between computer operating systems, use the optional xdr keyword with the open procedure.

    FITS files: To write a FITS file from IDL variables, use fits_write.

    SDAS or IRAF files: Use stwrt and irafwrt, respectively. These (deprecated) routines each produce two output files (*.hhh and *.hhd for SDAS; *.imh and *.pix for IRAF).

    Other image formats

    Instant Storage of Session Data



    8. IMAGE PHOTOMETRY

    [Up to Contents]



    9. DATABASE ACCESS

    [Up to Contents]

    Thanks to the efforts of Don Lindler, Wayne Landsman and others at GSFC, the IDL Astronomy User's Library offers convenient and powerful access to on-line databases. Information from these can be directly incorporated into your IDL sessions. For instance, the routine imdbase marks the locations of sources in selected catalogues on your current image display (assuming you have accurate astrometry for your image.)

    Individual commands in the database package are described on the IDL Astronomy User's Library home page. More details are given in the 2001 description AUL-database-info-2001.pdf.

    The databases must have been put in a special IDL-readable format before you can access them (commands to do this are part of the package). A selection of IDL databases of general interest is publicly available from the IDL Astronomy User's Library. At UVa, approximately 95 such databases, weighted toward UV science, are available. They are presently linked to: /astro/idl/zdbase.

    In order to use the databases, you must have defined the environment variable $ZDBASE to point to the directory containing them.

    To see what databases are available, use the command dbhelp,1. To see what information is included in a given database and to display information for selected entries (here numbers 10,100,1000), type

       dbopen,'[data base name]
                 dbhelp,1
                 dbprint,[10,100,1000],'*
    
    To retrieve and use the data entries, you will need to use the more sophisticated commands described in the documentation cited above.

    The creation of new databases using the provided tools is relatively straightforward, so you can convert other datasets to online IDL databases as you wish.



    10. PLOTS

    [Up to Contents]

    IDL makes plots by default in the "current" terminal graphics window, which can be adjusted in size, location, etc, by using the same basic commands as for image displays (see above). Plots are scaled to the dimensions of the current window.

    The basic IDL commands for making plots are plot, for creating a new plot, and oplot, for overplotting on an existing plot. The destination for the plot (terminal, PostScript file, etc.) is determined by the set_plot command (see next section).

    This, however, is only the tip of an immense iceberg. IDL contains many options for making plots---so many, in fact, that the hardest part of the job can be keeping track of the multiplicity of optional parameters. Options in the form of keywords can be specified in the calls to the plotting functions. Alternatively, they can be invoked in the form of system variables, such as !p.title, which will apply to all later plot calls until changed.

    The IDL defaults are not as "nice" as those in SUPERMONGO, for example. However, you can quickly customize to obtain as sophisticated a plotting style as you like. All the functionality of SUPERMONGO and other scientific graphics packages is inherent in IDL. Many of the 2-D and 3-D graphics routines are illustrated in the IDL Demos that come with the system.

    Color tables for plots:

    Sample plotting scripts for displaying plots on your terminal follow. These involve a mixture of intrinsic IDL and AstUseLib routines:

    1. Plot an analytic function, using default axis labeling.

      
      x=findgen(10000)/10.           ; Create the independent variable,
                                     ;  here running between 0 and 1000
                                     ;  in intervals of 0.1
      
      y=x*sin(x/15.)                 ; Create the function of interest
      
      plot,x,y                       ; Plot the function against the
                                     ;  independent variable with a solid line. 
                                     ;  Plot will appear in the current window
                                     ;  (or Window 0 if none are open) with axes
                                     ;  scaled to match the maximum x,y ranges.
                               
      plot,x,y,xrange=[0,50]         ; Restrict plot to x values in range
                                     ;   [0,50].  Y range automatically scaled.
      
      plot,x[99:199],y[99:199]       ; Restrict plot to the 100th through
                                     ;   200th elements of the arrays.
      
      plot,x,y,xrange=[0,5],psym=1   ; Restrict plot to x values between
                                     ;   0 and 5, and plot points separately
                                     ;   as plus signs.
      
      oplot,x,y                      ; Add continuous line plot to previous display,
                                     ;   using existing axis scales
      
      plot,x,1+y^2,/ylog             ; Plot a new function (defined to be positive
                                     ;  definite), using a base-10 log scale on the
                                     ;  y axis
      
      
    2. Plot galaxy surface brightness data (in magnitudes) with error bars versus radius to the one-quarter power. Use discrete symbols (not a connected line). On this plot, a galaxy with a standard "de Vaucouleurs" brightness profile will produce a straight line. This example also shows how one can quickly edit bad data points out of plots.

      Assume that the (ordered) vectors sb, rads, and sberr already exist, with sb and sberr in units of ergs/s/cm^2/Angstrom/arcsec^2 and rads in units of arcsec. Assume that because of bad data points you must truncate the plot to eliminate the first 3 entries and those after the 20th.

       
      
      mags=-2.5*alog10(sb(3:19)) -21.1   ; Convert SB to magnitudes per arcsec^2
                                         ; in the STMAG system, ignoring bad data.
                                         ; Assumes all SB entries are > 0. 
                                       
      
      r25=rads(3:19)^0.25                ; Compute fourth root of radius vector,
                                         ; ignoring bad data entries
      
      magerr=1.086*sberr(3:19)/sb(3:19)   ; Convert uncertainties 
                                          ;   in SB to magnitudes
      
      !y.range=[25,18]               ; Set a non-default y-axis range; in
                                     ; this case the magnitude scale has 
                                     ; smaller (brighter) values higher
                                     ; on the y-axis
      
                                     ; Make the titles
      
      !p.title='Sample Surface Brightness Profile"
      !x.title='Radius(arcsec)^0.25'
      !y.title='Surface Bright (mags/arcsec^2)'
      
      plotsym,4,1.5,/fill            ; Choose filled triangles for plotting symbol,
                                     ;   50% larger than the default
      
      ploterror,r25,mags,magerr,psym=8
                                    ; Make plot on terminal screen with error bars; 
                                    ; Psym=8 specifies a user-created symbol, which
                                    ;    in this case was defined by plotsym.
                                    ;    No connecting lines between points.
                                    ; If psym were omitted or set to 0, no
                                    ;    symbols would be plotted and the points
                                    ;    would be connected by straight lines.
      
      xyouts,1.5,20,'NGC 4151'      ; Add a label within the plot area.  The
                                    ;    positional parameters (data units) set the 
                                    ;    leftmost position of the displayed string.  
                                    ;    Choose these so the string is clear of 
                                    ;    the data and axes. 
      
       
    3. Make a contour plot of a smoothed image.
       
      chan,3                              ; Open plotting Window 3
      
      !x.title='X'                        ; Make titles
      !y.title='Y'
      !p.title=' Contours for Image'
      
      square                              ; Set aspect ratio to make a square plot
                                          ;   (Note that you must also give this command
                                          ;   after the PostScript device is called
                                          ;   when making hardcopies.)
      
      smooth_one=smooth(image,5)          ; Smooth the image by a 5 pixel boxcar
      
      clev=[10,20,40,80,160]              ; Define trial contour levels--assume
                                          ;     these values span the range of interest
      
      contour,smooth_one,levels=clev       ; Do fast test of contour plot.  Check &
                                           ;   iterate clev for best appearance
       
      contour,smooth_one,levels=clev,/follow  ; Do more accurate (slow) plot,
                                              ;  with labels
       
    4. Read, sort, and plot data from an ASCII file; make & display trial polynomial fits

      Assume x is a vector containing values in the range 0 to 5 and that y is the corresponding dependent variable. Assume that these are to be read from an ASCII file named xy.dat, which contains x and y in separate columns. xy.dat can contain an initial explanatory section and other separator headers, as long as none of these contain only one or two floating point numbers (since readcol will mistake those for data lines). The data columns need not be aligned. By default, the readcol routine will read in the numerical x,y data ignoring any line containing alphabetic characters. No labels are put on the plot in this example.

       
      readcol,'xy.dat',xx,yy         ; Read data from the file.  Note that a format 
                                     ;    statement and 'open' command are not required,
                                     ;    nor do xx and yy have to be pre-defined
      
      index=sort(xx)                 ; Sort the arrays in order of increasing x 
      x=xx(index)
      y=yy(index)
      
      chan,1                         ; Open Window 1 for plotting
      
      plot,x,y,psym=5                ; Plot the data with open triangles
      
      quadcoeff=poly_fit(x,y,2)      ; Derive coefficients for best quadratic
                                     ;    polynomial fit
      print,quadcoeff                ; Print these out (optional) [Note: 
                                     ;    quadcoeff is a vector]
      
      tx=findgen(101)/20.            ; Create independent variable vector for fitted
                                     ;    values (uniform x interval of 0.05 units)
      
      quadtesty=poly(tx,quadcoeff)   ; Create the fitted quadratic values
      
      oplot,tx,quadtesty,psym=1      ; Overplot the quadratic fit with plus signs
      
      cubcoeff=poly_fit(x,y,3)       ; Derive coefficients for cubic fit
      cubtesty=poly(tx,cubcoeff)     ; Create the fitted cubic values
      
      oplot,tx,cubtesty,psym=3       ; Overplot the cubic values with small dots
      
                                     ; [Assume the quadratic fit was adequate]
      
      delta=y-poly(x,quadcoeff)      ; Compute the difference between y data and
                                     ;    the best quadratic fit 
      
      nix=where(abs(delta) gt 2.)     ; Locate those y values which are more
                                      ;    than 2 units from the best fit
      
      weight=fltarr(n_elements(x))+1. ; Create a weight vector corresponding to
                                      ;     x with unit entries
      
      weight(nix)=0.0                 ; Give deviant points zero weight
      
      newquadcoeff=polyfitw(x,y,weight,2) ; Derive improved quadratic coeffs. 
                                          ;   employing weights just assigned
      
      final=poly(tx,newquadcoeff)         ; Create improved fit values
      
      chan,2                         ; Open Window 2 for final, clean plot.
                                     ; (Window 1 is retained for comparison.)
      
      plot,x,y,psym=5,xrange=[1,3.5] ; Plot the data with open triangles in
                                     ;    Window 2.  
                                     ; Assume interest is limited to only 
      		 	       ;    part of data x-range.
      
      oplot,tx,final,psym=0          ; Overplot final fit with solid line
       
    5. Distinguish two samples in a histogram plot

      Suppose you have measurements of a quantity val for two samples of objects with a similar range of values and you want to compare the histograms of the two in a single plot. A simple way to do this in IDL is to create the union of the two datasets for a first plot and then overplot a shaded histogram of the second set. If the data consist of two vectors, val1 and val2,

      
           allval=[val1,val2]
           plothist,allval,bin=0.2,xrange=[0,8]
           plothist,val2,bin=0.2,/over,/fill,fcolor=200
      
      In the resulting plot, the val1 entries will be unshaded, while the val2 entries will be shaded with the fill color value 200 (the on-screen appearance will be determined by the pre-loaded color table).

        By default, plothist will plot the full range of values in the data. In histograms, this is often undesirable because extreme values usually have small populations. It is helpful to restrict the range for better display of the important values; here we used xrange to do that.

      Similarly, in the case of a subset of data values that are to be distinguished by some second parameter:

      
           plothist,val1,bin=0.2,xrange=[0,8]
           good=where(param ge 0) 
           plothist,val1(good),bin=0.2,/over,/fill,fcolor=200
      
        Here, the vector param must have the same length as val1, and we assume the subset of interest is defined by having a non-negative value of param.



    11. GRAPHICS HARDCOPIES

    [Up to Contents]

    The most common method of obtaining hardcopies or permanent storage of graphics output (plots or images) is to use PostScript files, since these can be printed on most laser printers. PostScript files can be later edited and reformatted, though special (non-IDL) programs are needed.

    IDL also supports output of GIF, JPEG, TIFF, PNG, and other file formats. GIF and JPEG are standard for Internet Web browsers. GIF is recommended for line-drawing and plot output storage. TIFF is recommended for high quality, full color reproductions (e.g. transporting image files to vendors for hardcopies or publication).

    You should always experiment on the terminal screen with your plot format before dumping to an output file. It is easy to do this by working out the set of commands you want while plotting to the screen, then typing set_plot,'ps (in the case of PostScript output) and repeating the commands using the command recall buffer.

    For more complex plots, use the journal utility, then edit and re-execute the resulting file (or cut and paste across windows).

    The set_plot command determines which output graphics device you are using. The most common versions of this command are:

     
            set_plot,'x  :  Send output to X Windows (default) 
            set_plot,'ps :  Send output to the PostScript file "idl.ps"
    

    The subsequent commands for sending data to the PS file are (mostly) the same as for putting data on your monitor screen, since monitors and PS files are interchangeable output devices for IDL.

    You can always check the properties of the current graphics output device by typing help,/dev. You can change these defaults by using the device command. The hardware/software interfaces are sometimes non-trivial, and you will want to plan for a significant learning curve in doing things which are not "vanilla." Before sending large jobs to printers, vendors, etc., be sure to check the files using LINUX ghostview, xv, or other screen display programs.

    Here are some graphics output methods for common situations:

    Technical Issues:



    12. IDL HINTS AND ANNOYANCES

    [Up to Contents]

    Not many people have experienced fully interactive computing before they start to use IDL. There are tremendous advantages but also many pitfalls for the unwary. The pitfalls will, of course, mostly seem obvious and trivial in retrospect---i.e. after you have learned to avoid them. A number of tips & warnings for IDL beginners are discussed in this section.

    Paths, Procedures, Directories

    Special IDL Interactive Aids & Accelerators

    Definition & Stability of Variables

    Differences from FORTRAN & C

    Punctuation, Syntax, etc

    Other Issues

    Programming Philosophy



    III. APPENDICES


    APPENDIX A: IDL SETUP

    [Up to Contents]

    Before executing IDL, you must define a special set of IDL environment variables, so that LINUX and IDL know where to look for the various IDL executable and program packages. Normally, you do this by source-ing a special idl_env file. Default environment files are contained in each IDL distribution from Exelis.

    On the other hand, if you want to customize your IDL configuration, you need to know the details of the various environment variables.

    The first item that must be defined is $IDL_DIR, the location of the main IDL directory. Intrinsic IDL executables are kept there. Which $IDL_DIR you define determines which version of IDL you will execute.

    You must also specify the location of the IDL license file that authorizes your host to execute IDL. Licenses are issued for single or multiple hosts. The UVA license supports 50 hosts simultaneously. The environment variable pointing to the license file is named $LM_LICENSE_FILE.

    You must also specify the search path for all *.pro routines you wish to use (other than the intrinsic Exelis routines).

    Separately, if you wish to use the databasing software, you must specify the location of the IDL databases (the zdbase directory)..

    In order to customize your IDL session further, you can define the $IDL_STARTUP environment variable. The file to which this variable points will be executed at the beginning of your IDL session. This startup file can execute an arbitrary number of other IDL programs and scripts. The default at UVa is to execute the startup file mousse_startup.pro. Among other items, this executes the procedure cinit.pro; many MOUSSE routines will fail if cinit is not executed.

    Finally, the idl_env file also establishes aliases for the various IDL executable files.

    When you give the command idl under the following setup, this is what happens:

    1. The executable idl located in $IDL_DIR is retrieved

    2. The Exelis/IDL license is checked

    3. If your host computer is authorized to use IDL, the normal program is executed

    4. If you are not licensed for IDL, the program runs in a 7-minute "Demo" mode

    5. The shell variables starting with IDL are converted to corresponding internal IDL system variables

    6. The file $IDL_STARTUP is executed, if defined.



    APPENDIX B: ENVIRONMENT SETUP EXAMPLE

    [Up to Contents]

    
    
    ### This is the UVa Astronomy LINUX system default env file 
    ### as installed on the department server, March 2009.
    ### Location is:  /astro/idl/idl_env.csh
    
    
    # C shell commands to define IDL environment variables and aliases.
    #
    # Script name: idl_env.csh
    #
    # This script is used by C shell users to define 
    # the environment variables and aliases required by IDL
    # related commands.
    #
    # To execute this script, give the following command; or,
    # for automatic execution, place it in your .cshrc file.
    #
    #    source /astro/idl/idl_env.csh
    #
    # To overwrite the defaults set in this file, you can source other
    # setup files from your individual .cshrc file.
    #
    
    ##### 
    # System choice - deprecated 03-07-11
    #####
    # set sys=`uname -a | awk '{print $3}'`
    # if ( $sys == 5.6 ) then
    #          setenv IDL_DIR          /net/astsun.astrosw/idl/
    #       else
    #          setenv IDL_DIR          /net/jeeves.common/rsi/idl/
    # endif
    
    # Note: the March 2009 configuration substitutes a departmental version
    #       of IDL v7.0 (/astro/itt/idl) for the UVa v6.4 version
    #       (/common/rsi/idl). 
    
    #setenv IDL_DIR          /common/rsi/idl/
    setenv IDL_DIR          /astro/itt/idl/
    setenv IDL_HOME         ${IDL_DIR}
    #setenv IDL_HELP                /common/rsi/idl/help
    setenv IDL_HELP         /astro/itt/idl/help
    
    # License file
    
    #setenv LM_LICENSE_FILE /common/rsi/license/license.dat
    setenv LM_LICENSE_FILE /astro/itt/license/license.dat
    
    # Local packages
    
    setenv ASTROLIB_DIR     /astro/idl/Astrolib
    setenv MOUSSE_DIR       /astro/idl/Mousse
    setenv MOUSSE98_DIR     /astro/idl/Mousse.98
    setenv PIA_LOC_GEN      /astro/idl/PIA/
    setenv ATV_DIR          /astro/idl/atv
    setenv FUSE_DIR         /astro/idl/FUSE
    
    setenv IDL_HELP_PATH    "+${MOUSSE_DIR}:+${ASTROLIB_DIR}:+${IDL_HELP}"
    
    # ZDBASE is the location of the IDL-formatted databases 
    
    setenv ZDBASE           /astro/idl/zdbase
    
    # Notes on IDL_PATH
    # 
    # Symbol "+" means that the path will be expanded to include all
    #   subdirectories within the given directory.  
    # The user's own IDL programs are assumed to be in a directory named,
    #   or linked to, "idl" under the user's home directory
    # Directory $IDL_HOME/lib contains the IDL User's Library of
    #   standard user-written utility routines.  
    
    setenv IDL_PATH         ".:+~/idl:/astro/idl/UVAlocal:+${ASTROLIB_DIR}:${ATV_DIR}:+${MO
    USSE_DIR}:+${MOUSSE98_DIR}:+${IDL_HOME}/lib:+${IDL_HOME}/examples:+${PIA_LOC_GEN}:+${FU
    SE_DIR}"
    
    setenv UIT_DATA         ${IDL_HOME}/data
    
    setenv IDLUSR           ${HOME}
    setenv IDLUSER          ${HOME}
    
    alias idl $IDL_DIR/bin/idl
    alias idlde $IDL_DIR/bin/idlde
    alias idldeclient $IDL_DIR/bin/idldeclient
    alias idlhelp $IDL_DIR/bin/idlhelp
    alias idlrpc $IDL_DIR/bin/idlrpc
    alias insight $IDL_DIR/bin/insight
    alias idldemo $IDL_DIR/bin/idldemo
    
    alias pia $PIA_LOC_GEN/PIA
    
    # The following IDL startup file will be executed before
    #    each IDL session.  User can specify a different
    #    startup in his/her .cshrc file.
    
    setenv IDL_STARTUP /astro/idl/startup/mousse_startup.pro
    
    

    APPENDIX C: MOUSSE STARTUP FILE EXAMPLE

    [Up to Contents]

    "Startup" files are batch files which are automatically executed when IDL begins running. They enable to user to customize his/her IDL session.

    The startup file is often placed in $IDL_DIR. However, it can be anywhere. In order for IDL to locate it, you must point the environment variable $IDL_STARTUP to it. If this variable is not defined, no startup file will be executed.

    This particular example defines special "systems variables" and common blocks used by the MOUSSE image processing package. These improve the efficiency of the package and reduce the number of typed entries the user must make to use it.

    Contents of mousse_startup.pro:

    
      ; STARTUP FILE mousse_startup.pro
      ; Last Modified: 15 May 1992, 30 Sep 93, 27 Dec 93, 22 Aug 97
    
    print , "Running mousse_startup.pro..."
    print , "For help on AstUseLib and Mousse routines, use the Mousse MAN procedure."
    
    setplot, 'X'
    DEFSYSV,'!DEBUG',0
    DEFSYSV,'!TEXTUNIT',0
    DEFSYSV,'!TEXTOUT',1
    DEFSYSV,'!PRIV',0
    defsysv,'!psprinter','astro-hp'     ;  UVA default printer
    cinit              ; CINIT sets up MOUSSE common blocks 
    !PROMPT = 'IDL>'
    !EDIT_INPUT = 100
    on_error,1         ; Return to main program in case of error
    print,' *** on_error,1 is default ***'
    print,' *** If terminal is vt100/tek, type SETPLOT,0'
    
    

    APPENDIX D: X-WINDOWS NOTES

    [Up to Contents]

    This guide assumes you are executing IDL from within an X-WINDOWS environment. So: first, you must start your windows system and open an appropriate window to enter IDL commands. The location of display windows opened by various IDL applications can be controlled in many instances. Consult the description of xpos under the window command. It is useful to place display windows away from your command window, wherever possible. You can move existing windows using your mouse.

    On an Apple MAC, you must ordinarily manually start the X11 program. You can select a monitor display class from its Preferences menu; changes here will not take effect until after you exit and re-start X11. You must execute IDL from within an X-window in order to use graphics.

    Normally, you run IDL on your local machine with display on your local terminal. To run IDL on a remote host which supports X-windows with display on your local terminal, do the following:

    1. Edit the .xinitrc file on your local machine to include the statement xhost [remote host], where [remote host] = the fully qualified network name of the IDL host you wish to use. E.g.:

      xhost parfait.gsfc.nasa.gov

    2. Start X-windows on your local machine.

    3. ssh or telnet to the remote IDL host and login

    4. To obtain X-windows output from the IDL host on your local machine, type

      setenv DISPLAY [local]:0.0

      where [local] is the fully qualified network name of your local machine, e.g.

      setenv DISPLAY bonkers.astro.virginia.edu:0.0

    5. Start IDL on the remote machine

    6. New windows created by IDL should now appear on your local terminal. Of course, data display in this mode will be slower than if you were running on the local machine.


    [To top of IDL GUIDE ]


    This Guide is copyright © 1991-2012 by Robert W. O'Connell. All rights reserved.
    Reproduction or distribution without attribution and commercial uses are prohibited.


    Last modified June 2012 by RWO. Email: rwo [at] virginia [dot] edu