------------------------------------------------------------------------ 12/6/07 - NEG AUTOMATED Total Power Analysis ------------------------------ Work in progress... You need (check also http://www.astro.virginia.edu/~neg9j/codes/) - tp_doall.sh - Miriad (you will be using uvaver and uvlist) - standard IDL package - additional IDL scripts (put in your IDL_PATH) from idl_scripts.tar.bz - the .uvbrx data that you want (check paper@gasp0.gb.nrao.edu) - Weather fits files from GB (/home/gbtlogs/Weather-Weather1-weather1) *Note* You will want to change line 174 for your own startup file, or just set it to blank if you don't want one. Makes lots of lists, textfiles, and finally, plots. But you can get summary of results at tp_results.txt. --------------------------------------------------------------------------- --------------------------------------------------------------------------- --------------------------------------------------------------------------- All the Stuff below is OLD, but I'm keeping it around for my own reference. --------------------------------------------------------------------------- --------------------------------------------------------------------------- --------------------------------------------------------------------------- 8/7/07 - NEG tsys_t.pro no longer requires a weather file. Use plot = 5 if no ambient temperature data are given. ------------------------------------------------------------------------ 6/5/07 - NEG Update to 5/8 instructions: 1) Note that may07_eort.pro only works for the 4 antenna data so far. Needs to be edited to write out files for more or less than that (but formatting is same). Also, output file is lst_nnn_(d)d.(m)m.yyyy.txt, where nnn is antenna number and polarization (logical, not physical), and the date is self-explanatory. 2) tsys_t input is now tsys_t,datafile,lsti,lstf,pl,wfile,pol,lsti2,lstf2 pol can be 1 for NS and 2 for EW and chooses the correct polarization for the Haslam map (1 gives model_142.txt and 2 gives ew_model_150.txt). Default is 1. lsti2 and lstf2 gives lst fitting ranges for the temperature corrected fitting. Default is 0 to 24. This is good for avoiding weird temperature effects during the day. 3) Also, new file, power.pro. .compile this, and then run power_plot with the only input as the filename. Ex: IDL> power_pl,'lst.thisday.txt' and the output would be lst.thisday.txt_corr. This does the non-linear correction using Don's code, which is in this file labelled power_db. ------------------------------------------------------------------------ 5/8/07 - NEG New procedures for reading tsys files to eliminate any editing by hand (I finally learned how to format!). 1) Get weather files from /home/gbtlogs/Weather-Weather1-weather1 and make a list of the filenames in a text file. ex: ls weather/2007_05* > myweather.txt 2) Open IDL and do a .compile weather.pro. Run weathers.pro with the following inputs: weathers,filename,filenum,station,timeform,recnum IDL>weathers,'myweather.txt',24,1,0,30 Note that station=1 is default, timeform=0 means LST and is default, recnum=30 and it default. This creates forprint.prt which you should rename to something else, and it lists LST time and ambient temperature. 3) Bring over whatever .uv files you want to work with. Use uvaver and uvlist as described below. Ex: uvaver vis=07may/zen.2454226*.uv 'select=auto' line=channel,1,620,50,1 interval=2 options=relax out=zen.2454226.uv uvlist vis=zen.2454226.uv options=brief,data recnum=1e7 log=uvlist_xx_05may07.txt 4) To convert the uvlisting into lst, you will need the IDL program may07_eort.pro. .compile this, then run it as may07_eort,file,dayinit,month,year,longitude. Ex: IDL>may07_eort,'uvlist_xx_05may07.txt',5,5,2007 Longitude defaults to GBT. This will plot the power vs LST for you and create four files like lst_1xx_552007.txt, where the 1 is the antenna number, XX is still polarization, and the date is (d)d(m)myyy (needs work). 5) Now you are ready for tsys_t.pro. .compile this, and tsys_t,datafile,lst_i,lst_f,plot,weatherfile IDL>tsys_t,'lst_1xx_05may07.txt',2,22,6,'bigweatherfile.lst' where you can set the lst fitting range, and plot=6 gives the multi-plot. For all of these IDL routines, just typing in the name, once it has been compiled, returns a brief summary of inputs. ------------------------------------------------------------------------ 3/27/07 Update to weather.pro - NEG Weather.pro has added functionality for choosing a weather station and time format. Make sure you know what station you are using ahead of time and what time format you need. To get input help, type IDL> weather weather,filename,station,timeform,recnum filename = weather fits file station = weather station (1 or 2; default 1) timeform = output time format (0 -> LST, 1 -> UT; default 0) recnum = number of records per hour (default 30) ---------------------------------------------------------------------- 3/18/07 New procedure for system temperature fitting. - NEG Need only the uvlist file (modified to remove non-numerical characters, up to "Now IDL can deal with it." from 2/20) and the weather file (as described on 3/07). Get tsys_t.pro and mastert.pro. .compile both files once (maybe twice to get the functions right). You can just type IDL> mastert and it will return a brief explanation. Mastert takes 7 inputs mastert,filein,day,mon,yr,lsti,lstf,wfile filein = file with uvlist output day = first day of observation mon = first month of obs year = year of obs lsti = initial LST of first fit window lstf = end of window for LST fit wfile = text file with weather at 2 minute intervals Calls tsys_t.pro An example: IDL> mastert,'txt/uvlist_2xx_feb22.txt',22,2,2007,0,12,'txt/feb22_wea.txt' % TEXTOPEN: Output is being directed to a file txt/lst_2xx_feb22.txt % TEXTOPEN: Output is being directed to a file txt/lst_2xx_feb22.txt_mod Reading data and model... Fitting polynomials of degree 6 Using normalized time scale in LST Selecting LST range from 0 to 12 Fit for Trec T_rec in Kelvin: 226.79418 Gain*BW*k is: 12507.133 T(K)= -215.69626 + 7.8533628e-05 * Power G_0,G_1,Tsys= 18876.164 -360.14989 10.964355 This does the lst conversion for you, and applies power.pro for quantization. It will produce a plot like g1_fit.jpg, which is explained in captions.txt. You still need the procedues in useful.idl.tar.gz. ----------------------------------------------------------------- 3/07/07 Procedure for reading weather files from Green bank stations - NEG Weather files are kept on the Green bank system at /home/gbtlogs/Weather-Weather2-weather2 At this location are weather files in fits table format that have four temperature readings, and I suspect that this is one of the ground stations. Files are created every hour and files a few months old are gzipped for space. Each file consists of weather data, including four data points for temperature, taken every second. Use the IDL task weather.pro, provided at http://www.astro.virginia.edu/~neg9j/codes/ along with other IDL tasks and procedures necessary to run in useful.idl.tar.gz. Start IDL, and run the program with the filename of the weather file that you want to read and the number of records desired for the hour. IDL> weather,'weather/2007_02_25_19:24:57.fits',30 % TEXTOPEN: Output is being appended to file forprint.prt This will create and append to a file forprint.prt (so rename it once you are done creating it!) with the LST and median temperature (from the four sensors) for each record. There are 3600 records per hour (once per second) so to choose 30 records is to choose one record every two minutes. If no number of records is chosen, the default is 30 to match the 2 minute averaging used for determining the Tsys. ---------------------------------------------------------------------- 2/20/07 TSYS procedure with Nov06 data - NEG Be where zenith files are available and you have Miriad. uvaver vis=zen.2454053*.xrfi 'select=polarization(xx),antenna(2)(2)' line=channel,1,145,1,1 interval=2 options=scalar,relax out=test.uv.xrfi Continue with zen.2454054*.xrfi, etc. up to zen.2454059*.xrfi Will typically call outfiles something like zen.2454057.ant3.uv.xrfi Once time averaging is done on selected channel and antenna, create list uvlist vis=zen.2454057.ant3.uv.xrfi options=brief,data scale=1e-6 recnum=1e7 log=test.log Please use that scaling factor, since later scripts compensate for it! Compile all the results into one log file. Example, after each uvlist: more test.log >> uvlist_3xx_145.txt Replace all non-numerical characters (except .) in columns with a space (I prefer emacs, ESC % with ! to replace all) Now IDL can deal with it. Open IDL. You will need the file located at http://www.astro.virginia.edu/~neg9j/codes/useful.idl.tar.gz to get the scripts I wrote for this, plus a bunch of really, really handy scripts from NASA GSFC. These are all text files, just put them in your IDL_PATH in your .rc file. (Be careful not to overwrite existing paths! Trust me, I did that.) Let me know if any are missing (that is usually an error when you try to run the script saying that a procedure is missing.) So now, in IDL, compile and run the program nov_eor2.pro, give inputs IDL> .compile nov_eor2 % Compiled module: NOV_EOR2. IDL> nov_eor2 Enter input filename: uvlist_3xx_145.txt Enter output filename: lst_3xx_145.txt Enter number of first day: 13 Enter number for month (0 for Default 11): 0 Enter year (0 for Default 2006): 0 Enter longitude (0 for Default -79.8398): 0 and it gives you a pretty plot of LST vs. Power and outputs a file of these values. Type 0 (zero) if you want to keep the default. Now you have a data file. I will try and make a list of text files for the HASLAM model at different frequencies and keep it up to date with the latest beam models. For now, it's using model_142.txt, also in http://www.astro.virginia.edu/~neg9j/codes, along with the next script, tsys.pro. Compile and run tsys, give inputs: IDL> .compile tsys % Compiled module: TSYS. IDL> tsys Enter file with data: lst_3xx_145.txt Enter start LST for fit: 0 Enter end LST for fit: 10 Plot (1) raw data, (2) data and fit, (3) polynomials, (4) temp vs power, or (5) multi? 5 then the program gives lots of fun messages and outputs, plus the plot of your choice. Example outputs: Reading data and model... Fitting polynomials of degree 6 Using normalized time scale in LST Selecting LST range from 0.00000 to 10.0000 Fit for Trec T_rec in Kelvin: 164.18577 Gain*BW*k is: 91287.479 T(K)= -161.05949 + 1.0895023e-05 * Power That's it!