This is a note on how to set up the ctio data transfer program. 1) Discription of scripts 2) Setting up account at ctio. 2) Setting up scripts at lbl. 3) Running and monitoring scripts. 1) Descriptiong of scripts In order to facilitate the transfer and automonitoring of data from ctio, a series of scripts have been developed. They have gradually become more complex, alowing for more intricate operations. And there is certainly room for improvement. At this time there are four scripts running in parrallel. The first is actualy run at CTIO. This looks for cleaned data in a certain directory at CTIO. When an image has the proper label, ie the cleaning is done, this script picks it up and compresses it, using the hcompress routine developed by white. When the compression is complete the routine adds a new label to the image, this acts as a signal for the next script. The transfer routine, runs rsh (remote shell comands) to search the CTIO directory for compressed data. It then uses the rcp (remote copy) routine to pick up the image and copy it into a directory at lbl. When the transfer of the image is complete, the tranfer routine adds a new label to the image name, notifying the next script. The decompression/surfacing script picks up the image, and runs hdecomp to decompress the image. The image is then surfaced using the redgsurf idl routine writen by Ivan. After the image is surfaced it is loaded onto the data base and a blank file is created to let the monitoring routine know a new image has been loaded The monitoring routine looks for a dummy file signifying the loading of a new image. The monitor then runs several idl routines. The first calculates seeing and signal to noise of the image. The next one checks the focus of the image, producing a focus curve. Finally if there are subtractions to be done, you can turn on or off the transfer update option. This is a program written by Ivan to automate the creation of subtractions. After the idl routines are finished, the script then transfers the monitor data and the focus curve to CTIO using rcp. Finally it prints up the focus curves on the printer at CTIO using the rsh command. 2) SETTING UP THE CTIO ACCOUNT When you recieve a new account at CTIO, you need to set it up for data cleaning and transfer. There is something odd about the CTIO computer network. There are three separate machines. Each machine has a different login directory under your name. The computers you need to know about are ctio4m.ctio.noao.edu 139.229.1.1 ctio0.ctio.noao.edu 139.229.1.32 ctioa1.ctio.noao.edu 139.229.1.71 In each login directory, you want to set up the .rhosts file. The file will already contain quite a few CTIO machines. You will want to add LBL machines to the end of the list. For example I would add the following lines. barneys.lbl.gov jmel panisse.lbl.gov jmel rivoli.lbl.gov jmel This allows you to pop over to the CTIO account from lbl machines without entering a password. Now you can use the rsh (Remote SHell) and rcp (Remote CoPy) comands. The next thing you need to create is a directory to contain iraf cleaning scripts, and compression scripts. Copy the lblpost.cl and ctiocompress.csh to this directory. I usually put these in the ctio0 machine. The next thing to do is copy the hcompress source code over to ctio0. Compile the hcompess routine using the directions given with the routine. The reason for compiling over at CTIO is that different machines will compile different versions of hcompress. So you want the properly compiled version for the current CTIO hardware. Create a bin directory in the ctio0 computer and place the hcomress compiled routines in there. Now you want to create directories in which to put the cleaned images. The format is the following: mkdir /ua13/iraf/v?/clean_fits (where ? is your acount number) mkdir /ua13/iraf/v?/hotstuff The first directory is where the cleaning script (lblpost.cl) will send the clean images. The second directory allows one to change the priority of an image. If one image is deep in the que for transfer, and you want that image next, just move it to the hotstuff directory. The Compression routine will look in hotstuff for something to compress. If there is nothing there, it will then look in clean_fits. Similarly, the transfer routine will look in hotstuff first and then in clean_fits. The next thing to check at ctio, is that the paths are correct in the in the ctiocompress.csh. At the top of the routine will be several directory names. There will be the path to clean_fits,hotsuff and the bin directory containing the hcompress executables. Check to make sure these are correct. Specifically, that the v? is set right as our user name changes from run to run. If they are not correct, change them. Finally you will want to create a monitor directory in the login directory of one the machines a ctio (usually 139.229.1.1). I usually just call it monitor. This will be the location where our monitor script at lbl sends it's output when it is done. Once this is done, CTIO is ready for the data run. 3) SETTING UP SCRIPTS AT LBL Just as you did with CTIO. The first thing that needs to be done is to set up your .rhosts file. You want to include the CTIO computer names with your CTIO login name into your .rhosts. So for example: cat >> .rhosts 139.229.1.1 v19 139.229.1.32 v19 Make sure you include all the computers you will be accessing remotely. Now, you need to get the remaining scripts set up: hotrcp.csh (transfer), ctiosurfload.csh (decompression, surfacing, loading), monitor.csh (S/N, seeing, focus). The following items need to be initialized in the begining of the scripts hotrcp.csh 1) datadir - path to clean_fits at ctio, note that v19 will be different durring next run. 2) hotdir - path to hotstuff, note same as above 3) souraddr - name of computer you will talk to usually 139.229.1.32 4) sourid - login name at ctio 5) destaddr - name of machine running the script at lbl 6) destid - login name at lbl 7) destdir - directory in which you will put the compressed files when you transfer. This needs to be created before the run. 8) logdir - The script will keep a log of all that is done while running the script. Logdir is the place you keep the log. ctiosurfload.csh 1) DATADIR - location of the compressed files at lbl 2) FINALDIR - place to send the surfaced and loaded images. Note, give full path to a deepsearch directory ie. /home/astro28/deepsearch 3) LOGDIR - location of the logfile the script creates. monitor.csh 1) DATADIR - ctiosurfload.csh will create a file to signal monitor.csh that a new file hase been loaded onto the data base. This signal file is put in the same directory as the DATADIR of ctiosurfload.csh 2) LOGDIR - location of the logfile the script creates. 3) MONDIR - location of the monitor output, ie signal to noise and seeing of image. The routine creates a file that holds all this info. 4) RSHDIR - the directory at CTIO where you wish to copy the monitor data file. This directory needs to be created ahead of time. 5) CTIOADDR - the name of the machine you will tranfer to. I usually chose a different machine from the hotrcp machine, ie. 139.229.1.1 6) CTIOID - login name at CTIO. 7) Monitor.csh calls the idl routine, automon.pro. This routine calls transferupdate, which allows ivan to automatically create and run subtractions of our data. Howver, durring a reference run, you should turn off the update transfer. Just put a ';' in front of the line calling the routine. If you want transferupdate, check to make sure it is on. Once the scripts are set up, you need to make sure the DATADIR directories have all been created, with the same path you gave to the respective programs. If you have changed the title of one of the scripts, you will need to initialize it for execution. For instance if you make monitor2.csh type the following: chmod a+x monitor2.csh This allows you to execute the program as a shell script. 4) RUNNING THE SCRIPTS Each script should be run in its own window. If your path is set up correctly, you should be able to just type the name of the script in the appropriate window and it will begin to run. If it crashes right away, this just meens on of your initializtion directories is not correct. Just fix it. The programs should continue to run without problem. However, on ocation the transfer program hotrcp.csh crashes, because of internet problems. If this occurs, you should be able to just start it up again. Occationally there will be bigger problems. The ctiosfl.csh will start going wild printing out a message over and over. This means there is a problem with the loading of an image. Usualy it has to do with the fact that you are trying to load two images with the same name. Last run, I was sent a set of raw images and then moments later, I was sent the same set of clean iamges. This kicked off the ctiosfl.csh problem. In order to solve this problem, you will have to C the program and rm the old file and load the new ones by hand. Then if you want to monitor the new ones you need to create the correctly named monitor file. Once this is done, restart the ctiosfl.csh. I have not known the monitor.csh to crash but since it is a non crucial part of the system, you can just start it up again if it does. I hope this gives you a good idea of the things that need to be done for the transfer. It should all be relatively easy to learn, once you have given it a look over. Good luck in all your SNe searches! Yours, Jason Melbourne