Loading Images to the Image Database

Introduction

The phrase "loaded into our database" actually means "registered to our database" in this group. The main purpose of this step is to standardize the usual header informations since other than the fits format, there are no standards for the keywords in the header. Another mundane yet important task done at the same time is to rename the image files according to our standard. So this step of loading is usually required before any real analysis on the images can be performed properly.

(The easiest way to see the entire header of a FITS file is to use "more" in unix.But make sure that your unix terminal is 80 charater wide.)

For an example of header variations, let's consider the following :

DATE-OBS= 'jan/24/98' UT = '13:12:13.3' vs. DATEOBS = '24/01/1998' UTSTART = 13.203694 For any intelligent person, the two are giving the same information. Unfortunately, computer programs usually do not recognize that fact. Thus the job of "loading" boils down to matching which keyword for which entry and what format used and saving the result to the database.

This step was first coded as loadtelescope.c in C(C++) and executed on unix prompt. This was OK for a while since the header usually stays consistent for a particular set-up of telescope-CCD combination. But as the project goes on, new telescopes and new cameras have been used, each with its slight variations in the header. And more importantly our personnel changes. To update the C code all the time is simply not practical at all. Thus at one point, copycat of the routine is done in IDL, in which codes can be modified very easily (too easily according to Reynold Pain and some others). Gradually the C code became out of date making the IDL routine the only way to load. Eventually the code became very messy, full of all kinds of special one-shot keywords intended to circumvent problems for one particular telescope on one particular run. An entirely seperate program called loadbtc was required to load btc images. This was beginning to lead to difficulties in maintaining the software, so when the new database system was put in by Rob Knop, a new version of the loading software was written.

IDL routine : ltelescope.pro

The program consists of two parts

tformat.pro

This is a subroutine that holds the header pattern for each telescope so the code is nothing more than a long list of CASE statements. In the new version it is possible to specify more than one possibility for the keywords. The possibilities are searched in order, and the first one that finds a match in the header is used. A new telescope/CCD combination never before used by the group requires a new case statement to be added to tformat.pro. Once updated, the subsequent images from the same instruments can be handled with little trouble.

calc_keywords.pro

This performs the meat of the work of ltelescope. It takes the keywords given by tformat and hunts through the header to find the values which must be entered into the database. Sadly, several telescopes still require specific hacks to work. In particular, as of this writing, HST, BTC, CHFT, and APT require some modifications in order to load successfully. In the new version, these hacks are localized together at the end of calc_keywords - hopefully the user will not have to ever specifically deal with these, but at least now it's easy to tell where they live.

EXAMPLES

This is what needs to be done for a typical incoming data (except for the compressed-decompressed "com" files for the real time search run where the images are loaded automatically).

Let's say there are images n2.0***.fits (like n1.0005.fits, n1.0127.fits) in /home/astro20/deepsearch/ctio-night02/ where the images came from the second night at the CTIO

% cd /home/astro20/deepsearch/ctio-night02/ # it's easier to work here % ls -C1 n2.*.fits > imlist # creat a image name list file % idl # start IDL session IDL> ltelescope,file='imlist',order='MDY' ;The order keyword is necessary for BTC because it stores the ; date in a different order than most other telescopes. ;when prompted for telescope, enter btc ;for some documentation on the two IDL routines here, use the following : IDL> dlib,'ltelescope' more text on loadtelescope.pro