Loading Images to the Image Database
Introduction
The phrase "loaded into our database" actually means
"registered to our database" in this group. The main purpose of this step is
to standardize the usual header informations since other than the fits format,
there are no standards for the keywords in the header. Another mundane yet
important task done at the same time is to rename the image files according
to our standard. So this step of loading is usually required before any real
analysis on the images can be performed properly.
(The easiest way to see the entire header of a FITS file is to use "more"
in unix.But make sure that your unix terminal is 80 charater wide.)
For an example of header variations, let's consider the following :
DATE-OBS= 'jan/24/98'
UT = '13:12:13.3'
vs.
DATEOBS = '24/01/1998'
UTSTART = 13.203694
For any intelligent person, the two are giving the same information.
Unfortunately, computer programs usually do not recognize that fact.
Thus the job of "loading" boils down to matching which keyword for which
entry and what format used and saving the result to the database.
This step was first coded as loadtelescope.c
in C(C++) and executed on unix prompt. This was OK for a while since the
header usually stays consistent for a
particular set-up of telescope-CCD combination. But as the project goes on,
new telescopes and new cameras have been used, each with its slight variations
in the header. And more importantly our personnel changes.
To update the C code all the time is simply not practical at all. Thus at one
point, copycat of the routine is done in IDL, in which codes can be modified
very easily (too easily according to Reynold Pain and some others). Gradually
the C code became out of date making the IDL routine the only way to load.
Eventually the code became very messy, full of all kinds of special one-shot
keywords intended to circumvent problems for one particular telescope on
one particular run. An entirely seperate program called loadbtc was
required to load btc images. This was beginning to lead to difficulties
in maintaining the software, so when the new database system was put
in by Rob Knop, a new version of the loading software was written.
IDL routine : ltelescope.pro
The program consists of two parts
- Get the header format pattern : tformat.pro
- Use the header to determine keyword values: calc_keywords.pro
tformat.pro
This is a subroutine that holds the header pattern for each telescope so the
code is nothing more than a long list of CASE statements. In the new version
it is possible to specify more than one possibility for the keywords. The
possibilities are searched in order, and the first one that finds a match
in the header is used. A new telescope/CCD combination never before
used by the group requires a new case statement to be added to tformat.pro.
Once updated, the subsequent images from the same instruments
can be handled with little trouble.
calc_keywords.pro
This performs the meat of the work of ltelescope. It takes the
keywords given by tformat and hunts through the header to find
the values which must be entered into the database. Sadly, several
telescopes still require specific hacks to work. In particular,
as of this writing, HST, BTC, CHFT, and APT require some modifications
in order to load successfully. In the new version, these hacks
are localized together at the end of calc_keywords - hopefully
the user will not have to ever specifically deal with these,
but at least now it's easy to tell where they live.
EXAMPLES
This is what needs to be done for a typical incoming data (except for the
compressed-decompressed "com" files for the real time search run where
the images are loaded automatically).
Let's say there are images n2.0***.fits (like n1.0005.fits, n1.0127.fits)
in /home/astro20/deepsearch/ctio-night02/ where the images came from the
second night at the CTIO
% cd /home/astro20/deepsearch/ctio-night02/ # it's easier to work here
% ls -C1 n2.*.fits > imlist # creat a image name list file
% idl # start IDL session
IDL> ltelescope,file='imlist',order='MDY'
;The order keyword is necessary for BTC because it stores the
; date in a different order than most other telescopes.
;when prompted for telescope, enter btc
;for some documentation on the two IDL routines here, use the following :
IDL> dlib,'ltelescope'
more text on loadtelescope.pro