From the point of view of the IDL programmer writing software to analyze images and data, hopefully a low-level understanding of the database is unnecessary. Rather, documentation of the relevant routines to read the database (such as TRACKER) and to write the database (such as UPDATEIMAGEDB) are all that is necessary. The actual details of how the database is stored may be insulated from the reader.
If you want or need to know more, read on.
The deepsearch database is actually a collection of different databases. At the time of writing, I could think of the following different database:
For the images database, each image "in our database" exists (in principle) as a FITS file somewhere in one of our image directories. Those directories are usually listed in your DEEPIMAGEPATH environment variable, and usually are of the form /home/astro*/deepsearch. In addition to the actual physical image, there is a NetCDF database file in $DEEPHOME/database/images. (NOTE! They may also be in $DEEPHOME/nearbydb/images, depending on whether your talking about the deep search or the nearby search!) This has a standardized set of header information, as well as an echo of (at least the first part of) the image FITS header. Finally, there is the "image summary" database, which is an PostgreSQL database stored on dara.lbl.gov. This has just the standardized header information from the NetCDF files, and is most useful for searching for sets of images.
The reducedimages database is supplemental to the images database. It is a set of NetCDF files, one for each image which has been "reduced". These files include some header information (APM RA/Dec solution, etc.) which is also in the SQL Image summary databse. These files also include {star_struct} entries for every object (star, galaxy, cosmic ray, etc.) found on the image by reduceimages.
The transformations database is also supplemental to the images database. It is a giant SQL table that stores the (up to) third order polynomial transformation between any two images on which ftransimages has been run. Normally, the user will only need to interact with this database by using the IDL routine transimages or ftransimages.
The imzeropoints database is yet another supplemental database to the images database. For every image for which Robert Quimby's zeropoint software has been used to find a zeropoint, there will be a NetCDF file. Te "quimbyzpstat" field of the image summary SQL database should indicate whether or not such a Quimby zeropoint exists.
The subng database is a specialized combination of NetCDF files and SQL records that the Next Generation subtraction and searchscan software uses to keep track of an ungodly number of things.
The lightcurve databases are NetCDF files that store specific information for those specific programs, and will not be discussed further here.
The candidates database is produced by the search software. It produces both NetCDF files and an SQL summary table. It is used by tracker to find where various candidates are in the sky. It is also used by sntrak to add candidates to it's own private database. As of October 1998, the NetCDF files are being phased out. They still exist for old candidates, but new candidates have their information stored only in (new) SQL tables.
The sntrak database is used by the program sntrak, and is documented elsewhere.
This section is not yet written.
The transformation database is a Postgres database ("transformations") which stores the transformation coefficients for every pair of images for which the transformation has been calculated. Because this represents a lot of entries, this database tends to be rather large. To improve the performance of looking up a transformation, an index "trans_file1" exists. This may (we shall find out) slow down the storage of new transformations, but presumably that is done less frequently.
This is the one which should be of most use to most people. It's the most high end, and the "safest" in that you are less likely to completely screw over the database.
This function is more flexible than tracker, but also lower level. This means that not only is it harder to use, but you could conceivably muck up the database if you using it wrong. It allows you to construct your SQL predicate (usually starting with "where" and having an "order by", but it can be whatever you want), and tacks it on to the end of "select * from images". You get back an array of IMSUM_STRUCT structures.
This reads the transformation between two specified images into the trns() common variable.
Returns a structure with the most basic information about any supernova candidate.
This function takes an IMSUM_STRUCT, and perhaps a couple of other things, and creates a new database entry. It creates both the NetCDF file and the SQL summary file entry. By default, it won't overwrite an exiting entry, but you can override that.
Modifies selected columns (parameters, attributes, whatever you want to call them) of an existing entry in the image database. Modifies both the SQL summary entry and (if it's one of the releveant parameters) the NetCDF .im.nc file. Before doing the modification, it makes sure that both exist. This routine will not create new entries.
Reads the transformation between two buffers from the trns() common block, and writes two corresponding entries to the transformation database. WARNING! Normally, you shouldn't have to use this! Rather, you should just use ftransimages().
This is the lowest level of all. It just takes the SQL command specified as the first command line arguement and sends it to the database. You get back a 1 or a 0 depending on whether the program thinks the command worked. Some of the above routines use this to modify the database. In general, you shouldn't need to use this one; anything you'd have to do can be done with one of the above commands. If you are going to use this, you should probably talk to Rob before you do it.
This is a Unix binary, written in C, and as such if you use it from IDL you have to use the SPAWN command (preferably with /NOSHELL).
When setting up a Deepsearch database, you need to create the following SQL tables:
Rob has written a Perl script that he keeps carefully hidden called "createdeepdatabases.perl", which calls several other individual perl scripts that create these tables.
In addition, several subdirectories need to be created in your "database" directory. By default, this is $DEEPHOME/database, although on our systems $DEEPHOME/nearby is another database used for the nearby search. These subdirectories are:
All of these subdirectories must exist in the "database" directory, although they may be symbolic links to elsewhere. The "reducedimages" subdirectory in particular can tend to get rather large, so you may need to locate it on a disk with scads of space.
This section is not yet written.
This section is not yet written.
This section is not yet written.