There are a finite number of routines that are the "legal," documented pathways to read and write the IDL database. Every program which needs to access the database should go through these programs. Do NOT use any other method (e.g. direct access of NetCDF files, other routines not documented here but "which work"). If you do, your code may well break the next time we have to upgrade the database. Worse, your code may still appear to work, but may trash the database in the process.
If you need to do something for which instructions do not appear here, either (a) tough, or (b) talk to Rob or whoever deals with the database about getting it done.
These routines cover the following database: images, reduceimages, transformations, candidates, sntrak. For the subtractions, lightcurves, and perhaps one or two other databases, you should still do it the "illegal" way.
When you run tracker with "sum=", "/yesc,csum=", or "/yesg,gsum=", you get back these summary structures that have information about the images, candidates, or grids that tracker found. This section documents the elements of those structures:
Many, but not all, of these structure elements may be found in the ims_struct structure, which is the type of the ims[] variables. Hopefully before long Rob will get his butt in enough gear to actually make ims_struct and imsum_struct the same.
The name of the image, ending in '.fts'.
The exposure time of the image, in seconds.
The time (UT or GMT) when the image was taken; hour, minute, second.
The RA of the center of the image (hours). This is generally what the telescope originally put in the raw image's header, not what was later determined from an APM solution. Hopefully, it has been adjusted to represent the proper RA for a given chip of a multiple chip camera, but it may not have been, so tread with caution.
The Dec of the center of the image (degrees). See ra.
The epoch of ra and dec.
? I think this is the Zenith Angle. (90-altitude).
The airmass through which the image was taken. 1 for straight vertical, 0 for HST. Needed for calibration purposes.
The name of the detector, usually out of the header from the raw image.
What was formerly "telescope". This is a tag such that for one night, all images with the same tag were from the same chip (or quadrant) on the same telescope, and all images from a given chip (or quadrant) and telescope have the same tag. This is usually a brief telescope specification, with additional alphabetic letters to indicate the chip. For the CFHT12K camera, the det_tag is cfht12ka, cfht12kb, ..., cfht12kl. Note that you aren't guaranteed that two images from different nights with the same det_tag will be from the same detector; usually this will be the case, but not always. See det_spec for something which is supposed to uniquely identify the detector. The det_tag is used in the filename of the image.
That string to which you add an alphabetic specification of the chip number in order to get det_tag. For the CFHT12K camera, det_tag_base is "cfht12k". You add a through l for chips 0 through 11 to get the det_tag.
The total number of chips which were in the camera that exposed simultaenously with the chip in this image.
A string which uniquely identifies the chip and detector. Might be CFHT12K0 for chip 0 (or chip a) on the CFHT12K camera. If the chip is swapped out, subsequent images should have the det_spec set to CRHT12K0_1.
Which chip, of a multi-chip camera. You ought to be able to generate det_tag from det_tag_base and this number. WARNING: This value may NOT match the "chip number" given by the observatory, and hence in the header. This id starts from 0 and counts up; many observatories will count their chips starting at 1.
The number the image should be multiplied by to get it to photoelectrons. Note that normally we multiply our images by the gain in the reduction process, and therefore this should be set to 1.0. It used to mean the original gain of the image, but this is no longer true.
A string representing the filter. Usually something like "R" or "Rctio", but it can also be "Harris_R" or "R_Harris" or "R " or a variety of other things. Some aren't even R.
A standard integer which represents what standard filter the filter for this image best approximates. 0:?, 1:C, 2:U, 3:B, 4:V, 5:R, 6:I, 7:Z, 8:W, 9: J,10: H, 11:K, 100: none.
A integer represented the type of the image. 5=cleaned, 6=compressed search image, 16=subtraction intermediate sum.
The average sky level of the image, which may have been subtracted (if "surfaced" is 1). Ideally, the noise on a blank area of the image should be close to sqrt(sky).
The "background" noise of the image, due to readout noise and sky noise. Add the shot noise sqrt(counts) properly in quadrature to an appropraite fraction of this value, and you have the uncertainty in the photometry counts of an object on the image.
? I mean, there's an obvious interpretation, but is it really that?
Readout noise.
The welldepth of the image. I am not sure if this is before or after gain division- that's a pretty important distinction! Pixels greater than this value (or this value divided by the gain, I'm not sure which) may be considered to be "saturated".
A standard string indicating the telescope that this image came from. Examples are "wiyn", "ctio4m", "btc", "hst", "keck2", etc. Hopefully all images from the same telescope will have this string the same. This will probably usually be grabbed from the header.
1? 0?
The number of seconds since midnight on Jaunary 1, 1970 when this database record was last modified. This field suffers from the year 2046 bug.
The "run number" or image number of the image. Most telescopes ascribe an increasing number to all images taken on a given night. Sometimes they put it in the header, sometimes they just put it in the filename. We put it here. (Sometimes, the telescope gives no run number, so the person who reduces the data assigns his own ordinal.)
The UT date (day, month, year) of the observation.
Width of the image in pixels
Height of the image in pixels
? There is an obvious interpretation again, but it's hard to know if this is an "at telescope" binfactor or a "since" binfactor... it's also hard to know how this interacts with sigma and the like.
The name of the object. Usually this is whatever title the observer at the telescope told the telescope to put in the image header. Sometimes it has no connection to reality.
X, Y FWHM of the seeing, in pixels. Calcuated by reduceimage, using a really cheasy algorithm on which you should not greatly depend.
sig[0] is the Gaussian sigma of the seeing of the image, in pixels. Calculated by reduceimage. I'm not sure why this is an array of three values, since usually sig[1] and sig[2] seem to be zero. I don't think it keeps separate X and Y sigs, even though they can easily be different.
A zeropoint for the image, so that a point source measured in 1 FWHM may have it's magnitude calculated by
mag=-2.5log(counts)+apmzeropoint(Check this.) Frequently these apmzeropoints are not great, and may be wrong by tenths of a magnitude for R-band images and more for I-band images. (And it's often utterly terrible for Keck I-band images.)
Internal coefficients used to convert pixel coordinates (x,y) to RA and Dec. Never use these coefficients directly! If you do, your code will break in the future. Rather, use the IDL routines "transpho" and "apmscale" to transform coordinates between coordinate systems.
Coefficients to go from RA and Dec in degrees to pixel (x,y) on the image. Never use these directly, use only transpho.
When the APM match for the image was performed, this is the number of APM stars that were matched to objects found on the image.
The median residual in... pixels? arcseconds?... (pixels, I think)... between the matched objects and APM stars.
1950. All APM coordinates are 1950. In fact, I think some software just assumes this without ever checking apmepoch. Caveat Programmor.
Scary stuff hardcoded in for KPNO 4m, that produced lots and lots of big ugly code to deal with, but is ignored by most things now. We need to come up with a more general way of storing variable seeing.
The range on the sky that the image spans. I _think_ RA is in hours, Dec is in degrees. This is epoch 1950. If reduceimage has been run on the image, this will be from the APM solution. Otherwise, it will be identical to basicminra et al. below. WARNING: Sometimes there is a bad APM match which the software saves anyway. This means that these variables could have wrong values in them. If you're worried about that, use basicminra et al. Of course, basicminra et al. assumes that the RA and Dec in the header are correct. You just can't win.
1 if minra et al. is from the APM solution, 0 otherwise.
1950 coordinates that represent the patch on the sky that the image subtends. This is based on the RA and Dec from the header and the software's built-in "knowledge" of the telescope's scale. Unless the software was confused about the telescope's scale, these values should be reasonable. However, minra et al. from a good APM match will be far more precise and accurate.
The Julian date of the observation. Should correspond to dateobs and ut above, but it's a single, easy-to-sort-on number.
(Documentation needed)
For many images (esp. the reference and search images from the March, 1999 nearby campaign), this gives the location on HPSS where the image may be found (since most of the images are in fact not online in the DEEPIMAGEPATH).
The location on the local system (where "local system" will depend on which database you're looking at; the SCP workstations for the deep database) where you ought to be able to find this image.
The location of the .im.nc NetCDF file for this image. Mostly, this file has the FITS imag eheader.
The location of the .pho.nc NetCDF file for this image. This file mostly has the catalog of objects found by Isofind.
The location of the Quimby Zeropoint .zp.nc NetCDF file for this image.
This is where we store all sorts of information about the things we have found in an image (stars,galaxies, and the like). Quite frankly, I don't know what most of these are, so if you see one in here you know, feel free to update the page.
All of this should be done with tracker. Tracker takes a number of keywords that lets you search by position, date, filter, and telescope of the image. If you specify the keyword "summ=imsum", you will get an array of imsum_struct structures back from tracker in the variable imsum. If you specify /silent, tracker won't output anything to the screen when it does it.
If all you are interested in is image filenames, you can use the keyword "images=images" to get the filenames back in the array images.
From IDL, do dlib,'tracker' for more information about tracker.
Two ways to do this. One, readimsum will read the db summary information for a single image. Two, use the routine readimage (or freadimage). Normally, this reads the header information into one of the Deepsearch common block "buffers" and also reads the image data into a passed variable. If all you are interested in is the database/header information, then you can set the flag /noimage to freadimage. To make the calling sequence work, you still have to put a variable for the image data, but no image data is actually loaded. Because several megabytes of FITS data don't have to be transferred over the network, using /noimage makes freadimage run significantly faster (and saves memory too).
You read a photometry list the same way you create it: with reduceimage (or freduceimage). That will read the photometry list if it exists, create and write it if it doesn't.
Some ways of customizing the behavior of freduceimage:
Use the routine updateimagedb. That's the only supported way. Do dlib,'updateimagedb' from within IDL for more information.
Most people will use ltelescope, which is the current replacement for "loadtelescope". The lower level routine is addimagetodb, documented below.
Use the routine addimagetodb. This routine will both add the image to the summary database and create the image NetCDF file. If the image already exists in either database, it will fail unless you specify /overwrite.
This routine only updates the database entries. It does not verify that actual image data exists or has the right filename. It is your responsibility, before or after calling this routine, to get the actual image file somewhere in the DEEPIMAGEPATH with the proper filename. The proper filename is ".fts" appended on to either what you specified with a FILENAME= keyword to addimagetodb, or what was returned from addimagetodb. Don't forget to append the .fts!
Don't.
The only way to write a photometry list is when you create it with reduceimage (or freduceimage). See the dlib help on that routine for more information.
The easy way: use transimages (or ftransimages). This will read the transformation into the trns common variable if it exists, and will calculate it and write it to the database if it doesn't. Use "/clobber" to force it to redo the transformation.
The hard way: use readtransimages to read the transformation, writetransimages to write the transformation. Both have 'dlib' documentation. You might want to do this if you are, say, running ftranscorrect (which doesn't access the database) to fix up a transformation. Normally, this is no longer necessary, as ftransimages has all that kind of thing built in.
Just like images: use tracker. Specifying the option "csum=csum" will return the candidate summary structures in the array csum. Specifying the option "nameid=cands" will return the list of candidate names (standard LBL names) in the array cands. Do dlib,'tracker' for more information. Note if you are interested in just the candidate summary structure for a single candidate, you have to clip down the return from tracker, beacuse it's possible you'll also get back summary structures for nearby candidates. Do this with:
IDL> tracker,auto=['n','9819'],/yesc,csum=csum
IDL> w=where(csum.officialname eq '9819',nw)
IDL> if (nw gt 0) csum=csum[w[0]] else // error //
Replace '9819' with the name of your candidate. If your candidate name is in an integer variable, you will probably need to convert it to a string using: strtrim(string(mycand),2). The strtrim takes off leading and trailing spaces, which can cause trouble for you.
A second routine, findcandbyncname tries to find a candidate of a given ncfilename... which is what a lot of the software seems to refer to as the name, although I prefer thinking of the name as the LBL name or the "officialname.
getcandinfo ought to do the trick. dlib it to get documentation.
Don't. I've got something like this built into the Next Generation searchscan software, but nobody else should need to do it.
Use readsndb. Do dlib,'readsndb' for more information.
It's entirely possible that this no longer works.
Of course, you can also use the tool "sntrak" from the Unix command line. On the Suns, just run "sntrak" (it's in $DEEPHOME/bin). On the PC's, run "perl $DEEPHOME/bin/sntrak".
You can't. Not yet.
Right now, the only way to edit it is using sntrak itself.