Princeton/NYU SDSS Imaging Home Page

(Link to spectroscopy data.)

NEW! Second Orion Data Release: /
For the first Orion Data Release, see: /

In this document:

Tools for using the data

The imaging reductions are done using the "photo" product and its many dependencies, which is C and TCL code. Independent of that, we have a number of 6154444450 for using the data. The latter is described below.

IDL setup

If you have never used IDL before, you will need to set a few environment variables. This is usually done by placing the following lines in a file called .idlenv and sourcing it in your .cshrc file BEFORE any "setup" commands:
# set directory
setenv IDL_DIR          /usr/peyton/common/licensed/idl

# clear path
setenv IDL_PATH

# set generic path
setenv IDL_PATH $IDL_PATH{:}+$IDL_DIR/lib
If you are not at Princeton, you will need to type "which idl" and maybe follow a symlink to find out what IDL_DIR should be set to. Princeton

Several environment variables should be set to point to the raw and the reduced data. At Princeton, the following could be put in your .cshrc file:

  setenv SPECTRO_DATA /u/dss/spectro
  setenv PHOTO_DATA /u/dss/data
  setenv PHOTO_REDUX /u/dss/redux

At Princeton, the perl scripts and IDL tools can be put in your path with the following:

  setenv EUPS_FLAVOR `/bin/uname`
  source /u/dss/products/evilups/bin/setups.csh
  setup idlspec2d
  setup photoop

...not at Princeton

Install the "evilups" product, which is a set of scripts written by Nikhil Padmanabhan for managing different versions of software. The example below puts the code in the directory $HOME/products:
  setenv CVS_RSH ssh
  setenv CVSROOT
  cvs login
    (hit a carriage return when it asks for the "CVS password")
  cvs export -d evilups-temp -r v0_4 evilups
  setenv PROD_DIR_PREFIX $HOME/products
  setenv EUPS_DIR $PROD_DIR_PREFIX/evilups
  cd evilups-temp
  make clean
  make install
  setenv EUPS_FLAVOR `/bin/uname`
  source $EUPS_DIR/bin/setups.csh
Download, build, and install the products "idlutils", "idlspec2d", and "photoop" using the scripts available from evilups:
  evilinstall idlutils v4_10_4
  setup idlutils
  evilinstall idlspec2d v4_10_1
  evilinstall photoop v1_0
You'll probably want the following in your .cshrc file to set up your paths:
  setenv EUPS_FLAVOR `/bin/uname`
  source $HOME/products/evilups/bin/setups.csh
  setup idlspec2d
  setup photoop
If you will be accessing the reduced data remotely (as described below), then set
and copy the file runList.par into your home directory (or wherever $PHOTO_REDUX points).

Accessing the data (images and reduced)

All 8 Tb of raw data and 4 Tb of reduced data are spinning on disk at Princeton. The raw images can be found in the directories:

The reduced data can be found in the directories:
These data are fully mirrored at MIT, and partially mirrored at NYU.

In order to view the actual images (e.g., using ATVSDSS), you either need a copy of the raw images, or need to be logged into the Princeton machines. In order to access the reduced data, you either need to have a copy of the data (and use SDSS_REAODBJ) or have permission in the alfred account at Princeton (and use SDSS_READOBJ_REMOTE).

  1. Where have we observed?
    Once you have the above tools set up, you can determine where we have observed w/out downloading any data whatsoever. For example, to see which images correspond to RA=180,DEC=1:
      IDL> foo=sdss_findimage(180,1,/print)
    To plot the locations of all the imaging data using an equal-area projection:
      IDL> plotrunsky, /equalarea
  2. What has been reduced?
    Each set of reductions is given a "rerun name". Rerun 137 is complete for all the data taken through October 2003, including all of the images used for spectroscopic follow-up. It contains 14000 square degrees of imaging data, covering a footprint of 7000 square degrees (the difference is repeat imaging). There are 400,000,000 objects. You can plot the location of these runs (and overplot the spectroscopic plate locations) with:
      IDL> plotrunsky, rerun=137
      IDL> platelist, plist=plist
      IDL> soplot,plist.ra,plist.dec,psym=4,color='red'
    The list of these run numbers and other parameters can be printed with:
      IDL> struct_print, sdss_runlist(rerun=137)

    These reductions replace rerun 131, which contains 7500 square degrees of data.

  3. Display the sky at a particular location
    To plot the sky at RA=180,DEC=1 (if we have observed it, which we have):
      IDL> atvsdss,ra=180,dec=1
    Make the same plot using the Princeton reductions known as "rerun 137", and over-plotting the SDSS spectral classifictions, and objects from the 2MASS, FIRST and Tycho-2 catalogs:
      IDL> atvsdss,ra=180,dec=1,rerun=137,/catalog,/spec
  4. Generate a FITS table for a particular field
    If you have a particular RUN+RERUN+CAMCOL+FIELD (for example, found above with SDSS_FINDIMAGE), you can use Nikhil's perl script to generate a FITS binary table with all of the reduced objects. The example below creates the file "foo.fits", and includes matches to the 2MASS, USNO-B and FIRST catalogs:
      % sdss_readobj_remote --user alfred --run 752 --rerun 137 --col 6 \
        --filename foo.fits --fieldrange "243" --extra="/catalog"
    The above command can be run outside of Princeton, as long as you have access to the alfred account, and a copy of the following file on your local machine:
    The above file tells this script how to tunnel into the appropriate machine at Princeton that has the data locally.
  5. Load reduced data for a particular field into an IDL structure
    The IDL command SDSS_READOBJ can be used to load data into an IDL structure. This loads the same data as the example above:
      IDL> objs=sdss_readobj(752,6,243,rerun=137,/catalog)
    If you wish to load 50 fields for this run, but trim it to only a few columns of interest (but including all of the various flux measurements):
      IDL> objs=sdss_readobj(752,6,200+lindgen(50),rerun=137,/catalog, $
    The commands can be run on machines outside of Princeton by using "sdss_readobj_remote" rather than "sdss_readobj". This does the same tunnelling trick as the perl script above.
  6. Locate the nearest object (and load into memory)
    Example: Find the nearest object to the location of one of our spectra (plate 400 MJD 51820 fiber 1) from the rerun 137 reductions. Then load that object into memory:
      IDL> list = sdss_findobj(24.600318,-1.0067119,rerun=137)
      IDL> objs = sdss_readobjlist(inlist=list)
    The above can be done with lists of more than 1 object.
  7. Getting help with any of the IDL commands
    You can get help on any IDL commands by using "doc_library", for example:
      IDL> doc_library,'plotrunsky'
    This is equivalent to 970-666-6386.

Data Model

Data Model: calibObj data structure

Data are released in "calibObj" files. These are binary FITS tables covering one SDSS field per file. These files are "virtual" in the sense that they are generated on-the-fly by the IDL or perl commands described in this document. The files used are fpObjc, asTrans, calibPhotom, psField, reObj. Even if information is the same for every object in the field (e.g. run, camcol, field), it is repeated in the data structure so that data from many files can be concatenated with no loss of information. Description of the calibObj files (below) is broken into 6 sections:
  1. Book-keeping values -- Observation "run" and time information
  2. Classifications, flags, and dimensionless values
  3. Calibration parameters -- photometricity, plate scale, orientation
  4. Uncalibrated quantities -- raw counts, pixel coordinates
  5. Calibrated quantities -- calibrated fluxes, RA, DEC
  6. (385) 440-1063 -- 2MASS, FIRST, USNO-B
  1. calibObj book-keeping values:
       RUN             LONG      Run number
       RERUN           STRING    Rerun name
       CAMCOL          LONG      Camera column [1..6]
       FIELD           LONG      Field number
       ID              LONG      Object ID, starting at 1 and unique in each field
       PARENT          LONG      ID of parent, -1 if this is a parent object
       NCHILD          LONG      Number of children deblended from this object
       MJD             LONG      Modified Julian Day for date of observation
       TAI             DOUBLE[5] Mean time of observation for the object center
                                 in each filter, where TAI = 24 * 3600 * MJD
       AIRMASS         FLOAT[5]  Airmass approximated as csc(zenith-angle)
       PSP_STATUS      LONG[5]   "Status" field from the psField file
       CALIB_STATUS    INT[5]    Status of photometric calibration
       RESOLVE_STATUS  LONG      (see below)
  2. calibObjclassifications, flags, and dimensionless values:
       OBJC_TYPE       LONG      Classification:
                                   0=unknown (only used for some edge objects)
                                   2=defect (unused)
                                   4=ghost (unused)
                                   5=known object (unused)
                                   7=trail (unused)
       OBJC_PROB_PSF   FLOAT    Classification: -9999=unknown, 0=galaxy, 1=star
       CATID           LONG     (Unused)
       OBJC_FLAGS      LONG     See
       OBJC_FLAGS2     LONG
       FLAGS           LONG[5]
       FLAGS2          LONG[5]
       TYPE            LONG[5]  Same as OBJC_TYPE, but for each filter
       TEXTURE         FLOAT[5] (Unused.)
       STAR_L          FLOAT[5]
       STAR_LNL        FLOAT[5]
       EXP_L           FLOAT[5]
       EXP_LNL         FLOAT[5]
       DEV_L           FLOAT[5]
       DEV_LNL         FLOAT[5]
       FRACPSF         FLOAT[5]
       PROB_PSF        FLOAT[5]
  3. calibObj calibration parameters:
       NMGYPERCOUNT    FLOAT[5]    Calibration of nano-maggies per count
       NMGYPERCOUNT_IVAR  FLOAT[5] Formal error in the photometric calibration as
                                   an inverse variance; =0 if there wer no calibration
                                   stars and a default calibration was used instead
       CLOUDCAM        INT[5]      Photometricity in each filter:
                                    -1=unknown, 0=cloudy, 1=clear
       EXTINCTION      FLOAT[5]    Galactic extinction in magnitudes, defined as
                                   [5.155, 3.793, 2.751, 2.086, 1.479] * E(B-V)
                                   where E(B-V) is from the SFD98 dust maps.
                                   Fluxes (and their errors) can be corrected
                                   for Galactic extinction as follows:
                                     Corrected-Flux = Flux * 10^(Extinction/2.5)
       PIXSCALE        FLOAT[5]    Pixel scale [arcsec/pix]
       PSF_FWHM        FLOAT[5]    PSF FWHM [arcsec]
       PHI_OFFSET      FLOAT[5]    Calibration angle [degrees] to subtract from any
                                   uncalibrated position angles such that they mean
                                   degrees east of north.
  4. calibObj uncalibrated quantities: (straight from fpObjc files, but with position angles rotated to degrees E of N)
       OBJC_ROWC       FLOAT     Canonical row center (0.5-indexed) in r-band,
                                 even if not detected in that band
       OBJC_COLC       FLOAT     Canonical column center (0.5-indexed) in r-band,
                                 even if not detected in that band
       ROWV            FLOAT     -- velocities ???
       ROWVERR         FLOAT
       COLV            FLOAT     -- velocities ???
       COLVERR         FLOAT
       ROWC            FLOAT[5]  Row center (0.5-indexed) in each filter
       ROWCERR         FLOAT[5]
       COLC            FLOAT[5]  Column center (0.5-indexed) in each filter
       COLCERR         FLOAT[5]
       SKY             FLOAT[5]  Sky level [raw counts/pixel]
       SKYERR          FLOAT[5]
       PSFCOUNTS       FLOAT[5]
       PETRORAD        FLOAT[5]   Petrosian radius [pix]
       PETROR50        FLOAT[5]   Radius with 50 percent of Petrosian light [pix]
       PETROR50ERR     FLOAT[5]
       PETROR90        FLOAT[5]   Radius with 50 percent of Petrosian light [pix]
       PETROR90ERR     FLOAT[5]
       Q               FLOAT[5]
       QERR            FLOAT[5]
       U               FLOAT[5]
       UERR            FLOAT[5]
       M_E1            FLOAT[5]
       M_E2            FLOAT[5]
       M_E1E1ERR       FLOAT[5]
       M_E1E2ERR       FLOAT[5]
       M_E2E2ERR       FLOAT[5]
       M_RR_CC         FLOAT[5]
       M_RR_CCERR      FLOAT[5]
       M_CR4           FLOAT[5]
       M_E1_PSF        FLOAT[5]
       M_E2_PSF        FLOAT[5]
       M_RR_CC_PSF     FLOAT[5]
       M_CR4_PSF       FLOAT[5]
       ISO_ROWC        FLOAT[5]
       ISO_ROWCERR     FLOAT[5]
       ISO_COLC        FLOAT[5]
       ISO_COLCERR     FLOAT[5]
       ISO_A           FLOAT[5]
       ISO_AERR        FLOAT[5]
       ISO_AGRAD       FLOAT[5]
       ISO_B           FLOAT[5]
       ISO_BERR        FLOAT[5]
       ISO_BGRAD       FLOAT[5]
       ISO_PHI         FLOAT[5]
       ISO_PHIERR      FLOAT[5]
       ISO_PHIGRAD     FLOAT[5]
       R_DEV           FLOAT[5]
       R_DEVERR        FLOAT[5]
       AB_DEV          FLOAT[5]   Isophotal ratio a/b in the range [0,1] or -9999
       AB_DEVERR       FLOAT[5]
       PHI_DEV         FLOAT[5]
       PHI_DEVERR      FLOAT[5]
       COUNTS_DEV      FLOAT[5]
       R_EXP           FLOAT[5]
       R_EXPERR        FLOAT[5]
       AB_EXP          FLOAT[5]
       AB_EXPERR       FLOAT[5]
       PHI_EXP         FLOAT[5]
       PHI_EXPERR      FLOAT[5]
       COUNTS_EXP      FLOAT[5]
       NPROF           LONG[5]     Number of measured radial profile SBs, where
                                   the maximum is 15.
       PROFMEAN        FLOAT[15,5] Surface brightness in circular annuli
                                   [mean raw counts/pixel], -9999 if not measured
       PROFERR         FLOAT[15,5]
  5. calibObj structure calibrated quantities:
       RA              DOUBLE      J2000 RA [deg] in canonical (r-band) filter
       DEC             DOUBLE      Declination [deg] in canonical (r-band) filter
       OFFSETRA        FLOAT[5]    RA of center in this filter relative to RA [arcsec]
       OFFSETDEC       FLOAT[5]    DEC of center in this filter relative to RA [arcs
       PHI_ISO_DEG     FLOAT[5]    Isophotal fit, angle E of N [deg]
       PHI_DEV_DEG     FLOAT[5]    De Vaucouleurs fit, angle E of N [deg]
       PHI_EXP_DEG     FLOAT[5]    Exponential fit, angle E of N [deg]
       SKYFLUX         FLOAT[5]    Sky level [nano-maggies/arcsec^2]
       PSFFLUX         FLOAT[5]
       FIBERFLUX       FLOAT[5]
       MODELFLUX       FLOAT[5]
       PETROFLUX       FLOAT[5]
       DEVFLUX         FLOAT[5]
       EXPFLUX         FLOAT[5]
       APERFLUX        FLOAT[NAPER,5]  Aperture fluxes in radii of size 0.223,
                                       260.37 arcsec; our default is to return
                                       only those fluxes out to 11.3 arcsec,
                                       unless NAPER is explicitly specified
  6. calibObj structure external catalog matches:

    2MASS: If the /CATALOG option is used, then we include the closest 2MASS infrared source within 3 arcsec. The number of matches is reported in TMASS_NMATCH, and the closet match distance [in degrees] is TMASS_MDIST. Other 2MASS object parameters are specified in parameters whose names begin with "TMASS_". Details of 2MASS parameters can be found at: 605-763-8692

    FIRST: If the /CATALOG option is used, then we include the closest FIRST radio source within 3 arcsec. The number of matches is reported in FIRST_NMATCH, and the closet match distance [in degrees] is FIRST_MDIST. Other FIRST object parameters are specified in parameters whose names begin with "FIRST_". Details of FIRST parameters can be found at: /

    USNO-B: If the /CATALOG option is used, then we include the closest USNO-B1.0 source within 3 arcsec. The number of matches is reported in USNO_NMATCH, and the closet match distance [in degrees] is USNO_MDIST. Other USNO object parameters are specified in parameters whose names begin with "USNO_". Details of USNO parameters can be found at: 704-844-8813

Data Model: Resolving primary+repeat observations

RESOLVE_STATUS lets you know whether the object is unique within a particular run or within the survey.

The bit values are:

RESOLVE_STATUS  0 RUN_PRIMARY - Primary observation within a run
RESOLVE_STATUS  1 RUN_RAMP - At the very beginning or end of the run (in time)
RESOLVE_STATUS  2 RUN_OVERLAPONLY - Only appears in the overlap
RESOLVE_STATUS  3 RUN_IGNORE - "Bright" or "parent" object that should be ignored
RESOLVE_STATUS  4 RUN_EDGE - Near the NU boundary (X pixel position)
RESOLVE_STATUS  5 RUN_DUPLICATE - Duplicate measurement of the same object
                                  (from the same raw pixels)
RESOLVE_STATUS  8 SURVEY_PRIMARY - Primary observation within the full survey,
                                   where it appears in the primary observation
                                   of this part of the sky
RESOLVE_STATUS  9 SURVEY_BEST - Best observation within the full survey, but it
                                   does not appear in the primary observation
                                   of this part of the sky
RESOLVE_STATUS 10 SURVEY_SECONDARY - Repeat (independent) observation of an
                                   object that has a different primary or
                                   best observation
The following bits are exclusive, and exactly one of these bits will be set for each object: RUN_PRIMARY, RUN_RAMP, RUN_OVERLAPONLY, RUN_IGNORE, RUN_DUPLICATE. In addition, RUN_EDGE can be set for any objects. One typically wants all unique objects within a run that are not at the edge of field. To select such objects, the RUN_PRIMARY bit should be set, and the RUN_EDGE bit should not be set. The IDL code would be:
  (resolve_status AND 2L^0) NE 0 AND (resolve_status AND 2L^4) EQ 0
More properly, one could use the sdss_flagval() function:
  (resolve_status AND sdss_flagval('RESOLVE_STATUS','RUN_PRIMARY')) NE 0 $
  AND (resolve_status AND sdss_flagval('RESOLVE_STATUS','RUN_EDGE')) EQ 0

Data Model: Flux measurements + errors

Calibrated quantities are presented in nano-Maggies (abbreviated nMgy, a flux density), where 1 Maggie is the flux density of a 0th magnitude star on the "SDSS natural magnitude system." Our 6-sigma detection limit is approximately at a magnitude of 22.5, corresponding to 1 nMgy. The spectroscopic limit of 17.67 for the SDSS main galaxy survey corresponds to 85.5 nMgy.

Frequently asked questions

  1. I am familiar with tsObj files, the files output by the standard survey calibration pipeline. What are the main differences between tsObj files and calibObj files?
  2. Why use nano-maggies instead of Luptitudes?

    The official SDSS data releases present flux measurements in Asinh magnitudes, fondly known as Luptitudes (Lupton 1999, AJ, 118, 1406). This flux scaling is like magnitudes bright flux levels, and become approximately linear at a reference magnitude near the detection limit of the survey. This allows one to store zero and negative fluxes, and provides a quantity with (nearly) gaussian noise near the detection limit.

    Looking ahead to reduction of co-added scans (we have as many as 20 observations of some parts of the sky), the noise limit would vary across the sky. If Asinh magnitudes were used, this would require a different reference magnitude in different parts of the sky, which would be extraordinarily confusing. A linear flux quantity (such as the Maggie) preserves the advantages of gaussian noise and allowing zero fluxes, and scales to co-added data reduction with no problems.

    The nano prefix is used for convenience, since the faintest objects in the survey are ~ 1 nMgy.

  3. Why use ivar (inverse variance) instead of sigma?

    Inverse variance is the quantity most often wanted (e.g. inverse variances add when computing a weighted average) and the situation of zero signal to noise (no measurement) can often be handled by setting ivar=0 rather than requiring an additional flag.

    Implementation is also easy - to demand a S/N > 5 positive measurement, one would demand e.g. FLUX * sqrt(IVAR) GT 5 which always behaves correctly. The more conventional FLUX/SIG GT 5 requires checking that SIG is nonzero. Try it, you'll like it.

  4. What are your conventions for the adaptive moments?

    Exactly the same as the main survey. See the DR1 documentation at /

  5. How does one convert to AB magnitudes or to Janskys?

    All quoted fluxes are in terms of the "SDSS natural system". There are small corrections to convert to AB magnitudes, which is a physical system described by Oke & Gunn (1983). An estimation of these corrections as of Aug 2002 are (courtesy of David Hogg sdss-calib/845):

      u(AB) = 22.5 - 2.5*log10(flux_u) - 0.042
      g(AB) = 22.5 - 2.5*log10(flux_g) + 0.036
      r(AB) = 22.5 - 2.5*log10(flux_r) + 0.015
      i(AB) = 22.5 - 2.5*log10(flux_i) + 0.013
      z(AB) = 22.5 - 2.5*log10(flux_z) - 0.002
    Daniel Eisenstein has been working on improved estimations from white dwarf models (c.f., sdss-calib/1043).

    These quantities are related to physical units such that zero-th AB magnitude corresponds to 3631 Jy, where 1 Jy = 10-26W m-2 Hz-1 = 10-23 erg s-1 cm-2 Hz-1. This conversion to physical units is only known to about 5% (Oke & Gunn 1983). The measured calibrated magnitude or flux density of an object at the nominal central wavelength of a broad-band filter depends on the spectral shape of the source.

  6. What area of the sky do the "Orion" runs cover?

    The Galactic plane runs (nicknamed "Orion") actually also cover parts of Taurus and other parts of the Galactic plane.
    Run Date MJD Strip Field start Field end Area l b
    211 1998/10/29 51115 82S 11 385 71.25 210.16 -3.33
    250 1998/11/16 51133 82N 11 455 84.55 201.89 -18.03
    259 1998/11/17 51134 82N 11 567 105.83 206.66 -9.54
    273 1998/11/19 51136 82S 11 534 99.56 206.07 -11.11
    297 1998/11/22 51139 82O 11 203 36.67 206.00 -10.95
    307 1998/11/23 51140 82O 11 186 33.44 212.38 -0.00
    308 1998/11/23 51140 10O 11 216 39.14 215.95 4.19
    994 1999/10/06 51457 76S 11 133 23.37 209.88 -4.99
    1923 2000/12/08 51886 0O 29 71 8.17 86.40 0.01
    1924 2000/12/08 51886 0O 34 78 8.55 168.61 11.91
    1925 2000/12/08 51886 0O 17 48 6.08 165.26 6.56
    2955 2002/02/07 52312 82N 21 166 27.74 209.30 -4.54
    2960 2002/02/08 52313 82S 11 127 22.23 208.03 -7.43
    2968 2002/02/09 52314 82N 19 94 14.44 207.84 -7.33
    3511 2002/12/06 52614 0O 14 89 14.44 163.50 -8.21
    3512 2002/12/06 52614 0O 11 90 15.20 177.31 -9.27
    3557 2002/12/31 52639 0O 11 87 14.63 165.56 -10.47
    3559 2002/12/31 52639 0O 11 86 14.44 176.43 -9.95
    3610 2003/01/27 52666 61N 17 148 25.08 139.02 -10.77
    3628 2003/01/28 52667 61S 17 165 28.31 137.32 -10.88
    3629 2003/01/28 52667 61N 11 129 22.61 142.03 -10.91
    3634 2003/01/29 52668 62N 18 162 27.55 159.08 -13.00
    3642 2003/02/01 52671 62S 11 97 16.53 135.63 -13.17
    3643 2003/02/01 52671 62S 11 119 20.71 159.07 -13.21
    4114 2003/09/20 52902 0S 11 129 ?? ?? ??
    4115 2003/09/20 52902 0S 11 71 ?? ?? ??
    4116 2003/09/20 52902 0O 11 79 ?? ?? ??
    4119 2003/09/20 52902 0O 11 72 ?? ?? ??

Internal pages

Reduced data
Disk used
(401) 312-6024

Web page maintained by David Schlegel, 4162087159, and Nikhil Padmanabhan