Re: Removing the blindness

From: Chris Lidman (clidman@eso.org)
Date: Fri Oct 15 2004 - 09:57:49 PDT

  • Next message: Alexander Conley: "Re: Removing the blindness"

    Dear Alex,
      Many thanks for your e-mail. I have read your reply in detail and
    I am satisfied with your answers. I have some additional, minor
    comments for you to consider.

    Cheers, Chris.

    Figure 3
    --------

    To make this figure a little clearer, you might want to add arrows
    linking the line ends so that readers can see more clearly how the
    lines move.

    A comment on the high redshift residuals
    -----------------------------------------

    The SNe you use from Rob's paper are

    97eq
    98as
    98aw
    98ax
    98ba
    00fr

    Note that 98as, 98ax and 98aw were excluded from Rob's low extinction
    subset. Do these 3 SNe dominate the rightmost blue histogram in figure
    30 of your methods paper? If this is the case, then this
    adds extra support to the notion that extinction is the cause in
    the split of the high redshift residuals.

    Section 4
    ---------

    Second paragraph.

    ... joint fit is preferred.

    I think that this needs to be stronger

    ... joint fit is used.

    Lightcurve fits
    ---------------

    There was an earlier discussion on the behaviour of the snminuit program
    that
    may have affected the V band fits in P99. Saul mentioned that this
    feature may have already been known. Was this actually the case? I have
    lost the thread of the discussion since Saul's e-mail.
    See http://supernova.lbl.gov/collab/archive/cmag/0020.html.

    Figure 2
    --------

    Ideogram. I am not familiar with the way in which this word is used in
    the
    figure caption. From the Webster dictionary
      

    \I*de"o*gram\, n. [Ideo- + -gram; cf. F. id['e]ograme.]
    1. An original, pictorial element of writing; a kind of
       hieroglyph expressing no sound, but only an idea.

             Ideograms may be defined to be pictures intended to
             represent either things or thoughts. --I. Taylor
                                                   (The
                                                   Alphabet).

             You might even have a history without language
             written or spoken, by means of ideograms and
             gesture. --J. Peile.

    2. A symbol used for convenience, or for abbreviation; as, 1,
       2, 3, +, -, ?, $, ?, etc.

    3. A phonetic symbol; a letter.

    On Wed, 2004-10-13 at 18:16, Alexander Conley wrote:
    > Hello all,
    >
    > I have finished (I hope) trying to answer all of the questions that
    > you
    > have asked me about my analysis. What I need from you is for you all
    > to tell me if you are satisfied with my attempt at an answer, or if I
    > need
    > to look at a particular issue more deeply.
    >
    > There is a new version of the paper (1.32) available in the usual place
    > ( http://panisse.lbl.gov/~aconley/cmag ) which incorporates many of the
    > changes you suggested, plus a few more.
    >
    > The biggest change is that probcut is no more. I discovered a subtle
    > bias that probcut was introducing in the analysis (it tended to punish
    > extremely well observed low redshift SNe with slopes that were a bit
    > away from the central value). I ended up excluding 2 low-z SNe by hand
    > that probcut had been catching: 97br, 97bp. Their data appears to
    > be internally inconsistent, with a much larger degree of scatter than
    > can be accounted for by their quoted photometric errors.
    >
    > There is also a (blinded) w fit. I haven't combined it with the other
    > priors, but there isn't much point in doing so in blind mode anyways.
    > Finally, I started putting the systematic errors together, although the
    > job isn't done yet.
    >
    > Below I have listed all of you in alphabetical order and tried to give
    > a brief response to each of the questions you asked of me.
    >
    > Ariel:
    > Original message:
    > http://supernova.lbl.gov/collab/archive/cmag/0006.html
    >
    > Q: Have you tried not applying the MW extinction correction and
    > seeing
    > if that behaves as you expect?
    > A: As expected, the m_B residuals (as measured by the RMS) got much
    > worse and the B_BV residuals did not.
    > More info:
    > http://supernova.lbl.gov/collab/archive/cmag/0010.html
    >
    > Q: Why do you set s_B = s_V for some fits but not for others?
    > A: I think it would be internally inconsistent to force s_B=s_V
    > given what
    > we know from CMAGIC -- namely, that when there is a bump, s_V <
    > s_B.
    > Note that I have only allowed s_B ne s_V when there is a bump.
    > And it's not like the s_B = s_V fits to SNe with bumps are good
    > -- most
    > of them are absolutely awful, clearly missing the shape of the
    > light curve.
    > On a related note, on Saul's urging I did redo all fits so that
    > s_B was fit
    > to the B data only, and found that using these values for
    > stretch, etc. made
    > very little difference. See Saul's section below.
    > More info:
    > http://supernova.lbl.gov/collab/archive/cmag/0007.html
    >
    > Q: If the split in the high redshift residuals (between the Knop and
    > Barris sample)
    > is really due to dust, can you see this in the residuals of
    > m_B vs B_BV?
    > A: Yes, the matched residuals are consistent with this story. This
    > is probably
    > my single greatest area of concern with my result -- but I'm
    > not sure what
    > to do about it. Note that if a 'by-hand' extinction
    > correction is applied, the
    > sample no longer splits.
    > More info:
    > http://supernova.lbl.gov/collab/archive/cmag/0007.html
    >
    > Q: Have you tried to set limits on intergalactic dust?
    > A: No, but we can talk about this after the blindness comes off.
    >
    > Chris:
    > Original email:
    > http://supernova.lbl.gov/collab/archive/cmag/0001.html
    >
    > Q: Could you add a stretch vs. scriptM plot?
    > A: Done
    >
    > Q: You seem less sensitive to the U-enhanced K-correction
    > systematic
    > than K03. Could you emphasize this more?
    > A: Am I? K03 was completely insensitive to changing the U-B
    > color of
    > the uberspectrum as long as an extinction correction was not
    > applied,
    > and very sensitive when one was. I don't apply a extinction
    > correction,
    > and my sensitivity lies somewhere between the two K03 values.
    > See the latest version of the paper for actual numbers.
    >
    > Q: s_B ne s_V?
    > A: Ariel also asked this (although you beat him to the punch).
    > See the
    > answer in his section.
    >
    > Greg:
    > Greg's comments:
    > http://supernova.lbl.gov/collab/archive/cmag/0013.html
    > Greg had many comments. I won't be able to hit them all here. See
    > http://supernova.lbl.gov/collab/archive/cmag/0013.html for many
    > of my responses.
    > Some highlights, and things I didn't get to in that email.
    >
    > Q: The first 3 high redshift SNe all seem high. Is their anything
    > funny with them?
    > A: Nothing I can find. They come from 3 different reductions
    > (P99, K03 and Riess 98).
    > Applying a by-hand extinction correction does not save them.
    > They are also
    > all high in a maximum magnitude fit. There was a problem
    > with one of the
    > lightcurve points for SN1995ba (a photometry error -- I have
    > informed Rob),
    > but that doesn't really 'save' it.
    >
    > Q: Can you break up the residuals by various subsamples and see if
    > they
    > differ.
    > A: By photometry source: The SCP and hi-z team samples are quite
    > compatible
    > in terms of mean and scatter. SCP mean: 0.025 avdev:
    > 0.183 rms: 0.218
    > high-z team mean: 0.006 avdev: 0.183 rms: 0.233. The
    > Hawaii (basically
    > Barris) sample has an offset mean and lower rms -- mean:
    > -0.13
    > avdev: 0.11 rms: 0.19. The offset is well documented
    > (the split in the
    > residuals) -- see Ariel's section.
    > By redshift: Split into three chunks: [0,0.45),
    > [0.45,0.55),[0.55) (to get roughly
    > equal numbers of SNe), they are consistent. The RMS gets
    > marginally
    > smaller at the high redshift end (0.23 -> 0.17), but I
    > expect this to some
    > extent because the mean extinction will be lower because
    > of selection effects.
    > Summary: I see no evidence for any real problems, except
    > the already noted
    > shift in the mean between the Barris sample and everybody
    > else (which seems
    > to arise because the Barris sample has unusually little
    > host galaxy extinction).
    >
    > Q: In the matched residuals (m_B vs B_BV) is the distribution
    > narrowing as you
    > move towards positive residuals? Do bump SNe have anything
    > to do with this?
    > A: Splitting the sample in half, the lower RMS is 0.16, the upper
    > is 0.15, so it is
    > slightly narrower, but not by much. Removing all of the
    > SNe with bumps
    > decreases the RMS values slightly to lower: 0.16 upper:
    > 0.14.
    > On a related note, of the 8 SNe with bumps, 6 have
    > positive residuals.
    > This is significant at the 1.4 sigma (equiv: 14.4%) level,
    > according to
    > the binomial distribution. Bottom line: This doesn't seem
    > significant.
    >
    > Q: What does the chisquare distribution for the fits to the CMAG
    > relations look like?
    > A: Too good -- as expected. This is why W03 rescaled their error
    > bars -- people
    > are clearly underestimating them. I decided that the
    > rescaling was far too
    > dangerous given the quality of the high-z data. The plot of
    > the chisq probability
    > (the prob that the chisq should be higher than it is
    > observed to be) is
    > attached. It is very difficult to say what this
    > distribution should look like because
    > it is different for every number of points in the CMAG
    > region, but it's fair
    > to say it should be peaked around 0.5. The big spike at
    > around 1 is a bunch
    > of SNe that are too well fit.
    >
    > ______________________________________________________________________
    >
    > Mark:
    > Original email:
    > http://supernova.lbl.gov/collab/archive/cmag/0011.html
    > Q: Could you include the intrinsic error in the stretch vs scriptM
    > plot?
    > A: Done
    > Q: How about propogating the z error into the vertical error bars
    > in the
    > Hubble residual plots?
    > A: Done
    > Q: On the resid vs. resid plot, do you take the errors into
    > account when
    > calculating the pearson's coefficient?
    > A: Not yet, but I think this can wait until after the blindness is
    > removed
    > (since that shouldn't affect these values).
    >
    >
    > Saul:
    >
    > Q: Could you do a blinded w fit?
    > A: Done. See the most recent version of the paper. The blindness
    > scheme
    > is a bit rough given the almost pathological shape of the
    > error contours.
    > I have not combined these contours with the other priors yet,
    > but there is
    > no point in doing so now. Adding the priors and then blinding
    > would probably allow the blindness scheme to be violated, so I
    > won't do it.
    > More info:
    > http://supernova.lbl.gov/collab/archive/cmag/0021.html
    >
    > Q: Could you redo all of the lightcurve fits using the prescription
    > of P99 --
    > that is, do a B only fit, then fix the stretch and date of
    > maximum to those values
    > for the V fit.
    > A: Done. Didn't make a difference (see the most recent version of
    > paper). As
    > detailed at great length in
    > http://supernova.lbl.gov/collab/archive/deepnews/1361.html
    > a very long and boring email which nobody probably read, this
    > is (a) harder
    > than it looks, and (b) wasn't done right in P99. Luckily it
    > only affects the V
    > magnitudes, so doesn't change fit C. Note that I didn't
    > follow this prescription
    > for the 8 bump SNe, since the quality of the fit in that case
    > is god-awful.
    >
    > Q: What effect does applying an extinction correction have?
    > A: It shifts the contours almost purely along the long axis inward
    > (by 0.4ish).
    > Note that this fit can NEVER be published. It's a piece of
    > junk found by
    > combining lots of extinction values that other people
    > calculated, and so
    > is horribly internally inconsistent.
    >
    > Q: What effect does removing all SNe with bumps have?
    > A: A shift of 0.016 along the short axis and -0.14 along the long
    > axis, giving
    > lower values of om, ol, but a slightly higher value (+0.016)
    > for om in a flat
    > universe. Not a really large systematic.
    >
    > Vitaliy:
    > Original email:
    > http://supernova.lbl.gov/collab/archive/cmag/0015.html
    > Q: Are you going to combine the shifts into a final systematic error?
    > A: Yes. I have started doing this in the latest paper draft.
    >
    > Okay -- hopefully I didn't miss anything. Please let me know if you are
    > happy with my answers, and if not, what I can do to try and answer your
    > questions better. Then, the blindness can be removed.
    >
    > Alex

    -- 
    European Southern Observatory
    Alonso de Córdova 3107, Vitacura
    Casilla 19001, Santiago 19
    CHILE
    

    Ph. +56 2 463 3106 FAX +56 2 463 3101



    This archive was generated by hypermail 2.1.4 : Fri Oct 15 2004 - 09:58:30 PDT