hispecdrp.primitives.ramp_fitting.ramp_fit_primitive

Functions

_nan_safe_gaussian_filter(rate, sigma)

Gaussian smooth that ignores non-finite pixels (normalized by weight).

clip_noise_driven_poisson_variance(rate, ...)

Adjust reported Poisson variance after the ramp fit.

flag_nonfinite_rate_pixels(rate, rate_err, dq, *)

Backward-compatible wrapper around sanitize_ramp_fit_rate_err().

sanitize_ramp_fit_rate_err(rate, rate_err, dq, *)

Make ramp-fit rate_err safe for downstream extraction.

Classes

FitRampPrimitive([dataset, config, context, ...])

This step fits the slope (DN/s) from the UTR for each pixel.

class FitRampPrimitive(dataset=None, config=None, context=None, log_config=None, parent=None, **kwargs)[source]

Bases: BasePrimitive

This step fits the slope (DN/s) from the UTR for each pixel.

While fitting the ramps, this step will also calculate the wavelength dependent mid-exposure times, which is then used to calculate the wavelength dependent barycentric correction. HISPEC leverages the UTR instead of having an exposure meter for this purpose.

The entire ramp does not need to be discarded for a single jump, which is a significant advantage of UTR. Each ramp might be divided into separate chunks after the jump detection step. The final slope will be the combined slope estimate from every valid chunk in that pixel.

Descriptions of the different ramp fitting algorithms:

  • Ordinary Least Squares (‘ols’): This method fits a straight line to the ramp data using the ordinary least squares approach. This method is simple and computationally efficient but is not optimal with significant photon noise.

  • Correlated Double Sampling (‘cds’): This method uses the difference between last read and the first read to estimate the slope. It is very fast but not optimal for readnoise dominated ramps.

  • Multiple Correlated Double Sampling (‘mcds’): This method generalizes CDS by using multiple reads (see argument mcds_num_coadd) at the beginning and end of the ramp to reduce the readnoise.

  • JWST Fixsen (‘jwst_fixsen’): This method assumes that the covariance is piece-wise constant by dividing 9 different regimes of various ratios of readnoise and photon noise. This is referenced as OLS in the JWST pipeline. Fixsen+2000 https://ui.adsabs.harvard.edu/abs/2000PASP..112.1350F/abstract

  • JWST Likely (‘jwst_likely’): Uses the maximum-likelihood from Brandt (2024) to fit the ramps, which is the optimal approach (but assuming constant illumination). Reference: Brandt, T. D., PASP, 136, 045005 (2024) https://ui.adsabs.harvard.edu/abs/2024PASP..136d5005B/abstract

  • Cython Likely (‘cython_likely’): Same as JWST Likely but implemented in Cython for better performance (author: Brandt, Tim).

todo: As a future research avenue, although not a DRP requirement, we plan to explore using the UTR and the ATC images to optimize the slope fitting to account for the variable slopes due to cloud or seeing variability. For example, a simple strategy could be that reads acquired during spikes in seeing (as monitored in the ATC data) could simply be masked in the same manner as cosmic rays. This could be particularly beneficial for high contrast spectroscopy of companions because spikes in seeing means that there is more speckles and less planet signal getting through the fiber.

Parameters:
  • dataset (str or RampModel) – Input file path or data model to process. This must be a raw up-the-ramp (UTR) ramp data product in which ramps are to be fitted.

  • suffix (str or None, optional) – Suffix to append to the output filename. Default is 'rampfit'.

  • method ({'ols', 'cds', 'mcds', 'jwst_likely', 'jwst_fixsen', 'cython_likely'}, optional) – Ramp fitting algorithm to use. See descriptions above for details. Default is 'cython_likely'.

  • mcds_num_coadd (int or None, optional) – Number of reads to coadd for MCDS ramp fitting. If None, it will be set to n_reads/4. Only used if method is set to ‘mcds’. Default is None.

  • max_cores (int, optional) – Maximum number of CPU cores to use for parallel processing. Default is 1 (no parallel execution).

  • jwst_jump_detection (bool, optional) – If True, use shared memory optimization for jump detection in combination with JWST ramp fitting methods. Default is False.

  • jwst_jump_rejection_threshold (float, optional) – Rejection threshold (approximately in signal-to-noise) above which discontinuities are flagged as jumps when using JWST ramp fitting methods. Default is 4.0.

  • poisson_variance_mode ({'smooth_detect', 'positive_only', 'detect_nsigma', 'stcal'}, optional) – How to adjust reported rate_var_poisson / rate_err after the fit. 'smooth_detect' (default) gates Poisson on a smoothed rate so empty sky stays RN-limited while faint illuminated structure keeps Poisson; 'positive_only' clears Poisson only for rate <= 0; 'detect_nsigma' uses a per-pixel poisson_detect_nsigma gate (legacy); 'stcal' leaves the fit Poisson term unchanged.

  • poisson_detect_nsigma (float, optional) – Per-pixel detection threshold for poisson_variance_mode='detect_nsigma'. Default is 3.0.

  • poisson_smooth_sigma (float, optional) – Gaussian smoothing scale (pixels) for smooth_detect. Default is 1.0.

  • poisson_smooth_nsigma (float, optional) – Smoothed-rate detection threshold in RN σ for smooth_detect. Default is 1.0.

Calibrations:
Returns:

output (Level0Model) – Level 0 data model with fitted slope (rate) and associated arrays.

Notes

Roman calibration pipeline reference for ramp fitting algorithms: https://github.com/spacetelescope/romancal

Ramp fitting roman reference: https://www.stsci.edu/files/live/sites/www/files/home/roman/_documents/Roman-STScI-000394_DeterminingTheBestFittingSlope.pdf

See JWST primitive: https://jwst-pipeline.readthedocs.io/en/latest/jwst/ramp_fitting/description.html

CHARIS reference: https://ui.adsabs.harvard.edu/abs/2017JATIS…3d8002B/abstract

HPF DRP: https://github.com/indiajoe/HxRGproc

clip_noise_driven_poisson_variance(rate, rate_var_poisson, rate_var_rnoise, *, mode='smooth_detect', n_sigma=3.0, smooth_sigma=1.0, smooth_nsigma=1.0, rate_err=None)[source]

Adjust reported Poisson variance after the ramp fit.

Ramp fits (including stcal likely) evaluate photon-noise variance at a non-negative function of the fitted rate (Wald plug-in). Modes:

  • 'smooth_detect' (default): keep the fit Poisson term where rate > 0 and a Gaussian-smoothed rate exceeds smooth_nsigma * sqrt(rate_var_rnoise). Empty sky stays RN-limited (isolated positive noise spikes are dropped); faint illuminated structure keeps Poisson because neighbors hold the smooth rate up.

  • 'positive_only': zero Poisson only where rate <= 0.

  • 'detect_nsigma': also zero Poisson unless rate > n_sigma * sqrt(rate_var_rnoise) (legacy; underestimates faint-source rate_err).

  • 'stcal': leave rate_var_poisson unchanged; only rebuild rate_err = sqrt(var_poisson + var_rnoise).

If rate_err is provided, it is preserved wherever var_poisson / var_rnoise cannot form a finite rebuilt uncertainty (e.g. legacy fitters that omit the variance split).

Returns:

rate_var_poisson_out, rate_err_out, n_clipped

sanitize_ramp_fit_rate_err(rate, rate_err, dq, *, rate_var_rnoise=None, do_not_use=1)[source]

Make ramp-fit rate_err safe for downstream extraction.

Ramp fits often leave empty / inter-order pixels as rate==0 with NaN rate_err. Flagging those as DO_NOT_USE and letting box extract renormalize the aperture caused large spectral spikes for undersampled PSFs. Instead:

  • Where rate is finite and rate_err is not: fill rate_err with sqrt(rate_var_rnoise) when that variance is finite and positive, otherwise 0. Do not mark DO_NOT_USE.

  • Where rate is non-finite: OR do_not_use into dq and set rate_err to 0 for array hygiene.

Returns:

rate_err_out, dq_out, n_filled_err, n_flagged_rate – rate_err / dq are copied only when modified.