hispecdrp.utils.line_mask_utils
Line-mask utilities (mask I/O, MaskLines container, template-built masks).
The canonical overview of mask formats and generation lives in the documentation:
docs/source/algorithms/line_mask.rst.
Module Attributes
Defaults for ideal-spectrum paint + line-mask RV (alias for notebooks/tests). |
|
Defaults for |
|
Packaged default delta line mask basename under |
Functions
|
Interpolated λ where |
|
If |
|
Two-stage grid search over mask λ near |
|
Single-line spline CCF + Gaussian fit at injected RV ≈ 0. |
|
Union lines with |
|
|
|
Second derivative w.r.t. |
|
Infer wavelength column scale for legacy two-column text masks: median of positive finite values → Å (>1e3) /1e4, nm (>10) /1e3, else assume µm. |
|
Per centroid, minimum of \(F/F_c\) over discrete samples in |
|
|
|
Drop duplicates closer than |
|
Map legacy |
|
Sub-pixel wavelength of a local absorption minimum. |
|
Mean \(1 - F/F_c\) over discrete template samples inside |
Replace wide |
|
Build a CCF-opt delta mask from an injected-RV=0 L2 via line-by-line stellar RV. |
|
Build a static rest-frame CCF-opt delta mask by zeroing single-line CCF bias. |
|
Build a CCF-opt delta mask from a line-by-line stellar-RV |
|
Refine a seed delta mask so each line maximizes single-line Gaussian CCF contrast at RV=0. |
|
|
Build a delta line mask aligned with e2e WAVE / RV truth (template centers). |
|
Detect lines in a template spectrum and write a filtered |
|
Match two line-center lists (vacuum µm) within |
Absorption line centers (vacuum µm) from gradient zeros with positive curvature. |
|
Line centers (vacuum µm) from discrete local flux minima + parabolic refinement. |
|
|
Fit |
|
Load Gemini |
Serializable Astropy/FITS unit strings for |
|
Mean atmospheric transmission over |
|
Absolute path to |
|
Absolute path to the packaged default delta mask (same as |
|
|
Resolve the packaged line mask paired with a template basename or path. |
If |
|
|
Pixel bin left/right edges (µm) from a sorted wavelength array (midpoints + end padding). |
Scan the start of a |
|
|
Resolve |
|
Keep lines whose rest centers can Doppler-shift into |
|
Resolve mask lines for line-mask CCF (path, default package mask, or in-memory arrays). |
Classes
|
One template-built mask row: vacuum wavelength (nm) and CCF weight. |
|
Tabulated CCF line mask (delta zero-width lines or tophat finite segments). |
- DEFAULT_PACKAGED_LINE_MASK_BASENAME = 'lte049.0-2.5-0.0a+0.0.BT-Settl.spec.7.ccfopt.delta.mas'
Packaged default delta line mask basename under
package_data_masks_dir(). Prefer the e2e-validated mask used to bakeHB.sim_hdf_wavesol.line_zp.fits.
- E2ERV_MASK_BUILD_KW = {'continuum_filter_size': 181, 'deblend_sep_resels': 1.5, 'min_continuum_normalized_depth': 0.2, 'min_continuum_normalized_flux_floor': 0.001, 'resel_um': 0.00015}
Defaults for
build_e2erv_line_mask_from_template()(RV-aligned e2e mask).
- IDEAL_SPECTRUM_STELLAR_RV_KW = {'ccf_input_velocity_m_s': 0.0, 'ccf_min_good_pixels': 9, 'mask_ccf_pixel_interp_order': 3, 'max_abs_rv_m_s': 500.0, 'max_rv_err_m_s': 10.0, 'min_contrast': 0.0, 'rezero_verify_abs_rv_m_s': 0.001, 'v_half_window_m_s': 20000.0, 'v_step_m_s': 10.0, 'window_line_by_line': True, 'window_size_um': 5.65e-05}
Defaults for ideal-spectrum paint + line-mask RV (alias for notebooks/tests).
- class LineMaskEntry(wavelength_nm, weight)[source]
Bases:
objectOne template-built mask row: vacuum wavelength (nm) and CCF weight.
- class MaskLines(mask_style, line_center, line_left_edge, line_right_edge, line_width, weight, metadata=<factory>)[source]
Bases:
objectTabulated CCF line mask (delta zero-width lines or tophat finite segments).
delta — two on-disk columns (vacuum line-center nm, weight):
line_left_edge,line_right_edge, andline_centercoincide;line_widthis 0. tophat — ≥3 columns (vacuum segment start nm, end nm, weight):line_centeris the segment midpoint;line_width = line_right_edge - line_left_edge.Wavelengths are vacuum; internal arrays use microns (nm÷1000 on read/write for
.mas). Useload()andsave()for disk I/O with# HISPEC_DRP_MASK_METAJSON (read_mask_generation_meta()).CCF hot paths use
line_lambda_um(centers, µm) andweight.- classmethod _from_mas_path(mask_file)[source]
- classmethod _from_two_column_path(mask_file)[source]
- classmethod empty_like_style(mask_style, *, metadata=None)[source]
Zero-line mask preserving
mask_style(e.g. after wavelength restriction).
- classmethod from_delta_centers_um(line_lambda_um, weight, *, metadata=None)[source]
Build a delta mask: zero width at each center (µm).
- classmethod from_kpf_segment_edges_um(lo_um, hi_um, weight, *, metadata=None)[source]
Deprecated alias for
from_tophat_segment_edges_um().
- classmethod from_tophat_segment_edges_um(lo_um, hi_um, weight, *, metadata=None)[source]
Build a tophat (finite-width) mask from rest segment bounds in µm.
- classmethod load(path)[source]
Load a mask from disk:
.mas(vacuum nm on disk → µm) or two-column text (.txt/.dat/.csv; wavelength column Å/nm/µm via median heuristic). Other extensions try.masparsing first, then two-column.
- metadata_dict()[source]
Shallow copy of
metadatafor inspection or JSON serialization.
- save(output_path=None, *, masks_directory=None, basename=None)[source]
Write this mask to a
.masfile.On-disk layout:
delta→ vacuum line-center (nm), weight;tophat→ vacuum segment start (nm), end (nm), weight. One# HISPEC_DRP_MASK_METAJSON line (read_mask_generation_meta()).When
output_pathisNone, writes{masks_directory}/{basename}.{mask_style}.maswithbasenamefrommetadata["template_path"]when set, else"mask".masks_directorydefaults topackage_data_masks_dir().
- apply_ccfopt_defaults_for_line_by_line(*, window_size_um, v_half_window, v_step_coarse)[source]
Replace wide
StellarRVPrimitivedefaults with ccfopt mask-build settings.When
window_line_by_line=True, line-mask CCF should use the same narrowwindow_size_umand velocity grid asCCFOPT_SINGLE_LINE_STELLAR_RV_KWunless the caller explicitly set non-default values.
- build_ccfopt_line_mask_from_l2_line_by_line_rv(seed_mask_path, input_model, *, fiber=1, template_path=None, output_path=None, masks_directory=None, ccf_input_velocity_m_s=0.0, outlier_sigma=1000000.0, use_single_line_builder=False, match_half_m_s=5000.0, max_abs_rv_m_s=500.0, min_contrast=0.0, window_size_um=5.65e-05, window_line_by_line=True, v_half_window_m_s=20000.0, v_step_m_s=10.0, mask_ccf_pixel_interp_order=3, zb=0.0, max_seed_lines=None)[source]
Build a CCF-opt delta mask from an injected-RV=0 L2 via line-by-line stellar RV.
Two-step workflow (default
use_single_line_builder=False):Run
StellarRVPrimitive(algorithm="ccf_line_mask")oninput_modelwith the seed mask and line-by-line windowing atccf_input_velocity_m_s(typically 0). Setoutlier_sigmavery large (default1e6) so chunk outlier rejection does not drop mask lines before aggregation.Pass the per-line
RVCCFtable tobuild_ccfopt_line_mask_from_rvccf_table(), which drops seed lines when any matching per-order calibration RV exceedsmax_abs_rv_m_s(defaultCCFOPT_MAX_ABS_RV_M_S), then shifts each kept lineλ' = λ × (1 + RV/c)and sets weights proportional to|Gaussian contrast|.
When
use_single_line_builder=True, delegate to the slower per-line CCF pathbuild_ccfopt_line_mask_from_l2_single_line_ccf()(gold-standard).
- build_ccfopt_line_mask_from_l2_single_line_ccf(seed_mask_path, input_model, *, fiber=2, template_path=None, output_path=None, masks_directory=None, window_size_um=5.65e-05, window_line_by_line=True, v_half_window_m_s=20000.0, v_step_m_s=10.0, ccf_input_velocity_m_s=0.0, mask_ccf_pixel_interp_order=3, ccf_min_good_pixels=9, max_abs_rv_m_s=500.0, min_contrast=0.0, rezero_verify_abs_rv_m_s=0.001, max_rv_err_m_s=10.0, zb=0.0, max_seed_lines=None)[source]
Build a static rest-frame CCF-opt delta mask by zeroing single-line CCF bias.
For each seed line, run the same line-mask CCF path as
StellarRVPrimitive(algorithm="ccf_line_mask")with line-by-line settings (CCFOPT_SINGLE_LINE_STELLAR_RV_KW, matchingrv_recovery_sim.ipynb) but with a mask containing only that line. Shift the line center so the recovered RV equals the injected prior (ccf_input_velocity_m_s, typically 0):λ' = λ_seed × (1 + (RV − ccf_input_velocity) / c)The output
.masis a fixed rest-frame list (does not vary with bulk RV). Weights are proportional to|Gaussian CCF contrast|at the measured (pre-shift) CCF and are renormalized to sum to 1.Lines are dropped when:
no valid chunk RV,
|RV|abovemax_abs_rv_m_s, or contrast at/belowmin_contrast;formal RV uncertainty
≥ max_rv_err_m_s(default 10 m/s);after the λ shift, re-measured
|RV − center| ≥ rezero_verify_abs_rv_m_s(default 0.001 m/s = 0.1 cm/s), or the re-measured uncertainty fails the samemax_rv_err_m_sgate.
- Parameters:
seed_mask_path – Packaged
*.delta.masseed.input_model – Blaze-corrected L2-like model (
WAVE_FIB*/SPEC_FIB*).fiber – Fiber label matching the L2 attributes (default
2).
- build_ccfopt_line_mask_from_rvccf_table(seed_mask_path, rvccf_table, *, template_path=None, output_path=None, masks_directory=None, match_half_m_s=5000.0, max_abs_rv_m_s=500.0, min_contrast=0.0)[source]
Build a CCF-opt delta mask from a line-by-line stellar-RV
RVCCFtable.For each seed line, gather all
RVCCFrows withinmatch_half_m_s(any echelle order). Drop the seed when any matching row has|RV| > max_abs_rv_m_s(failed / aliased calibration in at least one order). Otherwise pick the nearest unused matching row, shift the mask center so that chunk’s measured RV becomes 0:λ' = λ_seed × (1 + RV / c),and set the weight proportional to
|Gaussian contrast|of that chunk’s stored CCF. Weights are renormalized to sum to 1.- Parameters:
seed_mask_path – Path to the seed
*.delta.mas(typically the packaged depth-weighted mask).rvccf_table – Astropy table from
StellarRVPrimitivewithwindow_line_by_line=True(columnsRV,WAVELENGTH,CCF,V_MIN,V_MAX,V_STEP).template_path – Optional template path for metadata / default output naming.
output_path – Output
.maspath (default<stem>.ccfopt.delta.masundermasks_directory).masks_directory – Output
.maspath (default<stem>.ccfopt.delta.masundermasks_directory).match_half_m_s – Maximum
|Δv|between seed λ and chunkWAVELENGTHfor a match.max_abs_rv_m_s – Drop a seed line when any matching
RVCCFrow (across orders) has|RV|above this gate (failed / aliased per-line calibration).min_contrast – Drop lines whose absolute Gaussian contrast is at or below this floor.
- build_ccfopt_line_mask_from_template_and_seed(template_path, seed_mask_path, *, wl_min_um=0.8, wl_max_um=3.0, spectrum_window_half_um=0.01, ccf_v_half_m_s=8000.0, ccf_v_step_m_s=25.0, search_half_m_s=2500.0, coarse_step_m_s=250.0, fine_half_m_s=300.0, fine_step_m_s=25.0, rv_gate_m_s=2000.0, min_good_pixels=32, output_path=None, masks_directory=None, wave_um=None, flux=None)[source]
Refine a seed delta mask so each line maximizes single-line Gaussian CCF contrast at RV=0.
For every seed center
λ₀, search trial centersλ = λ₀ (1 + δv/c)on the template spectrum (injected RV = 0) and keep the wavelength that maximizes|Gaussian contrast|fromcalc_ccf_metrics(). Output weights are proportional to that contrast and are renormalized to sum to 1.- Parameters:
template_path – Path to a BT-Settl
.h5template (used for I/O naming / metadata; also loaded unlesswave_um/fluxare provided).seed_mask_path – Path to an existing delta
.masseed (typically the packaged depth-weighted mask).wave_um – Optional in-memory vacuum-µm spectrum (for tests). When either is set, both must be set and
template_pathis used only for metadata / default output naming.flux – Optional in-memory vacuum-µm spectrum (for tests). When either is set, both must be set and
template_pathis used only for metadata / default output naming.output_path – Full output path. Default:
<stem>.ccfopt.delta.masundermasks_directory.
- Returns:
path_written, entries – Output
.maspath and in-memory rows.
- build_e2erv_line_mask_from_template(template_path, *, wl_min_um=0.8, wl_max_um=3.0, min_continuum_normalized_depth=0.2, deblend_sep_resels=1.5, resel_um=0.00015, continuum_filter_size=181, min_continuum_normalized_flux_floor=0.001, output_path=None, masks_directory=None)[source]
Build a delta line mask aligned with e2e WAVE / RV truth (template centers).
Uses parabolic absorption centroids (same family as the sim-HDF line WAVE correction), a resolution-element deblend that keeps the deepest line in each blend, and continuum-normalized depth weights (sum → 1). No CCF-opt λ shifts — centers stay at template truth so CCF at injected RV=0 is unbiased when WAVE labels apparent line centers.
Default output:
<template_stem>.e2erv.delta.mas.
- build_line_mask_from_template(template_path, *, wl_min_um=0.8, wl_max_um=3.0, min_continuum_normalized_depth=0.1, low_pass_width_multiplier=7.283985083415576, telluric_velocity_half_width_km_s=288.2674944075802, telluric_min_mean_transmission=0.0, transmission_file=None, continuum_filter_size=181, min_continuum_normalized_flux_floor=0.0001, apply_deblend=True, apply_telluric_filter=True, local_min_fn_window_half_width_nm=None, output_path=None, masks_directory=None)[source]
Detect lines in a template spectrum and write a filtered
.masmask.Centroids come from
detect_template_line_centers_gradient_um(). Each output row is(wavelength_vacuum_nm, weight)with no segment edges. Raw weights are continuum-normalized depth \(1 - F/F_c\) at each line center (interpolated), then are divided by their sum so output weights sum to 1.Deblending (when
apply_deblend) uses0.5 * low_pass_width_multiplier * median(Δλ_nm)as the merge radius per line (deepest line kept in each cluster), wheremedian(Δλ_nm)is from the sorted template grid. Passapply_deblend=False/apply_telluric_filter=False(seeRELAXED_TEMPLATE_MASK_BUILD_KW) to maximize line count.- Parameters:
template_path – Path to a supported template (BT-Settl
.h5).wl_min_um – Wavelength clip in vacuum microns before line search.
wl_max_um – Wavelength clip in vacuum microns before line search.
min_continuum_normalized_depth – Passed to
detect_template_line_centers_gradient_um()and re-applied after interpolation to centers.low_pass_width_multiplier – Scales the deblending radius vs. the template median wavelength step (nm). Ignored when
apply_deblendis false.telluric_velocity_half_width_km_s – Half-width in km/s for the telluric averaging interval (± this value). Ignored when
apply_telluric_filteris false.telluric_min_mean_transmission – Lines with mean transmission below this value are removed (telluric path only).
transmission_file – Two-column
mktrans-style path (µm, transmission). Default: bundleddata/atmosphere/mktrans_zm_10_10.dat.continuum_filter_size – Odd-ish pixel size for the maximum-filter continuum envelope.
min_continuum_normalized_flux_floor – If not
None, drop candidate lines whose local minimum of \(F/F_c\) (over±local_min_fn_window_half_width_nmaround the centroid) is below this value. Saturated cores have \(F/F_c \to 0\) and are skipped. SetNoneto keep saturated cores (relaxed seed).apply_deblend – If false, skip nearest-neighbor deblend clustering (keep all depth-passing lines).
apply_telluric_filter – If false, skip the atmospheric-transmission gate.
local_min_fn_window_half_width_nm – Half-width of the wavelength interval (nm, vacuum) used for the local \(\min(F/F_c)\) test. If
None, uses5 ×the template median step (nm).output_path – Full path for the output
.mas. IfNone, writes undermasks_directoryas<template_basename>.delta.mas(deltais the on-diskMASK_STYLE_DELTAlayout; distinct stems per style matchMaskLines.save()).masks_directory – Default output directory (package
data/masksifNone).
- Returns:
path_written, entries – Output file path and in-memory rows (
wavelength_nm,weight).
- compare_line_center_lists_um(centers_a, centers_b, *, match_tol_nm=0.05)[source]
Match two line-center lists (vacuum µm) within
match_tol_nm(nanometers).Returns counts and boolean masks over the sorted union of inputs for which centers fall in only A, only B, or both (matched pairs use greedy nearest-neighbor within tolerance on the merged sorted list — for diagnostics only).
Notes
Greedy matching: sort all centers from A and B with tags; sweep λ increasing, pairing unmatched A with next unmatched B if
|Δλ|≤ tol (in µm:tol_nm * 1e-3). Simpler approach: for each a in A, match if any b in B with|a-b|<= tol_um.- Returns:
dict – Keys:
n_a,n_b,n_match,n_a_only,n_b_only,match_tol_nm,matched_pairs(list of (a_um, b_um)),centers_a_only_um,centers_b_only_um.
- detect_template_line_centers_gradient_um(wave_um, flux, *, continuum_filter_size=181, min_continuum_normalized_depth=0.04, merge_min_sep_um=None)[source]
Absorption line centers (vacuum µm) from gradient zeros with positive curvature.
Intervals where the first derivative of flux w.r.t. λ (
numpy.gradient) crosses zero are refined to a wavelength; the point is kept only if the second derivative of flux w.r.t. λ is positive (flux minimum, not maximum) and the usual continuum-normalized depth exceedsmin_continuum_normalized_depth.This is an alternative to chaining
_local_minima_indices()with_parabolic_minimum_wavelength_um(); lists can be compared withcompare_line_center_lists_um().- Parameters:
wave_um – Sorted vacuum-µm template grid (as from
load_template_from_path).flux – Sorted vacuum-µm template grid (as from
load_template_from_path).continuum_filter_size – Odd-ish size for
_continuum_envelope()(depth definition).min_continuum_normalized_depth – Minimum
1 - F/F_cat the interpolated stationary point.merge_min_sep_um – Merge roots closer than this (µm). Default:
0.25 * median(Δλ)on the grid.
- Returns:
np.ndarray – Centers in vacuum microns, sorted ascending.
- detect_template_line_centers_parabolic_um(wave_um, flux, *, local_min_order=2, continuum_filter_size=181, min_continuum_normalized_depth=None)[source]
Line centers (vacuum µm) from discrete local flux minima + parabolic refinement.
This matches the centroid step in
build_line_mask_from_template()before depth / deblend / telluric filters:_local_minima_indices()onfluxand_parabolic_minimum_wavelength_um()at each index.- Parameters:
wave_um – Same sorted vacuum-µm grid as
load_template_from_path(ascending λ).flux – Same sorted vacuum-µm grid as
load_template_from_path(ascending λ).local_min_order –
orderforscipy.signal.argrelextrema().continuum_filter_size – Used only if
min_continuum_normalized_depthis set.min_continuum_normalized_depth – If not
None, drop centers with1 - F/F_cbelow this threshold at the refined λ (F,F_cinterpolated), matching the spirit of the mask depth cut inbuild_line_mask_from_template().
- Returns:
np.ndarray – 1-D float64 centers in vacuum microns, sorted ascending.
- fit_half_width_linear_from_mas(mas_path)[source]
Fit
half_width_nm = slope * center_nm + interceptfrom a SPIROU.masfile.Comment lines (
#,;,//) are skipped; centers and half-widths are computed from the first two columns (start_nm, end_nm).
- load_atmospheric_transmission_um(path=None)[source]
Load Gemini
mktranstwo-column ASCII: wavelength (µm), transmission (0–1).Comment lines (
#,;,//) are skipped. The bundled file underdata/atmosphere/ships with a documented header; data rows are fixed-format decimals. External copies that omit a leading0on the first column (e.g..900180) are still accepted.
- masklines_wavelength_unit_meta(unit)[source]
Serializable Astropy/FITS unit strings for
MaskLines.metadatawavelength keys.
- mean_transmission_over_velocity_interval_um(line_wavelength_um, trans_wavelength_um, transmission, half_width_km_s)[source]
Mean atmospheric transmission over
line_wavelength_um * (1 ± v/c)(vacuum, µm).half_width_km_sis a half-width in km/s; the interval is \(\pm\) that value (total span \(2 v / c\) in fractional wavelength). Sampling uses a fixed number of points on the interval; values are linearly interpolated in the transmission table and clipped to the table bounds.
- package_data_masks_dir()[source]
Absolute path to
hispecdrp/data/masks.
- packaged_default_line_mask_path()[source]
Absolute path to the packaged default delta mask (same as
resolve_mask_path(None)).
- paired_line_mask_path_for_template(template_filename, *, mask_style='delta', masks_directory=None)[source]
Resolve the packaged line mask paired with a template basename or path.
For template
lte049.0-2.5-0.0a+0.0.BT-Settl.spec.7.h5, returns.../lte049.0-2.5-0.0a+0.0.BT-Settl.spec.7.delta.maswhenmask_styleisdelta.
- parse_mask_generation_meta_line(line)[source]
If
linestarts withLINE_MASK_META_PREFIX, return the parsed JSON object.The remainder of the line after the prefix must be a single JSON object.
- pixel_edges_from_wavelength(wave_um)[source]
Pixel bin left/right edges (µm) from a sorted wavelength array (midpoints + end padding).
- read_mask_generation_meta(path)[source]
Scan the start of a
.masfile forparse_mask_generation_meta_line().
- resolve_mask_path(mask_path)[source]
Resolve
mask_pathto an absolute filesystem path, or the packaged default whenNone.- Raises:
FileNotFoundError – If
mask_pathisNoneand the default mask file is missing.
- restrict_mask_to_chunk(mask_lines, w_min_um, w_max_um, *, v_min_m_s, v_max_m_s, zb=0.0)[source]
Keep lines whose rest centers can Doppler-shift into
[w_min_um, w_max_um]for some velocity in[v_min_m_s, v_max_m_s](m/s) with denominator(1+zb).
- stellar_rv_ccf_mask_lines(*, mask_path, mask_wavelength=None, mask_weight=None)[source]
Resolve mask lines for line-mask CCF (path, default package mask, or in-memory arrays).