hispecdrp.calibrations.koa_sync

Helpers to sync KOA calibration metadata and download missing cache files.

Functions

_discover_koa_cache_sources(dest, *[, extra])

Candidate seed caches, richest first (by FITS count), excluding dest.

_is_recoverable_remote_error(exc)

_koa_cache_fit_count(cache_dir)

_koa_cache_has_db(cache_dir)

_verify_koa_calibrations_once([...])

Single-pass KOA verify (no local fallback).

ensure_koa_calibrations([spectrograph, ...])

Seed the local KOA cache if thin, then verify/download calibrations.

seed_koa_calibration_cache([dest, sources, ...])

Copy a populated KOA cache into dest when the destination is empty/thin.

verify_koa_calibrations([spectrograph, ...])

Sync KOA catalog metadata and download missing calibration files for one spectrograph.

ensure_koa_calibrations(spectrograph='BSPEC', *, cache_dir=None, seed_sources=None, ignore_missing_cal_types=None, require_complete=True, connect_remote=True)[source]

Seed the local KOA cache if thin, then verify/download calibrations.

Tries a remote sync first. On auth / network / API failures, falls back to the local cache only (after seeding from another cache when needed).

seed_koa_calibration_cache(dest=None, *, sources=None, min_fits=1, force=False)[source]

Copy a populated KOA cache into dest when the destination is empty/thin.

Parameters:
  • dest – Target KOA_CALIBRATION_CACHE directory. Defaults to the current KOA_CALIBRATION_CACHE environment variable.

  • sources – Optional explicit seed directories (tried first). Otherwise discovers ~/hispec*/cals/KOA_CALIBRATION_CACHE and HISPEC_KOA_CACHE_SEED.

  • min_fits – Destination is considered populated once it has at least this many FITS files under calibrations/.

  • force – If True, replace the destination from the richest source even when dest already meets min_fits.

Returns:

dict – dest, seeded, source, n_fits_before, n_fits_after, candidates.

verify_koa_calibrations(spectrograph='BSPEC', *, connect_remote=True, sync_remote=True, download_missing=True, require_complete=True, ignore_missing_cal_types=None, fallback_local=True)[source]

Sync KOA catalog metadata and download missing calibration files for one spectrograph.

Uses KOA_CALIBRATION_CACHE for the local cache. When connect_remote / sync_remote are True, also requires KECK_OBSERVER_EMAIL and KECK_OBSERVER_PASSWORD for remote KOA access.

Each download is verified on disk (exists, non-trivial size, expected basename) before it is counted as successful. Metadata sync alone is not treated as a successful file download.

Parameters:
  • spectrograph (str, optional) – Spectrograph to retain / download ('BSPEC' or 'RSPEC'). Default 'BSPEC'.

  • connect_remote (bool, optional) – Open a remote KOA connection. Default True.

  • sync_remote (bool, optional) – Sync remote catalog IDs into the local SQLite DB. Default True.

  • download_missing (bool, optional) – Download calibrations listed in the local DB but missing from the cache, filtered to spectrograph. Default True.

  • require_complete (bool, optional) – If True (default), raise RuntimeError when any download fails or files remain missing after the pass. Set False to only return the summary.

  • ignore_missing_cal_types (list of str or None, optional) – Calibration types that may be absent from the cache without failing require_complete (and are skipped for download attempts). Useful for simulation notebooks that register a packaged lfc_wavesol instead of the remote geometric product. Example: ["lfc_wavesol"].

  • fallback_local (bool, optional) – If True (default) and a remote connect/sync fails with a recoverable auth/network/API error, retry once with connect_remote=False and sync_remote=False so notebooks need no try/except boilerplate.

Returns:

dict – Summary with keys spectrograph, n_records, cal_types, n_missing, n_downloaded, n_failed, downloaded, failed, verified_paths, still_missing, ignored_missing. On local fallback, also remote_mode='local_fallback' and remote_error.