hispecdrp.calibrations.hispec_store

Functions

_default_connect_remote()

Remote KOA unless HISPEC_KOA_CONNECT_REMOTE / KOA_CONNECT_REMOTE is false.

_is_recoverable_remote_init_error(exc)

Classes

HISPECCalibrationStore(*args[, ...])

class HISPECCalibrationStore(*args, instrument_name=None, connect_remote=None, **kwargs)[source]

Bases: CalibrationStore

_convert_origin(cal, origin)[source]

Convert an existing calibration file’s origin (record and file) to a new origin. This creates a new file and record. This does not modify the input file or record.

TODO: Need to generate new claibration ID.

Parameters:
  • cal (str | CalibrationModel) – The calibration file or model to copy the origin from.

  • origin (str) – The new origin string to set.

Returns:

str – The standardized origin string.

_existing_calibration_path_and_record(existing)[source]

Return (path, record) for a pre-existing cache row, or (None, None).

Upstream calibration_record_in_cache(..., mode="version-family") returns a list[dict] when multiple version-family matches exist (default include_version=False). Normalize to a single record before path lookup.

_finalize_cal_record(cal, cal_record=None, local_filepath=None)[source]

Finalize a calibration record after the file has been saved to disk.

Adds filename/MD5 via the base store, then fills KOA identifiers that are set on the saved file during generate_filename() but are not yet present on the pre-save metadata record.

Compatible with both koa_middleware call signatures:

  • _finalize_cal_record(cal, cal_record, local_filepath) (current git)

  • _finalize_cal_record(cal_record, local_filepath) (older releases)

calibration_record_in_cache(cal, mode='id')[source]

Checks if a calibration is already present in the local cache.

Parameters:
  • calibration (dict | str | DataModel) –

    Can be one of:
    • str : A calibration ID string or filepath.

    • dict : A calibration metadata dict.

    • DataModel : A calibration data model instance.

  • mode (str) –

    The mode to check the cache. Can be one of:
    • ’id’ : Check by calibration ID (cal_id), the primary key in the database.

    • ’version-family’ : Check by the version family (cal_type, datetime_obs, master_cal, spectrograph) + version (cal_version).

    • ’md5’ : Check by the MD5 checksum of the calibration file (Not Implemented yet!).

Returns:

dict | None – The calibration metadata record if found, otherwise None.

download_calibration_file(calibration)[source]

Download a remote calibration into data_dir under the DB filename.

Wraps koa_middleware so a zip that uses a different member name is moved to the expected cache path, then verifies the file exists and is non-trivial before returning.

generate_calibration_version(calibration, origin=None)[source]

Generate the next version string; accept a model or a metadata dict.

Upstream koa_middleware assigns calibration['origin'] and therefore only works with dicts. Datamodel callers (and generate_calibration_version()) pass the model itself, so convert to a record first and write the version back onto the model when applicable.

get_calibration(cal)[source]

Retrieve a calibration file and verify it is present on disk.

get_version_family_column_names(cal_type)[source]

Retrieves the column names for the version family attributes. This takes in cal_type in case we want to have different version family fields for different calibration types in the future.

Parameters:

cal_type (str) – The type of calibration (e.g. ‘dark’, ‘flat’, etc.) to get the version family fields for.

register_calibration(cal, origin=None, new_version=False)[source]

Registers a calibration to the local cache and metadata database.

Parameters:
  • cal (CalibrationModel | str) – The filepath or Calibration Datamodel object to register.

  • origin (str, optional) – The origin to register the calibration under.

  • new_version (bool, optional) – Whether to generate a new version for this calibration. If False, the method will check if a calibration with the same version family already exists in the cache and skip registration if so. Defaults to False.

Returns:

tuple[str, dict] –

A tuple containing:
  • str: The local file path where the calibration was saved.

  • dict: The calibration metadata dictionary as added to the database.

Notes

Upstream koa_middleware.CalibrationStore.register_calibration snapshots cal.to_record() before cal.save() and discards the return value of generate_calibration_version, so filename / koa_id / cal_version can be stored as None even though the FITS file is written. This override assigns those fields on the model and record before insert, and returns any existing version-family match instead of (None, None).

save_calibration_file(cal, cal_record=None)[source]

Saves a calibration file to the local cache directory.

Parameters:
  • cal (CalibrationModel) – The calibration data model instance to save.

  • cal_record (dict | None) – The corresponding record.

Returns:

str – The absolute local file path where the calibration file was saved.

select_and_get_calibration(input, selector)[source]

Selects the best calibration based on input data and a selection rule, then retrieves it.

This method uses a CalibrationSelector to identify the most appropriate calibration for the given input data. Once selected, it retrieves the calibration file, downloading it if it’s not already cached locally.

Parameters:
  • input – The input data product for which a calibration is needed.

  • selector (CalibrationSelector) – An instance of a CalibrationSelector class.

Returns:
  • tuple[str, dict] –

    • str: The local file path of the retrieved calibration file.

    • dict: The record of the selected calibration from the local database.

  • Example – >>> # Assuming my_input_data and my_selector are defined >>> local_filepath, calibration_record = store.select_and_get_calibration(my_input_data, my_selector) >>> print(f”Calibration file: {local_filepath}”) >>> print(f”Calibration ID: {calibration_record[‘id’]}”)

sync_records_from_cached_files(cals=None)[source]

Populates the local database from existing cached calibration files.

Parameters:

cals (str | CalibrationModel | Sequence[str | CalibrationModel]) – A single calibration metadata dictionary or a data model instance, or a list of these.

Notes

This method may be removed in the future if not found useful.

sync_records_from_remote(cals=None, mode='id')[source]

Synchronize the local DB with the remote KOA catalog.

cals is accepted for compatibility with callers that pass an explicit remote query result; when omitted, records are discovered via get_missing_records (same as koa_middleware).

Remote metadata failures (e.g. KOA API 404) are logged and treated as a no-op so pipelines can continue with the local caldb/cache. Callers that need a hard failure should inspect logs or sync explicitly with their own error handling.