Radio Interferometry
====================

Dataset name: ``radio_interferometry``

Data and Inverse Problem
------------------------

Radio interferometers do not measure a sky image directly. Pairs of antennas
sample complex spatial frequencies called *visibilities*. Reconstructing the sky
therefore amounts to recovering an image from incomplete, non-Cartesian Fourier
measurements, with sampling weights determined by the telescope observation.

This use case downloads a FITS sky image and uses Karabo to simulate a MeerKAT
observation. The simulation produces a Measurement Set and a metadata file. At
dataset load time, the cached visibilities are converted into a DeepInv physics
operator and measurement tensor by the benchmark's dirty-imaging interface. The
FITS image is retained as ground truth, and visibility weights are passed to the
solver.

There are two distinct noise controls. ``add_noise`` belongs to the Karabo
observation simulation and therefore changes the prepared cache. ``noise_level``
adds reproducible Gaussian noise through the DeepInv physics when the benchmark
dataset is loaded.

What ``benchopt install`` Does
------------------------------

Radio interferometry provides a custom shell installer. It checks for Apptainer
or Singularity, loads the ``singularity`` module when the environment supports
modules, and pulls the Karabo image from an OCI registry if it is absent. The
image is stored as ``benchmark_inference/tools/karabo.sif``.

On Jean Zay, the installer also detects ``SINGULARITY_ALLOWED_DIR`` and ensures
that ``karabo.sif`` is present there, using the site container-copy command when
available. An existing image is reused. The registry can be overridden with the
``KARABO_IMAGE_URI`` environment variable.

What ``benchopt prepare`` Does
------------------------------

Preparation first downloads the FITS file selected by ``fits_size``, either "1024" or "10k". It then
builds a deterministic cache key from the observation parameters and checks for a
matching Measurement Set and metadata file below the dataset's
``meerkat_cache/`` directory.

If the cache is missing, preparation runs the Karabo container either directly
or in a separate SLURM job according to ``run_on_slurm``. Preparation waits for a
submitted simulation to finish and verifies that both cache files were created.
Matching completed simulations are reused.

Run preparation for every observation configuration before launching the main
benchmark, especially on clusters where compute nodes have restricted network or
container-registry access. The simulation SLURM settings below control this
preparation job; they are separate from the parallel configuration used by
``benchopt run``.

Image and Observation Parameters
--------------------------------

``fits_size`` (default ``"1024"``)
   Selects the source-image collection. Supported values are ``"1024"`` and
   ``"10k"``.

``pos_ra`` and ``pos_dec`` (defaults ``0.0``)
   Right ascension and declination used for a fixed observation position.

``random_position`` (default ``true``)
   Ask the simulator to select the sky position randomly rather than relying
   only on the fixed coordinates.

``number_of_time_steps`` (default ``64``)
   Number of temporal samples in the simulated observation.

``start_frequency_hz`` and ``end_frequency_hz``
   Frequency interval of the observation; defaults are ``1.300e9`` and
   ``1.340e9`` Hz.

``number_of_channels`` (default ``8``)
   Number of frequency channels across the interval.

``pol_mode`` (default ``"Full"``)
   Polarization mode passed to the simulator.

``use_gpus`` (default ``true``)
   Enable GPU use inside the Karabo simulation.

``add_noise`` (default ``true``)
   Enable the simulator's observation noise.

``noise_level`` (default ``0.1``)
   Standard deviation of the additional Gaussian noise applied when creating
   the DeepInv measurements.

``seed`` (default ``42``)
   Seed for the additional DeepInv noise.

Preparation-job Parameters
--------------------------

``run_on_slurm`` (default ``false``)
   Run Karabo immediately in the current allocation, or submit a dedicated
   simulation job when set to ``true``.

``slurm_folder`` (default ``debug_output/slurm_logs``)
   Submitit log directory for the simulation job.

``slurm_job_name`` (default ``karabo_simulator``)
   Name of the simulation job.

``slurm_nodes`` (default ``1``)
   Number of nodes requested for the simulation.

``slurm_ntasks_per_node`` (default ``1``)
   Number of simulation tasks started on each node.

``slurm_cpus_per_task`` (default ``40``)
   CPU cores requested for each simulation task.

``slurm_gres`` (default ``gpu:4``)
   Generic GPU resource request for the simulation.

``slurm_time`` (default ``60``)
   Wall-time value passed to Submitit for the simulation job.

``slurm_hint`` (default ``nomultithread``)
   SLURM hint applied to the simulation job.

``slurm_account`` (default ``null``)
   Optional SLURM account used for the simulation.

``slurm_constraint`` (default ``v100-32g``)
   GPU constraint requested for the simulation.

``slurm_setup`` (default loads the Singularity module)
   Commands executed before the simulation job starts.

``slurm_poll_interval_seconds`` (default ``30``)
   Interval at which preparation checks the submitted job.

``slurm_wait_timeout_seconds`` (default ``1200``)
   Maximum time preparation waits before cancelling the simulation job.

See ``benchmark_inference/configs/examples/radio_interferometry.yml`` for a
complete configuration.