This README file was generated on 2026-06-10 by Kabir Siddiqi.

# README: Representative Nightly Optical SSA Data Product Archives

## General Information

**Title of dataset:** Nightly optical survey catalogues for resident space objects above LEO

**Associated publication:**  
Siddiqi et al., *On-sky performance of a scalable, wide-field optical survey node for persistent real-time space situational awareness above low Earth orbit*.

**Dataset summary:**  
This dataset contains representative processed nightly outputs from a low-cost, fixed-field, wide-field optical survey node for space situational awareness (SSA) above low Earth orbit. The data products are per-object catalogues of identified and unidentified resident space object (RSO) detections in medium Earth orbit (MEO), geosynchronous Earth orbit (GEO), and related high-altitude regimes. Each catalogue includes astrometric and photometric measurements, uncertainties, identifiers where available, confidence metrics, and supporting metadata.

## Contact Information

### Corresponding author / principal contact

**Name:** Kabir Siddiqi  
**ORCID:** 0009-0004-2214-0351  
**Institution:** Department of Physics, Durham University  
**Address:** Durham DH1 3LE, United Kingdom  
**Email:** kabir.i.siddiqi@durham.ac.uk

### Co-author / alternate contact

**Name:** James Osborn  
**Institution:** Department of Physics, Durham University  
**Address:** Durham DH1 3LE, United Kingdom  
**Email:** james.osborn@durham.ac.uk

### Additional authors

Mark Swinbank, Kathryn Barrett, Jason Anderson, Phineas Allen, and Amber Sedgley.  
Affiliations include the Department of Physics, Durham University, and the Department of Physics, University of Warwick.

## Collection Information

**Dates of data collection:** 2025-02-04, 2025-02-05, 2026-04-12, 2026-04-24, 2026-04-28, 2026-04-29, 2026-04-30, 2026-05-24, 2026-05-25.

**Geographic location of data collection:**  
Department of Physics, Durham University, Durham, United Kingdom.

**Instrument and observing setup:**  
The observations were collected using a ZWO ASI6200MM-Pro CMOS camera paired with a William Optics RedCat 71 APO telescope. The system has a pixel scale of approximately 2.21 arcsec/pixel and a field of view of approximately 5.9 deg × 3.9 deg. The instrument is housed in a 7 ft Astrohaven clamshell dome and operated on a fixed altitude-azimuth platform for persistent monitoring of a fixed regions of the visible GEO belt. The Durham site has a median seeing of approximately 3.5 arcsec.

**Observing sequence:**  
The system uses a repeating sequence of one 0.5 s calibration exposure followed by three 8 s science exposures. Calibration images are used for astrometric and photometric calibration. Science images are processed to detect candidate RSOs.


## Sharing and Access Information

**Licence / restrictions:**  
Creative Commons Attribution 4.0 International (CC BY)

**Recommended dataset citation:**  
[to add once repository DOI, archive URL, or publication details are finalised]

## Data and File Overview

The dataset is organised as one archive per observing night, for example:

```text
YY-MM-DD_outputs.zip
```

Each nightly archive contains processed per-object catalogues and a TLE text file.

### Files contained in each nightly archive

| File pattern | Description |
|---|---|
| `norad_xxxxx.csv` | Per-object catalogue for an identified object assigned a NORAD ID `xxxxx`. Each row is one detection of that object and contains measurements, uncertainties, identifiers, confidence metrics, and metadata. |
| `track_xx.csv` | Per-object catalogue for an unidentified extracted track assigned an internal track identifier `xx`. Each row is one detection of that object and contains measurements, uncertainties, identifiers, confidence metrics, and metadata. | These tracks were not confidently identified with a NORAD ID but were retained as coherent RSO-like tracks. |
| `yy-mm-dd_tles_identified.txt` | Text file containing the TLE list for objects predicted to enter the field of view. These TLEs were used for identification and for producing the CSV data products. |

### Relationship between files

Each CSV file corresponds to one object from one observing night. Each row in a CSV corresponds to one identified detection of that object in one science frame. Identified objects are saved using the `norad_xxxxx.csv` naming pattern. Unidentified but extracted tracks are saved using the `track_xxx.csv` naming pattern. The TLE text file records the orbital-element set used to predict objects entering the field of view and to associate detections with known NORAD objects.

### Data not included

The released archives are representative processed system outputs. They are intended to support interpretation and reuse of the methods and results in the associated publication. They are not intended to provide the complete raw image archive.

### Dataset versions

1

## Methodological Information

### Collection and generation of data

The instrument repeatedly observes a fixed wide field near the visible GEO belt from Durham. Candidate RSOs include point-like geostationary objects and streak-like moving objects in MEO, GEO, and related high-altitude or transfer regimes.

### Processing workflow

The processing pipeline operates in real time alongside image acquisition. The main stages are:

1. Source detection in calibration images using Source Extractor.
2. Astrometric plate solving of calibration images using Astrometry.net.
3. Photometric calibration using cross-matches with REFCAT2.
4. Source detection in science images.
5. Star masking using predicted stellar positions, leaving non-sidereal candidate RSO detections.
6. Centroid, flux, magnitude, SNR, and uncertainty measurement for candidate RSOs.
7. Fragment merging for split streak detections where appropriate.
8. TLE-based NORAD identification using predictions from Space-Track TLEs.
9. Geometric tracklet extraction for detections not matched with TLEs.
10. End-of-night post-processing to consolidate detections into final per-object catalogues.

### Software and services used

- Source Extractor / SExtractor: source detection and measurement.
- Astrometry.net: astrometric plate solving and WCS generation.
- REFCAT2: stellar reference catalogue for astrometric and photometric calibration.
- Space-Track: TLE source for object prediction.
- Python-based pipeline tools for coordinate conversion, star masking, identificaiton, tracklet extraction, post-processing, and catalogue generation.

## CSV Catalogue Format

Each CSV file is a per-object catalogue. Each row corresponds to one detection of that object in one science frame and contains the measured astrometric, photometric, identification, uncertainty, and processing metadata for that detection.

**Number of variables:** 30-31  
**Number of rows:** varies by object and observing duration.  
**Missing values:** blank, `NaN`, or equivalent empty values may appear where a quantity is not applicable. This is most common for unidentified `track_xxx.csv` files, which do not have a NORAD/TLE identification.

### Column descriptions

| Column | Description |
|---|---|
| `TIME_JD` | Start-exposure time of the detection, expressed as Julian Date. |
| `CONFIDENCE` | Per-detection heuristic confidence score assigned by the processing pipeline, ranging from 0 to 1, where larger values indicate higher confidence. |
| `TRACK_CONFIDENCE_LEVEL` | Overall confidence class for the final object track: 0 = unlikely, 1 = possible, 2 = likely. |
| `MATCH_NORAD` | NORAD ID identifier assigned to the detection, if it was identified. |
| `MATCH_NAME` | Object name associated with the assigned NORAD ID identifier, if available. |
| `MATCH_TLE_EPOCH_JD` | Epoch of the TLE used for the matched object, expressed as Julian Date. |
| `GLOBAL_TRACK_ID` | Internal global identifier assigned to unidentified tracks.|
| `COMBINED_ROW` | Boolean flag indicating whether the row was produced by combining fragmented detections. If `true`, the photometric quantities should generally be treated as invalid or unreliable, if not already removed. |
| `ALT_DEG` | Measured altitude of the detected object, in degrees. |
| `AZ_DEG` | Measured azimuth of the detected object, in degrees. |
| `MATCH_TLE_AZ_DEG` | TLE-predicted azimuth of the identified object at the detection time, in degrees. |
| `MATCH_TLE_ALT_DEG` | TLE-predicted altitude of the identified object at the detection time, in degrees. |
| `MATCH_SEP_R_DEG` | Angular separation between the measured position and the matched TLE prediction, in degrees. |
| `CENTROID_ERR_ARCSEC` | Centroiding uncertainty from the image measurement, in arcseconds, derived from SExtractor centroiding. |
| `ANG_ERR_ARCSEC` | Total angular positional uncertainty for the detection, in arcseconds. |
| `CENTER_ALT_DEG` | Altitude of the image centre associated with the detection, in degrees. |
| `CENTER_AZ_DEG` | Azimuth of the image centre associated with the detection, in degrees. |
| `ZPT` | Photometric zero-point used to convert measured fluxes to calibrated magnitudes, derived from photometric calibration against REFCAT2. |
| `ZPTSTERR` | Standard error of the photometric zero-point. |
| `FLUX_APER` | Source flux measured using the fixed-aperture method, in instrumental flux units. |
| `FLUXERR_APER` | Uncertainty on `FLUX_APER`, in instrumental flux units. |
| `MAG_APER` | Calibrated aperture magnitude derived from `FLUX_APER`. |
| `MAGERR_APER` | Uncertainty on `MAG_APER`, in magnitudes. |
| `SNR_APER` | Signal-to-noise ratio of the fixed-aperture flux measurement. |
| `FLUX_ISOCOR` | Corrected isophotal source flux, in instrumental flux units. |
| `FLUXERR_ISOCOR` | Uncertainty on `FLUX_ISOCOR`, in instrumental flux units. |
| `MAG_ISOCOR` | Calibrated magnitude derived from `FLUX_ISOCOR`. |
| `MAGERR_ISOCOR` | Uncertainty on `MAG_ISOCOR`, in magnitudes. |
| `SNR_ISOCOR` | Signal-to-noise ratio of the corrected isophotal flux measurement. |
| `ELLIPTICITY` | Ellipticity of the detected source image, derived from image moments. |
| `THETA_IMAGE` | Orientation angle of the detected source image in image coordinates, in degrees. A value of 0 corresponds approximately to the east-west image direction. |
| `AST_RMS` | Astrometric RMS fit error from the relevant plate-solving/WCS solution, in arcseconds. |

## Notes for Users

- `norad_xxxxx.csv` files correspond to identified objects with a NORAD/TLE identification.
- `track_xxx.csv` files correspond to extracted unidentified tracks. These are retained as RSO detections, but they were not confidently identified with a TLE during processing.
- For most analyses, users should apply cuts on `CONFIDENCE` and `TRACK_CONFIDENCE_LEVEL` appropriate to their use case. As a general starting point, we suggest `CONFIDENCE >= 0.8` and `TRACK_CONFIDENCE_LEVEL > 0`, which retains tracks classified as possible or likely.
- Rows with `COMBINED_ROW = true` should be handled carefully. These rows are produced by merging fragmented detections, so their photometric quantities are generally not reliable, if not already removed.
- Magnitudes and fluxes should be interpreted with the measurement type in mind. Aperture quantities are usually more suitable for point-like and short-streak sources, while corrected isophotal quantities may be more appropriate for extended or streaked detections, this can be deduced by the detections `ELLIPTICITY`.
