This readme file was generated on 2025-07-29 by Tanqiu Qiao GENERAL INFORMATION Title of Dataset: Geometric Visual Fusion Graph Neural Networks for Multi-Person Human-Object Interaction Recognition in Videos: MPHOI-120 [dataset] Author/Principal Investigator Information Name: Tanqiu Qiao ORCID: 0000-0002-6548-0514 Institution: Durham University Address: 18 Cutter Lane London SE10 0YD Email: qiaotq@icloud.com Author/Associate or Co-investigator Information Name: Ruochen Li ORCID: 0000-0001-8966-9613 Institution: Durham University Address: Neville Cross Durham DH1 4FL Email: ruochen.li@durham.ac.uk Date of data collection: 2023-06 --- 2023-09 Geographic location of data collection: Durham University, Durham, UK DH1 3LE Information about funding sources that supported the collection of the data: None SHARING/ACCESS INFORMATION Licenses/restrictions placed on the data: Creative Commons Attribution 4.0 International (CC BY) Links to publications that cite or use the data: http://doi.org/10.1016/j.eswa.2025.128344 Links to other publicly accessible locations of the data: No Links/relationships to ancillary data sets: No Was data derived from another source? No If yes, list source(s): Recommended citation for this dataset: Qiao, T., Li, R., Li, F.W.B., Kubotani, Y., Morishima, S. and Shum, H.P.H. (2025): Geometric visual fusion graph neural networks for multi-person human-object interaction recognition in videos. Durham University. (dataset). DOI: http://doi.org/10.15128/r26682x402f DATA & FILE OVERVIEW File List: Subject123 rgb.zip Subject137 rgb.zip Subject124 rgb.zip Subject156 rgb.zip Subject167 rgb.zip Subject234 rgb.zip Subject567 rgb.zip Subject237 rgb.zip Subject457 rgb.zip Subject456 rgb.zip visual_features.zip human_joints.zip Human_subactivity_ground_truth.zip dataset_info.json Relationship between files, if important: Additional related data collected that was not included in the current data package: No Are there multiple versions of the dataset? No If yes, name of file(s) that was updated: Why was the file updated? When was the file updated? METHODOLOGICAL INFORMATION Description of methods used for collection/generation of data: http://doi.org/10.1016/j.eswa.2025.128344 Methods for processing the data: Leveraging the Azure Kinect SDK along with the Body Tracking SDK, we acquire RGB-D videos to capture the comprehensive dynamics of multiple individual skeletons. Instrument- or software-specific information needed to interpret the data: Python 3 (or later) NumPy (v1.21.0 or later) Pandas (v1.3.0 or later) Matplotlib (v3.4.2 or later) Standards and calibration information, if appropriate: Environmental/experimental conditions: https://github.com/tanqiu98/GeoVis-GNN Describe any quality-assurance procedures performed on the data: https://github.com/tanqiu98/GeoVis-GNN People involved with sample collection, processing, analysis and/or submission: Tanqiu Qiao – Data collection, preprocessing, annotation, and submission Ruochen Li – Assistance with data processing and analysis Hubert Shum – Project supervision and guidance DATA-SPECIFIC INFORMATION FOR: Subject123 rgb Number of variables: 3 Number of cases/rows: 5,132 Variable List: Variable Name Description Unit / Values subject_id ID of the subject Integer (123) task_id Task number and name String (task1_signing, task_2_cheering, task_3_teaching, task_4_snooker) take_id ID of the take String String (take_0, take_1, take_2) Missing data codes: -1 – Data not recorded or lost Specialized formats or other abbreviations used: No DATA-SPECIFIC INFORMATION FOR: Subject124 rgb Number of variables: 3 Number of cases/rows: 5,510 Variable List: Variable Name Description Unit / Values subject_id ID of the subject Integer (124) task_id Task number and name String (task1_signing, task_2_cheering, task_3_teaching, task_4_snooker) take_id ID of the take String String (take_0, take_1, take_2) Missing data codes: -1 – Data not recorded or lost Specialized formats or other abbreviations used: No DATA-SPECIFIC INFORMATION FOR: Subject137 rgb Number of variables: 3 Number of cases/rows: 5,408 Variable List: Variable Name Description Unit / Values subject_id ID of the subject Integer (137) task_id Task number and name String (task1_signing, task_2_cheering, task_3_teaching, task_4_snooker) take_id ID of the take String String (take_0, take_1, take_2) Missing data codes: -1 – Data not recorded or lost Specialized formats or other abbreviations used: No DATA-SPECIFIC INFORMATION FOR: Subject156 rgb Number of variables: 3 Number of cases/rows: 4,872 Variable List: Variable Name Description Unit / Values subject_id ID of the subject Integer (156) task_id Task number and name String (task1_signing, task_2_cheering, task_3_teaching, task_4_snooker) take_id ID of the take String String (take_0, take_1, take_2) Missing data codes: -1 – Data not recorded or lost Specialized formats or other abbreviations used: No DATA-SPECIFIC INFORMATION FOR: Subject167 rgb Number of variables: 3 Number of cases/rows: 5,342 Variable List: Variable Name Description Unit / Values subject_id ID of the subject Integer (167) task_id Task number and name String (task1_signing, task_2_cheering, task_3_teaching, task_4_snooker) take_id ID of the take String String (take_0, take_1, take_2) Missing data codes: -1 – Data not recorded or lost Specialized formats or other abbreviations used: No DATA-SPECIFIC INFORMATION FOR: Subject234 rgb Number of variables: 3 Number of cases/rows: 5,400 Variable List: Variable Name Description Unit / Values subject_id ID of the subject Integer (234) task_id Task number and name String (task1_signing, task_2_cheering, task_3_teaching, task_4_snooker) take_id ID of the take String String (take_0, take_1, take_2) Missing data codes: -1 – Data not recorded or lost Specialized formats or other abbreviations used: No DATA-SPECIFIC INFORMATION FOR: Subject237 rgb Number of variables: 3 Number of cases/rows: 5,742 Variable List: Variable Name Description Unit / Values subject_id ID of the subject Integer (237) task_id Task number and name String (task1_signing, task_2_cheering, task_3_teaching, task_4_snooker) take_id ID of the take String String (take_0, take_1, take_2) Missing data codes: -1 – Data not recorded or lost Specialized formats or other abbreviations used: No DATA-SPECIFIC INFORMATION FOR: Subject456 rgb Number of variables: 3 Number of cases/rows: 5,179 Variable List: Variable Name Description Unit / Values subject_id ID of the subject Integer (456) task_id Task number and name String (task1_signing, task_2_cheering, task_3_teaching, task_4_snooker) take_id ID of the take String String (take_0, take_1, take_2) Missing data codes: -1 – Data not recorded or lost Specialized formats or other abbreviations used: No DATA-SPECIFIC INFORMATION FOR: Subject457 rgb Number of variables: 3 Number of cases/rows: 5,883 Variable List: Variable Name Description Unit / Values subject_id ID of the subject Integer (457) task_id Task number and name String (task1_signing, task_2_cheering, task_3_teaching, task_4_snooker) take_id ID of the take String String (take_0, take_1, take_2) Missing data codes: -1 – Data not recorded or lost Specialized formats or other abbreviations used: No DATA-SPECIFIC INFORMATION FOR: Subject567 rgb Number of variables: 3 Number of cases/rows: 5,306 Variable List: Variable Name Description Unit / Values subject_id ID of the subject Integer (567) task_id Task number and name String (task1_signing, task_2_cheering, task_3_teaching, task_4_snooker) take_id ID of the take String String (take_0, take_1, take_2) Missing data codes: -1 – Data not recorded or lost Specialized formats or other abbreviations used: No DATA-SPECIFIC INFORMATION FOR: Human_subactivity_ground_truth Number of variables: 3 Number of cases/rows: 120 Variable List: Variable Name Description Unit / Values subject_id ID of the subject Integer task_id Task number and name String (task1_signing, task_2_cheering, task_3_teaching, task_4_snooker) take_id ID of the take String String (take_0, take_1, take_2) Missing data codes: -1 – Data not recorded or lost Specialized formats or other abbreviations used: No DATA-SPECIFIC INFORMATION FOR: visual_features Number of variables: 1 Number of cases/rows: 120 Variable List: Variable Name Description Unit / Values file_name File name String (e.g. Subject123_task_1_signing_take_0.npy) Missing data codes: -1 – Data not recorded or lost Specialized formats or other abbreviations used: No DATA-SPECIFIC INFORMATION FOR: human_joints Number of variables: 1 Number of cases/rows: 120 Variable List: Variable Name Description Unit / Values file_name File name String (e.g. Subject123_task_1_signing_take_0.npy) Missing data codes: -1 – Data not recorded or lost Specialized formats or other abbreviations used: No DATA-SPECIFIC INFORMATION FOR: dataset_info Number of variables: 7 top-level keys Number of cases/rows: Not applicable (JSON metadata file) Variable List: Key Name Description Example Value or Type max_num_human Maximum number of human subjects present in any frame Integer (e.g., 3) max_num_object Maximum number of objects in any frame Integer (e.g., 5) coordinates_dimension Dimensionality of recorded coordinates Integer (2 = 2D) allowed_test_subject List of subject IDs designated for testing split List of strings (e.g., ["Subject123", ...]) object_class_name All object categories used in the dataset List of strings (e.g., "bottle", "laptop") subactivity_class_name List of all fine-grained human subactivity labels List of strings (17 total) activity_mapping Dictionary mapping each high-level activity to its objects and subactivities Nested dict with used_object and used_subactivity Missing data codes: Not applicable Specialized formats or other abbreviations used: JSON format; Used for programmatic loading of dataset configuration