HAPACT: A Benchmark For Human-Centric
Physical Impact Localization in Movies
Loading…
Abstract
Physical reasoning about humans in videos has largely focused on everyday interactions, where contact, pressure, pose, and motion support embodied perception and contact-aware applications. Still, short and forceful properties like impact events remain relatively underexplored, despite their relevance to broader understanding of how abrupt physical events affect the human body. To fill this gap, we introduce human-centric impact localization, a new task that localizes target physical impacts temporally within a video and spatially on the human body. We further present HAPACT, a HumAn-centric imPACT dataset and benchmark built from movie clips, containing over 2.8K video-query pairs with more than 20K fine-grained human-annotated frames. We built HAPACT around viewer-grounded annotation, text-conditioned localization, and supervision at frame-level temporal and SMPL-X-based spatial granularity. To demonstrate the effectiveness of our benchmark, we conduct extensive evaluations of existing methods, employing video moment retrieval models for temporal localization and human-scene contact models for spatial localization. For the spatial task, we propose a text-conditioned baseline by reformulating localization as query-guided body-region selection. Benchmark results show that physical impact localization remains challenging for existing methods, motivating HAPACT as a dedicated benchmark to support physical impact reasoning in videos.
212
Movies
2,884
Video-Query Pairs
4,272
Moments
20,370
Annotated Frames
HAPACT (HumAn-centric imPACT) is a video dataset and benchmark providing frame-level temporal intervals and SMPL-X vertex-level annotations of body regions affected by each queried impact. Adopting the definition of impact by Gilardi and Sharf (2002), we focus on short, forceful events in which the human body is suddenly subjected to external force. Source videos were collected from the Movieclips YouTube channel, and model-generated impact candidates were manually reviewed to retain visible, reliably inferable, and physically plausible events involving a clearly defined human target. In two annotation stages, annotators labeled impact intervals at 24 FPS and the affected body regions in each frame.
In total, HAPACT contains 20,370 annotated frames from 2,884 video-query pairs. The dataset covers three representative categories: (1) human-to-human contact (e.g., grabbing), (2) human-to-environment contact (e.g., being thrown against a surface), and (3) impacts from external forces (e.g., blast explosions). More detailed dataset statistics are presented below.
Dataset Statistics
Distributions of on-body impact occurrences, temporal positions, movie genres, moment lengths, and query characteristics.
Distribution of nouns (left) and verbs (right) in the query descriptions.
Movie-genre distributions for the full dataset and each split.
Start-end positions of impact moments, normalized by clip duration, across splits.
Distribution of affected body parts on the SMPL-X mesh.
Distribution of impact occurrences on the body mesh for each split.
Each example consists of a (1) movie clip, a (2) textual query, one or more (3) temporal intervals, and (4) frame-level on-body impact labels. A visualization video for these examples is included in the supplementary material.
Since existing human-scene contact models do not incorporate text queries, we introduce a text-conditioned multimodal large language model (MLLM) baseline for spatial on-body impact localization. We reformulate the task as query-guided region selection by mapping dense SMPL-X vertex-level annotations into 18 body regions. Given a movie frame, the target impact query, and these candidate region labels, a LoRA-fine-tuned Cosmos-Reason2 model predicts the body regions physically affected by the queried impact. This provides a stronger reference while vertex-level estimation remains challenging.
We expect HAPACT to extend video-based physical reasoning beyond user-intended interactions to externally driven bodily impacts, with potential applications in body-aware video analysis and media-based haptic generation.
Supplementary Video
Acknowledgments and Disclosure
HAPACT is developed for academic research using videos sourced from the Movieclips YouTube channel. We do not own the copyright to the underlying movie footage, and all rights remain with the respective copyright holders. To support reproducibility, we provide source video identifiers, timestamps, metadata, annotations, and evaluation code, but do not redistribute the original movie footage.
BibTeX
@inproceedings{kim2026hapact,
title={HAPACT: A Benchmark For Human-Centric Physical Impact Localization in Movies},
author={Youngrae Kim and Yejin Jang and Eunho Kim and Youjin Sung and Kun-Woo Song and Sangpil Kim and Sang Ho Yoon},
booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
year={2026}
}