Towards better utilization of pseudo labels for weakly supervised temporal action localization
Abstract
Weakly supervised temporal action localization (WS-TAL) aims to simultaneously recognize and localize action instances of interest in untrimmed videos with the use of the video-level label only. Some works have demonstrated that pseudo labels play an important role for performance improvement in WS-TAL. Since pseudo labels are inevitably inaccurate, direct adoption of noisy labels can lead to inappropriate knowledge transfer. Although some previous studies have shown the benefits of using only “reliable” pseudo labels, performance improvement is still limited. In this work, we experimentally analyze how the noise in pseudo labels affects model performance within the self-distillation framework. Motivated by the finding that incorrect pseudo labels with large confidence scores have a significant impact on performance, we propose the overconfidence suppression (OCS) strategy to mitigate the effect of the overconfident pseudo labels, and thus prevent over-fitting of the student model. In addition, a simplified contrast learning method is utilized to fine-tune the feature representation by increasing the separation of the foreground and background snippets. Equipped with the proposed methods, the benefits of pseudo labels can be better exploited and allow the model to achieve state-of-the-art performance on THUMOS’14 and ActivityNet-1.2 benchmarks.
Kaitai Guo
Assistant Professor
My research interests include broad-spectrum substance identification, microwave and infrared imaging, and system simulation and evaluation.
Yang Zheng
Assistant Professor
My research interests include human behaviour analysis for intelligent diagnosis of developmental coordination disorder, aritifical intelligence, and computer vision.
Haihong Hu
Associate Professor
My research interests include signal and image processing and computer vision.
Jimin Liang
Professor of Electronic Engineering
My research interests include artificial intelligence and computer vision.