SegmentCodeList: Unsupervised Representation Learning for Human Skeleton Data Retrieval

Investor logo

Warning

This publication doesn't include Faculty of Economics and Administration. It includes Faculty of Informatics. Official publication website can be found on muni.cz.
Authors

SEDMIDUBSKÝ Jan CARRARA Fabio AMATO Giuseppe

Year of publication 2023
Type Article in Proceedings
Conference 45th European Conference on Information Retrieval (ECIR)
MU Faculty or unit

Faculty of Informatics

Citation
Doi http://dx.doi.org/10.1007/978-3-031-28238-6_8
Keywords 3D skeleton sequence;segment similarity;unsupervised feature learning;Variational AutoEncoder;segment code list;action retrieval
Description Recent progress in pose-estimation methods enables the extraction of sufficiently-precise 3D human skeleton data from ordinary videos, which offers great opportunities for a wide range of applications. However, such spatio-temporal data are typically extracted in the form of a continuous skeleton sequence without any information about semantic segmentation or annotation. To make the extracted data reusable for further processing, there is a need to access them based on their content. In this paper, we introduce a universal retrieval approach that compares any two skeleton sequences based on temporal order and similarities of their underlying segments. The similarity of segments is determined by their content-preserving low-dimensional code representation that is learned using the Variational AutoEncoder principle in an unsupervised way. The quality of the proposed representation is validated in retrieval and classification scenarios; our proposal outperforms the state-of-the-art approaches in effectiveness and reaches speed-ups up to 64x on common skeleton sequence datasets.
Related projects:

You are running an old browser version. We recommend updating your browser to its latest version.