TY - GEN
T1 - Resource-Efficient Gesture Recognition Using Low-Resolution Thermal Camera via Spiking Neural Networks and Sparse Segmentation
AU - Safa, Ali
AU - Mommen, Wout
AU - Wambacq, Piet
AU - Keuninckx, Lars
N1 - Publisher Copyright:
© 2024 IEEE.
PY - 2024
Y1 - 2024
N2 - This work proposes a novel approach for hand gesture recognition using an inexpensive, low-resolution ( 24 x 32) thermal sensor processed by a Spiking Neural Network (SNN) followed by Sparse Segmentation and feature-based gesture classification via Robust Principal Component Analysis (R-PCA). Compared to the use of standard RGB cameras, the proposed system is insensitive to lighting variations while being significantly less expensive compared to high-frequency radars, time-of-flight cameras and high-resolution thermal sensors previously used in literature. Crucially, this paper shows that the innovative use of the recently proposed Monostable Multivibrator (MMV) neural networks as a new class of SNN achieves more than one order of magnitude smaller memory and compute complexity compared to deep learning approaches, while reaching a top gesture recognition accuracy of 93.9% using a 5-class thermal camera dataset acquired in a car cabin, within an automotive context. Our dataset is released for helping future research.
AB - This work proposes a novel approach for hand gesture recognition using an inexpensive, low-resolution ( 24 x 32) thermal sensor processed by a Spiking Neural Network (SNN) followed by Sparse Segmentation and feature-based gesture classification via Robust Principal Component Analysis (R-PCA). Compared to the use of standard RGB cameras, the proposed system is insensitive to lighting variations while being significantly less expensive compared to high-frequency radars, time-of-flight cameras and high-resolution thermal sensors previously used in literature. Crucially, this paper shows that the innovative use of the recently proposed Monostable Multivibrator (MMV) neural networks as a new class of SNN achieves more than one order of magnitude smaller memory and compute complexity compared to deep learning approaches, while reaching a top gesture recognition accuracy of 93.9% using a 5-class thermal camera dataset acquired in a car cabin, within an automotive context. Our dataset is released for helping future research.
UR - http://www.scopus.com/inward/record.url?scp=85199485189&partnerID=8YFLogxK
U2 - 10.1109/FG59268.2024.10582024
DO - 10.1109/FG59268.2024.10582024
M3 - Conference contribution
AN - SCOPUS:85199485189
SN - 979-8-3503-9495-5
T3 - Ieee International Conference On Automatic Face And Gesture Recognition And Workshops
BT - 2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition, FG 2024
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 18th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2024
Y2 - 27 May 2024 through 31 May 2024
ER -