Resource-Efficient Gesture Recognition Using Low-Resolution Thermal Camera via Spiking Neural Networks and Sparse Segmentation

Ali Safa, Wout Mommen, Piet Wambacq, Lars Keuninckx

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

This work proposes a novel approach for hand gesture recognition using an inexpensive, low-resolution ( 24 x 32) thermal sensor processed by a Spiking Neural Network (SNN) followed by Sparse Segmentation and feature-based gesture classification via Robust Principal Component Analysis (R-PCA). Compared to the use of standard RGB cameras, the proposed system is insensitive to lighting variations while being significantly less expensive compared to high-frequency radars, time-of-flight cameras and high-resolution thermal sensors previously used in literature. Crucially, this paper shows that the innovative use of the recently proposed Monostable Multivibrator (MMV) neural networks as a new class of SNN achieves more than one order of magnitude smaller memory and compute complexity compared to deep learning approaches, while reaching a top gesture recognition accuracy of 93.9% using a 5-class thermal camera dataset acquired in a car cabin, within an automotive context. Our dataset is released for helping future research.
Original languageEnglish
Title of host publication2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition, FG 2024
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages5
ISBN (Electronic)9798350394948
ISBN (Print)979-8-3503-9495-5
DOIs
Publication statusPublished - 2024
Externally publishedYes
Event18th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2024 - Istanbul, Turkey
Duration: 27 May 202431 May 2024

Publication series

NameIeee International Conference On Automatic Face And Gesture Recognition And Workshops

Conference

Conference18th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2024
Country/TerritoryTurkey
CityIstanbul
Period27/05/2431/05/24

Fingerprint

Dive into the research topics of 'Resource-Efficient Gesture Recognition Using Low-Resolution Thermal Camera via Spiking Neural Networks and Sparse Segmentation'. Together they form a unique fingerprint.

Cite this