This study reports our initial work on multimodal sensory representation for object classification. To form a sensory representation we used the spatial pooling phase of the Hierarchical Temporal Memory - a Neocortically-inspired algorithm. The classification task was carried out on the Washington RGB-D dataset in which the employed method provides extraction of non-hand engineered representations (or features) from different modalities which are pixel values (RGB) and depth (D) information. These representations, both early and lately fused, were used as inputs to a machine learning algorithm to perform object classification. The obtained results show that using multimodal representations significantly improve (by 5 %) the classification performance compared to a when a single modality is used. The results also indicate that the performed method is effective for multimodal learning and different sensory modalities are complementary for the object classification. Therefore, we envision that this method can be employed for object concept formation that requires multiple sensory information to execute cognitive tasks.

Multimodal sensory representation for object classification via neocortically-inspired algorithm

Kirtay M.;Vannucci L.;Albanese U.;Falotico E.;Laschi C.
2018

Abstract

This study reports our initial work on multimodal sensory representation for object classification. To form a sensory representation we used the spatial pooling phase of the Hierarchical Temporal Memory - a Neocortically-inspired algorithm. The classification task was carried out on the Washington RGB-D dataset in which the employed method provides extraction of non-hand engineered representations (or features) from different modalities which are pixel values (RGB) and depth (D) information. These representations, both early and lately fused, were used as inputs to a machine learning algorithm to perform object classification. The obtained results show that using multimodal representations significantly improve (by 5 %) the classification performance compared to a when a single modality is used. The results also indicate that the performed method is effective for multimodal learning and different sensory modalities are complementary for the object classification. Therefore, we envision that this method can be employed for object concept formation that requires multiple sensory information to execute cognitive tasks.
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: http://hdl.handle.net/11382/530004
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
social impact