Data preprocessing is a main step in data mining because real data can be corrupted for different causes and high performance data mining systems require high quality data. When a database is used for training a neural network, a fuzzy system or a neuro-fuzzy system, a suitable data selection and pre-processing stage can be very useful in order to obtain a reliable result. For instance, when the final aim of a system trained through a supervised learning procedure is to approximate an existing functional relationship between input and output variables, the database that is exploited in the system training phase should not contain input-output patterns for which the same input or similar input sets are associated to very different values of the output variable. In this paper a procedure is proposed for detecting non-coherent associations between input and output patterns: by comparing two distance matrices associated to the input and output patterns, the elements of the available dataset, where similar values of input variables are associated to quite different output values can be pointed out. The efficiency of the proposed algorithm when pre-processing data coming from an industrial database is presented and discussed.

A procedure for the detection of anomalous input-output patterns

MATARESE, NICOLA;COLLA, Valentina;VANNUCCI, Marco;
2013-01-01

Abstract

Data preprocessing is a main step in data mining because real data can be corrupted for different causes and high performance data mining systems require high quality data. When a database is used for training a neural network, a fuzzy system or a neuro-fuzzy system, a suitable data selection and pre-processing stage can be very useful in order to obtain a reliable result. For instance, when the final aim of a system trained through a supervised learning procedure is to approximate an existing functional relationship between input and output variables, the database that is exploited in the system training phase should not contain input-output patterns for which the same input or similar input sets are associated to very different values of the output variable. In this paper a procedure is proposed for detecting non-coherent associations between input and output patterns: by comparing two distance matrices associated to the input and output patterns, the elements of the available dataset, where similar values of input variables are associated to quite different output values can be pointed out. The efficiency of the proposed algorithm when pre-processing data coming from an industrial database is presented and discussed.
2013
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11382/365427
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
social impact