FAHES: Detecting disguised missing values

Abdulhakim Qahtan; Ahmed Elmagarmid; Mourad Ouzzani; Nan Tang

doi:10.1109/ICDE.2018.00188

FAHES: Detecting disguised missing values

Abdulhakim Qahtan, Ahmed Elmagarmid, Mourad Ouzzani, Nan Tang

Qatar Computing Research Institute

Hamad bin Khalifa University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

8 Citations (Scopus)

Abstract

It is well established that missing values, if not dealt with properly, may lead to poor data analytics models, misleading conclusions, and limitation in the generalization of findings. A key challenge in detecting these missing values is when they manifest themselves in a form that is otherwise valid, making it hard to distinguish them from other legitimate values. We propose to demonstrate FAHES, a system for detecting different types of disguised missing values (DMVs) which often occur in real world data. FAHES consists of several components, namely a profiler to generate rules for detecting repeated patterns, an outlier detection module, and a module to detect values that are used repeatedly in random records. Using several real world datasets, we will demonstrate how FAHES can easily catch DMVs.

Original language	English
Title of host publication	Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1609-1612
Number of pages	4
ISBN (Electronic)	9781538655207
DOIs	https://doi.org/10.1109/ICDE.2018.00188
Publication status	Published - 24 Oct 2018
Event	34th IEEE International Conference on Data Engineering, ICDE 2018 - Paris, France Duration: 16 Apr 2018 → 19 Apr 2018

Publication series

Name	Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018

Conference

Conference	34th IEEE International Conference on Data Engineering, ICDE 2018
Country/Territory	France
City	Paris
Period	16/04/18 → 19/04/18

Keywords

Disguised missing values

Access to Document

10.1109/ICDE.2018.00188

Cite this

Qahtan, A., Elmagarmid, A., Ouzzani, M., & Tang, N. (2018). FAHES: Detecting disguised missing values. In Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018 (pp. 1609-1612). Article 8509409 (Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDE.2018.00188

@inproceedings{f6c6819a269a4792bbf8b6e524bc6d90,

title = "FAHES: Detecting disguised missing values",

abstract = "It is well established that missing values, if not dealt with properly, may lead to poor data analytics models, misleading conclusions, and limitation in the generalization of findings. A key challenge in detecting these missing values is when they manifest themselves in a form that is otherwise valid, making it hard to distinguish them from other legitimate values. We propose to demonstrate FAHES, a system for detecting different types of disguised missing values (DMVs) which often occur in real world data. FAHES consists of several components, namely a profiler to generate rules for detecting repeated patterns, an outlier detection module, and a module to detect values that are used repeatedly in random records. Using several real world datasets, we will demonstrate how FAHES can easily catch DMVs.",

keywords = "Disguised missing values",

author = "Abdulhakim Qahtan and Ahmed Elmagarmid and Mourad Ouzzani and Nan Tang",

note = "Publisher Copyright: {\textcopyright} 2018 IEEE.; 34th IEEE International Conference on Data Engineering, ICDE 2018 ; Conference date: 16-04-2018 Through 19-04-2018",

year = "2018",

month = oct,

day = "24",

doi = "10.1109/ICDE.2018.00188",

language = "English",

series = "Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1609--1612",

booktitle = "Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018",

address = "United States",

}

Qahtan, A, Elmagarmid, A , Ouzzani, M & Tang, N 2018, FAHES: Detecting disguised missing values. in Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018., 8509409, Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018, Institute of Electrical and Electronics Engineers Inc., pp. 1609-1612, 34th IEEE International Conference on Data Engineering, ICDE 2018, Paris, France, 16/04/18. https://doi.org/10.1109/ICDE.2018.00188

FAHES: Detecting disguised missing values. / Qahtan, Abdulhakim; Elmagarmid, Ahmed ; Ouzzani, Mourad et al.
Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 1609-1612 8509409 (Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - FAHES

T2 - 34th IEEE International Conference on Data Engineering, ICDE 2018

AU - Qahtan, Abdulhakim

AU - Elmagarmid, Ahmed

AU - Ouzzani, Mourad

AU - Tang, Nan

PY - 2018/10/24

Y1 - 2018/10/24

N2 - It is well established that missing values, if not dealt with properly, may lead to poor data analytics models, misleading conclusions, and limitation in the generalization of findings. A key challenge in detecting these missing values is when they manifest themselves in a form that is otherwise valid, making it hard to distinguish them from other legitimate values. We propose to demonstrate FAHES, a system for detecting different types of disguised missing values (DMVs) which often occur in real world data. FAHES consists of several components, namely a profiler to generate rules for detecting repeated patterns, an outlier detection module, and a module to detect values that are used repeatedly in random records. Using several real world datasets, we will demonstrate how FAHES can easily catch DMVs.

AB - It is well established that missing values, if not dealt with properly, may lead to poor data analytics models, misleading conclusions, and limitation in the generalization of findings. A key challenge in detecting these missing values is when they manifest themselves in a form that is otherwise valid, making it hard to distinguish them from other legitimate values. We propose to demonstrate FAHES, a system for detecting different types of disguised missing values (DMVs) which often occur in real world data. FAHES consists of several components, namely a profiler to generate rules for detecting repeated patterns, an outlier detection module, and a module to detect values that are used repeatedly in random records. Using several real world datasets, we will demonstrate how FAHES can easily catch DMVs.

KW - Disguised missing values

UR - http://www.scopus.com/inward/record.url?scp=85051518156&partnerID=8YFLogxK

U2 - 10.1109/ICDE.2018.00188

DO - 10.1109/ICDE.2018.00188

M3 - Conference contribution

AN - SCOPUS:85051518156

T3 - Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018

SP - 1609

EP - 1612

BT - Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 16 April 2018 through 19 April 2018

ER -

FAHES: Detecting disguised missing values

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this