A comparison of dimensionality reduction techniques for text retrieval

Vishwa Vinay; Ingemar J. Cox; Ken Wood; Natasa Milic-Frayling

doi:10.1109/ICMLA.2005.2

A comparison of dimensionality reduction techniques for text retrieval

Vishwa Vinay^*, Ingemar J. Cox, Ken Wood, Natasa Milic-Frayling

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

14 Citations (Scopus)

Abstract

The growth of digital information increases the need to build better techniques for automatically storing, organizing and retrieving it. Much of this information is textual in nature and existing representation models struggle to deal with the high dimensionality of the resulting feature space. Techniques like Latent Semantic Indexing address, to some degree, the problem of high dimensionality in information retrieval. However, promising alternatives, like Random Mapping (RM), have yet to be completely studied in this context. In this paper, we show that despite the attention RM has received in other applications, in the case of text retrieval it is outperformed not only by Principal Component Analysis (PCA) and Independent Component Analysis (ICA) but also by a simple noise reduction algorithm.

Original language	English
Title of host publication	Proceedings - ICMLA 2005
Subtitle of host publication	Fourth International Conference on Machine Learning and Applications
Pages	293-298
Number of pages	6
DOIs	https://doi.org/10.1109/ICMLA.2005.2
Publication status	Published - 2005
Externally published	Yes
Event	ICMLA 2005: 4th International Conference on Machine Learning and Applications - Los Angeles, CA, United States Duration: 15 Dec 2005 → 17 Dec 2005

Publication series

Name	Proceedings - ICMLA 2005: Fourth International Conference on Machine Learning and Applications
Volume	2005

Conference

Conference	ICMLA 2005: 4th International Conference on Machine Learning and Applications
Country/Territory	United States
City	Los Angeles, CA
Period	15/12/05 → 17/12/05

Access to Document

10.1109/ICMLA.2005.2

Cite this

Vinay, V., Cox, I. J., Wood, K., & Milic-Frayling, N. (2005). A comparison of dimensionality reduction techniques for text retrieval. In Proceedings - ICMLA 2005: Fourth International Conference on Machine Learning and Applications (pp. 293-298). Article 1607465 (Proceedings - ICMLA 2005: Fourth International Conference on Machine Learning and Applications; Vol. 2005). https://doi.org/10.1109/ICMLA.2005.2

@inproceedings{c5bca2e3196b4019bc7c55b6a2949e12,

title = "A comparison of dimensionality reduction techniques for text retrieval",

abstract = "The growth of digital information increases the need to build better techniques for automatically storing, organizing and retrieving it. Much of this information is textual in nature and existing representation models struggle to deal with the high dimensionality of the resulting feature space. Techniques like Latent Semantic Indexing address, to some degree, the problem of high dimensionality in information retrieval. However, promising alternatives, like Random Mapping (RM), have yet to be completely studied in this context. In this paper, we show that despite the attention RM has received in other applications, in the case of text retrieval it is outperformed not only by Principal Component Analysis (PCA) and Independent Component Analysis (ICA) but also by a simple noise reduction algorithm.",

author = "Vishwa Vinay and Cox, {Ingemar J.} and Ken Wood and Natasa Milic-Frayling",

year = "2005",

doi = "10.1109/ICMLA.2005.2",

language = "English",

isbn = "0769524958",

series = "Proceedings - ICMLA 2005: Fourth International Conference on Machine Learning and Applications",

pages = "293--298",

booktitle = "Proceedings - ICMLA 2005",

note = "ICMLA 2005: 4th International Conference on Machine Learning and Applications ; Conference date: 15-12-2005 Through 17-12-2005",

}

Vinay, V, Cox, IJ, Wood, K & Milic-Frayling, N 2005, A comparison of dimensionality reduction techniques for text retrieval. in Proceedings - ICMLA 2005: Fourth International Conference on Machine Learning and Applications., 1607465, Proceedings - ICMLA 2005: Fourth International Conference on Machine Learning and Applications, vol. 2005, pp. 293-298, ICMLA 2005: 4th International Conference on Machine Learning and Applications, Los Angeles, CA, United States, 15/12/05. https://doi.org/10.1109/ICMLA.2005.2

A comparison of dimensionality reduction techniques for text retrieval. / Vinay, Vishwa; Cox, Ingemar J.; Wood, Ken et al.
Proceedings - ICMLA 2005: Fourth International Conference on Machine Learning and Applications. 2005. p. 293-298 1607465 (Proceedings - ICMLA 2005: Fourth International Conference on Machine Learning and Applications; Vol. 2005).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A comparison of dimensionality reduction techniques for text retrieval

AU - Vinay, Vishwa

AU - Cox, Ingemar J.

AU - Wood, Ken

AU - Milic-Frayling, Natasa

PY - 2005

Y1 - 2005

N2 - The growth of digital information increases the need to build better techniques for automatically storing, organizing and retrieving it. Much of this information is textual in nature and existing representation models struggle to deal with the high dimensionality of the resulting feature space. Techniques like Latent Semantic Indexing address, to some degree, the problem of high dimensionality in information retrieval. However, promising alternatives, like Random Mapping (RM), have yet to be completely studied in this context. In this paper, we show that despite the attention RM has received in other applications, in the case of text retrieval it is outperformed not only by Principal Component Analysis (PCA) and Independent Component Analysis (ICA) but also by a simple noise reduction algorithm.

AB - The growth of digital information increases the need to build better techniques for automatically storing, organizing and retrieving it. Much of this information is textual in nature and existing representation models struggle to deal with the high dimensionality of the resulting feature space. Techniques like Latent Semantic Indexing address, to some degree, the problem of high dimensionality in information retrieval. However, promising alternatives, like Random Mapping (RM), have yet to be completely studied in this context. In this paper, we show that despite the attention RM has received in other applications, in the case of text retrieval it is outperformed not only by Principal Component Analysis (PCA) and Independent Component Analysis (ICA) but also by a simple noise reduction algorithm.

UR - http://www.scopus.com/inward/record.url?scp=33847287398&partnerID=8YFLogxK

U2 - 10.1109/ICMLA.2005.2

DO - 10.1109/ICMLA.2005.2

M3 - Conference contribution

AN - SCOPUS:33847287398

SN - 0769524958

SN - 9780769524955

T3 - Proceedings - ICMLA 2005: Fourth International Conference on Machine Learning and Applications

SP - 293

EP - 298

BT - Proceedings - ICMLA 2005

T2 - ICMLA 2005: 4th International Conference on Machine Learning and Applications

Y2 - 15 December 2005 through 17 December 2005

ER -

A comparison of dimensionality reduction techniques for text retrieval

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this