ORCID Identifiers
Document Type
Article
Source of Publication
Journal of Big Data
Publication Date
12-1-2020
Abstract
© 2020, The Author(s). Similarity measures have long been utilized in information retrieval and machine learning domains for multi-purposes including text retrieval, text clustering, text summarization, plagiarism detection, and several other text-processing applications. However, the problem with these measures is that, until recently, there has never been one single measure recorded to be highly effective and efficient at the same time. Thus, the quest for an efficient and effective similarity measure is still an open-ended challenge. This study, in consequence, introduces a new highly-effective and time-efficient similarity measure for text clustering and classification. Furthermore, the study aims to provide a comprehensive scrutinization for seven of the most widely used similarity measures, mainly concerning their effectiveness and efficiency. Using the K-nearest neighbor algorithm (KNN) for classification, the K-means algorithm for clustering, and the bag of word (BoW) model for feature selection, all similarity measures are carefully examined in detail. The experimental evaluation has been made on two of the most popular datasets, namely, Reuters-21 and Web-KB. The obtained results confirm that the proposed set theory-based similarity measure (STB-SM), as a pre-eminent measure, outweighs all state-of-art measures significantly with regards to both effectiveness and efficiency.
DOI Link
ISSN
Publisher
Springer
Volume
7
Issue
1
Last Page
43
Disciplines
Computer Sciences
Keywords
Empirical study, Information retrieval, Similarity measures, Text classification, Text retrieval
Scopus ID
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Recommended Citation
Amer, Ali A. and Abdalla, Hassan I., "A set theory based similarity measure for text clustering and classification" (2020). All Works. 269.
https://zuscholars.zu.ac.ae/works/269
Indexed in Scopus
yes
Open Access
yes
Open Access Type
Gold: This publication is openly available in an open access journal/series