tsta

Home / دومین کنفرانس ملی عصر انفجار تکنولوژی؛ هوش مصنوعی، تحولی در صنعت، تجارت و زنجیره تامین و دومین کنفرانس ملی علم داده در کاربردهای مهندسی

Document Clustering Using Deep Pre-trained Language Model Embeddings for Information Retrieval

Authors :

Mahdi Mohammadiha¹ Mohammad Hassan Sadreddini² Morteza Mohammadi Zanjireh³

1- International University of Imam Khomeini 2- International University of Imam Khomeini 3- International University of Imam Khomeini

Keywords :

Document Clustering،Information Retrieval،SBERT،UMAP،HDBSCAN

Abstract :

Document clustering is critical to information retrieval (IR) as it enhances user navigation, semantic organization, and exploration of large text collections. Current clustering techniques, though, are marred by poor accuracy and semantic inconsistency, with many misclassifying relevant documents as noise and using superficial textual representations. This study aims to develop a clustering pipeline that produces semantically meaningful and structurally coherent groups of documents to support more effective IR. We propose a method that combines SBERT embeddings for deep semantic representation, UMAP for structure-preserving dimensionality reduction, and HDBSCAN for flexible, density-based clustering without needing to predefine the number of clusters. Experimental evaluations on the 20 Newsgroups dataset reveal that our optimal setting with the paraphrase-mpnet-base-v2 model obtains a Silhouette Score of 0.6853, ARI of 0.7865, and NMI of 0.8186. These results illustrate the promise of embedding-based clustering methods to greatly improve the interpretability and effectiveness of IR systems on real-world text collections.