Research Papers Classification System
Presented by
Jill Wang, Junyi Yang, Yu Min Wu, Chun Kit (Calvin) Li
Introduction
This paper introduces a paper classification system that utilizes the Term frequency-inverse document frequency (TF-IDF), Latent Dirichlet Allocation (LDA), and K-means clustering. The most important technology the system used to process big data is the Hadoop Distributed File Systems (HDFS). The system can handle quantitatively complex research paper classification problems efficiently and accurately.
General Framework
Data Preprocessing
Crawling of Abstract Data Managing Paper Data
Topic Modeling Using LDA
Term Frequency Inverse Document Frequency (TF-IDF) Calculation
Term Frequency (TF)
Document Frequency (DF)
Inverse Document Frequency (IDF)