Research Papers Classification System

From statwiki
Revision as of 18:02, 24 November 2020 by Ck6li (talk | contribs) (Created page with "== Presented by == Jill Wang, Junyi Yang, Yu Min Wu, Chun Kit (Calvin) Li == Introduction == This paper introduces a paper classification system that utilizes the Term freque...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Presented by

Jill Wang, Junyi Yang, Yu Min Wu, Chun Kit (Calvin) Li

Introduction

This paper introduces a paper classification system that utilizes the Term frequency-inverse document frequency (TF-IDF), Latent Dirichlet Allocation (LDA), and K-means clustering. The most important technology the system used to process big data is the Hadoop Distributed File Systems (HDFS). The system can handle quantitatively complex research paper classification problems efficiently and accurately.

General Framework


Data Preprocessing

Crawling of Abstract Data Managing Paper Data

Topic Modeling Using LDA

Term Frequency Inverse Document Frequency (TF-IDF) Calculation

Term Frequency (TF)

Document Frequency (DF)

Inverse Document Frequency (IDF)


Paper Classification Using K-means Clustering

System Testing Results

Conclusion

Critique

Reference