Skip to Main Content
Frequently Asked Questions
Submit an ETD
Global Search Box
Need Help?
Keyword Search
Participating Institutions
Advanced Search
School Logo
Files
File List
WSUThesis.pdf (379.52 KB)
ETD Abstract Container
Abstract Header
Slim Embedding Layers for Recurrent Neural Language Models
Author Info
Li, Zhongliang
Permalink:
http://rave.ohiolink.edu/etdc/view?acc_num=wright1531950458646138
Abstract Details
Year and Degree
2018, Doctor of Philosophy (PhD), Wright State University, Computer Science and Engineering PhD.
Abstract
Recurrent neural language (RNN) models are the state-of-the-art method for language modeling. When the vocabulary size is large, the space taken to store the model parameters becomes the bottleneck for the use of these type of models. We introduce a simple space compression method that stochastically shares the structured parameters at both the input and output embedding layers of RNN models to significantly reduce the size of model parameters, but still compactly represents the original input and the output embedding layers. The method is easy to implement and tune. Experiments on several data sets show that the new method achieves perplexity and BLEU score results comparable to the best existing methods, while only using a tiny fraction of the parameters required by other approaches.
Committee
Michael Raymer , Ph.D. (Advisor)
Shaojun Wang, Ph.D. (Advisor)
Jack Jean, Ph.D. (Committee Member)
Xinhui Zhang, Ph.D. (Committee Member)
Krishnaprasad Thirunarayan, Ph.D. (Committee Member)
Pages
75 p.
Subject Headings
Computer Science
Keywords
Language Modeling
;
Embedding Layers
Recommended Citations
Refworks
EndNote
RIS
Mendeley
Citations
Li, Z. (2018).
Slim Embedding Layers for Recurrent Neural Language Models
[Doctoral dissertation, Wright State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=wright1531950458646138
APA Style (7th edition)
Li, Zhongliang.
Slim Embedding Layers for Recurrent Neural Language Models.
2018. Wright State University, Doctoral dissertation.
OhioLINK Electronic Theses and Dissertations Center
, http://rave.ohiolink.edu/etdc/view?acc_num=wright1531950458646138.
MLA Style (8th edition)
Li, Zhongliang. "Slim Embedding Layers for Recurrent Neural Language Models." Doctoral dissertation, Wright State University, 2018. http://rave.ohiolink.edu/etdc/view?acc_num=wright1531950458646138
Chicago Manual of Style (17th edition)
Abstract Footer
Document number:
wright1531950458646138
Download Count:
454
Copyright Info
© 2018, all rights reserved.
This open access ETD is published by Wright State University and OhioLINK.