Skip to Main Content
 

Global Search Box

 
 
 
 

Files

ETD Abstract Container

Abstract Header

Slim Embedding Layers for Recurrent Neural Language Models

Abstract Details

2018, Doctor of Philosophy (PhD), Wright State University, Computer Science and Engineering PhD.
Recurrent neural language (RNN) models are the state-of-the-art method for language modeling. When the vocabulary size is large, the space taken to store the model parameters becomes the bottleneck for the use of these type of models. We introduce a simple space compression method that stochastically shares the structured parameters at both the input and output embedding layers of RNN models to significantly reduce the size of model parameters, but still compactly represents the original input and the output embedding layers. The method is easy to implement and tune. Experiments on several data sets show that the new method achieves perplexity and BLEU score results comparable to the best existing methods, while only using a tiny fraction of the parameters required by other approaches.
Michael Raymer , Ph.D. (Advisor)
Shaojun Wang, Ph.D. (Advisor)
Jack Jean, Ph.D. (Committee Member)
Xinhui Zhang, Ph.D. (Committee Member)
Krishnaprasad Thirunarayan, Ph.D. (Committee Member)
75 p.

Recommended Citations

Citations

  • Li, Z. (2018). Slim Embedding Layers for Recurrent Neural Language Models [Doctoral dissertation, Wright State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=wright1531950458646138

    APA Style (7th edition)

  • Li, Zhongliang. Slim Embedding Layers for Recurrent Neural Language Models. 2018. Wright State University, Doctoral dissertation. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=wright1531950458646138.

    MLA Style (8th edition)

  • Li, Zhongliang. "Slim Embedding Layers for Recurrent Neural Language Models." Doctoral dissertation, Wright State University, 2018. http://rave.ohiolink.edu/etdc/view?acc_num=wright1531950458646138

    Chicago Manual of Style (17th edition)