Skip to Main Content
 

Global Search Box

 
 
 

ETD Abstract Container

Abstract Header

Improvement and Implementation of Gumbel-Softmax VAE

Abstract Details

2022, Master of Computer Science (M.C.S.), University of Dayton, Computer Science.
Variational autoencoders (VAE) have recently become one of the most interesting developments in deep learning, as they take input data (e.g., images or text), learn its latent space, and then generate new similar and smooth data. The ability of discovering the latent space and creating new data makes VAEs powerful generative models, having applications in dimensionality reduction, data reconstruction, text automatic generation, art design, unsupervised clustering, semi-supervised classification, anomaly/outlier detection, and so on. Classic VAEs consider a Gaussian latent space, which does not allow for more complex representations. Gumbel-Softmax VAEs are an interesting extension, which provides practical solutions to implement the reparameterization trick for sampling a one-hot vector from a categorical distribution. During training, Gumbel-Softmax VAE needs to rely on softmax temperature tau, which guides the annealing process for categorical latent variables. Prior work simply decreases the temperature by a fixed factor and ignores the impact of the starting value and the active range of the temperature. We find that the temperature directly determines the performance of training. We present a novel parallel structure for VAEs, which combines two symmetric VAEs with different updating mechanics for the temperature and adjusts it at each training epoch based on the loss from these two VAEs. We show that our model offers a better performance than the original Gumbel-Softmax VAEs and can be used for data reconstruction, anomaly detection, and renovation of the imperfect with relatively lower distortion and noises.
Zhongmei Yao (Committee Chair)
Luan V Nguyen (Committee Member)
Xin Chen (Committee Member)
53 p.

Recommended Citations

Citations

  • Fangshi, Z. (2022). Improvement and Implementation of Gumbel-Softmax VAE [Master's thesis, University of Dayton]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=dayton1659485365234826

    APA Style (7th edition)

  • Fangshi, Zhou. Improvement and Implementation of Gumbel-Softmax VAE. 2022. University of Dayton, Master's thesis. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=dayton1659485365234826.

    MLA Style (8th edition)

  • Fangshi, Zhou. "Improvement and Implementation of Gumbel-Softmax VAE." Master's thesis, University of Dayton, 2022. http://rave.ohiolink.edu/etdc/view?acc_num=dayton1659485365234826

    Chicago Manual of Style (17th edition)