2018 November 26

Location: Coors Tech

Attending:
Andy Antoine Thomas Iga Jihyun Bane Hayden

TODO

  • Post theorem relating sparsity, DoF, and latent space (compressed dimension) (Thomas)
  • Book Coors Tech 381 for Spring semester (Andy)
  • Send Andy KL-divergence ICA paper (Thomas)

Key takeaways

  • We now have basic understanding on autoencoders and variational autoencoders.

Problems

  • Bane has a high dimensional dataset - reduces dimensionality using PCA.
  • How to pick number of dimensions to keep after PCA? Consider looking at the explained variance of each principle component, only keeping components that explain most of the variance will lead to lower reconstruction error.
  • Need multiple class classification capabilities. Consider a multivariate regression approach.

Questions

  • Why use Leaky ReLu? Because it allows values below zero to be outputted. Also, there is no gradient for ReLu inputs below zero (maybe not a problem).
  • Why call it z_log_sigma ? Because it is in VAE tutorials and the loss function code is simpler after taking a logarithm.
  • Where does the kl_loss definition in vae_loss() come from? Look into independent component analysis (ICA) papers and try rederiving.
  • Why is total_loss variable in vae_loss() computing the mean? Typo?

Comments

  • Leaky ReLu is expensive - maybe - but could help with vanishing gradient problem - maybe.
  • Note the use of K. in the definition of loss function in VAE.
  • Note that decresing the loss function regularization pushes standard deviations to zero, making the VAE more deterministic.

Topics of discussion for next week

  • (???)

Upcoming/backup topics of discussion

  • Arnab - existing project
  • PyTorch vs. Keras

Comments