Topic: Awesome List: deep-learning-foundation

← Back to home

1. Deep Residual Learning for Image Recognition

   Paper • Dec 10, 2015 • arxiv.org • Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun

   Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly...

2. Identity Mappings in Deep Residual Networks

   Paper • Apr 12, 2016 • arxiv.org • Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun

   Deep residual networks have emerged as a family of extremely deep architectures showing compelling accuracy and nice convergence behaviors. In this paper, we analyze the propagation formulations be...

3. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning

   Paper • Aug 23, 2016 • arxiv.org • Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, Alex Alemi

   Very deep convolutional networks have been central to the largest advances in image recognition performance in recent years. One example is the Inception architecture that has been shown to achieve...

4. Building high-level features using large scale unsupervised learning

   Paper • Jul 12, 2012 • arxiv.org • Quoc V. Le, Marc'Aurelio Ranzato, Rajat Monga, Matthieu Devin, Kai Chen, Greg S. Corrado, Jeff Dean, Andrew Y. Ng

   We consider the problem of building high-level, class-specific feature detectors from only unlabeled data. For example, is it possible to learn a face detector using only unlabeled images? To answe...

5. Auto-Encoding Variational Bayes

   Paper • Dec 10, 2022 • arxiv.org • Diederik P Kingma, Max Welling

   How can we perform efficient inference and learning in directed probabilistic models, in the presence of continuous latent variables with intractable posterior distributions, and large datasets? We...

6. DRAW: A Recurrent Neural Network For Image Generation

   Paper • May 20, 2015 • arxiv.org • Karol Gregor, Ivo Danihelka, Alex Graves, Danilo Jimenez Rezende, Daan Wierstra

   This paper introduces the Deep Recurrent Attentive Writer (DRAW) neural network architecture for image generation. DRAW networks combine a novel spatial attention mechanism that mimics the foveatio...

7. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

   Paper • Jan 7, 2016 • arxiv.org • Alec Radford, Luke Metz, Soumith Chintala

   In recent years, supervised learning with convolutional networks (CNNs) has seen huge adoption in computer vision applications. Comparatively, unsupervised learning with CNNs has received less atte...

8. Pixel Recurrent Neural Networks

   Paper • Feb 29, 2016 • arxiv.org • Aaron van den Oord, Nal Kalchbrenner, Koray Kavukcuoglu

   Modeling the distribution of natural images is a landmark problem in unsupervised learning. This task requires an image model that is at once expressive, tractable and scalable. We present a deep n...

9. Improving neural networks by preventing co-adaptation of feature detectors

   Paper • Jul 3, 2012 • arxiv.org • Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, Ruslan R. Salakhutdinov

   When a large feedforward neural network is trained on a small training set, it typically performs poorly on held-out test data. This "overfitting" is greatly reduced by randomly omitting ha...

10. Adam: A Method for Stochastic Optimization

    Paper • Jan 30, 2017 • arxiv.org • Diederik P. Kingma, Jimmy Ba

    We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. The method is straightforward to i...

11. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

    Paper • Mar 2, 2015 • arxiv.org • Sergey Ioffe, Christian Szegedy

    Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the t...

12. DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition

    Paper • Oct 6, 2013 • arxiv.org • Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, Trevor Darrell

    We evaluate whether features extracted from the activation of a deep convolutional network trained in a fully supervised fashion on a large, fixed set of object recognition tasks can be re-purposed...

13. Visualizing and Understanding Convolutional Networks

    Paper • Nov 28, 2013 • arxiv.org • Matthew D Zeiler, Rob Fergus

    Large Convolutional Network models have recently demonstrated impressive classification performance on the ImageNet benchmark. However there is no clear understanding of why they perform so well, o...

14. Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images

    Paper • Apr 2, 2015 • arxiv.org • Anh Nguyen, Jason Yosinski, Jeff Clune

    Deep neural networks (DNNs) have recently been achieving state-of-the-art performance on a variety of pattern-recognition tasks, most notably visual classification problems. Given that DNNs are now...

15. Distilling the Knowledge in a Neural Network

    Paper • Mar 9, 2015 • arxiv.org • Geoffrey Hinton, Oriol Vinyals, Jeff Dean

    A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions. Unfortunately, making...