Deep learning from a statistical perspective

As one of the most rapidly developing artificial intelligence techniques, deep learning has been applied in various machine learning tasks and has received great attention in data science and statistics. Regardless of the complex model structure, deep neural networks can be viewed as a nonlinear and nonparametric generalization of existing statistical models. In this review, we introduce several popular deep learning models including convolutional neural networks, generative adversarial networks, recurrent neural networks, and autoencoders, with their applications in image data, sequential data and recommender systems. We review the architecture of each model and highlight their connections and differences compared with conventional statistical models. In particular, we provide a brief survey of the recent works on the unique overparameterization phenomenon, which explains the strengths and advantages of using an extremely large number of parameters in deep learning. In addition, we provide a practical guidance on optimization algorithms, hyperparameter tuning, and computing resources.

This is the peer reviewed version of the following article: [Deep learning from a statistical perspective. Stat 9, 1 (2020)], which has been published in final form at https://doi.org/10.1002/sta4.294. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions: https://authorservices.wiley.com/author-resources/Journal-Authors/licensing/self-archiving.html#3.

Files

Metadata

Work Title Deep learning from a statistical perspective
Access
Open Access
Creators
  1. Yubai Yuan
  2. Yujia Deng
  3. Yanqing Zhang
  4. Annie Qu
Keyword
  1. Autoencoder
  2. Convolutional neural network
  3. Generative adversarial network
  4. Long-short term memory
  5. Overparameterization
  6. Restricted Boltzmann machine
License In Copyright (Rights Reserved)
Work Type Article
Publisher
  1. Stat
Publication Date June 13, 2020
Publisher Identifier (DOI)
  1. https://doi.org/10.1002/sta4.294
Deposited March 14, 2023

Versions

Analytics

Collections

This resource is currently not in any collection.

Work History

Version 1
published

  • Created

    March 14, 2023 15:53 by Researcher Metadata Database

  • Added full_manuscript.pdf

    March 14, 2023 15:53 by Researcher Metadata Database

  • Added Creator Yubai Yuan

    March 14, 2023 15:53 by Researcher Metadata Database

  • Added Creator Yujia Deng

    March 14, 2023 15:53 by Researcher Metadata Database

  • Added Creator Yanqing Zhang

    March 14, 2023 15:53 by Researcher Metadata Database

  • Added Creator Annie Qu

    March 14, 2023 15:53 by Researcher Metadata Database

  • Published

    March 14, 2023 15:53 by Researcher Metadata Database

  • Updated Keyword, Description, Publication Date Show Changes

    April 06, 2023 13:23 by avs5190

    Keyword
    • Autoencoder, Convolutional neural network, Generative adversarial network, Long-short term memory, Overparameterization, Restricted Boltzmann machine
    Description
    • <p>As one of the most rapidly developing artificial intelligence techniques, deep learning has been applied in various machine learning tasks and has received great attention in data science and statistics. Regardless of the complex model structure, deep neural networks can be viewed as a nonlinear and nonparametric generalization of existing statistical models. In this review, we introduce several popular deep learning models including convolutional neural networks, generative adversarial networks, recurrent neural networks, and autoencoders, with their applications in image data, sequential data and recommender systems. We review the architecture of each model and highlight their connections and differences compared with conventional statistical models. In particular, we provide a brief survey of the recent works on the unique overparameterization phenomenon, which explains the strengths and advantages of using an extremely large number of parameters in deep learning. In addition, we provide a practical guidance on optimization algorithms, hyperparameter tuning, and computing resources.</p>
    • As one of the most rapidly developing artificial intelligence techniques, deep learning has been applied in various machine learning tasks and has received great attention in data science and statistics. Regardless of the complex model structure, deep neural networks can be viewed as a nonlinear and nonparametric generalization of existing statistical models. In this review, we introduce several popular deep learning models including convolutional neural networks, generative adversarial networks, recurrent neural networks, and autoencoders, with their applications in image data, sequential data and recommender systems. We review the architecture of each model and highlight their connections and differences compared with conventional statistical models. In particular, we provide a brief survey of the recent works on the unique overparameterization phenomenon, which explains the strengths and advantages of using an extremely large number of parameters in deep learning. In addition, we provide a practical guidance on optimization algorithms, hyperparameter tuning, and computing resources.
    Publication Date
    • 2020-01-01
    • 2020-06-13
  • Updated

    April 04, 2024 10:22 by [unknown user]