Revisiting spatial dropout for regularizing convolutional neural networks

Sanghun Lee, Chulhee Lee

Research output: Contribution to journalArticlepeer-review

16 Citations (Scopus)


Overfitting is one of the most challenging problems in deep neural networks with a large number of trainable parameters. To prevent networks from overfitting, the dropout method, which is a strong regularization technique, has been widely used in fully-connected neural networks. In several state-of-the-art convolutional neural network architectures for object classification, however, dropout was partially or not even applied since its accuracy gain was relatively insignificant in most cases. Also, the batch normalization technique reduced the need for the dropout method because of its regularization effect. In this paper, we show that conventional element-wise dropout can be ineffective for convolutional layers. We found that dropout between channels in the CNNs can be functionally similar to dropout in the FCNNs, and spatial dropout can be an effective way to take advantage of the dropout technique for regularizing. To prove our points, we conducted several experiments using the CIFAR-10 and CIFAR-100 databases. For comparison, we only replaced the dropout layers with spatial dropout layers and kept all other hyperparameters and methods intact. DenseNet-BC with spatial dropout showed promising results (3.32% error rates with CIFAR-10, 3.0 M parameters) compared to other existing competitive methods.

Original languageEnglish
Pages (from-to)34195-34207
Number of pages13
JournalMultimedia Tools and Applications
Issue number45-46
Publication statusPublished - 2020 Dec

Bibliographical note

Funding Information:
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (2017R1E1A2A01079495).

Publisher Copyright:
© 2020, Springer Science+Business Media, LLC, part of Springer Nature.

All Science Journal Classification (ASJC) codes

  • Software
  • Media Technology
  • Hardware and Architecture
  • Computer Networks and Communications


Dive into the research topics of 'Revisiting spatial dropout for regularizing convolutional neural networks'. Together they form a unique fingerprint.

Cite this