nn.Dropout随机丢神经元的用法-Toy模板网

这篇具有很好参考价值的文章主要介绍了nn.Dropout随机丢神经元的用法。希望对大家有所帮助。如果存在错误或未考虑完全的地方，请大家不吝赐教，您也可以点击"举报违法"按钮提交疑问。

前言：

pytorch与tensorflow中均有nn.dropout,两者之间的使用方法，下面将介绍。

一、torch.nn.dropout

说明文档：

r"""During training, randomly zeroes some of the elements of the input
tensor with probability :attr:`p` using samples from a Bernoulli
distribution. Each channel will be zeroed out independently on every forward
call.

This has proven to be an effective technique for regularization and
preventing the co-adaptation of neurons as described in the paper
`Improving neural networks by preventing co-adaptation of feature
detectors`_ .

Furthermore, the outputs are scaled by a factor of :math:`\frac{1}{1-p}` during
training. This means that during evaluation the module simply computes an
identity function.

大致的翻译：

在训练期间，随机地将输入的一些元素归零，以概率为`p`，使用伯努利分布的样本。每个通道将在每次前向调用时被独立清零。

这已被证明是一种有效的正则化技术，可以防止神经元的共同适应，如论文中所述 "Improving neural networks by preventing co-adaptation of feature detectors"。

此外，训练过程中，输出的比例为:math:`frac{1}{1-p}`【a1*(1-p)=a】。这意味着，在评估过程中，该模块只需计算一个识别函数。

1.1对于一维度向量的dropout

import torch
a = torch.randn(10)
p = 0.3 # probability of an element to be zeroed
Dropout = torch.nn.Dropout(p)
a1 = Dropout(a)
print(a)
print(a1)

程序输出结果为

nn.dropout,pytorch,python,源码简介-pytorch,深度学习,人工智能,python

由上图可以知道,：

0.p的作用：元素被清零的概率

1.红色框框部分：10个元素，其中有三个被置零，概率为0.3

2.紫色框框的元素：output = input/(1-p)

1.2对于二维卷积特征图的操作

import torch
a = torch.randn([2,2,3,4])
p = 0.3 # probability of an element to be zeroed
Dropout = torch.nn.Dropout(p)
a1 = Dropout(a)
print(a)
print(a1)

程序的输入：batch_size = 2, channel=2,w=3,h=4=>[2,2,3,4]

输出结果如图所示:

nn.dropout,pytorch,python,源码简介-pytorch,深度学习,人工智能,python