本文共 3036 字，大约阅读时间需要 10 分钟。

pytorch神经网络模型会自动初始化嘛？

搭好的神经网络，可以自定义初始化方法，如下方式：

from torch.nn import init#define the initial function to init the layer's parameters for the networkdef weigth_init(m):    if isinstance(m, nn.Conv2d):        init.xavier_uniform_(m.weight.data)        init.constant_(m.bias.data,0.1)    elif isinstance(m, nn.BatchNorm2d):        m.weight.data.fill_(1)        m.bias.data.zero_()    elif isinstance(m, nn.Linear):        m.weight.data.normal_(0,0.01)        m.bias.data.zero_()

当然你不初始化也可以，每一部分原生的网络模块都调用了初始化网络参数的函数，例如torch.nn.Linear的源码：

class Linear(Module):    r"""Applies a linear transformation to the incoming data: :math:`y = xA^T + b`    Args:        in_features: size of each input sample        out_features: size of each output sample        bias: If set to ``False``, the layer will not learn an additive bias.            Default: ``True``    Shape:        - Input: :math:`(N, *, H_{in})` where :math:`*` means any number of          additional dimensions and :math:`H_{in} = \text{in\_features}`        - Output: :math:`(N, *, H_{out})` where all but the last dimension          are the same shape as the input and :math:`H_{out} = \text{out\_features}`.    Attributes:        weight: the learnable weights of the module of shape            :math:`(\text{out\_features}, \text{in\_features})`. The values are            initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})`, where            :math:`k = \frac{1}{\text{in\_features}}`        bias:   the learnable bias of the module of shape :math:`(\text{out\_features})`.                If :attr:`bias` is ``True``, the values are initialized from                :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where                :math:`k = \frac{1}{\text{in\_features}}`    Examples::        >>> m = nn.Linear(20, 30)        >>> input = torch.randn(128, 20)        >>> output = m(input)        >>> print(output.size())        torch.Size([128, 30])    """    __constants__ = ['bias', 'in_features', 'out_features']    def __init__(self, in_features, out_features, bias=True):        super(Linear, self).__init__()        self.in_features = in_features        self.out_features = out_features        self.weight = Parameter(torch.Tensor(out_features, in_features))        if bias:            self.bias = Parameter(torch.Tensor(out_features))        else:            self.register_parameter('bias', None)        self.reset_parameters()    def reset_parameters(self):        init.kaiming_uniform_(self.weight, a=math.sqrt(5))        if self.bias is not None:            fan_in, _ = init._calculate_fan_in_and_fan_out(self.weight)            bound = 1 / math.sqrt(fan_in)            init.uniform_(self.bias, -bound, bound)    def forward(self, input):        return F.linear(input, self.weight, self.bias)    def extra_repr(self):        return 'in_features={}, out_features={}, bias={}'.format(            self.in_features, self.out_features, self.bias is not None        )

其中的reset_parameters就是初始化参数的方法~ 每个网络的初始化方法（采用正态分布还是均匀分布）都在pytorch官方文档里有所说明，自定义还是直接用内建初始化方法就看任务需要啦~( •̀∀•́ )~

转载地址：https://meteorrain.blog.csdn.net/article/details/104598363 如侵犯您的版权，请留言回复原文章的地址，我们会给您删除此文章，给您带来不便请您谅解！