本文共 2649 字，大约阅读时间需要 8 分钟。

• Tensor.requires_grad = True 记录对Tensor的所有操作，后序.backward() 自动计算所有梯度到 .grad 属性

import torchx = torch.ones(2,2, requires_grad=True) # 默认是Falseprint(x)tensor([[1., 1.],        [1., 1.]], requires_grad=True)

• 停止记录调用.detach()

x.detach_()print(x.requires_grad) # False

• .grad_fn 保存了创建张量的 Function 的引用

x = torch.ones(2,2, requires_grad=True)y = x + 2print(y)print(y.grad_fn)tensor([[3., 3.],        [3., 3.]], grad_fn=
   
    )
    
   

z = y*y*3out = z.mean()print(z, out)tensor([[27., 27.],        [27., 27.]], grad_fn=
   
    ) tensor(27., grad_fn=
    
     )
    
   

# requires_grad 默认为 Falsea = torch.randn(2, 2)a = ((a*3)/(a-1))print(a.requires_grad)  # Falseb = (a*a).sum()print(b.grad_fn)  # Nonea.requires_grad_(True)  # 设置为 Trueprint(a.requires_grad)  # Trueb = (a*a).sum()print(b.grad_fn)# 
   

• backward() 后向传播

z = y*y*3y = x+2计算 d(out)/dx

$out=\frac{1}{4}(\sum 3(x_i+2{)}^2)\to \frac{d_{out}}{d{x}_i}=\frac{3}{2}(x_i+2)$

out.backward()print(y.grad) # None, 为什么？是 Noneprint(x.grad)tensor([[4.5000, 4.5000],        [4.5000, 4.5000]])

$$J=\left(\begin{array}{ccc}\frac{\partial y_1}{\partial x_1}& \cdots & \frac{\partial y_m}{\partial x_1}\\ ⋮& \ddots & ⋮\\ \frac{\partial y_1}{\partial x_n}& \cdots & \frac{\partial y_m}{\partial x_n}\end{array}\right)$$

• 当又使用了一个函数 $l=g(y)$，v 是 $l$ 对 $y$ 的导数，链式求导相乘，得到 $l$ 对 $x$ 的导数
  $$J\cdot v=\left(\begin{array}{ccc}\frac{\partial y_1}{\partial x_1}& \cdots & \frac{\partial y_m}{\partial x_1}\\ ⋮& \ddots & ⋮\\ \frac{\partial y_1}{\partial x_n}& \cdots & \frac{\partial y_m}{\partial x_n}\end{array}\right)\left(\begin{array}{c}\frac{\partial l}{\partial y_1}\\ ⋮\\ \frac{\partial l}{\partial y_m}\end{array}\right)=\left(\begin{array}{c}\frac{\partial l}{\partial x_1}\\ ⋮\\ \frac{\partial l}{\partial x_n}\end{array}\right)$$

上面代码改为：

v = torch.tensor(2, dtype=torch.float)out.backward(v)print(x.grad)# 梯度乘以了 2tensor([[9., 9.],        [9., 9.]])

• 评估阶段可以使用 with torch.no_grad(): 不需要梯度计算和更新

print(x.requires_grad) # Trueprint((x ** 2).requires_grad) # True# 取消梯度记录with torch.no_grad():    print((x ** 2).requires_grad) # False

转载地址：https://michael.blog.csdn.net/article/details/111657579 如侵犯您的版权，请留言回复原文章的地址，我们会给您删除此文章，给您带来不便请您谅解！