loss function

损失函数的基本用法：

criterion = LossCriterion() # 构造函数有自己的参数
loss = criterion(x, y) # 调用标准时也有参数

得到的loss结果已经对mini-batch数量取了平均值

注：

reduction ( string , optional ) – Specifies the reduction to apply to the output: 'none' | 'mean' | 'sum'. 'none': no reduction will be applied, 'mean': the sum of the output will be divided by the number of elements in the output, 'sum': the output will be summed. Note: size_average and reduce are in the process of being deprecated, and in the meantime, specifying either of those two args will override reduction. Default: 'mean'

weight (Tensor, optional) – a manual rescaling weight given to each class. If given, it has to be a Tensor of size C. Otherwise, it is treated as if having all ones.

L1Loss

创建一个criterion计算input x和target y的每个元素的平均绝对误差(mean absolute error (MAE))

$$loss(x_n,y_n) = \sum_{n=1}^N | x_n - y_n |$$

from torch import nn

loss = nn.L1Loss()
# Input: (N, *)where ∗ means, any number of additional dimensions
input = torch.randn(3, 5, requires_grad=True)
# Target: (N, *), same shape as the input
target = torch.randn(3, 5)
# Output: scalar. If reduction is 'none', then (N,∗), same shape as the input
output = loss(input, target)
output.backward()

MSELoss

创建一个criterion计算input x和target y的每个元素的均方误差(mean absolute error (MAE))

torch.nn.MSELoss(size_average=None, reduce=None, reduction='mean')

$$loss(x_n,y_n) = \sum_{n=1}^N (x_n - y_n)^2$$

from torch import nn

loss = nn.MSELoss()
# Input: (N, *)where ∗ means, any number of additional dimensions
input = torch.randn(3, 5, requires_grad=True)
# Target: (N, *), same shape as the input
target = torch.randn(3, 5)
# Output: scalar. If reduction is 'none', then (N,∗), same shape as the input
output = loss(input, target)
output.backward()

CrossEntropyLoss

该criterion将nn.LogSoftmax()和nn.NLLLoss()方法结合到一个类中

torch.nn.CrossEntropyLoss(weight=None, size_average=None, ignore_index=-100, reduce=None, reduction='mean')

$$loss(x,class) = -log(\frac {e^{x[class]}} {\sum_j e^{x[j]}})$$

from torch import nn

loss = nn.CrossEntropyLoss()
# Input: (N, C) where C = number of classes
input = torch.randn(3, 5, requires_grad=True)
# Target: (N) where each value is 0 <= targets[i] <= C-1
target = torch.empty(3, dtype=torch.long).random_(5)
# Output: scalar. If reduction is 'none', then the same size as the target: (N)
output = loss(input, target)
output.backward()

NLLLoss

用于多分类的负对数似然损失函数(negative log likelihood loss)

torch.nn.NLLLoss(weight=None, size_average=None, ignore_index=-100, reduce=None, reduction='mean')

from torch import nn

m = nn.LogSoftmax(dim=1)
loss = nn.NLLLoss()
# input is of size N x C = 3 x 5
input = torch.randn(3, 5, requires_grad=True)
# each element in target has to have 0 <= value < C
target = torch.tensor([1, 0, 4])
output = loss(m(input), target)
output.backward()

# 2D loss example (used, for example, with image inputs)
N, C = 5, 4
loss = nn.NLLLoss()
# input is of size N x C x height x width
data = torch.randn(N, 16, 10, 10)
conv = nn.Conv2d(16, C, (3, 3))
m = nn.LogSoftmax(dim=1)
# each element in target has to have 0 <= value < C
target = torch.empty(N, 8, 8, dtype=torch.long).random_(0, C)
output = loss(m(conv(data)), target)
output.backward()

BCELoss

创建一个衡量目标和输出之间二进制交叉熵的criterion

torch.nn.BCELoss(weight=None, size_average=None, reduce=None, reduction='mean')

$$loss(x_n,y_n) = -w_n[y_n \cdot logx_n + (1-y_n) \cdot log(1-x_n)]$$

from torch import nn

m = nn.Sigmoid()
loss = nn.BCELoss()
# Input: (N, *) where ∗ means, any number of additional dimensions
input = torch.randn(3, requires_grad=True)
# Target: (N, *) , same shape as the input
target = torch.empty(3).random_(2)
# Output: scalar. If reduction is 'none', then (N, *) , same shape as input.
output = loss(m(input), target)
output.backward()

BCEWithLogitsLoss

将sigmoid函数和BCELoss方法结合到一个类中

这个版本在数值上比使用一个带着BCELoss损失函数的简单的Sigmoid函数更稳定，通过将操作合并到一层中，我们利用log-sum-exp技巧来实现数值稳定性。

torch.nn.BCEWithLogitsLoss(weight=None, size_average=None, reduce=None, reduction='mean', pos_weight=None)

多出参数：

• pos_weight (Tensor,**可选) –正值例子的权重，必须是有着与分类数目相同的长度的向量

from torch import nn

loss = nn.BCEWithLogitsLoss()
# Input: (N, *) where ∗ means, any number of additional dimensions
input = torch.randn(3,requires_grad=True)
# Target: (N, *) , same shape as the input
target = torch.empty(3).random_(2)
# Output: scalar. If reduction is 'none', then (N, *) , same shape as input.
output = loss(input, target)
output.backward()

参考

https://pytorch.org/docs/stable/index.html