参考：PyTorch 神经网络
实现下面这个网络：
- 第一层：卷积 5*5*6、ReLU、Max Pooling
- 第二层：卷积 5*5*16、ReLU、Max Pooling
- 第三层：Flatten、Linear NN
- 第四层：Linear NN
- 第五层：Linear NN
这是一个简单的前馈神经网络，它接收输入，让输入一个接着一个的通过一些层，最后给出输出。
一个典型的神经网络训练过程包括以下几点：
- 定义一个包含可训练参数的神经网络
- 迭代整个输入
- 通过神经网络处理输入
- 计算损失(loss)
- 反向传播梯度到神经网络的参数
- 更新网络的参数，典型的用一个简单的更新方法：weight = weight - learning_rate *gradient
定义神经网络：
import torch
import torch.nn as nn
import torch.nn.functional as F
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
# 1 input image channel, 6 output channels, 5x5 square convolution kernel
# 第一层
self.conv1 = nn.Conv2d(in_channels=1, out_channels=6, kernel_size=5)
# 第二层
self.conv2 = nn.Conv2d(in_channels=6, out_channels=16, kernel_size=5)
# an affine operation: y = Wx + b
# 第三层
self.fc1 = nn.Linear(in_features=16 * 5 * 5, out_features=120)
# 第四层
self.fc2 = nn.Linear(in_features=120, out_features=84)
# 第五层
self.fc3 = nn.Linear(in_features=84, out_features=10)
def forward(self, x):
# 第一层 (conv1 -> relu -> max pooling)
x = self.conv1(x)
x = F.relu(x)
# Max pooling over a (2, 2) window
x = F.max_pool2d(x, (2, 2))
# 第二层 (conv2 -> relu -> max pooling)
x = self.conv2(x)
x = F.relu(x)
# If the size is a square you can only specify a single number
x = F.max_pool2d(x, 2)
# 第三层 (fc -> relu)
x = x.view(-1, self.num_flat_features(x))
x = self.fc1(x)
x = F.relu(x)
# 第四层 (fc -> relu)
x = self.fc2(x)
x = F.relu(x)
# 第五层 (fc -> relu)
x = self.fc3(x)
x = F.relu(x)
return x
def num_flat_features(self, x):
size = x.size()[1:] # all dimensions except the batch dimension
num_features = 1
for s in size:
num_features *= s
return num_features
net = Net()
print(net)
输出：
Net( (conv1): Conv2d(1, 6, kernel_size=(5, 5), stride=(1, 1)) (conv2): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1)) (fc1): Linear(in_features=400, out_features=120, bias=True) (fc2): Linear(in_features=120, out_features=84, bias=True) (fc3): Linear(in_features=84, out_features=10, bias=True) )
在Pytorch中训练模型包括以下几个步骤:
- 在每批训练开始时初始化梯度
- 前向传播
- 反向传播
- 计算损失并更新权重
import torch.optim as optim
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)
for epoch in range(2): # loop over the dataset multiple times
running_loss = 0.0
for i, data in enumerate(trainloader, 0):
# get the inputs
inputs, labels = data
# zero the parameter gradients
optimizer.zero_grad()
# forward + backward + optimize
outputs = net(inputs)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
# print statistics
running_loss += loss.item()
if i % 2000 == 1999: # print every 2000 mini-batches
print(‘[%d, %5d] loss: %.3f‘ %
(epoch + 1, i + 1, running_loss / 2000))
running_loss = 0.0
print(‘Finished Training‘)
通用
# 在数据集上循环多次
for epoch in range(2):
for i, data in enumerate(trainloader, 0):
# 获取输入; data是列表[inputs, labels]
inputs, labels = data
# (1) 初始化梯度
optimizer.zero_grad()
# (2) 前向传播
outputs = net(inputs)
loss = criterion(outputs, labels)
# (3) 反向传播
loss.backward()
# (4) 计算损失并更新权重
optimizer.step()