YouTip

PyTorch torch.nn Reference Manual

---

torch.nn.Sequential is a container in PyTorch used for quickly building sequential models.

It executes modules in the order they are declared, suitable for constructing simple linear networks.

Function Definition

torch.nn.Sequential(*args)
Parameter Description:

• *args: Any number of modules that will be executed in the order passed in.

---

Usage Examples

Example 1: Basic Usage

Using Sequential to build a simple network:

Instance

import torch
import torch.nn as nn

# Create sequential container
model = nn.Sequential(
    nn.Linear(10,20),
    nn.ReLU(),
    nn.Linear(20,5)
)

# Test forward propagation
input_tensor = torch.randn(3,10)
output = model(input_tensor)

print("Model structure:")
print(model)
print("\nInput shape:", input_tensor.shape)
print("Output shape:", output.shape)

Example 2: Using OrderedDict to Name Layers

Naming each layer via OrderedDict:

Instance

import torch
import torch.nn as nn
from collections import OrderedDict

# Use OrderedDict to name layers
model = nn.Sequential(OrderedDict([
    ('fc1', nn.Linear(784,256)),
    ('relu1', nn.ReLU()),
    ('fc2', nn.Linear(256,128)),
    ('relu2', nn.ReLU()),
    ('output', nn.Linear(128,10))
]))

# Access layers by name
print("fc1 layer:", model.fc1)
print("relu1 layer:", model.relu1)

# Test
x = torch.randn(32,784)
output = model(x)
print("\nOutput shape:", output.shape)

Example 3: CNN Example

Building a convolutional neural network:

Instance

import torch
import torch.nn as nn

cnn = nn.Sequential(
    # First convolution block
    nn.Conv2d(3,32, kernel_size=3, padding=1),
    nn.BatchNorm2d(32),
    nn.ReLU(),
    nn.MaxPool2d(2,2),
    
    # Second convolution block
    nn.Conv2d(32,64, kernel_size=3, padding=1),
    nn.BatchNorm2d(64),
    nn.ReLU(),
    nn.MaxPool2d(2,2),
    
    # Flatten
    nn.Flatten(),
    nn.Linear(64 * 8 * 8,256),
    nn.ReLU(),
    nn.Dropout(0.5),
    nn.Linear(256,10)
)

# Test
x = torch.randn(4,3,32,32)
output = cnn(x)
print("CNN structure:")
print(cnn)
print("\nInput shape:", x.shape)
print("Output shape:", output.shape)

Example 4: Accessing Intermediate Layer Outputs

Access intermediate layers using forward hook:

Instance

import torch
import torch.nn as nn

# Simple network
model = nn.Sequential(
    nn.Linear(10,20),
    nn.ReLU(),
    nn.Linear(20,10)
)

# Method 1: Using hooks
feature = None
def hook_fn(module, input, output):
    global feature
    feature = output.clone()

# Register hook
model.register_forward_hook(hook_fn)

# Forward pass
x = torch.randn(1,10)
output = model(x)
print("ReLU output shape:", feature.shape)
print("ReLU output:", feature.squeeze().tolist())

---

Sequential vs Manually Defining Forward

Method	Advantages	Disadvantages
nn.Sequential	Concise, fast construction	Poor flexibility, cannot share layers
Manually define forward	Flexible control over data flow	More code required

---

Frequently Asked Questions

Q1: How to get a specific layer from Sequential?

Access via index or name: model or model.fc1

Q2: Is Sequential suitable for all networks?

No, it's not suitable for networks with skip connections, branching structures, or complex data flows.

Q3: How to modify layers in Sequential?

You can replace a specified layer using model = new_module.

---

Use Cases

Main application scenarios for nn.Sequential include:

• Simple Networks: Linearly stacked MLPs, CNNs
• Rapid Prototyping: Quickly validate model ideas
• Feature Extraction: Fixed network structures

Tip: For complex networks, it is recommended to inherit nn.Module and manually define the forward method.

---

PyTorch torch.nn Reference Manual