Pytorch Torch Matmul

* * Pytorch torch Reference Manual](#) `torch.matmul` is a function in PyTorch used to perform matrix multiplication. It supports inputs of different dimensions and can handle 1D, 2D, and higher-dimensional tensor multiplication. This is one of the most commonly used operations in deep learning; the forward propagation of neural networks is essentially a series of matrix multiplications. ### Function Definition torch.matmul(input, other, out=None) **Parameters**: * `input` (Tensor): The first input tensor. * `other` (Tensor): The second input tensor. * `out` (Tensor, optional): The output tensor. **Return Value**: * `torch.Tensor`: Returns the result of the matrix multiplication. * * * ## Usage Examples ### Example 1: 2D Matrix Multiplication ## Example import torch # Create two 2D matrices a = torch.randn(3,4)# 3x4 matrix b = torch.randn(4,5)# 4x5 matrix # Matrix multiplication c = torch.matmul(a, b) print("Shape of a:", a.shape) print("Shape of b:", b.shape) print("Shape of c:", c.shape) The output result is: Shape of a: torch.Size([3, 4]) Shape of b: torch.Size([4, 5]) Shape of c: torch.Size([3, 5]) ### Example 2: Vector and Matrix Multiplication ## Example import torch # Create vector and matrix vector = torch.randn(4)# 4-dimensional vector matrix = torch.randn(4,5)# 4x5 matrix # Multiply vector and matrix result = torch.matmul(vector, matrix) print("Shape of vector:", vector.shape) print("Shape of matrix:", matrix.shape) print("Shape of result:", result.shape) print(result) The output result is: Shape of vector: torch.Size()Shape of matrix: torch.Size([4, 5])Shape of result: torch.Size() tensor([-0.7837, 0.3684, -0.6542, -0.4594, 1.5328])

Example 3: Batch Matrix Multiplication

Example

import torch

# Create batch matrices
batch_a = torch.randn(10, 3, 4) # 10 3x4 matrices
batch_b = torch.randn(10, 4, 5) # 10 4x5 matrices

# Batch matrix multiplication
batch_c = torch.matmul(batch_a, batch_b)

print("Shape of batch a:", batch_a.shape)
print("Shape of batch b:", batch_b.shape)
print("Shape of batch result c:", batch_c.shape)

The output result is:

Shape of batch a: torch.Size([10, 3, 4])Shape of batch b: torch.Size([10, 4, 5])Shape of batch result c: torch.Size([10, 3, 5])
Batch matrix multiplication is a common operation in deep learning, for example, in the attention mechanism of Transformers.

### Example 4: Matrix Multiplication in Neural Networks

## Example

import torch

# Simulate neural network layer: input xW + b

 x = torch.randn(32,128)# Batch size 32, feature dimension 128

 W = torch.randn(128,256)# Weight matrix

 b = torch.randn(256)# Bias vector

# Linear transformation: y = x @ W^T + b (W is usually transposed in PyTorch)

# Here demonstrates x @ W

 y = torch.matmul(x, W) + b

print("Shape of input x:", x.shape)

print("Shape of weight W:", W.shape)

print("Shape of output y:", y.shape)

The output result is:

Shape of input x: torch.Size([32, 128])Shape of weight W: torch.Size([128, 256])Shape of output y: torch.Size([32, 256])
This example simulates the forward propagation process of a fully connected layer in a neural network.

* * *

## Notes

*   `torch.matmul` supports broadcasting, but the last two dimensions of the two inputs must satisfy the dimension requirements for matrix multiplication.
*   For 2D matrix multiplication, `torch.mm()` can also be used, but `torch.matmul` is more general.
*   Pay attention to the distinction between `torch.matmul` (matrix multiplication) and `torch.mul` (element-wise multiplication).

* * Pytorch torch Reference Manual](#)

YouTip