Introduction to Tensors

Tensors are the building blocks of PyTorch, serving as the fundamental data structure for all operations in the library. Think of tensors as multi-dimensional arrays, capable of representing scalar values, vectors, matrices, and even higher-dimensional data.

Creating Tensors

Let's start by exploring different ways to create tensors in PyTorch:

import torch

# From a Python list
tensor_from_list = torch.tensor([1, 2, 3, 4])

# From a NumPy array
import numpy as np
numpy_array = np.array([1, 2, 3, 4])
tensor_from_numpy = torch.from_numpy(numpy_array)

# Zeros and ones
zeros_tensor = torch.zeros(3, 3)
ones_tensor = torch.ones(2, 2)

# Random tensors
random_tensor = torch.rand(4, 4)

print(tensor_from_list)
print(tensor_from_numpy)
print(zeros_tensor)
print(ones_tensor)
print(random_tensor)

Tensor Attributes

Tensors come with several important attributes:

1. Shape: Describes the dimensions of the tensor.
2. Dtype: Specifies the data type of the tensor elements.
3. Device: Indicates whether the tensor is stored on CPU or GPU.

Let's examine these attributes:

tensor = torch.rand(3, 4)

print(f"Shape: {tensor.shape}")
print(f"Data Type: {tensor.dtype}")
print(f"Device: {tensor.device}")

Tensor Operations

PyTorch provides a wide array of operations for manipulating tensors. Here are some common ones:

Arithmetic Operations

a = torch.tensor([1, 2, 3])
b = torch.tensor([4, 5, 6])

# Addition
print(a + b)

# Multiplication
print(a * b)

# Matrix multiplication
c = torch.tensor([[1, 2], [3, 4]])
d = torch.tensor([[5, 6], [7, 8]])
print(torch.matmul(c, d))

Indexing and Slicing

Tensors can be indexed and sliced similar to NumPy arrays:

tensor = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

# Get the first row
print(tensor[0])

# Get the second column
print(tensor[:, 1])

# Slice the tensor
print(tensor[1:, 1:])

Reshaping Tensors

Changing the shape of tensors is a common operation in deep learning:

tensor = torch.tensor([1, 2, 3, 4, 5, 6])

# Reshape to 2x3
reshaped = tensor.reshape(2, 3)
print(reshaped)

# Transpose
transposed = reshaped.t()
print(transposed)

Tensor Computation and Gradients

One of the most powerful features of PyTorch is automatic differentiation. This is achieved through the requires_grad attribute:

x = torch.tensor([1.0, 2.0, 3.0], requires_grad=True)
y = x.pow(2).sum()

# Compute gradients
y.backward()

print(f"Gradients: {x.grad}")

Moving Tensors to GPU

To leverage the power of GPUs for faster computations, you can move tensors to the GPU:

if torch.cuda.is_available():
    tensor = torch.tensor([1, 2, 3]).cuda()
    print(f"Device: {tensor.device}")
else:
    print("CUDA is not available. Using CPU.")

Practical Example: Linear Regression

Let's put our tensor knowledge into practice with a simple linear regression example:

import torch
import torch.nn as nn
import torch.optim as optim

# Generate some fake data
X = torch.rand(100, 1) * 10
y = 2 * X + 1 + torch.randn(100, 1)

# Define the model
class LinearRegression(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(1, 1)
    
    def forward(self, x):
        return self.linear(x)

model = LinearRegression()

# Define loss function and optimizer
criterion = nn.MSELoss()
optimizer = optim.SGD(model.parameters(), lr=0.01)

# Training loop
for epoch in range(100):

# Forward pass
    y_pred = model(X)
    loss = criterion(y_pred, y)

# Backward pass and optimize
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    
    if (epoch + 1) % 10 == 0:
        print(f'Epoch [{epoch+1}/100], Loss: {loss.item():.4f}')

# Print model parameters
print(f"Weight: {model.linear.weight.item():.2f}")
print(f"Bias: {model.linear.bias.item():.2f}")

This example demonstrates how tensors are used in a real-world scenario, from data representation to model parameters and computations.