Appendix F: Useful PyTorch Code Templates

Prof.(Dr.) D G Mahto, Founder & CEO -

Abstract:

Below is Appendix F: Useful PyTorch Code Templates, written as a complete and polished appendix for PyTorch book. It includes ready-to-use templates for models, datasets, training loops, evaluation, logging, and deployment—everything beginners and practitioners need.

Appendix F: Useful PyTorch Code Templates

This appendix provides ready-made, reusable code templates frequently used in PyTorch projects. These templates simplify development by offering standard structures for:

  • Model definitions (MLP, CNN, RNN, Transformers)

  • Custom datasets and dataloaders

  • Training, validation, and testing loops

  • Logging and checkpointing

  • Inference and deployment

  • Mixed precision training

  • GPU/multi-GPU support

  • ONNX export and TorchScript

These code blocks can be directly copied into real-world PyTorch projects.

F.1 Template: Basic Project Structure

project/
│── data/
│── models/
│── utils/
│── train.py
│── model.py
│── dataset.py
│── inference.py

This structure works for almost any PyTorch workflow.

F.2 Template: Importing PyTorch Properly

import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import Dataset, DataLoader

F.3 Model Templates

F.3.1 Simple Feed-Forward Neural Network (MLP)

class MLP(nn.Module):
    def __init__(self, input_dim, hidden_dim, num_classes):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(input_dim, hidden_dim),
            nn.ReLU(),
            nn.Linear(hidden_dim, num_classes)
        )

    def forward(self, x):
        return self.net(x)

F.3.2 Convolutional Neural Network (CNN) Template

class SimpleCNN(nn.Module):
    def __init__(self, num_classes=10):
        super().__init__()
        self.features = nn.Sequential(
            nn.Conv2d(3, 32, 3, padding=1),
            nn.ReLU(),
            nn.MaxPool2d(2),

            nn.Conv2d(32, 64, 3, padding=1),
            nn.ReLU(),
            nn.MaxPool2d(2)
        )
        
        self.classifier = nn.Sequential(
            nn.Flatten(),
            nn.Linear(64 * 8 * 8, 128),
            nn.ReLU(),
            nn.Linear(128, num_classes)
        )

    def forward(self, x):
        x = self.features(x)
        return self.classifier(x)

F.3.3 LSTM-Based Sequence Model

class LSTMClassifier(nn.Module):
    def __init__(self, vocab_size, embed_dim, hidden_dim, num_classes):
        super().__init__()
        self.embedding = nn.Embedding(vocab_size, embed_dim)
        self.lstm = nn.LSTM(embed_dim, hidden_dim, batch_first=True)
        self.fc = nn.Linear(hidden_dim, num_classes)

    def forward(self, x):
        x = self.embedding(x)
        _, (h, _) = self.lstm(x)
        return self.fc(h[-1])

F.3.4 Transformer Encoder Template

class TransformerClassifier(nn.Module):
    def __init__(self, vocab_size, embed_dim, num_heads, num_layers, num_classes):
        super().__init__()
        self.embedding = nn.Embedding(vocab_size, embed_dim)
        
        encoder_layer = nn.TransformerEncoderLayer(
            d_model=embed_dim, 
            nhead=num_heads
        )
        self.transformer = nn.TransformerEncoder(encoder_layer, num_layers)

        self.fc = nn.Linear(embed_dim, num_classes)

    def forward(self, x):
        x = self.embedding(x)  # (B, L, D)
        x = x.permute(1, 0, 2)  # (L, B, D) for transformer
        out = self.transformer(x)
        out = out.mean(dim=0)  # Global average pooling
        return self.fc(out)

F.4 Template: Custom Dataset

F.4.1 Custom Dataset for Images

from PIL import Image
import os

class CustomImageDataset(Dataset):
    def __init__(self, root, transform=None):
        self.root = root
        self.files = os.listdir(root)
        self.transform = transform

    def __len__(self):
        return len(self.files)

    def __getitem__(self, idx):
        img_path = os.path.join(self.root, self.files[idx])
        image = Image.open(img_path).convert("RGB")

        if self.transform:
            image = self.transform(image)

        label = int(self.files[idx].split("_")[0])  # assumes label_filename.jpg
        return image, label

F.4.2 Custom Dataset for Text

class TextDataset(Dataset):
    def __init__(self, texts, labels, tokenizer):
        self.texts = texts
        self.labels = labels
        self.tokenizer = tokenizer

    def __getitem__(self, idx):
        tokens = self.tokenizer(self.texts[idx])
        return torch.tensor(tokens), torch.tensor(self.labels[idx])

    def __len__(self):
        return len(self.texts)

F.5 Template: DataLoader

train_loader = DataLoader(dataset, batch_size=32, shuffle=True, num_workers=2)
valid_loader = DataLoader(dataset, batch_size=32)

F.6 Template: Training Loop

def train(model, dataloader, loss_fn, optimizer, device):
    model.train()
    total_loss = 0

    for x, y in dataloader:
        x, y = x.to(device), y.to(device)

        optimizer.zero_grad()
        preds = model(x)
        loss = loss_fn(preds, y)
        loss.backward()
        optimizer.step()

        total_loss += loss.item()

    return total_loss / len(dataloader)

F.7 Template: Validation Loop

def validate(model, dataloader, loss_fn, device):
    model.eval()
    total_loss = 0
    correct = 0

    with torch.no_grad():
        for x, y in dataloader:
            x, y = x.to(device), y.to(device)
            preds = model(x)
            loss = loss_fn(preds, y)
            total_loss += loss.item()
            correct += (preds.argmax(1) == y).sum().item()

    accuracy = correct / len(dataloader.dataset)
    return total_loss / len(dataloader), accuracy

F.8 Template: Training Script

device = "cuda" if torch.cuda.is_available() else "cpu"

model = SimpleCNN().to(device)
optimizer = optim.Adam(model.parameters(), lr=1e-3)
loss_fn = nn.CrossEntropyLoss()

for epoch in range(10):
    train_loss = train(model, train_loader, loss_fn, optimizer, device)
    val_loss, val_acc = validate(model, val_loader, loss_fn, device)

    print(f"Epoch {epoch+1}: Train Loss={train_loss:.4f}, "
          f"Val Loss={val_loss:.4f}, Val Acc={val_acc:.4f}")

F.9 Template: Mixed Precision Training (AMP)

scaler = torch.cuda.amp.GradScaler()

for x, y in train_loader:
    x, y = x.to(device), y.to(device)

    optimizer.zero_grad()

    with torch.cuda.amp.autocast():
        preds = model(x)
        loss = loss_fn(preds, y)

    scaler.scale(loss).backward()
    scaler.step(optimizer)
    scaler.update()

F.10 Template: Saving & Loading Models

Save

torch.save(model.state_dict(), "model.pth")

Load

model = SimpleCNN()
model.load_state_dict(torch.load("model.pth"))
model.eval()

F.11 Template: Inference (Prediction)

def predict(model, x, device):
    model.eval()
    with torch.no_grad():
        x = x.to(device)
        preds = model(x)
        return preds.argmax(1)

F.12 Template: Early Stopping

class EarlyStopping:
    def __init__(self, patience=3):
        self.patience = patience
        self.counter = 0
        self.best_loss = float("inf")

    def step(self, loss):
        if loss < self.best_loss:
            self.best_loss = loss
            self.counter = 0
        else:
            self.counter += 1

        return self.counter >= self.patience

F.13 Template: TensorBoard Logging

from torch.utils.tensorboard import SummaryWriter
writer = SummaryWriter()

writer.add_scalar("Loss/train", train_loss, epoch)
writer.close()

F.14 Template: Export to ONNX

dummy = torch.randn(1, 3, 224, 224).to(device)
torch.onnx.export(model, dummy, "model.onnx")

F.15 Template: Export to TorchScript

traced = torch.jit.trace(model, dummy)
traced.save("model_traced.pt")

F.16 Template: Multi-GPU Training (DDP)

torchrun --nproc_per_node=4 train.py

Inside the script:

ddp_model = nn.parallel.DistributedDataParallel(model)

F.17 Summary

This appendix provided a rich set of PyTorch templates including:

  • Model templates (MLP, CNN, LSTM, Transformers)

  • Custom Dataset and DataLoader templates

  • Training, validation, and inference loops

  • Logging, checkpointing, AMP, and multi-GPU support

  • Exporting models (TorchScript/ONNX)

  • Early stopping and project structure

These templates equip learners and practitioners with ready-to-use building blocks to build scalable, clean, and production-ready PyTorch projects.


Prof. (Dr.) D. G. Mahto: A Visionary Director, Professor, Leader & Educator With over 25 years of excellence, Prof. (Dr.) D. G. Mahto is a Director, Professor, Philanthropist, HEI Builder, writer, consultant, reviewer, and career advisor at "Career Education for Success—Discover, Apply, Succeed!" He has empowered individuals, brands, and institutions through insightful content, innovation, and mentorship. Holding a Ph.D. in Engineering from NIT Jamshedpur, he has pursued Post-Doctoral Programs at IITs, NITs, and IIITs, reinforcing his expertise. A member of prestigious global organizations like NPA (USA), IET (UK), ASME, IAENG, and MRSI, he actively contributes to engineering, research, and education. An accomplished writer and researcher, he shares knowledge via Career Education Success Now, Career Edus, and Discover Knowledge Wisely. His vision integrates technology, innovation, and leadership, bridging gaps between aspirations and achievements. Connect with him for insights into education, research, and professional growth.

Comments

Post a Comment

"Thank you for seeking advice on your career journey! Our team is dedicated to providing personalized guidance on education and success. Please share your specific questions or concerns, and we'll assist you in navigating the path to a fulfilling and successful career."