Fine-Tuning Guide

How to fine-tune pre-trained language models for your specific NLP tasks.

When to Fine-Tune vs. Use Zero-Shot

Fine-tune when: You have labeled data, need low latency, want to avoid API costs, or need domain-specific accuracy that general models cannot match.

Choosing a Base Model

Quick Start: Classification Fine-Tuning

from nlp_starter_kit.fine_tuning import FineTuningConfig, HuggingFaceFineTuner

config = FineTuningConfig(
    model_name="distilbert-base-uncased",
    num_epochs=3,
    batch_size=16,
    learning_rate=2e-5,
)

tuner = HuggingFaceFineTuner(config)
tuner.prepare_data(
    train_texts=["Great product!", "Terrible service", ...],
    train_labels=["positive", "negative", ...],
    eval_texts=["Loved it!", "Would not recommend", ...],
    eval_labels=["positive", "negative", ...],
)

metrics = tuner.train()
print(f"Accuracy: {metrics['eval_accuracy']}")

tuner.save_model("./my-classifier")

LoRA Fine-Tuning (Parameter-Efficient)

LoRA (Low-Rank Adaptation) freezes the base model and injects small trainable matrices into attention layers. Benefits:

• 95%+ parameter reduction — train 1M params instead of 100M
• Fits on consumer GPUs — fine-tune BERT-large on 8GB VRAM
• Fast training — fewer gradients to compute
• Easy model merging — combine multiple LoRA adapters

config = FineTuningConfig(
    model_name="bert-large-uncased",
    use_peft=True,
    lora_r=8,            # Rank — lower = fewer params, less expressive
    lora_alpha=16,        # Scaling factor — typically 2x lora_r
    lora_dropout=0.05,    # Regularization
    lora_target_modules=["query", "value"],  # Which attention matrices
    learning_rate=3e-4,   # LoRA uses higher LR than full fine-tuning
    num_epochs=5,
    batch_size=8,
    gradient_accumulation_steps=4,  # Effective batch = 32
    fp16=True,
)

LoRA Rank Selection

Learning Rate Scheduling

The kit provides two schedulers, both matching Hugging Face Trainer behavior:

Linear with Warmup (Default)

LR
│  /‾‾‾‾‾‾‾‾‾‾‾‾\
│ /                \
│/                  \
└──────────────────── Steps
  warmup    decay

Good for most tasks. The warmup prevents early training instability.

Cosine with Warmup

LR
│  /‾‾‾‾‾\_
│ /         ‾‾\_
│/               ‾‾‾‾
└──────────────────── Steps
  warmup    cosine decay

Better for longer training runs. The gradual decay lets the model explore longer before converging.

Hyperparameter Recommendations

Dataset Size → Hyperparameters

Gradient Accumulation

If your GPU cannot fit batch_size=32, use gradient accumulation:

batch_size=8,
gradient_accumulation_steps=4,  # Effective batch = 8 × 4 = 32

The optimizer updates after accumulating gradients from 4 mini-batches, giving the same training dynamics as a larger batch without the memory cost.

Evaluation During Training

Always evaluate during training to catch overfitting early:

config = FineTuningConfig(
    eval_strategy="epoch",      # Evaluate after each epoch
    save_strategy="epoch",      # Save checkpoint after each epoch
    save_total_limit=3,         # Keep only 3 most recent checkpoints
)

Watch for:

• Training loss drops, eval loss rises → Overfitting. Reduce epochs or increase regularization.
• Eval loss plateaus → Learning rate may be too low, or model has converged.
• Training loss doesn't decrease → Learning rate too low, or data issues.

Common Mistakes

1. Using the wrong casing. If you're doing NER, use a cased model (bert-base-cased). Uncased models lose important entity signals.

2. Too many epochs on small data. With <1000 examples, 3 epochs of full fine-tuning will overfit. Use LoRA or early stopping.

3. Not shuffling training data. If all positive examples come before negative examples, the model learns order, not content. Hugging Face Trainer shuffles by default.

4. Ignoring class imbalance. If 90% of your data is one class, the model will just predict that class. Use weighted loss or oversample the minority class.

5. Not saving the tokenizer. When you save a fine-tuned model, always save the tokenizer too — it defines how text is converted to input IDs.