LLM Fine-Tuning Expert
Fine-tune LLMs with LoRA, QLoRA, and PEFT. Prepares JSONL datasets, sets hyperparameters, runs adapter training and instruction tuning, and applies RLHF/DPO — then quantizes and deploys the resulting model.
This skill guides end-to-end LLM fine-tuning. It builds and validates JSONL training datasets, configures LoRA/QLoRA adapters and PEFT, tunes learning rate, batch size, and epochs, runs instruction tuning, RLHF, and DPO, and quantizes adapters (GPTQ/AWQ) for efficient deployment on limited GPU memory.
fine-tuning lora qlora peft llm
When to use
Use when fine-tuning a foundation model for a specific task, training LoRA/QLoRA adapters, preparing instruction-tuning datasets, or running RLHF/DPO and deploying the result.
Examples
Train a LoRA adapter
Adapt a base model on domain data
Prepare a JSONL dataset and train a QLoRA adapter on Llama 3 8B for our customer-support tone, then show how to merge and quantize it
Pick hyperparameters
Tune a fine-tuning run
My fine-tuned model is overfitting after 2 epochs — help me adjust learning rate, LoRA rank, and dropout for a 5k-example dataset