Post-training 0/0

Post-training

Fine-tuning

Training an existing model further on your own examples, in full or with parameter-efficient methods such as LoRA and QLoRA, and when prompting or retrieval does the job instead.

Reading

WhiteBox lessons1

Papers and essays2

  1. LoRA: Low-Rank Adaptation of Large Language Models(opens in a new tab)

    Hu et al. 2022 ICLR 2022 50 min

    LoRA: freeze the pre-trained weights and train two small low-rank matrices per layer whose product is added to a frozen weight matrix, which for GPT-3 175B cut trainable parameters by 10,000 times and GPU memory by 3 times.

  2. QLoRA: Efficient Finetuning of Quantized LLMs(opens in a new tab)

    Dettmers et al. 2023 NeurIPS 2023 60 min

    QLoRA: train LoRA adapters through a frozen model stored in a new 4-bit NormalFloat format, with double quantization of the quantization constants and paged optimizers for memory spikes, which fits fine-tuning of a 65B model on a single 48GB GPU.