Adapter

An adapter is a small trainable block you insert into a frozen pretrained model so it can do a new task without moving most of the weights. The usual shape is a bottleneck: project down, apply a nonlinearity, project back up. You put it after attention or feedforward layers, with a residual so it only learns what to add.

5% to 5% of the parameters in the layers they sit in. On many tasks they match full fine-tuning. That buys practical things. Different adapters can share one base model. You can train on smaller hardware. You can mix adapters. Only adapter weights need gradients and optimizer state, so training is cheaper. The base forward pass stays the same, which pairs well with quantization.

The family grew. LoRA updates weight matrices with a low-rank patch. Prefix tuning prepends learnable tokens. Prompt tuning learns input embeddings. The shared idea is freeze almost everything, learn a small targeted change.

That is how more teams customize a large language model. You do not need to fine-tune every weight. You need a small module, a task, and a frozen base that already knows language.

Swap the adapter, keep the base. That is the operational win for a team that serves many tasks from one GPU footprint. Adapters (2019) insert small bottleneck modules into a frozen Transformer so each task has its own tiny add-on.