Vision Transformers (ViT)

Vision Transformers, ViT, adapt the transformer, first built for text, to images.

Instead of convolutional filters sliding over pixels, a ViT splits the image into a grid of fixed-size patches, flattens each patch into a vector, and feeds that sequence into a standard transformer encoder. The model attends to relationships among all patches at once, including long-range spatial links that CNNs handle poorly.

On large image-classification benchmarks, ViTs often reach higher accuracy in fewer training epochs than comparable convolutional models.

The sequence design makes it easier to mix images with text or audio in one transformer, which simplifies multimodal systems. Teams that process satellite imagery, medical scans, or e-commerce product photos benefit from that scaling. In research, ViT is a backbone for object detection, segmentation, and video understanding, with the same transformer code reused across those tasks.

Patch, flatten, encode: that is the whole inductive bias swap versus a CNN. Global attention sees far-apart patches in one layer, which helps long-range layout. Large classification benchmarks often show higher accuracy and fewer epochs than a matched conv net. The same sequence interface accepts text or audio tokens beside patches, which is why multimodal stacks like this backbone.

Satellite, medical, and product-photo pipelines care about that scaling. Detection, segmentation, and video models reuse the encoder rather than writing a new vision architecture per task. ViT treats image patches like tokens. That is why the same attention stack can do vision and language.