SAM

SAM is a Meta AI segmentation model that can mask objects from a point, box, or text prompt without extra training.

It was trained on over 1 billion masks from 11 million images, using a data engine that improved the model and the dataset together. That scale is why zero-shot generalization works: SAM can mask objects and domains it never saw in training. A heavy image encoder runs once per image. A light mask decoder runs per prompt, so a user can click and get a mask immediately.

Point prompts mark locations. Box prompts mark regions. Mask prompts give a coarse outline to refine. SAM 2 adds text prompts that describe what to segment, and extends tracking across video frames. The name states the goal: a general segmentation foundation, the way GPT-3 became a general language foundation. Annotation tools use it to label faster. Editors use it for precise object selection.

Larger vision systems use it as a component.

The data engine looped: model predicts masks, annotators correct them, corrected masks retrain the model, repeat, until more than 1 billion masks on 11 million images existed. That is the source of zero-shot transfer. Interactive use is: run the image encoder once, then click. Each click is a point prompt into the lightweight decoder. Boxes and coarse masks are other prompts.

SAM 2 tracks a mask through video and accepts text. Labeling tools, photo editors, and downstream vision pipelines all call this decoder instead of training a new segmenter per object type. Meta's Segment Anything Model (2023) was trained to mask objects from a click, box, or text prompt.