Meta's AI tool can cut-out objects from images: Here's how
Haven't we all had that one great image that couldn't be posted anywhere because of something annoying in the background? Well, Meta has a solution for this problem - SAM. SAM or Segment Anything Model can cut out anything from an image. The company claims its AI model can make segmentation easy. It has also released the Segment Anything 1-Billion mask dataset (SA-1B).
Why does this story matter?
Computer vision is a fast-evolving sphere of AI. Companies such as Google and Amazon have been working on computer vision for a while now. Image segmentation is an integral part of computer vision technologies and algorithms. However, creating an accurate segmentation model for specific data is beyond most AI researchers due to the complexity involved. Meta aims to solve that problem with SAM.
What is SAM?
SAM is a generalized segmentation model. It is a mixture of both interactive and automatic segmentation, the two classic approaches to segmentation. The model's purpose is to reduce the need for "task-specific modeling expertise, training compute, and custom data annotation." SAM is a 'promptable' model that is general enough to cover a wide range of tasks.
SA-1B is the largest image segmentation dataset in the world
SAM's generality comes from the dataset it was trained on. SA-1B has over 1.1 billion segmentation masks (a specific portion of an image isolated from others), which is 400 times more masks than any other existing dataset. The enormity of the dataset gives SAM the ability to generalize images and objects not part of its training as well.
SAM can be prompted in multiple ways
With SAM, users can segment images by either clicking on any object in the image or by "interactively clicking on points to include and exclude from the image," Meta said. To prompt the model to segment an object, users can also draw a bounding box. If there is any ambiguity, SAM can show multiple valid masks. Text prompts capability is not released yet.
SAM will be a good fit with other AI models
SAM will be useful in any field that requires finding and segmenting objects. Its promptable design will make it easier to use SAM with other larger AI systems. One of the use cases pointed out by Meta is AR/VR. SAM could be part of a user's gaze and then lift into 3D. It will also be useful for scientific studies and content creation.
