Meta tests 3D modeling tools for marketplace

Meta on Tuesday announced SAM 3D, a new addition to its Segment Anything Model collection designed to generate 3D information from standard images. The company said the tools aim to improve 3D understanding for uses such as augmented and virtual reality, game development and computer vision research.

The release includes two models. SAM 3D Objects is built for object and scene reconstruction, producing 3D shapes, textures and layouts from a single image. SAM 3D Body focuses on human-body estimation. Both are designed to convert 2D images into detailed 3D representations.

It also published model checkpoints and inference code for researchers and developers. The company said it plans to release a new dataset, SAM 3D artist objects, which pairs images with object meshes to support evaluation of real-world 3D reconstruction.

A web-based tool, SAM, allows users to upload images, select objects or people, and generate 3D reconstructions. The platform also includes SAM 3, the company’s latest model for image and video understanding.

According to the company, SAM 3D and SAM 3 are being used to test a ‘View in Room’ feature on Facebook Marketplace, which allows users to preview home décor items in their own spaces. The company has not announced a date for wider deployment.

Meta said earlier that 3D models were limited by the small amount of real-world 3D data available, much of it synthetic. The new system is intended to address challenges such as occlusion, indirect views and varied environments that often occur in everyday photos.