BigShinyThing

MIT’s Technology Review highlights the new web service Make3D, which does a truly amazing job of extracting 3D data from normal 2D images.

A spinoff from research at Stanford, Make3D works its magic using:

a machine-learning algorithm that associates visual cues, such as color, texture, and size, with certain depth values based on what they have learned from studying two-dimensional photos paired with 3-D data. For example [...] grass has a distinctive texture that makes it look very different close up than it does from far away. The algorithm learns that the progressive change in texture gives clues to the distance of a patch of grass.

Note the key phrase ‘machine learning’. They haven’t tried to understand the world — they’ve built a tool which can learn to understand depth cues in visual imagery. Cool.

Currently the system only understands scene cues in outdoor landscapes and (rather curiously, we think) indoor scenes which feature staircases (but why not?). Future work will help their system learn about other kinds of scenes. But what it does, it does very well indeed as proof-of-concept. See the Make3D site for demos, or to upload your own scenes for processing.

Impressive as it stands. But as we see it, the most exciting place for this technology to turn up will be at the point of capture — in cameras. Our Nikon D200 already features a ‘programme’ mode for autoexposure, which uses scene cues to understand something of what’s in front of the lens: a big blue rectangle top of image , for example, is probably the sky, and should maybe be overexposed relative to the lower half of the scene so that whatever’s underneath doesn’t come out pitch-black in the photo. Add in Make3D, which could profit from a whole slew of data available at time of capture (by, for example, capturing more depth data before and after the photo is taken by playing with autofocus…) and you’ve got consumer 3D photography done and dusted. We can’t wait.