Capture, an iPhone 3D scanner app, hints at the future of augmented reality

3D scanners don’t have to cost a fortune: As it turns out, you may already have one in your pocket. A new free app called Capture uses the TrueDepth sensors on X-series iPhones to make crude 3D models of small objects, which you can then plop into different settings through augmented reality or share with other folks through iMessage or other means.

Or let’s put it this way: With Capture, I can use my iPhone XS Max to scan a 3D model of Macworld’s old Macintosh SE, being careful to scan each angle I can reach (or until it times out). Once done, I can flip and spin the 3D model around inside the app as though the old Mac were a prop in a video game. I could also send it to a friend, who’d then be able to use her iPhone’s camera see how it looks on her own desk through augmented reality. This will work even with her iPhone 7, which doesn’t have the TrueDepth sensor built in.

Leif Johnson/IDG

The real Macintosh SE on the table is in the photo on the left, while the AR version I “placed” on the table is to the right. Unfortunately, the scans are always colorless when used in AR.

Sounds revolutionary, right? Alas, for now it looks as basic as biscuits, which is why you likely haven’t heard of it. Even the publicity images from developer Standard Cyborg look like they’re being disintegrated at the end of Avengers: Infinity War. And as with many other scanning apps, you’re never going to be able to make a true 3D model with Capture’s current version. Even when I could get a decent scan of one side of an object, I wasn’t able to scan its underside.

Facing forward

But don’t blame Standard Cyborg for this. Capture’s awkwardness springs from the current design of the TrueDepth sensors, which are made to read faces from only a few inches away. That means Capture only works well if you’re holding something in your hands (or scanning your face) as the iPhone only has TrueDepth sensors on the front-facing camera.

If you try to scan something in the same way you’d take a video with the rear camera, you’ll be scanning blindly, guided only by subtle vibrations that remind me of notification taps on an Apple Watch. If you move too quickly, Capture ends the scan. As a result, my folder of finished scans looks like a grab bag filled with more tricks than treats. Frankly, I’d be angry if Standard Cyborg were charging any cash for this.

But according to Standard Cyborg co-chief Garrett Spiegel, this is still better than what you’ll find in competing apps. In a conversation on Twitter, Spiegel told me that most competing apps are made by hobbyists who lack his company’s venture financing and engineers with experience in computational geometry or computer vision. These apps are often slow and unstable, he says, and they’re designed so poorly that they even make iPhones run hot.

“By comparison, our algorithms are written in a way such that our app does over 30 frames per second (more than twice the next best app), which means the scanning is more seamless and similar to just taking a panorama (and the speed is already getting better),” Spiegel said.

Spiegel also highlighted Standard Cyborg’s integrated cloud storage system with machine learning models, which makes it easy to import Capture scans into other apps.

Source link