Let’s talk about depth camera in smartphones. Smartphone cameras have come a long way in the last decade. What started as basic rear cameras for taking photos and videos has now evolved into multi-lens systems capable of professional-grade photography.
One of the most interesting developments in smartphone cameras recently is the addition of depth cameras, also known as 3D cameras.
In this post, we’ll take a look at what exactly depth cameras are, their benefits and drawbacks, their use cases, how they compare to other depth sensing technologies like Apple’s TrueDepth camera, and the future of this technology in smartphones.
What is a Depth Camera in a Smartphone?
A depth camera, also known as an RGB-D camera, is a camera system that adds depth perception capability to smartphones and other devices.
While regular smartphone cameras capture the color and brightness of a scene, a depth camera also captures the third dimension of depth. This allows the camera to perceive distances between objects in the scene.
Depth cameras work by projecting infrared light onto the scene. An infrared sensor then reads this pattern of infrared dots, compares it to what was originally projected, and triangulates the distance to objects in the scene based on distortions in how the pattern falls on them.
Additional processing is then done by the smartphone processor to create a depth map with estimated distances to subjects in focus.
Benefits of having a Depth Camera in a Smartphone
There are several benefits to having depth cameras in smartphones:
1. Better Portrait mode photos
By determining distances to subjects, the depth camera allows portrait photos to add artificial blurred backgrounds, making subjects “pop” from the scene. This works best on people and objects in the foreground and is difficult to achieve with regular single lenses.
2. AR applications
Depth information allows for more accurate augmented reality effects and measurements. This helps anchoring of virtual objects onto real structures more convincingly by understanding 3D spaces better.
3. 3D scanning capability
With depth data, smartphones can map small three-dimensional areas and objects for the purposes of 3D scanning.
4. Potential for other computational photography features
Other camera features may leverage depth data as an additional input, spurring algorithmic innovation for even better photos and videos.
Drawbacks of having a Depth Camera in a Smartphone
However, depth camera systems also come with some downsides:
1. Increased cost and complexity
Depth cameras require additional components over regular cameras, increasing costs for manufacturers. The additional data requires more smartphone processing power too.
2. Impact on smartphone design
Many depth camera implementations require fairly large sensor arrays and lens openings, leading to larger camera bumps that can impact smartphone aesthetics and ergonomics.
3. Performance limitations in some use cases
The small sensors and lenses limit the quality of depth maps calculated, so performance may suffer in low light compared to larger professional systems. Range is also limited for mapping larger areas and objects.
What is a Depth Camera Used For?
Here are some of the current and potential use cases where smartphone depth cameras offer unique value:
1. Portrait mode photos
This is the most popular current use, adding artificial background blur to make subjects stand out.
2. Augmented reality effects
Fun effects like face filters, masks, 3D animoji, virtual furniture placement in rooms for shopping apps. Accuracy is improved by understanding real-world depths and surfaces better.
3. Room/space mapping for interior design apps
Apps can map and measure rooms, detect walls and objects, helping plan space layouts and design.
4. 3D scanning of small objects
While limited in object size, simple smartphones scans can be useful for product design, 3D printing, gaming, virtual spaces.
5. Camera Autofocus
Dual pixel autofocus and other features on some smartphone cameras also leverage depth data.
Is Depth Camera on Android and TrueDepth Camera on iPhone same?
While the core technology principles behind depth cameras are similar across smartphones, Apple’s TrueDepth camera system on newer iPhones is its own custom implementation and has a few key differences:
- Rather than relying on the normal front camera, TrueDepth has dedicated hardware with more capable sensors for depth mapping. This allows it to work better in low light.
- More advanced face mapping: Apple uses over 30,000 invisible dots to map faces in 3D, enabling advanced FaceID security and expressive 3D avatar Animoji that can mimic facial movements.
- Apple was among the first to popularize depth cameras on phones with the iPhone X in 2017. Android phones have been catching up since with similar TrueDepth-like capabilities.
Is Time-of-flight (ToF) Camera sensor the same as Depth Camera?
Strictly speaking, time-of-flight (ToF) sensors are one specific type of depth camera technology used in smartphones. But the terms are sometimes used interchangeably.
Depth cameras refer to any camera system capable of capturing depth information – the third dimension of distance/depth in addition to standard cameras that capture color images.
Time-of-flight sensors are one common approach used to implement depth cameras in smartphones. They measure depth/distance by emitting light pulses and calculating distance based on the time it takes for the light to bounce back to the sensor.
But smartphone makers and tech publications often use “depth camera” as a catch-all term to refer to the depth mapping capability and features enabled by time-of-flight sensors in smartphones.
Conclusion
In conclusion, depth cameras bring new creative possibilities to smartphone photography while also enabling more immersive AR experiences and use cases involving 3D spatial data. Dual camera systems combining depth and traditional RGB sensors allow innovative hybrid applications.
Apple set the bar high with their custom-designed TrueDepth system, but Android smartphones have caught up impressively with similar capabilities now baked into many flagship devices too.
As hardware and computational photography advances further, so will the practical uses for depth cameras in future smartphones.
They’re an exciting development opening up new experiences previously only possible with substantially more expensive professional 3D scanning and measurement equipment.
