Intel RealSense shutdown reminiscent of Kinect & Tango

The Robot Report

Intel discontinuing the RealSense line is a bombshell for roboticists. For the past six years, RealSense has been a go-to for industry and academia for low-cost, high-quality depth sensing. It found its way into the production design for many now-deployed robot platforms, like the recently announced Corvus warehouse inspection drone and Avidbots’ Neo 2 automated floor cleaner.

It was featured in many Intel keynotes as a cornerstone product for the company’s future. And it was a very active project, with new sensors being released regularly, and updates to the RealSense SDK as recently as last week.

The original Microsoft Kinect. | Photo Credit: Microsoft

Surveying the depths of history

While the sudden announcement of its demise is a shock, the cancellation isn’t. After all, this has happened again and again in the world of 3D sensors.

Microsoft Kinect
One need not look too far back to recall the depth sensor that started it all – the original Microsoft Kinect. Upon its launch in 2010, roboticists seized upon it as a source for low-cost, high-quality depth sensing (sound familiar?) that could unlock sensing capabilities that were previously only available to companies and institutions that could afford multi-thousand dollar industrial 3D sensors. Co-developed by Israeli startup PrimeSense, the Kinect quickly found its way into academic and commercial robotic environments, being employed for tasks like obstacle avoidance, object detection, and skeletal tracking.

With the backing of Microsoft, the Kinect seemed like a sure thing, a safe choice as part of a vision-enabled system’s final design. And then, suddenly, Apple snapped up PrimeSense for $360 million in November 2013. The original Kinect became history. Roboticists who had relied on the Kinect – and, more importantly, Microsoft – were suddenly left in the lurch.

Google Project Tango

Google’s Project Tango. | Photo Credit: Google

Project Tango
Just about a half year after Apple’s acquisition of PrimeSense, another promising depth sensing platform emerged from a huge tech company: Google’s Project Tango. Like the Kinect, Project Tango promised low-cost, high-quality 3D sensing along with a robust developer SDK to rapidly create applications that used depth (and, interestingly, the very first Project Tango developer devices used PrimeSense chips as part of their design).

Adoption from the developer community was swift, and Google subsidized and promoted some of the more promising applications, including a collaboration with NASA. Soon, Tango-specific handsets were being launched, including one co-developed with Intel that included a built-in RealSense. And then, after just three years, Google announced it was winding down Project Tango. It would be joining the Kinect in the ranks of defunct depth sensing platforms.

What, then, happened in the wake of the cancellations of the Kinect and Tango? What lessons can we learn from those examples that may apply to the RealSense platform? There are a few.

Future for RealSense and its users

When Kinect and Tango were cancelled, there were a few distinct reactions by their user bases: hoarding, switching, and innovating. Here at Tangram Vision, we believe these will be the paths followed by RealSense users as well.

Hoarding: Integrating a depth sensor into a complex, vision-enabled platform is no small feat. For many companies that had to do so before the advent of plug-and-play sensor management platforms like the Tangram Vision SDK, this process could take as long as six to 18 months.

Switching: Many of the more forward-thinking companies that had come to rely on Kinect and Tango decided to bite the bullet and simply switch to a new sensor that had guaranteed supply. For many, the simplest option was to switch to the original Structure Sensor by Occipital, which included similar components to the Kinect and was the closest sensor possible to a direct replacement. But many others chose to redesign their depth sensing systems entirely and integrated a completely different sensor — ironically, in many cases, that different sensor was an Intel RealSense.

It’s very likely that many companies that currently rely on RealSense will choose to redesign their products to incorporate a different 3D sensor. Many engineering teams, no doubt, will have a sense of dread that whatever sensor they choose to replace RealSense will ultimately suffer the same fate.

Further down in this article, we’ll explore a few of the alternatives, along with our best guess as to the supply stability of the platform and the financial stability of the company behind the sensor.

Innovating: The proliferation of depth sensing integrated into mobile handsets, advanced driver assistance systems (ADAS) and other rapidly growing markets has allowed optics component manufacturers like Sony, ST Micro and OmniVision to offer a wider range of off-the-shelf optics for those companies that want to “roll their own” depth systems. Indeed, the components required to build a passive stereo depth sensing system are now widely available and inexpensive, yet require a talented engineering team (including hardware, software, and firmware engineers) to pull it off. This path is very real – at Tangram Vision, we’ve been surprised at the number of robotics teams we’ve spoken with who have chosen to develop their own bespoke depth systems.

The Structure Core by Occipital. | Photo Credit: Occipital

Alternatives to RealSense

RealSense was an excellent choice for near- to mid-range depth sensing for tasks like obstacle avoidance, object detection, and depth information for SLAM systems used for robotic navigation. It was also widely used in the current crop of machine-learning driven pick-and-place robotic systems used in supply chain contexts. Fortunately, there are many alternatives. For the sake of brevity, we’ll note what we believe to be the best direct replacement options.

Structure Core by Occipital
The Structure Core features an optics and sensing package that is very much like that which was found in the Intel RealSense D435i: a pattern projector, a frequency-matched infrared camera, an RGB camera, and an onboard IMU. From a range and field of view perspective, this sensor is a near match for what the D435i offers. Now entering its 13th year, Occipital has raised nearly $50 million from investors, and appears to retain stability to continue offering its products well into the future.

(Editor’s Note: the author was formerly a member of the Occipital team).

Orbbec Astra
Orbbec’s Astra range is extensive, much like the RealSense range was. The Astra+ module nearly matches the Intel D435’s 10 meter range, as well as its relatively wide field of view. According to Crunchbase, Orbbec has raised $200M, a formidable war chest. It has also recently partnered with Microsoft to further expand the possibilities of depth sensing.

Luxonis OAK-D
Marketed in partnership with OpenCV, the OAK-D is a unique depth camera that incorporates Intel’s Myriad X vision processing chip for on-device AI capabilities. This places it in a slightly different class than other depth sensors that don’t offer this feature, but may also make it difficult to use as a direct replacement for RealSense. And, for those who are wary of supporting Intel in the wake of its sudden elimination of RealSense, there are no guarantees that Intel’s refocusing on its core PC chip business won’t spell the demise of Myriad X either.

The OAK-D was launched with a very successful Kickstarter campaign, which saw it benefit from the relationship with OpenCV. We don’t have much insight into Luxonis’ financial stability; however, in the field we are increasingly observing many companies experimenting with its sensors. We think the future is bright for Luxonis, assuming Intel continues to support Myriad X.

Tangram Vision’s Depth Sensor Visualization Tool. | Photo Credit: Tangram Vision

For a simple way to compare many currently available depth sensors to the Intel RealSense range of depth sensors, take a look at the depth sensor visualizer and comparison tool we created at Tangram Vision. There’s also a great thread on ROS Discourse looking at RealSense alternatives.

So what went wrong?

Put simply, RealSense never materialized into a big enough business to matter to Intel. It also barely contributed to Intel’s core business of selling chips. And that core business has been under pressure with Apple’s introduction of their bespoke M1 chip and AMD’s increasing competitiveness. Therefore, its future was always in question. Some might even say it’s surprising that Intel committed itself to the platform for the eight years it has been in existence.

The demise of RealSense will be painful for many, but beneficial for others. Competing platforms from Occipital, Orbbec, Mynt, Stereolabs and others will benefit as companies that were once reliant on RealSense look for alternatives. Sensor SDK companies like Tangram Vision will benefit from companies seeking a simpler integration path as they face adopting a new sensor into their existing and forthcoming product designs. And it’s likely that the market will reward Intel shareholders as it recognizes the benefit of Intel further refocusing on its core product lines.

Yesterday’s news was a shock, but this story is long from over. We’ll continue to see RealSense-integrated products; engineers will contribute to open-source repositories that support RealSense; and the proliferation of depth sensing due to RealSense will be a lasting legacy that all of us in the robotics industry should be thankful for.

Adam Rodnitzky

About the Author

Adam Rodnitzky is a co-founder and COO of Tangram Vision, which creates DevOps for perception. Adam was formerly General Manager of the Structure division for Occipital, and has worked in the computer vision industry for the past 15 years.

Prior to launching Tangram Vision, Rodnitzky was a mentor at StartX, a seed stage accelerator focused on commercializing startups connected with Stanford University, and VP of marketing and GM of Occipital.

The post Intel RealSense shutdown reminiscent of Kinect & Tango appeared first on The Robot Report.

Source: therobotreport

Leave a Reply