Toyota has unveiled technology for cabin awareness that uses millimetre-wave 4D radar to detect and track occupants, both human and animal. If any occupant is left behind, it can alert the driver. The new technology can support better assistance of disabled and elderly passengers, both in saloon and patient transport vehicles. It is also designed to improve the safety of child and pupil transport buses.

One application for the new technology is preventing passenger or pet harm during extreme weather, such as heatstroke or exposure to extreme cold.

The technology is being tested in prototype Toyota Sienna minivans and in May Mobility Sienna AutonoMaas shuttle buses.

The technology can sense micro movements, such as a heartbeat, motion and respiration of occupants across three seating rows, the cargo area and footwells.

It classifies occupants according to size, posture and position – supporting advanced safety applications. The platform can sense even if the occupant is covered with a blanket, a scenario which a passerby would be unable to see the occupant.

According to Toyota, it differs from other technologies on the market such as weight sensors, that can be prone to false alerts and mis-detects, cameras limited by blind spots or other radar systems with a limited range of passenger detection.

The cabin awareness technology uses a 4D imaging radar sensor mounted out of sight above a vehicle’s headliner that can detect presence of a life form in the vehicle, even after a driver exits.

The new technology is being trialed through Toyota’s partner May Mobility. The autonomous-vehicle company is testing the technology in its fleet of Toyota Sienna AutonoMaaS (Autonomous Mobility as a Service) minivans at its headquarters in Michigan and will soon begin testing in public AV deployments in Arlington, Texas, and Ann Arbor, Michigan, with additional deployments coming in late 2022. Recently, May Mobility announced that a portion of the fleet will be modified to become ADA-compliant and accommodate passengers in wheelchairs.

In the future, one possible application for autonomous shuttles is alerting parents when their children complete a ride. Conversely, a shuttle may delay driving to its next stop if it senses someone is still in the vehicle upon reaching its destination.

As currently designed, the Cabin Awareness concept provides a number of warnings to help alert the driver, and passersby, to check the back of the vehicle if a living being is detected. A warning light signals first on the instrument cluster followed by a horn honking, then emergency lights flashing. Following all the early warnings, the owner may get a notification on their phone through the Toyota app as well as text messages.

If programmed, Cabin Awareness can send alerts through smart home devices or send text messages to designated emergency contacts assigned by the primary user. Further alert options include having Toyota’s Safety Connect emergency assistance system contact first responders through the integrated SOS vehicle function.

The team is continuing to explore additional notification alerts such as vehicle-to-vehicle (V2V) communications and the incorporation of smart infrastructure retail signage to further expand the reach of this alert.

“Toyota Connected’s talented software engineers and data scientists are leveraging cutting-edge technology to bring innovation and advanced technologies to customers’ vehicles,” said Zack Hicks, CEO and president, TCNA and executive vice president and chief digital officer, TMNA. “We are extremely proud of our efforts to take this idea from the drawing board to a full-blown concept, and, hopefully, developing a technology that has the potential to save lives.”

“The key difference with this system is the improved resolution and accuracy, full-cabin detection and scope of functionality Cabin Awareness provides,” said Simon Roberts, managing engineer, TCNA, who has spearheaded the development process. “With the precision of these sensors, we’re designing Cabin Awareness with the aim of reducing false positives and false negatives.”

“We are excited to explore and leverage new technologies, connected intelligence and existing safety platforms, with the ultimate goal of reducing child fatalities in hot cars. This Cabin Awareness concept has the potential to do exactly that,” said Brian Kursar, chief technology officer, TCNA and group vice president, chief technology officer and chief data officer, Toyota Motor North America (TMNA). “I’m so proud of the Toyota Connected and TMNA’s Connected Technologies teams for developing this concept and believe in the talent and dedication of our engineers.”

“May Mobility was founded with a mission of making roads safer through autonomous technology – and we’ve kept safety as our top priority as we’ve advanced our technology and worked with key strategic partners like Toyota,” said Edwin Olson, CEO, May Mobility. “As we move closer to driver-out operations next year and continue to scale our global business with more public AV deployments, technology like the Cabin Awareness concept is essential for our riders.”

The idea for Cabin Awareness was born in 2019 out of the inaugural Toyota Connected Hackathon, a 36-hour innovation event that challenged teams of software designers and engineers to ideate, develop and test real-world solutions. This winning idea quickly grew support following the Hackathon, allowing Roberts to assemble a team of engineers and further develop the technology.

The Cabin Awareness concept also allowed Toyota Connected to evaluate and develop hardware for the first time in addition to building innovative software systems and creating new processes. The Toyota Connected team evaluated millimeter-wave suppliers and began working with technology provider Vayyar Imaging’s in-cabin monitoring platform.

Vayyar’s single chip 4D imaging radar has the capability to monitor an entire vehicle cabin with a single sensor. The platform provides dense point cloud imaging, enabling developers to develop a wide range of applications and deploy them remotely using over-the-air software updates.

Inspiration for the Cabin Awareness concept came from microwave radar technology created by NASA’s Jet Propulsion Laboratory to support underground rescues after a 7.8-magnitude earthquake struck Nepal in 2015. In this application, NASA engineers and rescuers were able to detect human breathing and heartbeats under more than 30 feet of rubble, helping responders know where to dig holes.

“NASA’s use of radar technology was inspiring,” said Kursar. “The idea that you can listen to heartbeats using contact-less technology opens up new possibilities to give Toyota the potential to produce a service that is beneficial to the evolution of our in-vehicle services.”

“The Cabin Awareness programme is an innovation first for Toyota Connected and Toyota in so many ways,” said Roberts. “The most important goal we’re working on, though, is to build technologies that empower occupants and give them greater peace of mind.”