STMicroelectronics has launched a dual image sensor that promises full interior monitoring of cars, including both driver and all passengers. The new sensor can support passenger safety-belt checks, vital-sign monitoring, child-left behind detection, gesture recognition, and high-quality video and picture recording.

The new hybrid sensor features hybrid rolling and global-shutter imaging, extending basic driver monitoring to enable more adaptive and convenient vehicles for all occupants. The sensor, which is priced for mass market adoption, is designed to address a driver monitoring market that is growing at more than 10% per year.

The VD/VB1940 sensor combines the sensitivity and high resolution of infrared sensing with high dynamic range (HDR) colour imaging in a single component. It can capture frames alternatively in rolling-shutter and global-shutter modes. With 5.1-megapixels, it captures the high-dynamic-range (HDR) colour images needed for an occupant monitoring system (OMS) in addition to the high-quality near-infrared (NIR) images typically captured by standard DMS sensors. DMS uses NIR imaging to analyse driver head and eye movements in all lighting conditions.

The new automotive image sensor uses ST’s second-generation 3D-stacked back-side illuminated (BSI) wafer technology, which maximises the optical area and on-chip processing in relation to die size. This lets the sensor perform sophisticated algorithms locally for optimal performance in both colour and NIR imaging, saving power and relieving demand for an external co-processor.

Algorithms performed on-chip include Bayer conversion and HDR merging for optimal image-quality and frame rate. On-chip Bayerisation processing enables the user to reshuffle the colour pixels of the RGB NIR 4X4 pattern into RGGB format compatible with a wide range of SoCs. In addition, local processing also handles independent colour and NIR pixel-exposure optimisation for optimum image quality in both modes, as well as smart upscaling to maximise NIR image resolution by capturing extra NIR information from RGB pixels.

The embedded processor also manages enhanced cybersecurity to ensure privacy in connected-vehicle applications, including mutual authentication and pairing of camera and electronic control unit (ECU), as well as video-stream authentication.

The sensor captures NIR images in global-shutter mode, which permits synchronising with the infrared LED emitter to capture fast-moving scenes without motion blur. By also supporting rolling-shutter mode, which reads pixel data row by row, the VD/VB1940 provides optimum colour-imaging performance. Using the on-chip HDR-merge feature in rolling-shutter mode, the VB/VD1940 produces a 100dB full-resolution colour image.

Qualified to AEC-Q100, the VD/VB1940 is ISO 26262 compliant to support use in functional-safety systems up to ASIL-B.

“The DMS market is growing at a double-digit pace. ST’s new image sensor is set to push that forward by enabling brands to create new services and deliver even greater value for vehicle users, leveraging complete in-cabin monitoring that covers multiple occupants,” said Eric Aussedat, Executive Vice President, Imaging Sub-Group General Manager, STMicroelectronics. “Our market-leading DMS sensor provided the perfect launchpad to take the technology further and empower new innovations.”