To monitor the environment and navigate the real world, the robot should be able to acquire images and environmental measurements under different background lighting conditions. In recent years, researchers and engineers around the world have been working to develop more and more advanced sensors to integrate into robots, surveillance systems or other devices that can sense their surroundings.
According to Memes Consulting, researchers from Hong Kong Polytechnic University, Peking University, Yonsei University and Fudan University have recently developed a new type of bionic vision sensor that uses a mechanism that artificially simulates retinal function and can be used in a variety of Data were collected under light conditions. This bionic vision sensor is based on phototransistors made of molybdenum disulfide.
Photo of the biomimetic vision sensor array (left); schematic structure of the vision sensor unit and optical microscope image (right)
"Our research team started work on optoelectronic memory five years ago," said Yang Chai, one of the researchers who developed the vision sensor. "This emerging device can output light-dependent and history-dependent signals, enabling image integration. , Weak signal accumulation, spectral analysis and other complex image processing functions, the multi-functional integration of sensing, data storage and data processing into one device."
In 2018, Yang Chai and his colleagues published the first paper on optoelectronic memory, in which they introduced a resistive switching memory device that can perform light sensing and logic operations. A year later, the team introduced a new type of photoresistive random access memory with three different functions. Specifically, the new device can sense the environment, store the information in memory, and perform neuromorphic visual preprocessing operations.
"We studied the concepts of near-sensor and in-sensor computing paradigms in 2020 and published our views in the field." Yang Chai continued, "This new research on biomimetic vision sensors builds on our On top of all previous efforts."
The intensity of ambient natural light varies widely, with a total range of 280 dB. When the human retina senses external light signals, it adjusts the light sensitivity of its photoreceptors (i.e., rods and cones) according to the strength of the signal. This ultimately enables the human eye to gradually adapt to varying levels of lighting, allowing it to see clearly in both dark and bright environments, an ability known as "visual adaptation."
“For example, when you enter a dark cinema from a bright hall, you can hardly see anything at first, but after a while in the cinema, it becomes easier to see things,” explains Yang Chai. "This phenomenon is called scotopic adaptation. Conversely, if you go from a dark movie theater to a sunny outdoors, you'll feel very dazzled at first, and it takes a while to get used to seeing what's going on around you. The process The opposite of dark adaptation is called photopic adaptation."
The main goal of Yang Chai and his colleagues' recent work is to build a vision sensor inspired by the structure and function of the human retina. To do this, they first started by studying the human retina and then tried to design perceptual strategies that would allow them to artificially simulate visual adaptations.
State-of-the-art image sensors based on CMOS technology typically have a limited dynamic range of 70 dB. However, this dynamic range is much narrower than the lighting range of natural scenes (280 dB).
"To achieve visual perception over a wide range of light intensities, researchers have explored the use of controlled optical apertures, liquid lenses, adjustable exposure times, and denoising algorithms in post-processing," said Yang Chai. "However, these Methods often require complex hardware and software resources."
Dark and light adaptation of biomimetic vision sensor arrays. (a) Schematic of the dark adaptation test: recognition of low-light images using an 8 x 8 pixel array in a dark environment. (b) Schematic diagram of light adaptation test: recognition of high-illuminance images using an 8 x 8 pixel array in a bright environment. (c) Dark adaptation process to identify the "8" pattern. (d) The photoadaptation process to identify the "8" pattern.
Optoelectronic devices with light-adaptive vision and broad sensing range at sensory terminals could have very valuable applications. For example, they can help improve the performance of computer vision tools, reduce the hardware complexity required to build robots or other sensing systems, and improve the accuracy of image recognition systems.
Although, other research teams have developed optoelectronic devices that can adapt to different lighting conditions in the past. However, most of the previously demonstrated devices can only mimic the light adaptation mechanism of the retina. The dark adaptation process has so far proved more difficult to simulate.
"There is still a long way to go to fully replicate the visual adaptation function of the retina," explains Yang Chai. "To achieve this, we designed a phototransistor-based vision sensor using ultra-thin semiconductors that can The degree of dark adaptation and light adaptation in the same device was controlled by applying different gate voltages. In this way, we simulated photoreceptors and horizontal cells in the retina and successfully achieved a sensing range of 199 dB. Vision-adaptive devices in biomimetic sensors."
Artificial simulation of photoreceptors and horizontal cells in the retina for visual adaptation (dark adaptation and light adaptation)
The biomimetic vision sensor developed by Yang Chai and colleagues is based on phototransistors made of an ultrathin semiconductor material known as molybdenum disulfide. The phototransistors they used have multiple charge trap states that can trap or release electrons within the channel at different gate voltages.
Ultimately, these states allow researchers to dynamically tune the conductance of their devices. This, in turn, allowed them to artificially simulate the dark- and light-adaptive mechanisms of the human retina, thereby expanding the range of their sensor's perception of different lighting conditions.
"Our bionic vision sensor has several advantages and features," said Yang Chai. "First, the visual adaptation function is implemented in a single device, which greatly reduces the footprint. Second, multiple functions can be implemented on a single device. , including light sensing, memory, and processing. Finally, dark and light adaptation under different light intensities can be achieved by controlling its gate voltage."
Yang Chai and his colleagues evaluated the bionic vision sensor in a series of tests and found that it could effectively mimic the function of the human retina, achieving remarkable results in both dark and light adaptation. Furthermore, it has a significantly higher perceptual range (199 dB) compared to previously proposed solutions.
"Our vision sensor can enrich machine vision functions, reduce hardware complexity, and achieve high image recognition efficiency," said Yang Chai, "All these advantages are available in areas such as autonomous driving, face recognition, and industrial manufacturing in complex lighting environments. great application prospects.”
In future studies, the researchers plan to further improve the performance of the vision sensor, while also using it to fabricate large-scale systems consisting of sensor arrays. Ideally, they want to build this sensor array on a flexible or hemispherical substrate to enable a wider field of view.
"One area that needs improvement is the adaptation time of our vision sensor, as it is still not enough to support machine vision applications." Yang Chai added, "Our goal is to reduce the adaptation time to the microsecond level. In addition, the vision sensor array scale Further improvements are also needed. Our near-term target for array size is greater than 100 x 100 pixels. Finally, the heterogeneous integration of vision sensors and post-processing units, including silicon-based control circuits, is a very important step toward practical applications.”
GMKJ Technology is deeply engaged in healthy and smart light sources, providing a full range of UVA UVB UVC LED, infrared IR LED VCSEL products and solutions to the market. It has hundreds of high-quality partners in domestic and foreign markets to jointly promote the use of light technology to create a healthy and smart life. .
