An ultra-fast image sensor with a built-in neural network has been developed at TU Vienna.
Automatic image recognition is widely used today: There are computer programs that can reliably diagnose skin cancer, navigate self-driving cars, or control robots. Up to now, all this has been based on the evaluation of image data as delivered by normal cameras – and that is time-consuming. Especially when the number of images recorded per second is high, a large volume of data is generated that can hardly be handled.
Scientists at TU Wien therefore took a different approach: using a special 2D material, an image sensor was developed that can be trained to recognize certain objects. The chip represents an artificial neural network capable of learning. The data does not have to be read out and processed by a computer, but the chip itself provides information about what it is currently seeing – within nanoseconds. The work has now been presented in the scientific journal “Nature”.
Neural networks are artificial systems that are similar to our brain: Nerve cells are connected to many other nerve cells. When one cell is active, this can influence the activity of neighbouring nerve cells. Artificial learning on the computer works according to exactly the same principle: A network of neurons is simulated digitally, and the strength with which one node of this network influences the other is changed until the network shows the desired behaviour.
Typically, the image data is first read out pixel by pixel and then processed on the computer. The Vienna researchers, on the other hand, integrate the neural network with its artificial intelligence directly into the hardware of the image sensor. This makes object recognition many orders of magnitude faster.
The chip is based on photodetectors made of tungsten diselenide – an ultra-thin material consisting of only three atomic layers. The individual photodetectors, the “pixels” of the camera system, are all connected to a small number of output elements that provide the result of object recognition.
The technology can be usefully applied wherever extremely high speed is required.
News Source: TU Wien
