|
| Projects | Chip Gallery | People | Publications | PhD Study | Contact |
The SCAMP Vision Sensor integrates a massively parallel SIMD processor array into the pixels of the image sensor device. Unlike a conventional image sensor, it does not output raw images, but rather the results of on-sensor computations, for instance a feature map, optic flow map and/or address-events describing locations of pixels of interest. The device is fully programmable, to execute a variety of vision algorithms. As early vision computations are done entirely on-sensor, high speed and low-power consumption of the entire camera system is obtained, enabling new embedded vision applications in areas such as robotics, VR, automotive, toys, surveillance, etc.
On this page, you can find out more about the SCAMP-5 vision chip architecture and implementation, the smart camera development kit based on this device, and see the demonstrations of various algorithms implemented on the device.
In a conventional machine vision system, images are acquired, digitised, and transmitted off camera, frame after frame. The processor system then needs to keep up with the resulting pixel avalanche, with large amount of data continually exchanged between the sensor, processors and the memory, to implement required vision algorithms. This is inefficient. Our approach, inspired by the biology, is based on moving the computation from the processor system to the sensor itself. Essentially, we embed a parallel processor array into the image sensor. The approach has a number of advantages:
The SCAMP-5 vision chip contains a 256x256 processor array, operating in a SIMD mode, with one processing element per image pixel. The processors are simple albeit fully software-programmable entities, comprising local memory, ALU, control and I/O circuits. Mixed-mode datapath allows the execution of some operations in the analog domain, achieving low-power and bypassing the need for A/D conversion. The results of computation can be read-out as grayscale images (mostly for debugging purposes) and binary images. If the results of computations are sparse (e.g. when points of interest are identified), an address-event read-out allows to minimise the bandwidth between the sensor chip and the rest of the system.
Each of the 65,536 processors contains six analog registers (capable of storing a real-valued variable, e.g. a gray-level pixel value), 13 bits digital memory, and circuits for performing arithmetic and logic operations on the local data, as well as neighbourhood operations. It also contains hardware accelerators for asynchronous binary flood-fill and spatial gray-scale low-pass filters, as well as control and I/O circuits. The processing elements execute instructions broadcast to the entire array from a common controller, with local autonomy provided via an ativity flag. The photosensor circuit is tightly coupled to the processor.
The chip is a custom mixed-mode integrated circuit implemented in 180nm CMOS technology. It can carry out over 500 GOPS (billion operations per second) at power consumption below 1W. We have demonstrated processing speeds up to 100,000 fps (frames per second), although a typical high-speed operation in a robotic application may be at 1000s fps or less. The power depends on the processing load, i.e. the algorithm and frame rate, and can be below 1 mW for simple low frame-rate operations.
We have integrated the SCAMP chip into a reference smart camera design, and have provided software tools that allow the development of SCAMP algorithms for various applications. The SCAMP camera includes a USB interface for development, as well as SPI and GPI/O interfaces for embedded applications. It also includes an application processor (Cortex-M4) an inertial measurement unit (IMU) for custom applications.
The chip is programmed using a C based program flow, using provided libraries that allow parallel processing kernels to be executed on the processor array. The SCAMP processor emulator, running on a PC (Windows), allows debugging of the programs that will be executed on the SCAMP chip. The SCAMP kernel code can be stepped-through, with full insight into the contents of all registers on the device.
(see Martel et al. 2016)
(see Martel et al. 2018)
(see Chen et al. 2017)
Earlier developments of the SCAMP chip technology:
|
|
