Publicreport_wp35


Public Report Of The Project's Results: WP3 - IC Design and WP5: Validation

WP3: IC Design and WP5: Validation

Because the design and validation of the CMOS circuits are so closely linked, they are reported together.

1. Advances in artificial silicon retinas

SeeBetter accomplished significant advances in improving the performance of artificial vision sensors based on retina organization and function. In particular, the SeeBetter collaboration between UZH and imec has resulted in the first back-side illuminated (BSI) dynamic vision sensor silicon retina. This BSI sensor combines the latest DAVIS pixel technology with BSI to improve the optical sensitivity by an estimated 5x compared with front side illumination (FSI), even when the FSI uses microlenses to focus light on the photodiode. This improved sensitivity will allow the DAVIS sensor to be used under lower-light conditions and will reduce image blur for both the dynamic and static imaging capabilities, which will be valuable for visually guided robots, which are a main target of DAVIS technology.

In the first phases of the project, UZH designed two prototype DAVIS sensors: The first (SBRET10) was designed and fabricated in Taiwanese UMC 0.18um technology. The second prototype (SBRET20) was designed and fabricated in Israeli Towerjazz 0.18um CIS technology. In the last phase of the SeeBetter project, UZH designed 11 different ICs for tapeout on Towerjazz CIS technology on 20 July 2014 (Figure 1). The main target design was design #4, a version of the 346x280 pixel DAVIS sensor with bonding pads targeting the BSI process flow at imec.



Figure 1: Reticle floorplan for SeeBetterFinal dedicated wafer run.

These wafers were received from fabrication on 27 Oct 2014 (Figure 2).





Figure 2: Received silicon wafers.

By 26 Jan 2015 a single FSI wafer with RGBW color filters was diced and packaged (Figure 3).



Figure 3: UZH team celebrating receipt of packaged designs.

Testing of the designs then started, using a completely redesigned USB 3.0 SuperSpeed motherboard, with daughter boards for each design.

By March 9, 2015, it was confirmed that at least one design was fully functional (Figure 4).



Figure 4: Initial testing of CDAVIS.

On 7 April, 2015, post-processed BSI wafers were received by UZH from imec and sent for dicing and bonding. The backside wafer only shows the pad openings (Figure 5).



Figure 5: BSI design image. Only the backside pad openings are visible.

A side-by-side comparison of the BSI vs FSI optical performance of the identical CMOS design (Figure 6) showed that with the same conditions, the BSI design was about 15 times more sensitive to light. Since the FSI design came from the RGBW wafer, the FSI design would be expected to be about ⅓ as sensitive due to the color filters. The additional factor of 5 in sensitivity would be explained if the effective pixel fill factor were 100% rather than the 20% of the drawn photodiode fill factor.



Figure 6: Side by side comparison of BSI vs. FSI with RGB color filters.

As of the final review of the SeeBetter project (June 4th, 2015), the testing status of all designs was as listed in the table below.


Status of all wafer designs as of end of SeeBetter project
Design Test status
240x180 SBRet21 fail-safe mono DAVIS Fully functional and now in limited production as Davis240C camera with 10 bare-board cameras built and used in workshops and internal projects.
346x260 DAVIS with CFA Fully functional.
346x260 SBRetFinal BSI DAVIS Verified fully functional, with improved sensitivity compared with RGB FSI SBRetFinal design.
640x480 RGB DAVIS Fully functional and live demonstration shown publicly during presentation of paper at ISCAS 2015 in Lisbon on 25.5.15 by C. Li. Initial publications at ISCAS 2015 and IISW 2015.
128x128 FSI DAVIS Verified fully functional.
640x480 VGA DAVIS Verified functional. Waiting for 1" optics to test optical performance and possible AER bus bandwidth limitations.
192x192 sensitive DVS/DAVIS Verified functional, with initial testing complete. Coupling from APS to more sensitive DVS is larger than was expected, but can be worked around by selectively ignoring DVS events during electronic exposure clocking. High pass pixel is functional and removes background activity effectively.
>64x64 exptl. Correlation Filter Still in testing phase. IC is alive but might have logic error in event decoding.
346x260 "failsafe" DAVIS (No Salicide block, no deep pwell, conservative pad frame grounding scheme.) Possible short in bonding pads (not yet debugged)
Low power 0.5V binaural cochlea PCB fabricated and assembled, and waiting for logic and software design.
4 ear AER-EAR cochlea PCB fabricated and assembled, waiting for PCB testing, logic and software design

Below you can see the first videos taken with the RGB DAVIS chips.


2. Software infrastructure

jAER is an open-source project started in 2007. In 2013, we also obtained DNS domains dedicated to this software project: All of www.jaerproject.net, www.jaerproject.org, or www.jearproject.ch are directed to the home of the project on SourceForge.

The particular Java classes for software control of SEEBETTER ICs are in the Java package rooted at eu.seebetter.ini.chips, and the firmware and logic designs for the PCBs are open-sourced at the subversion repository in the jAER folder devices, in particular in the subfolders firmware and logic.

The major developments by the end of the project were:

  1. We moved from using the closed-source commercial windows USB driver Thesycon and full-custom kernel-mode drivers for linux to the open-source multi-platform library libUSB. libUSB is now stable enough at least for experimental purposes and the move to libUSB hugely simplifies supporting the sensors on multiple OSs (Windows, Linux and MacOS) and with multiple host side APIs (jAER and cAER). As of July 2015, the DAVIS sensor is used in all 3 OSs with no changes to build or runtime configuration needed, although peculiarities in Apple’s OS X and the complexity of supporting OpenGL graphics on multiple GPUs (in particular, bugs in integrated Intel graphics GPUs) still cause niggling problems.
  2. For the last 8 years we had been using the Cypress FX2 USB interface IC. With the development of USB 3.0 SuperSpeed, we now developed a new hardware interface to our cameras based on the next-generation Cypress FX3 IC. This FX3 USB device supports SuperSpeed which increases maximum throughput by a factor of about 10 over USB 2.0 HighSpeed mode. Another advantage of USB3.0 is that it removes host side polling, potentially allowing even further decreases in latency below the minimum 125us allowed by USB2.0 HighSpeed mode.
  3. We had also been using a CPLD reconfigurable logic device on our cameras to interface between the vision sensor IC and the USB IC. Although CPLDs are low power and inexpensive, they have a limited number of registers and embedded block RAM memory. We are now using a Lattice FPGA for our cameras. This change greatly expands our ability to do post-sensor computation in logic, i.e. to remove background noise and hot pixels. It also allows us to integrate new sensors like inertial measurement units more easily into our cameras.
  4. We had been using firmware and logic design on the cameras that had accreted over many generations of designs starting around 2008, when we first incorporated a reconfigurable logic device on the camera. Over the past two years, the firmware and logic was rewritten from scratch to enable much simpler and modular support of numerous camera devices sharing a common underlying logic IP base. We have gradually been integrating new functionality such as noise filtering and ROI readout of the APS frames to this new logic framework.

2.1. The DAVIS cameras

All the latest generation SEEBETTER DAVIS cameras starting with SeeBetter10, SeeBetter20, SBRet10, SBRet20 and now all the designs on the wafer run have a combined DVS AER event stream and APS image frame output. The complexity of dealing with these new cameras results from the combination of these two very different data types and the addition of IMU to the camera.

2.2 User-friendly interface to the SEEBETTER cameras

The configuration of the SEEBETTER cameras has become very complex compared to the 128x128 DVS camera developed in the CAVIAR project. A much more configurable on-chip bias generator, other on-chip configuration bits, off-chip configuration of the USB controller, configuration of the logic designs on the PCB’s CPLD, configuration of the rendering of frame-based images etc., have necessitated a major effort in software engineering. The control of the frame-based APS capture and output rendering, the control of the DVS event rendering, and the control of DVS pixel parameters have been encapsulated into a simplified "user-friendly" control panel (Figure 7). This control panel hides a great deal of configuration that would be cryptic to an untrained user. The control panel allows control of Image Sensor parameters, DVS parameters, and IMU parameters. Additional panels control the auto-exposure controller properties and the detailed video display properties. The Chip configuration panel and the Debug Output MUX control enabled advanced testing modes for debugging and measuring internal analog and digital signals.



(a)



(b)

Figure 7: (a)User-Friendly control panel for Davis346, the 346x244 pixel design on the Tower wafer run. (b) Compare
(particularly the number of expert tabs) with CAVIAR DVS128 control panel.


3. USB 3.0 Camera Interface

To take advantage of the latest technologies, a large effort over the past two years was spent on developing a USB 3.0 SuperSpeed interface to the SEEBETTER cameras. SuperSpeed offers a maximum of 5Gbps throughput compared with 480Mbps of USB 2.0 HighSpeed mode. In addition, USB 3.0 is also very favorable for low-latency applications because the basis of USB 2.0 of using host-controller polling has been done away with; now all host controller requests are handled by direct interrupts and DMA memory transfers. This can decease USB latency even lower than the 125 μs minimum polling interval of USB 2.0.

Two generations of USB 3.0 camera boards were developed (Figure 8, Figure 9). The latest generation board specifically targets testing the IC designs on the Tower dedicated wafer run.



Figure 8: USB 3.0 Motherboard layout. Daughter boards plug into the connectors at left and right. The legacy CAVIAR connector at bottom allows debugging AER data transfers. The USB 3.0 device controller and USB connector at top left interface using a USB micro cable. The Lattice FPGA sits under the daughter board. The motherboard includes a SATA connector and 2MB of SRAM for future embedded FGPA signal processing.



Figure 9: Photos of USB 3.0 motherboard used to test the Tower wafer designs.