February 28

Software Update

After receiving the Intel Neural Compute Stick 2, our team benchmarked the device using two popular object detection models: MobileNet v1 SSD 0.75 depth, and MobileNet v2 SSD model trained on the Common Objects in Context (COCO) dataset with an input size of 300 x 300. These models were selected as they were already available in the OpenVINO Intermediate Representation. The Intel Neural Compute Stick 2 requires the OpenVINO toolkit and thus models must be converted beforehand to the appropriate format. This process requires a specific distribution of Linux and is challenging, so pre-existing models were chosen to save time. Additional models may be converted if the need arises.

The performance of the Intel Neural Compute Stick 2 exceeded the Raspberry Pi CPU as expected. The average inference time for MobileNet v1 over 10 runs was 51.95ms. For MobileNet v2 it was 80.41ms. The later model produces more accurate detections, and thus there is a tradeoff for speed. Compared to the 134.54ms inference time using the CPU only, this represents a large improvement, and theoretically would allow us to meet our 10 Hz specification. However, this would require installing 4 Intel Neural Compute Stick 2 devices inside our system, which is not mechanically feasible. Our hope is to install 2 devices in the front module and settle for slightly less than 10 Hz frequency to preserve the accuracy of detections.

Additionally, our team performed testing with the USB cameras that will be used for blindspot detection. The front and rear facing cameras have a small blindspot which is monitored by a single USB camera on each side. These units are not arranged in a stereo configuration and will use relative vehicle size as an indicator of depth. The resolution and framerate of the cameras were tested and both satisfied our expectations. The connector was found to be poor quality, but can be soldered directly to the board to avoid signal losses.

While testing the USB cameras we also had an opportunity to test our wrapper for the OpenCV VideoCapture class, used to interface with the cameras. This class only provides methods to retrieve the last frame in a queue, meaning that bufferless frames (i.e., discard frames until they are required) is not possible. Using multithreading, we implemented a wrapper around the VideoCapture class which polls the camera and keeps only the most recent frame using a queue. This works well and allows us to grab frames in real time while doing CPU intensive computation in-between.

Lastly, we developed and tested our stereo calibration software. The mechanical team mounted a set of stereo cameras inside a prototype of the rear module. A standard calibration procedure for stereo cameras was followed. Several photos of a checkerboard were taken in all regions of the image frame, and at different angles. The intrinsic and extrinsic camera parameters were computed. From these, the depth map, which provides the estimated depth on a pixel-basis, can be derived. This is a fundamental component of our vehicle positioning system. Subsequent calibration of the front-facing stereo camera unit will be performed when the front module is assembled.