Enabling Edge AI Vision with RISC-V and a Silicon Platform
By Tom Simon, the Moderator at SemiWiki on 03-15-2021.
AI vision processing moving to the edge is an undeniable industry trend. OpenFive, the custom silicon business unit of SiFive, discusses this trend with compelling facts in their recent paper titled “Enabling AI Vision at the Edge.” AI vision is being deployed in many applications, such as autonomous vehicles, smart cities, agriculture, industrial & warehouse robotics, delivery drones, augmented reality, and smart retail & home.
Initially, it was only feasible to run AI vision processing in the cloud due to its capacity and processing power requirements. However, as billions of devices are deployed, processing solely in the cloud becomes unscalable. The network bandwidth requirements from billions of devices capturing high-resolution video from multiple cameras would exceed 5 petabytes per second!
On top of the logistical issues, cloud-based AI vision processing exacerbates privacy and latency issues. I for one would not want my self-driving car to rely on a wireless internet connection for making real-time driving decisions.
Associated with the push to move AI vision processing to the edge, there is large growth in the chipsets used to perform this processing. As shown in the chart, custom ASIC will become a dominant solution to provide the performance, power and functional advantage in AI Vision applications.
Edge AI Vision – Deep Learning Market
SiFive, OpenFive’s parent company, was founded on applying the ideas that have made software development so productive by eliminating the inefficiencies typically encountered. Yunsup Lee, co-founder and CTO of SiFive, participated in development of the RISC-V open- source instruction set architecture (ISA) in 2010. His vision has been to reduce the barriers for hardware design. The work of OpenFive is bearing fruit with impressive reductions in the cost, manpower and time needed to develop custom ASICs.
OpenFive’s use of SiFive’s RISC-V processor IP gives developers access to a well-developed set of operating systems, compilers, development packages and debugging tools. OpenFive’s AI vision platform is intended to speed up development of custom AI vision SoCs by providing multiple customizable subsystems that enable designers to focus on their key differentiators.
The platform contains just about every subsystem needed and can be tailored to eliminate unnecessary ones or to add specialized new blocks for specific applications. At the heart of the platform are SiFive’s multicore super-scalar Linux-capable U74 CPU complex, with support up to 8 cores and 2 MB of L2 cache. 32/64-bit LPDDRx with 6400MT/s provides gigabytes of high-bandwidth DRAMs required by edge AI applications. Powered by SiFive’s S21 embedded CPU, the platform management unit is responsible for power, boot and system health. The platform is secured by SiFive Shield that performs many security functions such as crypto, secure boot and key management. There is a vision subsystem with a vision DSP as well as MIPI interfaces. OpenFive includes an AI accelerator subsystem, of course, or users can add their own. Other customer specific accelerators can be added as well. The audio subsystem offers a wide range of features such as echo suppression and noise cancellation with its audio DSP. For visualization and graphics output, there is an integrated GPU. Naturally there is a wide range of high speed I/Os. There is even a die-2-die interface to improve performance with additional chiplets.
OpenFive’s business model allows their customers to engage with them during all stages of the ASIC development process. Customers can easily and quickly leverage OpenFive to complement their own skills, instead of needing to have in-house expertise in every one of the several dozen fields needed to produce a custom ASIC.
With open-source hardware and platform-based ASIC development, it is certain that we will see new products coming to market quickly that offer much hardware innovation. The rapid progress and growth that SiFive (and OpenFive) is experiencing are proof that there is pent-up demand for this. “Enabling AI Vision at the Edge” offers more details about OpenFive’s AI Vision platform. The paper is available for download on their website.