Protocol and Interface Agnostic Universal D2D Controller for HPC and Chiplets
Overview
Demand for die-to-die and chip-to-chip interfaces has been growing steadily in the past few years due to new applications in cloud/data centers, AI (training and edge applications), and High-Performance Computing (HPC). The demand is driven by the requirements of high throughput, low latency and low power in these applications. Advances in packaging technology are further helping the adoption of heterogeneous systems with chiplets/known-good-die(KGD) assembly to create solutions that require the evolution of a new kind of interface popularly known as “Die-to-Die” interfaces.
Figure 1 – D2D Interface Use Cases
Interface Types: Serial vs Parallel Approach
Driven by high-performance networking applications, SerDes has been the primary choice for connecting multiple chips and dies as switches and routers are at the forefront of the bandwidth requirements. For homogenous applications where multiple dies are on the same process nodes, USR/XSR SerDes are widely used to connect and scale the performance. However, this comes with a penalty of higher latency and power for each of the SerDes links as they are primarily PAM-4 based at 56G/112G speeds. The new trend is to connect dies using wider parallel IOs. These IOs are single-ended, similar to HBM and DDR memory technologies, and usually forward the clock to eliminate using power-hungry clock and data recovery circuits. With parallel IOs one can expect almost half the latency and power as compared to SerDes interfaces. There are many consortiums and standards working on parallel IOs such as OpenHBI, BoW, AIB, etc. OpenFive is a proponent of open interfaces and is actively involved in standardization efforts.
Packaging Choices: Interposer and Organic Substrate
Depending on the overall throughput requirements and the number of signals connecting the dies, one can use the interposer (various types) or organic substrate to stitch the dies together. Silicon interposers and other technologies allow more signal density, whereas organic substrates provide cost-effective packaging options. Organic substrate options are particularly attractive to chiplets with heterogenous dies (using different process nodes). There are many new advances in this area which we will cover in a future blog.
In this blog, we will talk about OpenFive’s recently launched D2D Controller. It sits on top of the D2D PHY (XSR SerDes/Parallel PHYs). The D2D Controller provides seamless connectivity to both types of interfaces.
D2D Controller
Figure 2 – D2D Controller Block Diagram
The D2D Controller is essentially a link layer of the protocol stack. As compared to other protocols such as PCIe and Ethernet, this controller has the least overhead in terms of area and latency primarily with the intent to transfer data across two dies reliably in a few nanoseconds versus tens to hundreds of nanoseconds.
On the system side (left), the D2D Controller has Client Adaption Logic. This logic is configurable, and the customer can use either AXI-4, GMII, CXS, TileLink, or a native user interface, depending on the system’s needs. The AXI bridge supports master/slave interfaces with customer-defined widths and frequency.
The Protocol Layer takes care of end-to-end error-free delivery of the data. It has optional flow control to manage and back-pressure the traffic. This block also has optional re-transmission that stores user-defined packets based on the latency of the link.
Finally, the Framing Layer creates a seamless interface to industry-standard SerDes of various reach (LR, MR, VSR/XSR, etc.). The interface is also compile-time configurable to attach to upcoming parallel interfaces such as BoW, OpenHBI, and AIB. The framing layer also includes optional Forward Error Correction (FEC) blocks. Some of the lightweight FEC engines can improve the BER to below 1e-17.
Being configurable and agnostic to both the user interface and PHY interface is the key feature of this IP that allows SoC designers to not worry about the complex protocols.
For more information about OpenFive’s Protocol and Interface Agnostic D2D Controller, click on the following links: