How PCIe® Technology is Connecting Disaggregated Systems for Generative AI
David Kulansky, Director of Solutions Engineering, Alphawave Semi
Published by: www.pcisig.com
PCIe technology is set to be leveraged as an important component in the AI infrastructure marketplace. According to the “PCI Express Market Vertical Opportunity” report from ABI Research, the expected total addressable market (TAM) for PCIe technology in AI will grow from $449.33 million to $2.784 billion by 2030, at a compound annual growth rate (CAGR) of 22%. One emerging use case for AI is generative Artificial Intelligence or GenAI. GenAI is a type of AI technology that is used to produce content, including text, images, video, audio and more. As GenAI evolves, some unique challenges in GenAI applications are becoming clear, such as the need for low power, low-latency robust technologies to connect these systems together. Due to the continuing increase in complexity and scale of Large Language Models (LLMs), the most advanced generative AI models can’t fit on one GPU, one server, one rack, or even a single data center.
PCI Express® (PCIe®) technology offers numerous benefits for generative AI applications, since its inherent DNA is perfectly suited to enable disaggregated systems including distributed multiplication functionality of value for LLMs. In this blog, we’ll touch on how PCIe technology is used in generative AI today, how the PCIe technology features perfectly aligned with growing AI demands, and how the relationship between PCIe technology and AI will continue to evolve for future applications.
PCIe Technology Features Meet the Technical Demands of Generative AI
PCIe technology is a ubiquitous I/O interconnect that provides the structure to connect nodes together by enabling low-latency, low-power connections, and always ensuring backwards compatibility. PCIe technology connects the entire data center, creating a pooled resource of compute, memory, and storage to fit the unique and specific needs of generative AI applications.
As the need for higher data rates continues and the industry makes the switch from NRZ to PAM4 signaling, Forward Error Correction (FEC) becomes essential to maintaining reliability. PCIe addresses this for generative AI, and all low-latency applications, by utilizing FLIT (Flow Control Unit) Mode. FLITs help maintain the low latency of PCIe technology while still delivering low post-FEC error rates. Additionally, PCIe architecture includes Low Power Modes, which conserves energy when less data throughput is needed and allows for even greater savings when links are temporarily unused through L0p and L1 substates.
Digging further into the importance of low-latency benefits of PCIe, hardware coherency plays a crucial role in scale-up networks to enhance efficiency. It’s not just about the overall bandwidth – latency in data exchanges can cause GPUs and CPUs to stall as they wait for data. Pipelined algorithms often depend on distributed results, and even a single node’s delay can lead to significant slowdowns, idling valuable compute resources. PCIe technology, now with FLIT mode, keeps data transport delays minimized and consistent, allowing for efficient performance.
Future Evolutions of Generative AI With Emerging PCIe Technology
PCIe technology will help to evolve the applications of generative AI due to its scalability for both electrical and optical links in the back-end network, where AI operates. As bandwidths increase and electrical reaches decrease, CopprLink™ Internal and External Cables can extend the reach of PCIe signals within generative AI applications, with the CopprLink Internal cable having a maximum reach of 1m within a single system, while the CopprLink External cable extends the maximum reach to 2m. Additionally, the PCI-SIG Optical Work Group is currently investigating a path for enabling PCIe technology over optical links to ensure any PCIe link will be possible in the future of generative AI applications.
Join PCI-SIG to Support the Future of PCIe Technology and Generative AI
If you would like to support the future development of PCIe technology and generative AI, we encourage you to join PCI-SIG. Follow PCI-SIG on LinkedIn and Twitter/X for the latest information about the PCIe specifications, events and more.