Is Chiplets the Answer to the End of Moore’s Law?

Is Moore’s Law dead? Many would argue–yes. Yet, this is not the end of the advancement in the semiconductor realm. We have just entered the age of generative AI, and the requirement for newer and more efficient chips is ever-growing.

Many in the industry believe chiplets could be the answer. In a research paper in 1965, Intel co-founder Gordon Moore said that the number of transistors on a chip would double approximately every two years, leading to exponential increases in computing power and decreases in cost.

But in today’s time, given the constraints of single-reticle chip sizes, fitting more transistors onto one chip is becoming increasingly challenging. Chiplets, however, offer greater flexibility and scalability in semiconductor design by allowing the integration of smaller, specialised components into a single package.

This, in turn, can help overcome some of the physical and economic challenges associated with traditional scaling methods.

Why Chiplets?

Sid Sheth, the co-founder and CEO of d-Matrix, an AI chip startup based in Santa Clara, California, believes the amount of functionality one can pack into a chip doesn’t increase if the chip size remains the same, and when it comes to chips, it can’t exceed a certain size.

“To overcome this, chiplets offer a solution. Instead of one large chip, you can divide the functionality into multiple smaller chips. For example, if you break one 800 mm² chip into four smaller 800 mm² chiplets and interconnect them, you can achieve the equivalent of a 3200 mm² chip, effectively increasing the overall functionality without being constrained by the single-chip size limit,” he told AIM.

d-Matrix is developing chiplets that employ digital in-memory computing (DIMC) to run Transformer-based AI inference models, particularly targeting large language models (LLMs).

Their Jayhawk II processor utilises chiplets technology, featuring approximately 16.5 billion transistors. It is designed to scale by accommodating up to eight chiplets per card and up to 16 cards per node.

(Jayhawk II by d-Matrix)

Another benefit of chiplets, besides compute density per package, is composability and reusability. Chiplets can be quickly and easily customised and upgraded leading to shorter development times and lower costs. 

This flexibility allows chip makers to rapidly respond to changing market demands and new technological advancements.

Stan Sokorac, senior fellow for AI hardware and software at Tenstorrent, believes the difference in inference and training (AI models) is an excellent showcase for chiplets. 

“A general AI compute chiplet can be built with high-bandwidth die-to-die (i.e., chiplet to chiplet) I/O, which can then be packaged with a large amount of DRAM bandwidth to build an ideal LLM inference chip, or with a mix of DRAM and Ethernet to allow for easy training scale-out. 

“Multiple AI chiplets can be packaged together for a high-end solution, or just one for a low-cost option. In all these cases, design and implementation costs are primarily in the packaging, once the initial chiplet is created,” he told AIM.

Among other things, Tenstorrent too is developing a modular chiplet-based architecture for AI and high-performance computing. Similarly, Netrasemi, a Kerala, India-based startup, is also developing a 12 nm AI chip based on chiplets technology.

The first chip, Netra A4000, represents a line of high-performance chips designed with advanced Chiplet (D2D integration) technology. These chips, offering between 32 and 100 TOPS (trillion operations per second), are targeted at the edge server market, smart NVR systems and the robotics industry.

Chiplets is the Future

The concept is not new. Multi-chip modules have existed for decades. Back in 1995, Intel combined a CPU chip and an SRAM chip in the Pentium Pro. However, much has changed since then.

(Intel Pentium Pro/ Source- Science Museum Group)

“Today, if you examine an Apple iPad or MacBook, you’ll find multiple chiplets integrated into the device. For instance, these devices often feature multiple CPUs communicating with each other, along with a neural engine interacting with the CPU,” Sheth said.

But given the slowdown in Moore’s Law, it’s evident that nearly every platform is now leveraging chiplets to enhance performance and functionality.”

Moreover, as packaging and interposer technology continues to advance, constructing very large packages with dozens or more chiplets will become easier and more affordable. 

This progress, in turn, will trigger a surge in the hardware startup market, enabling new companies to create highly sophisticated devices at a significantly lower cost. 

“The capability to rapidly assemble high-performance systems using off-the-shelf chiplets will pave the way for a broad range of custom devices, expanding market opportunities,” Sokorac said.

Moreover, he also adds that it’s in every chip company’s best interest to seriously consider chiplet technology. 

“As the ecosystem grows and the technology matures, end-user companies will be able to assemble products faster and more cheaply, while IP companies will reach larger markets more easily,” he added.

The post Is Chiplets the Answer to the End of Moore’s Law? appeared first on AIM.