The focus of this SEMICON TAIWAN's master forum undoubtedly falls on Jim Keller, executive director of Tenstorrent. This has served as a senior executive in the hardware departments of Intel, AMD, Tesla and other companies. He has successfully led several successful chip development plans, known as the "chip master". As soon as the lecture opened, Tenstorrent will set up an office in Taipei to start recruiting talents, and will also propose a new AI processor by opening the original code RISC-V architecture to build high-performance AI using ordinary DRAM and ethernet networks, so that anyone can build and expand, and AI applications will become more popular.
Jim Keller, unlike others, wore T-Shirt and said that this speech was partly invited, and more importantly, Tenstorrent will set up a office in Taipei and has begun recruiting talents. Due to Taipei's talent economy, not only recruits operators to help cooperate with local manufacturers, but also finds many excellent engineers to invest in key fields such as chiplet engineering, CPU design verification and AI models. In particular, Taiwan is famous for its outstanding manufacturing reputation for various factories such as motherboards, packaging, systems, and crystal circles. He hopes to become a part of the excellent manufacturing team.
A reverse engineering incident for those who are ill-inducing the Tenstorrent thinking banJim Keller said Tenstorrent is building a special computer factory with the goal of producing all the components needed for modern high-level AI computers and high-performance processors. Tenstorrent has been committed to building high-level AI computers and high-speed CPUs with a large number of IP and technology. Now products have started shipping, including expandable Galaxy box servers, and water-cooled quiet box designed to respond to customers’ complaints about noise. In addition, we also launched PCI Express cards and just started providing Eson TPU as IP to our customers. It is exciting that AI processors have also been shipped to multiple customers, including LG, which integrates AI processors into TV chips.
Jim Keller emphasizes that he is a mission-oriented person. Although he hopes that the company can make money from a long time, it is more important to have a mission, that is, the AI world is changing everything at the moment, and the demand for "gigawatt" and "megabit set" products can be seen everywhere. But all of this is getting bigger and bigger, more expensive, and many technologies are special. Although some models are open, most are still closed. Therefore, my mission is to create new architectures that are cheaper, faster and more open.
To this end, Tenstorrent has developed a macro plan, which is to build a RISC-V CPU based on an open source architecture. This open AI processor, which is developing, comes from an unintentional source incident, is that a young man on the Internet released all the details of our processor through reverse engineering and released specifications. This seemingly insulting incident actually prompted us to think this is a good idea. Because if someone wants to start building their own AI computer from the beginning, there will be complete specifications and reference models on the Internet soon.
Based on this, we also promised to build a software stack that completely opens the original code, from models, translators, fast operators, low-level instruction sets for the operational environment, etc., all will be open. This means that if you buy our hardware, you can clearly see all internal operations. The goal is to build high-performance AI using ordinary DRAM and ethernet so that anyone can build and expand it.
NTD 6 nanometers to create Black Hole chips, with a strong structure to expand Galaxy box serversJim Keller emphasized that in terms of hardware, the Black Hole is our tyre-electric 6-nanometer chip, which contains 140 meter processors and RISC-V processors, and supports GDDR6 DRAM and on-chip SRAM. We chose not to use HBM because it is too expensive and complicated to pack. Our strategy is to make more, smaller, cheaper chips. For example, there are eight Black Hole chips on a board and four boards form a box with a total of 32 chips, providing 1TB of DRAM and 16TB of frequency width. Each chip has an ethernet interface, and all connections are redundant to ensure the reliability of the system.
As for the current situation, the challenge of AI training is that although the model can be written, making it run quickly is a complex HPC problem. As a result, Tenstorrent's training computers have 2,000 Black Hole chips, which requires huge collaboration and optimization as compared to more than one million RISC-V processors. As for the low-cost AI training, many large AI models (such as OpenAI and XAI models) may be open source, but for those of us who need to train our own models, it is important to be able to train at a low cost. Our training computers are composed of nine Galaxy boxes, and all connections are redundant, so they can run normally even if there is a cable failure.. Multiple such units can further form a larger training cluster, providing a consistent perspective from top to bottom, making it possible to manage hundreds of Galaxy boxes.
Jim Keller emphasized that building a chip is expensive and our Black Hole chip costs about $80 million. Every time we upgrade AI, we need to spend tens of thousands of dollars to create new chips. Therefore, our next generation solution is to use chiplets. We design components such as CPU, memory controller and AI as modular wood. Despite the existing multiple small chip standards, the goal is to ensure that our chip family works together and is compatible with UCIe standards.
Small chip structure stacking + self-interpretation IP integration, easily embrace the AI marketTenstorrent is working to implement independent iteration and synthesis of wafers, and will provide solutions that use these small wafers, and will also open these small wafers to customers who want to develop solutions on their own. To this end, we launched an Open Chiplet Architecture plan, which is not only the interconnection standards between the chips, but also includes all components such as testing, resetting, system encryption, etc. We will pay all the relevant IPs to this alliance, and even plan to launch an empty chip, a die-hard chip containing all the necessary IP systems, allowing customers to easily integrate their IPs and quickly launch them to the market.
Finally, Jim Keller said that many people in the past would ask what exactly we sold? The answer is that we sell everything we do. Whether it is IP, chip or system, we will sell it. We even offer cloud services, allowing new companies to keep their applications on our machines. Finally, I hope to make the next generation of computer design more interesting. This makes people feel that computer design is becoming more expensive and more difficult to disappear. Through small chip technology, better IP, better verification, and sourced AI drive methods, computer design will become better and easier to implement. I hope to gather everyone's strength to jointly create a future full of endless possibilities.