As you can see, it consists of two Twin Lake CPUs and multiple accelerators (M.2 modules) connected to them using a PCIE switch. The figure below illustrates the design of our AI inference server. Our first-generation video accelerators delivered a 10x performance-per-watt improvement in processing 4K videos. These videos are viewable on any device and with varying internet bandwidth. This enables Facebook to process the nearly 250 million videos uploaded to our app each day. We also invested in specialized hardware for video encoding and decoding. They also delivered a 3-10x performance-per-watt improvement over a CPU. Today, our first-generation systems are 10-30x more performant on our largest AI models. Since 2019, Facebook has invested heavily in deploying accelerator-based servers to provide higher performance and energy efficiency. How do we remedy this? By designing hardware that is customized to accelerate AI operations via application-specific integrated circuits (ASICs). This far outpaces improvements in CPU performance. In recent times, model complexity and compute requirements for AI have grown by roughly a factor of 10 each year. As investigated by the OpenAI community, we’ve seen two distinct eras of compute in AI models. They also tend to exhibit inefficiency in terms of energy used per AI prediction. However, CPUs fail to meet the rising computational demands of AI applications today. Good old general-purpose processors (CPUs) offer versatility and have grown exponentially faster over the decades. The graph below of AI model adoption at Facebook illustrates this unmistakable pattern. And as these models improve, so do their computational requirements. Running these AI models comes with heavy infrastructure demands. Backgroundįacebook’s cloud infrastructure handles about 150 trillion AI predictions per day for tasks ranging from feed recommendations to combating harmful content. An initial version of Atrace allowed us to close a 10 percent performance gap between Caffe2 and PyTorch implementations of a large AI model. It allows us to inspect accelerator systems in detail and provides actionable trace summaries and analyses. It has shrunk our development time for onboarding a new accelerator from a month to under a week.Ītrace, an accelerator tracing solution that collects traces remotely on production servers. Asicmon has facilitated load balancing, performance monitoring, and automated health checks for hundreds of thousands of accelerators running in our data centers.Īsimov, a custom specification language that makes developing and rapid prototyping new accelerators easier.
Asicmon’s library abstracts an accelerator’s custom interfaces and provides a standard interface to our internal tools. To meet these challenges, we’ve introduced three new tools:ĪSIC Monitoring (Asicmon), a scalable observability framework.
To ensure that these complex accelerators operate smoothly, we need an excellent observability system with monitoring and tracing capabilities so we can understand the performance and interactions between CPUs and accelerators. However, it is challenging to operate these heterogeneous platforms efficiently at scale. And Application-specific hardware platforms play an important role in meeting the growing latency and compute demands of workloads like deep learning, content understanding, and video encoding.Īt Facebook, the inevitable rise in use of accelerators in our data centers has led to better performance and energy efficiency. Please submit any questions to before the event.Īccelerators are special-purpose hardware devices optimized for specific applications, like AI prediction and video encoding. PT on Wednesday, June 30, followed by a live Q&A session. We will be hosting a talk about our work on, “ A Platform Agnostic Observability System for AI Accelerators” during our virtual Systems event at 10:20 a.m.
0 kommentar(er)
