Idioma:

A 95.6-TOPS/W Deep Learning Inference Accelerator With Per-Vector Scaled 4-bit Quantization in 5 nm

Keller, Ben ; Venkatesan, Rangharajan ; Dai, Steve ; Tell, Stephen G. ; Zimmer, Brian ; Sakr, Charbel ; Dally, William J. ; Gray, C. Thomas ; Khailany, Brucek

IEEE journal of solid-state circuits, 2023-04, Vol.58 (4), p.1-13 [Revista revisada por pares]

New York: IEEE

Texto completo disponible

Citas Citado por

Acciones

Título:
A 95.6-TOPS/W Deep Learning Inference Accelerator With Per-Vector Scaled 4-bit Quantization in 5 nm
Autor: Keller, Ben ; Venkatesan, Rangharajan ; Dai, Steve ; Tell, Stephen G. ; Zimmer, Brian ; Sakr, Charbel ; Dally, William J. ; Gray, C. Thomas ; Khailany, Brucek
Materias: Accelerators ; Accuracy ; Accuracy-efficiency trade-off ; Arithmetic ; Artificial neural networks ; BERT ; Costs ; Deep learning ; deep neural network (DNN) inference accelerator ; Energy efficiency ; Inference ; Machine learning ; Measurement ; Natural language processing ; Optimization ; quantization ; Quantization (signal) ; Task analysis ; Transformers
Es parte de: IEEE journal of solid-state circuits, 2023-04, Vol.58 (4), p.1-13
Descripción: The energy efficiency of deep neural network (DNN) inference can be improved with custom accelerators. DNN inference accelerators often employ specialized hardware techniques to improve energy efficiency, but many of these techniques result in catastrophic accuracy loss on transformer-based DNNs, which have become ubiquitous for natural language processing (NLP) tasks. This article presents a DNN accelerator designed for efficient execution of transformers. The proposed accelerator implements per-vector scaled quantization (VSQ), which employs an independent scale factor for each 64-element vector to enable the use of 4-bit arithmetic with little accuracy loss and low energy overhead. Using a multilevel dataflow to maximize reuse, the 5-nm prototype achieves 95.6 tera-operations per second per Watt (TOPS/W) at 0.46 V on a 4-bit benchmarking layer with VSQ. At a nominal voltage of 0.67 V, the accelerator achieves 1734 inferences/s/W (38.7 TOPS/W) with only 0.7% accuracy loss on BERT-Base and 4714 inferences/s/W (38.6 TOPS/W) with 0.15% accuracy loss on ResNet-50 by using quantization-aware fine-tuning to recover accuracy, demonstrating a practical accelerator design for energy-efficient DNN inference.
Editor: New York: IEEE
Idioma: Inglés

Volver a la lista de resultados

Realización: Logos de Redes Sociales:

A 95.6-TOPS/W Deep Learning Inference Accelerator With Per-Vector Scaled 4-bit Quantization in 5 nm

Keller, Ben ; Venkatesan, Rangharajan ; Dai, Steve ; Tell, Stephen G. ; Zimmer, Brian ; Sakr, Charbel ; Dally, William J. ; Gray, C. Thomas ; Khailany, Brucek

New York: IEEE

Buscando en bases de datos remotas, por favor espere