Idioma:

A 95.6-TOPS/W Deep Learning Inference Accelerator With Per-Vector Scaled 4-bit Quantization in 5 nm

Keller, Ben ; Venkatesan, Rangharajan ; Dai, Steve ; Tell, Stephen G. ; Zimmer, Brian ; Sakr, Charbel ; Dally, William J. ; Gray, C. Thomas ; Khailany, Brucek

IEEE journal of solid-state circuits, 2023-04, Vol.58 (4), p.1-13 [Periódico revisado por pares]

New York: IEEE

Texto completo disponível

Citações Citado por

Enviar para

Título:
A 95.6-TOPS/W Deep Learning Inference Accelerator With Per-Vector Scaled 4-bit Quantization in 5 nm
Autor: Keller, Ben ; Venkatesan, Rangharajan ; Dai, Steve ; Tell, Stephen G. ; Zimmer, Brian ; Sakr, Charbel ; Dally, William J. ; Gray, C. Thomas ; Khailany, Brucek
Assuntos: Accelerators ; Accuracy ; Accuracy-efficiency trade-off ; Arithmetic ; Artificial neural networks ; BERT ; Costs ; Deep learning ; deep neural network (DNN) inference accelerator ; Energy efficiency ; Inference ; Machine learning ; Measurement ; Natural language processing ; Optimization ; quantization ; Quantization (signal) ; Task analysis ; Transformers
É parte de: IEEE journal of solid-state circuits, 2023-04, Vol.58 (4), p.1-13
Descrição: The energy efficiency of deep neural network (DNN) inference can be improved with custom accelerators. DNN inference accelerators often employ specialized hardware techniques to improve energy efficiency, but many of these techniques result in catastrophic accuracy loss on transformer-based DNNs, which have become ubiquitous for natural language processing (NLP) tasks. This article presents a DNN accelerator designed for efficient execution of transformers. The proposed accelerator implements per-vector scaled quantization (VSQ), which employs an independent scale factor for each 64-element vector to enable the use of 4-bit arithmetic with little accuracy loss and low energy overhead. Using a multilevel dataflow to maximize reuse, the 5-nm prototype achieves 95.6 tera-operations per second per Watt (TOPS/W) at 0.46 V on a 4-bit benchmarking layer with VSQ. At a nominal voltage of 0.67 V, the accelerator achieves 1734 inferences/s/W (38.7 TOPS/W) with only 0.7% accuracy loss on BERT-Base and 4714 inferences/s/W (38.6 TOPS/W) with 0.15% accuracy loss on ResNet-50 by using quantization-aware fine-tuning to recover accuracy, demonstrating a practical accelerator design for energy-efficient DNN inference.
Editor: New York: IEEE
Idioma: Inglês

Voltar para lista de resultados

Realização: Logos de Redes Sociais:

A 95.6-TOPS/W Deep Learning Inference Accelerator With Per-Vector Scaled 4-bit Quantization in 5 nm

Keller, Ben ; Venkatesan, Rangharajan ; Dai, Steve ; Tell, Stephen G. ; Zimmer, Brian ; Sakr, Charbel ; Dally, William J. ; Gray, C. Thomas ; Khailany, Brucek

New York: IEEE

Buscando em bases de dados remotas. Favor aguardar.