Cover Image

Dynamic point stochastic rounding algorithm for limited precision arithmetic in Deep Belief Network training

This paper reports how to train a Deep Belief Network (DBN) using only 8-bit fixed-point parameters. We propose a dynamic-point stochastic rounding algorithm that provides enhanced results compared to the existing stochastic rounding. We show that by using a variable scaling factor, the fixed-point...

Full description

Saved in:
Bibliographic Details
Main Authors: Essam, M., Tang, T.B., Ho, E.T.W., Chen, H.
Format: Article
Published: IEEE Computer Society 2017
Online Access:https://www.scopus.com/inward/record.uri?eid=2-s2.0-85028585216&doi=10.1109%2fNER.2017.8008430&partnerID=40&md5=3d565633d6b0ee148b8349a9b15f1d76
http://eprints.utp.edu.my/20033/
Tags: Add Tag
No Tags, Be the first to tag this record!
id my.utp.eprints.20033
record_format eprints
spelling my.utp.eprints.200332018-04-22T14:38:14Z Dynamic point stochastic rounding algorithm for limited precision arithmetic in Deep Belief Network training Essam, M. Tang, T.B. Ho, E.T.W. Chen, H. This paper reports how to train a Deep Belief Network (DBN) using only 8-bit fixed-point parameters. We propose a dynamic-point stochastic rounding algorithm that provides enhanced results compared to the existing stochastic rounding. We show that by using a variable scaling factor, the fixed-point parameter updates are enhanced. To be more hardware amenable, the use of common scaling factor at each layer of DBN is further proposed. Using publicly available MNIST database, we show that the proposed algorithm can train a 3-layer DBN with an average accuracy of 98.49, with a drop of 0.08 from the double floating-point average accuracy. © 2017 IEEE. IEEE Computer Society 2017 Article PeerReviewed https://www.scopus.com/inward/record.uri?eid=2-s2.0-85028585216&doi=10.1109%2fNER.2017.8008430&partnerID=40&md5=3d565633d6b0ee148b8349a9b15f1d76 Essam, M. and Tang, T.B. and Ho, E.T.W. and Chen, H. (2017) Dynamic point stochastic rounding algorithm for limited precision arithmetic in Deep Belief Network training. International IEEE/EMBS Conference on Neural Engineering, NER . pp. 629-632. http://eprints.utp.edu.my/20033/
institution Universiti Teknologi Petronas
building UTP Resource Centre
collection Institutional Repository
continent Asia
country Malaysia
content_provider Universiti Teknologi Petronas
content_source UTP Institutional Repository
url_provider http://eprints.utp.edu.my/
description This paper reports how to train a Deep Belief Network (DBN) using only 8-bit fixed-point parameters. We propose a dynamic-point stochastic rounding algorithm that provides enhanced results compared to the existing stochastic rounding. We show that by using a variable scaling factor, the fixed-point parameter updates are enhanced. To be more hardware amenable, the use of common scaling factor at each layer of DBN is further proposed. Using publicly available MNIST database, we show that the proposed algorithm can train a 3-layer DBN with an average accuracy of 98.49, with a drop of 0.08 from the double floating-point average accuracy. © 2017 IEEE.
format Article
author Essam, M.
Tang, T.B.
Ho, E.T.W.
Chen, H.
spellingShingle Essam, M.
Tang, T.B.
Ho, E.T.W.
Chen, H.
Dynamic point stochastic rounding algorithm for limited precision arithmetic in Deep Belief Network training
author_facet Essam, M.
Tang, T.B.
Ho, E.T.W.
Chen, H.
author_sort Essam, M.
title Dynamic point stochastic rounding algorithm for limited precision arithmetic in Deep Belief Network training
title_short Dynamic point stochastic rounding algorithm for limited precision arithmetic in Deep Belief Network training
title_full Dynamic point stochastic rounding algorithm for limited precision arithmetic in Deep Belief Network training
title_fullStr Dynamic point stochastic rounding algorithm for limited precision arithmetic in Deep Belief Network training
title_full_unstemmed Dynamic point stochastic rounding algorithm for limited precision arithmetic in Deep Belief Network training
title_sort dynamic point stochastic rounding algorithm for limited precision arithmetic in deep belief network training
publisher IEEE Computer Society
publishDate 2017
url https://www.scopus.com/inward/record.uri?eid=2-s2.0-85028585216&doi=10.1109%2fNER.2017.8008430&partnerID=40&md5=3d565633d6b0ee148b8349a9b15f1d76
http://eprints.utp.edu.my/20033/
_version_ 1738656154057179136
score 13.160551

Similar Items