Training Deep Belief Network with Sparse Hidden Units

摘要：

　　In this paper,we proposed a framework to train Restricted Boltzmann Machine (RBM) which is the basic block for Deep Belief Network (DBN).By introducing sparsity constraint to the Contrastive Divergence algorithm (CD algorithm),we trained RBMs with better performance than the off-the-shelf model in MNIST handwritten digit data set.The sparse model suffer from saturation slightly,however,by using a trade-off coefficient,the saturation problem can be solved well.To our knowledge,the sparsity constraint was first introduced to the hidden units of RBM.

关键词： sparsity DBN mnist

作者: Zhen Hu Wenzheng Hu Changshui Zhang

作者单位: The State Key Laboratory of Intelligent Technology and Systems,Tsinghua National Laboratory for Information Science and Technology(TNList)and the Department of Automation,Tsinghua University,Beijing,China,100084

会议类型: 国际会议

会议名称: Chinese Conference on Pattern Recognition, CCPR(2014年全国模式识别学术会议)

会议地点: 长沙

会议语种:英文

页码: 11-20

在线出版日期: 2014-11-01（万方平台首次上网日期，不代表论文的发表时间）