On exploiting sparsity in CANFIS neuro-fuzzy modular network learning by second-order stagewise backpropagation
We describe efficient evaluation of the (global) Hessian matrix of the sum-squared-error measure for CANFIS neuro-fuzzy modular network learning. Our network consists of multiple (local-expert) multilayer perceptrons (MLPs) mediated by fuzzy membership functions, leading to an iteratively reweighted nonlinear least squares problem. In the posed situation, we show how our second-order stagewise backpropagation procedure, recently developed for learning with a single MLP, efficiently exploits the sparsity (of the Hessian matrix) that arises in a multiple-response problem. In spite of its complex modular architecture, our procedure works excellently. Its computational convenience is immense since such an efficient evaluation is crucial in implementing Newton-type second-order algorithms that may exploit negative curvature when the Hessian matrix is indefinite as well as in the Hessian analysis for any type of modular neural-network learning.
Eiji Mizutani
Department of Industrial Management, National Taiwan University of Science and Technology, Taipei, 106. TAIWAN
国际会议
上海
英文
200-204
2011-07-26（万方平台首次上网日期，不代表论文的发表时间）