Controlling Competition by Structural Information
In this paper, we propose a new information theoretic method called structural information control for flexible feature discovery. The new method has three distinctive characteristics. First, the new method can directly control competitive unit activation patterns. Second, competitive units compete with each other by maximizing their information content about input patterns. Consequently, this information maximization makes it possible to flexibly control competition processes. Third, in structural information control, it is possible to define many different kinds of information content, and we con choose a specific type of information according to a given objective. When applied to competitive learning, structural information can be used to control the number of dead or spare units, and to extract macro as well as micro features of input patterns in explicit ways. We first applied this method to simple pattern classification to demonstrate that information can be controlled and that different neuron firing patterns can be generated. Second, we applied the method to a language acquisition problem in which networks must flexibly discover some linguistic rules by changing structural information.
Ryotaro Kamimura Taeko Kamimura Osamu Uchida Shohachiro Nakanishi
Future Science and Technology Joint Research Center and Information Science Laboratory, Tokai Univer Department of English, Senshu University, Japan, taekok Department of Network Engineering, Kanagawa Institute of Technology, Japan Future Science and Technology Joint Research Center and Department of Human and Information Science,
国际会议
8th International Conference on Neural Information Processing(ICONIP 2001)(第八届国际神经信息处理大会)
上海
英文
279-284
2001-11-14（万方平台首次上网日期，不代表论文的发表时间）