A New Geometric Approach to the Complexity of Model Selection

摘要：

Model selection is one of the central problems of machine learning. The goal of model selection is to select from a set of competing explanations the best one that capture the underlying regularities of given observations. The criterion of a good model is generalizability. We must make balance between the goodness of fit and the complexity of the model to obtain good generalization. Most of present methods are consistent in goodness of fit and differ in complexity. But they only focus on the free parameters of the model; hence they cannot describe the intrinsic complexity of the model and they are not invariant under re-parameterization of the model This paper uses a new geometrical method to study the complexity of the model selection problem. We propose that the integral of the Gauss-Kronecker curvature of the statistical manifold is the natural measurement of the non-linearity of the manifold of the model. This approach provides a clear intuitive understanding of the intrinsic complexity of the model. We use an experiment to verify the criterion based on this method.

关键词： Model selection Occams razor statistical manifold geometrical complexity Gauss-Kronecker curvature.

作者: Ziang Lv Siwei Luo Yunhui Liu Yu Zheng

作者单位: School of Computer and Information Technology, Beijing Jiaotong University

会议类型: 国际会议

会议名称: Firth IEEE International Conference on Cognitive Informatics(第五届认知信息国际会议)

会议地点: 北京

会议语种:英文

页码: 268-273

在线出版日期: 2006-07-17（万方平台首次上网日期，不代表论文的发表时间）