Multitask Learning for Sparse Failure Prediction
Sparsity is a problem which occurs inherently in many realworld datasets.Sparsity induces an imbalance in data,which has an adverse effect on machine learning and hence reducing the predictability.Previously,strong assumptions were made by domain experts on the model parameters by using their experience to overcome sparsity,albeit assumptions are subjective.Differently,we propose a multi-task learning solution which is able to automatically learn model parameters from a common latent structure of the data from related domains.Despite related,datasets commonly have overlapped but dissimilar feature spaces and therefore cannot simply be combined into a single dataset.Our proposed model,namely hierarchical Dirichlet process mixture of hierarchical beta process(HDP-HBP),learns tasks with a common model parameter for the failure prediction model using hierarchical Dirichlet process.Our model uses recorded failure history to make failure predictions on a water supply network.Multi-task learning is used to gain additional information from the failure records of water supply networks managed by other utility companies to improve prediction in one network.We achieve superior accuracy for sparse predictions compared to previous state-of-the-art models and have demonstrated the capability to be used in risk management to proactively repair critical infrastructure.
Multi-task learning Sparse predictions Dirichlet process Beta process Failure predictions
Simon Luo Victor W.Chu Zhidong Li Yang Wang Jianlong Zhou Fang Chen Raymond K.Wong
The University of Sydney,Sydney,Australia;Data61,CSIRO,Sydney,Australia Nanyang Technological University,Singapore,Singapore Data61,CSIRO,Sydney,Australia;University of Technology Sydney,Ultimo,Australia The University of New South Wales,Kensington,Australia
国际会议
澳门
英文
3-14
2019-04-14（万方平台首次上网日期，不代表论文的发表时间）