Validating Search Processes in Systematic Literature Reviews
Context: Systematic Literature Reviews (SLRs) need to employ a search process that is as complete as possible. It has been suggested that an existing set of known papers can be used to help develop an appropriate strategy. However, it is still not clear how to evaluate the completeness of the resulting search process. Aim: We suggest a means of assessing the completeness of a search process by evaluating the search results on an independent set of known papers. Method: We assess the results of a search process developed using a known set of papers by seeing whether it was able to identify papers from a different set of known papers. Results: Using a second set of known papers, we were able to show that a search process, which was based on a first set of known papers, was unlikely to be complete, even though the search process found all the papers in the first known set. Conclusions: When using a set of known papers to develop a search process, keep a “holdout sample to evaluate probable completeness.
Barbara Kitchenham Zhi Li Andrew Burn
School of Computer Science and Mathematics, Keele University, Staffs, ST5 5BG, U.K. College of Computer Science and Information Technology, Guangxi Normal University No.15 Yu Cai Road, School of Engineering and Computing Sciences, Durham University, Durham, DH1 3LE, U.K.
国际会议
13th International Conference on Enterprise Information System(第13届企业信息系统国际会议 ICEIS 2011)
北京
英文
1575-1581
2011-06-08（万方平台首次上网日期，不代表论文的发表时间）