국고보조사업 지원 대상 선정평가 신뢰성 개선

Improving Reliability of the Section Evaluation of Government Subsidy Program Support Targets Using Intraclass Correlation Coefficient(ICC) and Standard Score

(Purpose) Subsidy targets for a government subsidized project are mostly selected based on the results of written evaluation and on-site evaluation by external experts. However, there is a risk that the priorities of this evaluation change depending on the subjective propensity of evaluators (e.g. generous, strict) and the reliability of the evaluation get lowered. Measuring reliability of evaluation and correcting evaluators’ subjective propensity are, therefore, needed. This study suggests the intraclass correlation coefficient (ICC) as a method to measure the reliability of the selection evaluation of subsidy targets in government subsidized projects, and proposes the use of standardized scores to increase the evaluation reliability. (Design/methodology/approach) ICC and standardized scores are identified in theoretical analysis, and then the reliability of the evaluation scores of the 2022 support targets of the Fishing Village New Deal Project, a government subsidized project, is analyzed using ICC. Lastly the rankings are recalculated using standardized score. (Findings) Written evaluation reliability was low, with the ICC of less than 0.5, and on-site evaluation reliability was also not good with the ICC of less than 0.75. When using standardized scores, 14% of the ports selected by raw scores was changed. As a result of ICC analysis on the standardized score, it was found that ICC increased. This means that when the standardized score is used, the reliability of the evaluation score increase by reducing the statistical error. (Research implications or Originality) This study suggests methods to verify and increase the reliability of the selection evaluation of the support target of government subsidized projects to improve the performance.