Validating a concept inventory for cybersecurity

[摘要] The validity of a concept inventory is imperative to its adoption by the education community. In this thesis, we evaluate the validity of the Cybersecurity Concept Inventory for assessing student knowledge of core cybersecurity concepts after a first course on the topic. Our evaluation involved expert review and student performance data. A panel of 12 experts in cybersecurity reviewed each item of the Cybersecurity Concept Inventory (CCI), and the majority agreed that every item measured appropriate cybersecurity knowledge. We gave the CCI to 142 students from six different institutions taking a first cybersecurity course either online or proctored by the professor of the course. We used Classical Test Theory to evaluate the quality of the CCI. This evaluation showed that the CCI is sufficiently reliable for measuring students' knowledge of cybersecurity and that the CCI may be too difficult as a whole. We describe the results of both the expert review and the pilot test in further detail and provide recommendations for the continued improvement of the CCI.

[关键词] Cybersecurity Education, Cybersecurity Assessment Tools (CATS) Project, Cybersecurity Concept Inventory (CCI), Assessment Validation