The incremental validity of three tests of academic literacy in the context of a South African university of technology

[摘要] English: This study focuses on the incremental validity of three assessments of academic language readiness, compared to Grade 12 English results: the National Benchmark Test in Academic Literacy (NBT AL), the Proficiency Test English Second Language Advanced Level (PTESLAL) and the Test of Academic Literacy Levels (TALL) at the end of the first year of academic study. More specifically, the study investigates the ability of any of the four assessments to predict first year academic performance better than the others. Where those that are examined do not possess this ability, the further question is asked: can they at least add to the predictive power of the best predictor? Ultimately, the aim is to determine if the assessments designed to provide additional information about first year academic preparedness are valid for this purpose, and the extent to which this is the case.The study starts with a brief exploration of the literature on the reportedly low levels of academic language ability among first time entrants to higher education in South Africa in recent years, and the consequent need for the development and use of valid tests of academic language ability for channelling these students into academic language interventions that are aimed at dealing with this challenge. The literature on the current theories of validity is also explored in relation to the hypothesis of the study, which is that as a test designed to provide additional information about the academic language readiness of first year students, TALL will possess better incremental validity in relation to the best predictor of first year academic performance.Subsequently, an attempt is made to account for the ability of Grade 12 English results to predict first year academic performance better than the other three assessments investigated in the present study. Similarly, an effort is made to account for the ability of TALL to show evidence of incremental validity in relation to Grade 12 English results and the inability of NBT AL and PTESLAL to do the same. Furthermore, on the basis of the results of previous studies and the current one, a recommendation is made that Grade 12 results in general and Grade 12 English results in particular be used together with those of academic literacy tests to make access and placement decisions. The basis for this recommendation resides in the psychometric and other shortcomings of Grade 12 results that have been identified by previous studies as well as the evidence that similar studies have produced to show that tests of academic literacy possess better ability to partition test taker performance from different school backgrounds and at different levels of performance.Finally, the implications of the results of the study for current theories of test validity are discussed. In the main, the discussion focuses on demonstrating on the basis of these results and those of previous studies that the currently popular theory of validity wherein a unitary approach to validity is upheld and the interpretation and use of test scores are regarded as the essence of validation does not hold. At the same time, the discussion focuses on demonstrating that the traditional theory of validity, wherein validity is believed to reside in the objective ability of a test to produce valid scores and a distinction is made between the three traditional types of validity, namely construct, content and criterion-related validity is, with certain obvious qualifications, still defensible. Finally, the implications of the results of the study for validity theory are dealt with in relation to the validity of courses of academic literacy.

[发布机构] University of the Free State