ASSESSMENT
IN HIGHER EDUCATION
|
ASSESSMENT IN HIGHER EDUCATION
|
|
|
This site is no longer being funded by CQU. If you work for, or know of, an educational institution that might be interested in funding this site, please email me for further details.
This page aims to provide a one-stop shop for links and resources specifically devoted to the subject of multiple-choice quizzes or tests If there are others you'd like to see listed, please email details.
Of course, there are many books and online articles and offline articles on assessment which include references to multiple choice tests, but this page lists articles specifically devoted to MCQs There are two sections: (1) articles (many available online) and web sites from the last ten years, and (2) a subjective selection of influential articles from previous decades, mostly available only offline.
Ayers. S.F. (2001). Developing quality multiple-choice tests for physical education Journal of Physical Education, Recreation & Dance. Reston: Aug 2001. Vol. 72, Iss. 6; p. 23 (7 pages) Although the initial creation of such tests can be time-consuming, multiple-choice templates, once developed, can be easily modified for a variety of instructional topics. Ayers presents guidelines to help teachers develop quality multiple-choice assessments. Ballantyne, C. & The Teaching & Learning Centre of Murdoch University (2002). Multiple Choice Tests. Multiple Choice Questions (MCQ) are usually used to test a student's ability to recall information, to interpret data or diagrams and to analyse and evaluate material. Beller, M. & Gafni. N. (2000). Can item format (multiple choice vs. open-ended) account for gender differences in mathematics achievement? Sex Roles. New York: Jan 2000. Vol. 42, Iss. 1/2; p. 1 (21 pages) The purpose of this study was to investigate differential performance of boys and girls on open-ended (OE) and multiple-choice (MC) items on the 1988 and 1991 International Assessment of Educational Progress (IAEP) mathematics test. Burton, R. F. & Miller, D. J. (1999). Statistical modelling of multiple-choice and true/false tests: Ways of considering, and of reducing, the uncertainties attributable to guessing Assessment and Evaluation in Higher Education. Bath: Dec 1999. Vol. 24, Iss. 4; p. 399 (13 pages) Test unreliability due to guessing in multiple-choice and true/false tests is analyzed from first principles, and two new measures are described. It is shown that the results of many tests currently conducted are likely to be unacceptably unreliable. Burton, R. F. (2005). Multiple-choice and true/false tests: myths and misapprehensions Assessment & Evaluation in Higher Education Vol. 30, No. 1, February 2005, pp. 65-72. Examiners seeking guidance on multiple-choice and true/false tests are likely to encounter various faulty or questionable ideas. Twelve of these are discussed in detail, having to do mainly with the effects on test reliability of test length, guessing and scoring method (i.e. number-right scoring or negative marking). Bush M (1999). Alternative marking schemes for online multiple choice tests. Considers scoring options for MCQs, including use of negative marking and normalisation. Carneson J, Delpierre G and Masters K (n.d.) Designing and managing multiple choice questions. Explores the relationship between different types of question and Bloom's taxonomy. Center for Assessment and Research Studies (1999). Assessment Using the Multiple Choice Format discusses the construction of tests (wording and choices), pilot test refinement, student performance, reliability issues, test dimensionality, test refinement, validity, test standards, test maintenance, and test development. Chan, N. & Kennedy. P. E. (2002). Are multiple-choice exams easier for economics students? A comparison of multiple-choice and "equivalent" constructed-response exam questions Southern Economic Journal. Stillwater: Apr 2002. Vol. 68, Iss. 4; p. 957 (15 pages) This article investigates this issue by comparing student scores on MC and equivalent CR questions. Results show that, for certain types of MC questions, students do indeed score better, even after correcting for guessing, and on other types of questions, they produce similar scores. These results are unaffected by comparing males versus females or good versus poor students. Counselling Services - University of Victoria (2003). Learning Skills Program – The Multiple Choice Exam. Multiple-choice questions require fine distinctions between correct and nearly correct statements. Davies, P. (2002), Computer-aided assessment must be more than multiple-choice tests for it to be academically credible?, 5th International Conference on Computer-Aided Assessment. Note the question mark at the end of the title. Delgado, A. R. & Prieto, G. (2003). The effect of item feedback on multiple-choice test responses British Journal of Psychology. London: Feb 2003. Vol. 94; p. 73. This study examines the effects of item feedback on multiple-choice test responses. The discussion takes into account other evidence to state that recommendations of providing item feedback in the context of testing are neither theoretically nor empirically founded. Dewey, R.A. (2000). Writing multiple-choice items which require comprehension. A comprehensive list of suggestions for writing multiple-choice questions. Ediger. M. (2003). Teacher involvement to evaluate achievement Education. Chula Vista: Fall 2003. Vol. 124, Iss. 1; p. 137. There are selected criteria which teachers need to use in writing subject matter tests. A variety of kinds of test items should be written including multiple choice, true false, matching, completion, and essay. Epstein, M. L., Lazarus, A. D., Calvano, T. B., Matthews, K. A. et al. (2002). Immediate feedback assessment technique promotes learning and corrects inaccurate first responses The Psychological Record. Gambier: Spring 2002. Vol. 52, Iss. 2; p. 187 (15 pages) Multiple-choice testing procedures that do not provide corrective feedback facilitate neither learning nor retention. The results of three studies converged to indicate that the Immediate Feedback Assessment Technique method actively engages learners in the discovery process and that this engagement promotes retention and the correction of initially inaccurate response strategies. Hadj-Alouane, A. B. & Bean, J. C. (1997). A genetic algorithm for the multiple-choice integer program Operations Research. Linthicum: Jan/Feb 1997. Vol. 45, Iss. 1; p. 92 (10 pages) A genetic algorithm is presented for the multiple-choice integer program that finds an optimal solution with probability one. General constraints are relaxed by a non-linear penalty function for which the corresponding dual problem has weak and strong duality. Higgins, E. & and Tatham, L. (2003). Assessing by multiple choice question (MCQ) tests at Manchester Metropolitan University MCQ’s are "the traditional 'choose from a list' of possible answers". Higgins, E. & and Tatham, L. (2003). Exploring the potential of Multiple-Choice Questions in Assessment. Learning and Teaching in ACTION. Vol. 2 Iss. 1 Winter 2003. This article explores the potential for using MCQs for assessment in a subject discipline in which MCQ testing is not universally accepted as valid. Some of the advantages and misconceptions associated with MCQ tests are explored, and in particular the article looks at ways of testing more than knowledge-acquisition. Houseworth, S. & Plous, S. (1997-1999). Tips on Taking Multiple-Choice Tests Most of the recommendations are taken from an AATBS manual designed for the psychology licensure exam, from research on testing and test anxiety, and from the experience of both professors. Illinois State University (2001). Centre for the Advancement of Teaching. Writing Multiple Choice Tests. This information is intended to guide faculty in the design and writing of multiple-choice tests. The accurate assessment of student learning depends, in part, on the quality of the tests you write.
Iowa State University Centre for Teaching Excellence (2003). Teaching Tips:A Student Guide to Multiple Choice Exams Studying for a multiple choice exam requires a special method of preparation distinctly different from an essay exam. Kehoe J (1995) Writing multiple-choice test items The objective of this Digest is to set out some conventional wisdom for the construction of multiple-choice tests, which are one of the most common forms of teacher-constructed tests. The comments which follow are applicable mainly to multiple-choice tests covering fairly broad topic areas. Kehoe, J. (1995). Basic Item Analysis for Multiple-Choice Tests. ERIC/AE Digest. ERIC Identifier: ED398237. This Digest offers some suggestions for the improvement of multiple-choice tests using "item analysis" statistics. Kuechler, W. L. & Simkin. M. G. (2003). How Well Do Multiple Choice Tests Evaluate Student Understanding in Computer Programming Classes? Journal of Information Systems Education. West Lafayette: Winter 2003. Vol. 14, Iss. 4; p. 389 (11 pages) Despite the wide diversity of formats with which to construct class examinations, there are many reasons why both university students and instructors prefer multiple-choice tests over other types of exam questions. The purpose of the present study was to examine this multiple-choice/constructed-response debate within the context of teaching computer programming classes. Leon P (2002) Log on to student motivation. Times Higher Education Supplement 3 May 2002 describes the use of formative Web-based MCQs to complement face-to-face teaching Lowry, R. (2000), Formative Assessment via the Web using ELEN, Cal-laborate 5, October 2000. Using short MCQs for formative assessment. Maunder, P. (2002). In support of multiple choice questions: evidence from Curriculum 2000. Paper presented at the Annual Conference of the British Educational Research Association, University of Exeter, England, 12-14 September 2002. One of the key advantages of multiple-choice questions being marked by computer is the speed and savings of time effected by this means of assessment. Measures of the efficacy of such questions are available to establish their credibility in establishing what students know and understand about basic concepts in any discipline. McKenna C and Bull J (1999). Designing effective objective test questions: an introductory workshop (the Computer Assisted Assessment Centre's useful guide to question design, with examples of tested questions demonstrating alternative formats) MCEdit and MCRun allow you to create, edit and run multiple choice tests on a computer running Windows® (98, ME, NT, 2000 or XP). They feature graphic supported questions, the option of hints for incorrect responses, and self-correcting runs in normal or random sequence. Each comes with two sample tests. Both applications can be downloaded and used free of charge on any number of computers. The National Center for Fair & Open Testing (n.d.). FairTest A multiple-choice test usually has dozens of questions or "items." For each question, the test- taker is supposed to select the "best" choice among a set of four or five options. O'Neill. P. B. (2001). Essay versus multiple choice exams; an experiment in the principles of macroeconomics course American Economist. Los Angeles: Spring 2001. Vol. 45, Iss. 1; p. 62 (9 pages) Examining the impact of test type in Principles of Macroeconomics via 2-stage estimation of an education production function, the author finds that students do better on a post-course TUCE when they take multiple-choice exams. Pauk, W. (n.d.). How to Study in College. Multiple Choice Tests. The site displays some very specific strategies for approaching multiple-choice tests. Paxton. M. (2000). A linguistic perspective on multiple choice questioning Assessment and Evaluation in Higher Education. Bath: Jun 2000. Vol. 25, Iss. 2; p. 109 (11 pages) Paxton critiques the over-emphasis on multiple choice testing in some large first year classes as well as the poor design and construction of many of these tests, and calls for the use of multiple choice questions as part of a broader and more diverse range of assessment measures. Pennsylvania State University (2004). Writing Effective Questions to Promote Learning. Lots of good tips of writing effective multiple-choice and other questions. Prieto, G., & Delgado, A. (1999). The effect of instructions on multiple choice test scores. European Journal of Psychological Assessment, 15(2), 143-150. Rizzuto, G. T. & Walters, F. H. (1997). Cheating probabilities on multiple choice tests Journal of Chemical Education [H.W. Wilson - GS]. Oct 1997. Vol. 74; p. 1185. Cheating probabilities for multiple choice tests are outlined. The probability of a student having the same answers as another student can be calculated quite simply by using the binomial distribution. The probability of 2 students showing identical responses increases the better the students are, but the mathematical model is designed for all responses and provides a baseline for evaluation. Original exam papers should be inspected for details. Rodriguez, M.C. (2005). Three options are optimal for multiple-choice questions: A meta-analysis of 80 years of research. Educational Measurement: Issues and Practice, Summer, 3-13.Struyven, K., Janssens, S. & Dochy, F. (2002). Students' perceptions about assessment in higher education: a review. Paper presented at the Learning Communities and Assessment Cultures Conference organised by the EARLI Special Interest Group on Assessment and Evaluation, University of Northumbria, 28-30 August 2002. Within conventional assessment, multiple choice format exams are seen as favourable assessment methods in comparison to essay type questions. But when conventional assessment and alternative assessment methods are compared, students perceive alternative assessment as being more "fair" than the traditional 'normal' assessment methods. Tretter, T. T. & Jones, M. G. (2003). Relationships Between Inquiry-Based Teaching and Physical Science Standardized Test Scores School Science and Mathematics. Bowling Green: Nov 2003. Vol. 103, Iss. 7; p. 345. This exploratory case study investigates relationships between use of an inquiry-based instructional style and student scores on standardized multiple-choice tests. University of Alberta (n.d.). Scoring Multiple Choice Tests A site with many links and well worth a visit. Wesolowsky. G. O. (2000). Detecting excessive similarity in answers on multiple choice exams Journal of Applied Statistics. Sheffield: Sep 2000. Vol. 27, Iss. 7; p. 909 (13 pages). Provides a simple and robust method for detecting cheating. The model is suitable for screening large classes and the results are simple to interpret. It has received considerable application in practice and has been verified through the adjacent seating method. Wood. W. C. (1998). Linked multiple-choice questions: The tradeoff between measurement accuracy and grading time Journal of Education for Business. Washington: Nov/Dec 1998. Vol. 74, Iss. 2; p. 83 (4 pages) Business faculty face a trade off between accuracy of measurement and grading time. Simple true/false and multiple-choice questions are quickly graded but may be less accurate in measuring learning than free-response questions. Yunker. B. D. (1999). Adding authenticity to traditional multiple choice test formats Education. Chula Vista: Fall 1999. Vol. 120, Iss. 1; p. 82 (6 pages) The Challenge is a performance-based assessment strategy allowing students in pre-service teacher training programs to orally challenge the keyed answers following a multiple choice test.
If there are
other articles you'd like to
see listed, please email details.
Abu-Sayf, F.K. (1979).
The scoring of multiple-choice tests: a closer look. Educational
Technology, 19, 5-15. Aiken, L.R. (1964).
Item context and position effects on multiple-choice tests. The Journal of
Psychology, 58, 369-373. Aiken, L.R. (1982). Writing
multiple-choice items to measure high-order educational objectives.
Educational and Psychological Measurement, 42, 803-806. Aiken, L.R. (1987). Testing with
multiple-choice items. Journal of Research Albanese, M.A. (1986). The correction for guessing: a further analysis of Angoff and Schrader. Journal of Educational Measurement, 23, 225-235. Albanese, M.A. (1988). The projected input of the correction for guessing on individual scores. Journal of Educational Measurement, 25, 149-157. Beck M.D. (1978). The effect of item response changes on scores on an elementary reading achievement test. Journal of Educational Research 71, 153-156. Benjamin, L.T., Cavell, T.A. and Shallenberger III, W.R. (1984). Staying with Initial Answers on Objective Tests - Is It a Myth?. Teaching of Psychology, October 1984, 133-141. Ben-Shakhar, G. and Sinai, Y. (1991) Gender differences in multiple-choice tests: the role of differential guessing tendencies. Journal of Educational Measurement, 28, 23-35. Ben-Simon, A., Budescu, D.V., & Nevo, B. (1997). A comparative study of measures of partial knowledge in multiple-choice tests. Applied Psychological Measurement, 21(1), 65-88. Bokhorst, F.D. (1986). Confidence weighting and the validity of achievement tests. Psychological Reports, 59, 383-386. Budescu, D., & Bar-Hillel, M. (1993). To guess or not to guess: a decision-theoretic view of formula scoring. Journal of Educational Measurement, 8(3), 209-214.Cangelosi, J. (1990). "Designing Tests for Evaluating Student Achievement." NY: Longman Collet, L.S. (1971). Elimination scoring: an empirical evaluation. Journal of Educational Measurement, 30(4), 277-291.Coombs, C.H., Milholland, J.E., & Womer, F.B. (1956). The assessment of partial knowledge. Educational and Psychological Measurement, 16, 13-37.Diamond, J., & Evans, W.J. (1973). The correction for guessing. Review of Educational Research, 43(2), 181-191.Dirkzwager, A. (1996). Testing with personal probabilities:11-year-olds can correctly estimate their personal probabilities. Educational and Psychological Measurement, 56(6), 957-971.Ebel, R.L. (1970). The case for true-false items, School Review, 78, 373-389. Edouiard, L.J. & Harris, F.T.C. (1983). Multiple-choice questions: fact, fallacy, and fancy. International Journal of Mathematical Education in Science and Technology, 14, 685-691. Eisner, M. P. (1998). The probability of passing a multiple-choice test The College Mathematics Journal [H.W. Wilson - GS]. Nov 1998. 29, (5), 421-427 Feinberg, L. (1990). Multiple-Choice and Its Critics. In College Board Review, No. 156, 12-17+. Frary, R. B. (1995). More Multiple-Choice Item Writing Do's and Don'ts. ERIC/AE Digest. ERIC Identifier: ED398238. Gives suggestions on content, structure, options, and errors to avoid in the creation of multiple-choice tests. A series of guidelines and examples are discussed. Frary, R.B. (1980). The effect of misinformation, partial information, and guessing on expected multiple-choice item scores. Applied Psychological Measurement, 4 , 79-90.Frary, R.B. (1989). Partial credit scoring methods for multiple choice tests. Applied Measurement in Education, 2 (1), 79-96.Frary, R.B., Cross, L.H., & Lowry, S.R. (1977). Random guessing, correction for guessing and reliability of multiple-choice tests scores. Journal of Experimental Education, 46, 9-15. Gannon, T.J. & Sannito, T. (1969). Discrimination weighting on a multiple-choice exam, Journal of the Iowa Academy of Science, 75, 318-323. Glass, G.V, & Wiley, D.E. (1964). Formula scoring and test reliability. Journal of Educational Measurement, 1, 43-47. Haladyna, Thomas M. (1994). Developing and Validating Multiple-Choice Test Items. Hillsdale, NJ: Lawrence Erlbaum Associates. Haladyna, Thomas M. (1997). Writing Test Items to Evaluate Higher Order Thinking. Needham Heights, MA: Allyn & Bacon. Hambleton, R.K., Roberts, D.M., & Traub, R.E.(1970). A comparison of the reliability and validity of two methods for assessing partial knowledge on a multiple-choice test. Journal of Educational Measurement, 7(2), 75-82.Hanna, G.S. (1977). Study of reliability and validity effects of total and partial immediate feedback in multiple-choice testing. Journal of Educational Measurement, 14, 1-7. Hanna, G.S. & Johnson, F.R. (1978). Reliability and validity of multiple-choice tests developed by four distractor selection procedures. Journal of Educational Research, 71, 203-206. Hsu, T.C., Moss, P.A., & Khampalikit, C. (1984). The merits of multiple-choice items as evaluated by using six scoring formulas. Journal of Experimental Education, 52, 152-158. Jackson, R.A. (1955). Guessing and test performance. Educational and Psychological Measurement, 15, 74-79. Jacobs, Lucy Cheser & Chase, Clinton I. (1992). Developing and Using Tests Effectively: a Guide for Faculty. San Francisco: Jossey-Bass Inc.,. Jaradat, D., & Tollefson, N. (1988). The impact of alternative scoring procedures for multiple-choice tests on test reliability, validity, and grading. Educational and Psychological Measurement, 48(3), 627-635. Little, E.B. (1966). Overcorrection and undercorrection in multiple-choice test scoring,. Journal of Experimental Education, 35, 44-47. Lord, F.M. (1963). Formula scoring and validity. Educational and Psychological Measurement, 23(4), 663-673. Lord, F.M. (1964). The effect of random guessing on test validity. Educational and Psychological Measurement, 24(4), 745-747. Lord, F.M. (1975). Formula scoring and number-right scoring. Journal of Educational Measurement, 12(1), 7-11. Lord, F.M. (1977). Optimal number of choices per item: a comparison of four approaches. Journal of Educational Measurement, 14, 33-38. Marcus, A. (1963). Effect of correct response location on the difficulty level of multiple-choice questions. Journal of Applied Psychology, 47, 48-51. Messick, S. (1989). Validity. In R.L. Linn (Ed.), Educational measurement (3rd. ed., pp. 13-103). New York: American Council on Education and MacMillan. Osterlind, S. J. (1998). Constructing Test Items. 2nd ed. Boston: Kluwer Academic Publishers. Pollard, G. H. (1985). Scoring in multiple-choice examinations. The Mathematical Scientist, 10, 93-97.Prieto, G., & Delgado, A. (1999). The effect of instructions on multiple choice test scores. European Journal of Psychological Assessment, 15(2), 143-150.Ricketts, C. & Wilks, S. J. (2002). Improving Student Performance Through Computer-based Assessment: insights from recent research. Assessment & Evaluation in Higher Education, Sep 2002, Vol. 27 Issue 5, p475, 5p. A comparison between online multiple choice tests and OMR marked multiple choice tests. Rippey, R.M. (1970). A comparison of five different scoring functions for confidence tests. Journal of Educational Measurement, 7(3), 165-170. Rogers, W. T., & Ndalichako, J. (2000). Number-right, item-response, and finite-state scoring: Robustness with respect to lack of equally classifiable options and item option independence. Educational and Psychological Measurement, 60(1), 5-19. Ruch, G.M., & Degraff, M.H. (1926). Corrections for chance and "guess" vs. "do not guess" instructions in multiple-response tests. Journal of Educational Psychology, 17, 368-375. Ruch, G.M., & Stoddard, G.D. (1925). Comparative reliabilities of five types of objective examinations. Journal of Educational Psychology, 16, 89-103. Sax, G., & Collet, L.S. (1968). The effects of differing instructions and guessing formulas on reliability and validity. Educational and Psychological Measurement, 28(4), 1127-1136. Savage, L.J. (1971). Elicitation of personal probabilities and expectations. Journal of the American Statistical Association, 66(336), 783-801. Sherriffs, A.C. & Boomer, D.S. (1954), Who is penalized by the penalty for guessing? Journal of Educational Psychology, 45, 81-90. Shuford, E.H. (1993). In pursuit of the fallacy: resurrecting the penalty. In D.A. Leclercq & J.E. Bruno (Eds.), Item Banking: Interactive Testing and Self-Assessment (pp. 76-98). Berlin: Springer- Verlag. Shuford, E.H., Albert, A.., & Massengill, H.E. (1966). Admissible probability measurement procedures. Psychometrika, 31(2), 125-145. Shuford, E.H., & Brown, T.A., (1975) Elicitation of personal probabilities and their assessment. Instructional Science, 4, 137-188. Siddick, J.T., Barrett, G.V., and Doverspike, D. (1994). Three-alternative multiple-choice tests: an attractive option. Personnel Psychology, 47, 829-835. Slakter, M. (1968). The penalty for not guessing, Journal of Educational Measurement, 5, 141-144. Wang, M.W., & Stanley, J.C. (1970). Differential weighting: a review of methods and empirical studies. Review of Educational Research, 40, 663-705. Warren, G. (1979). Essay versus multiple-choice tests. Journal of Research in Science Teaching, 16, 563-567. Weitzman, E. & McNamara, W.J. (1946). Apt use of the inept choice in multiple-choice testing. Journal of Educational Research, 39, 517-522. Williams, S.I., & Jones, C.O. (1974). Multiple-choice mathematics questions: how students attempt to solve them. Mathematics Teacher, 67, 34-40. Willson, V.L. (1982). Maximising reliability in multiple-choice questions. Educational and Psychological Measurement, 42, 69-72. Willson, V.L. (1989). Cognitive and developmental effects of item performance in intelligence and achievement tests in young children. Journal of Educational Measurement, 26, 103-119. Wood, R. (1977). Multiple choice: a state of the art report. Evaluation in Education, 1, 191-280. . |
|
|
