The Optimal Number of Choices in Multiple-Choice Tests : Some Evidence for Science and Technology Education

The purpose of this study is to compare the psychometric properties of the tests with different numbers of choices used in science and technology education courses. With this purpose in mind, 9 tests were developed consisting of 3-, 4 – and 5-choice questions respectively on a science and technology subject and they were applied throughout 1 year to 41 pre-service teachers. The findings demonstrate that the level of reliability of the items with 3 or 5 choices was found to be higher than that of the items with 4 choices and no significant differences were found among other psychometric properties. In addition, according to the effect size values, the most effective test type was found to be the 3-choice test. According to these results supporting the literature, it can be said that using 3-choice questions in science education is more advantageous than 4 – or 5-choice questions because they are easy both to prepare and analyze.

REFERENCES:

• Anderson, L.W. (2003). Classroom assessment: Enhancing the quality of teacher decision making. New Jersey: Lawrence Erlbaum Associates, Inc.
• Araz, G. (2001). Aynı davranışı ölçmeye yönelik kısa cevaplı, 3 ve 5 seçenekli çoktan seçmeli testlerin madde ve test özelliklerinin şans başarısı ile birlikte incelenmesi [Research on the item and tests stastistics of the short answer, three – and five – multiple choice tests aimed at measuring the same behaviour together with a chance factor]. Unpublished master thesis, Ankara: Hacettepe University.
• Bruno, J.E., & Dirkzwager, A. (1995). Determining the optimal number of alternatives to a multiple-choice test item: An information theoretic perspective. Educational and Psychological Measurement, 55(6), 959–966.
• Budescu, D.V., & Nevo, B. (1985). Optimal number of options: An investigation of the assumption of proportionality. Journal of Educational Measurement, 22, 183–196.
• Bulska, J. (2008). Teacher’ work evaluation as an element of education quality. The New Educational Review, 14(1), 219–224.
• Comrey, A.L., & Lee, H.B. (1992). A first course in factor analysis (2th ed.). New Jersey: Lawrence Erlbaum Associates.
• Costin, F. (1970). The optimal number of alternatives in multiple-choice achievement tests: Some Empirical evidence for a mathematical proof. Educational and Psychological Measurement, 30, 353–358.
• Çakan, M. (2004). Öğretmenlerin ölçme-değerlendirme uygulamaları ve yeterlik düzeyleri: İlk ve Ortaöğretim [Comparison of elementary and secondary school teachers in terms of their assessment practices and perceptions toward their qualification levels]. Ankara University Journal of Faculty of Educational Sciences,, 37(2), 99–114.
• Geering, M. (1993). Gender differences in multiple choice assessment. Unpublished Master Thesis, University of Canberra, Australia.
• Haladyna, T.M., & Downing, S.M. (1989). A taxonomy of multiple-choice ıtemwriting rules. Applied Measurement in Education, 2(1), 37–50.
• Haladyna, T.M., & Downing, S.M. (1993). How many options is enough for a multiple-choice test item? Educational and Psychological Measurement, 53, 999–1010.
• Haladyna, T.M., Downing, S.M., & Rodriguez, M.C. (2002). A review of multiplechoice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309–334.
• Hogben, D. (1973). The reliability, discrimination and difficulty of word knowledge tests employing multiple choice ıtems containing three, four or five alternatives. Australian Journal of Education, 17, 63–68.
• Lord, F.M. (1944). Reliability on multiple-choices tests as a function of number of choices per item. Journal of Educational Psychology, 35(3), 175–180.
• Lord, F.M. (1977). Optimal number of choices per item a comparison of four approaches. Journal of Educational Measurement, 14(1), 33–38.
• Morgil, İ., & Yılmaz, A. (2001). Kimya eğitiminde farklı madde türlerinin psikometrik özellikleri ve öğrenci başarısı bakımından karşılaştırılması [Comparison of the effects of different types of items on the psychometric characteristics of students]. Hacettepe University Journal of Education Faculty, 20, 111–116.
• Popham, W.J. (2002). Classroom assessment: What teachers need to know? Boston: Allyn and Bacon.
• Race, P., Brown, S., & Smith, B. (2005). 500 tips on assessment. (2th ed.), Abingdon: Taylor & Francis Group.
• Rodriguez, M.C. (2005). Three options are optimal for multiple-choice items: A meta-analysis of 80 years of research. Educational Measurement: Issues and Practice, 24(2), 3–13.
• Seinhorst, G. (2008). Are three options better than four? Investigating the effect of reducing the number of options per item on the quality of a multiple-choice reading test. Unpublished Master Thesis, Department of Linguistics and English Language, Lancaster University, England.
• Sidick, J.T., Barret, G.V., & Deverspike, D. (1994). Three-alternative multiple choice tests: An attractive option. Personnel Psychology, 47, 829–835.
• Simkin, M.G., & Kuechler, W.L. (2005). Multiple-choice tests and student understanding: What is the connection? Decision Sciences Journal of Innovative Education, 3(1), 73–97.
• Straton, R.G., & Catts, R.M. (1980). A Comparison of two, three and four-choice item tests given a fixed total number of choices. Educational and Psychological Measurement, 40, 357–365.
• Struyven, K., Dochy, F., & Janssens, S. (2005). Students’ perceptions about evaluation and assessment in higher education: A review, Assessment and Evaluation in Higher Education, 30(4), 331–347.
• Tekin, H. (2004). Eğitimde ölçme ve değerlendirme [Assessment and evaluation in education]. (17th ed), Ankara: Yargı Publishing.
• Torabi-Parizi, R., & Campbell, N.J. (1982). Classroom test writing: effects of item format on test quality. The Elementary School Journal, 83(2), 155–160.
• Trevisan, M.S., Sax, G., & Michael, W.B. (1991). The Effects of the number of options per item and student ability on test validity and reliability. Educational and Psychological Measurement, 51, 829–837.
• Zhang, Z., & Burry-Stock, J.A. ( 2003). Classroom assessment practices and teachers’ self-perceived assessment skills. Applied Measurement in Education, 16(4), 323–342.