top of page
Asian Institute of Research, Journal Publication, Journal Academics, Education Journal, Asian Institute
Asian Institute of Research, Journal Publication, Journal Academics, Education Journal, Asian Institute

Education Quarterly Reviews

ISSN 2621-5799

asia institute of research, journal of education, education journal, education quarterly reviews, education publication, education call for papers
asia institute of research, journal of education, education journal, education quarterly reviews, education publication, education call for papers
asia institute of research, journal of education, education journal, education quarterly reviews, education publication, education call for papers
asia institute of research, journal of education, education journal, education quarterly reviews, education publication, education call for papers
crossref
doi
open access

Published: 24 May 2019

Evaluating Quality of Teacher-Developed English Test in Vocational High School: Content Validity and Item Analysis

Putu Irmayanti Wiyasa, I Ketut Darma Laksana, Ni Luh Ketut Mas Indrawati

Universitas Udayana, Indonesia

asia institute of research, journal of education, education journal, education quarterly reviews, education publication, education call for papers
pdf download

Download Full-Text Pdf

doi

10.31014/aior.1993.02.02.67

Pages: 344-356

Keywords: Content Validity, Test Items Analysis, Validity, Reliability, Index of Difficulty (I), Discrimination Index (DI) and Distractor Efficiency (DE)

Abstract

Teacher-developed test, in the form of multiple-choice questions has widely used for measuring the students' final learning process. Content validity and items analysis is used to evaluate the quality of the test that is developed by the teacher. The study focused on content validity that analyzed the content of the test and basic competency which have been asserted on the syllabus. The content validity analysis using Gregory Formula calculation. In addition, the items test includes validity, reliability, index of difficulty (I), discrimination index (DI) and distractor efficiency (DE) is measured by using respective formula that processing in Microsoft Excel program. This mixed-method study was conducted among 211 students of grade X, XI, and XII of Senior Vocational High School, Sekolah Menengah Kejuruan Negeri 2 Denpasar. Total test items that analyzed in this study were 180 items. The result of this study showed that according to the basic competency and also the topic that occurred on the test, there are 6 out of 180 test items which are not in line with the basic competency on the syllabus of K13. In addition, the items test analysis resulted that in terms of validity, reliability, index of difficulty (I), discrimination index (DI) and distractor efficiency (DE) the tests are categorized as moderate.

References

  1. Brown, H. D., (2004). Language assessment: Principles and classroom practices. White Plains, NY: Pearson Education

  2. Dornyei, Z. (2007). Research methods in applied linguistics: Quantitative, qualitative, and mixed methodologies. New York: Oxford University Press.

  3. Fadarwati, I. P. (2015). Penggunaan Teknik Evaluasi Non-tes dan Hambatannya pada Penilaian Pembelajaran PKn SD di DABIN IV Kecamatan Tonjong Kabupaten Brebes [Skripis]. Semarang (ID): Universitas Negeri Semarang.

  4. Selamat, Suzanah, and Kaur Gurnam Sidhu. 2011. "Student Perceptions of Metacognitive Strategy Use in Lecture Listening Comprehension." Language Education in Asia 185-198. Accessed November 23, 2015. http://dx.doi.org/10.5746/LEiA/11/V2/12/A02/Selamat_Sidhu.

  5. Suchitra, Nair, Yew Li Koo, and Abu Bakar Kesumawati. 2014. "Exploring the Listening Processes of Pre-university ESL Students." Procedia Social and Behavioral Sciences 118: 475-472. www.sciencedirect.com.

  6. Robinson, Martin, Evelina D Galaczi, Coreen Docherty, Anthony King, and Hanan Khalifa. 2014. "Supporting national education reform: The Cambridge Malaysia Baseline Project." Cambridge English: Research Notes, November: 40-44. Accessed November 8, 2016. www.cambridgeenglish.org/research-notes.

  7. 2010-2015. "Official Portal Malaysian Examination Council." STPM and MUET Examination Reports. Accessed May 1, 2016. portal.mpm.edu.my.

  8. Mehrabian, A. 1981. Silent Messages: Implicit Communication of Emotions and Attitudes. Wadsworth: Belmont.

  9. McDevitt, T.M., E.P. Sheenan, and N McMenamin. 1991. "Self-reports of academic listening activities by traditional and non-traditional college students." College Student Journal 25: 478-489.

  10. Brown, J.D. 2005. Testing in language programs: A comprehensive guide to English language assessment. New York: Mc Graw-Hill.

  11. Vivien, Berry, and Barry O'Sullivan. 2014. "Assessing General Language Proficiency." Introducing Language Assessment. London: British Council Digital Services. Accessed November 15, 2016. https://www.britishcouncil.org/exam/aptis/research/project/assessment-literacy/general-language-proficiency.

  12. Taylor, Lynda, and Ardeshir Geranpayeh. 2011. "Assessing listening for academic purposes: Defining and operationalising the test construct." Journal of English for Academic Purposes (Elsevier Ltd) (10): 89-101. doi:10.1016/j.jeap.2011.03.002.

  13. Buck, Gary. 2001. Assessing Listening. Cambridge Language Assessment Series. Edited by J.Charles Anderson and Lyle F Bachman. Cambridge: Cambridge University Press.

  14. English Language Standards and Quality Council. 2015. English Language Education Reform in Malaysia: The Roadmap 2015-2025. Ministry of Education Malaysia.

  15. Weir. 2005. Language Testing and Validation An Evidence-Based Approach. Hampshire: Palgrave Macmillan.

  16. North, Brian; Ortega, Angeles; Sheehan, Susan;. 2010. A Core Inventory for General English. British Councils / EQUALS. Accessed February https://englishagenda.britishcouncil.org, 2017.

  17. Mehrens, William A, and Irvin J Lehmann. 1991. Measurement and Evaluation in Education and Psychology. 4th. Belmont, CA: Wadsworth Thomson Learning.

  18. Field, John. 2008. Listening in the Language Classroom. Cambridge: Cambridge University Press.

  19. Vandergriff, Larry. 2007. "Listening: theory and practice in modern foreign language competence." Subject Centre for Languages, Linguistics and Area Studies Guide to Good Practice. February 22. Accessed January 3, 2016. http://www.llas.ac.uk/resources/gpg/67#toc_1.

  20. Mendelsohn, David. 2001. "Listening Comprehension: We've come a long way but..." Research Symposium, Fall: 33-37. Accessed February 16, 2016. http://www.teslontario.net/publications/researchsympossium/ResearchSympossium2001.pdf.

  21. Rost, Michael. 2002. Teaching & Researching Listening. London: Longman.

  22. D.Wolvin, Andrew, ed. n.d. Listening and Human Communication in the 21st Century. 2010: Wiley-Blackwell.

  23. 2018. "DepEd vision, mission, core values and goals." Department of Education. October 30. www.deped.gov.ph.

  24. 2018. "K to 12 Curriculum Guide in Science." DepEd Tambayan. October 30. https://depedtambayan.blogspot.com.

  25. 2018. "Creating Learning Objectives." Convergence Training. October 30. https://www.convergencetraining.com.

  26. Lucas, Maria Rita D., and Brenda B. Corpuz. 2014. Facilitating Learning : Metacognitive Process. Quezon City, Metro Manila: Lorimar Publishing, Inc.

  27. Mizbani, Maryam, and Azizeh Chalak. 2017. "Analyzing Listening and Speaking Activities of Iranian EFL Textbook Prospect 3 Through Bloom's Revised Taxonomy." Advances in Language and Literacy Studies 1-6.

  28. Zareian, Gholamreza, Mohammad Davoudi, Zahra Heshmatifar, and Javad Rahimi. 2015. "An Evaluation of Questions in Two ESP Coursebooks Based on Bloom's New Taxonomy of Cognitive Learning Domain." International Journal of Education and Research 1-14.

  29. Mizbani, Maryam, and Azizeh Chalak. 2017. "Analyzing Reading and Writing Activities of Iranian EFL Textbook Prospect 3 Based on Bloom's Revised Taxonomy." Journal of Applied Linguistics and Language Research 1-15.

  30. Assaly, Ibtihal R., and Oqlah M Smadi. 2015. "Using Bloom's Taxonomy to Evaluate the Cognitive Levels of Master Class Textbook's Questions." English Language Teaching 1-11.

  31. Olimat, Mohammad. 2015. "Analyzing Action Pack Textbooks' Questions according to Revised Taxonomy." Journal of Education and Practice 1-8.

  32. Al-hasanat, Hasan AbdRabbeh Ali. 2016. "Analyzing Assessment Questions in an Arabic Textbook (Communication skills) for Eight Grade in Jordan According to Bloom's Taxonomy of Levels of Knowledge Aims." World Journal of Education 1-12.

  33. Rahpeyma, Ayoub. 2015. "The Analysis of Learning Objectives in Iranian Junior High School English Textbooks based on Bloom's Revised Taxonomy." International Journal of Education and Literacy Studies 1-12.

  34. Abdi, Heidar, and Mohammad Mohammadi. 2014. "Textbook Evaluation: A Case Study." Procedia Social and Behavioral Sciences 1-8.

  35. 2018. "Effective Use of Performance OBjectives for Learning and Assessment." Teacher and Educational Development. September 26.

  36. Bigcas, Jimayma. 2016. "A Mini Critique of K-12 Basi Education Curriculum." ResearchGate 1-3.

  37. Wilson, Leslie Owen. 2018. "Anderson and Krathwol Bloom's Taxonomy Revised." The Second Principle. October 30. https://thesecondprinciple.com.

  38. 2018. "Revised Bloom's Taxonomy ." October 30. http://www.kurwongbss.qld.edu.au/thinking/Bloom/blooms.htm.

  39. 2018. "Revised Bloom's Taxonomy Action Verbs." October 30. https://www.apu.edu.

  40. Cherry, Kendra. 2018. "The 4 stages of Cognitive Development by Jean Piaget." very well mind. November 12. https://www.verywellmind.com.

  41. n.d. National Book Development Board. Accessed November 21, 2018. books.philippines.gov.ph.

  42. Abistado, Jonna M., Meliza P. Valdoz, Marites D. Aquino, and Mary Anne T. Bascara. 2014. Science Links 9. Quezon City: REX Printing Company Inc.

  43. Alvarez, Liza A., Dave G. Angeles, Hernan L. Apurada, Ma. Pillar P. Carmona, Oliver A. Lahorra, Judith f. Marcaida, Ma. Regaele A. Olarte, et al. 2014. Science Learner's Module 9. Pasig city: FEP Printing Corporation.

  44. Deauna, Melecio C. 1996. Elementary Statistics for Basic Education. Quezon City: Phoenix Publishing House, Inc.

  45. Harmer, Jeremy. 2007. The practice of English language teaching. England: Pearson Education Limited.

  46. Selamat, Suzanah, dan Kaur Gurnam Sidhu. 2011. “Student Perceptions of Metacognitive Strategy Use in Lecture Listening Comprehension.” Language Education in Asia 185-198. Diakses November 23, 2015. http://dx.doi.org/10.5746/LEiA/11/V2/12/A02/Selamat_Sidhu.

  47. Suchitra, Nair, Yew Li Koo, dan Abu Bakar Kesumawati. 2014. “Exploring the Listening Processes of Pre-university ESL Students.” Procedia Social and Behavioral Sciences 118: 475-472. www.sciencedirect.com.

  48. Robinson, Martin, Evelina D Galaczi, Coreen Docherty, Anthony King, dan Hanan Khalifa. 2014. “Supporting national education reform: The Cambridge Malaysia Baseline Project.” Cambridge English: Research Notes, November: 40-44. Diakses November 8, 2016. www.cambridgeenglish.org/research-notes.

  49. 2010-2015. “Official Portal Malaysian Examination Council.” STPM and MUET Examination Reports. Diakses May 1, 2016. portal.mpm.edu.my.

  50. Mehrabian, A. 1981. Silent Messages: Implicit Communication of Emotions and Attitudes. Wadsworth: Belmont.

  51. McDevitt, T.M., E.P. Sheenan, dan N McMenamin. 1991. “Self-reports of academic listening activities by traditional and non-traditional college students.” College Student Journal 25: 478-489.

  52. Brown, J.D. 2005. Testing in language programns: A comprehensive guide to English language assessment. New York: Mc Graw-Hill.

  53. Vivien, Berry, dan Barry O'Sullivan. 2014. “Assessing General Language Proficiency.” Introducing Language Assessment. London: British Council Digital Services. Diakses November 15, 2016. https://www.britishcouncil.org/exam/aptis/research/project/assessment-literacy/general-language-proficiency.

  54. Taylor, Lynda, dan Ardeshir Geranpayeh. 2011. “Assessing listening for academic purposes: Defining and operationalising the test construct.” Journal of English for Academic Purposes (Elsevier Ltd) (10): 89-101. doi:10.1016/j.jeap.2011.03.002.

  55. Buck, Gary. 2001. Assessing Listening. Cambridge Language Assessment Series. Disunting oleh J.Charles Anderson dan Lyle F Bachman. Cambdrige: Cambdrige University Press.

  56. English Language Standards and Quality Council. 2015. English Language Education Reform in Malaysia: The Roadmap 2015-2025. Ministry of Education Malaysia.

  57. Weir. 2005. Language Testing and Validation An Evidence-Based Approach. Hampshire: Palgrave Macmillan.

  58. North, Brian; Ortega, Angeles; Sheehan, Susan;. 2010. A Core Inventory for General English. British Councils / EQUALS. Diakses February https://englishagenda.britishcouncil.org, 2017.

  59. Mehrens, William A, dan Irvin J Lehmann. 1991. Measurement and Evaluation in Education and Psychology. 4th. Belmont, CA: Wadsworth Thomson Learning.

  60. Field, John. 2008. Listening in the Language Classroom. Cambridge: Cambridge University Press.

  61. Vandergriff, Larry. 2007. “Listening: theory and practice in modern foreign language competence.” Subject Centre for Languages, Linguistics and Area Studies Guide to Good Practice. 22 February. Diakses January 3, 2016. http://www.llas.ac.uk/resources/gpg/67#toc_1.

  62. Mendelsohn, David. 2001. "Listening Comprehension: We've come a long way but..." Research Sympossium, Fall: 33-37. Diakses February 16, 2016. http://www.teslontario.net/publications/researchsympossium/ResearchSympossium2001.pdf.

  63. Rost, Michael. 2002. Teaching & Researching Listening. London: Longman.

  64. D.Wolvin, Andrew, penyunt. t.thn. Listening and Human Communication in the 21st Century. 2010: Wiley-Blackwell.

  65. 2018. “DepEd vision, mission, core values and goals.” Department of Education. 30 October. www.deped.gov.ph.

  66. 2018. “K to 12 Curriculum Guide in Science.” DepEd Tambayan. 30 October. https://depedtambayan.blogspot.com.

  67. 2018. “Creating Learning Objectives.” Convergence Training. 30 October. https://www.convergencetraining.com.

  68. Lucas, Maria Rita D., dan Brenda B. Corpuz. 2014. Facilitating Learning : Metacognitive Process. Quezon City, Metro Manila: Lorimar Publishing, Inc.

  69. Mizbani, Maryam, and Azizeh Chalak. 2017. "Analyzing Listening and Speaking Activities of Iranian EFL Textbook Prospect 3 Through Bloom's Revised Taxonomy." Advances in Language and Literacy Studies 1-6.

  70. Zareian, Gholamreza, Mohammad Davoudi, Zahra Heshmatifar, dan Javad Rahimi. 2015. “An Evaluation of Questions in Two ESP Coursebooks Based on Bloom's New Taxonomy of Cognitive Learning Domain.” International Journal of Education and Research 1-14.

  71. Mizbani, Maryam, dan Azizeh Chalak. 2017. “Analyzing Reading and Writing Activities of Iranian EFL Textbook Prospect 3 Based on Bloom's Revised Taxonomy.” Journal of Applied Linguistic and Language Research 1-15.

  72. Assaly, Ibtihal R., dan Oqlah M Smadi. 2015. “Usisng Bloom's Taxonomy to Evaluate the Cognitive Levels of Master Class Textbook's Questions.” English Language Teaching 1-11.

  73. Olimat, Mohammad. 2015. “Analyzing Action Pack Textbooks' Questions according to Revised Taxonomy.” Journal of Education and Practice 1-8.

  74. Al-hasanat, Hasan AbdRabbeh Ali. 2016. “Analyzing Assessment Questions in an Arabic Textbook (Communication skills) for Eight Grade in Jordan According to Bloom's Taxonomy of Levels of Knowledge Aims.” World Journal of Education 1-12.

  75. Rahpeyma, Ayoub. 2015. “The Analysis of Learning Objectives in Iranian Junior High School English Text books based on Bloom's Revised Taxonomy.” International Journal of Education and Literacy Studies 1-12.

  76. Abdi, Heidar, dan Mohammad Mohammadi. 2014. "Textbook Evaluation: A Case Study." Procedia Social and Behavioral Scciences 1-8.

  77. 2018. “Effective Use of Performance OBjectives for Learning and Assessment.” Teacher and Educational Development. 26 September.

  78. Bigcas, Jimayma. 2016. “A Mini Critique of K-12 Basi Education Curriculum.” ResearchGate 1-3.

  79. Wilson, Leslie Owen. 2018. “Anderson and Krathwol Bloom's Taxonomy Revised.” The Second Principle. 30 October. https://thesecondprinciple.com.

  80. 2018. “Revised Bloom's Taxonomy .” 30 October. http://www.kurwongbss.qld.edu.au/thinking/Bloom/blooms.htm.

  81. 2018. “Revised Bloom's Taxonomy Action Verbs.” 30 October. https://www.apu.edu.

  82. Cherry, Kendra. 2018. “The 4 stages of Cognitive Development by Jean Piaget.” verywell mind. 12 November. https://www.verywellmind.com.

  83. t.thn. National Book Development Board. Diakses November 21, 2018. books.philippines.gov.ph.

  84. Abistado, Jonna M., Meliza P. Valdoz, Marites D. Aquino, dan Mary Anne T. Bascara. 2014. Science Links 9. Quezon City: REX Printing Company Inc.

  85. Alvarez, Liza A., Dave G. Angeles, Hernan L. Apurada, Ma. Pillar P. Carmona, Oliver A. Lahorra, Judith f. Marcaida, Ma. Regaele A. Olarte, et al. 2014. Science Learner's Module 9. Pasig city: FEP Printing Corporation.

  86. Deauna, Melecio C. 1996. Elementary Statistics for Basic Education. Quezon City: Phoenix Publishing House, Inc.

  87. Harmer, Jeremy. 2007. The practice of English language teaching. England: Pearson Education Limited.

bottom of page