COMPARING HOLISTIC AND ANALYTIC WAYS OF SCORING IN THE ASSESSMENT OF SPEAKING SKILLS

Rastislav Metruk

DOI Number
10.22190/JTESAP1801179M
First page
179
Last page
189

Abstract


of testing oral performance are usually applied: holistic and analytic scoring. In the present study, these two ways of evaluating the spoken proficiency are explored in order to examine the relationship between them. English speaking skills of a total of 50 subjects, who are Slovak university EFL (English as a foreign language) students, were assessed by an interlocutor and an assessor. The interlocutor conducted the holistic scoring, while the assessor performed the analytic scoring. Categories within the analytic scoring consisted of content and organisation, pronunciation, vocabulary, and grammar. The overall average for the four criteria was 3.32, while the holistic scoring mean was 3.56. The results demonstrate that there exists a statistically significant difference between the holistic and analytic ways of assessment as the p-value was calculated at 0.001 (p < 0.05). It is, therefore, suggested that employing both ways of scoring in the assessment process might be considered appropriate as they appear to complement each other, and together contribute towards more objective assessment.

Keywords

assessing speaking skills, holistic scoring, analytic scoring, teaching English as a foreign language

Full Text:

PDF

References


Al-Amri, M. (2010). Direct Spoken English Testing is Still a Real Challenge to be Worth bothering About. English Language Teaching, 3 (1), 113-117.

Aleksandrzak, M. (2011). Problems and challenges in teaching and learning speaking at advanced level. Glottodidactica, 37, 37-48.

Allen, D., & Tanner, K. (2006). Rubrics: Tools for Making Learning Goals and Evaluation Criteria Explicit for Both Teachers and Learners. CBE— Life Sciences Education, 5 (3), 197–203. http://doi.org/10.1187/cbe.06-06-0168

Anderson, J. (2015). A Guide to the Practice of English language teaching for Teachers and Trainee Teachers. Nairobi: East African Educational Publishers Ltd.

Babik, D., Gehringer, E., Kidd, J., Pramudianto, F. & Tinapple, D. (2016). Probing the landscape: Toward a systematic taxonomy of online peer assessment systems in education. Unknown Journal 1633.

Baryla, E., Shelley, G., & Trainor, W. (2012). Transforming Rubrics using Factor Analysis. Practical Assessment. Practical Assessment Research & Evaluation, 17 (4).

Bilá, M. (2010). Perception and Production of a Second Language and the Concept of a Foreign Accent. In S. Pokrivčáková et al. (Eds.) Modernization of Teaching Foreign Languages: CLIL, Inclusive and Intercultural Education, pp. 123-143. Brno: Masaryk University.

Cambridge English: Understanding Results Guide. (2014). http://www.gml.cz/prof/zajickova/Cambridge%20exams_information/Understanding%20results%20guide.pdf

Chen, G. (2016). Developing a Model of Analytic Rating Scales to Assess College Students’ L2 Chinese Oral Performance. International Journal of Language Testing, 6 (2), 50-71.

Chuang, Y. (2009). Foreign Language Speaking Assessment: Taiwanese College English Teachers’ Scoring Performance in the Holistic and Analytic Rating Methods. The Asian EFL Journal, 11 (1), 150-173.

Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge: Press Syndicate of the University of Cambridge.

Davies, A. (1999). Dictionary of Language Testing. Cambridge: Cambridge University Press.

Dogan, C., and Uluman. M. (2017). A Comparison of Rubrics and Graded Category Rating Scales with Various Methods Regarding Raters’ Reliability. Educational Sciences: Theory & Practice, 17 (2), 631-651. doi:10.12738/estp.2017.2.0321

Finson, K., Ormsbee, C., & Jensen, M. (2011). Differentiating Science Instruction and Assessment for Learners with Special Needs, K-8. Thousand Oaks: Corwin, A SAGE Company.

Goh, C., & Burns, A. (2012). Teaching Speaking. A Holistic Approach. New York: Cambridge University Press.

Gondová, D. (2014). Taking First Steps in Teaching English: Assessing Learners. Žilina: EDIS.

Göktürk, N. (2016). Examining the Effectiveness of Digital Video recordings on Oral Performance of EFL Learners. Teaching English with Technology, 16 (2), 71-96.

Green, A. (2014). Exploring Language Assessment and Testing. Language in Action. New York: Routledge.

Griffith, W., & Lim, H. (2012). Performance-Based Assessment: Rubrics, Web 2.0 Tools and Language Competencies. Mextesol Journal, 36 (1).

Helvoort, J. (2010). A Scoring Rubric for Performance Assessment of Information Literacy in Dutch Higher Education. Journal of Information Literacy, 4 (1), 22-39. http://dx.doi.org/10.11645/4.1.1256

Jankowska, A., & Zielińska, U. (2015). Designing a Self-Assessment Instrument for Developing the Speaking Skill at the Advanced Level. In

M. Pawlak and E. Waniek-Klimczak (Eds.) Issues in Teaching, Learning and Testing Speaking in a Second Language. Heidelberg: Springer. https://doi.org/10.1007/978-3-642-38339-7_16

Jonnson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2 (2), 130-144. https://doi.org/10.1016/j.edurev.2007.05.002

Kaba, Y., & Sengül, S. (2016). Developing the rubric for Evaluation Problem Posing (REPP). International Online Journal of Educational Sciences, 8 (1), 8-25.

Khamkhien, A. (2010). Teaching English Speaking and English Speaking Tests in the Thai Context: A reflection from Thai Perspective. English Language Teaching, 3 (1), 184-190.

Kondo-Brown, K. (2002). A FACETS Analysis of Rater Bias in Measuring Japanese Second Language Writing Performance. Language Testing, 19 (1), 3-31. https://doi.org/10.1191/0265532202lt218oa.

Llach, M. (2011). Lexical Errors and Accuracy in Foreign Language Writing. New York: Multilingual Matters.

Luoma, S. (2004). Assessing Speaking. Cambridge: Cambridge University Press.

Metruk, R. (2017). Pronunciation of English Dental Fricatives by Slovak University EFL Students. International Journal of English Linguistics, 7 (3), 11-16. doi:10.5539/ijel.v7n3p11

Murugaiah, P. (2016). Pecha Kucha Style Powerpoint Presentation: An Innovative Call Approach to Developing Oral Presentation Skills of Tertiary Students. Teaching English with Technology, 16 (1), 88-104.

Myford, C., & Wolfe, E. (2003). Detecting and Measuring Rater Effects Using Many-Facet Rasch Measurement: Part I. Journal of Applied Measurement, 4 (4), 386-422.

O’Sullivan, B. (2012). Assessing Speaking. In C. Coombe, P. Davidson, B. O’Sullivan, & S. Stoynoff (Eds.) The Cambridge Guide to Second Language Assessment, pp. 234-246. New York: Cambridge University Press.

Pan, M. (2016). Nonverbal Delivery in Speaking Assessment. From an Argument to a Rating Scale Formulation and Validation. Singapore: Springer.

Pokrivčáková, S. (2010). Modern Teacher of English. Nitra: ASPA.

Razali, K., & Isra, M. (2016). Male and Female Teachers Roles in Assessment of Speaking Skill. Gender Equality: International Journal of Child and Gender Studies, 2 (1), 1-10.

Reddy, Y. (2007). Effect of Rubrics on Enhancement of Student Learning. Educate~, 7 (1), 3-17.

Richards, J., & Schmidt, R. (2013). Longman Dictionary of Language Teaching and Applied Linguistics (4th ed.). New York: Routledge.

Ruammai, P. (2014). Constructing Scoring Instrument for Writing Assessment and Fostering Critical Thinking. In H. Lee (Ed.) The International Conference on Language and Communication Innovative Inquiries and Emerging Paradigms in Language, Media and Communication, pp. 127-137.

Saritha, K. (2016). Rubric for English Language Teaching Research. Research Journal of English Language and Literature, 4 (2), 725-731.

Sarwar, M., Alam, M., Hussain, S., Shah, A., & Jabeen, M. (2014). Assessing English speaking skills of prospective teachers at entry and graduation level in teacher education program. Language Testing in Asia, 4 (5). https://doi.org/10.1186/2229-0443-4-5

Schunn, C., Godley, A., & DeMartino, S. (2016). The Reliability and Validity of Peer Review of Writing in High School AP English Classes. Journal of Adolescent & Adult Literacy, 60 (1), 13-23. doi:10.1002/jaal.525

Shatrova, Z., Mullings, R., Blažejová, S., & Üstünel, E. (2017). English Speaking Assessment: Developing a Speaking Test for Students in a Preparatory School. International Journal of English Language Teaching, 5 (3), 27-40.

Taylor, L., & Galaczi, E. (2011). Scoring Validity. In L. Taylor (Ed.), M. Milanovic & C. Weir (Series Eds.) Studies in Language Testing 30. Examining Speaking. Research and Practice in Assessing Second Language Speaking, pp. 171-233. Cambridge: Cambridge University Press.

Thornbury, S. (2005). How to Teach Speaking. Harlow: Pearson Education Limited.

Topkaraoğlu, M., & Dilman, H. (2014). Effects of Studying Vocabulary Enhancement Activities on Students’ Vocabulary Production Levels. Procedia - Social and Behavioral Sciences, 152, 931-936. https://doi.org/10.1016/j.sbspro.2014.09.345

Tuan, L. (2012). Teaching and Assessing Speaking Performance through Analytic Scoring Approach. Theory and Practice in Language Studies, 2 (4), 673-679. doi:10.4304/tpls.2.4.673-679.

Ur, P. (2012). A Course in English Language Teaching. Cambridge: Cambridge University Press.

Vafee, P., & Yaghmaeyan, B. (2015). Providing Evidence for the Generalizability of a Speaking Placement Test Scores. Iranian Journal of Language Testing, 5 (2), 78-95.

Xi, X. (2007). Evaluating Analytic Scoring for the TOEFL® Academic Speaking Test (TAST) for Operational Use. Language Testing, 24 (2), 251-286. https://doi.org/10.1177/0265532207076365.




DOI: https://doi.org/10.22190/JTESAP1801179M

Refbacks

  • There are currently no refbacks.


ISSN 2334-9182 (Print)
ISSN 2334-9212 (Online)