Angela C. de M. Garcia

DOI Number
First page
Last page


Since the publication of the International Civil Aviation Organization (ICAO)’s language proficiency requirements, a number of different tests have been developed and implemented around the world to assess pilots and air traffic controllers’ proficiency in English. Meanwhile, researchers have questioned the clarity and appropriateness of the policy, and the reliability of tests (e.g., Alderson 2011; Douglas 2004; Emery 2014). ICAO has recently acknowledged that, over the years, multiple interpretations of the policy have led to practices that might undermine the meaningfulness of aviation English tests (ICAO 2022b). However, the ICAO Rating Scale remains as the instrument to be used in assessing pilots and air traffic controllers’ aeronautical language proficiency. Thus, this article explores the construct of aeronautical English listening tests stated in the comprehension descriptors of the ICAO rating scale, as well as the elements of the other descriptors that may inform the definition of this construct. An in-depth content analysis of the rating scale was conducted by using the “interview technique”, as described by O’Leary (2021). Results provide useful information for the development of listening tests in the aeronautical context. A better interpretation of the construct informed by the policy can help to reduce the differences among test implementations around the world and further contribute to more standardized and meaningful testing practices.


language testing, ICAO rating scale, listening comprehension, construct definition

Full Text:



Alderson, J. Charles, Neus Figueras, Henk Kuijper, Guenter Nold, Sauli Takala, and Claire Tardieu. “Analysing Tests of Reading and Listening in Relation to the Common European Framework of Reference: The Experience of The Dutch CEFR Construct Project.” Language Assessment Quarterly 3, no. 1 (2006): 3–30.

Alderson, J. Charles. “A Survey of Aviation English Tests.” Language Testing, 27(1), (2010): 51-72. doi: 10.1177/0265532209347196

Alderson, J. Charles. “The Politics of Aviation English Testing. Language.” Assessment Quarterly, 8:4 (2011): 386-403. doi: 10.1080/15434303.2011.622017References

Aragão, Beatriz F. “O Uso de Critérios Autóctones no Contexto Aeronáutico: Contribuições para uma Nova Escala de Proficiência para Controladores de Tráfego Aéreo” [The Use of Indigenous Assessment Criteria in the Aeronautical Context: Contributions to a New Proficiency Rating Scale for Air Traffic Controllers]. In Matilde V. R. Scaramucci, Patricia Tosqui-Lucks, and Silvia M. Damião (Eds.), Pesquisas sobre Inglês Aeronáutico no Brasil. Campinas: Pontes Editores, 2018: 243−269.

Bonk, William J. “Second Language Lexical Knowledge and Listening Comprehension.” International Journal of Listening 14, no. 1 (2000): 14–31.

Borowska, Anna P. Avialinguistics:The Study of Language for Aviation Purposes. Frankfurt am Main: Peter Lang Edition, 2017.

Bowen, Glenn A. “Document Analysis as a Qualitative Research Method.” Qualitative Research Journal 9, no. 2 (2009): 27–40.

Brazil. The Need for Revision of Annex 1 Language Proficiency SARPS for Pilots and Air Traffic Controllers. International Civil Aviation Organization Working Paper A39-WP/249, 2016. Retrieved 14 December, 2022, from


Buck, Gary. Assessing Listening. Cambridge: Cambridge University Press, 2001.

Cardno, Carol. “Policy Document Analysis: A Practical Educational Leadership Tool and a Qualitative Research Method.” Educational Administration: Theory and Practice 24, no. 4 (2018): 623-640.

Douglas, Dan. “Three Problems in Testing Language for Specific Purposes: Authenticity, Specificity and Inseparability.” In Experimenting with Uncertainty: Essays in Honour of Alan Davies. Cambridge: Cambridge University Press, 2001: 45–52.

Douglas, Dan. “Assessing the Language of International Civil Aviation: Issues of Validity and Impact.” Proceedings from the International Professional Communication Conference, IEEE Professional Communication Society. IEEE (2004): 248-252

Douglas, Dan. Understanding Language Testing. London: Hodder Education, 2010.

Douglas, Dan. “Nobody Seems to Speak English Here Today: Enhancing Assessment and Training in Aviation English.” Iranian Journal of Language Teaching Research, 2(2), (2014): 1-12.

Emery, Henry J. “Standardization in Language Rating: An Association of Language Raters.” Second ICAO Aviation Language Symposium. (May, 2007), available at

Emery, Henry J. “Developments in LSP Testing 30 Years On? The Case of Aviation English.” Language Assessment Quarterly 11, no. 2 (2014): 198–215.

Estival, Dominique, Candace Farris, and Brett Molesworth. Aviation English: A Lingua Franca for Pilots and Air Traffic Controllers. New York: Routledge, 2016.

Field, John. “Idle Chatter: What Really Goes On in Tests of Interactive Communication?”, CRELLA Symposium. (2020), available at


Garcia, Angela C. M. What Do ICAO Language Proficiency Test Developers and Raters Have to Say about the ICAO Language Proficiency Requirements 12 years after their Publication? A Qualitative Study Exploring Experienced Professionals’ Opinions. Unpublished Master’s Thesis. Lancaster University, 2015.

George, Alexander. “Quantitative and Qualitative Approaches to Content Analysis.” In Klaus Krippendorff and Mary A. Bock (Eds.), The Content Analysis Reader. Thousand Oaks, CA: Sage Publications, 2009: 144-155.

Howard, John W. “‘Tower, Am I Cleared to Land?’: Problematic Communication in Aviation Discourse.” Human Communication Research 34, no. 3 (2008): 370–391.

ICAEA. Guidelines: ICAO LPR Test Design Guidelines. International Civil Aviation English Association. Retrieved December 14, 2022, from


ICAO. ICAO Doc 9835. Manual on the Implementation of ICAO Language Proficiency Requirements. International Civil Aviation Organization. 2004.

ICAO. ICAO Cir 318. Language Testing Criteria for Global Harmonization. International Civil Aviation Organization. 2009.

ICAO. ICAO Doc 9835. Manual on the Implementation of ICAO Language Proficiency Requirements (2nd ed.). International Civil Aviation Organization, 2010.

ICAO. Implementation Status of English Language Proficiency Requirements. Working Paper A38.WP/37. International Civil Aviation Organization, 2013. Retrieved 08 March, 2023, from

ICAO. Report of the Technical Commission on Agenda Item 35. Working Paper A39.WP/514. International Civil Aviation Organization, 2016. Retrieved 14 December, 2022, from

ICAO. Annex 1 to the Convention on International Civil Aviation – Personnel Licensing (14th ed.). International Civil Aviation Organization, 2022a.

ICAO. ICAO Language Proficiency Requirements: Achieving Standardization | Catalin Popa [Video]. Skytalks. ICAO TV. 2022b October. Retrieved from

Jacoby, Sally, and Tim McNamara. “Locating Competence.” English For Specific Purposes 18, no. 3 (1999): 213–241.

Kim, Hyejeong, and Catherine Elder. “Understanding Aviation English as a lingua franca: Perceptions of Korean aviation personnel.” Australian Review of Applied Linguistics 32(3), (2009): 23.1-23.17.

Kim, Hyejeong, and Catherine Elder. “Interrogating the Construct of Aviation English: Feedback from Test Takers in Korea.” Language Testing 32, no. 2 (2015): 129–149.

Kim, Hyejeong. “Exploring the Construct of Radiotelephony Communication: A critique of the ICAO English testing policy from the perspective of Korean aviation experts.” Papers in Language Testing and Assessment 2(2), (2013): 103-110

Kim, Hyejeong, and Rosey Billington. “Pronunciation and Comprehension in English as a Lingua Franca Communication: Effect of L1 Influence in International Aviation Communication.” Applied Linguistics 39, no. 2 (2018): 135–158.

Knoch, Ute. “Collaborating with ESP Stakeholders in Rating Scale Validation: The Case of the ICAO Rating Scale.” Spaan Fellow Working Papers in Second or Foreign Language Assessment 7 (2009): 21-46.

Knoch, Ute. “Using Subject Specialists to Validate an ESP Rating Scale: The Case of the International Civil Aviation Organization (ICAO) Rating Scale.” English for Specific Purposes 33 (2014): 77-86.

Knoch, Ute, Bart Deygers, and Apichat Khamboonruang. “Revisiting Rating Scale Development for Rater-Mediated Language Performance Assessments: Modelling Construct and Contextual Choices Made by Scale Developers.” Language Testing 38, no. 4 (2021): 602–626.

Knoch, Ute., and Susan Macqueen. Assessing English for Professional Purposes. Milton: Routledge, 2020.

Lam, Daniel M. K. “Don’t Turn a Deaf Ear: A Case for Assessing Interactive Listening.” Applied Linguistics 42, no. 4 (2021): 740–764.

McNamara, T. F., Ute Knoch, and Fan Jason. Fairness, Justice, and Language Assessment: The Role of Measurement. Oxford, United Kingdom: Oxford University Press, 2019.

Merriam, Sharan B. Case Study Research in Education: A Qualitative Approach. 1st ed. San Francisco: Jossey-Bass, 1988.

Merriam, Sharan B. Qualitative Research and Case Study Applications in Education. 2nd ed. San Francisco: Jossey-Bass Publishers, 1998.

Messick, Samuel. “Validity.” In Educational Measurement (3rd ed.), ed. Robert L. Linn, 13–103. Phoenix: Macmillan, 1989.

Monteiro, Ana L. T. Reconsidering the Measurement of Proficiency in Pilot and Air Traffic Controller Radiotelephony Communication: From Construct Definition to Task Design. PhD’s Dissertation. Carleton University, 2019.

Monteiro, Ana L. T. “The Assessment of English in Aeronautical Radiotelephony Communications: A Mixed Methods Study.” In Aviation English: A Global Perspective, ed. Patricia Tosqui-Lucks. Sao Paulo: Bookerfield, 2022, 224-245.

Munro, Murray, and Tracey Derwing. “Foreign Accent, Comprehensibility, and Intelligibility in the Speech of Second Language Learners.” In Language Learning, 49 (sup. 1), (1999): 285-310.

O’Leary, Zina. The Essential Guide to Doing Your Research Project. 4th edition. London: SAGE Publications, 2021.

Pfeiffer, Anja. Inter-Rater Reliability in an Aviation Speaking Test. Master’s Thesis. Lancaster University, 2009.

Popa, Catalin. “ICAO Language Proficiency Requirements: Occurrence, Developments and Associated Documents.” Uniting Aviation. 05 April 2019. Retrieved December 14, 2022, from

Prado, Malila C. A. Levantamento de Padrões Léxico-Gramaticais do Inglês para Aviação: Um Estudo Vetorado pela Linguística de Corpus [Survey on the Lexicogrammatical Patterns in Aviation English: A Study Conducted by corpus linguistics]. Master’s Thesis. Universidade de São Paulo, 2015.

Prinzo, O.Veronika, A. M. Hendrix, and R. Hendrix. The Outcome of ATC Message Length and Complexity on En Route Pilot Readback Performance. DOT/FAA/AM-09/2. Washington, DC: Federal Administration, 2009.

Read, John, and Ute Knoch. “Clearing the Air: Applied Linguistic Perspectives on Aviation Communication.” Australian Review of Applied Linguistics, 32(3), (2009): 21.1−21.11.

Rost, Michael, and Steven Brown. “Second Language Listening.” In Handbook of Practical Second Language Teaching and Learning, ed. Eli Hinkel. Routledge, 2022.

Shohamy, Elana G. The Power of Tests: A Critical Perspective on the Uses of Language Tests. Harlow, England: Longman, 2001.

Silva, Ana L. B. C. “Language Proficiency Test Design for a Group of Military Pilots: Criteria, Construct and Purpose.” The Journal of Teaching English for Specific and Academic Purposes, 2023.

Tosqui- Lucks, Patricia, and Ana L. B. C. Silva. Da Elaboração de um Glossário Colaborativo à Discussão sobre os Termos Inglês para Aviação e Inglês Aeronáutico. [From Writing a Collaborative Glossary to Discussing the Terms “Aviation English” and “Aeronautical English”]. Estudos Linguísticos, 48(4), (2020): 97– 116. doi:10.21165/ el.v49i1.25610

Treadaway, Maria. “The ICAO Scale and Language Testing for Ab Initio Cadets: Is There a Fit?” In Aviation English: A Global Perspective, ed. Patricia Tosqui-Lucks. Sao Paulo: Bookerfield, 2022, 246-261.

Trippe, Julia E. (2018). Aviation English is Distinct from Conversational English: Evidence from Prosodic Analyses and Listening Performance. ProQuest Dissertations Publishing.

Upshur, John A, and Carolyn E Turner. “Constructing Rating Scales for Second Language Tests.” ELT Journal 49, no. 1 (1995): 3–12.

Wodak, Ruth. “Linguistic Analyses in Language Policies.” In T. Ricento (Ed.), An Introduction to Language Policy: Theory and Method. Oxford: Blackwell, 2006: 170-193.



  • There are currently no refbacks.

ISSN 2334-9182 (Print)
ISSN 2334-9212 (Online)