A COMPARATIVE ANALYSIS OF AI AND HUMAN EVALUATION OF HEDGES AND BOOSTERS IN STUDENT ACADEMIC WRITING

Farida Hidayati

Abstract


Hedges (such as, might, perhaps) and boosters (such as, clearly, undoubtedly) are central epistemic devices and epistemic pragmatics devices in academic writing. However, student writers often find it tough to engage in their certain participation. Hence, it is significant to evaluate how such AI models such as ChatGPT-4.5 compare with human teachers on assessing these epistemic features. This study aims to compare the scores and evaluative feedback provided by an AI model (ChatGPT-4.5) and a human writing instructor in assessing the deployment of hedges and boosters in four undergraduate argumentative essays. The mixed methods were used to analyze the feedback from ChatGPT and an experienced writing instructor and their ratings of each essay across 10 components based on Hyland’s framework on a 5-point scale. The Mann-Whitney U test showed that there was no statistically significant difference in overall scores between the AI and human rater (U-statistic: 59.5000, P-value: 0.4644), indicating general alignment. However, differences were shown at the component level: AI was less variable when it came to identifying hedges and boosters, whereas the instructor added greater context to his comments regarding the appropriateness of use. Both raters were found to correlate moderately for stance evaluation. From a qualitative perspective, thematic analysis revealed an AI's generic phrase usage within a limited context and a teacher pedagogy-grounded, rhetorically informed comment. It turns out that students also stated a more constructive approach for the teacher due to clarity and helpful relevance. 


Keywords


Academic writing; AI evaluation; boosters; evaluation; hedges

Full Text:

PDF

References


Algaraady, J., & Mahyoob, M. (2023). ChatGPT's capabilities in spotting and analyzing writing errors experienced by EFL learners. Arab World English Journal (AWEJ) Special Issue on CALL, 9, 3-17. https://dx.doi.org/10.24093/awej/call9.1

Al-Mudhaffari, M., Hussin, S., & HoAbdullah, I. (2020). Interactional strategies in L2 writing: An exploration of Hedging and Boosting Strategies in Applied Linguistics research articles. International Journal of Arabic-English Studies, 20(1), 171–186. https://doi.org/10.33806/ijaes2000.20.1.9

Charpentier-Jiménez, W. (2024). Evaluación de la inteligencia artificial y de la calibración de docentes en los cursos de escritura de inglés como lengua extranjera en una universidad pública costarricense [Evaluation of artificial intelligence and teacher calibration in English as a foreign language writing courses at a Costa Rican public university]. Actualidades Investigativas en Educación, 24(1), 1–25. https://doi.org/10.15517/aie.v24i1.55612

Dontcheva-Navratilova, O. (2016). Cross-Cultural Variation in the Use of Hedges and Boosters in Academic Discourse. Prague Journal of English Studies, 5(1), 2016. 163-184. https://doi.org/10.1515/pjes-2016-0009

Escalante, J., Pack, A., & Barrett, A. (2023). AI-generated feedback on writing: Insights into efficacy and ENL student preference. International Journal of Educational Technology in Higher Education, 20(1), 57. https://doi.org/10.1186/s41239-023-00425-2

Geçkin, V., Kızıltaş, E., & Çınar, Ç. (2023). Assessing second-language academic writing: AI vs. Human raters. Journal of Educational Technology and Online Learning, 6(4), 1096-1108. https://doi.org/10.31681/jetol.1336599

Herbold, S., Hautli-Janisz, A., Heuer, U., & Meurers, D. (2023). A large-scale comparison of human-written versus ChatGPT-generated essays. Scientific Reports, 13, 18617. https://doi.org/10.1038/s41598-023-45644-9

Hyland, K. (2005). Metadiscourse: Exploring interaction in writing. Continuum.

Karademir Coşkun, T. (2024). Unlocking the Future: The Role of Digital Learning Materials in Fostering 21st-Century Skills Among University Students. In R. Aggarwal, P. Gupta, S. Singh, & R. Bala (Eds.), Augmented Reality and the Future of Education Technology (pp. 229-252). IGI Global Scientific Publishing. https://doi.org/10.4018/979-8-3693-3015-9.ch015

Ningrum, S., Puspita, H., & Mulyadi, A. I. (2024). Hedges and boosters in academic writing of ASEAN EFL learners. Journal of English Education and Teaching, 8(1), 202–218. https://doi.org/10.33369/jeet.8.1.202-218EJournal Universitas Bengkulu

Ryoo, S. M. (2023). Efficacy of hedges in formative feedback on L2 writing. International Journal of Studies in Education, 5(2), 550–567. https://doi.org/10.46328/ijonse.171

Shalevska, E. (2024). Hedges and boosters in AI and human writing: A comparative analysis. Knowledge - International Journal, 60(1), 123–135. https://eprints.uklo.edu.mk/id/eprint/10825/

Shen, Z. (2023). Functions of hedges and boosters in academic writing. Journal of Education Reform and Innovation, 1(1), 13. (https://doi.org/10.61957/joerai-20230102)

Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. https://doi.org/10.3102/0034654307313795

Solak, E. (2024). Revolutionizing Language Learning: How ChatGPT and AI Are Changing the Way We Learn Languages. International Journal of Technology in Education, 7(2), 353-372. https://doi.org/10.46328/ijte.732

Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J., ... & Olson, C. B. (2024). Comparing the quality of human and ChatGPT feedback of students’ writing. Learning and Instruction, 91, 101894. https://doi.org/10.1016/j.learninstruc.2024.101894

Tran, T. S., Nguyen, S. D., Lee, H. J., & Tran, V. P. (2023). Advanced crack detection and segmentation on bridge decks using deep learning. Construction and Building Materials, 400, Article 132839. https://doi.org/10.1016/j.conbuildmat.2023.132839

Wingate, U. (2014). The impact of formative feedback on the development of academic writing. In Approaches to Assessment that Enhance Learning in Higher Education (pp. 29-43). Routledge.

Xiao, F. (肖斐文), Zhu, S. (朱思宇), & Xin, W. (辛闻). (2025). Exploring the landscape of generative AI (ChatGPT)-powered writing instruction in English as a foreign language education: A scoping review. ECNU Review of Education, 0(0). https://doi.org/10.1177/20965311241310881

Yoon, S.-Y., Miszoglad, E., & Pierce, L. R. (2023). Evaluation of ChatGPT feedback on ELL writers' coherence and cohesion. arXiv preprint arXiv:2310.06505. https://arxiv.org/abs/2310.06505




DOI: https://doi.org/10.17509/ije.v18i2.85627

Refbacks

  • There are currently no refbacks.


Copyright (c) 2025 Farida - Hidayati



Lisensi Creative Commons
This work is licensed under Creative Commons Attribution-ShareAlike 4.0 International License.