PENGEMBANGAN SISTEM KLASIFIKASI GESTUR BAHASA ISYARAT BERBASIS GAMIFIKASI ADAPTIF MENGGUNAKAN SVM DENGAN MEKANISME ADAPTIVE GESTURE DIFFICULTY LEVELING (AGDL)

Reno Nilam Sari, Dedy Irfan, Syafrijon Syafrijon, Randi Proska Sandra

Abstract


Penelitian ini bertujuan untuk mengembangkan sistem klasifikasi gestur Sistem Isyarat Bahasa Indonesia (SIBI) berbasis peramban web yang interaktif guna memfasilitasi pembelajaran inklusif bagi komunitas Tuli. Pendekatan yang digunakan mengintegrasikan ekstraksi landmark MediaPipe dan algoritma Support Vector Machine (SVM) yang dieksekusi murni di sisi klien (client-side inference) menggunakan format ONNX, serta dilengkapi dengan mekanisme Adaptive Gesture Difficulty Leveling (AGDL). Hasil evaluasi menunjukkan model SVM mencapai akurasi sebesar 98,92% dalam mengenali 36 kelas gestur, dengan skor System Usability Scale (SUS) mencapai 86,0 (Best Imaginable). Implementasi arsitektur client-side terbukti mampu mengeliminasi latensi server dan mengamankan privasi data pengguna, sementara penerapan elemen gamifikasi adaptif secara dinamis berhasil mempertahankan keterlibatan pengguna pada zona flow. Dampak dari inovasi ini adalah terciptanya solusi produk edukasi teknologi (EdTech) yang skalabel, aman, dan dapat diadopsi secara luas untuk menjembatani kesenjangan komunikasi tanpa memerlukan infrastruktur cloud yang berbiaya tinggi.

Keywords


Client-Side Inference; EdTech; Gamification; MediaPipe; SIBI; Support Vector Machine

Full Text:

PDF

References


Aggarwal, S., & Verma, J. (2022). Comparative analysis of ReactJS and AngularJS in web development. International Journal of Computer Applications, 183(45), 34–38. https://doi.org/10.5120/ijca2022921868

Bora, J., Dehingia, S., Boruah, A., Chetia, A. A., & Gogoi, D. (2023). Real-time Assamese Sign Language Recognition using MediaPipe and Deep Learning. Procedia Computer Science, 218, 1384–1393. https://doi.org/10.1016/j.procs.2023.01.117

Choi, J., Kim, S., & Park, J. (2020). Efficient client-side deep learning for privacy-preserving web applications. Web Intelligence, 18(2), 115–128. https://doi.org/10.3233/WEB-200435

Dichev, C., & Dicheva, D. (2017). Gamifying education: What is known, what is believed and what remains uncertain: A critical review. International Journal of Educational Technology in Higher Education, 14, 9. https://doi.org/10.1186/s41239-017-0042-5

Grandini, M., Bagli, E., & Visani, G. (2020). Metrics for multi-class classification: An overview. ArXiv Preprint ArXiv:2008.05756. https://arxiv.org/abs/2008.05756

Jin, T., Bercea, G.-T., Le, T. D., Chen, T., Su, G., Imai, H., Negishi, Y., Leu, A., O’Brien, K., Kawachiya, K., & Eichenberger, A. E. (2020). Compiling ONNX Neural Network Models Using MLIR. http://arxiv.org/abs/2008.08272

Khekare, G., Panigrahi, G. R., Singh, V., Majumder, G., & Shelke, N. (2025). Adaptive ensemble learning for real time sign language recognition. 2025 3rd International Conference on Networks & Advances in Computational Technologies (NetACT). https://doi.org/10.1109/NetACT65906.2025.11188939

Krath, J., Schürmann, L., & von Korflesch, H. F. O. (2021). Revealing the theoretical basis of gamification: A systematic review and analysis of theory in research on gamification, serious games and game-based learning. Computers in Human Behavior, 125. https://doi.org/10.1016/j.chb.2021.106963

Kupidura, P., Kępa, A., & Krawczyk, P. (2024). Comparative analysis of the performance of selected machine learning algorithms depending on the size of the training sample. Reports on Geodesy and Geoinformatics, 118(1). https://doi.org/10.2478/rgg-2024-0015

Legaki, N.-Z., Xi, N., Hamari, J., Karpouzis, K., & Assimakopoulos, V. (2020). The effect of challenge-based gamification on learning: An experiment in the context of statistics education. International Journal of Human-Computer Studies, 144, 102496. https://doi.org/10.1016/j.ijhcs.2020.102496

Liu, H.-I., Galindo, M., Xie, H., Wong, L.-K., Shuai, H.-H., Li, Y.-H., & Cheng, W.-H. (2024). Lightweight Deep Learning for Resource-Constrained Environments: A Survey. http://arxiv.org/abs/2404.07236

Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C.-L., Yong, M. G., Lee, J., Chang, W.-T., Hua, W., Georg, M., Grundmann, M., & Kalakrishnan, R. (2019). MediaPipe: A framework for building perception pipelines. ArXiv Preprint ArXiv:1906.08172. https://arxiv.org/abs/1906.08172

Niswati, Z., Mustajib, F. S., & Sujatmiko, A. (2021). Sign language recognition using Support Vector Machine (SVM) and convex hull algorithm. Journal of Physics: Conference Series, 1842(1), 012014. https://doi.org/10.1088/1742-6596/1842/1/012014

Nugraheni, A. S., Husain, A. P., & Unayah, H. (2023). OPTIMALISASI PENGGUNAAN BAHASA ISYARAT DENGAN SIBI DAN BISINDO PADA MAHASISWA DIFABEL TUNARUNGU DI PRODI PGMI UIN SUNAN KALIJAGA. Jurnal Holistika, 5(1), 28. https://doi.org/10.24853/holistika.5.1.28-33

Ostermann, A., Vollmuth, P., & Ziemsky, V. (2023). Design and Application of the unIT-e2 Project Use Case Methodology. World Electric Vehicle Journal, 14(1). https://doi.org/10.3390/wevj14010013

Paudyal, P., Banerjee, A., & Gupta, S. (2020). On evaluating the effects of feedback for sign language learning using explainable AI. Companion Proceedings of the 25th International Conference on Intelligent User Interfaces (IUI ’20 Companion), 83–84. https://doi.org/10.1145/3379336.3381469

Romão, L. C., Villamizar, H., Oliveira, R., Alonso, S., & Kalinowski, M. (2025). Agile management for machine learning: A systematic mapping study. ArXiv Preprint ArXiv:2506.20759. https://arxiv.org/abs/2506.20759

Saleh, A. (2025). A Comparative Analysis of CNN and SVM for Static Sign Language Recognition Using MediaPipe Landmarks. Journal of Intelligent System and Telecommunication, 1(2), 225–238. https://doi.org/10.26740/jistel.v1n2.p225-238

Sruthi, C. J., & Sasikala, A. L. (2021). Sign language recognition using SVM and neural network. 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS), 774–778. https://doi.org/10.1109/ICACCS51430.2021.9441865

Wang, Q., Jiang, S., Chen, Z., Cao, X., Li, Y., Li, A., Ma, Y., Cao, T., & Liu, X. (2025). Anatomizing Deep Learning Inference in Web Browsers. ACM Transactions on Software Engineering and Methodology, 34(2). https://doi.org/10.1145/3688843

Wibowo, A. S., Purnama, B. E., & Wardati, I. U. (2023). Aplikasi pembelajaran bahasa isyarat SIBI berbasis Android menggunakan metode Linear Congruent Method. Jurnal Ilmiah SINUS, 21(1), 1–12. https://doi.org/10.30646/sinus.v21i1.667

Zhang, F., Bazarevsky, V., Vakunov, A., Tkachenka, A., Sung, G., Chang, C.-L., & Grundmann, M. (2020). MediaPipe Hands: On-device Real-time Hand Tracking. http://arxiv.org/abs/2006.10214




DOI: https://doi.org/10.17509/ijdb.v5i4.99529

Refbacks

  • There are currently no refbacks.


Copyright (c) 2026 Universitas Pendidikan Indonesia (UPI)

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Indonesian Journal of Digital Business is published by Universitas Pendidikan Indonesia (UPI)
and managed by Department of Digital Business
Jl. Dr. Setiabudi No.229, Kota Bandung, Indonesia - 40154
View My Stats