Real-Time Hand Gesture Recognition for Controlling Web Video of Traditional Dance Utilizing MediaPipe

Mahardika Pratama; Mochamad Rabbani Fathur; Reza Aziiz Maulana; Trisna Gelar; Iwan Awaludin; Muhammad Rizqi Sholahuddin; Aprianti Nanda Sari

doi:10.17509/edsence.v7i1.82344

Real-Time Hand Gesture Recognition for Controlling Web Video of Traditional Dance Utilizing MediaPipe

Mahardika Pratama, Mochamad Rabbani Fathur, Reza Aziiz Maulana, Trisna Gelar, Iwan Awaludin, Muhammad Rizqi Sholahuddin, Aprianti Nanda Sari

Abstract

The interaction between humans and digital devices has significantly evolved with a notable progression in Human-Computer Interaction (HCI) being the increasing adoption of contactless device control through hand gesture recognition. This research details the development and evaluation of a real-time hand gesture recognition system utilizing the MediaPipe framework for controlling web video players, with a specific focus on its application in managing the playback of traditional dance videos. The system enables users to manipulate video playback functions via hand gestures. The research methodology comprised key stages: a literature review, system workflow design, practical implementation, and performance evaluation. The MediaPipe framework facilitated real-time detection of hand landmark coordinates, processed, and mapped to specific video player control commands (e.g., play, pause, volume adjustment) using a rule-based approach. The findings indicate that the system performs very well in bright lighting conditions and at close user distances (0.5 meters) from the camera, accurately recognizing gestures with high responsiveness. However, recognition accuracy significantly decreases at further distances (>3 meters) and in dark lighting conditions, highlighting the importance of optimal camera placement and sufficient illumination for effective system operation. The evaluation results indicate effective control of web video players, demonstrating optimal performance in environments with bright illumination and close user proximity.

Keywords

hand gesture recognition; mediapipe framework; web video player control; contactless interaction; traditional dance

Full Text:

PDF

References

Acosta, T., Zambrano-Miranda, J., & Lujan-Mora, S. (2020). Techniques for the Publication of Accessible Multimedia Content on the Web. IEEE Access, 8, 55300–55322.

Agustiani, A. D., Putri, S. M., Hidayatullah, P., & Sholahuddin, M. R. (2024). Penggunaan MediaPipe untuk Pengenalan Gesture Tangan Real-Time dalam Pengendalian Presentasi. Media Jurnal Informatika, 16(2).

Awaludin, I., Sholahuddin, M. R., Gelar, T., Engineering, I., & Bandung, P. N. (2024). 3D Large-Screen Immersive Video Mapping Installations & Interactive Light Game for Conference Seminars. Journal of Computer Engineering , Electronics and Information Technology ( COELITE ), 3(2), 89–100.

Dewangga, S., & Subianto, Mochamad Swastika, W. (2024). Implementation of Hand Gesture Recognition as Smart Home Devices Controller. INSYST, 6(2), 63–68.

Earl, C., & Neal, E. (2016). HTML5 accessible video player. Proceedings of the 13th International Web for All Conference, 1–4.

Google Developers. (2025). Hand landmarks detection guide. https://ai.google.dev/edge/mediapipe/solutions/vision/hand_landmarker

Haria, A., Subramanian, A., Asokkumar, N., Poddar, S., & Nayak, J. S. (2017). Hand Gesture Recognition for Human Computer Interaction. Procedia Computer Science, 115, 367–374.

Indriani, Harris, M., & Agoes, A. S. (2021). Applying Hand Gesture Recognition for User Guide Application Using MediaPipe. Proceedings of the 2nd International Seminar of Science and Applied Technology (ISSAT 2021), 207(Issat), 101–108.

Khairianto, D., & Firdaus, R. (2024). Penerapan Hand Gesture Recognition Sebagai Media Kontrol Presentasi Aplikasi Powerpoint. JATI (Jurnal Mahasiswa Teknik Informatika), 8(2), 1852–1860.

Kriznar, V., Leskovsek, M., & Batagelj, B. (2021). Use of Computer Vision Based Hand Tracking in Educational Environments. 2021 44th International Convention on Information, Communication and Electronic Technology, MIPRO 2021 - Proceedings, 804–809.

Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C.-L., Yong, M. G., Lee, J., Chang, W.-T., Hua, W., Georg, M., & Grundmann, M. (2019). MediaPipe: A Framework for Building Perception Pipelines. Arxiv.

Maharana, S., Singh, R. K. B., Priya, M. P., & Aswini, M. (2024). SMART Virtual Painter and Mouse System to Improve Online Education with Computer Vision and Deep Learning Features. ESIC 2024 - 4th International Conference on Emerging Systems and Intelligent Computing, Proceedings, 768–773.

Panwar, M., & Singh Mehra, P. (2011). Hand gesture recognition for human computer interaction. 2011 International Conference on Image Information Processing, 1–7.

Roy, K., & Akif, M. A. H. (2022). Real Time Hand Gesture Based User Friendly Human Computer Interaction System. 2022 International Conference on Innovations in Science, Engineering and Technology (ICISET), 260–265.

Saputra, W., & Hariyanto, S. (2024). Implementasi Hand Gesture Mediapipe Pada Game Interaktif Untuk Stimulasi Motorik Anak Down Syndrome Pada Sekolah Luar Biasa Autis Kalimantan Barat. JATI (Jurnal Mahasiswa Teknik Informatika), 8(5), 10776–10784.

DOI: https://doi.org/10.17509/edsence.v7i1.82344