DOI : https://doi.org/10.18517/ijaseit.14.3.18115

A Design and Implementation of an Online Video Lecture System based on Facial Expression Recognition

Seung Ho Seo (1), Min Young Kim (2), Yong Kim (3)
(1) Department of smart convergence consulting, Hansung Univ., 16-gil, Seongbuk-gu, Seoul, 02876, Republic of Korea
(2) Intube Inc., 330, Neungdong-ro, Gwangjin-gu, Seoul, 04928, Republic of Korea
(3) Department of Edutech., Graduate School, Korea National Open Univ., 86 Daehak-ro, Jongno-gu, Seoul, 03087, Republic of Korea
Fulltext View | Download
How to cite (IJASEIT) :
Seo, Seung Ho, et al. “A Design and Implementation of an Online Video Lecture System Based on Facial Expression Recognition”. International Journal on Advanced Science, Engineering and Information Technology, vol. 14, no. 3, June 2024, pp. 866-72, doi:10.18517/ijaseit.14.3.18115.
Citation Format :
Real-time online education facilitates education without requiring instructors and students to be physically present in the exact location, allowing for interactions similar to face-to-face teaching. However, when many students participate simultaneously, instructors face challenges, such as spending a lot of time understanding each student's learning status. Mainly, systems utilized for real-time online education, initially developed for business meetings, have limitations when repurposed for educational uses. Therefore, this paper presents the design and implementation of a video lecture system for real-time online education based on facial expression recognition. The system is a video lecture system based on Facial Expression Recognition, implemented with a web browser method, a classroom tool platform supporting teacher-student interaction, and facial expression recognition features based on an artificial intelligence engine. Engagement metrics utilize fundamental values like facial recognition, total learning time, and video playback to measure student engagement through facial expressions. Additionally, structured numerical data such as eye-tracking, motion tracking, drowsiness tracking, speaking, effective chatting, hand-raising, polling, and screen sharing are weighted and aggregated for calculation. Weights can be adjusted according to the nature of the lecture, such as discussion-based learning or cooperative learning. The system calculates student engagement levels, categorizing them as 'Active (90 points)', 'Moderate (80 points)', and 'Insufficient (70 points)', and is designed to provide this data to instructors asynchronously. Furthermore, a focus group interview was conducted with Edtech experts to validate the implemented system, and positive responses were obtained.

M. A. Camilleri and A. C. Caterina, "Remote learning via video conferencing technologies: Implications for research and practice," In Technology in Society, 2022 Feb; 68: 101881. Published online 2022 Jan 10. doi: 10.1016/j.techsoc.2022.101881.

I. K. Noh, H. R. Jung) and Y. H. Woo, "Exploring the sustainability of implementing online education as a higher education’s New Normal in Post-Corona Era," The Journal of Learner-Centered Curriculum and Instruction, Vol. 21, no. 17, pp. 245-263, 2021.

G. Bubas and A. Čižmešija, "Measuring Video Conferencing System Success in Higher Education: Scale Development and Evaluation," International Journal of Emerging Technologies in Learning., Vol. 18 Issue 19, pp. 227-254, 2023, DOI: 10.3991/ijet.v18i19.42225.

R. R. Lexman and R. Baral, "Video-conferencing applications for educational continuity: an inquiry towards building institutional resilience," Development and Learning in Organizations: An International Journal, Vol. 37, Issue 2, pp. 18-22, 2022, DOI: 10.1108/DLO-01-2022-0030.

M. A. Camilleri and A. C. Caterina, "The Acceptance of Learning Management Systems and Video Conferencing Technologies: Lessons Learned from COVID-19," Technology, Knowledge and Learning., Vol. 27, no. 4, pp. 1311-1333, 2022, DOI: 10.1007/s10758-021-09561-y.

S. N. Roman, A. V. Roza and A. L. Liliya, "Video Conferencing Solutions for Students – Future Teachers’ Professional Socialization," Интеграция образования, Vol 26, Issue 2, pp. 229-246. 2022, DOI: 10.15507/1991-9468.107.026.202202.229-246.

A. A. Bennett, E. D. Campion, K. R. Keeler and S. K. Keener, “Videoconference Fatigue? Exploring Changes in Fatigue After Videoconference Meetings During COVID-19,” The Journal of Applied Psychology., Vol. 106, no. 3, pp. 330–344, 2021, DOI: DOI: 10.1037/apl0000906.

C. Wu, Y. Zhang, N. Li and A. Rezaeipanah, "An intelligent fuzzy-based routing algorithm for video conferencing service provisioning in software defined networking," Telecommunication Systems: Modelling, Analysis, Design and Management., pp. 1-12, 2023, 10.1007/s11235-023-01044-y.

J. Y. Zhang and O. Rebaz, "DRL-based routing algorithm with guaranteed loss, latency and bandwidth in SDN networks: Application of online video conferencing," In Journal of King Saud University - Computer and Information Sciences., Vol. 35, Issue 10, 2023, DOI: 10.1016/j.jksuci.2023.101805.

I. A. Doush, A. Al-Jarrah, N. Alajrmeh and M. Alnfiai, “Learning features and accessibility limitations of video conferencing applications: are people with visual impairment left behind,:" Universal Access in the Information Society: International Journal., vol. 22, no. 4, pp. 1353-1368, 2023, DOI:10.1007/s10209-022-00917-4.

M. F. Valstar, B. Jiang, M. Mehu, M. Pantic and K. Scherer, "The first facial expression recognition and analysis challenge," IEEE International Conference on Automatic Face & Gesture Recognition (FG), Santa Barbara, CA, USA, pp. 921-926, 2021, DOI: 10.1109/FG.2011.5771374.

C. Gilham and B. A. Moody, “Face to Face:Video Conferencing Creates Opportunities for Incarcerated Youth,” Journal of Correctional Education, Vol. 52, no. 1, pp. 29–31, 2021. http://www.jstor.org/stable/23294032

N. Kucirkova, G. Brod, and N. Gaab, “Applying the science of learning to EdTech evidence evaluations using the EdTech Evidence Evaluation Routine (EVER),” Science of Learning, Vol. 8, no. 1, pp. 1-3, 2023, DOI: 10.1038/s41539-023-00186-7.

Q. T. Thao, M. D. Tham and D. T. H. Nguyen, “The Use of Edtech Apps in English Language Learning: EFL Learners' Perspectives,” Theory & Practice in Language Studies, Vol. 13, no. 5, pp. 1115-1123, 2023, DOI: 10.17507/tpls.1305.04.

J. H. Lee and S. H. Lee, “Development and Application of Elementary Convergence Education Program Using EduTech,” Journal of Creative Information Culture, Vol. 9, no. 3, pp. 163-171, 2023.

E. G. Kim and J. H. Kim, “A Case Study on Teaching and Learning Practices Utilizing EduTech to Improve Elementary School Students’Aesthetic Experience Competence,” Korean journal of culture and arts education studies, Vol. 17, no. 6, pp. 107-132, 2022.

J. O. Park, “Study on Potential of VR/AR-based EduTech for Floral Design Education,” Journal of the Korean society of floral art & design, Vol. 47, pp. 91-106, 2022.

K. S. Oh, K. and H. W. Kim, “Edutech-based AI Convergence Platform and App Technology Trend in Physical Education,” Journal of Korean Society for the Study of Physical Education, Vol. 28, no. 2, pp. 51-62, 2023.

M. K. Lee, and M. G. Kim, “Study on the Utilization of Augmented Reality as EduTech for Students with Developmental Disabilities in the Field of Special Education,” Journal of special education: theory and practice, Vol. 23, no. 1, pp. 251-287, 2022.

Y. S. Lee, J. Y. Lim, J. G. Yang and S. Y. Sin, “A study on ways to vitalize the operation of realistic content to improve the educational abilities of students with disabilities,” Journal of the Korea Institute of the Spatial Design, Vol. 18, no. 4, pp. 233-240, 2023.

J. E. Lee, “Crisis and Opportunities in Higher Education Stimulated by Edutech,” Korea Business Review, Vol. 24, pp. 151-171, 2020.

H. R. Yoon, “A Study on Edu-tech Activation Methods for Learners in University Education,” The Journal of Humanities and Social science, Vol. 13, no. (1, pp. 3135-3148, 2022.

M. H. Jung, Y. J. Na and S. Y. Kim, “An Analytical Study on the Awareness and Demands of Professors and Students in Introducing EduTech in University - Focusing on the case of D University,” CNU Research Institute of Education, Vol. 41, no. 4, pp. 31-53, 2020.

S. H. Chin, Y. B. Kang and Y. S. Song, “Research Trends Analysis of Corporate EduTech Based on Keyword Network Analysis,” Korean Journal of Human Resource Development, Vol. 25, no. 4, pp. 27-58, 2022.

K. H. Rim, J. M. Shin and J. R. Kim, “A Study on the Establishment of Edutech-based Vocational Education and Training Model,” Journal of practical engineering education, Vol. 14, no. 2, pp. 425-437, 2022.

W. Ho, H. S. Woo, D. H. Lee and Y. Kim, “Development and Distribution of Deep Fake e-Learning Contents Videos Using Open-Source Tools,” Journal of Distribution Science, Vol. 20, no. 11, pp. 121-129, 2022, DOI : 10.15722/jds.20.11.202211.121.

W. Ho, N. K. Pahm, D. H. Lee and Y. Kim, “Proposition for LMS Integration for Share, Exchange, and Spread of Online Lectures under Covid-19 Environment,” International Journal On Informatics Visualization,” Vol. 5, no. 4, pp. 475-480, 2021, DOI: http://dx.doi.org/10.30630/joiv.5.4.744.

H. Ha, “2023 Innovation in classes based on big data and AI platform iTokTalk”, Korea Institute foe Curriculum and Evaluation, Education Square, Vol.82, pp.14-17, 2023.

J. H. Lim, S. C. Kim and Y. Kim, “A Study on Feature of Online Platform with Exploiting Blockchain for International Onshore Students,” International Journal on Advanced Science, Engineering and Information Technology, Vol. 12, no. 1, pp. 92-100, 2022, DOI: http://dx.doi.org/10.18517/ijaseit.12.1.14095.

H. S. Cho, H. and Y. Kim, “Doc2Vec based Question and Answer Search System,” International Journal on Advanced Science, Engineering and Information Technology, Vol. 11, no. 1, pp. 31-36, 2021, DOI: http://dx.doi.org/10.18517/ijaseit.11.1.14079.

Ahmed Tlili, et. al., "Automatic modeling learner’s personality using learning analytics approach in an intelligent Moodle learning platform," Interactive learning environments, Vol. 31, no. 5, pp. 2529-2543, 2023.

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors who publish with this journal agree to the following terms:

    1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
    2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
    3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).