DOI : https://doi.org/10.18517/ijaseit.13.3.18457

Design and Implementation of a Programming Automatic Assessment System in Jupyter Notebook

HakNeung Go (1), Seong-Won Kim (2), Youngjun Lee (3)
(1) Korea National University of Education, Cheongju, 28173, Republic of Korea
(2) Silla University, Sasang-Gu, Busan, 46958, Republic of Korea
(3) Korea National University of Education, Cheongju, 28173, Republic of Korea
Fulltext View | Download
How to cite (IJASEIT) :
Go, HakNeung, et al. “Design and Implementation of a Programming Automatic Assessment System in Jupyter Notebook”. International Journal on Advanced Science, Engineering and Information Technology, vol. 13, no. 3, June 2023, pp. 1080-6, doi:10.18517/ijaseit.13.3.18457.
Citation Format :
Learning programming is challenging. So, computer educators have developed various tools to help students. In this paper, we have developed a tool that combines the advantages of a Programming Automatic Assessment (PAA) system and Jupyter Notebook (JN) to support learning programming. The design direction of this system is free to use, easy to set up, and supports interactive computing. The Programming Automatic Assessment in Jupyter Notebook (PAAinJN) is available free of charge using the assessment module released on Git and the personal JN. The initialization is completed by executing in a code cell with two lines of code that downloads and executes the assessment module. In an interactive computing environment, presenting problems, writing code to be evaluated, and evaluating code can be executed in the code cells, and the problems and the results of the assessment are presented in the code cell outputs. The performance was verified by the examples presented in a high school informatics textbook using the programming automatic assessment system as teaching learning material. In addition, we propose a way to develop teaching-learning materials using PAAinJN in consideration of teachers and students and a way of distributing and collecting teaching-learning materials using the free Learning Management System. PAAinJN is expected to help students learn programming by eliminating assessment and feedback delays through PAA while learning to program in an interactive computing environment.

Lee, D., Hwang, J. Y., Lee, Y., and Kim, S. W, “Informatics and Artificial Intelligence (AI) Education in Korea: Situation Analysis Using the Darmstadt Model”. JOIV: International Journal on Informatics Visualization., vol. 6(2), pp. 427-444, 2022, DOI: 10.30630/joiv.6.2.1000.

Shin, S., and Bae, Y, “A Study on the Hierarchical Instructional System Design of Software Education by School System,” Journal of The Korean Association of Information Education, vol. 19(4), pp. 533-544, 2015, DOI: 10.14352/jkaie.2015.19.4.533.

Wing, J. M, “Computational thinking,” Communications of the ACM, vol. 49(3), pp. 33-35, 2006, DOI:10.1145/1118178.1118215.

Lee, Y. J., Paik, S. H., Shin, J. H., You, H. C., Jeong, I. G., Ahn, S. J., and Jeon, S. G, “Research for introducing computational thinking into primary and secondary education,” Korea Foundation for the Advancement of Science and Creativity, BD14060010, 2014.

C. Cachero, P. Barra, S. Melií¡ and O. López, “Impact of Programming Exposure on the Development of Computational Thinking Capabilities: An Empirical Study,” IEEE, vol. 8, pp. 72316-72325, 2020, DOI: 10.1109/ACCESS.2020.2987254.

Kim, J. M, “Problem-solving skills based on computing thinking,” Korea Information Processing Society Review, vol. 24(2), pp. 13-21, 2017.

Durak, H. Y., Yilmaz, F. G. K., and Yilmaz, R, “Computational thinking, programming self-efficacy, problem solving and experiences in the programming process conducted with robotic activities,” Contemporary Educational Technology, vol. 10(2), pp. 173-197, 2019, DOI:10.30935/cet.554493.

Malmi, L., Utting, I., and Ko, A, “Tools and Environments,” In S. Fincher & A. Robins (Eds.), The Cambridge Handbook of Computing Education Research. Cambridge, United Kingdom, pp. 173-197, 2019, doi:10.1017/9781108654555.022.

Ramí­rez-Echeverry, J. J., Restrepo-Calle, F., and Gonzí¡lez, F. A., “A case study in technologyenhanced learning in an introductory computer programming course,” Global Journal of Engineering Education, vol. 24(1), pp. 65-71, 2022.

Ullah, Z., Lajis, A., Jamjoom, M., Altalhi, A., Al”Ghamdi, A., and Saleem, F, “The effect of automatic assessment on novice programming: Strengths and limitations of existing systems,” Computer Applications in Engineering Education, vol. 26(6), pp. 2328-2341, 2018, DOI: 10.1002/cae.21974.

Kurnia, A., Lim, A., & Cheang, B, “Online judge,” Computers & Education, vol 36(4), pp. 299-315, 2001, DOI: 10.1016/S0360-1315(01)00018-5.

Rahmani, A., and Min, J. L, “Automatic Grading System to Supporting Blended Learning in Basic Programming Practice-an Experience Report,” In International Conference on Online and Blended Learning 2019 (ICOBL 2019), pp. 96-101, Atlantis Press, DOI: 10.2991/assehr.k.200521.020.

Cardoso, A., Leití£o, J. and Teixeira, C, “The Challenges of the Digital Transformation in Education,” Proceedings of the 21st International Conference on Interactive Collaborative Learning (ICL2018), vol. 2, pp. 227-236, DOI: 10.1007/978-3-030-11935-5_22.

Wang, J., Kuo, T. Y., Li, L., and Zeller, A, “Assessing and restoring reproducibility of Jupyter notebooks,” In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering, pp. 138-149, 2020, DOI: 10.1145/3324884.3416585.

Ochkov, V. F., Stevens, A., and Tikhonov, A. I, "Jupyter Notebook, JupyterLab-Integrated Environment for STEM Education," In 2022 VI International Conference on Information Technologies in Engineering Education (Inforino), pp, 1-5, 2022, DOI: 10.1109/Inforino53888.2022.9782924.

Cardoso, A., Leití£o, J., and Teixeira, C, "Using the Jupyter Notebook as a Tool to Support the Teaching and Learning Processes in Engineering Courses," In: Auer, M., Tsiatsos, T. (eds) The Challenges of the Digital Transformation in Education. ICL 2018. Advances in Intelligent Systems and Computing, Springer, Cham, vol 917, 2019 DOI: 10.1007/978-3-030-11935-5_22.

Gonzí¡lez-Carrillo, C. D., Restrepo-Calle, F., Ramí­rez-Echeverry, J. J., and Gonzí¡lez, F. A, “Automatic grading tool for jupyter notebooks in artificial intelligence courses,” Sustainability, vol. 13(21), pp. 12050, 2021, DOI: 10.3390/su132112050.

Zamfir, F. S. and Pricop, E, “On the design of an interactive automatic Python programming skills assessment system,” 2022 14th International Conference on Electronics, Computers and Artificial Intelligence (ECAI), pp. 1-5, 2022, DOI: 10.1109/ecai54874.2022.9847414.

Chang, W. Y. and Kim, S. S, “A review on trends of programming(algorithm) automated assessment system and it`s application”, The Journal of Korean Association of Computer Education, vol. 20(1), pp. 13-26, 2017.

Han, S., Liu, X., and Woo G., "Evaluation Methods and Classification of Online Judge Systems Based on Education Level and Purpose," KIISE Transactions on Computing Practices, vol.28(10), vol.487-492, 2022, DOI: 10.5626/KTCP.2022.28.10.487.

Ministry of Education, “Informatics Science Guidelines”, 2015.

Chang, W. Y., and Kim, S. S, “Development and application of algorithm judging system: analysis of effects on programming learning.” The Journal of Korean Association of Computer Education, vol. 17(4), pp. 45-57, 2014.

Jeong, S. S, “The effects of programming education using an automatic programming assessment system on learning flow of general high school students” M.S. thesis, Dept. Computer Education, Graduate School of Korea National University of Education., Cheongju, Korea, 2019.

Johnson, J. W, “Benefits and pitfalls of jupyter notebooks in the classroom,” In Proceedings of the 21st Annual Conference on Information Technology Education, pp. 32-37, 2020, DOI: 10.1145/3368308.3415397.

Zhao, P., and Xia, J, “Use JupyterHub to Enhance the Teaching and Learning Efficiency of Programming Related Courses,” 23rd International Conference on ISO & TQM, Florida, USA, pp.1-6, 2019.

Klever, N., “Jupyter Notebook, JupyterHub and Nbgrader. Becoming Greener—Digitalization in My Work”, The Publication Series of LAB University of Applied Sciences, pp. 37-43, 2020.

Blank, D. S., Bourgin, D., Brown, A., Bussonnier, M., Frederic, J., G ranger, B., and Willing, C, “nbgrader: A tool for creating and grading assignments in the Jupyter Notebook”, The Journal of Open Source Education, vol. 2(11), 2019, DOI: 10.21105/jose.00032.

Barba, L. A., Barker, L. J., Blank, D. S., Brown, J., Downey, A. B., George, T., ... and Zingale, M., “Teaching and learning with Jupyter,” Recuperado: https://jupyter4edu. github. io/jupyter-edu-book, 2019.

Go, H., and Lee, Y, “Exploring how to present the problem of Automatic Assessment system in Jupyter Notebook,” Proceedings of the Korean Society of Computer Information Conference, vol. 31(1), pp. 221-222, 2022.

Gyeonggi Science High School Informatics Teachers, Problems. [Online]. available: http://koistudy.net/?mid=viewProblems (accessed: October 1st, 2022).

Han. G. W, Go. G. S, Kim. E. K, Lim. S. K, Jeon. H. S, Jeong. S. S, Jeong. W.Y, Jeong. J. K, Jeong. J. W, and Jo. H. J, “[2015 Revised-- High School] Informatics Guide,” Seoul, Korea: SamYangMedia, 2019.

Gupta, A., and Pathania, P., “To study the impact of Google Classroom as a platform of learning and collaboration at the teacher education level,” Education and Information Technologies, vol. 26(1), pp. 843-857, 2021.

Authors who publish with this journal agree to the following terms:

    1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
    2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
    3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).