Using Cipherbot: An Exploratory Analysis of Student Interaction with an LLM-Based Educational Chatbot

Joni Salminen, Soon Gyo Jung, Johanne Medina, Kholoud Aldous, Jinan Azem, Waleed Akhtar, Bernard J. Jansen

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Citations (Scopus)

Abstract

Cipherbot, an educational chatbot using large language models to answer student questions concerning learning materials uploaded by the educator, was pilot tested in a classroom setting. Forty-four students used Cipherbot for seven weeks, sending 8077 messages. The average number of messages sent per student was 184 (SD = 80), with an average length of 98 characters (SD = 80). The engagement followed a non-normal distribution, with few power users, implying that most students are still hesitant to adopt tools like Cipherbot. Cipherbot was able to answer 82.5% of the student questions, demonstrating a scalable ability to address students' learning queries, with some room for improvement.

Original languageEnglish
Title of host publicationL@S 2024 - Proceedings of the 11th ACM Conference on Learning @ Scale
PublisherAssociation for Computing Machinery, Inc
Pages279-283
Number of pages5
ISBN (Electronic)9798400706332
DOIs
Publication statusPublished - 9 Jul 2024
Event11th ACM Conference on Learning @ Scale, L@S 2024 - Atlanta, United States
Duration: 18 Jul 202420 Jul 2024

Publication series

NameL@S 2024 - Proceedings of the 11th ACM Conference on Learning @ Scale

Conference

Conference11th ACM Conference on Learning @ Scale, L@S 2024
Country/TerritoryUnited States
CityAtlanta
Period18/07/2420/07/24

Keywords

  • LLMs
  • cipherbot
  • generative AI
  • student interaction

Fingerprint

Dive into the research topics of 'Using Cipherbot: An Exploratory Analysis of Student Interaction with an LLM-Based Educational Chatbot'. Together they form a unique fingerprint.

Cite this