Quizmaster App

With the Quizmaster app Pepper and Yuki were used (i.e., copies of the robot model Pepper). The goal of the app is to prepare the students - completely controlled by the robot - for potential exam questions using a quiz and at the same time to provide the teacher with the freedom to help students during this simulated exam situation.

 Here is a short video on the Quizmaster app, how our robot could perform it with one or a few students. (For larger groups, the quiz would be done without input and projected onto a large screen.)

Description of the App

How the questions (questions + answers + explanations) are worked through depends on whether the quiz is set manually or automatically:

• Manual: user must click on next question or solution
• Automatic: Robot performs the entire quiz. After each question or solution, the students are given a short time before the quiz continues (about 10 seconds).

In the quiz, the questions are worked through one after the other.

 The questions are presented verbally and textually by the robot, while a timer runs on the top right of the tablet. After a predetermined time (quiz setting parameter), there is an intermediate voice warning from the robot that only X seconds are left.

 When all the questions have been gone through, the solutions can be inquired. The solutions are both spoken and displayed on the tablet, and do not contain any explanations for the solution yet. Explanations come automatically (spoken and on tablet). Should there not be enough space on the tablet, one can scroll down.
A voice command or a tablet entry ("Exit Program") quits the quiz app. The robot thanks you for taking part in the quiz - "I really enjoyed our quiz together!" - and shows the interface of the app launcher again. From there, if necessary, you can shut down the robot or put it in sleep mode.

Semantic Script

Program Components

Use and Evaluation

In both courses, the Quizmaster app was used several times and the response of the students was qualitatively evaluated.

Two types of data collection were used for the evaluation: On the one hand, the use of the Quizmaster app was personally monitored by the team member responsible for the evaluation, and questionnaires were handed out, with partly closed and partly open questions to get the opinion of the students on using the robot.

 Following the execution of the use of the robot, the data was evaluated and summarized. Overall, the Quizmaster app has been used and evaluated four times.

1. Use
On November 9, 2017 the app was used for the first time, with Pepper (robotic model "Pepper") and 4 students of the course History of English.

• 4 participants
• 3m, 1w
• Teacher Degree English
• Average age 24

As this course is taught in the FLOCC format (flexible on-campus course), the students are subdivided into different learning rhythms and are grouped accordingly in the seminar room during the lecture. The 4 students were all 3-day rhythm participants (a new online session every 3 days). They sat in the front row, with Pepper in front of them. Using voice commands and tablet controls - a pen was also provided as the tablet response to the fingers is sometimes unreliable - the students themselves were able to design a quiz, for which they first had to select the framework conditions / settings, e.g. the subject area, the type of questions, the number of questions, the processing time for the questions etc. and then the quiz itself. Then Pepper read the quiz questions and displayed them parallelly on his tablet.

 The remaining time to process the question was also clearly visible on the tablet. After 60 seconds, Pepper spoke briefly and pointed out the remaining time. Imitating a mock exam, all questions were first displayed and edited one after the other, before finally revealing the solutions on the tablet.

 Apart from the slight problem with the response time of the tablet - at one point the project team member who observed the usage had to intervene and briefly take away the tablet pen from one of the students because he pushed too hard on the tablet - the preparation of the quiz proceeded quite unproblematic. The quiz itself went smoothly as well, the students discussed the answers among each other and then entered the solution together. This interaction lasted about 15 minutes, during which the students worked on 2 quizzes. Subsequently, an evaluation sheet for this robot use was filled in.

 Evaluation 1. attempt
The first attempt to use Pepper as a quizmaster with a small group of students turned out to be mixed. The students thought Pepper was hard to hear, one of the reasons given was that it was too loud overall. This could be because the robots were used with the 4 students, while the rest of the students did their usual course work a few meters away, which includes discussions with each other and group work.

 This first attempt envisaged that the students would control Pepper themselves and generate their quizzes, then carry them out and work on their own. The handling of the robot they found, however, to be not very intuitive or unintuitive. Pepper was too slow for them - the students complain about the robot's frequent crashes and hope for a "quicker implementation of the instructions" in the future. However, the main criticism is the lack of detailed explanations of the solutions. The solutions provided by Pepper were not enlightening enough for the students.

 A positive learning effect through the quiz with Pepper could not be found in the opinion of respondents. Only one out of 4 respondents said that the quiz helped to prepare for the upcoming exam. It remains to be asked if the result would have been the same with a human quiz master. 3 of the 4 students could imagine working with Pepper as Quizmaster again.

2. Use
On November 23, 2017, Pepper was tested for the second time as a quizmaster as part of the course "History of English".
This course is held in FLOCC format, divided into three rhythms. For 7 students of the 5-day rhythm Pepper was provided to ask questions relevant for the exam.

• 7 participants
• 1m, 6
• Teacher Degree English
• Average age 20

For this run, the tablet display was also projected on a screen through a projector, so that all students of the quiz group could see everything well. In addition, the handling of the robot has been simplified.
The teacher started the quiz but let the students keep interacting with Pepper. Their first voice input was not understood. Problems continued to be occur during the quiz, which at one point led to one of the students commenting on Pepper's performance disparagingly and imitating it. The repeated crashes of the quiz resulted in the students laughing.

 Evaluation 2. attempt
The first attempt with Pepper was mixed. For the second attempt not all criticisms of the first evaluation could be implemented: Admittedly, the handling is now theoretically easier, but it is not running properly. The explanations of the solutions are not yet implemented. This point of criticism then came back on the part of the students again.

 The acoustic problems - with this assignment too, the students found Pepper as bad to hear and the environment as too loud - suggest that the working groups with Pepper would better be kept in a separate room in the future, so the communication is not affected by a loud environment or the quiz should be done with the complete course.

 As a contribution to the internalization of the study material and preparation for the exam, the use of robots as a quizmaster was considered as irrelevant by the students. The students themselves see no added value there. Nevertheless, everyone without exception agreed to rehearse with Pepper as Quizmaster again. However, 5 of the 7 students linked this to one condition - in most cases these conditions affect a more reliable and fluid way of working of the robot / app.

 The lecturer also received constructive criticism: Although the handling of the quiz was made easier, it was still too time-consuming to take care of the smooth running of the robot.

Evaluation of the third and fourth use follows soon.

Obstacles and Special Features

The use of the robots has revealed the obstacles that arise beyond the app itself.

 The use of robots usually requires quite a bit of effort, as described above. A sole lecturer would not be able to cope with that. With sudden problems with the robot, the app, or the otherwise necessary technical equipment of the seminar room (for example, wireless connection), it is clearly advantageous as well to have an employee with programming skills and local robot experience. Ideally, the more stable the developed apps are, the lower the need should be.

 The realization of the desired changes is not always easy. The self-controlled running through the quiz, which would require no intervention of the teacher and thus give him/her more freedom, for example, requires a complete overhaul of the entire application code, so this change request had to be postponed for the time being.

 A feature of the robots in the project H.E.A.R.T. is that theoretically they are capable of recognizing emotions and reacting accordingly. In order to include these capabilities usefully, scenarios would still have to be devised, which will then be programmed accordingly. At the present time, the robots do not use the emotion recognition, but only express emotion themselves to some extent: the robot praises with a happy voice when a correct answer has been given, and comforts the quiz participant in the event of a wrong answer.

Conclusion

As the previous evaluations have shown, the students are generally in a positive mood, even if they were not able to determine any direct added value for themselves - apart from a not to be underestimated entertainment factor.

The added value of this app is clearly in the execution of a quiz - completely controlled by the robot. This creates freedom for the teacher for his most important task: to support students in learning.

 The surveys have also clarified how the app could be optimized.

Disclaimer Impressum