Composing the Atmosphere of a Virtual Classroom witha Group of Student Agents

Masato Fukuda, Hung-Hsuan Huang*, Naoki Ohta, and Kazuhiro Kuwabara

College of Information Science & Engineering, Ritsumeikan University, Japan*contact: hhhuang@acm.org

1 Introduction

In Japan, the training of teachers mainly relies on in-classroom lectures in universities.It is compensated with the practice for a relatively short period, say only two to threeweeks in real schools. The teacher-training programs in Japan therefore lacks the prac-tice of teaching skills and the admission of classes. The result is, many young teachersleft their jobs in the first year due to frustration and other mental issues. In order torelieve this situation, we are developing a Wizard-of-OZ (WOZ) based simulation plat-form of a school environment with computer graphics (CG) animated virtual students.The trainees can interact with the virtual students in this immersive and realistic vir-tual classroom and practice their teaching and administration skills.The virtual studentsare operated by an operator (the wizard) from remote with a dedicated interface. Inaddition to the training purpose, the system is considered to be able to be used in theexamination of teacher recruitment as well. In that case, the operator is supposed to bethe examination investigator.

On the other hand, there are usually dozens of students in a high school class inJapan. In a WOZ system that is operated by one or few people, how to simultaneouslycontrol the relatively large number of students can be a challenge. A research issueemerges here, how to effectively control a group of virtual students to represent a real-istic class in a real-time WOZ system. In this paper, we propose an atmosphere modelof a group of virtual students based on empirical results as a solution to this issue.

2 Related Works

Virtual environments have also been shown to be an effective tool for various trainingtasks. Jones et al. [2] developed a job interview simulation platform, which supportssocial training and coaching in the context of job interviews. Williamon et al. [5] de-signed and tested the efficacy of simulated performance environments as a new trainingfacility for musician trainees. Kenny et al. [3] designed the training systems of mentaltherapeutic with virtual simulated patients.

On the other hand, few studies have focused on the training system of teachers invirtual environment. TechLive [1] is one of the examples. This application is also aVR simulated classroom with operator(s). In this system, one operator can only controlone of the virtual students by selecting pre-defined animation sequences or driving itwith a motion capture device in real-time. There are no severe issues in the US where

the number of students in a class is small. However, due to more limited resources,the number of students in one class is much larger in Japan. How to efficiently andrealistically control dozens of students at the same time is not a trivial problem. Ourwork addresses this by proposing an atmosphere model that is described in the followingsections.

3 Atmosphere Model of the Virtual Classroom WOZ System

This WOZ system is composed of two front ends, one is a simulated classroom for thetrainee, the other one is the interface for the system operator / investigator. The expectedusage of the system is: the trainee front end is projected on a large screen (say 100inches or even larger) while the trainee stands in front of the screen and practice his/herteaching skill. The trainee’s teaching is captured by a Web cam and is displayed at theoperator’s interface (see the virtual class from the rear side) in real-time. The operatorthen control the virtual students while observing the trainee’s teaching performance.

We divided the control of virtual students into two modes, individual mode andwhole-class mode. When the operator choose an arbitrary student, that student will shiftto the individual mode and then the operator can fully control that student manually. Bydefault, all virtual students are in the whole-class mode and are controlled by an atmo-sphere model. The atmosphere model serves as a template and all the virtual studentstogether create the atmosphere of their group. This atmosphere is supposed to be thefeedback sent from the operator to the trainee. The trainee can then adopt his/her teach-ing style in responding to the atmosphere. The class atmosphere is model is defined tobe driven by three elements, concentration, arousal, tension which are inspired fromthe pleasure-arousal-dominance (PAD) model [4] of an individual’s emotional state.The CAT space is defined as the follows:

Concentration: how much the students are concentrating on the lecture. How well thetrainee is explaining important topics of the lecture.

Arousal: the activity level of the students. How well the trainee is keeping the interestof the students.

Tension: the level of the tension of the students. How well the trainee is maintainingthe order of the class.

The idea is: the values of these parameters have the effect in the possibility of the vir-tual students to express corresponding behaviors. For example, in a low-concentration,low-arousal, and low-tension situation, many of the virtual students may show sleepyanimation, while in a high-concentration, high-arousal, and medium-tension situation,the virtual students may concentrate in the lecture, take memos, nodding frequentlyand so on. However, another research issue emerges here, how to determine the actualbehaviors of the virtual students in the various CAT state? The perception and inter-pretation from student behaviors to the atmosphere of the whole class is subjective andheavily depends on the experience in education of the observer. Therefore, we con-ducted a subject experiment to gather the interpretation from unspecified people andhopefully in large number.

!"#

!$#

!%#

Fig. 1. Interface dedicated for experiment participants to compose the atmosphere in CAT states.(1) the area indicating the CAT state that the participant should compose (2) the virtual studentsshowing the behaviors selected by the participant (3) the dashboard where the participant adjuststhe number of students performing each behavior with scroll bars

4 Data Collection Experiment and Conclusions

A dedicated interface of the virtual classroom was developed the experiment (Fig. 1).Although the value of the CAT dimensions can be arbitrary, we assume that if we canget the parameters of the maximum and the minimum of each axis, the arbitrary in-termediate values can be interpolated from them. We recruited 12 participants for theexperiment, three of them are professors and nine of them are students of our university.Among these students, three of them are receiving teacher-training course provided byour university.

The participants were asked to compose the atmosphere of the class according tothe specified CAT states one by one. They adjust the number of virtual students who isperforming one of 15 implemented behaviors (Table 1) with scroll bars. The number ofstudents (xk,i) assigned with a specific behavior k by all participants (i from 1 to n) ineach state is then used to compute the probability (Pk) of a virtual student to performbehavior k in that state by the following equation. Here N denotes the total number ofvirtual students (28 in current implementation).

Pk =1N{1n

n∑i=1

xk,i} (1)

The experiment results were shown in Table 2. From these results, we can find sometendencies of the probability distribution. Behavior A to F are more frequently used inpositive states and the remaining ones are more frequently used in negative states. Be-havior A, B, and E are dominating ones while the others are used as accents. The resultsare still in a very preliminary state due to the small number of experiment participants.We plan to increase the number by recruiting the participants from crowdsourcing ser-vices. After gathering reliable enough results, we will conduct further analyses andintegrate the model to the WOZ prototype system

Table 1. Student behaviors used in the experiment with their corresponding code names

Code Behavior Code BehaviorA look at the teacher H make chin rest on handsB write something on a note I swing upper body slightlyC read text book J look aroundD nod K dozeE nod strongly L sleep with face downward to the deskF raise right hand M out of itG cross hands behind head N whisper

Table 2. Probability of each student behavior in each characteristic CAT state. h denotes a highlevel while m denotes medium level, and l denotes a low level. Column A-N denotes the behaviorcode listed in Table 1 while “–” denotes default behavior (sit down and look forward)

A B C D E F G H I J K L M N –ChAhTh 0.14 0.20 0.04 0.05 0.40 0.11 0.00 0.00 0.00 0.01 0.00 0.00 0.00 0.00 0.05ClAlTl 0.00 0.00 0.01 0.03 0.04 0.02 0.09 0.12 0.04 0.09 0.10 0.28 0.09 0.08 0.01ChAhTl 0.32 0.17 0.03 0.07 0.14 0.13 0.02 0.02 0.00 0.03 0.05 0.00 0.01 0.02 0.00ChAlTh 0.20 0.08 0.04 0.06 0.06 0.10 0.03 0.03 0.10 0.04 0.04 0.02 0.05 0.08 0.07ClAhTh 0.49 0.10 0.03 0.05 0.14 0.00 0.04 0.00 0.01 0.00 0.04 0.05 0.00 0.01 0.04ChAhTm 0.43 0.10 0.06 0.06 0.19 0.05 0.00 0.04 0.02 0.00 0.01 0.00 0.00 0.00 0.04ChAmTh 0.27 0.09 0.03 0.08 0.16 0.08 0.00 0.00 0.11 0.04 0.02 0.00 0.04 0.05 0.03CmAhTh 0.44 0.10 0.03 0.13 0.04 0.05 0.00 0.00 0.01 0.02 0.00 0.00 0.00 0.01 0.17ClAlTm 0.07 0.01 0.04 0.02 0.00 0.00 0.07 0.15 0.14 0.14 0.17 0.03 0.07 0.04 0.05ClAmTl 0.06 0.08 0.06 0.00 0.00 0.00 0.04 0.12 0.02 0.05 0.10 0.15 0.14 0.09 0.09CmAlTl 0.01 0.02 0.03 0.02 0.00 0.02 0.03 0.13 0.12 0.13 0.05 0.07 0.15 0.21 0.01ChAlTl 0.04 0.06 0.04 0.00 0.03 0.11 0.00 0.01 0.16 0.19 0.03 0.00 0.09 0.24 0.00ClAhTl 0.31 0.12 0.22 0.11 0.01 0.02 0.00 0.07 0.02 0.00 0.02 0.02 0.02 0.00 0.06ClAlTh 0.23 0.05 0.07 0.10 0.02 0.04 0.00 0.07 0.05 0.04 0.13 0.04 0.04 0.05 0.07

References

1. R. Barmaki and C. E. Hughes. Providing real-time feedback for student teachers in a virtualrehearsal environment. In 17th International Conference on Multimodal Interaction (ICMI2015), pages 531–537, November 2015.

2. H. Jones, N. Sabouret, I. Damian, T. Baur, E. Andrre, K. Porayska-Pomsta, and P. Rizzo.Interpreting social cues to generate credible affective reactions of virtual job interviewers. InarXiv:1402.5039v2, 2014.

3. P. Kenny, T. D. Parsons, J. Gratch, A. Leuski, and A. A. Rizzo. Virtual patients for clinicaltherapist skills training. In Proceedings of the 7th International Conference on IntelligentVirtual Agents (IVA’07), pages 192–210, Paris, France, 2007.

4. A. Mehrabian. Pleasure-arousal-dominance: A general framework for describing and measur-ing individual differences in temperament. Current Psychology, 14(4):261–292, 1996.

5. A. Williamon, L. Aufegger, and H. Eiholzer. Simulating and stimulating performance: intro-ducing distributed simulation to enhance learning and performance. Frontiers in Psychology,5(25):1–9, February 2014.