a video analysis and retrieval agent system for family art...

11
A Video Analysis and Retrieval Agent System for Family Art Therapy Assessments 1 Seunggeung Oh, 2 Younghee Im, 3 Heesu Choi, 4 Zhenshun Seo, 5 Teaewoong Jeon, 6 Jinkyung Chang, 7 Daihee Park 1 Korea University, [email protected] *7 Korea University, [email protected] 2,3,5 Korea University, {yheeim, chlgmltn420, jeon}@korea.ac.kr 4 Korea University, [email protected] 6 Sookmyung Women’s University, [email protected] Abstract A Kinetic Family Drawing (KFD) web database system is a prototype system for managing large amounts of assessment data and facilitating an effective implementation of assessment tasks. However, such a system has an intrinsic flaw of not providing clients’ behaviors, attitudes, facial expressions, voices, and other significant information that can be otherwise observed whilst they are drawing. We propose to add functions to improve the existing system to address such a problem. Thus, a plurality of information regarding the client’s behaviors whilst drawing, in addition to the sketching, can be recorded with the aid of a web camera and a drawing tool and later retrieved. More specifically, this new system is designed to deliver two kinds of services: the client video retrieval service and the sketch video retrieval service, accompanied by a summary report of events and dynamic behavior relative to each family member, respectively. This is due to the extra facilities provided 1) from the client video data, from which events are extracted according to definitions by which some noticeable behaviors are formed into ontology-based events, and 2) from the sketch video data, from which various dynamic behaviors are obtained whilst the sketch is being drawn. Thus, this system allows therapists access more qualitative and objective information regarding their clients’ process of producing KFD. Thereby, it serves as a more supportive KFD assessment tool. Keywords: KFD, KFD Web Database System, Ontology-based Event Analysis, Event-based Video Retrieval, Multi-Modal Synchronization 1. Introduction Family art therapy uses the client’s own art works depicting their family as a tool to identify relational dynamics between the client and his or her family members and to treat psychological disorders at a family level. In particular, in situations where the client is too young or has been too shocked to be treated with verbal protocols, its usefulness is highly appreciated [1]. Since Burns and Kaufman [2-3] developed the Kinetic Family Drawing (KFD) as a projective assessment tool, it has been the most widely used in family art therapy field due to its nature of emphasizing family dynamics and children's behavioral problems. Recently, there has been growing interest in computer-aided art therapy, as part of interdisciplinary collaboration between art therapy and computer science. Hartwich and Brandecker [4] investigated the responses from clients who were directed to use a computer graphic tool, such as Adobe Photoshop. Even though their efforts were at an infant level, their significance as a new direction for the fusion of computer technology and art therapy was not trivial. Recent studies in this line [5-8] have reached a point where computer-aided art therapy is defined as a new methodology to prove the validity of drawing techniques based on projection theories and as a new realm for development of an expert system that seeks to ensure practices of automation, quantification, objectivity, and accuracy in art therapy assessment. Interestingly, what was shared in common among those studies conducted in recent years was a proposal for an expert system designed to overcome the controversy over the reliability and validity of the projective drawing techniques [9-14]. Despite controversy over the validity of drawing assessments, it is common to see many experienced family art therapists producing their own assessment results based on clues in drawings. They are often talking about the difficulties they meet when trying to access information relating to KFD that is being A Video Analysis and Retrieval Agent System for Family Art Therapy Assessments Seunggeung Oh, Younghee Im, Heesu Choi, Zhenshun Seo, Teaewoong Jeon, Jinkyung Chang, Daihee Park International Journal of Information Processing and Management (IJIPM) Volume 6, Number 1, February 2015 75

Upload: others

Post on 13-Aug-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: A Video Analysis and Retrieval Agent System for Family Art …idb.korea.ac.kr/publication/all/IJIPM312PPL.pdf · 2015-03-02 · A Video Analysis and Retrieval Agent System for Family

A Video Analysis and Retrieval Agent System for Family Art Therapy Assessments

1Seunggeung Oh, 2Younghee Im, 3Heesu Choi, 4Zhenshun Seo, 5Teaewoong Jeon,

6Jinkyung Chang, 7Daihee Park 1Korea University, [email protected]

*7 Korea University, [email protected] 2,3,5Korea University, {yheeim, chlgmltn420, jeon}@korea.ac.kr

4Korea University, [email protected] 6Sookmyung Women’s University, [email protected]

Abstract

A Kinetic Family Drawing (KFD) web database system is a prototype system for managing large amounts of assessment data and facilitating an effective implementation of assessment tasks. However, such a system has an intrinsic flaw of not providing clients’ behaviors, attitudes, facial expressions, voices, and other significant information that can be otherwise observed whilst they are drawing. We propose to add functions to improve the existing system to address such a problem. Thus, a plurality of information regarding the client’s behaviors whilst drawing, in addition to the sketching, can be recorded with the aid of a web camera and a drawing tool and later retrieved. More specifically, this new system is designed to deliver two kinds of services: the client video retrieval service and the sketch video retrieval service, accompanied by a summary report of events and dynamic behavior relative to each family member, respectively. This is due to the extra facilities provided 1) from the client video data, from which events are extracted according to definitions by which some noticeable behaviors are formed into ontology-based events, and 2) from the sketch video data, from which various dynamic behaviors are obtained whilst the sketch is being drawn. Thus, this system allows therapists access more qualitative and objective information regarding their clients’ process of producing KFD. Thereby, it serves as a more supportive KFD assessment tool.

Keywords: KFD, KFD Web Database System, Ontology-based Event Analysis,

Event-based Video Retrieval, Multi-Modal Synchronization

1. Introduction

Family art therapy uses the client’s own art works depicting their family as a tool to identify relational dynamics between the client and his or her family members and to treat psychological disorders at a family level. In particular, in situations where the client is too young or has been too shocked to be treated with verbal protocols, its usefulness is highly appreciated [1]. Since Burns and Kaufman [2-3] developed the Kinetic Family Drawing (KFD) as a projective assessment tool, it has been the most widely used in family art therapy field due to its nature of emphasizing family dynamics and children's behavioral problems.

Recently, there has been growing interest in computer-aided art therapy, as part of interdisciplinary collaboration between art therapy and computer science. Hartwich and Brandecker [4] investigated the responses from clients who were directed to use a computer graphic tool, such as Adobe Photoshop. Even though their efforts were at an infant level, their significance as a new direction for the fusion of computer technology and art therapy was not trivial. Recent studies in this line [5-8] have reached a point where computer-aided art therapy is defined as a new methodology to prove the validity of drawing techniques based on projection theories and as a new realm for development of an expert system that seeks to ensure practices of automation, quantification, objectivity, and accuracy in art therapy assessment. Interestingly, what was shared in common among those studies conducted in recent years was a proposal for an expert system designed to overcome the controversy over the reliability and validity of the projective drawing techniques [9-14].

Despite controversy over the validity of drawing assessments, it is common to see many experienced family art therapists producing their own assessment results based on clues in drawings. They are often talking about the difficulties they meet when trying to access information relating to KFD that is being

A Video Analysis and Retrieval Agent System for Family Art Therapy Assessments Seunggeung Oh, Younghee Im, Heesu Choi, Zhenshun Seo, Teaewoong Jeon, Jinkyung Chang, Daihee Park

International Journal of Information Processing and Management (IJIPM) Volume 6, Number 1, February 2015

75

Page 2: A Video Analysis and Retrieval Agent System for Family Art …idb.korea.ac.kr/publication/all/IJIPM312PPL.pdf · 2015-03-02 · A Video Analysis and Retrieval Agent System for Family

kept in paper documents scattered in many places. In response to this issue, Im et al. [15] proposed to build up a KFD web database system to ensure that large amounts of KFD data in paper documents are effectively maintained, so that working therapists gain timely support in their KFD assessments. There are many advantages once such a KFD web database system has been implemented. In addition to the basic database functions provided, the system enables the recording of the process of drawings, content-based KFD image search via object-oriented image retrieval, the provision of assessment sheets available online and other KFD information search services. Additionally, a web-based database allows those therapists to meet their clients online at clients’ convenient times. However, there is a limitation that a full picture of the process of drawing is not provided. Using the replay function in the existing system does not suffice, because critical peripheral information, such as facial expressions, behaviors, or murmuring voices, that accompany the action of drawing, is not available. Hence, the existing system needs to be complemented by additional functions, such as recording and replaying the client’s behaviors and voice using a web camera and other devices.

In the KFD assessment, therapists should pay close attention to their clients’ behaviors and attitudes. These are important information for the examiner seeking to identify therapeutic needs [16]. Instances of showing hesitance and lack of attention or using an eraser too frequently, whilst producing a drawing of a particular family member, cannot be negligible in KFD interpretation. Nonetheless, there is no diagnostic model for specifically handling such behaviors and attitudes. There are a wealth of quantitative systems developed to score measurable styles, actions, and characteristics of KFD [9], [17-22] all of which were used to evaluate only a completed drawing by referring to the elements identified by Burns and Kaufman [2-3], not including the client’s behaviors and attitudes shown during the action of drawing except for some case studies dealing with qualitative analysis of drawings, where behaviors and emotional changes of the client whilst drawing were mentioned briefly [16]. Despite the emphasis on the rule of “observe a child’s behavior while drawing” in administering psychological evaluation of drawings [23], the practices involved have not yet been systematically embedded in actual settings, and therefore there is little academic literature to support their use. However, it is important to be mindful of the argument of Watson [24], a U.S. psychologist called a father of behaviorism in psychology, that psychology should be an objective experimental branch of science whose methods are direct observation and measurement of behavior. Following his insights, it is possible to understand that considering both the client’s drawing and his/her behaviors observed whilst drawing will lead to interpretation that is more accurate.

This paper has the goal of enhancing the completeness of the KFD web database to provide family art therapists with not only information regarding the completed drawings but also additional important information related to the process of producing such drawings. Two types of KFD video data are provided by the system proposed in this study. First, one for the client engaged in the process of producing a drawing and one for the client’s drawing. The client scene video clip (referred to as client video hereinafter) is a video record of the upper body of the client in focus from the web camera’s viewpoint during the art work (Figure. 3(a)). Second, the sketch scene video clip (referred to as sketch video hereinafter) is about the sketch drawing, which is done on the drawing board from the start until its culmination (Figure. 3(b)). In the proposed KFD video analysis and retrieval system, ontology-based event analysis and event-based video image retrieval are provided using indices of key behavior events obtained from client video data. From sketch video clips, information such as the sequence in which each object was drawn, the time of drawing, the amount of time taken to draw each family member, and the number of times the eraser was used by the client is extracted and summarized for each family member for therapists, who can also have access to the retrieval function based on the family member. This system also supports the multi-modal synchronization replay function through which therapists are allowed to easily infer semantic linkage between the two types of video data. To illustrate, when sketch scenes are played back, the client scenes synchronized to the time of drawing are simultaneously played back, thereby enabling the observation of the changes in behaviors, facial expressions, and voices. Synchronizing the playback backwards, starting from the client scenes, where some unusual way of behaving was observed from the client, the synchronization function can be used to see what object (family member) was being drawn at that time. Thus, this proposed system gives practical aid to therapists in their activities towards more accurate assessments.

The remainder of this paper is organized as follows. In the next two sections, the structure of the proposed digital video analysis and retrieval system is debriefed and then the process of defining and extracting events is accounted. Then, how to analyze the client and sketch video data is described,

A Video Analysis and Retrieval Agent System for Family Art Therapy Assessments Seunggeung Oh, Younghee Im, Heesu Choi, Zhenshun Seo, Teaewoong Jeon, Jinkyung Chang, Daihee Park

76

Page 3: A Video Analysis and Retrieval Agent System for Family Art …idb.korea.ac.kr/publication/all/IJIPM312PPL.pdf · 2015-03-02 · A Video Analysis and Retrieval Agent System for Family

followed by the introduction of the search and playback functions supported by the multi-modal synchronization replay function. Then, the results from measuring recognition accuracy of extracted events are presented to verify the reliability of analyzed information from the client video. In the final section, our discussion and conclusions are presented. 2. The structure of KFD video analysis and retrieval agent system

Figure 1 shows the structure of the proposed KFD video analysis and retrieval agent system. Our

agent system was designed as a subsystem of the KFD web database system [15], whose client module (Figure 2) creates client video clips (Figure 3(a)) from shots of the upper body of the client taken from a web camera and captures sketch video clips (Figure 3(b)) out of recordings inside the drawing board. Our KFD video analysis and retrieval system provide therapists with summarized information regarding both client and sketch video. It additionally offers a multimodal synchronization replay function.

Figure 1. Structure of KFD video analysis and retrieval agent system

Our proposed system was designed to perform event recognition based on ontology. Ontology is an

explicit specification of conceptualization. When the knowledge of a domain is represented in a declarative formalism, the set of objects that can be represented is called the universe of discourse. This set of objects, and the describable relationships among them, are reflected in the representational vocabulary with which a knowledge-based program represents knowledge. Thus, we can describe the ontology of a program by defining a set of representational terms. In such an ontology, definitions associate the names of entities in the universe of discourse (e.g., classes, relations, functions, or other objects) with human-readable text describing what the names are meant to denote, and formal axioms that constrain the interpretation and well-formed use of these terms [25]. Ontology for human activities describes entities, environment, interaction between them and the sequence of events that is semantically identified with an activity. It specifies how an activity can be constructed using lower-level primitive events by identifying the role played by each entity in the sequence of events [26]. Considering our web camera’s characteristics of covering only the subject’s upper body, we attempted to structure the client’s

A Video Analysis and Retrieval Agent System for Family Art Therapy Assessments Seunggeung Oh, Younghee Im, Heesu Choi, Zhenshun Seo, Teaewoong Jeon, Jinkyung Chang, Daihee Park

77

Page 4: A Video Analysis and Retrieval Agent System for Family Art …idb.korea.ac.kr/publication/all/IJIPM312PPL.pdf · 2015-03-02 · A Video Analysis and Retrieval Agent System for Family

behavioral features into a set of events by computing facial coordinates and defining ontology-based event rules.

Figure 2. Drawing board and video recording module for clients

Figure 3. Client video snapshot (a) and sketch video snapshots (b)

Facial region detection It is necessary to extract facial regions from the original image of the client’s upper body to define

and recognize events from the client video. Our system used the Haar-like feature method [27-28] for facial region extraction. The Haar-like feature method, unlike other facial region extraction algorithms, is a model-based approach that does not use pixel values directly. This allows less learning input and more diverse output results in training algorithms, resulting in an easier classification procedure. The facial region detection process is as follows. First, a rectangle-shaped window of the target size, as shown in Figure 3(a), is slid over the input image and the feature values are calculated. Second, the obtained feature values are run through the learned classifier. Once the facial region is correctly detected, the central coordinates are calculated using the edges of the rectangle and then the calculated coordinate values are stored in a log file per frame.

A Video Analysis and Retrieval Agent System for Family Art Therapy Assessments Seunggeung Oh, Younghee Im, Heesu Choi, Zhenshun Seo, Teaewoong Jeon, Jinkyung Chang, Daihee Park

78

Page 5: A Video Analysis and Retrieval Agent System for Family Art …idb.korea.ac.kr/publication/all/IJIPM312PPL.pdf · 2015-03-02 · A Video Analysis and Retrieval Agent System for Family

Event generation and production rule Major events in a client’s video represent significant behavioral anomalies that can be recognized

during the client’s drawing work. Table 1 defined those events. We selected 10 client’s behaviors that we thought frequently occur and that are relatively simple taking into account the current level of video pattern analysis technology, and formed them into events. Then, each of such predefined events was defined again so that it can be mapped to a vector of low level features created based on changes in the traced facial coordinates (see Table 2: here, 푋 and 푌 respectively represent the central X- and Y- coordinates in the facial region in the 푛th frame). That is, in more detail, the central coordinates in the facial region for each frame are stored in the log file, as metadata of the client video, with event recognition being conducted in a unit of three frames. Then, if the value corresponding to the difference between the sum of the central coordinates in the facial region for the first and second frames and those for the third frame exceeds the threshold value defined in each event, the event is recognized as such. Each event that may appear is summed individually to be provided as a summary report to therapists.

Table 1. Major events defined for client video

Event Definition Enter The client’s face has been detected for the first time since the start of the video recording. Leave The client’s face is absent (is not detected) after the events of Enter or Re-enter.

Re-enter After the Leave event, the Enter event has been detected again. Stay The client maintains a particular posture or shows almost no body movement. Stand The client is standing. Curl The client is in a curling motion or keeps the head lowered.

Left peak The client keeps the head leant to the left. Right peak The client keeps the head leant to the right. Approach The client keeps the body close to the screen.

Recede The client keeps the body away from the screen.

Table 2. Production rules for event recognition Event Production rule

Enter While a plurality of log information for the nth frame is being serially input, the values of 푋 , 푌 (the central coordinates of the facial region) are greater than 0 for the first time.

Leave The values of 푋 and 푌 are 0. Re-enter Events take place in the order of Enter → Leave → Enter.

Stay While a plurality of log information is being input in a unit of three frames, the values of 푋 and 푌 remain below the threshold.

Stand While a plurality of log information is being input in a unit of three frames, the value of 푌 exceeds the threshold value.

Curl While a plurality of log information is being input in a unit of three frames, the value of 푌 falls below the threshold.

Left peak While a plurality of log information is being input in a unit of three frames, the value of 푋 falls below the threshold.

Right peak While a plurality of log information is being input in a unit of three frames, the value of 푋 exceeds the threshold value.

Approach While a plurality of log information is being input in a unit of three frames, the size of the facial region exceeds the threshold value.

Recede While a plurality of log information is being input in a unit of three frames, the size of the facial region falls below the threshold value.

A Video Analysis and Retrieval Agent System for Family Art Therapy Assessments Seunggeung Oh, Younghee Im, Heesu Choi, Zhenshun Seo, Teaewoong Jeon, Jinkyung Chang, Daihee Park

79

Page 6: A Video Analysis and Retrieval Agent System for Family Art …idb.korea.ac.kr/publication/all/IJIPM312PPL.pdf · 2015-03-02 · A Video Analysis and Retrieval Agent System for Family

3. KFD video analysis

A video analysis module for family art therapists, as exemplified in Figure 4, is provided. In an event analysis module, pressing the video analysis button starts the module to perform the summarization and analysis of information in the client and sketch video data. Once the video analysis has been finished, key information extractable from sketch video and the summary of the events retrievable from client video are provided in a table form, as shown in Figure 4.

Figure 4. KFD video analysis module for therapists

Client video analysis In the case of client video, the coordinates of the facial region are extracted and sorted according to

time-frame and stored into a log file in real time. The stored video data and the log file are then transmitted to an ontology-based event analysis system, where each event is recognized as pre-defined production rules and coordinates in the log file are appropriately mapped. The result is data that can be summarized and sorted by object to provide rudimentary information available for therapists, as shown in Figure 5.

Figure 5 indicates that the client showed more motion when drawing the father than other family member objects. Since this suggests a possible presence of an unusual relationship between the client and his or her father, therapists need to check further the relationship in a later interview session. This advantage cannot be gained only from using completed drawings. This is an important point where the necessity for analysis of the client’s dynamic behaviors that accompany drawing becomes manifest in a KFD assessment process.

A Video Analysis and Retrieval Agent System for Family Art Therapy Assessments Seunggeung Oh, Younghee Im, Heesu Choi, Zhenshun Seo, Teaewoong Jeon, Jinkyung Chang, Daihee Park

80

Page 7: A Video Analysis and Retrieval Agent System for Family Art …idb.korea.ac.kr/publication/all/IJIPM312PPL.pdf · 2015-03-02 · A Video Analysis and Retrieval Agent System for Family

Figure 5. Sample of client video analysis summary (video analysis for the client in Figure 4.)

Sketch video analysis Data retrieved for KFD sketch video analysis were chosen from quantitative data with respect to the

client’s dynamic behaviors that can be automatically extracted from meta-data stored in the database, reflecting the needs of family art therapists. The following data are provided for sketch video analysis by the proposed system in this study.

- Sketch sequence: the sequence in which each object was drawn - Sketch time (sec): the amount of time taken to complete an object.

- Relative sketch time (%): a percentage of the amount of time taken to complete each object in the

entire duration of the drawing process. - Idle time (sec): The time interval between the completion of previous object and the initiation of

current object. Long idle time can be interpreted to mean that the client is experiencing hesitancy and other perceived difficulties in drawing.

- Number of erasures: the number of times that the eraser function was used whilst an object is being

drawn. The data retrieved from sketch video analysis are displayed in a summarized table form as below,

together with client video analysis data. Figure 6, a table for a KFD sample, corresponds to Figure 4. There is a plurality of dynamic behavioral information in this table that cannot be gained by using completed drawings. For example, looking at the data in Figure 6, the provided indicators, including the last drawing of the client in the sketch sequence, 47 seconds of sketch time for client accounting for only 7% of the total drawing duration, and about 47 seconds of idle time, lend themselves to a highly likely interpretation that the client was experiencing hesitancy and other difficulties in drawing himself/herself. It might be predictable that the client has relatively low self-esteem. Thus, this type of detailed information is very useful for art therapists in performing their jobs and is not otherwise accessible when dealing with only completed drawings.

Figure 6. Sample of sketch video analysis (from sketch video analysis in Fig. 4)

Video retrieval and replay Pressing the video retrieval button shown in Figure 4 starts the video retrieval module (see Figure 7

and Figure 8). The video retrieval module offers, depending on the mode of query requested through the user interface, the following three video retrieval (including sound data retrieval) functions: 1) event-based client video retrieval, 2) family member object-based sketch video retrieval, and 3) multi-modal synchronized replay for retrieved video clips.

In the case of event-based client video retrieval, the video clips are replayed from the time a specific event that was queried occurred. Figure 7 is a sample display of event-based scene retrieval where the ‘approach’ event has been retrieved. In addition to the retrieval function, the synchronized playback

A Video Analysis and Retrieval Agent System for Family Art Therapy Assessments Seunggeung Oh, Younghee Im, Heesu Choi, Zhenshun Seo, Teaewoong Jeon, Jinkyung Chang, Daihee Park

81

Page 8: A Video Analysis and Retrieval Agent System for Family Art …idb.korea.ac.kr/publication/all/IJIPM312PPL.pdf · 2015-03-02 · A Video Analysis and Retrieval Agent System for Family

function is offered so that the therapist can see what object was being drawn at the time the corresponding event occurred.

Figure 7. Sample displays of event-based scene retrieval and multi-modal synchronized replay

(where the ‘approach’ event has been retrieved)

For the family member object-based sketch video retrieval, if a query is submitted, the sketch video clips are replayed from the point where the corresponding object has been indexed. Figure 8 shows the results obtained when the keyword ‘father’ was input. Therapists are now allowed to have the desired displays of sketch video starting from the time the client started to draw the father. With the synchronized replay function now available, as with the client video retrieval, the client’s attitudes and facial expressions whilst drawing the corresponding object can be observed when necessary (see Figure 8).

Figure 8. Sample displays of object-based scene retrieval and multi-modal synchronized replay

(where the ‘father’ object has been retrieved).

4. Discussion and Conclusion In this paper, a new video analysis and retrieval system was designed and developed as a proposal to

enhance the accuracy and reliability of the KFD web database system. It provides therapists key with information about the client’s psychological status via a summary report on client’s behaviors and attitudes towards family members during drawing that cannot be gained from a completed drawing alone. In addition, the multi-modal synchronized replay function was provided to support therapists in inferring

A Video Analysis and Retrieval Agent System for Family Art Therapy Assessments Seunggeung Oh, Younghee Im, Heesu Choi, Zhenshun Seo, Teaewoong Jeon, Jinkyung Chang, Daihee Park

82

Page 9: A Video Analysis and Retrieval Agent System for Family Art …idb.korea.ac.kr/publication/all/IJIPM312PPL.pdf · 2015-03-02 · A Video Analysis and Retrieval Agent System for Family

semantic linkage between client’s behaviors and family members more efficiently. In the KFD assessment process, the interpretation of client’s behaviors is as important as that of completed drawings. Needless to say, close attention should be paid to the semantic association between client behaviors and each object in drawings. Now with this development, we expect that family art therapists will be able to obtain more semantically meaningful outcomes in their assessment.

In family art therapeutic scenes, therapists should pay attention to the client, including the client’s behaviors and attitudes that accompany each drawing, the client’s state of mind, such as whether he/she is well aware of the therapist’s directions or is concentrating sufficiently. In some cases, depending on the client’s mood or the severity of the issues faced by the client, there are cases when the client does not want a particular family member or even the therapist to be with him/her whilst drawing. In these cases, the drawing is done by the client alone or together with one of his/her family members, instead of with the therapist. This means that since the therapist has no chance to observe otherwise noticeable client’s behaviors, which are likely to play an important part in interpreting his/her drawing, the only artifact available for assessment is the completed drawing. However, even in such cases, if a solution similar to that proposed here is available, therapists will secure a tool by which more accurate and objective analysis is possible, because they are now able to find, check, and compare the completed drawing and the information about the client’s behaviors, attitudes, and voices that is retrieved from the web database. That is, a potentially important contribution that this study will make is that having such an objective analysis tool can be seen as strength in the field of art therapy, considering the currently prevailing view that the assessment techniques used in the art therapy realm lack validity and reliability, even though their practical usefulness must be admitted. In fact, we expect that a richer collection of video data analyses provided through the KFD web database system will lay solid a foundation for researchers to more deeply delve into the semantic association between drawings and children’s behaviors with the use of statistical modeling and data mining analysis.

The methodology adopted in this study was intended for application in KFD assessment, but its application to art therapy in general is straightforward. In this paper, the number of client’s behaviors and attitudes was limited to only ten events, considering the current level of video pattern analysis technology. However, it should be noted that through a statistical process, more client’s behaviors and attitudes can be defined and further abstracted into higher level concepts. For example, a collection of events consisting mainly of such events as stay, approach, and recede may be incorporated into an event group designated as ‘highly concentrated’, and conversely, events with a large amount of motion, such as leave, stand, curl, left peak, and right peak may be incorporated into an event group represented by ‘distracted’. However, since the grouping and naming of such high-level concepts are dependent on the assessor’s subjectivity, leaving such a job to practicing therapists would be more reasonable. In addition, practicing therapists might suggest that other events that are harder to recognize and involve delicate and momentary changes in facial expressions could be added. ‘Smiling’, ‘frowning’, ‘yawning’, and ‘blinking’ could represent examples of such events. It is impossible for therapists to capture all behaviors, attitudes, and facial expressions of their client, whatever efforts they make. Moreover, psychological projection by the client might be more naturally and directly made on his or her behaviors, attitudes, or facial expressions than on his/her hand drawings on paper. If so, further study on the usefulness of the multi-modality techniques dealt with in this paper is mandatory. In this respect, there are some interesting observation points including the multi-modal synchronization technique proposed here, how client’s behaviors and attitudes change whilst a particular object is being drawn, and what family member certain behaviors and attitudes are attributable. As Watson [24] argued, psychology is an objective experimental branch of science whose methods should be direct ‘observation’ and ‘measurement’ of behavior. Along the same line, given the importance of the use of the client’s behaviors whilst drawing, as well as the drawings themselves, in the assessment process, taking an approach of observation and evaluation covering both aspects, in particular, multi-modality techniques incorporating the two aspects seems to be more appropriate. Therefore, a new mission for art therapy researchers is to develop and validate more objective and systematic multi-modality techniques.

5. Acknowledgement

This research was supported by the BK (Brain Korea) 21 Plus Program.

A Video Analysis and Retrieval Agent System for Family Art Therapy Assessments Seunggeung Oh, Younghee Im, Heesu Choi, Zhenshun Seo, Teaewoong Jeon, Jinkyung Chang, Daihee Park

83

Page 10: A Video Analysis and Retrieval Agent System for Family Art …idb.korea.ac.kr/publication/all/IJIPM312PPL.pdf · 2015-03-02 · A Video Analysis and Retrieval Agent System for Family

6. References

[1] Kim. H, The theory and practice of family therapy, Hakjisa, Korea, 2008 [2] Burns. R. C, Kafuman. S. H, Kinetic family drawings (K-F-D): An introduction to understanding

children through kinetic drawings, NY:Burnener/Mazel, USA, 1970 [3] Burns. R. C, Kafuman. S. H, Action, styles and symbols in kinetic family drawings (KFD): An

interpretive manual. NY:Burner/Mazel, USA, 1972 [4] Hartwich. P, Brandecker. R, “Computer-based art therapy with inpatients: Acute and chronic

schizophrenics and borderline cases”, The Arts in Psychotherapy, vol. 24, no. 4, pp.367-373, 1997. [5] Kim. S. I, Ghil. J. H, Choi. E. Y, Kwon. O. S, Kong. M, “A computer System using a structured

mandala to differentiate and identify psychological disorders”, The Arts in Psychotherapy, vol. 41, no. 2, pp.181-186, 2014.

[6] Kim. S. I, Kim, J. H, H. E. J, “A computer system for face stimulus assessment with application to the analysis of dementia”, The Arts in Psychotherapy, vol. 40, no. 2, pp.245-249, 2013.

[7] Kim. S. I, Kang. H. S, Chung. S, Hong, E. J, “A statistical approach to comparing the effectiveness of several art therapy tools in estimating the level of a psychological state”, The Arts in Psychotherapy, vol. 39, no. 5, pp.397-403, 2012.

[8] Kim. S. I, Betts. D. J, Kim. H, Kang. H, “Statistical model to estimate level of psychological disorder based on a computer rating system: An application to dementia using structured mandala drawings”, vol. 36, no. 4, pp.214-221, 2009.

[9] McPhee. J. P, Wegner. K. W, “Kinetic family drawing styles and emotionally disturbed childhood behavior”, Journal of Personality Assessment, vol. 40, no. 5, pp.487-491, 1976.

[10] Monahan. M, “Situation influences on children’s kinetic family drawings”, Dissertation Abstracts International”, vol. 46, pp.4444, 1986.

[11] Layton. M, “Specific features in the kinetic family drawings of children”, A dissertation from Temple University, 1983.

[12] Sobel. H, Sobel. W, “Discriminating adolescent male delinquents through the use of kinetic family drawings”, Journal of Personality Assessment, vol. 40, no. 1, pp.91-94, 1976.

[13] Rhine. P, “Adjustment indicators in kinetic family drawings by children: A validation study”, Unpublished doctoral dissertation of Purdue University, 1977.

[14] Cumming. J. A, “An evaluation of an objective scoring system for KFD’s”, Dissertation Abstract International , 1980.

[15] Im. Y. H, Oh. S. K, Chung. M. J, Yu. J. H, Lee. H. S, Chang. J. K, Park. D. H, “A KFD web database system with an object-based image retrieval for family art therapy assessments”, The Arts in Psychotherapy, vol. 37, no. 3, pp.163-171, 2010.

[16] Choi. W. S, Jeong. H. H, Psychology diagnosis, JungMoon, Korea, 2004 [17] O’Brien. R. P, Patton. W. F, “Development of an objective scoring method for the kinetic family

drawing”, Journal of Personality Assessment, vol. 38, no. 2, pp.156-164, 2974. [18] Mysers. D. V, “Toward an objective evaluation procedure of the kinetic family drawings (KFD)”,

Journal of Personality Assessment, vol. 42, no. 4, pp.358-365, 1978. [19] Reynolds. C. R, “A quick-scoring guide to the interpretation of children’s kinetic family drawings

(KFD)”, Psychology in the Schools, vol. 15, no. 4, pp.489-492, 1978. [20] Mostkoff. D. L, Lazarus. P. J, “The kinetic family drawing: The reliability of an objective scoring

system”, Psychology in the Schools, vol. 20, no. 1, pp.16-20, 1983. [21] Kasuya. M, Sawaki. Y, Ohno. Y, Ueda. M, “Psychological study of cleft palate children with or

without cleft lip by Kinetic Family Drawing”, Journal of Cranio-Maxill of Acial Surgery, vol. 28, no. 6, pp.373-379, 2000.

[22] Wegman. P, Lusebrink. V. B, “Kinetic family drawing scoring method for cross-cultural studies”, The Arts in Psychotherapy, vol. 27, no. 3, pp.179-190, 2000.

[23] Skybo. T, Ryan-Wenger. N, Su. Y, “Human figure drawings as a measure of children’s emotional status: Critical review for practice”, Journal of Pediatric Nursing, vol. 22, no. 1, pp.15-28, 2007.

[24] Waston. J. B, “Psychology as the behaviorist views it”, Psychological Review, vol. 20, no. 2, pp.158-177, 1913.

[25] Gurber. T. R, “A translation approach to portable ontology specifications”, Knowledge Acquisition, vol. 5, no. 2, pp.199-220, 1993.

A Video Analysis and Retrieval Agent System for Family Art Therapy Assessments Seunggeung Oh, Younghee Im, Heesu Choi, Zhenshun Seo, Teaewoong Jeon, Jinkyung Chang, Daihee Park

84

Page 11: A Video Analysis and Retrieval Agent System for Family Art …idb.korea.ac.kr/publication/all/IJIPM312PPL.pdf · 2015-03-02 · A Video Analysis and Retrieval Agent System for Family

[26] Akdemir. U, Turaga. P, Chellappa. R, “An ontology based approach for activity recognition from video”, Proceeding of the 16th ACM international conference on Multimedia, pp.709-712, 2008.

[27] Toan. T. D, Khiem. N. D, Thai. H. L, Bac. H. L, “Boosted of Haar-like feature and local binary pattern based face detection”, Computing and Communication Technologies, RVF '09. International Conference, pp.1-8, 2009.

[28] Lin. L, Li. X, Zhou. W, “Face detection algorithm based on haar and local binary pattern features”, JCIT, vol. 7, no. 19, pp.602-610, 2012.

A Video Analysis and Retrieval Agent System for Family Art Therapy Assessments Seunggeung Oh, Younghee Im, Heesu Choi, Zhenshun Seo, Teaewoong Jeon, Jinkyung Chang, Daihee Park

85