The Big ASC

RA TrainingFriday 17 – Saturday 18 June 2011

UWS

Programme for Friday 17/06/11 09:00 Welcome – Paperwork for casual payment (to be returned at Morning

tea)09:30 Introduction (Denis Burnham / Dominique Estival)

Overview of the Big ASC projectResponsibilities of the recording RA

10:00 The Black Box (Steven Fazio / Lei Jing): Unpacking and setting up 11:00 Morning tea 11:30 The data collection protocol (Dominique Estival / Lei Jing)

Setting up and starting the SSCP / Video and audio calibration13:00 Lunch (UWS) 13:30 Running a recording session (Dominique Estival)14:00 Practice Session 115:30 Afternoon tea16:00 Interview and Story re-retelling (John Ingram): Practice Session 217:30 Pack up the Black Boxes18:30 Dinner at The Mill

Programme for Saturday 18/06/11 09:30 Unpack and set up the Black Boxes10:00 The Map Task (Dominique / Denis / Maria): Practice Session 3 11:00 Morning tea 11:30 Recruitment, keeping records, etc. (Dominique Estival / Denis Burnham)13:00 Lunch (Bankstown Golf Club) 14:00 Data saving and data upload (Lei Jing / Steve Cassidy)15:00 Questions?

More practice 16:00 Pack up the Black Boxes17:00 Departure

Introduction

The Big ASC: Introduction

Linkage Infrastructure & Equipment Fund (LIEF) 13 partners - UWS lead institution

Who are we?• Denis Burnham – MARCS – Lead CI• Dominique Estival – MARCS – Project officer• Steven Fazio – MARCS – Technical officer• Lei Jing – MARCS – Software engineer• Maria Christou – Expert Assistant

The Big ASC University of Western SydneyMacquarie UniversityThe Australian National UniversityThe Flinders University of South AustraliaThe University of MelbourneThe University of New South WalesThe University of QueenslandThe University of SydneyThe University of Western AustraliaUniversity of CanberraUniversity of TasmaniaAustralian Speech Science and Technology AssociationMax Planck Institute for Psycholinguistics

RA and IT-support from partner institutions

MQ: Greg Flannery UNE: Mark ConroyUNSW: Ronny Kurniawan Sydney U: Sarah de LauneyUQ: Nahyun Kwon, John Ingram Melbourne U: Joshua Clothier, Cara Penry-Williams Flinders: Marissa Milne, Hywel Stoakes, Tom Anderson UWA: Damien Pontifex, Serajul Haque U Tas: Rosie Maunder UC: Sharifa Alghowinem ANU: Katharine Gosling, Geoff CollinCSU/Bathurst: Rebekah Lockart

+ Catherine Watson from Auckland U

The Big ASC: aims and goals

1. Establish the infrastructure for the collection of large quantities of AV speech data.

2. Provide standardised recording equipment and protocols for data collection.

3. Collect AV data from many locations across multiple sessions per speaker. 4. Provide standardised annotation for (at least a subset of) the data.5. Provide access to the annotated data and the meta-data via a centralised

storage facility.

• AusTalk: not only a large Australian speech corpus, but also the only Australian corpus of AV speech in the public domain. designed to cater to various clients with a variety of interests

(Phonetics and Linguistics; Psycholinguistics; Language Technology; Speech Engineering; Speech pathology; Forensic Speech Science)

The Big ASC: Infrastructure

• Corpus Collection: Equipment and Protocol• Standard Speech Science Infrastructure Black Box (SSSIBB)

• Standard Speech Collection Protocol (SSCP) - Session protocol- Software Implementation

• Corpus Distribution: Data Storage and Server

The Big ASC: Data CollectionSTATE UNI N REGIONAL N OTHER N TOTAL

NSW UWS   TOWNSVILLE 48     48

  MQ   ARMIDALE 48     48

  UNSW 48     EMOTION 36 84

  USYD 48     DISORDERED 16 64

QLD UQ 120       120

VIC MELB 120 GEELONG 48     168

SA FLINDERS 96 AS/DARWIN 48 AUSAB 48 192

WA UWA 96         96

TAS UTAS 48         48

ACT UC 36 BATHURST 48     84

  ANU 48         48

TOTAL   660   240   100 1000

Responsibilities of the RA

from the Position Description for the job:

• Manage the recruitment of participants• Set up the recording equipment• Conduct the recording sessions• Upload the data from the local computers to a

central server.

Concretely ....• Manage the recruitment of participants:

Contact the participants who have registered through the website for your area.Advertise in your local uni, church, RSL, etc... if more are needed.

• Set up the recording equipment:Responsible for the BB and running it.

• Conduct the recording sessions:Set up a recording schedule, contact the participants, welcome them, ensure the

questionnaire is completed and the consent form signed.Be patient and manage successful interactions with the public.

• Upload the data from the local computers to a central server:Responsible for saving the data at the end of each session, checking it and transferring

it to the HDD at the end of the day.

The Black Box

Infrastructure for data collection

Standard Speech Science Infrastructure Black Box (SSSIBB)Hardware:• 1 computer, 1 digital audio acquisition device, 3 desktop microphones,

2 head-worn microphones, 2 stereo cameras • packed together in a reinforced box which folds out into a table with

integrated shelving

• standardised: same equipment used at all locations, in the same configuration and with the same setup and spacing.

• portable: data collection can be conducted in any location and can be extended to new locations as needed.

No Item Make Model Number Company Serial No Vendor

T01-1 16U MIXER RACK WORKSTATION cases.com.au cases.com.au

T01-2 1ru sliding shelf cases.com.au cases.com.au

T01-3 2RU Rackmount Drawer cases.com.au cases.com.au

T01-4 Phillips Head Screwdriver Bunnings Bunnings

T01-5 Flat Head Screwdriver Bunnings Bunnings

T01-6 Head worn microphone 1 AudioTechnica AT892c AudioTechnica Turra Music

T01-7 xlr adapter 1 AT8539 Phantom Power / XLR adapter AudioTechnica Turra Music

T01-8 Head worn microphone 2 AudioTechnica AT892c AudioTechnica Turra Music

T01-9 xlr adapter 2 AT8539 Phantom Power / XLR adapter AudioTechnica Turra Music

T01-10 boundary mics Behringer c-2 mic array Mic 1 Behringer Turra Music

T01-11 boundary mics Behringer c-2 mic array Mic 2 Behringer Turra Music

T01-12 boundary mics stereo adapter Behringer c-2 mic array stand Behringer Turra Music

T01-13 xlr cable 1 xlr budget cable Turra Music

T01-14 xlr cable 2 xlr budget cable Turra Music

T01-15 Far-Field microphone Shure MX391/O Shure Turra Music

T01-16 far-Field microphone xlr adapter shure shure Turra Music

T01-17 Stereo Camera 1 BumbleBee2 Pointgrey Pointgrey

T01-18 Stereo Camera 2 BumbleBee2 Pointgrey Pointgrey

T01-19 firewire cable camera 1 pointgrey custom cable Pointgrey Pointgrey

T01-20 firewire cable camera 2 pointgrey custom cable Pointgrey Pointgrey

T01-21 dual bus firwire pci express card pointgrey pci express card Pointgrey Pointgrey

T01-22 GPIO Sync Cable Custom made GPIO to audio cable Flinders Uni Flinders Uni

T01-23 Capture Computer core i7 rackmount 4ru Impact Custom Impact

T01-24 Keyboard MS MS Impact

T01-25 Mouse MS MS Impact

T01-26 RA monitor Dell E170S 17 inch Flat Panel Monitor Dell ITP

T01-27 Subject monitor Dell E170S 17 inch Flat Panel Monitor Dell ITP

T01-28 Audio DAQ M-Audio FastTrack Ultra8R M-Audio Turra Music

T01-29 power supply audio DAQ M-Audio Turra Music

T01-30 USB cable audio DAQ D type USD cable M-Audio Turra Music

T01-31 serial cable audio DAQ M-Audio Turra Music

T01-32 External hard drive Samsung STORY Station 2TB Samsung ITP

T01-33 power supply external HDD Samsung ITP

T01-34 USB cable external HDD Samsung ITP

T01-35 Light Meter Lux & Fc Light Meter Jaycar Jaycar

T01-36 RA Head Phones KOSS UR-20 headphones KOSS Turra Music

T01-37 RA Head Phones 3.5MM STEREO SOCKET to 6.5MM STEREO PLUG adaptor KOSS Turra Music

T01-38 Monitor arm / stand subject Atdec Visidec Focus MICRO LCD Single Arm, VF-M Atdec Pacific Notebooks

T01-39 Monitor arm / stand tool Allen Key tool to fix bolts to stand Atdec Pacific Notebooks

T01-40 Monitor arm / stand base plate black base plate Atdec Pacific Notebooks

T01-41 Monitor arm / stand bolt 1 Long allen key bolt to mount the monitor stand arm to the table Atdec Pacific Notebooks

T01-42 Monitor arm / stand bolt 2 Short allen key bolt to mount custom mounting plate Atdec Pacific Notebooks

T01-43 Monitor arm / stand vesa bolts bag of VESA bolts to attach monitor Atdec Pacific Notebooks

T01-44 Tripod mount for stero camera 1 jaycar QC3333 + custom mounting plate Jaycar Jaycar

T01-45 Tripod mount for stero camera 1B Manfrotto 700RC2 tripod head Manfrotto Dirt Cheap Cameras

T01-46 quick shoe plate Manfrotto Manfrotto Dirt Cheap Cameras

T01-47 Tripod mount for stero camera 2 Manfrotto 700RC2 tripod head Manfrotto Dirt Cheap Cameras

T01-48 quick shoe plate Manfrotto Manfrotto Dirt Cheap Cameras

T01-49 umbrella light Main Subject Umbrella Soft Ebay Hypop Ebay Hypop

T01-50 Umbrella Refector Ebay Hypop Ebay Hypop

T01-51 Tripod Ebay Hypop Ebay Hypop

T01-52 Dual lamp adapter Ebay Hypop Ebay Hypop

T01-53 65W lamps x4 Ebay Hypop Ebay Hypop

T01-54

T01-55

T01-56

T01-57 umbrella light maptask only Umbrella Soft Ebay Hypop Ebay Hypop

T01-58 Umbrella Refector Ebay Hypop Ebay Hypop

T01-59 Tripod Ebay Hypop Ebay Hypop

T01-60 Dual lamp adapter Ebay Hypop Ebay Hypop

T01-61 65W lamps x4 Ebay Hypop Ebay Hypop

T01-62

T01-63

T01-64

T01-65 Background / screen Main Subject blue screen + base plate

T01-66 carry bag

T01-67 mounting pole 1

T01-68 mounting pole2

T01-69 Background / maptask only blue screen + base plate

T01-70   carry bag

T01-71 mounting pole 1

T01-72 mounting pole2

T01-73 Partition for maptask 400x500 3mm black acrylic board Lingel Lingel - Milperra

T01-74 Main Subject Chair

T01-75 Maptask Only Chair

List in your pack

The Standard Protocol

Standard Corpus Components

• Isolated Words• Digits• Read Sentences• Interview• Story: reading/retelling• Map Task• Yes/No• (Emotions)

Standard ProtocolSession 1 Session 2 Session 3

Task Time Task Time Task Time

Calibration(+ 3D face)

10

Calibration 3 Calibration 3

Opening Yes/No 3 Opening Yes/No 2 Opening Yes/No 2

Words 10 Words 10 Words 10

Read Narrative 5  Interview  15 Map Task (First run) 20

Re-told Narrative 10 Switch Sp.A and Sp.B 5

Read Digits 5 Read Digits 5 Map Task (Second run)   20

Read Sentences 8 Conversation   5

Words 10

Closing Yes/No 2 Closing Yes/No 2 Closing Yes/No 2

  44   45   77

Running a session

Session Set-up1. General setup – to be done every day before recording sessions - Measure location of light sources and distance to camera(s) (manual

measurement) : 80cm between light and camera- Check Bumblebee camera output is being recorded - Check microphone output is being recorded - Audio and video recording of the recording environment (ambient

noise and lighting profile) without a speaker present (60s) 2. Session setup – to be done before each session- Ensure the equipment is running: computer, camera, audio.- Ensure the 4 microphones are still plugged in.- Set chair for speaker and adjust background screen behind speaker

CalibrationCamera setup

Ensure face of speaker is within the red square. If necessary, adjust seating position.Check light level. If necessary adjust brightness.

3D procedureSpeaker is on chair, facing the camera. Ask the speaker to turn 90° to the right. Turn speaker to every 15° starting from -90° (left profile) to +90° ( right profile), take 5s at each position (total = 135s). It is important that the speaker turns their head such that the face points in the required direction.

Facial calibration While the speaker repeats these sequences, check on the M-Audio display that the levels are within the limits.

Setting audio levels Adjust the levels for each microphone so they are set within the levels shown.

Session 1

Yes/No Questions• Ask questions as casually as possible to elicit natural responses. • The questions can be modified as necessary. The only requirement is that they be “Yes/No”

questions, and that you can categorise the answer as positive or negative.

1) So let’s see, for the record, you are [Animal ID], right?2) And is this your first recording session for the project?3) Is there anything unclear with the instructions so far?4) Would you like me to explain the process again?5) Do you have any questions about what you need to do?6) Can you see the screen?7) Are you comfortable with the position of the chair and the microphone?8) Shall we start then?

• After the response to each question, click on “Answer Yes” or “Answer No”, depending on the response. Then click on “Start” for the next question.

• If the yes/no answer to a question requires clarification, give your explanations before clicking “Start” for the next question.

Isolated Words

• A set of 323 words is presented one at a time on the screen.

• Different orders for S1, S2, S3.• Ask the speaker to read each one silently before

reading them aloud.• Some are not real words, but made-up to elicit

particular sound combinations.For these, we have provided a guide for pronunciation on the screen under the word.

hun

sounds like fun

Words

• If the speaker makes a mistake: click “Pause”

then when you are both ready to record again click “Start”.

Numbers

• A set of 12 strings of 4 digits is presented one string at a time on the screen.

1 2 3 4

We do not want:“one thousand two hundred and thirty-four”

We want: “one two three four”

0837: zero eight three seven

942o: nine four two oh

Story reading

Arthur the Rat

• 7 successive screens• Ask the speaker to read each silently before

reading aloud.

Story re-telling: suggestions• Well, what did you think of the story of ‘Arthur the rat’?• If they answer ‘a bit weird’, ask: In what way did it seem weird or odd?• If they ‘liked the story’, ask: What did you like about it?• Stories often convey a ‘moral’ or a lesson. What was the moral of this story, Arthur

the Rat? And did you agree with it?• Most kids like being told stories. Why do you think that is so?• What sorts of stories used to appeal to you as a kid? • Stories are often used to educate the young about their heritage or their identity.

How did this apply to you, or to your age group when you were growing up? What stories were you brought up with?

• Can you recall a particular story that made a strong impression on you when you were young?

• Where do you mainly get your stories from nowadays? From books, from movies, from yarns told by friends? From the TV?

• What story caught your attention recently?

Session 2

Sentences

• A set of 59 sentences is presented one at a time on the screen.

• Different orders for S1 and S2 (this is taken care of by the program).

• Ask the speaker to read each one silently through before reading them aloud.

• If the speaker makes so many mistakes they get frustrated, you can skip to the next component (only as a last resort)

Interview: Topics

1. A dangerous or exciting episode in my life: Something I’m never likely to forget…

2. A topic that I feel quite strongly about: Something that really gets on my goat… something that really ought to be changed …

3. My best holiday ever was when …4. A most embarrassing incident …5. Changes in the district: How this place has changed over the years;

what it used to be like around here compared with the way things are now…

6. What we used to get up to as kids: Ways that we used to entertain ourselves, but are perhaps not so available to kids today …

7. One of the most interesting characters that I have ever met …

Interview: Guidelines

No “PARSNIP” !PoliticsAlcoholReligionSexNarcotics-ismsPersonal

Session 3

Map Task

• Two people, one the instruction giver (IG) and one the instruction follower (IF).

• Each has a map of the same environment, but there are small differences between the maps.

• The IG's map has a route marked to some target location from some start point.

• The IG's task is to communicate this route to the IF so that he or she can mark the route on their copy of the map.

Map Task: Information Giver

Map Task: Information Follower

Map Task: Instructions

MapTask set up for Speaker B

Map Task: logisticsOpen Session 3.1:

Speaker 1 at Main TableCalibration – Opening – Words Introduce Speaker 2 Sit Speaker 2 at Second 2 – Calibration for Speaker 2 at Second TableCalibration for Speaker 1 at Main Table (lower camera)Map Task 1 (20mns):Speaker 1 = IG / Speaker 2 = IF Speaker 2 goes over the route they took and holds up the map to the cameraClose and finalise Session 3.1

Switch Speaker 1 and Speaker 2 (position and role)Open Session 3.2:

Calibration for Speaker 1 at Second Table – Calibration for Speaker 2 at Main TableMap Task 2 (20mns)Speaker 2 = IG / Speaker 1 = IFSpeaker 1 goes over the route they took and holds up the map to the camera

Conversation between Speaker 1 and Speaker 2 (5mns)Escort Speaker 1 out of the room Calibration for Speaker 2 at Main Table (upper camera)Words – Closing for Speaker 2

Close and finalise Session 3.2

Conversation after Map Task

Ask the participants to talk to each other about:• whether they enjoyed the task, • how difficult it was, • what part of the instructions at the beginning

helped them understand the task, • what they liked or didn't like about the other

participant's directions, • what features of the landmarks helped or

hindered, • what they found useful.

Conversation after Map Task

Ask the participants to talk to each other about:• whether they enjoyed the task, • how difficult it was, • what part of the instructions at the beginning

helped them understand the task, • what they liked or didn't like about the other

participant's directions, • what features of the landmarks helped or

hindered, • what they found useful.

Data saving and upload

Data Handling Procedure (1)https://austalk.edu.au/trac/wiki/DataHandlingProcedureData is recorded on the Black Box for each session and stored on the local hard drive (D:) of the

machine. However, the local drive can't store a lot of data (how many sessions?) and we need to move data to backup storage and forward it to Macquarie to be uploaded to the server. This page details the procedure to be followed at the end of each day's recording.

The initial working assumption is that no compression of video is to be done on the black box machine. This means that a single session is up to 120G of raw data. If we later introduce compression, these procedures may be able to be altered.

During RecordingAll data will be stored on the D: drive in the D:\recordings\ folder, each speaker will be stored in a

folder named for the session and the speaker. The folder name will be something like Spkr1_1241_Session1 where 1_1241 is the speaker numerical id (corresponding to the colour and animal) and Session1 is the first session. There will be folders within this for each component that was recorded.

After Each SessionAfter each session...

Data Handling Procedure (2) https://austalk.edu.au/trac/wiki/DataHandlingProcedureEnd of DayAt the end of each day you should have all of the data from that day's recordings on the hard

drive of the black box. This could amount to around 500G if you ran 4 separate recording sessions. We need to move this to backup storage and produce something that can be sent to Macquarie for uploading to the server.

Write to two USB drives, one will be kept locally, one sent to MQ Verify that the data was copied correctly to both drives Label the drives with the dates, speaker and session IDs they contain Delete original data from the internal hard drive ready for the next day's recording Copying 500G to a single USB drive should take around 4hrs (est. around 40M/s transfer rate).

Copying to two different disks shouldn't take twice as long as the limiting factor is the USB transfer speed not the read speed.

End of Week (or every few days)Send the full hard drives to MQ for uploading to server. Drives should be properly packed (bubble

wrap, plastic bag) to avoid data loss due to shocks/water damage. Local USB drives should be stored in a secure location away from possible water damage etc. If

you have access to local storage (eg. your institution has a large scale data storage facility) then a copy could be uploaded to that as an additional backup.

Data Handling Procedure (3)https://austalk.edu.au/trac/wiki/DataHandlingProcedureEnd of Week (or every few days)Send the full hard drives to MQ for uploading to server. Drives should be properly packed (bubble

wrap, plastic bag) to avoid data loss due to shocks/water damage. Local USB drives should be stored in a secure location away from possible water damage etc. If

you have access to local storage (eg. your institution has a large scale data storage facility) then a copy could be uploaded to that as an additional backup.

At MacquarieOnce drives are received at MQ the data will be uploaded to the server, verified, and the original

USB drives will be cleared and sent back to the collection site. All data on the server will be backed up to a secondary storage facility (tape). The uploaded data

will be visible via the web server. Once the drive is returned to the site, you can confirm that the data is available via the web and

clear the data from the local USB drive copy. Alternately, this could be kept as a local backup if sufficient USB drives are available.

Recruitment and keeping records

Recruitment

Registered

16/06/2011 RequiredACT 24 132NSW 423 196NT 3 96QLD 70 168SA 49 96TAS 124 48VIC 148 168WA 51 96 892 1000

Expressions of interest: https://austalk.edu.au/express-interest.html

Demographic spread

Age/SES/Gender Prof-M NProf-M Prof-F Nprof-F

< 30 10 10 10 10

         

31-49 10 10 10 10

         

> 50 10 10 10 10

         

Total: 120        

See doc on wiki:https://austalk.edu.au/trac/attachment/wiki/CollectionSchedule/Demographics.xlsx

Consent forms

• Each institution must have their own(see model from UWS)

• Use your own Ethics approval number

• Need to be signed before Questionnaire

Questionnaire

• Website: https://austalk.edu.au/forms/participant-questionnaire.html

Need to be completed before recording.

• Speaker ID: colour + Australian animal– e.g. “Blue Northern funnelweb spider”– Same ID for all 3 sessions

Payment to participants

• Each speaker receives $25 for each of the first and second sessions, and $40 for the third.

• Payment is organised by each institution. 2 options:1) Cash2) Bank transfer

• Model forms from UWS – to be adapted by each institution:

1) Cash advance for the RA 2) Claim for payment Cash payment to participant ATO form

Bonus

• We really need each speaker to come back and do the 3 sessions.

• Incentive for YOU: $25 for each full set of 3 sessions

To be paid in a travel fund by your institution Complete the spreadsheet

Documentation

• Wiki: https://austalk.edu.au/trac/wiki

http://austalk.edu.au