j. peter rosenfeld, john meixner, michael winograd, elena labkovsky, alex sokolovsky, xiaoxing...

44
J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

Upload: itzel-bircher

Post on 19-Jan-2016

218 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena

Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

Page 2: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

Karis, Fabiani, & Donchin (1984) demonstrated two important phenomena:

1) A list of words was learned and then a subset of these was presented among a larger set of other, novel words in a later test. It was found that the recalled, previously exposed, familiar words evoked larger P300s than novel words.

2) Some of the old words were initially presented in an unusual (oddball) font size, which made them more memorable.

Page 3: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University
Page 4: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University
Page 5: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University
Page 6: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

80% to 95% correct detection rates….but….

*Rosenfeld et al. (2004) and Mertens, Allen et al. (2008):These methods are vulnerable to Counter-measures (CMs).

(A CM is an attempt to defeat the test)

Page 7: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

… leads to 2 tasks for each stimulus:

1. implicit probe recognition vs.2. explicit Target/Non-Target

discrimination

Possible Result: Mutual Interference more task demand reduced Probe P300 that is not as big as it could be. This is why CMs hurt Old test.

Page 8: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

When you see a specific irrelevant, SECRETLY make some response, mental/physical.

After all, if you can make a special response to TARGET on instruction from operator, you can secretly instruct yourself to do the same thing to other irrelevants.

Irrelevant becomes secret target that evokes big P300. If P = I, no diagnosis.

Page 9: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

Results from Rosenfeld et al. (2004): Farwell-Donchin paradigm(BAD and BCAD are 2 analysis methods.)Diagnoses of Guilty

Guilty Group Innocent Group CM Group

9/11(82%) 1/11(9%) 2/11(18%)

Amplitude Difference (BAD) method,p=.1

Cross-Correlation(BC-AD) Method, p=.1

6/11(54%) 0/11(0%) 6/11(54%)

Page 10: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

Week BAD* BC-AD*

1: no CM 12/13(.92) 9/13(.69)

2: CM 6/12(.50) 3/12(.25)

3: no CM 7/12(.58) 3/12(.25)

*Note: BCD and BAD are 2 kinds of analytic bootstrap procedures.

Page 11: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

First study to follow… it was based on detection of autobiographical information: birth dates

Page 12: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

2 stimuli, separated by about 1 s, per trial,

S1; Either P or I…..then…..S2 ; either T or NT. *There is no conflicting discrimination task

when P is presented, there is simple “I saw it!”…

so P300 to probe is expected to be as large as possible due to P’s salience, which should lead to good detection; 90-100 % in Rosenfeld et al.(2008) with autobiographical information. It is also CM resistant. (Delayed T/NT still holds attention.)

* “I saw it” response to S1. RT indexes CM use.

Page 13: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University
Page 14: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

WEEK Hit RateWeek 1 (no CM): 11/12* (92%) Week 2 (CM): 10/11* (91%)Week 3 (no CM): 11/12* (92%)

Results with innocent (control) group.

 Confidence=.9 Confidence=.95 Test FPs Hits A’ FPs Hits A’ Iall .08 .92 .95

0 .92 .98Imax 0 .92 .98

0 .92 .98 

Page 15: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

• Subjects were divided into three groups (n=12)• Simple Guilty (SG), Countermeasure (CM), and Innocent Control

(IC)

• All subjects first participated in a baseline reaction time (RT) test in which they chose a playing card and then completed the CTP using cards as stimuli.

• SG and CM subjects then committed a mock crime.• Subjects stole a ring out of an envelope in a professor’s mailbox.

Subjects were never told what the item would be, to ensure any knowledge would be incidentally acquired through the commission of the mock crime.

• All subjects were then tested for knowledge of the item that was stolen. There were 1 P (the ring) and 6 I( necklace,watch,etc).

• CM subjects executed covert assigned responses to irrelevant stimuli in an attempt to evoke P300s to these stimuli to try and beat the Probe vs. Irrelevant P300 comparison.

Page 16: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University
Page 17: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University
Page 18: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

Condition Detections Percentage

SG 10/12 83

CM 12/12 100

IC 1/12 8

Page 19: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University
Page 20: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

• As with autobiographical information, the CTP was found to be highly sensitive at detecting incidentally acquired concealed knowledge in a mock-crime scenario.

• Detection rates using the CTP compare favorably to similar polygraph CITs. The main advantage of the CTP over the old P300 or polygraph CIT is its resistance to CM use. The traditional covert-response CMs used to defeat past P300 CITs were found to be ineffective against the CTP, and actually led to larger Probe-Irrelevant amplitude differences and detection rates.

• CM use was also easily identified by a large increase in RT between the baseline and experimental blocks.

Page 21: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

So now we have a 5-button box for the left hand. The subject is instructed to press, at random*, one of the 5 buttons as the “I saw it” response to S1 on each trial with no repeats. T and NT (S2) stimuli and responses are as previously.

We also hoped that this would make CMs harder to do. It didn’t, but we caught the CM users anyway.

* We have done other studies with non-random, explicitly assigned responses also.

Page 22: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

Autobiographical information (birthdates): One P and 4 I (other, non-meaningful dates).

*3 Groups as before: SG,CM, IC. *NEW: mental CMs to only 2 of the 4 Irrelevants: Say to

yourself your first name was the CM1, your last name as CM2. These are assigned prior to run.

*Why 2 irrels? Meixner &Rosenfeld (2010) showed countering all Irrels, not probe gives probe extra, special significance. We did a study with only irrels (5), one of which was not countered. It had big P300. So doing CMs to all irrels is not a good strategy from perp’s perspective.

*Why mental CMs? They should be faster and a bigger challenge for our CTP.

Only one block per group (no baseline).

Page 23: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University
Page 24: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

Group BT/Iall.9 BT/Imax.9

SG 13/13 (100%) 13/13 (100%)

IC 1/13 (7.6%) 1/13 (7.6%)

CM 12/12 (100%) 10/12 (83%)**These are screened via RT, which still nicely represents CM use

within a block.

Page 25: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University
Page 26: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University
Page 27: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

Elena Labkovsky & Peter Rosenfeld

Page 28: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University
Page 29: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

John Meixner & Peter Rosenfeld

How do you catch bad guys before crimes are committed, and before you know what was done, where,

when?

Page 30: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University
Page 31: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

0

4

8

12

Guilty Innocent

Group

Am

plit

ud

e (µ

v)

Probe

Iall

A Mock Terrorism Application of the P300-based Concealed Information TestDepartment of Psychology, Northwestern University, Evanston, IL 60208-2700

Page 32: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

Iall Imax Blind Imax

Guilty Innocent Guilty Innocent Guilty Innocent

1000 648 985 287 985 603

1000 610 999 416 998 602

955 598 889 476 892 649

996 611 898 430 893 605

994 150 946 17 943 689

909 475 698 284 761 547

945 600 677 365 702 536

997 555 959 250 961 569

999 586 908 217 907 565985 690 888 382 886 706

912 390 667 129 698 650

903 644 837 215 842 702

966 546 863 289 872 619

12/12 0/12 12/12 0/12 10/12 0/12

AUC = 1.0 AUC = 1.0 AUC = .979

Table 1. Individual bootstrap detection rates. Numbers indicate the average number of iterations (across all three blocks) of the bootstrap process in which probe was greater than Iall or Imax. Blind Imax numbers indicate the average number of iterations in which the largest single item (probe or irrelevant) was greater than the second largest single item. Mean values for each column are displayed in bold above detection rates.

Page 33: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

CTP is a promising, powerful paradigm, against any number of CMs, mental and/or physical and RT reliably indicates CM use. The new “P900” might also.

[email protected]

Page 34: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

Separated or split away from are called “splitting CMs”.

What happens if subjects are instructed to do CM and “I saw it” response at the same time? They lump these acts together. This is called “Lumping CMs.”

Page 35: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University
Page 36: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

Xiaoxing Hu to the rescue! (with Dan Hegeman and Elizabeth Landry).

He simply increased irrelevants from 4 to 8, which should increase demand and RT…

Page 37: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University
Page 38: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University
Page 39: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University
Page 40: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University
Page 41: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University
Page 42: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

Remember, Allen Hu gave the CMs to Ss in advance and let them rehearse.

And his subjects were geniuses, like you all…

Page 43: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

So we are now working with 10 Irrelevant items… and 3,5,7 CMs.

Page 44: J. Peter Rosenfeld, John Meixner, Michael Winograd, Elena Labkovsky, Alex Sokolovsky, Xiaoxing Hu,Alex Haynes, Northwestern University

… it is obvious that having to form—on the spot-- and hold 6 CMs for 6 of 8 Irrels in your head –as must happen in the field--is probably too hard for most bad guys to do.