TDT2000 Link Detection

Evaluation Results

Jonathan Fiscus and George Doddington

NIST Speech Group

TDT Link Detection Task

One Participant: University of Massachusetts

System Goal:– To detect whether a pair of stories discuss the same topic.

(Can be thought of as a “primitive operator” to build a variety of applications)

?

2000 Link Detection Results• A lot was learned last year:

– The test set must be properly sampled • “Linked” story pairs were selected by randomly sampling all

possible on-topic story pairs• “Unlinked” pairs were selected using all on-topic stories as one

of the pair, and a randomly chosen story was chosen as the second

– This year, the task was made multilingual

– More story pairs were used

0

10000

20000

30000

40000

50000

60000

70000

Overall Eng-to-Eng Eng-to-Man Man-to-Man

Num

ber

of S

tory

Pai

rs Linked Pairs

UnLinked PairsLink Detection Test Set Composition

Link Detection Results

• Required Condition– Multilingual texts– Newswire + Broadcast News ASR,– Reference story boundaries– 10 file decision deferral

0.31

0.17

0.330.41

0.10

1.00

UMass1

Nor

mal

ized

Lin

k D

etec

tion

Cos

t

OverallEng-to-EngMan-to-ManEng-to-Man

Overall