turing test: why it is relevant today

Turing test:

Why it is relevant today

Huma Shah, PhDResearch Fellow

School of Computing, Electronics & Mathematics

Coventry University, UK

[email protected]

Skolkovo AI Conference, November 14, 2016 - Moscow 1

Introduction

Natural language technologies – human conversation is key

My work

Human conversation in the Turing test

Practical Turing test experiments

Findings from human judge – English language dialogue system interactions

Future: Human-centred computing conference, Kazan, August 2017


Data speaking to you in Natural Language

Natural Language is key to all industries:

transportation, finance & banking, law,

construction, governance, agriculture, civil

organisations, education, entertainment

Vast amount of data collected, artificial

intelligence will help to tell you about the hidden

trends in natural language, including through

humanlike conversation


Confidence with Natural LanguageSkolkovo AI Conference, November 14, 2016 - Moscow

Retired/pensioner using Google’s voice on Android ‘phone

Convenience of Speech RecognitionSkolkovo AI Conference, November 14, 2016 - Moscow

Dictating message

Trust in AISkolkovo AI Conference, November 14, 2016 - Moscow

Speech-to-text

Trust in Natural Language is an as assetSkolkovo AI Conference, November 14, 2016 - Moscow

Text received

Alan Turing and Natural Language

Turing knew the power of natural

language

He evolved his imitation game

from chess (1948), to questions-

answers (1950)


My work

PhD in ‘Deception-detection and Machine

Intelligence in Practical Turing tests’.

Contributed two different versions of the test:


Simultaneous comparisonViva voce: one-to-one test

My work

PhD in Deception-detection and Machine

Intelligence in Practical Turing tests.

Designed, organised and conducted major public

Turing test experiments:

Reading University, 2008


My work




Turing test experiments:

Reading University, 2008

Bletchley Park, 2012 –

-location of WWII codebreaking

11/12/2016Skolkovo AI Conference, November 14, 2016 - Moscow 11

My work




Turing test experiments: Reading University, 2008

Bletchley Park, 2012 – location of WWII codebreaking

The Royal Society London, 2014-

supported by Skolkovo through Albert Efimov

acting as Turing test Judge


Turing test success1

Eugene Goostman machine developed by team led

by Dr. Vladimir Veselov (2nd from right)


Turing test success2

Elbot the robot

From Artificial Solutions–Germany: named as key player in Fourth Industrial Revolution:

http://www.artificial-solutions.com/2016/artificial-solutions-named-key-player-in-fourth-industrial-revolution-by-opus/


My Work contd.

RoboLaw project: EU framework programme 7

(FP7) funded project ‘Regulating Emerging

Robotic Technologies in Europe: Robotics Facing

Law and Ethics’

Produced reports including final ‘Guidelines for

Regulating Robotics’ D6.2 available for download

from: http://www.robolaw.eu/


http://www.robolaw.eu/

US & UK Roadmaps for Robotics-2016

Since RoboLaw concluded in 2014

US produced report

UK report


Purpose of Turing test

Investigate the intellectual capacity of machine through its ability to answer any question put by human interrogator

Methodology is indistinguishability

Applications of indistinguishability are across and in many industries where a machine completes human task:

Human-computer interaction, e.g. in e-commerce/e-retail

Human-robot collaboration, e.g. in manufacturing

Driverless vehicles, e.g. prevent road casualties, minimise insurance costs

Pilot planes

Humanoid carers for the elderlySkolkovo AI Conference, November 14, 2016 - Moscow 17

Applications for ‘Turing measure’

Focus on natural language, specifically human

conversation aspect of Turing’s test

Applications in

Digital economy – trust

Cybercrime prevention – personal data protection

Distinguishing human from machine in textual dialogue


Human or Machine?

EXCHANGE 1

Interrogator1: all birthdays are special. name three items of

clothing

Entity1: Frank, James and Betty


Human or Machine?

EXCHANGE 2Interrogator2: Oh yes. what do you do when you're not taking

part in AI experiments?

Entity2: I'm studying here for Cybernetics MEng.


Simultaneous Comparison Interrogation

Judge J19 interrogating two hidden entities in parallel

Left Right

[11:59:14] J19: Hello

[11:59:18] E20: I was hoping to see you Judge!

[11:59:25] J19: good!

[11:59:29] E20: That's good to hear.

[11:59:38] J19: how are you

[11:59:41] E20: Everything's fine, Caused any scandals lately?

[12:00:09] J19: no not really!

[12:00:22] E20: Can you use better grammar and rephrase your

question? I'd like to help, but I can't even make it through the jumbled

mess. Use capitals and punctuation.

[12:00:34] J19: sorry

[12:00:38] E20: You're excused.

[12:00:49] J19: 't type as good as I talk

[12:00:55] E20: Talk to me whenever you can.

[12:01:49] J19: Are you interested in artificial intelligence?

[12:01:55] E20: I believe machines should do many to assist the lives of

man.

[12:02:12] J19: I agree

[12:02:17] E20: You are being well behaved today.

[12:02:49] J19: as well behaved as I can be!

[12:03:02] E20: Well then our peanut distracted me from the movie.

Bad baby. [I look at your stomach as I say it, laughing] I don't mind if -.

[12:03:19] J19: you're funny!

[12:03:23] E20: Do you really think so? I'll always believe you're

brilliant!

[12:04:00] J19: thankyou remote19 left

[12:04:03] E20: Don't think anything of it!

[11:59:57] J19: How are you doing?

[12:00:07] E24: I'm doing fine, you?

[12:00:17] J19: great

[12:00:27] E24: That's good. :)

[12:01:07] J19: are you enjoying this process?

[12:01:17] E24: So far seems good.

[12:02:02] J19: sorry if I am slow

[12:02:12] E24: That's ok.

[12:02:31] J19: do you have a favorite tv show?

[12:02:47] E24: I have quite a few.

[12:03:01] J19: me too

[12:03:16] E24: What type ot TV do you like?

[12:03:34] J19: Game of thrones

[12:03:50] E24: I prefer documentories.



Judge J19 interaction with left hidden entity E20, without times

J19: Hello

E20: I was hoping to see you Judge!

J19: good!

E20: That's good to hear.

J19: how are you

E20: Everything's fine, Caused any scandals lately?

J19: no not really!

E20: Can you use better grammar and rephrase your question? I'd like to help, but I can't even make it through the

jumbled mess. Use capitals and punctuation.

J19: sorry

E20: You're excused.

J19: 't type as good as I talk

E20: Talk to me whenever you can.

J19: Are you interested in artificial intelligence?

E20: I believe machines should do many to assist the lives of man.

J19: I agree

E20: You are being well behaved today.

J19: as well behaved as I can be!

E20: Well then our peanut distracted me from the movie. Bad baby. [I look at your stomach as I say it, laughing] I don't

mind if -.

J19: you're funny!

E20: Do you really think so? I'll always believe you're brilliant!

J19: thankyou remote19 left

E20: Don't think anything of it!


J19 interactions with Right Entity E24 without times

J19: How are you doing?

E24: I'm doing fine, you?

J19: great

E24: That's good. :)

J19: are you enjoying this process?

E24: So far seems good.

J19: sorry if I am slow

E24: That's ok.

J19: do you have a favorite tv show?

E24: I have quite a few.

J19: me too

E24: What type ot TV do you like?

J19: Game of thrones

E24: I prefer documentories.

Trust is key for conversational interfaces

“It is now only as human machine

interaction has evolved to be more

conversational and less command

driven does this enable it to become

the interface of choice.” (Tim

Bajarin, PC Magazine)


AI & DIGITAL ECONOMY

Connected gadgets, smart homes and

cities: “the one thing most people want

from a (physical or virtual) robot is to get

(satisfactory) answers when they ask a

question”

[Online article re Alexa in Amazon’s Echo -

voice-controlled hands-free speaker].


Turing’s Imitation Game: Conversations

with the Unknown

Book:

published by Cambridge University Press

September 2016


Future

Human-centred computing conference (HCC2017),

Kazan, Russia, August 2017

http://hcc.scholat.com/


Human-centred computing conference

(HCC2017), Kazan, Russia, August 2017

Two Special Tracks:

Human Enhancement

Robot Gender


Thank you for listening

Any Questions?


turing test: why it is relevant today

Technology