developers.io 2017 iphoneによるalexa/lex/pollyを利用した...
Post on 21-Jan-2018
866 Views
Preview:
TRANSCRIPT
⾃⼰紹介
クラスメソッド(株)アプリケーションエンジニア平内真⼀
・モバイルアプリサービス部(iOSエンジニア)・フリーソフト BlackJumboDog・Microsoft MVP (2013/1〜)
Alexa Alexa Voice Service
https://developer.amazon.com/public/solutions/alexa/alexa-voice-service/content/avs-api-overview
Alexa Alexa Voice Service
http://dev.classmethod.jp/smartphone/alexa-client-friendly-voice-assistant/
Alexa Endpoints/Protocol
Region SupportedCountries
URL
NorthAmerica US https://avs-alexa-na.amazon.com
Europe UK,Germany https://avs-alexa-eu.amazon.com
Alexa Endpoints/Protocol
:method=GET:scheme=https:path=/{{APIversion}}/directivesauthorization=Bearer{{YOUR_ACCESS_TOKEN}}
ダウンチャンネルストリームタイマー・アラームなど接続後10秒以内クライアントからの半閉状態で開いたまま接続の存続期間中はAVSから開く⻑い休⽌があることは珍しいことではない v20160207
Alexa Endpoints/Protocol
:method=GET:scheme=https:path=/pingauthorization=Bearer{{YOUR_ACCESS_TOKEN}}
PingandTimeout5分に1回
Alexa Endpoints/Protocol
:method=POST:scheme=https:path=/{{APIversion}}/eventsauthorization=Bearer{{YOUR_ACCESS_TOKEN}}content-type=multipart/form-data;boundary={{BOUNDARY_TERM_HERE}}
Alexa Endpoints/Protocol
https://developer.amazon.com/public/solutions/alexa/alexa-voice-service/docs/avs-http2-requests
JSONイベント・ディレクティブ
Audio⾳声
Alexa Endpoints/Protocol
audioencoded16bitLinearPCM(LPCM16)16kHzsamplerateSinglechannelLittleendianbyteorder
Alexa Interface
Interface Ddescription
SpeechSynthesizer AlexaのスピーチインターフェースSpeechRecognizer AVSのコアインターフェース 各ユーザ発話は、認識イベントを利⽤するSpeaker ミュートやミュート解除を含む、デバイスやアプリケーションの⾳量コントロールの
ためのインターフェイスSettings ロケールなど、製品のAlexa設定を管理するためのインタフェースPlaybackController
ボタンアフォーダンスを介して再⽣キューをナビゲートするためのインターフェイスAudioPlayer オーディオ再⽣を管理および制御するためのインターフェイスAlerts タイマーとアラームの設定、停⽌、および削除のためのインターフェースSystem Alexaにクライアント情報を提供するためのインタフェース
Alexa Interface
SpeechRecognizer Interface
Event/Directive
RecognizeEvent ユーザ⾳声の送信StopCapture Directive 録⾳停⽌指⽰ExpectSpeech Directive 録⾳開始指⽰(会話継続)ExpectSpeechTimedOut Event タイムアウト
Alexa Interface
SpeechSynthesizer Interface
Event/Directive
SpeakDirective Alexaの送話SpeechStarted Event 再⽣開始SpeechFinished Event 再⽣終了
Alexa Interface
https://developer.amazon.com/public/solutions/alexa/alexa-voice-service/reference/speechrecognizer
Alexa Interface
Event Code Directive1 録⾳開始2 録⾳終了3 SpeechRecognizer.Recognize
+ 録⾳⾳声200 SpeechSynthesizer.Speak
+ Alexa⾳声4 SpeechSynthesizer.SpeechStarted 204 会話が継続する場合
SpeechRecognizer.ExpectSpeech会話が継続しない場合DIRECTIVEなし
5 再⽣開始6 再⽣終了7 SpeechSynthesizer.SpeechFinished 204
SpeechRecognizer.ExpectSpeechが返っている場合は、1に戻る
Alexa Authorization
リダイレクトをスキームで受けれるように、URLSchemesで、BundleIdentifireの前にamzn- を付けたスキームを指定します。
amzn-jp.classmethod.us.ios.Friendly
Alexa Authorization
class AppDelegate:UIResponder,UIApplicationDelegate {
func application(_application:UIApplication,openurl:URL,
sourceApplication:String?,annotation:Any)->Bool{
return AIMobileLib.handleOpen(url,sourceApplication:sourceApplication)
}
Alexa Authorization
@IBAction func tapLoginButton(_sender:Any){
letSCOPE_DATA= "{alexa:all:{productID:プロダクトID,productInstanceAttributes:{
{deviceSerialNumber:デバイスシリアル番号}}
}AIMobileLib.authorizeUser(forScopes:["alexa:all"],
delegate:self,options:[kAIOptionScopeData:SCOPE_DATA])
}
開発者ポータル
Lex Endpoints/API
Service Region URL
Model buildingservice
USEast(N.Virginia)
https://models.lex.us-east-1.amazonaws.com
Runtimeservice
USEast(N.Virginia)
https://runtime.lex.us-east-1.amazonaws.com
Lex Endpoints/API
CreateBotDeleteBotDeleteBotAliasGetBotGetBotAliasGetBotAliasesGetBuiltinIntentGetBuiltinSlotTypesPutBotPutBotAliasPutIntentPutSlotTypeEtc.
AmazonLexModelBuildingService
Lex Endpoints/APIPostContent Sendsuserinput(textorspeech)toAmazonLex
PostText Sendsuserinput(text-only) toAmazonLex
POST/bot/botName/alias/botAlias/user/userId/contentHTTP/1.1x-amz-lex-session-attributes:sessionAttributesContent-Type:contentTypeAccept:acceptinputStream
POST/bot/botName/alias/botAlias/user/userId/textHTTP/1.1Content-type:application/json
{"inputText":"string","sessionAttributes":{"string":"string"
}}
Lex AWS mobile SDK for iOS
http://dev.classmethod.jp/smartphone/amazon-lex-ios-sdk/
1. pod install2. マイク利⽤許可(info.plist)3. CognitoによるIdentity発⾏とLexの初期化4. AWSLexVoiceBottn
Lex AWS mobile SDK for iOS
source'https://github.com/CocoaPods/Specs.git’target'BotSampleApp'doplatform:ios,'9.0’use_frameworks!pod'AWSLex’pod'AWSCognito’
end
Lex AWS mobile SDK for iOS
<key>NSMicrophoneUsageDescription</key><string>ForinteractionwithAmazonLex</string>
Lex AWS mobile SDK for iOS
let poolId ="us-east-1:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx"let credentialsProvider =AWSCognitoCredentialsProvider(regionType:.USEast1,identityPoolId:poolId)let configuration =AWSServiceConfiguration(region:.USEast1,credentialsProvider:credentialsProvider)AWSServiceManager.default().defaultServiceConfiguration =configuration
let BotName =“LexSample"let BotAlias =“$LATEST"let chatConfig =AWSLexInteractionKitConfig.defaultInteractionKitConfig(withBotName:BotName,botAlias:BotAlias)
AWSLexInteractionKit.register(with:configuration!,interactionKitConfiguration:chatConfig,forKey:"AWSLexVoiceButton")AWSLexInteractionKit.register(with:configuration!,interactionKitConfiguration:chatConfig,forKey:"chatConfig")
cognito
Lex
Lex AWS mobile SDK for iOS
•AWSLexInteractionDelegate•AWSLexAudioPlayerDelegate•AWSLexMicrophoneDelegate
Lex SessionAttribute
{"currentIntent":{"name":"intent-name","slots":{"slot-name":"value",},"confirmationStatus":"None, Confirmed, or Denied (intent confirmation, if configured)",},"bot":{"name":"bot-name","alias":"bot-alias","version":"bot-version"},"userId":"UserIDspecified inthe POSTrequest to AmazonLex.","inputTranscript":"Textused to process the request","invocationSource":"FulfillmentCodeHook or DialogCodeHook","outputDialogMode":"Textor Voice, based onContentType request header inruntime APIrequest","sessionAttributes":{"key1":"value1","key2":"value2"
}}
LambdaInput
sessionAttribute
http://docs.aws.amazon.com/ja_jp/lex/latest/dg/lambda-input-response-format.html
Lex SessionAttribute
{"sessionAttributes":{"key1":"value1","key2":"value2"},"dialogAction":{"type":"Close","fulfillmentState":"FulfilledorFailed","message":{"contentType":"PlainText orSSML","content":"Messagetoconveytotheuser.Forexample,Thanks,yourpizzahasbeenordered."},"responseCard":{"version":integer-value,"contentType":"application/vnd.amazonaws.card.generic","genericAttachments":[{"title":"card-title","subTitle":"card-sub-title","imageUrl":"URLoftheimagetobeshown",
LambdaResponse
sessionAttribute
http://docs.aws.amazon.com/ja_jp/lex/latest/dg/lambda-input-response-format.html
Lex SessionAttribute
POST/bot/botName/alias/botAlias/user/userId/text HTTP/1.1Content-type:application/json
{"inputText":"string","sessionAttributes":{"string":"string"
}}
ClientPOST
sessionAttribute
http://docs.aws.amazon.com/ja_jp/lex/latest/dg/API_runtime_PostText.html
Lex SessionAttribute
HTTP/1.1200Content-type:application/json
{"dialogState":"string","intentName":"string","message":"string","responseCard":{"contentType":"string","version":"string"
},"sessionAttributes":{"string":"string"
},"slots":{"string":"string"
},"slotToElicit":"string"
}
ClientResponse
sessionAttribute
http://docs.aws.amazon.com/ja_jp/lex/latest/dg/API_runtime_PostText.html
Lex SessionAttribute
"sessionAttributes":{"cards":[{"title":"Mocha","subtitle":"Pleaseenjoythefragrantmocha","body":"$4.15","imageUrl":"https://exsample.com/mocha.png","slotName":"CoffeeType","slotValue":"Mocha"
},
タップされた場合は、Textで送る
Polly アーキテクチャー
https://www.slideshare.net/AmazonWebServices/amazon-polly?qid=c9c8e2c4-1c27-41c1-bc5f-9b1e0c651c21&v=&b=&from_search=11
Polly アーキテクチャー
https://www.slideshare.net/AmazonWebServices/amazon-polly?qid=c9c8e2c4-1c27-41c1-bc5f-9b1e0c651c21&v=&b=&from_search=11
Polly AWS mobile SMD
source'https://github.com/CocoaPods/Specs.git’target'BotSampleApp'doplatform:ios,'9.0’use_frameworks!pod'AWSPolly’pod'AWSCognito’
end
Polly AWS mobile SMD
letinput=AWSPollySynthesizeSpeechURLBuilderRequest()input.text=textView.textinput.outputFormat=AWSPollyOutputFormat.mp3input.voiceId=AWSPollyVoiceId.joannaletbuilder=AWSPollySynthesizeSpeechURLBuilder.default().getPreSignedURL(input)
builder.continueOnSuccessWith{(awsTask:AWSTask<NSURL>)->Any?inleturl=awsTask.result!self.audioPlayer.replaceCurrentItem(with:AVPlayerItem(url:urlasURL))self.audioPlayer.play()returnnil
}
http://dev.classmethod.jp/smartphone/amazon-polly-sdk/
AWSPollySynthesizeSpeechURLBuilderRequest
テキスト
声の種類
AVFoundation.AudioPlayer
Summary
SkillSkillSkillSkill
Intent, Slots
Lambda
Skill
Intent, Slots,SessionAttributes
Intent, Slots,SessionAttributes
top related