Slide 1

Sudo Chef Dan Abbate Kyle Despins Tom Murphy Eli Siskind Advisor: Professor Tessier Slide 2 Current State of the Kitchen Difficulties of following instructions across one or more recipes. Picking a recipe involves multiple cross references to your inventory of ingredients. Slide 3 What is Sudo Chef? Home Automation + Recipe Chooser and Ingredients Tracker Slide 4 System Overview Bluetooth Hub 123 456 789 0 Slide 5 Block Diagram (Before) Ingredients Tracking Recipe Assistance Voice Recognition Appliance Devices Stove controller Oven Controller Microwave Controller Bluetooth Hub Image Processing Ingredients Database Interpret Results Audio input Mobile Bluetooth Adaptor Voice Processing Recipe helper mobile interface Thermometer Mobile Camera (Barcode Capture) Recipe API Kitchen Actions Dan Tom Eli Kyle Slide 6 Block Diagram (After) Ingredients Tracking Recipe Assistance Voice Recognition Appliance Devices Microwave controller Stove Controller Oven Controller Bluetooth Hub UPC lookup Ingredients Database Interpret Results Audio input Mobile Bluetooth Adaptor Voice Processing Recipe helper mobile interface Thermometer Camera system controller Yumlee API Kitchen Actions Dan Tom Eli Kyle Slide 7 MDR Deliverables Digitally controlled hot plate Standard handshake via Bluetooth between tablet and microcontroller hub Bar code module up and running GUI prototype and basic recipe parsing Slide 8 Recipe Fetch, Parsing, and Guidance Eli Siskind Slide 9 Context Recipe Database Bluetooth Hub 123 456 789 0 Slide 10 Getting Recipes Need a way to search for recipes based on ingredients User must be able to view recipes and select one A parsed recipe must be converted step by step actions performed by the Sudo Chef system Slide 11 Implementation Slide 12 Recipe Fetching Yummly API Search Request Recipe Request Ingredients Yummly URL Parsing Module Recipe Results String External Site Recipe Fetch Recipe URL Recipe Metadata String HTML File Recipe Results Recipe ID Client Side Slide 13 Optimization Asynchronous HTTP Get threads for all requests Thumbnails transformed in separate threads Slide 14 Recipe Guidance Steps Instructions Push notifications Voice commands Appliance Actions Preheat oven Set hot plate temperature Feedback Request Check if meat is cooked Slide 15 MDR Progress Ingredients Tracking Recipe Assistance Voice Recognition Appliance Devices Microwave controller Stove Controller Oven Controller Bluetooth Hub UPC lookup Ingredients Database Interpret Results Audio input Mobile Bluetooth Adaptor Voice Processing Recipe helper mobile interface Thermometer Camera system controller Yumlee API Kitchen Actions Dan Tom Eli Kyle Slide 16 Barcode Scanner Kyle Despins Slide 17 Context Bluetooth Hub 123 456 789 0 Barcode Decoding Product Lookup Barcode Slide 18 Barcode Purpose Easiest way for user to put ingredients into inventory Slide 19 How do you get a product from a barcode? Slide 20 Barcode Process Barcode Capture Processing Grayscale Gaussian ZXing Format image Decode call UPC Query HTTP GET Parse JSON Ingredient Database Raw Image Smoothed Image UPC Integer Product Object Slide 21 Gaussian Smoothing Collins, Robert Penn State Slide 22 MDR Progress Ingredients Tracking Recipe Assistance Voice Recognition Appliance Devices Microwave controller Stove Controller Oven Controller Bluetooth Hub UPC lookup Ingredients Database Interpret Results Audio input Mobile Bluetooth Adaptor Voice Processing Recipe helper mobile interface Thermometer Camera system controller Yumlee API Kitchen Actions Dan Tom Eli Kyle Slide 23 Going Forward Switch focus to Voice Processing Optimization and Integration of Barcode scanner CDR: Barcode integration Prototype voice processing Slide 24 Bluetooth Handshake Tom Murphy Slide 25 Context Bluetooth Hub 123 456 789 0 Slide 26 Why Bluetooth ? Need to be able to connect application to appliances 123 456 789 0 Slide 27 Android App to create connection from phone to Bluetooth module Bluetooth Hub implemented Atmega32 with Uart connected to HC-05 Bluetooth module Slide 28 Android App Main Activity used to discover possible Bluetooth devices SendData activity used to send data from device to chip Slide 29 Bluetooth Module HC-05 blinks constant ready light when on and searching for device to connect to HC-05 indicates connected when blinking light is stalled and not constant Slide 30 MDR Progress Ingredients Tracking Recipe Assistance Voice Recognition Appliance Devices Microwave controller Stove Controller Oven Controller Bluetooth Hub UPC lookup Ingredients Database Interpret Results Audio input Mobile Bluetooth Adaptor Voice Processing Recipe helper mobile interface Thermometer Camera system controller Yumlee API Kitchen Actions Dan Tom Eli Kyle Slide 31 CDR Deliverables Data being sent from Android application and forwarded to corrected corresponding device (i.e hot plate or convection oven) Slide 32 Hotplate Controller Dan Abbate Slide 33 Context Bluetooth Hub 123 456 789 0 Slide 34 Hot Plate Automation USART Signal Pulse Width Modulation LPF Control Signal Slide 35 Microcontroller Setup Input: USART Output to solid state relay: PWM Non-inverted, Fast PWM mode, with no clock prescaling Sent through low pass filter (Cutoff 4.8 Hz) USART SignalController Action PWM Duty Cycle SSR Input Voltage 0x00Turned Off0%0v 0x01Low43%1.8v 0x02High100%5v 0x03Send Current Status to App -- Functions Slide 36 MDR Progress Ingredients Tracking Recipe Assistance Voice Recognition Appliance Devices Microwave controller Stove Controller Oven Controller Bluetooth Hub UPC lookup Ingredients Database Interpret Results Audio input Mobile Bluetooth Adaptor Voice Processing Recipe helper mobile interface Thermometer Camera system controller Yumlee API Kitchen Actions Dan Tom Eli Kyle Slide 37 Plan for CDR Digitally controlled Toaster Oven Begin interfacing with microwave Slide 38 MDR Deliverables Digitally controlled hot plate Standard handshake via Bluetooth between tablet and microcontroller hub Bar code module up and running GUI prototype and basic recipe parsing Slide 39 CDR Deliverables Digitally controlled convection oven Data being sent from Android application and forwarded to corrected corresponding device (i.e hot plate or convection oven) Barcode module full integration and prototype voice processing Full recipe parsing, communication with hardware via Bluetooth, inventory tracking coupled with barcode module Slide 40 Demonstration Slide 41 Communication Bluetooth Protocol Slide 42 UPC-A format Slide 43 Sphinx Voice Recognition Use grammar language model: Java Speech Grammar Format Statistical Language Model Slide 44 Yummly Two API calls Search recipes Get recipe Supports ANDing or ORing terms Returns JSON objects with multiple fields Ingredients Allergens Source and attribution information Prep time Number of servings Slide 45 Specifications Keep track of bought food using bar codes Suggest recipes containing ingredients in inventory while they are fresh Parse 100% of recipes, and recognize 95% of recipe terms Respond to natural voice commands Guide user through recipes by preheating oven, heating hot plate, microwaving food, and alerting users with push notifications and voice instructions Always shut off appliances when not in use, including when mobile app fails Slide 46 Budget Slide 47 Bit No76543210 NameFOC0WGM00COM01COM00WGM01CS02CS01CS00 Initial Val01101001 TCCR0 Register: http://www.atmel.com/Images/doc2503.pdf Slide 48 PWM Modes Slide 49 16 Mhz Baud Rate UBRR (dec) UBRR (hex) % of error 30033320x0D040.0 60016660x06820.0 12008320x03400.0 24004160x01A00.0 48002070x00CF0.2 96001030x00670.2 14400680x00440.6 19200510x00330.2 28800340x00220.8 38400250x00190.2 57600160x00102.1 76800120x000C0.2 11520080x00083.7 23040030x00037.8 25000030x00030.0 http://www.wormfood.net/avrbaudcalc.ph p?postbitrate=19200&postclock=16 Slide 50 http://www.grainger.com/product/OMRON-Relay-5ZH25