![Page 1: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/1.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C. © 2016 IBM Corporation
Distributed Development of Cognitive Systems at IBM Research - Brazil
Alan Braz, MSc, Software [email protected], @alanbrazMaíra Gatti de Bayser ([email protected]), PhD, Research Staff Member, Master Inventor Claudio Pinhanez ([email protected]), PhD, Senior Manager
![Page 2: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/2.jpg)
© 2016 IBM Corporation
IBM Research Labs IBM Research – Openings in 2011 IBM Research – Openings in 2012
China
WatsonAlmaden
Austin
TokyoZurich
India
Dublin
Australia Brazil
Africa▪ Next Gen Public Sector ▪Water & transportation▪ Human Capacity Development
▪ Natural Resources▪ Disaster management▪ Healthcare/Life Sciences
▪ Natural Resources▪ Social Data Analytics (FSS)▪ Visual Analytics & Insights▪ Industrial S&T
▪ Analytics & Intelligence▪ Systems &Software▪ Industry research
▪ Internet of Things▪ Big Data / Analytics▪ Enterprise Cloud Services▪ Energy, Commerce, Traffic
▪ Big Data Analytics▪ HW & SW Quality▪ Cloud▪ Mobile
Haifa
▪ Smarter Cities▪ Analytics
▪ Services ▪ Big Data Analytics▪ Front Office Digitization
▪ Semiconductors▪ Systems▪ Software▪ Services▪ Analytics
▪ Semiconductors▪ Processors
▪ Analytics▪ Storage ▪ Nanotech
Healthcare
▪ Nanotech▪ Security▪ Business
Analytics▪ Systems
Industry Solution Lab
IBM Research: 12 Labs Worldwide in 10 Countries - 3,000 employees
![Page 3: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/3.jpg)
© 2016 IBM Corporation
IBM inventors received a record 7,355 U.S. patents in 2015, marking the 23th consecutive year it has topped the list of the world’s most inventive companies. IBM became the first company to be granted as many as 5,000 U.S. patents in a single year. It took IBM's inventors more than 50 years to receive their first 5,000 patents after the company was established in 1911. IBM's first patent: U.S. Patent #998,631 issued July 25, 1911.
23 Consecutive Years of IBM Patent Leadership
![Page 4: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/4.jpg)
© 2016 IBM Corporation
IBM Research I Brasil
Mission: To be known for our science and technology and vital to IBM, Brazil, our partners in the region, and globally.
IBM @ Tutoia Street
São Paulo
IBM @ Avenue Pasteur
Rio de Janeiro
First IBM Research lab in the South America, founded in 2010
Strategic Focus AreasSmarter natural resources discovery and logistics with emphasis on NR analytics. Optimization, SW technologies
Cognitive Computing with focus in Social Data Analytics reinvent large-scale service systems and enterprises.
Industrial Technology & Science with emphasis in sensors and microfluidics for Health and Natural Resources
Visual analytics & comprehension: methods and technologies for expert knowledge capturing, modeling, and representation
![Page 5: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/5.jpg)
© 2016 IBM Corporation
IBM Watson: From Jeopardy to Cognitive Solutions and ....
5
![Page 6: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/6.jpg)
© 2016 IBM Corporation
... bringing Cognitive Computing to Everyone: Watson Developer Cloud
6
http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/
![Page 7: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/7.jpg)
© 2016 IBM Corporation
Platform as a Service
7http://www.bluemix.net
![Page 8: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/8.jpg)
© 2016 IBM Corporation
SOCIAL COGNITIVE TECHNOLOGIES
social knowledge mining
conversational cognitive agents
2016: Social Data Analytics Research Workstreams
social knowledge gatherer
social knowledge organizer
cognitive dialogue
framework
cog agent interaction
design
CognIA: self-service investment adviser
mission: to make Watson socially skilled
8
![Page 9: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/9.jpg)
© 2016 IBM Corporation
Social Cognitive Technologies Research
9
Claudio PinhanezManager,Social Data AnalyticsPhD MIT
Maíra GattiPhD PUC-Rio
Ana AppelPhD USP-SC
Paulo CavalinPhD Quebec
Alan BrazMSc. Unicamp
Heloisa CandelloPhD Brighton
CapabilitiesKDD, HCI, Simulation, Machine Learning, Social Media, Agents, Design, Graph Mining, Computer Vision, Deep Learning, Large Databases, Agile.
+10 undergraduate/graduate interns
Marcia ItoPhD USP-Poli
Renan SouzaMsc UFRJ
Flavio FigueiredoPhD UFMG
Marisa VasconcelosPhD UFMG
![Page 10: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/10.jpg)
©2015 IBM Corporation10 29 August 2016
Creating a new generation of self-service automated systems for investment and wealth management.
Investment Advisors
![Page 11: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/11.jpg)
© 2015 IBM Corporation
Conversational Systems, the 4th Interface Paradigm
11
1. batch 2. command line 3. point & click 4. conversational
“There are more than 2 billion users of mobile apps” (The Wall Street Journal, December, 2015)
“Forget Apps, Now The Bots Take Over”(TechCrunch, Sep 29, 2015 by Beerud Sheth)https://techcrunch.com/2015/09/29/forget-apps-now-the-bots-take-over/
![Page 12: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/12.jpg)
©2015 IBM Corporation12 IBM Research
Prototype: Direct, Simple, Conversational
● Natural Dialogue System○ Mixed initiative
● Dialogue Mediation○ Coordination control
● Investment Options○ Investment Concepts & Options – Q&A○ Investment Simulation – Task-oriented○ Investment Opinion – Hard to answer
● Portuguese● Brazilian Finance
○ Saving Accounts○ CD (Certificate of Deposit)
■ CDB (Certificado de Depósito Bancário)
![Page 13: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/13.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Challenges on Distributed Development
13
Picture source: https://www.linkedin.com/pulse/how-organize-efficient-work-distributed-development-team-soloshchuk
![Page 14: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/14.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Remote working… ?
14
![Page 15: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/15.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
How do distributed teams interact with each other…?
“Meetings-driven development”“Email-driven task management”
15
![Page 16: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/16.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Issues on Remote Working / Distributed Cognitive Teams
▪Communication style▪Expectations and establishing common ground▪Performance control▪ Interdependence
– Lacks share of information -> less creativity▪Communication patterns
– Remote working ≈ Closed door offices– Social interactions– Lack of social communication is associated with lower trust and cohesion
▪High volume of messages can lead to confusion▪Need skilled managers▪Balance onsite and remote working▪ Integrate social business technologies▪Software integration▪Privacy concerns
16
![Page 17: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/17.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Challenges on Cognitive Systems Development
17
Picture source: http://www.forbes.com/sites/ibm/2015/10/06/four-steps-to-get-started-with-cognitive-computing/
![Page 18: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/18.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Cognitive Computing
18
White paper: http://www.research.ibm.com/software/IBMResearch/multimedia/Computing_Cognition_WhitePaper.pdf
![Page 19: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/19.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Software Engineering & Cognitive Systems Development
▪Current well established SE practices:– Agile process– Extreme Programming (XP)– Continuous process– Design improvement– Shared understanding– Collective code ownership– Simple design
19
Waterfall Model(Royce, W.)
Prototyping Model (Brooks, F.)
V Model (Forsberg, K.)
Incremental Dev. (IBM)
Spiral Model (Boehm, B.)
1970 1975 1988 1991 2001
Agile Soft Dev. Kent Beck XP
Scrum
1996
FDD
RUP
Applicability Degree for Cognitive Systems Development
![Page 20: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/20.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Software Engineering & Cognitive Systems Development
▪New challenges:– Interdisciplinary areas– Innovation– Complexity– Algorithms evolution– Data management– Models management
20
Standard Waterfall Model
Spiral Model
eXtreme Programming
predictive adaptive
rigidmethodologies
iterativemethodologies
agilemethodologies
Extreme Programming. Kent Beck, 1996
![Page 21: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/21.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Complexity and Uncertainty
21
http://iq3group.blogspot.com.br/2012/12/solving-data-governance-by-scaling.html
![Page 22: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/22.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Challenges
22
White paper: http://www.research.ibm.com/software/IBMResearch/multimedia/Computing_Cognition_WhitePaper.pdf
![Page 23: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/23.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Methods and Tools for Managing Distributed Development of Cognitive Systems
23
![Page 24: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/24.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
DevOps
https://www.ibm.com/developerworks/br/devops/principles.html
24
![Page 25: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/25.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
DevOps
▪Enable collaboration, co-development and reuse (a.k.a. social coding).
▪Minimize the friction and fuss required to build and deploy your code. This will be a combination of automating what are often manual or custom processes, in combination with providing ready-to-go platforms and specific guidance on release requirements.
▪Provide ready-to-go tools, platforms and methods. Many of these services are currently available from various internal and external corporate offerings. We're working on providing many of these services automatically and without the friction of concerns about funding, signups, approvals, etc.
▪Make it easy (or at least, as easy as possible). This is all about providing ready and useful guidance on using tools, getting help, and learning what you need to know.
25
![Page 26: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/26.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Tools
26
![Page 27: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/27.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.27
![Page 28: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/28.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Managing tasks with GitHub issues
28
![Page 29: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/29.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
GitHub org structure
29
▪14 code repos– Java, Node.js, Python, Go, HTML
▪1 planning repo only with Issues– 81 Open – 325 Closed (since Jun 2015)
▪2 types of Infrastructure– Softlayer– Bluemix
▪2 Database paradigms– SQL (DB2 and DashDB)– NoSQL (Cloudant)
![Page 30: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/30.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Investment Advisor Team
30
Undergrad interns
Senior Manager
PhD
MSc
SoftwareEngineer
HCI, Machine Learning
Designer, UserExperience
Database,Graph Analytics
DataMining
Data Mining
Databases
Software Engineer,Multi-Agent Systems
Machine Learning
![Page 31: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/31.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Architecture - Several Micro-services
31
![Page 32: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/32.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
The Architecture
▪The Investment Advisor Prototype is built on top of SABIÁ - the Speech-Act-Based Intelligent Agents framework
▪SABIÁ is a conversational framework for multi-party chabots built on top of Akka middleware that enables mixed-initiative dialogues.
– Gatti de Bayser, M.; Cavalin, P.; Programming Conversational Agents with Akka Actors. Tutorial at 18th European Agent Systems Summer School (EASSS 2016), Sicily, Italy. Available to download.
▪Akka is a toolkit and runtime that implements the Actor model on the JVM and was inspired by Erlang in many ways.
▪Akka can be used to build highly concurrent, distributed, and resilient message-driven applications. A large ecosystem of more than 250 Github projects has grown on Akka, in programming languages like Scala, Java, Groovy or JRuby.
Akka.io. Akka is Open Source and available under the Apache 2 License.
32
![Page 33: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/33.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Actor Model
▪Originated in 1973
▪The actor model in computer science is a mathematical model of concurrent computation that treats "actors" as the universal primitives of concurrent computation.
▪In response to a message that it receives, an actor can:– make local decisions,– create more actors,
– send more messages, and
– determine how to respond to the next message received.
▪Actors may modify private state, but can only affect each other through messages (avoiding the need for any locks).
33
Carl Hewitt; Peter Bishop; Richard Steiger (1973). "A Universal Modular Actor Formalism for Artificial Intelligence". IJCAI.
![Page 34: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/34.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Cloud Deployment
34
![Page 35: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/35.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Main back-end project - Akka agents
35
▪Technologies– Java runtime– Akka toolkit (http://akka.io/)– Socket.io
▪11 team members contributed▪Average of 50 commits/member▪2 main developers
– 80+ commits▪Automated unit and integration tests▪Continuous deploy▪Performance profiling ▪Stress tests
![Page 36: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/36.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Main front-end project - Socket.io Web/mobile chat
36
▪Technologies– Node.js runtime– Ionic framework (Angular.js)– Socket.io
▪6 team members contributed▪Average of 29 commits/member▪2 main developers
– 60+ commits
▪Continuous deploy
▪Mobile and web interfaces
![Page 37: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/37.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Evolution
37
▪Sort of a 2-week Scrum Sprint called Step▪Review and Planning at the same meeting (2h)▪Weekly status meeting (like a “Daily Scrum”)▪Not fully self-organized team
Akk
a ag
ents
Cha
t
![Page 38: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/38.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Conclusions
38
![Page 39: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/39.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
Conclusions
▪It is not trivial to manage distributed teams on the development of cognitive applications
▪Several communication tools and methods are needed
▪Software engineering practices and DevOps can provide methods and tools to help tackling some challenges
▪There is still a need on improving the process so it can become more mature while with space to innovation and creativity.
39
![Page 40: Distributed Development of Cognitive Systems at IBM Research - Brazil - CBSoft2016](https://reader035.vdocuments.site/reader035/viewer/2022070601/588677ac1a28ab962a8b4ffd/html5/thumbnails/40.jpg)
© 2016 IBM CorporationGatti de Bayser, M.; Braz, A.; Pinhanez, C.
IBM Research – Brazilhttp://www.research.ibm.com/brazil/[email protected], @alanbraz