a sociotechnical systems study of virtual r&d organizations
DESCRIPTION
TRANSCRIPT
A Sociotechnical Systems Study of Virtual R&D Organiza;ons
Co-Chairs Ronald E. Purser & Ramkrishnan V. Tenkasi
Panelists: Doug Austrom Betty Barrett Bert Painter
Pamela Posey
Discussant: William A. Pasmore
Sponsors: ODC, OCIS, TIM
Supported by NSF-VOSS award #0943237
Central Research Ques;on How do virtual modes of communica3on influence the quality of delibera3ons at various stages of the R&D process?
-‐ STS theore3cal lens -‐ Compara3ve case study -‐ Grounded theory building
Session Overview • Introduc3on and Framing: 1:15-‐1:30
Panel Presenta3ons • Large Video Game Inc 1:30-‐1:40 • Alzheimer’s Disease center 1:40-‐1:50
• CALTECH Micro & Nano-‐photonics 1:50-‐2:00 • Summarizing across the cases 2:00-‐2:15
• Discussant comments 2:15-‐2:30
• Q& A 2:30-‐2:45
Central Research Ques;on How do virtual modes of communica3on influence the quality of delibera3ons at various stages of the R&D process?
-‐ STS framework -‐ Delibera3ons -‐ Stages of the R&D
Evolu3on of Socio-‐Technical Systems
STS v1.0 Rou3ne work in a single organiza3on – e.g., coal mines, factories, oil refineries
• Work groups with pooled iden3ty • Unitary conversion process • Linear conversion sequence
STS v2.0 Non-‐rou3ne knowledge work in single organiza3ons – e.g., white collar office work, professional services firm, NPD and R&D
• Individual performers, specialized exper3se
• Mul3ple, concurrent conversion processes
• Nonlinear conversion flow
STS v3.0 Virtual, non-‐rou3ne work – e.g., R&D consor3a, complex supply chains
• Individual performers and work groups distributed across mul3ple loca3ons and/or organiza3ons
• ICT enabled • Mul3ple, concurrent independent and interdependent conversion processes
• Nonlinear conversion flows
*
Delibera;ons: Defini;on and Elements
Pava, 1983
• Delibera;ons are sense making exchanges in which people engage with themselves or others to reduce the uncertainty of a problema9c issue. -‐-‐i.e. an issue subject to two or more interpreta9ons
• The salient elements of a delibera3on include the … • Topics or problema3c issues facing the social en3ty about which people reflect and communicate
• Forums in which they occur which may structured, semi-‐structured, or unstructured or ad hoc
• Par;cipants both those who are currently involved and those who ideally should be involved in the delibera3on.
A Six Stage Con;nuum of the R&D Process
Sampling of Virtual R&D Projects across Sites and R&D Stages
Type of Research Development Program/Project
Research Site R1 Pure
R2 Applied
D1 Exploratory
D2 Advanced
D3 Start-up
D4 Scale-up
Large Video Game Inc.
X X
NIA Alzheimer’s Centers
X X X
Caltech Micro & Nano-Photonics Research Group
X X
Large Video Game, Inc.
Case Study
Major support and funding for this project has been provided by the Na;onal Science Founda;on.
NSF Award #0943237
LVG Inc. Core Team
Florida
QA Tes;ng
Southern USA
Engineering and Web Development
USA
LVG Inc.
Canada
Artwork (Philiphines, India, China)
Virtual Organiza;on for LVG Inc. Game 2012
Product Development
Systems Engineering
China
NSF Award #0943237
Systems View of Game Development
NSF Award #0943237
Core Game Team
NSF Award #0943237
QA
Art
Engineering Production
Design
Org Structure
NSF Award #0943237
Producer & Director
Art Director Lead Designer Technical Director
Engine/System Coders Gameplay Coders Network Coders
Hardware Specialists Audio Specialists
Rendering Specialists
Gameplay Designers SP Level Designers Scripting Specialists Cinematic Specialists MP Level Designers
Concept Artists Character Artists
Environment Artists 3D Modelers
Texture Artists Technical Artists
Animators
Audio Director
Assistant Producer Writers
Sound Engineers Music Composer Recording Studio
Voice Actors Localization Studio
AGILE -‐ SCRUM • Rapid prototyping process + mul3ple itera3ons to feature comple3on
• Episodic surveys of “POD” members, assessing their virtual interac3ons with vendors aler every SPRINT cycle (3 week cycles, 13 total SPRINT cycles, we entered at SPRINT 6).
• AGILE has significant implica3ons for virtual organiza3ons insofar as expecta3ons may change during the life of a contract, during various “Sprints”
NSF Award #0943237
SCRUM BOARD
Feature to develop Tasks in queue
Tasks in progress Tasks complete Feature complete
NSF Award #0943237
déjà vu
• Souder’s NSF research – “Task-‐dominant approach to R&D” (Souder, 1987)
• “Rugby vs. Relay-‐race” project management
• How and to what extent do various vendors synchronize and entrain with itera3ve nature of AGILE/SCRUM at the host organiza3on?
NSF Award #0943237
NSF Award #0943237
Data Collec3on to Date • Ini3al Scoping Phone Interviews with Various General and
Project Managers (Spring/Summer 2010) • On-‐site interviews (14) with “Franchise” team (Sept. 2010)
– (General Manager, Project & Program Managers, Ar3sts, Solware Engineers, Crea3ve Director, Audio Engineers, Computer Graphics, Outsourcing Managers, Quality Assurance)
• Milestone telephone interviews with Project Managers, Ar3sts, Engineers and Quality Assurance (Jan. 2011)
• Episodic electronic surveys at the end of each “Sprint” cycle, and at the end of “Alpha” (@ 3 weeks, 10/10 – 05/11)
• Electronic surveys of key vendors (May-‐June 2011)
• Follow-‐up phone interviews with Project Managers, Ar3sts, Engineers and Quality Assurance (Aug. 2011)
NSF Award #0943237
Uniqueness of Producing Art in Virtual Organiza;ons
• Art studios produce art assets, not games – Large knowledge gap between making art, and making art that works for games
– Oversea vendors – gap larger • How is tacit knowledge obtained in virtual exchanges? – Some vendors more sophis3cated than others
• Ar3s3c and aesthe3c judgment is part of the socio-‐technical system
• Rich, mul3-‐media documenta3on is key
NSF Award #0943237
Socio-‐Technical Systems of Virtual Art Produc;on: Emergent Themes
– Rou;ne vs. Nonrou;ne Work. Art assets vary in terms of “Likeness.” The more authen3c and life-‐like the art asset, the less rou3ne, but this actuality reduces “ar3s3c license.”
– Unit Opera;ons: How to assign and parcel out pieces of work? Too litle turns the vendor into a “monkey,” too large is overwhelming.
– Expecta;on se`ng is key to joint op3miza3on – Variances take the form of revisions/ reitera3ons and delays in delivery
– Managing front-‐end delibera;ons is key to seung flexible and realis3c contracts
NSF Award #0943237
Scoping Vendors
Vendor Selection
Defining & Estimating Project Work
Pre-test & Trial Run
Key Delibera;ons in Virtual Art Produc;on
Detailed Documentation & Requirements
Art Assets Deliveries
Initial Testing of Assets
Debugging Re-negotiating Project Work
Contract Negotiation
Critiques
Critiques
NSF Award #0943237
Reducing Equivocality in Art Delibera;ons
• Contrary to tradi3onal studies on virtual teams that strive for 3ght coupling, ar3s3c produc3on relies more on reducing equivocality of front-‐end delibera3ons by providing well defined documenta3on, visual targets, and reference materials
– “I write my emails out properly (addressing the person, wri3ng organized paragraphs, and then using closing remarks with a signature). I started no3cing that a lot of that properness started to fade as the sprint carried on. We are star3ng to make the transi3on into alpha and the workload has been plenty, so I thought those emails were a great reflec3on of it. (just something funny I no3ced)” 2/23/11
• Key delibera3ons in virtual art produc3on involve gaining shared agreement on “standardiza3on of outputs” (Mintzberg, 1993; Ramesh & Dennis, 2002)
• The use of both “seman3cally and aesthe3cally rich media” (e-‐mails, documenta3on, visual targets, screen shots)
NSF Award #0943237
Virtuality Impacts the Nego;a;on of Art-‐Based Project Work
• Ambiguity in defining 3me per asset • Pricing es3mates are olen inaccurate
• Cost effec3veness is hard to predict – “Cost nego3a3on. There was a task that required modifying heads. It took us
30min to 1hr to do one head in-‐house, so I tasked them out as 4hr jobs. The outsourcers wanted anywhere from 6-‐16hrs to do, and one agreed to setle at 4hrs per head aler doing 20 of them. Some3mes you win with outsourcing, some3mes it makes no sense to outsource.”
NSF Award #0943237
Management of Risk Virtual Engineering
• A Key Delibera3on involves managing risk in providing access to source code packages – Key Variances: Delays in obtaining approvals, incomplete documenta3on, changing expecta3ons/func3onality, mis-‐matches in preferred modes of communica3ons
• Our biggest challenge with Vendor E has been geung them set up with VPN and the appropriate permissions. There is a lot of red tape that we are having to go through wi3n our company to get them properly set up. It ul3mately took up to 5 weeks to get them fully ready to start work for us. Thus, the project is behind schedule and we are looking at the project coming in 3 weeks late from the original final date. …we have had a ton of email and conference alls trying to figure out the next
steps in geung Egeniq set up.
NSF Award #0943237
Emerging Theory and Prac;ce Ques;ons
• Can we observe and theorize how to design “high performance” virtual organiza3ons?
• Can non-‐rou3ne STS principles be applied to virtual art produc3on?
• Can theories of organiza3onal aesthe3cs help to understand arts-‐based virtual delibera3ons? [Aesthe3cs = sensory knowledge and felt meaning of objects and experiences.]
• How does aesthe3cs factor into virtual art delibera3ons to create value and performance?
NSF Award #0943237
These Slides are Op3onal
NSF Award #0943237
NSF Award #0943237
NSF Award #0943237
NSF Award #0943237
LVG Core Team
NSF Award #0943237
Vendors: Factors Impeding Virtual Communica3ons
NSF Award #0943237
NSF Award #0943237
Research Site and Project
• A virtual R&D eco-‐system comprised of 29 NIA-‐funded Alzheimers Disease Centers (ADCs) and the Na3onal Alzheimers Coordina3ng Center Center (NACC)
• Uniform Data Set
Key Peformance Outcomes of of this Alzheimers Research Eco-‐System
Number of enrollees*
23,577 subjects Both demented and non-‐demented
Number of variables
725 variables using up to 18 standardized forms at each visit
Longitudinal data is collected by clinicians, neuropsychologists, and other ADC research personnel each visit -‐-‐ ini3al, annual follow-‐up and milestones such as death or dropout
Number of research projects**
157 research projects have been approved to use the NACC data
• 104 by researchers from the ADCs • 27 by NACC staff • 26 by external researchers
Number of publica;ons using NACC data***
Since 2001 367 publica3ons • 180 NACC-‐funded projects • 53 by NACC staff • 53 NACC-‐ADC collabora3ve studies • 72 by non-‐NACC researchers • 9 from Indirect NACC support
* As of June 2011 ** As of August 2011 *** As of January 2011
Alzheimers Research Eco-‐System and a “Culture of Collabora3on”
• Culture of collabora3on among Alzheimers researchers has extended and has led to other major collabora3ve research ini3a3ves • Alzheimers Disease Neuro-‐Imaging Ini3a3ve (ADNI) • Genome-‐Wide Associa3on Study (GWAS) • Alzheimers Disease Coopera3ve Study (ADCS) • Dominantly Inherited Alzheimer Network (DIAN) • Interna3onal collabora3ve ini3a3ve with Worldwide ADNI
Research Process to Date • Conducted 14 interviews with a representa3ve cross-‐sec3on of key par3cipants in the UDS project including ADC Center Directors, Na3onal Ins3tute of Aging (NIA) ADC Program Director, NACC Director, Clinical Core Directors, and Data Core Directors
• Determined that the UDS project is D1 in the R&D con3nuum, Exploratory Development work – “They know what; but they didn’t know how to achieve it.”
• Iden3fied key delibera3ons in the ini3al development, ongoing data collec3on/submission/quality control, and revision of the Uniform Data Set including … • Iden3fying and choosing the “best fit” data collec3on methods, diagnos3c protocols and instruments, variables for the UDS
• Achieving agreement on the UDS across the 29 ADCs • Ensuring adherence with UDS protocols, data quality, and 3meliness of data submission at all 29 centers
• Revising assessment instruments and protocols in the UDS and achieving agreement on the new instruments
Factors that Contributed to Delibera3on Effec3veness
• Delibera3on processes itera3vely evolved within the informal network and the formal structure, the NACC
• Face-‐to-‐face forums twice a year have enabled the ICT-‐enabled (informa3on and communica3on technologies) forums and delibera3ons to be more effec3ve and efficient
• Relied on high levels of input and par3cipa3on – directly on commitees and sub-‐commitees and via surveys, bulle3n boards, direct feedback to Commitee Chair and Members – to build agreement on the UDS
• Rou3ne delibera3ons (3mely data submission and data quality are predominantly ICT-‐enabled
Contextual Factors that Contributed to Delibera3on Effec3veness
• Highly effec3ve and highly collabora3ve virtual socio-‐technical eco-‐sytem has evolved itera3vely in large part because …
• Leaders in the Alzheimers research community (from the ADCs and NIA) with considerable source credibility (trusted and respected, necessary exper3se) who were commited to the success of the network and a transcendent purpose – i.e., addressing Alzheimers disease
• NACC has provided the infrastructure for effec3ve and efficient delibera3ons within this virtual STS and serves the func3ons of a referent organiza3on (Trist, 1983) – coordina3on, apprecia3on of emergent issues, and infrastructure support
• Dense network of rela3onships among par3cipants in the network has also strengthened collabora3on
CALTECH (California Ins;tute of Technology) &
D.A.R.P.A. (Defense Advanced Research Projects Agency)
‘ORCHID’ PROGRAM of Fundamental Research -‐ Op;cal Radia;on Cooling & Hea;ng in Integrated Devices
Theore3cal & Experimental “Opto-‐Mechanics” Research within a Virtual Organiza3on -‐2 project phases, June 2010-‐12 & 2012-‐14:
VOSS study of Phase One-‐-‐Exploratory
-‐Tightly-‐Linked Collabora;on for Design of Experiments & Device Fabrica;on among Laboratories
using 3 Technology plajorms in USA & Europe
-‐ At-‐distance Theore;cal scien;fic Study & Input to work of the Experimental groups
-‐ Major CHALLENGE: Crea3ve Research Design & Knowledge Genera3on
in a complex Virtual se`ng
-‐2 project phases, June 2010-‐12 & 2012-‐14:
VOSS study of Phase One-‐-‐Exploratory
-‐Tightly-‐Linked Collabora;on for Design of Experiments & Device Fabrica;on among Laboratories
using 3 Technology plajorms in USA & Europe
-‐ At-‐distance Theore;cal scien;fic Study & Input to work of the Experimental groups
-‐ Major CHALLENGE: Crea3ve Research Design & Knowledge Genera3on
in a complex Virtual se`ng
Orchid Project Network Core Experimental Groups at CALTECH Physics & Applied Physics Depts.
Quantum Op;cs Group Physics -‐ Caltech
Micro & Nano-‐Photonics Research Group
Applied Physics -‐ Caltech
Meso-‐Op;cs Research Group Applied Physics -‐ Caltech
Quantum Op;cs Group Physics -‐ Caltech
Micro & Nano-‐Photonics Research Group
Applied Physics -‐ Caltech
Meso-‐Op;cs Research Group Applied Physics -‐ Caltech
Photonics & Quantum Measurement Laboratory Federal Inst.of Technology Lausanne, Switzerland
Quantum Op;cs & Nanophysics Group Univ. of Vienna
-‐ Austria
Orchid Project Network Data-‐Sharing & Collabora3on between Caltech & EUROPEAN Laboratories
Quantum Op;cs Group Physics -‐ Caltech
Micro & Nano-‐Photonics Research Group
Applied Physics -‐ Caltech
Meso-‐Op;cs Research Group Applied Physics -‐ Caltech
Photonics & Quantum Measurement Laboratory Federal Inst.of Technology Lausanne, Switzerland
Quantum Op;cs & Nanophysics Group Univ. of Vienna
-‐ Austria
Orchid Project Network THEORETICAL Physicists Study & Input to the work of the ‘Experimentalists’
Theore;cal Physics -‐
McGill Univ. Canada
Op;cal Sciences & Physics -‐
Univ.Arizona
Quantum Op;cs Group Physics -‐ Caltech
Micro & Nano-‐Photonics Research Group
Applied Physics -‐ Caltech
Meso-‐Op;cs Research Group Applied Physics -‐ Caltech
Photonics & Quantum Measurement Laboratory Federal Inst.of Technology Lausanne, Switzerland
Quantum Op;cs & Nanophysics Group Univ. of Vienna
-‐ Austria
Theore;cal Physics -‐
McGill Univ. Canada
Op;cal Sciences & Physics -‐
Univ.Arizona
Orchid Project Network European Graduate Students “Embedded”in Caltech Laboratories
VOSS -‐ Preliminary Finding -‐ Orchid Project -‐ ‘Embedded Researcher(s)’ provide temporary F2F ‘presence’ + ‘transla;on’ assis;ng at-‐distance collaborators
to resolve high ambiguity delibera;ons
-‐NOW, ’Habit’ and sufficient Shared Understandings are established for Skype-‐based collabora;on across groups without ‘embedded’ researcher(s)
-‐Another coincidental forum for F2F delibera;ons -‐ frequent Interna;onal Conferences (>5 per year)
-‐Result: Dynamic Complementary F2F + Technology-‐mediated Interac3ons in ‘hybrid’ Virtual SeHng
(Note: Dixon, K. & Panteli, N. (2010). From Virtual Teams to Virtuality in Teams, Human Rela3ons, 63, 1177-‐1197.)
-‐ ‘Embedded Researcher(s)’ provide temporary F2F ‘presence’ + ‘transla;on’ assis;ng at-‐distance collaborators
to resolve high ambiguity delibera;ons
-‐NOW, ’Habit’ and sufficient Shared Understandings are established for Skype-‐based collabora;on across groups without ‘embedded’ researcher(s)
-‐Another coincidental forum for F2F delibera;ons -‐ frequent Interna;onal Conferences (>5 per year)
-‐Result: Dynamic Complementary F2F + Technology-‐mediated Interac3ons in ‘hybrid’ Virtual SeHng
(Note: Dixon, K. & Panteli, N. (2010). From Virtual Teams to Virtuality in Teams, Human Rela3ons, 63, 1177-‐1197.)