1. Integrated Media Systems CenterViterbi School of Engineering University of Southern California PROJECT MANAGEMENT IN THEUNIVERSITY ENVIRONMENT:THE IMSC EXPERIENCE Isaac Maya, Ph.D., P.E Director, Industry and Technology Transfer Programs 213-740-2592, imaya@imsc.usc.edu1st Annual NASA PROJECT MANAGEMENT CONFERENCE:Meeting the Project Management ChallengeMarch 30-31, 2004National Science Foundation Engineering Research Center

2. Integrated Media Systems CenterNSF Engineering Research Center:a partnership in pursuit of research andinnovation in multimedia and immersivetechnologies and their applications28 Investigators and 260 students in NSFpartnership with:National Science FoundationUniversity of Southern California Viterbi School of Engineering Ranked 8th in US, $115M/yr in grant funding Annenberg Center for CommunicationCommercial PartnersDowntown LA Computer Hardware and Software Aircraft, Aerospace, Defense Petroleum, Oil, Gas Telecommunications EntertainmentOther Government Agencies DARPA, NASA, JPL, NIMA, ONR, U.S. Army N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e IMSC Headquarters search Center 3. Education 209 students graduated with IMSC providing funding, classes, and research aspects of their education experience 112 with PhD, 82 with MS, and 15 with BS IMSC created six academic programs3 MS programs with 454 students enrolled (152 graduates)2 UG minor programs with 76 students enrolled (121 graduates)BSEE (IMS) enrollment starts F03 IMSC gave UG research fellowships to 44 students Created 23 new courses for IMSC and SoE programsHuman Factors in Integrated Media SystemsIntegrated Media Systems - SAI project courseEngineering Approaches to Music Perception and CognitionIntro to Art and Technology - SoE/FA courseNational Science Foundation Engineering Research Center 4. Faculty and AcademiaIMSC has a high quality array of 28 investigators 14 of whom (~ 50%) weresought specifically because of IMSC working with 166 PhD, 56 MS, and 38 UG students.Investigators come primarily from EE and CS others from Psychology,Industrial and System Eng., School of Cinema/Television, Annenberg Schoolfor Comm., School of Gerontology, Biomedical Engineering, and theInformation Sciences InstituteTwo IMSC investigators have PFF awardsEight IMSC faculty have CAREER awards (2 CAREER awards this year)Alexander Sawchuk elected to Board of Directors of the Optical Society ofAmericaGerard Medioni named a Fellow of Institute of Electrical and ElectronicsEngineersMathieu Desbrun received the 2003 ACM SIGGRPH Significant NewResearcher AwardIMSC faculty published 52 peer-reviewed journal articles and 184 peer-reviewed conference papersarticles actually appearing in print over a 12 month period (2002-2003) 5. StrategicPlanDrivingApplicationResearchProjectsEngineeringandIntegrationBasicResearch 6. Research HighlightsIMSC has produced ground breaking results and fundamentalresearch in: immersive audio multichannel and HRTF approaches - holistic DSP approach streaming servers and multimedia databases distributed and scalable streaming architecture, immersidata analysis and query computer vision computational framework for grouping based on tensor voting, tracking for augmented realities and SFX graphics & animation 3D DSP mesh processing, compression, mesh operations, hair modeling and animation multimodal emotive, 3D interfaces Speech and dialog, vision sensing of body and hands, facial expressions analysis and expressive avatars virtual reality and simulations applications to psychology (ADD diagnosis), and user studiesNational Science Foundation Engineering Research Center 7. Enabling the Vision: Application Research ProjectsImmersiNet EntertainmentProf. Alexander Sawchuk (EE)Prof. Roger Zimmermann (CS)InterAct CommunicationProf. Shri Narayanan (EE)Prof. Isaac Cohen (CS)2020Classroom EducationProf. Cyrus Shahabi (CS)Prof. Chris Kyriakakis (EE)National Science Foundation Engineering Research Center 8. 2020ClassroomThe future of immersive technologiesas applied to learning, encompassing: Software and hardware architecture for distributed learning Investigate innovative methods for student/teacher interaction with the curriculum Dynamic curriculum content, specifically designed for this unique immersive platform Development and assessment of high fidelity presence in learning Our two testbed sites are used to study the requirements for interface design, computational complexity, visual and aural fidelity, network performance, and data acquisition of presence for learning applications National Science Foundation Engineering Research Center 9. InterAct: Communications and CollaborationMedia-rich integration of sensory modes to support human tasksand communication Multimodal interfaces speech synthesis and recognition, vision tracking and interpretation of human behavior, facial gesture analysis and avatar rendering, haptics, ... Tele-immersion Hi-fidelity low-latency robust communication over IP networks, graceful incorporation of PDA or low-BW 3D/4D visualization and modeling of time-varying surfaces, volumes, and imagery Data fusion 3D models and video streams and sensor data Data streaming, synchronization, analysis, and queryNational Science Foundation Engineering Research Center 10. ApplicationsImmersiNet: P2P Streaming Media over IP NetworksA fusion of internet browsing with a theater-likeimmersive experienceImmersed in a college HD Video at up to 45 Mbits/secfootball game 10.2 channel Immersive audio (12 Mbits/sec)Steaming on-demand over the Internet StreamingImmersiveProtocols forDoctors assisting in amedia serversaudio capture error remote procedureand recordersand rendering management SynchronizationBusiness people Recent accomplishments: negotiating like theyare in the same roomBing Theater I2 ConfLive DuetHD video NWS Students visiting anaquarium a thousand miles away 11. IMSC Research ProgramTheory of PerceptualImmersive InformationMultimediaRemote Media ImmersiNET - fusionand Cognitive audio IntegrationNetworks, Immersionof internet andPleasureSpeech Transmission, Software cinemaImmersi-Recognitionand Comm- Architecture forEmotional Expressions data2020Classroom andunication Immersi-Content immersion,User State SensingSynthesis Extraction & Ultra presence authoring, andand Perceptual UserFacialAnalysis Wideband assessmentInterfacesGestureWirelessCustomizedInterAct awareVirtual EnvironmentsAnalysisQuerying & Compressioncomputing andfor Performanceand Rendering multimodalTesting and TrainingAnimation Immersive interactionRobustMediaVisionReal-TimeAnalysisStorage &Digital RetrievalGeometryProcessingNational Science Foundation Engineering Research Center 12. Sensory Interfaces (SI) Robust vision with tensor voting Segmentation, motion, 3D body and hand tracking Speech recognition and synthesis Emotive dialog, children as users, translation Immersive audio Sonic visualization, spatialization, autocalibration Facial gesture analysis and animation Expression signatures (PCA) for emotion and recognition Video fusion with 3D models for surveillance Structure extraction from LiDAR and texture projection Digital geometry processing Compression, surface and topology smoothing, mesh simplification Mixed Reality and Visualization Merging of video, computer graphics, ubiquitous computing National Science Foundation Engineering Research Center 13. SI Research3D models fromuncalibratedimages3D models from laser scanners Automated caricature3D bodymodelingand poseanalysisSurface / volume deformations Model detail enhancementImplicit hapticsExpression analysis from parameterized modelsCR MusclesCR Muscles0 0 0 01 1, 31 1, 32 2 2 23 4, 63 4, 64 5, 74 5, 75 9, 11, 13 5 9, 11, 136 10, 12, 146 10, 12, 147 157 158 8 8 8National Science Foundation Engineering Research Center 14. Mixed Reality and Visualization Augmented Reality for Space Flight (NASA)Develop AR authoring tools for video-based trainingAnthony Majoros, Human Factors group, The Boeing Corp 4D Battlefield Visualization (MURI-ARMY)Develop fusion of video, images, and 3D modelsAvideh Zakhor (Berkeley), Suresh Lodha (UC Santa Cruz),Bill Ribarsky (Georgia Tech), Pramod Varshney (Syracuse) Wide Area AR Tracking (ONR)Novel sensors & fusion for tracking position/orientation National Science Foundation Engineering Research Center 15. Information Management (IM) Distributed streaming media systems P2P storage, indexing, and retrieval (YIMA+) Formal analysis and design of P2P systems using complex systems theory Multidimensional databases and data streams Progressive & approximate (wavelet-based) query evaluation Real-time analysis and query of sensor data streams Spatio-temporal information integration Music (MIDI) stream analysis Semantic information representation Use of dynamic ontologies to represent information about objects in immersispace Use of information semantics for customized experiences National Science Foundation Engineering Research Center 16. spiral array modelMusic InfoRetrieval APPLICATIONSpitch structuresExpressivePerformance time structuresAutomatedtranscriptionStream analysisMusic Information Processing Geospatial Data Integration Peer-to-Peer Discovery National Science Foundation Engineering Research Center 17. Media Communications (MC)CompressionCompression for Speech Recognition and Music ClassificationNew Compression Techniques for Robust and Scalable MediaCommunicationsNetworks, Transmission, Communication - QOSError Concealment Techniques and Channel Modeling forWireless Video CommunicationLoss Concealment in Multi-Channel Streaming AudioRobust Audio and Video Streaming over IP NetworksMultipath streaming over IP NetworksStereoscopic Video Acquisition, Display, Transmission andInteractionUltrawideband WirelessImpulse radio for communication and position tracking National Science Foundation Engineering Research Center 18. Large vocabularycontinuous speechSpeech recognizer recognizer networkedapplicationsClientsWireless networkCompression for Distributed SpeechRecognition Paul G. Allen Wireless Test Facility Conventional SpatialSpatial Domain Concealment Domain Concealmentwith Edge Recovery Stereo video acquisition; Robust wireless video transmission: burst noise: autostereoscopic and loss of 20 macroblocks in the I frameprojection display National Science Foundation Engineering Research Center 19. User-Centered Sciences (UCS)Virtual environments - technology and applicationsstudy, assessment, and rehabilitation of attention, memory,visuospatial, and executive cognitive processesBody and face gesture representation and analysisNeural basis for perceptual and cognitive affect and attentionVirtual classroom for attention & behaviorassessment and rehabilitation 360 video news and sportsNational Science Foundation Engineering Research Center 20. IMSC Industrial Members andKDDIOlympus CollaboratorsJapan EverFocus IIITaiwanBTGUK ITRIMicrosoft WA Eastman KodakNY NCROH IntelORSee California MapNVIS VACollaborations withGovernment Agenciesand Foundations:US ArmyDARPA, ONRNASA, NIMAToyota FoundationNational Science Foundation Engineering Research Center 21. IMSC Industrial Members andCollaborators Los Angeles/San Diego JPL Lord Foundation Silicon ValleyLos Angeles Times Northrop GrummanConceptLabs TMH CorporationF-X Palo Alto LaboratoryHewlett-PackardIBMLockheed Martin Silicon ValleyChevronTexacoHollywood USCIMSCLos Angeles National Science Foundation Engineering Research Center 22. IMSC Corporate Programs Memberships access to IMSC and faculty demos for internal/research use 20-30 per year over life of center Sponsors 3yr project collaboration pre-paid licenses collaboration on common agenda Centers multi-year collaboration in broad areas of activityPWICE (1Q 03)CiSoft -- IT for Petroleum Industry (1Q 04)Defense/HLS (1Q 04) National Science Foundation Engineering Research Center 23. IMSC Research/Technology TransferSuccess StoryHollywood company, Rhythm & Hues using IMSCsoftware for special effectsFeature tracking in image sequencesThree feature films -- X-Men II, Daredevil, Cat N Hat Reduces time for key part of process from minutes to seconds per frameTermed Fastrack by R&H Reduces need for hand- correction by robust tracking Two IMSC graduates hired National Science Foundation Engineering Research Center 24. IMSC Research/Technology TransferSuccess Story -- End-to-EndInvention DisclosurePrototype DevelopmentLicensingProduct DevelopmentCommercial/Strategic PartnerCommercializationDeploymentIMSCGeometrixVisionixHoyaOptical ShopsNational Science Foundation Engineering Research Center 25. Pratt & Whitney Institute for CollaborativeEngineering (PWICE)Demonstrated wireless,audio and videocommunication amongtechnicians in field (KoreanAir) with engineering helpdesk (P&W Hartford), withvisibility and interaction inKorean Air Board RoomExamining Future IssuesMulticast-based Mobility ArchitectureDifferentiated Service for Internet MultimediaRapid Symbol-by-Symbol Transmission AdaptationApplication Specific CompressionMicroserver Data Storage and QueryingNational Science Foundation Engineering Research Center 26. CiSoft -- IT for the Petroleum Industry IMSC expertise in data management, data transmission and visualization will assist ChevronTexaco in developing the oil field of the future Focus on the integration of field automation, reservoir simulation technologies, new and emerging well technologies, and real-time reservoir managementNational Science Foundation Engineering Research Center 27. IMSC PROJECT MANAGEMENTAPPROACH Organizational and Management Evolution Project InitiationIMSC Vision + Faculty Research Interests =>Projects of Mutual Interest Milestones and Schedules Scope Requirements Specifications Monitoring and Control Intellectual Property Protection Lessons LearnedNational Science Foundation Engineering Research Center 28. CENTER MANAGEMENTFUNCTIONAL ORGANIZATION National Science Foundation Engineering Research Center 29. FUNCTIONAL ORGANIZATION:USEFUL FOR MANAGING INDIVIDUALRESEARCH EFFORTSimmersive audiostreaming serversmultimedia databasescomputer visiongraphics & animationmultimodal emotive, 3D interfacesvirtual reality and simulations National Science Foundation Engineering Research Center 30. INTEGRATED MEDIA SYSTEMS REQUIREMULTIDISCIPLINARY RESEARCH INTEGRATIONNSF requires turbulence in research program Bottom-up basic research uncovers new capabilities that drive new applications Top-down application projects create requirements (to meet existing or desired needs) to be solved by researchIndustry-sponsored research projects oftenrequire integration with company projects National Science Foundation Engineering Research Center 31. PROJECT SELECTION DRIVEN BYIMSC VISION, NSF-ERC MISSION, ANDRESEARCHER AND INDUSTRY INTERESTSNational Science Foundation Engineering Research Center 32. INTEGRATION OF RESEARCH INTO PROJECTSNational Science Foundation Engineering Research Center 33. PROJECT MANAGEMENT EVOLVEDTO WEAK-MATRIX ORGANIZATION SensoryComm Info Mgt HF MIENational Science Foundation Engineering Research Center 34. OF THE 9 INFLUENCES TRADITIONALLYAVAILABLE TO PMs, ONLY 3 1/2 REALLYWORK IN UNIVERSITY ENVIRONMENT Authority Assignment Budget -- Research Assistants Promotion Money Penalty Challenge -- Research Expertise -- Recognition Friendship -- CollaborationNational Science Foundation Engineering Research Center 35. PROJECT SCOPE & BUDGETS DEVELOPEDITERATIVELY AMONG DIRECTOR, PMs & TEAMS Project Total Budget Project Integration Budget Research Budget National Science Foundation Engineering Research Center 36. MILESTONES AND SCHEDULESNormally driven by Academic Calendar Fall and Spring Semesters, Summer months Graduate Student hiresNSF-specified annual site-visit reviewIMSC instituted additional major calendar events 2 Scientific Advisory Board meetings Early Fall & Spring semester researcher retreats After-site-visit analysisAdditional 1-hour weekly Center-wide progressdiscussion meetingsSchedule granularity controlled by PM National Science Foundation Engineering Research Center 37. SCOPEREQUIREMENTS:Turbulence &MultidisciplinaryIntegration Analysis5-year MilestonePlanNational Science Foundation Engineering Research Center 38. SCOPE REQUIREMENTS To achieve Project Vision, early meetings used to clarify Vision, Scope and research requirementsWhat are we sure of? (Failure is not an option)What are extensions? (What is go-no go?)What is really hard?What are some expected problems ofintegration with what youre sure of? Uncompromising on Quality of Deliverables World-class research, innovative, new, Risk doing things never done before, so see above questions, identify risks plus back-up plan 5-Year Milestone PlanNational Science Foundation Engineering Research Center 39. SCOPE REQUIREMENTS National Science Foundation Engineering Research Center 40. REQUIREMENTS SPECIFICATIONSRequirements converted into Specifications at theResearcher levelMaintained in Project Specification Document National Science Foundation Engineering Research Center 41. EXAMPLE FLOW CHART AND TASK DESCRIPTIONCollaborativelymaintaineddocument National Science Foundation Engineering Research Center 42. PROJECT MONITORING AND CONTROLWeekly or bi-weekly formal Project Team meetingsAgendas a mustRound table discussionsAssignments/namesFollow-up itemsAttendance requiredPeriodic Technology Demonstrations and White Papers (inaddition to journal publications)Industry visitsMonthly Open House demonstrationsManagement by EmbarrassmentNational Science Foundation Engineering Research Center 43. PROJECT TEAM BUILDINGCo-location is also important in a university setting Brainstorming essential Informal discussions indispensable in integration Concurrent multidisciplinary discussions lead to higher efficiency in overall collaborationIn the absence of co-location, need a focal facilityor lab for technology integration Visible signs of team progress Sense of responsibility and team-nessHuman Factors/User Centered component helpful National Science Foundation Engineering Research Center 44. INTELLECTUAL PROPERTY PROTECTIONExtensive IP discussions, education effort amongboth faculty and students (highly interested)Wide-spread awareness of filing inventiondisclosures prior to publications and publicpresentationsProvisional patent process used to provide timecushion to find technology transfer and licensingopportunitiesFlexible IP terms to interested partiesEarly commercial successes maintain momentumNational Science Foundation Engineering Research Center 45. PM LESSONS LEARNEDMust have Faculty buy-in on projects, brainstormingand iterating on project selection and budgetsMust assign lead PM or designee on each projectMust adapt PM tools and techniques to universityenvironment (e.g., quality, milestones andschedules, requirements/specifications, WBS detaillevel, etc.)Must have regular meetings, communicate progressand reasons to justify hitting those milestonesMust do appropriate risk assessment (after all, this isresearch) and have viable back-up plans National Science Foundation Engineering Research Center 46. National Science FoundationEngineering Research Center programhttp://www.eng.nsf.gov/eec/erc.htmIMSC PARTNERSBTGChevronTexacoConceptLabsEastman KodakIMSC Industry partners..... EverFocusFuji-Xerox Palo AltoHewlett PackardHRLIBMKDDIKorean AirIIIMore information about IMSC IntelITRI/CCLJPLhttp/:imsc.usc.eduLockheed MartinLord FoundationLos Angeles TimesMicrosoftNCRNorthrop Grumman/TRWNVISOlympusPanoram TechnologiesPratt & WhitneyTMHToyota FoundationNational Science Foundation Engineering Research Center