Marcos SerranoAssociate Professor (HDR) in Human-Computer Interaction

ELIPSE group,IRIT Lab, University of ToulouseFRANCE

Tel.: (00 33) + (0)5 61 55 74 05E Mail: marcos.serrano@irit.frWebsite: www.marcosserrano.net

Born 05.08.1981 in Madrid, Spain

ResearchSince 2018 Associate Professor (HDR), ELIPSE group, IRIT Lab,

University of Toulouse, France.

Since 2013 Assistant Professor, ELIPSE group, IRIT Lab, Universityof Toulouse, France.

2012-2013 Post-Doctoral Research Fellow at HCI Lab (with PourangIrani), University of Manitoba, Winnipeg, Canada.

2011-2012 Post-Doctoral Research Fellow at Telecom ParisTech (withEric Lecolinet and Yves Guiard). Visualization and Ad-vanced Interfaces (VIA) Team, Paris, France.

2007–2010 PhD, University of Grenoble, Engineering Human-ComputerInteraction (EHCI) research group, Grenoble Laboratoryof Informatics (LIG). Title : Input Multimodal In-teraction : Design and Prototyping. Supervisor:Laurence Nigay.

2006-2009 Workpackage leader in european project OpenInterface,dedicated to multimodal interaction (FP6-35182, www.oi-project.org).

PhD supervisionOngoing:

Florent Cabric Phygital interaction techniques for explorating complexdata spaces. Start: September 2018. Co-supervised withEmmanuel Dubois and Christophe Hurter.

1

Elio Keddisseh Interaction techniques to support learning in railway sim-ulators. Start: September 2018. Co-supervised with Em-manuel Dubois.

Aziz Niyazov Interaction for pervasive freeform interfaces. Start: Novem-ber 2019. Co-supervised with Loic Barthe.

Kaixing Zhao Multimodal tangible user interfaces for visually impairedpeople. Start: September 2018. Co-supervised with ChristopheJouffrais.

Past:

Gary Perelman Advanced interaction for the fluid manipulation of bigdata. September 2018. Co-supervised with EmmanuelDubois and Celia Picard.

Houssem Saidi Design and evaluation of multidisplay techniques for mul-tidimensional data. October 2018. Co-supervised withEmmanuel Dubois.

Sandra Bardot Design and evaluation of werables interaction techniquesfor visually impaired users. June 2019. Co-supervisedwith Christophe Jouffrais.

Research projects and grantsOngoing:

2018-2022 Project leader of an ANR Jeune Chercheur (JCJC) granton freeform interfaces. Start: October 2018.

2017-2019 PICS research grant by CNRS to support collaborationwith the University of Manitoba (Canada)

2015-2019 Member of ANR Project AP2D

Past:

2014-2018 Workpackage leader in ANR Project Accessimap

2006-2009 Workpackage leader in european project OpenInterface,dedicated to multimodal interaction (FP6-35182, www.oi-project.org).

Research-related activitiesConference Chair General co-chair of MobileHCI 2021.

2

Paper Chair Paper co-chair of MobileHCI 2016.

PC Member CHI 2020, PerDIS 2019, MUM 2019, CHI 2018, MUM2018, CHI 2017, MUM 2017, PerDIS 2017, MUM 2016,MobileHCI 2015, IHM 2015, MobileHCI 2014, IHM 2014

Editor Gest editor in special Issue on advanced interactions forwearable devices, MTI Journal, MDPI. 2019.

Invited expert Natural Sciences and Engineering Research Council ofCanada (NSERC, Canada); Agenzia Nazionale per la Va-lutazione del sistema Universitario e della Ricerca (AN-VUR , Italy).

Committee Poster chair of MobileHCI 2015; Workshop chair of IHM’14and IHM’15

Reviewer CHI, UIST, ToCHI journal, IJHCI, Display journal, Mo-bileHCI, ICMI, DIS, IWC, Interact, TEI, ITS, EICS, IUI,AVI, IHM, JMUI

Awards Special recognition for reviewing: CHI 2016, UIST 2015

Organizations Member of AFIHM commitee (2013-2017), Member ofResearch Committee of IUT of Tarbes (2016-present),Member of IUT Research Council of Toulouse (2016-present)

SeminarsJuly 2018 Bristol Interaction Group, Bristol, UK.

February 2018 IIHM group, LIG, Grenoble.

April 2016 IIHM group, LIG, Grenoble.

May 2014 Forum sur l’Interaction Tactile 2014 (FITG), Lille.

May 2010 Forum sur l’Interaction Tactile 2010 (FITG), Lille.

May 2008 Rencontres Jeunes Chercheurs en Interaction Homme-Machine (RJC-IHM’08), Cap d’Agde, France.

July 2007 Workshop eNTERFACE’07 for young european researchers(4 weeks), Istanbul, Turkey.

July 2006 Workshop eNTERFACE’06 for young european researchers(4 weeks), Dubrovnik, Croatia.

3

OrganizationJune 2015 Organization of Junior HCI Researchers Meeting (Ren-

contres Jeunes Chercheurs en IHM), Ile d’Oleron, France.

June 2009 Organization of digital art festival Play In, interactiveinstallations in the streets of Grenoble, France.

November 2008 Responsable of LIG demonstration during Science Day(Fête de la Science), Grenoble, France.

July 2008 Organization of a multimodal design workshop during in-ternational conference DSV-IS’08, Kingston, Canada.

Publications

Habilitation a Diriger des Recherches (HDR)Title: Ubiquitous interaction: mobile, multi-display and freeform interfaces.Date: November 2018.University of Toulouse 3, Toulouse, France.Jury: Caroline Appert (President), Michel Beaudouin-Lafon (reviewer), StephenBrewster (reviewer), Andy Cockburn (reviewer), Antonio Kruger (jury mem-ber) and Emmanuel Dubois (warrantor)

PhDTitle: Multimodal input interaction: design and prototyping.Date: June 2010.University of Grenoble.Supervisor: Laurence Nigay

International refereed conference papersFlorine Simon, Anne Roudaut, Pourang Irani, and Marcos Serrano. Findinginformation on non-rectangular interfaces. In Proceedings of the 2019 CHIConference on Human Factors in Computing Systems, CHI ’19, pages 102:1–102:8, New York, NY, USA, 2019. ACM.

Houssem Saidi, Marcos Serrano, Pourang Irani, Christophe Hurter, and Em-manuel Dubois. On-body tangible interaction. In Proceedings of Interact2019, Interact’19. Springer, 2019.

Gary Perelman, Marcos Serrano, Christophe Bortolaso, Celia Picard,Mustapha Derras, and Emmanuel Dubois. Combining tablets with smart-

4

phones for data analytics. In Proceedings of Interact 2019, Interact’19.Springer, 2019.

Sandra Bardot, Marcos Serrano, Simon Perrault, Zhao Shengdong, andChristophe Jouffrais. Investigating feedback for two-handed exploration ofdigital maps without vision. In Proceedings of Interact 2019, Interact’19.Springer, 2019.

Florent Cabric, Emmanuel Dubois, Pourang Irani, and Marcos Serrano.Touchglass: Raycasting from a glass surface to point at physical objects inpublic exhibit. In Proceedings of Interact 2019, Interact’19. Springer, 2019.

Mathieu Pecchioli, Emmanuel Dubois, Pourang Irani, and Marcos Serrano.Investigating screen reachability on an articulated dual-display smartphone.In Proceedings of Interact 2019, Interact’19. Springer, 2019.

Emmanuel Dubois, Marcos Serrano, and Mathieu Raynal. Rolling-menu:Rapid command selection in toolbars using roll gestures with a multi-dofmouse. In Proceedings of the 2018 CHI Conference on Human Factors inComputing Systems, CHI ’18, pages 367:1–367:12, New York, NY, USA, 2018.ACM.

Marcos Serrano, Anne Roudaut, and Pourang Irani. Visual composition ofgraphical elements on non-rectangular displays. In Proceedings of the 2017CHI Conference on Human Factors in Computing Systems, CHI ’17, pages4405–4416, New York, NY, USA, 2017. ACM.

Houssem Saidi, Marcos Serrano, Pourang Irani, and Emmanuel Dubois.Tdome: A touch-enabled 6dof interactive device for multi-display environ-ments. In Proceedings of the 2017 CHI Conference on Human Factors inComputing Systems, CHI ’17, pages 5892–5904, New York, NY, USA, 2017.ACM.

Sandra Bardot, Marcos Serrano, Bernard Oriola, and Christophe Jouffrais.Identifying how visually impaired people explore raised-line diagrams to im-prove the design of touch interfaces. In Proceedings of the 2017 CHI Con-ference on Human Factors in Computing Systems, CHI ’17, pages 550–555,New York, NY, USA, 2017. ACM.

Marcos Serrano, Anne Roudaut, and Pourang Irani. Investigating text legi-bility on non-rectangular displays. In Proceedings of the 34rd Annual ACMConference on Human Factors in Computing Systems, CHI ’16, pages 498–508, New York, NY, USA, 2016. ACM.

Julie Ducasse, Marc Macé, Marcos Serrano, and Christophe Jouffrais. Tangi-ble reels: Construction and exploration of tangible maps by visually impaired

5

users. In Proceedings of the 34rd Annual ACM Conference on Human Fac-tors in Computing Systems, CHI ’16, pages 2186–2197, New York, NY, USA,2016. ACM.

Gary Perelman, Marcos Serrano, Mathieu Raynal, Celia Picard, MustaphaDerras, and Emmanuel Dubois. Deco: A design space for device composition.In Proceedings of the ACM SIGCHI Conference on Designing InteractiveSystems (DIS), DIS ’16, pages 435–446, New York, NY, USA, 2016. ACM.

Houssem Saidi, Marcos Serrano, and Emmanuel Dubois. Investigating theeffects of splitting detailed views in overview+detail interfaces. In Proceedingsof the 18th International Conference on Human-Computer Interaction withMobile Devices and Services, MobileHCI ’16, pages 180–184, New York, NY,USA, 2016. ACM.

Sandra Bardot, Marcos Serrano, and Christophe Jouffrais. From tactile tovirtual: Using a smartwatch to improve spatial map exploration for visu-ally impaired users. In Proceedings of the 18th International Conference onHuman-Computer Interaction with Mobile Devices and Services, MobileHCI’16, pages 100–111, New York, NY, USA, 2016. ACM.

Marcos Serrano, Barrett Ens, Xing-Dong Yang, and Pourang Irani. Gluey:Developing a head-worn display interface to unify the interaction experiencein distributed display environments. In Proceedings of the 17th InternationalConference on Human-Computer Interaction with Mobile Devices and Ser-vices, MobileHCI ’15, pages 161–171, New York, NY, USA, 2015. ACM.

Gary Perelman, Marcos Serrano, Mathieu Raynal, Celia Picard, MustaphaDerras, and Emmanuel Dubois. The roly-poly mouse: Designing a rollinginput device unifying 2d and 3d interaction. In Proceedings of the 33rd AnnualACM Conference on Human Factors in Computing Systems, CHI ’15, pages327–336, New York, NY, USA, 2015. ACM.

Marcos Serrano, Dale Hildebrandt, Sriram Subramanian, and Pourang Irani.Identifying suitable projection parameters and display configurations for mo-bile true-3d displays. In Proceedings of the 16th International Conference onHuman-computer Interaction with Mobile Devices & Services, Mobile-HCI ’14, pages 135–143, New York, NY, USA, 2014. ACM.

Louis-Pierre Bergé, Marcos Serrano, Gary Perelman, and Emmanuel Dubois.Exploring smartphone-based interaction with overview+detail interfaces on3d public displays. In Proceedings of the 16th International Conference onHuman-computer Interaction with Mobile Devices & Services, Mobile-HCI ’14, pages 125–134, New York, NY, USA, 2014. ACM.

6

Marcos Serrano, Barrett M. Ens, and Pourang P. Irani. Exploring the useof hand-to-face input for interacting with head-worn displays. In Proceed-ings of the 32Nd Annual ACM Conference on Human Factors in ComputingSystems, CHI ’14, pages 3181–3190, New York, NY, USA, 2014. ACM.

Marcos Serrano, Eric Lecolinet, and Yves Guiard. Bezel-tap gestures: Quickactivation of commands from sleep mode on tablets. In CHI ’13 : Proceed-ings of the 2013 international conference on Human factors in Computingsystems, 2013.

Sarah Fdili Alaoui, Baptiste Caramiaux, Marcos Serrano, and FrédéricBevilacqua. Movement qualities as interaction modality. In Proceedings ofthe Designing Interactive Systems Conference, DIS ’12, pages 761–769, NewYork, NY, USA, 2012. ACM.

Marcos Serrano and Laurence Nigay. Temporal aspects of care-based mul-timodal fusion: from a fusion mechanism to composition components andwoz components. In ICMI-MLMI ’09: Proceedings of the 2009 internationalconference on Multimodal interfaces, pages 177–184, New York, NY, USA,2009. ACM.

Marcos Serrano, David Juras, and Laurence Nigay. A three-dimensional char-acterization space of software components for rapidly developing multimodalinterfaces. In IMCI ’08: Proceedings of the 10th international conference onMultimodal interfaces, pages 149–156, New York, NY, USA, 2008. ACM.

Marcos Serrano, Laurence Nigay, Rachel Demumieux, Jérôme Descos, andPatrick Losquin. Multimodal interaction on mobile phones: development andevaluation using acicare. In MobileHCI ’06: Proceedings of the 8th conferenceon Human-computer interaction with mobile devices and services, pages 129–136, New York, NY, USA, 2006. ACM.

International refereed journal articlesAndrés Lucero and Marcos Serrano. Towards proxemic mobile collocatedinteractions. Int. J. Mob. Hum. Comput. Interact., 9(4):15–24, October 2017.

Marcos Serrano and Laurence Nigay. A wizard of oz component-based ap-proach for rapidly prototyping and testing input multimodal interfaces. Jour-nal on Multimodal User Interfaces, Springer Publ., 3(3):215–225, 2010.

Alexandre Benoît, Laurent Bonnaud, Alice Caplier, Yannie Damousis,Frédéric Jourde, Jean-Yves L. Lawson, Laurence Nigay, Marcos Serrano, andDimitrios Tzoradas. Multimodal signal processing and interaction for a driv-ing simulator : component-based architecture. Journal on Multimodal UserInterface, Springer Publ., 1(1):49–58, 2007.

7

Book chaptersMarc Macé, Sandra Bardot, Anke Brock, Emeline Brule, Julie Ducasse,Stéphanie Giraud, Bernard Oriola, Marcos Serrano, Grégoire Denis, andChristophe Jouffrais. Les nouvelles technologies pour les apprentissages spa-tiaux chez les personnes déficientes visuelles. In Dannyelle Valente andEdouard Gentaz, editors, Développement psychologique chez les enfants etadolescents déficients visuels. Enjeux théoriques et appliqués, chapter 8, page(support électronique). Approche Neuropsychologique des Apprentissageschez l’Enfant (ANAE), https://www.anae-revue.com, 2018.

Laurence Nigay, Jullien Bouchet, David Juras, Benoit Mansoux, Michael Or-tega, Marcos Serrano, and Lionel Lawson. Software Engineering for Multi-modal Interactive Systems, chapter 9, pages 201–218. D. Tzovaras (ed.), Lec-ture Notes in Electrical Engineering, Springer-Verlag, Berlin, 2007. Springer,2008.

C Coutrix, D Cunin, M Okura, and M Serrano. De part et d’autre del’interface : les continuites esthetiques et scientifiques. Editions l’Harmattan,2012.

National refereed conference papersHoussem Saidi, Marcos Serrano, Pourang Irani, Christophe Hurter, and Em-manuel Dubois. Utilisation du corps comme support à l’interaction à distanceavec un dispositif à multiple degrés de libertés. In AFIHM, editor, 30eme con-férence francophone sur l’interaction homme-machine, Articles Scientifiques,page 11p, Brest, France, October 2018.

Gary Perelman, Marcos Serrano, Célia Picard, Christophe Bortolaso, Em-manuel Dubois, and Mustapha Derras. Combinaison de dispositifs tactilespour interagir avec un tableur sur tablette. In AFIHM, editor, 30eme con-férence francophone sur l’interaction homme-machine, Articles Scientifiques,pages 20–29, Brest, France, October 2018.

Emmanuel Dubois, Mathieu Raynal, and Marcos Serrano. Taking advan-tage of rolling gestures on a multidimensionnal mouse for interacting with amenubar. In Proceedings of the 29th Conference on L’Interaction Homme-Machine, IHM ’17, pages 83–92, New York, NY, USA, 2018. ACM.

Gary Perelman, Marcos Serrano, Célia Picard, Mustapha Derras, and Em-manuel Dubois. Multi-device interaction for spreadsheet on tablets. In Pro-ceedings of the 29th Conference on L’Interaction Homme-Machine, IHM ’17,pages 105–115, New York, NY, USA, 2018. ACM.

Emeline Brulé, Gilles Bailly, Marcos Serrano, Marc Teyssier, and Samuel

8

Huron. Investigating the design space of smartwatches combining physi-cal rotary inputs. In Proceedings of the 29th Conference on L’InteractionHomme-Machine, IHM ’17, pages 13–20, New York, NY, USA, 2018. ACM.

Gary Perelman, Marcos Serrano, Mathieu Raynal, Emmanuel Dubois, CéliaPicard, and Mustapha Derras. Designing an input device to interact withmultidimensional data : Disco. In Proceedings of the 26th Conference onL’Interaction Homme-Machine, IHM ’14, pages 91–100, New York, NY, USA,2014. ACM.

Marcos Serrano and Laurence Nigay. Openwizard: une approche pour lacréation et l’évaluation rapide de prototypes multimodaux. In IHM ’09:Proceedings of the 21st International Conference on Association Francophoned’Interaction Homme-Machine, pages 101–109, New York, NY, USA, 2009.ACM.

Marcos Serrano, David Juras, Michael Ortega, and Laurence Nigay. OIDE :un outil pour la conception et le développement d’interfaces multimodales. InMay 28-30, 2008, pages 91–92, Saint-Malo, France, May 28-30, 2008 4èmesjournées Francophones Mobilité et Ubiquité (UbiMob’08).

DemonstrationsS. Fdili Alaoui, B. Caramiaux, and M. Serrano, “From dance to touch: move-ment qualities for interaction design,” in Proceedings of the 2011 annual con-ference extended abstracts on Human factors in computing systems, CHI EA’11, (New York, NY, USA), pp. 1465–1470, ACM, 2011.

M. Serrano, L. Nigay, J.-Y. L. Lawson, A. Ramsay, R. Murray-Smith, andS. Denef, “The openinterface framework: a tool for multimodal interaction.,”in CHI ’08: CHI ’08 extended abstracts on Human factors in computingsystems, (New York, NY, USA), pp. 3501–3506, ACM, 2008.

D. Juras, L. Nigay, M. Ortega, and M. Serrano, “Multimodal slideshow:demonstration of the openinterface interaction development environment,”in IMCI ’08: Proceedings of the 10th international conference on Multimodalinterfaces, (New York, NY, USA), pp. 193–194, ACM, 2008.

P. Gray, A. Ramsay, and M. Serrano, “A demonstration of the openinterfaceinteraction development environment,” in Adjunct Proceedings of UIST’07,pp. 39–40, ACM Press, 2007.

ReportsA. Benoît, L. Bonnaud, A. Caplier, Y. Damousis, F. Jourde, J.-Y. L. Lawson,L. Nigay, M. Serrano, and D. Tzoradas, “Multimodal signal processing and

9

interaction for a driving simulator : component-based architecture,” Journalon Multimodal User Interface, Springer Publ., vol. 1, no. 1, pp. 49–58, 2007.

IxDesign In June 2010, after my PhD defense, I founded We Inter-active People, an interaction design agency offering de-sign and prototyping services for interactive media cre-ation. I leaded this project for one year (June 2010 -Sept. 2011), before returning to research activities atTelecom ParisTech. The agency still runs and is com-posed of several interaction designers and developers.

June 2011 Presentation of "AugmentedWindow", tactile augmentedreality-based interactionPublic interactive installation, at Futur en Seine 2011Georges Pompidou Center, Paris.

June 2011 Presentation of "Discontrol Party", large-group tracking-based interactionPublic interactive installation, at Futur en Seine 2011La Gaieté Lyrique, Paris.

2010-2011 Design of novel tactile public displays for HiLabs (INRIAresearch start-up).

June 2010 Creation of We Interactive People agency.

Pedagogy My teaching activities concern design methods and pro-gramming techniques for human-computer interaction. Itis mainly delivered to computer-science and interactiondesign students.

Keywords Java, C++, JavaSwing, Web (HTML, CSS, JS), PHP,Databases, Graphical Interfaces

Since 2017 Mobile Interaction (20h)HCI Master, ENAC, Toulouse, France.

Since 2013 Assistant Professor at Multimedia Department, TechnicalInstitute (IUT) of Tarbes, France.

10

Jan-Feb 2012 Human-Computer Interaction (9h)ENSTA ParisTech, Paris, France.

Nov-Dec 2011 C++ Programming Workshop (6h)Telecom ParisTech, Paris, France.

Oct-Nov 2011 Processing Programming (8h)Mains d’Oeuvres, Paris, France.

Sept-Dec 2011 Java Programming Workshop (9h)Telecom ParisTech, Paris, France.

Sept-Dec 2011 Java Programming (6h)Telecom ParisTech, Paris, France.

Feb-June 2011 Supervision of design students project (24h), gestural in-teraction in urban contextStrate College, Paris, France.

Feb-June 2011 Processing Programming (8h)Mains d’Oeuvres, Paris, France.

Feb-June 2011 Graphical Interface Programming (50h)IUT d’Orsay, Paris South University, France.

Oct-Dec 2010 Workshop (30h) on interaction design and prototypingEnsad, Paris, France.

Education2007–2010 Phd in Computer Science, Engineering Human-Computer

Interaction (EHCI) research group, Grenoble Laboratoryof Informatics (LIG), University of Grenoble.Input Multimodal Interaction : Design and Pro-totyping| Supervisor: Laurence Nigay. June 30, 2010.

2009-2011 Research student at the Superior National School for Ap-plied Arts of Paris (Ensad), Relational Devices and In-teractive Installations program (Drii), Paris, France.

2005-2006 Master 2 Research Intelligence, Interaction, InformationUniversity Joseph Fourier, Grenoble, France

2003-2005 Master 2 at Superior National School of Applied Mathe-matics and Computer Science of Grenoble (ENSIMAG),Specialty Images and Virtual Reality, Double diploma eu-

11

ropean exchange program, Grenoble, France.ENSIMAG engineer diploma, delivered in June 2005.

2003-2004 Master 2 at ENSIMAG, Specialty Networks and distributedsystems, Grenoble, France.European exchange program ERASMUS.

1999-2005 Computer-science engineer, Polytechnic University ofMadrid (UPM), Madrid, Spain.Engineer diploma delivered in June 2005.

June 1999 Scientific school diploma, Good distinction.French College of Madrid, Spain.

Toulouse, France, August 2019

12