COHERENT: Collaborative Holographic Environments for Networked Tasks

Details
Funded by: EU Sixth Framework Program	Reference: IST-FP6-510166
Start: 2004-01-01	Duration: 39 months
Partners
Coordinator: Holografika	Hungary
Contractor: CRS4	Italy
Contractor: C-S	France
Contractor: University of Bonn	Germany
Contractor: Istituto Superiore di Sanita	Italy
Contractor: PSA Peugeot Citroen	France

Abstract

Advances in networked audiovisual communication facilitate the emergence of computer-supported collaborative work (CSCW). In the COHERENT project, six leading European organisations provide complementary competencies to create a new networked holographic audio-visual platform to support real-time collaborative 3D interaction between geographically distributed teams. The display component will be based on innovative holographic techniques that can present, at natural human interaction scale, realistic animated 3D images to an unlimited number of freely moving simultaneous viewers. The design of the basic networked audiovisual components will be driven by two innovative demanding applications - a collaborative medical visualisation system and a collaborative design review system for the automotive industry - that will constitute by themselves an advancement of the state of the art in their specific domains. Both applications will provide intuitive access and interaction with shared 3D models through a sensory rich 3D user interface based on non-intrusive wireless interaction devices and offering 3D audio cues. Research will strongly concentrate on enabling technology for intuitive multi user access and interaction with complex 3D signals and objects. This project proposes to build a working high-resolution display in the one metre size range that, thanks to its human scale work area, will be ideally suited for multi-user collaborative working in true 3D. The challenge of providing the large visualisation data flow needed to drive such a device will be met using a cost-effective parallel solution based on commercial-off-the-shelf graphics and computing technology. Using GEANT, the pan-European Gigabit Research Network, the project will conduct distributed testing and validation of the system concepts for the two representative application scenarios. The research will be conducted in a 30-month schedule, to guarantee evaluation and demonstration of tangible results.

Objectives

Rapidly evolving advances in networked audiovisual communication technology are facilitating the emergence of computer-supported collaborative work (CSCW) systems. These systems are striving to seamlessly support collaboration between geographically distant teams for the purpose of achieving higher levels of participation, productivity, and creativity. They therefore address a major societal and economic challenge. Since visualisation is one of the most natural and intuitive ways to exchange information between humans, it has become the principal medium used in co-operative and multi-user situations. At the present time, however, state of the art collaborative real-time audiovisual systems typically rely on essentially 2D environments (traditional flat screens) to share information. For many professional applications, however, the main goal is to share the physical 3D object of common interest. These applications typically include clinical discussions among teams of medical specialists, multi-disciplinary scientific debate, design reviews between OEM's and suppliers using computer aided design (CAD), where the objects may be anatomical, molecular and product models respectively. Since these are almost exclusively very complex 3D objects, providing collaborative environments able to process, transmit and display 3D data in ways that match human perceptual abilities is therefore of primary importance and would represent a significant technology breakthrough. However, at present the only computer displays able to provide all the depth cues processed by the human brain to reconstruct a three-dimensional scene are unfortunately limited to single user configurations. Quite ironically, these limitations have led to networked solutions that facilitate remote collaboration only at the expense of the isolation of each participant from their local physical environment. In the COHERENT project, six leading European organisations in their respective fields provide complementary competencies to create a new networked holographic audio-visual platform striving to seamlessly support real-time collaborative 3D interaction between geographically distributed teams. The display component will be based on innovative holographic techniques that can present, at natural human interaction scale, realistic animated 3D images to an unlimited number of freely moving simultaneous viewers. The design of the basic networked audiovisual components will be driven by two innovative demanding applications - a collaborative medical visualisation system and a collaborative design review system for the automotive industry - that will constitute by themselves an advancement of the state of the art in their specific domains. Both applications will provide intuitive access and interaction with shared 3D models through a sensory rich 3D user interface based on non-intrusive wireless interaction and offering 3D audio cues. Research will strongly concentrate on enabling technology for intuitive multi user access and interaction with complex 3D signals and objects. The technical feasibility of the proposed holographic display solution has been recently demonstrated with the development of a "small scale" proof-of-concept, using white light based, 24 bit true colour, holographic 3D display. This project proposes to build on this earlier success to produce a working high-resolution display in the one metre size range that, thanks to its human scale work area, will be ideally suited for multi-user collaborative working in true 3D. The challenge of providing the large visualisation data flow needed to drive such a device will be met using a cost-effective parallel solution based on commercial-off-the-shelf graphics and computing technology. The driving applications have been chosen in two important sectors where collaborative 3D technology and networked audiovisual communication have a clear potential impact and provide a sizeable market for the future exploitation of the project results. Moreover, the need for distant teams to work together for a collaborative goal is becoming increasingly common in many industrial and social situations. Therefore, the best practice and methods opened-up by this project will have implications in other application domains. In particular, they will concern high potential, industry-driven domains such as next generation 3D-TV, electronic cinema, virtual and tele-presence and future mixed-reality-based communication services. The consortium has centred the project workplan around continuous and detailed end-user involvement in the research, development, evaluation, and validation activities. The end-users will also play an instrumental role in reaching their larger community as part of the dissemination and exploitation strategy. The research will be conducted against an ambitious, but achievable, 30-month schedule, to guarantee early delivery, evaluation, and demonstration of tangible results.

Publications

	[1] Gavin Brelstaff, Marco Agus, Enrico Gobbetti, and Gianluigi Zanetti. Pseudo-holographic device elicits rapid depth cues despite random-dot surface masking. In Perception, ECVP 2007 Abstract Supplement. Volume 36. Pages 202, 2007.
	[2] Marco Agus, Fabio Bettio, Enrico Gobbetti, and Giovanni Pintore. Medical Visualization with New Generation Spatial 3D Displays. In Eurographics Italian Chapter Conference. Eurographics Association, February 2007.
	[3] Tibor Balogh, Zsuzsa Dobranyi, Tamas Forgacs, Attila Molnar, Laszlo Szloboda, Enrico Gobbetti, Fabio Marton, Fabio Bettio, Giovanni Pintore, Gianluigi Zanetti, Eric Bouvier, and Reinhard Klein. An Interactive Multi-User Holographic Environment. In SIGGRAPH 2006 Emerging Technologies Proceedings. ACM SIGGRAPH. Addison-Wesley, August 2006.
	[4] Tibor Agocs, Tibor Balogh, Tamas Forgacs, Fabio Bettio, Enrico Gobbetti, and Gianluigi Zanetti. A Large Scale Interactive Holographic Display. In Proc. IEEE VR 2006 Workshop on Emerging Display Technologies, 2006. CD ROM Proceedings.
	[5] Fabio Bettio, Francesca Frexia, Andrea Giachetti, Enrico Gobbetti, Giovanni Pintore, Gianluigi Zanetti, Tibor Balogh, Tamas Forgacs, Tibor Agocs, and Eric Bouvier. A Holographic Collaborative Medical Visualization System. In J. D. Westwood, editor, Medicine Meets Virtual Reality 2006, IOS, Amsterdam, The Netherlands, January 2006.
	[6] Tibor Balogh, Tamas Forgacs, Tibor Agocs, Olivier Balet, Eric Bouvier, Fabio Bettio, Enrico Gobbetti, and Gianluigi Zanetti. A Scalable Hardware and Software System for the Holographic Display of Interactive Graphics Applications. In EUROGRAPHICS 2005 Short Papers Proceedings, 2005.
	[7] Tibor Balogh, Tamas Forgacs, Olivier Balet, Eric Bouvier, Fabio Bettio, Enrico Gobbetti, and Gianluigi Zanetti. A Scalable Holographic Display for Interactive Graphics Applications. In Proc. IEEE VR 2005 Workshop on Emerging Display Technologies, 2005. CD ROM Proceedings.

Technical Reports

	[1] Eric Bouvier, Fabio Bettio, Enrico Gobbetti, Nicolas Baudrey, Romain Maurer, David Sanchez, Laurent Philippon, Tamas Forgacs, and Peter Breuer. Communication and Display SDK. Deliverable D9. EU Project COHERENT (IST-FP6-510166), December 2004.
	[2] Tibor Agocs, Tibor Balogh, Eric Bouvier, Enrico Gobbetti, and Gianluigi Zanetti. System Specification. Deliverable D8. EU Project COHERENT (IST-FP6-510166), August 2004.
	[3] Enrico Gobbetti and Gianluigi Zanetti. Initial User Requirement Definition. Deliverable D4. EU Project COHERENT (IST-FP6-510166), April 2004.

Related videos

	Marco Agus, Fabio Bettio, Enrico Gobbetti, and Giovanni Pintore Medical Visualization with New Generation Spatial 3D Displays CRS4 Video n. 138 - Date: February 2007 Eurographics Italian Chapter Conference. Eurographics Association, February 2007.
	Tibor Balogh, Zsuzsa Dobranyi, Tamas Forgacs, Attila Molnar, Laszlo Szloboda, Enrico Gobbetti, Fabio Marton, Fabio Bettio, Gianni Pintore, Gianluigi Zanetti, Eric Bouvier, and Reinhard Klein Live from SIGGRAPH 2006:An Interactive Multi-User Holographic Environment CRS4 Video n. 137 - Date: September 2006 In SIGGRAPH 2006 Emerging Technologies Proceedings - Boston Convention and Exhibition Center Boston,MA,USA,30 July - 3 August 2006
	Tibor Balogh, Zsuzsa Dobranyi, Tamas Forgacs, Attila Molnar, Laszlo Szloboda, Enrico Gobbetti, Fabio Marton, Fabio Bettio, Gianni Pintore, Gianluigi Zanetti, Eric Bouvier, and Reinhard Klein An Interactive Multi-User Holographic Environment CRS4 Video n. 135 - Date: August 2006 In SIGGRAPH 2006 Emerging Technologies Proceedings. ACM SIGGRAPH. Addison-Wesley, August 2006.
	Tibor Agocs, Tibor Balogh, Tamas Forgacs, Fabio Bettio, Enrico Gobbetti, and Gianluigi Zanetti A Large Scale Interactive Holographic Display CRS4 Video n. 134 - Date: 2006 In Proc. IEEE VR 2006 Workshop on Emerging Display Technologies, 2006. CD ROM Proceedings.
	Tibor Balogh, Tamas Forgacs, Eric Bouvier, Olivier Balet, Fabio Bettio, Enrico Gobbetti, and Gianluigi Zanetti A Scalable Holographic Display for Interactive Graphics Applications CRS4 Video n. 126 - Date: 02/2005 Presented at IEEE VR 2005 Workshop on Emerging Displays, conference held in Bonn, Germany, April 2005.