Research Group Perceptual User Interfaces
Fraunhofer Institute of Optronics, System Technologies and Image Exploitation
Deutsche Version
Group description | People | Photo gallery | Publications | External links
Group description
Our goal is to develop alternatives to the classic mouse-keyboard setting that most offices, meeting rooms, and control rooms still use. Users should be able to interact with smart environments naturally, much like they would interact with other people. To achieve this, the environment needs to know which users are present, where they are, and what they are doing. In our case, this information is obtained through visual perception with multiple cameras. As output device, we use a large-scale videowall. In the near future, some of the following will be integrated as well: speech recognition, standard workstations, a digital situation table, tablet-PCs, sound, and synthesized speech. Control rooms for fire brigade, ambulance, and police are our case study since they are one of the most promising applications.
There is an abundance of ways to use the information generated by the room's perceptual components. Let us observe a concrete example. A user's identity can be obtained through face recognition and voice identification. Then, a user model can be used to generate a personal interface, obeying the user's preferences, current tasks, and specialized knowledge. In a crisis response control room, the ambulance dispatcher requires different functionality than the center's director. Through person tracking, such personal interfaces can then be displayed at a position close to the user.
Machine perception gives rise to new and unobtrusive ways of interacting with the room. Instead of using mouse and keyboard, objects on the videowall can be manipulated simply by pointing and directing one's visual attention. When combined with a diversity in hand gestures and voice commands, powerful and unobtrusive interaction becomes possible. Also, we analyse the interaction between the people in the room. Through situation modeling, reasoning, and learning, the system will be able to adapt to the current social situation in the room.
The research group "Perceptual User Interfaces" focuses on machine perception and situation modeling to facilitate innovative user interfaces and smart environments. In the framework of the »Fraunhofer Attract« project on "Visual perception for the man-machine interface - Interaction of humans in and with »smart« rooms", we bring together progressive science and practical applications.
Videos of our ongoing work
» Interaction demonstration (11 MB)
» Technical components (29 MB)
» Visual hull algorithm (5 MB)
» User study videowall interaction (17 MB)
» Interaction accuracy evaluation (21 MB)
People
» Prof. Dr.-Ing. Rainer Stiefelhagen (supervisor)
» Dipl.-Inform. Florian van de Camp (tracking and identification)
» Joris Ijsselmuiden, MSc (situation modeling and human-machine interaction)
» Dipl.-Inform. Alexander Schick (gestures and body pose)
» Dipl.-Inform. Michael Voit (visual focus of attention)
Contact:
Prof. Dr.-Ing. Rainer Stiefelhagen
Phone: +49 721 608-6385
Fax: +49 721 6091-233
rainer.stiefelhagen@iitb.fraunhofer.de
Back to top
Photo gallery
Our smart control room laboratory, containing videowall and cameras
A camera image and its corresponding segmentation and 3D voxel representation
Person tracking and identification
Head pose estimation and gesture recognition
Back to top
Example: interaction with our videowall through Digital situation table and tablet PCs
head pose estimation and gesture recognition in a map application
F.l.t.r.: Michael Voit, Florian van de Camp, Rainer Stiefelhagen, Alexander Schick, Joris Ijsselmuiden
Publications
2009
Schick, A., van de Camp, F., Ijsselmuiden, J., Stiefelhagen, R. (2009) Extending Touch: Towards Interaction with Large-Scale Surfaces,
Proceedings of Interactive Tabletops and Surfaces (ITS), Banff, Canada
pdf
Voit, M., Stiefelhagen, R. (2009) A System for Probabilistic Joint
3D Head Tracking and Pose Estimation in Low-resolution, Multi-view
Environments, Proceedings of the 7th International Conference on Computer
Vision Systems (ICVS) Liège, Belgium
pdf
van de Camp, F., Bernardin, K., Stiefelhagen, R. (2009) Person Tracking in
Camera Networks Using Graph-Based Bayesian Inference, Proceedings of
the Third ACM/IEEE International Conference on Distributed Smart Cameras
(ICDSC), Como, Italy
pdf
Schick, A., Stiefelhagen, R. (2009) Real-Time GPU-Based Voxel Carving with
Systematic Occlusion Handling, Proceedings of the 31st Symposium of the
German Association for Pattern Recognition (DAGM), Jena, Germany
pdf
Ijsselmuiden, J., Stiefelhagen, R., van de Camp, F., Schick, A., Voit, M.
(2009) Towards a Smart Control Room for Crisis Response Using Visual
Perception of Users, Poster at the 7th International Conference on
Information Systems for Crisis Response and Management (ISCRAM, not in
proceedings), Goteborg, Sweden poster
paper
2008
Stiefelhagen, R., Bernardin, K., Ekenel, H. K., Voit, M. (2008) Tracking
Identities and Attention in Smart Environments - Contributions and Progress in
the CHIL Project, Proceedings of the IEEE International Conference on Face
and Gesture Recognition (FG) Amsterdam, Netherlands pdf
Voit, M., Stiefelhagen, R. (2008) Deducing the Visual Focus of Attention from
Head Pose Estimation in Dynamic Multi-view Meeting Scenarios, Proceedings of
the 10th International Conference on Multimodal Interfaces (ICMI) Chania,
Greece
pdf
Bader, T., Meissner, A., Tschnerney, R. (2008) Digital Map Table with
Fovea-Tablett®: Smart Furniture for Emergency Operation Centers, Proceedings
of the 5th International Conference on Information Systems for Crisis Response
and Management (ISCRAM) Washington DC, USA pdf
2007
Bernardin, K., Gehrig, T., Stiefelhagen, R. (2007) Multi-Level Particle
Filter Fusion of Features and Cues for Audio-Visual Person Tracking,
Multimodal Technologies for Perception of Humans: International Evaluation
Workshops CLEAR 2007 and RT 2007, Baltimore, MD, USA, May 8-11, 2007, Revised
Selected Papers, 70-81, Springer LNCS 4625. pdf
Nickel, K., Stiefelhagen, R. (2007) Visual Recognition of Pointing Gestures
for Human-Robot Interaction, Image and Vision Computing, 25, 12,
1875-1884. pdf
External links
» The research group Computer Vision for Human-Computer Interaction, Universität Karlsruhe (TH)
» The CHIL project (Computers In the Human Interaction Loop)
» The Fraunhofer project Crisis Management at the Digital Situation Table
» Barco Control Rooms
Group description | People | Photo gallery | Publications | External links
Internal
Bookmark at