This paper introduces a new methodology for performing multimodal data fusion. Each modality is considered as being an agent, capable of performing certain actions inside a given state-space. The different modalities form cooperative teams of agents in the same way as individual players form a football team. We argue that performing behavior recognition on the team formed by the modalities is equivalent to performing Multimodal Fusion. The Multi-agent Abstract Hidden Markov mEmory Model (M-AHMEM) is used to model the actions of the single agents and the teams, and Dynamic Bayesian Networks are used to perform inference. To prove our hypothesis, we have analyzed a set of real environment videos for use in domestic application providing an interface to proactively assist elderly people living alone at home to perform their daily activities. In the present part of research we track behavior of a person moving inside his/her living room and producing utterances in natural language. Thus we have a restricted domain to work with, but we deal with unrestricted natural human behavior. The results obtained from the analysis of these sequences, show that performing multimodal fusion in terms of cooperative agents can complement the natural strength of each modality improving robustness.
Benoit Macq was born in 1961. He is currently Professor at Université catholique de Louvain (UCL), in the Telecommunication Laboratory. He has done is military service in 1984-1985 at the Royal Military School of Belgium were he worked on Laser interferometer measurements. He worked on networks planning in 1985 for the Tractebel company, Brussels. He did his doctoral thesis on perceptual coding for digital TV under the supervision of Prof. Paul Delogne at UCL. He was researcher at Philips Research in 1990 and 1991. He has been senior researcher of the Belgian NSF. Benoit Macq has been visiting scientist at Ecole Polytechnique Fédérale de Lausanne and at the Massachussets Institute of Technology, Boston. He has been Visiting Professor at the "Ecole Nationale Supérieure des Télécommunications", ENST-Paris and at the "Université de Nice Sophia-Antipolis". Benoit Macq is teaching and doing is research work in image processing for visual communications. His main research interests are image compression, image watermarking and image analysis for medical and immersive communications.
Benoit Macq has been Guest Editor for the Proceedings of the IEEE and for the Signal Processing Journal and member of the program committee of several IEEE and SPIE conferences. He is co-Technical Chair of the IEEE Conference on Multimedia and Expo (ICME02) and member of the board of EUSIPCO2002 and ICPR2002. He will be in the TC of ICASSP06. He is co-guest editor for a special issue on security for image communications of the Image Communications journal to be published in 2003, co-guest Editor for a special issue on Watermarking for the IEEE Trans on CSVT to be published in 2003 and co-Guest editor for the Proceedings of the IEEE on Digital Rights Management to be published in 2004. He is Senior Member of the IEEE. He is member of the IEEE Technical Committee IMDSP (Image and Multidimensional Signal Processing). He is Editor Asoociate of the IEEE Transactions on Multimedia. He received the Bell Telephone award in 1990.