My research interests are mainly focused on active/interactive perception, i.e. understanding how perception and action are tighlty linked together inside the sensorimotor flow. My approach consists in studying sensorimotor contingencies (i.e. invariants) whose mathematical structure can explain how a naive robot can understand, by itself, the structure of its interaction with the environment.
I have also worked on Robot Audition (well, I still do sometimes), especially within the binaural paradigm. In Robotics, using only two microphones with a head is a challenging task when dealing with auditory scene analysis in realistic environments involving noises, reverberations, etc. On this topic, I have mainly worked on sound source localization, but also on active audition, mixing audition with the robot movement to better the scene analysis.
Habilitation (HDR), 2018
Sorbonne Université, Paris
PhD in Robotics, 2006
Université Paul Sabatier, Toulouse
M.Sc. in Robotics and Intelligent Systems, 2003
Université Pierre et Marie Curie, Paris
Agrégation externe in electrical engineering, 2002
Ecole Normale Supérieure, Cachan
My list of publications is available here.
Selected publications :
For naive robots to become truly autonomous, they need a means of developing their perceptive capabilities instead of relying on hand crafted models. The sensorimotor contingency theory asserts that such a way resides in learning invariants of the sensorimotor flow. We propose a formal framework inspired by this theory for the description of sensorimotor experiences of a naive agent, extending previous related works. We then use said formalism to conduct a theoretical study where we isolate sufficient conditions for the determination of a sensory prediction function. Furthermore, we also show that algebraic structure found in this prediction can be taken as a proxy for structure on the motor displacements, allowing for the discovery of the combinatorial structure of said displacements. Both these claims are further illustrated in simulations where a toy naive agent determines the sensory predictions of its spatial displacements from its uninterpreted sensory flow, which it then uses to infer the combinatorics of said displacements.
This paper deals with the perception of mobile robotic systems within the framework of interactive perception, and inspired by the sensorimotor contingencies (SMC) theory. These approaches state that perception arises from active exploration of an environment. In the SMC theory, it is postulated that information about the structure of space could be recovered from a quasi-uninterpreted sensorimotor flow. In a recent article, the authors have provided a mathematical framework for the construction of a sensorimotor representation of the interaction between the sensors and the body of a naive agent, provided that the sensory inputs come from the agent’s own body. An extension of these results, with stimulations coming from an unknown changing environment, is proposed in this paper. More precisely it is demonstrated that, through repeated explorations of its motor configurations, the perceived sensory invariants can be exploited to build a topologically accurate internal representation of the relative poses of the agent’s sensors in the physical world. Precise theoretical considerations are provided as well as an experimental framework assessed in simulated but challenging environments.
Over the last twenty years, a significant part of the research in exploratory robotics partially switches from looking for the most efficient way of exploring an unknown environment to finding what could motivate a robot to autonomously explore it. […] The Head Turning Modulation model presented in this paper is composed of two modules providing a robot with two different forms of intrinsic motivations leading to triggering head movements towards audiovisual sources appearing in unknown environments. First, the Dynamic Weighting module implements a motivation by the concept of Congruence, a concept defined as an adaptive form of semantic saliency specific for each explored environment. Then, the Multimodal Fusion & Inference module implements a motivation by the reduction of Uncertainty through a self-supervised online learning algorithm that can autonomously determine local consistencies. […] Results presented in this paper have been obtained in simulated environments as long as with a real robot in realistic experimental conditions.
I’m the teaching instructor for the following courses at Sorbonne Université:
I’m also involved in the following courses (tutorial classes or practical works):
Most of my courses are available on Sorbonne Université’s Moodle (only available to the University students, for now).
Research is definitely not something you can do alone. You work with colleagues, partners, students, all of them sharing their enthusiasm, knownledge and competences. I have/had the pleasure to work with:
Waradon Senzt Phokhinanan (2019 - to date), in cooperation with Nicolas Obin (IRCAM)
Waradon is just starting his PhD on active robotic audition, in cooperation with Nicolas Obin from IRCAM.
Jean-Merwan Godon (2017 - to date)
Jean comes from applied mathematics, and he tried during his M.Sc. internship to introduce some rigorous mathematical foundations to the notion of action, in the framework of interactive perception. He is now trying to pursue this work during his PhD, with very promising forthcoming results!
Valentin Marcel (2015 - 2019)
Valentin is working hard on active perception, trying to formalize the existence of contingencies in the sensorimotor flow. So far, he proposed a contribution on how to build a sensorimotor representation of an agent body. We are now trying to extend this formalization to the working space of a naive agent.
Benjamin Cohen-Lhyver (2013 - 2017), co-advised with Bruno Gas
Benjamin worked on the european project TWO!EARS. His PhD thesis, entitled "Modulating the head movement for the multimodal analysis of an unknown environment" dealt with how a robot, entailed with a rotating head, can use its action to (i) learn the interlink between the audio and visual modalities, in order to (ii) better its understand of the audiovisual scene (through the new notion of congruence).
Alban Laflaquière (2009-2013), co-advised with Bruno Gas
Alban was the first PhD student of the team to work on the sensorimotor point of view of perception. As such, he did all the spadework for our futur works on interactive perception. During his PhD thesis, he worked on the estimation of space dimension, inspired by Henri Poincaré's intuition. He also proposed a first intuitive way for an agent to build a representation of its interaction with the environement.
Karim Youssef (2010-2013), co-advised with Jean-Luc Zarader
Karim worked on the BINNAHR project. He was mainly interested in binaural sound localization in realistic acoustic conditions, involving noises and reverberations. He proposed a learning strategy which is robust to such considerations, together with other contributions on visualy-guided audio localization, or binaural speaker recognition.
Alban Portello (2010-2013), co-advised with Patrick Danès
Alban was in LAAS, Toulouse during his PhD, mainly advised by Patrick. Like Karim, he was involved in the BINNAHR project. The aim of his PhD was to develop active strategies, combining binaural signals with the sensor motor commands so as to overcome usual limitations in the case of a static world: resolve front-back ambiguities, recover observability of variables, etc.
Antonyo worked with us on the TWO!EARS project as an engineer. He was in charge of the ISIR robot of the projet (called ODI). He also developed, with our colleagues from LAAS, a ROS binaural frontend which is now available on Github.