Oculo Nimbus

Toward New Statistical Models for Interpreting Eye-Tracking Data at the Heart of Our Cognition

Action Team

The Oculo Nimbus research team (Toward New Statistical Models for Interpreting Eye-Tracking Data at the Heart of Our Cognition) is a project funded by LabEx PERSYVAL.

Summary of the Action Team

In the field of vision, human interaction with the environment is achieved through dynamic exploration of regions of visual interest via eye movements. Understanding the mechanisms responsible for this efficient sampling of information opens up new avenues for innovation in human-machine interaction, either by mimicking human visual exploration in robots or by creating "natural" interfaces for humans.

Eye tracking is a technique for continuously monitoring and recording eye movements. It is a widely used measurement tool that has become increasingly easy to use in real-world settings. In the future, this technology will be used in fields beyond its original applications (cognitive psychology or clinical settings), extending into cognitive ergonomics or the enhancement of multimedia content—to name just two of the most emblematic examples in the digital age.

Our project aims to develop new statistical tools for analyzing eye movements and multimodal data.

We plan to organize our research efforts around four main themes. The first theme concerns the development of statistical models for multimodal data and eye movements. The other three themes concern: (i) the segmentation of spatiotemporal data into complete cognitive phases, (ii) the analysis of spatiotemporal dependencies to explain intra- and inter-individual differences, and (iii) the modeling of eye fixations with higher spatial resolution to understand the functional roles of microsaccades in visual perception.

The objectives of this project team are aligned with those of PERSYVAL-Lab, and more specifically with the Advanced Data Mining research area.

Our theoretical work will focus on adapting and improving existing tools in spatial statistics—in particular, point process models and spatial Markov chains—to address the specific challenges posed by eye-movement data, both on their own and when combined with electroencephalographic signals. In addition, we will draw on the existing literature on hierarchical modeling to construct models capable of quantifying intra- and inter-individual differences.

Point process models are used to analyze data that takes the form of a set of points in space—essentially, a simple set of random points on a map. Such data is naturally very common and ranges from the observed locations of animals or trees (in ecology) to cases of disease (in epidemiology). The usual goal is to understand why things happen where they do and to predict future occurrences. In our specific applications, our data consists of eye-tracking fixations on an image or video, and the goal is to understand why subjects choose to fixate on one location rather than another. To do this, point process models attempt to link observed locations to various spatial covariates.
Animals are more likely to be found where there is food, and a map of food availability would therefore be a good spatial covariate for an animal localization model. Similarly, fixation points can be predicted based on local image characteristics (Privitera & Starck, 2000), with local contrast being a good example: homogeneous regions of the image are less interesting (Itti et al., 1998).

From a cognitive perspective, fixation sequences (known as "scanpaths") reveal non-stationary processes. For example, in an information-seeking task such as reading or visual exploration (visual search), the phases of reading, visual search, decision-making, or confirmation are intertwined. To infer latent states from the data, an approach based on HMMs and inverse problem modeling was proposed by Simola et al. (2008). These cognitive phases should be reflected in both eye movements and EEG sequences.
However, the use of these multimodal data raises theoretical questions concerning primarily the heterogeneity of the observations (continuous, discrete—categorical or nominal, ordinal), as well as the temporal lag between the different modalities, which implies asynchrony between the phases.

Previous studies (Simola et al., 2008) have shown that reading is not a static process, from the perspective of cognition and eye movements. Most often, the acquisition of information through reading involves several stages or phases, such as scanning, attentive reading, decision-making, or confirmation. While this study (Simola et al., 2008) focuses on reading, this issue is also found in visual search research and, more generally, in information-seeking tasks across all media. In this project, we plan to study this question in both contexts: reading and visual search.

Eye movement patterns vary considerably from one individual to another. Some differences are idiosyncratic, but others may be due to disease, and eye movements are important tools for diagnosis and rehabilitation in psychiatry, neurology, and ophthalmology. This issue will be addressed within the context of this project, but using the databases currently available to us, derived from healthy volunteer participants (eye movements of infants aged three to twelve months, and eye movements during visual exploration and reading in adults exhibiting different behaviors when speed-related instructions are mandatory). We hypothesize that there may be substantial differences in sequential dependencies that could potentially be captured in point-in-time processes or Markov models.

Visual fixation is accompanied by minute eye movements known as fixational eye movements (FEMs). There are three types of FEMs: drift, tremor, and microsaccades (Rolfs, 2009). However, the functional role of microsaccades was called into question in early studies: several groups provided evidence that microsaccades are nothing more than oculomotor noise, as argued by Collewijn & Kowler (2008). In contrast, Ko, Poletti & Rucci (2010) replicated the “threading a needle” task in a highly controlled virtual environment and observed that the frequency of the smallest microsaccades increased significantly toward the end of the task, whereas they were rare during threading or fixation. These potential properties of noise-enhanced microsaccades have recently been modeled using a basic model of retinal photoreceptors (Zozor, Amblard & Duchene, 2009). The term "noise-enhanced" refers to the ability of systems to improve their performance in the presence of noise. Since microsaccades generate rapid fluctuations in the retinal image, Zozor et al. (2009) suggested that these visual transients could be useful for spatial information processing.

Participants

  • GIPSA-lab (Theme 4): Simon Barthelmé (CNRS Research Fellow), Anne Guérin-Dugué (Professor, University of Grenoble Alpes), Nathalie Guyader (Assistant Professor, University of Grenoble Alpes), Ronald Phlypo (Assistant Professor, Grenoble Institute of Physics), Steeve Zozor (CNRS Research Fellow)
  • LIG (Topic 1): Francis Jambon (Assistant Professor, University of Grenoble Alpes)
  • LJK (Theme 3): Mariane Clausel (Assistant Professor, University of Grenoble Alpes, until August 2017), Jean-François Coeurjolly (Assistant Professor, University of Grenoble Alpes, until August 2016), Jean-Baptiste Durand (Assistant Professor, INP), Jean-Charles Quinton (Assistant Professor, University of Grenoble Alpes)
  • LPNC (Theme 4): David Alleysson (Research Fellow, CNRS), Alan Chauvin (Assistant Professor, UGA), Benoît Lemaire (Assistant Professor, UGA), David Meary (Assistant Professor, UGA)
  • Associate Participant (Theme 1): Jean-François Coeurjolly (Professor, UQAM, Montreal, Canada, since September 2016)

Doctoral dissertations

1. Joint analysis of eye movements and EEGs using coupled hidden Markov models

  • Ph.D. student: Brice Olivier
  • Thesis advisors: Jean-Baptiste Durand, Anne Guérin-Dugué
  • Starting in October 2015, at the MSTII Doctoral School (Mathematics, Information Sciences and Technologies, Computer Science)
  • Read the thesis abstract (in English)

2. Modeling eye movements

  • Ph.D. student: Camille Breuil
  • Thesis advisors: Nathalie Guyader, Simon Barthelmé
  • Starting in November 2015, funded by an “AGIR” fellowship (2015) through the ISCE doctoral school (Engineering for Health, Cognition, and the Environment)
  • Read the thesis abstract (in English)

3. Modeling micro-eye movements through multistable perception

  • Ph.D. student: Kevin Parisot
  • Thesis advisors: Steeve Zozor, Ronald Plypo, Alan Chauvin
  • Starting in October 2016, funded by a university scholarship (2015), in the EEATS doctoral school (Electronics, Electrical Engineering, Automation, Signal Processing)
  • Read the thesis abstract

Postdoctoral Fellow

1. Spatial/computational statistics

  • Postdoctoral researcher: Mélisande Albert
  • Advisors: Simon Barthelmé, Jean-François Coeurjolly
  • Started in December 2015, ended in August 2016

Publications

Spatial point process (collaborating partner, J.F. Coeurjolly)

Jean-François Coeurjolly, Jesper Møller, Rasmus Waagepetersen. (2017). Palm distributions for log Gaussian Cox processes. Scandinavian Journal of Statistics, Wiley, 44 (1), pp.192-203. <hal-01163672v4>

Jean-François Coeurjolly, Jesper Møller, Rasmus Waagepetersen. (2017). A tutorial on Palm distributions for spatial point processes. International Statistical Review, Wiley, 83 (5), pp.404-420. <hal-01241277v3>

Jean-François Coeurjolly (2017). Median-based estimation of the intensity of a spatial point process. Annals of the Institute of Statistical Mathematics, Springer Verlag, 69 (2), pp.303-331. <hal-01071605v2>

Jean-François Coeurjolly, Frédéric Lavancier. (2017). Parametric estimation of pairwise Gibbs point processes with infinite range interaction. Bernoulli, Bernoulli Society for Mathematical Statistics and Probability, 23 (2), pp.1299-1334. <hal-01092225v3>

Achmad Choiruddin, Jean-François Coeurjolly, Frédérique Letue. (2018). Convex and non-convex regularization methods for spatial point processes intensity estimation. Electronic journal of statistics , Shaker Heights, OH : Institute of Mathematical Statistics, 12 (1), pp.1210-1255. <hal-01484779v2>

Joint analysis of eye movements and EEG

Emmanuelle Kristensen, Bertrand Rivet, Anne Guérin-Dugué. (2017). Estimation of overlapped Eye Fixation Related Potentials: The General Linear Model, a more flexible framework than the ADJAR algorithm. Journal of Eye Movement Research, 10 (1):1-27. <hal-01568579>

Emmanuelle Kristensen, Anne Guérin-Dugué, Bertrand Rivet. (2017). Regularization and a general linear model for event-related potential estimation. Behavior Research Methods, 49(6):2255-2274. <hal-01539867>

Aline Frey, Benoit Lemaire, Laurent Vercueil, Anne Guérin-Dugué. (2018). An Eye Fixation-Related Potential Study in Two Reading Tasks: Reading to Memorize and Reading to Make a Decision. Brain Topography, 31(4):640-660. <hal-01741895v1>

Anne Guérin-Dugué, Raphaelle N. Roy, Emmanuelle Kristensen, Bertrand Rivet, Laurent Vercueil, Anna Tcherkassof. (2018). Temporal Dynamics of Natural Static Emotional Facial Expressions Decoding: A Study Using Event- and Eye Fixation-Related Potentials. Frontiers in Psychology, Frontiers, 2018, 9:1190.〈hal-01837209〉

Modeling eye movements

Jean-Charles Quinton, Laurent Goffart. (2018). A unified dynamic neural field model of goal directed eye-movements. Connection Science, Taylor & Francis, Embodied Neuronal Mechanisms in Adaptive Behaviour, 30 (1), pp.20-52. <hal-01637024>

International Conferences

Joint analysis of eye movements and EEG

Emmanuelle Kristensen, Raphaëlle N. Roy, Bertrand Rivet, Anna Tcherkassof, Anne Guérin-Dugué. (2017). Analyzing Emotional Facial Expressions’ Neural Correlates Using Event-Related Potentials and Eye Fixation-Related Potentials. 19th European Conference on Eye Movements (ECM 2017), Aug 2017, Wuppertal, Germany. <hal-01577643>

Anne Guérin-Dugué, Benoit Lemaire, Aline Frey. (2017). General Linear Model to isolate higher-level cognitive components from oculomotor factors in natural reading by using EEG and eye-tracking data coregistration. 19th European Conference on Eye Movements (ECM 2017), Aug 2017, Wuppertal, Germany. <hal-01841508>

Modeling eye movements

Camille Breuil, Simon Barthelme, Nathalie Guyader. (2017). How redundant are luminance and chrominance information in natural scenes?. Vision Sciences Society Annual Meeting, May 2017, St Pete's Beach, United States. <hal-01836541>

Camille Breuil, Simon Barthelme, Nathalie Guyader. (2017). Luminance modulates color detection threshold in natural scenes. International Colour Vision Society Symposium, Aug 2017, Erlangen, Germany. <hal-01836503>

Camille Breuil, Simon Barthelme, Nathalie Guyader. (2017). Luminance modulates color detection threshold in natural scenes. European Conference on Visual Perception, Aug 2017, Berlin, Germany. <hal-01836518>

Brice Olivier, Jean-Baptiste Durand, Anne Guérin-Dugué, Marianne Clausel. (2017). Eye-tracking data analysis using hidden semi-Markovian models to identify and characterize reading strategies. 19th European Conference on Eye Movements (ECM 2017), Aug 2017, Wuppertal, Germany. 2017. <hal-01671224>

Modeling micro-eye movements through multistable perception

Kevin Parisot, Alan Chauvin, Anne Guérin-Dugué, Ronald Phlypo, Steeve Zozor. (2017). Predictable motion on a Necker cube leads to micro-pursuit-like eye movements and affects the dynamics of bistability.. 19th European Conference on Eye Movements (ECM 2017), Aug 2017, Wuppertal, Germany. <hal-01726513>

Kevin Parisot, Alan Chauvin, Anne Guérin-Dugué, Ronald Phlypo, Steeve Zozor. (2017). A multistable gravitational potential approach to fixational eye movements. European Conference on Visual Perception (ECVP 2017), Aug 2017, Berlin, Germany. <hal-01724178>

National Conferences

Joint analysis of eye movements and EEG

Jean-Baptiste Durand, Anne Guérin-Dugué, Sophie Achard. (2016). Analyse de séquences oculométriques et d'électroencéphalogrammes par modèles markoviens cachés. 48èmes Journées de Statistique, May 2016, Montpellier, France. <hal-01339458>

Eye movements and EEG

Jean-Charles Quinton, Laurent Goffart. (2016). A neural field model of the dynamics of goal-directed eye movements. Colloque BioComp, Oct 2016, Lyon, France. <hal-01839803>

Published on November 26, 2024

Updated on April 3, 2025