New DFG project on Formation of crossmodal episodic memory

11 February 2021

Prof. Dr. Michael Rose, CML principal investigator in project B3, has acquired a new DFG-funded project titled "Relevance of neuronal pre-stimulus oscillatory activity for the formation of crossmodal episodic memory”, starting in 2021 and running until 2024.

For several cognitive processes it was demonstrated that the neuronal activity prior to stimulus presentation modulates the consecutive processing of that stimulus. For memory formation our group and several other researchers demonstrated a pronounced influence of pre-stimulus oscillatory activity on memory encoding. This effect was mainly demonstrated for item memory presented in one modality. However, a key feature of episodic memory in the real world is the integration of different sensory modalities. Therefore, the aim of the present proposal is the examination of the functional role of pre-stimulus oscillatory activity for the formation of crossmodal associations.

Based on theoretical assumptions and previous observations we hypothesize a functional relevance of oscillatory activity in the theta-band (4-8 Hz). The pre-stimulus theta band characteristics including amplitude and phase locking are hypothesized to be predictive for memory encoding for associations of information from different sensory modalities. Furthermore, the interactive effect of pre- and poststimulus activity for memory formation will be examined. The already developed and tested experimental design allows for assessing pure associative memory between visual and auditory stimuli independent from the memory for the individual stimuli.

In the first section of the proposal audio-visual memory formation will be examined using EEG- measurements. Here, the relevance of pre-stimulus theta band amplitude and phase coupling and the relation of both parameters to post-stimulus processing will be examined. Theoretical models assumes that the coincident input from different sensory modalities to the hippocampus is realized by theta- band oscillatory activity. In the second section this assumption will be tested using combined EEG- and fMRI measurements. Here, the hypothesis is tested that theta-band pre-stimulus activity affects the consecutive processing within the hippocampus.

The examination of the neuronal correlates of cross-sensory episodic memory should reflect the rich environment in which these memories are generated usually. Therefore, in the third section it is planned to test the reliability and validity of the laboratory results in a more realistic virtual reality (VR) setting. In the real world episodic memory is encoded in a natural context with is usually part of the memory. The VR setting is perfectly suited to test the encoding of more naturalistic contextual memory and the development of crossmodal association within that context. Furthermore, the behavior of the participants can be recorded in VR in detail to relate behavioral aspects to the formation of memory. Here, it is planned to test the influence of self-engaged actions and exploratory behavior (head- and eye movements) on the pre-stimulus oscillatory state and memory encoding. Therefore, EEG will be measured simultaneously with the VR- stimulation in the last experiments.