Publications - Thematic Area A

Publications project A1 - Dynamic adaptation of multisensory processes by crossmodal recalibration

Publications of the current funding period (2020-2024)

Bruns P, Thun C, Röder B (2024). Quantifying accuracy and precision from continuous response data in studies of spatial perception and crossmodal recalibration, Behavior Research Methods, 56, 3814-3830. https://doi.org/10.3758/s13428-024-02416-1
Bruns P, Röder B (2024). Improvement of sound localization after spatially congruent audiovisual exposure: A special case of crossmodal recalibration or higher-order visual location learning?, Fortschritte der Akustik - DAGA 2024, 237-240. https://pub.dega-akustik.de/DAGA_2024/files/upload/paper/43.pdf
Li C, Xuan Y, Bruns P, Fu X (2024). The role of arousal in the estimation of time-to-collision of threatening stimuli, PsyCh Journal, 13, 376-386. https://doi.org/10.1002/pchj.762
Bruns P, Röder B (2023). Development and experience-dependence of multisensory spatial processing, Trends in Cognitive Sciences, 27, 961-973. https://doi.org/10.1016/j.tics.2023.04.012
Bruns P, Li L, Guerreiro MJS, Shareef I, Rajendran SS, Pitchaimuthu K, Kekunnaya R, Röder B (2022). Audiovisual spatial recalibration but not integration is shaped by early sensory experience, iScience 25, 104439, June 17, 2022. https://doi.org/10.1016/j.isci.2022.104439
Rohlf S, Bruns P, Röder B (2021). The Effects of Cue Reliability on Crossmodal Recalibration in Adults and Children, Multisensory Research 34:7, pp. 743-761, 2021. https://doi.org/10.1163/22134808-bja10053
Bruns P, Dinse HR, Röder B (2020). Differential effects of the temporal and spatial distribution of audiovisual stimuli on cross-modal spatial recalibration, Eur J Neurosci. 2020, 52:3763–377. https://doi.org/10.1111/ejn.14779
Gudi-Mindermann H, Rimmele JM, Bruns P, Kloosterman NA, Donner TH, Engel AK, Röder B (2020). Post-training Load-Related Changes of Auditory Working Memory – An EEG Study, Front. Hum. Neurosci., vol 14, article 72, 17 March 2020 (Sec. Speech and Language). https://doi.org/10.3389/fnhum.2020.00072
Kramer A, Röder B, Bruns P (2020). Feedback Modulates Audio-Visual Spatial Recalibration, Front. Integr. Neurosci. 13:74. https://doi.org/10.3389/fnint.2019.00074
Li L, Rehr R, Bruns P, Gerkmann T, Röder B (2020). A Survey on Probabilistic Models in Human Perception and Machines, Front. Robot. AI 7:85. https://doi.org/10.3389/frobt.2020.00085
Rohlf S, Li L, Bruns P, Röder B (2020). Multisensory Integration Develops Prior to Crossmodal Recalibration, Current Biology 30, 1726–1732, May 4, 2020. https://doi.org/10.1016/j.cub.2020.02.048
Tong J, Li L, Bruns P, Röder B (2020). Crossmodal associations modulate multisensory spatial integration, Attention, Perception, and Psychophysics (2020) 82:3490–3506. https://doi.org/10.3758/s13414-020-02083-2

Publications of the first funding period (2016-2019)

Bruns P (2019). The ventriloquist illusion as a tool to study multisensory processing: An update. Frontiers in Integrative Neuroscience, 13, 51. 10.3389/fnint.2019.00051
Bruns P, Röder B (2019). Visual recalibration of auditory spatial perception decays at different time scales. Proceedings of the International Congress on Acoustics, 2019, 3915-3920. 10.18154/RWTH-CONV-239152
Zierul B, Tong J, Bruns P, Röder B (2019). Reduced multisensory integration of self-initiated stimuli, Cognition, 182, 349-359.
Bruns P, Röder B (2019). Cross-modal learning in the auditory system. In A. K. C. Lee, M. Wallace, A. Coffin, A. N. Popper, & R. R. Fay (Eds.), Multisensory processes: The auditory perspective. Springer Handbook of Auditory Research.
Bruns P, Röder B (2019). Repeated but not incremental training enhances cross-modal recalibration. Journal of Experimental Psychology: Human Perception and Performance, 45(4), 435.
Badde S, Röder B, Bruns P (2018). Task-irrelevant sounds influence both temporal order and apparent motion judgments about tactile stimuli applied to crossed and uncrossed hands. Attention, Perception, & Psychophysics, 80(3), 773-783.
Gudi-Mindermann H, Rimmele JM, Nolte G, Bruns P, Engel AK, Röder B (2018). Working memory training in congenitally blind individuals results in an integration of occipital cortex in functional networks. Behavioural Brain Research, 348, 31-41. 10.1016/j.bbr.2018.04.002
Parisi GI, Tong J, Barros P, Röder B, Wermter S (2018). Towards Modeling the Interaction of Spatial-Associative Neural Network Representations for Multisensory Perception. Workshop on Computational Models for Crossmodal Learning (ICDL-EpiRob).
Bruns P, Röder B, (2017). Spatial and frequency specificity of the ventriloquism aftereffect revisited. Psychological Research, 1-16.
Katzakis N, Tong J, Nunez O, Chen L, Klinker G, Röder B, Steinicke F (2017). Stylo and handifact: modulating haptic perception through visualisations for posture training in augmented reality. Proc. Spatial User Interaction. 10.1145/3131277.3132181
Zierul B, Röder B, Tempelmann C, Bruns P, Noesselt T (2017) The Role of Auditory Cortex in the Spatial Ventriloquism Aftereffect. NeuroImage, 162, 257-268.
Zhang D, Hong B, Gao S, Röder B (2017). Exploring the temporal dynamics of sustained and transient spatial attention using steady-state visual evoked potentials. Experimental Brain Research, 2017 May; 235(5):1575-1591. 10.1007/s00221-017-4907-6

Publications project A2 - Using multiperturbation analysis to reveal the neural circuit basis of multi-sensory processing

Publications of the current funding period (2020-2024)

Fakhar K, Dixit S, Hadaeghi F, Kording K, Hilgetag CC (2024). Downstream network transformations dissociate neural activity from causal functional contributions, Sci Rep 14, 2103 (2024). https://doi.org/10.1038/s41598-024-52423-7
Hadaeghi F, Fakhar K, and Hilgetag CC (2024). Controlling Reciprocity in Binary and Weighted Networks: A Novel Density-Conserving Approach, bioRxiv (preprint). https://www.biorxiv.org/content/10.1101/2024.11.24.625064v1
Xie H, Liu K, Li D, Zhang CS, Hilgetag CC, Guan JS (2024). Rectified activity-dependent population plasticity implicates cortical adaptation for memory and cognitive functions, Communications Biology 7:1487. https://doi.org/10.1038/s42003-024-07186-2
Zamanzadeh M, Pourhedayat A, Bakouie F, Hadaeghi F (2024). Exploring potential ADHD biomarkers through advanced machine learning: An examination of audiovisual integration networks, Computers in Biology and Medicine, 183, 109240. https://doi.org/10.1016/j.compbiomed.2024.109240
Zavaglia M, Malherbe C, Schlaadt S, Nachev P, Hilgetag CC (2024). Ground-truth validation of uni- and multivariate lesion inference approaches, Brain Communications 6(5):fcae251. https://doi.org/10.1093/braincomms/fcae251
Barbas H, Hilgetag C (2023). From Circuit Principles to Human Psychiatric Disorders, BIOL PSYCHIAT. 2023;93(5):388-390. https://doi.org/10.1016/j.biopsych.2022.08.007
Fakhar K, Dixit S, Hadaeghi F, Kording K, Hilgetag CC (2023). When Neural Activity Fails to Reveal Causal Contributions, bioRxiv. https://doi.org/10.1101/2023.06.06.543895
Moore J, Wilms M, Gutierrez A, Ismail Z, Fakhar K, Hadaeghi F, Hilgetag C, Forkert N (2023). Simulation of neuroplasticity in a CNN-based in-silico model of neurodegeneration of the visual system, FRONT COMPUT NEUROSC. 17:1274824.. https://doi.org/10.3389/fncom.2023.1274824
Wang F, Chen X, Bo B, Zhang T, Liu K, Jiang J, Wang Y, Xie H, Liang Z, Guan JS (2023). State-dependent memory retrieval: insights from neural dynamics and behavioral perspectives, Learning & Memory 30 (12), 325-337. https://psycnet.apa.org/doi/10.1101/lm.053893.123
Ansari Z, Pourhoseini F, Hadaeghi F (2022). Heterogeneous Reservoir Computing Models for Persian Speech Recognition., 2022 International Joint Conference on Neural Networks (IJCNN). https://doi.org/10.1109/IJCNN55064.2022.9892570
Chen QN, Ding XL, Guo XX, Zhou G, Guan JS (2022). Suv39h1 regulates memory stability by inhibiting the expression of Shank1 in hippocampal newborn neurons, Eur J Neurosci. 2022 Mar;55(6):1424-1441. https://doi.org/10.1111/ejn.15626
Damicelli F, Hilgetag CC, Goulas A (2022). Brain connectivity meets reservoir computing, PLoS Comput Biol 18(11): e1010639. https://doi.org/10.1371/journal.pcbi.1010639
Fakhar K, Hilgetag CC (2022). Systematic perturbation of an artificial neural network: A step towards quantifying causal contributions in the brain, PLoS Comput Biol 18(6): e1010250. https://doi.org/10.1371/journal.pcbi.1010250
Fakhar K, Hadaeghi F, Hilgetag CC (2022). Causal Influences Decouple From Their Underlying Network Structure In Echo State Networks, 2022 International Joint Conference on Neural Networks (IJCNN), pp. 1-8. https://doi.org/10.1109/IJCNN55064.2022.9892782
Goulas A, Damicelli F, Hilgetag CC (2022). Bio-instantiated recurrent neural networks: Integrating neurobiology-based network topology in artificial networks, Neural Netw. 2021 Oct;142:608-618. https://doi.org/10.1016/j.neunet.2021.07.011
Hilgetag CC, Zikopoulos B (2022). The highways and byways of the brain, PLOS BIOL. 20(3):e3001612. https://doi.org/10.1371/journal.pbio.3001612
Hilgetag CC, Goulas A, Changeux JP (2022). A natural cortical axis connecting the outside and inside of the human brain, Network Neuroscience, 6(4), 950-959, 2022. https://doi.org/10.1162/netn_a_00256
Lamothe-Molina PJ, Franzelin A, Beck L, Li D, Auksutat L, Fieblinger T, Laprell L, Alhbeck J, Gee CE, Kneussel M, Engel AK, Hilgetag CC, Morellini F, Oertner TG (2022). cFos ensembles in the dentate gyrus rapidly segregate over time and do not form a stable map of space, Nature Comm, 13 (6376). http://dx.doi.org/10.1101/2020.08.29.273391
Lamothe-Molina PJ, Franzelin A, Beck L et al. (2022). $\Delta$FosB accumulation in hippocampal granule cells drives cFos pattern separation during spatial learning, Nat Commun 13, 6376 (2022). https://doi.org/10.1038/s41467-022-33947-w
Li D, Wang G, Werner R, Xie H, Guan JS, Hilgetag CC (2022). Single Image-Based Vignetting Correction for Improving the Consistency of Neural Activity Analysis in 2-Photon Functional Microscopy, Front. Neuroinform. 15:674439. https://doi.org/10.3389/fninf.2021.6744
Liu KY, Li XY, Lai YR et al. (2022). Denoised Internal Models: A Brain-inspired Autoencoder Against Adversarial Attacks, Mach. Intell. Res. 19, 456–471 (2022). https://doi.org/10.1007/s11633-022-1375-7
Luo W, Yun D, Hu Y, Tian M, Yang J, Xu Y, Tang Y, Zhan Y, Xie H, Guan JS (2022). Acquiring new memories in neocortex of hippocampal-lesioned mice, Nat Commun. 2022 Mar 24;13(1):1601. https://doi.org/10.1038/s41467-022-29208-5
Yan Y, Tian M, Li M, Zhou G, Chen Q, Xu M, Hu Y, Luo W, Guo X, Zhang C, Xie H, Wu QF, Xiong W, Liu S, Guan JS (2022). ASH1L haploinsufficiency results in autistic-like phenotypes in mice and links Eph receptor gene to autism spectrum disorder, Neuron. 2022 Apr 6;110(7):1156-1172.e9. https://doi.org/10.1016/j.neuron.2021.12.035
Beul SF, Goulas A, Hilgetag CC (2021). An architectonic type principle in the development of laminar patterns of cortico-cortical connections, Brain Struct Funct. 226(4):979-987 (2021). https://doi.org/10.1007/s00429-021-02219-6
Changeux JP, Goulas A, Hilgetag CC (2021). A Connectomic Hypothesis for the Hominization of the Brain, Cereb Cortex. 31(5):2425-2449 (2021). https://doi.org/10.1093/cercor/bhaa365
Goulas A, Changeux JP, Wagstyl K, Amunts K, Palomero-Gallagher N, Hilgetag CC (2021). The natural axis of transmitter receptor distribution in the human cerebral cortex, Proc Natl Acad Sci USA 118(3):e2020574118 (2021). https://doi.org/10.1073/pnas.2020574118
Jiang Y, Fu X, Zhang Y et al. (2021). Rett syndrome linked to defects in forming the MeCP2/Rbfox/LASR complex in mouse models, Nat Commun 12, 5767 (2021). https://doi.org/10.1038/s41467-021-26084-3
Wang G, Xie H, Hu Y, Chen Q, Liu C, Liu K, Yan Y, Guan JS (2021). Egr1-EGFP transgenic mouse allows in vivo recording of Egr1 expression and neural activity, J Neurosci Methods. 363:109350 (2021). https://doi.org/10.1016/j.jneumeth.2021.109350
Beul SF, Hilgetag CC (2020). Systematic modelling of the development of laminar projection origins in the cerebral cortex: Interactions of spatio-temporal patterns of neurogenesis and cellular heterogeneity, PLoS computational biology. 2020 Oct 13;16(10):e1007991. https://doi.org/10.1371/journal.pcbi.1007991
Liu S, Tian M, He F et al. (2020). Spontaneous hyperactivity in Ash1l mutant mice, a new model for Tourette syndrome, Molecular Psychiatry (2020) 25:241–242. https://doi.org/10.1038/s41380-019-0642-7
Liu S, Tian M, et al., Guan JS (2020). Mutations in ASH1L confer susceptibility to Tourette Syndrome., Molecular Psychiatry, Feb; 25(2):476-490. https://doi.org/10.1038/s41380-019-0560-8
Zhang X, Wang F, Hu Y, Chen R, Meng D, Guo L, Lv H, Guan J, Jia Y (2020). In vivo stress granule misprocessing evidenced in a FUS knock-in ALS mouse model, Brain. 2020 May 1;143(5):1350-1367. https://doi.org/10.1093/brain/awaa076
Li D, Wang G, Xie H, Hu Y, Guan JS, Hilgetag CC (2019). Multimodal Memory Components and Their Long-Term Dynamics Identified in Cortical Layers II/III but Not Layer V, Front. Integr. Neurosci. 13:54. https://doi.org/10.3389/fnint.2019.00054
Zheng Q, Liu P, Gao G, Yuan J, Wang P, Huang J, Xie L, Lu X, Di F, Tong T, Chen J, Lu Z, Guan J, Wang G (2019). Mitochondrion-processed TERC regulates senescence without affecting telomerase activities, Protein Cell. (9):631-648 (2019). https://doi.org/10.1007/s13238-019-0612-5

Publications of the first funding period (2016-2019)

Damicelli F, Hilgetag CC, Hüt, MT, Messé A (2019). Topological reinforcement as a principle of modularity emergence in brain networks. Network Neuroscience, 3(2), 1-34.
Wang G, Xie H, Wang L, Luo W, Wang Y, Jiang J, ...Guan JS (2019). Switching From Fear to No Fear by Different Neural Ensembles in Mouse Retrosplenial Cortex. Cerebral Cortex pii: bhz050. 10.1093/cercor/bhz050
Li D, Zavaglia M, Wang G, Xie H, Hu Y, Werner R, ... Hilgetag CC (2019). Discrimination of the hierarchical structure of cortical layers in 2-photon microscopy data by combined unsupervised and supervised machine learning. Scientific Reports, 9(1), 7424.
Goulas A, Zilles K, Hilgetag CC. (2018). Cortical gradients and laminar projections in mammals. Trends in Neuroscience, 41(11), 775-788.
Luo W, Guan JS (2018). Do brain oscillations orchestrate memory? Brain Science Advances, 4(1), 16-33.
Jiang J, Wang G, Guan JS (2018). Mammillary body regulates state-dependent fear by alternating cortical oscillations Sci Rep, 8(1), 13471.
Messé A, Hütt MT, Hilgetag CC (2018). Toward a theory of coactivation patterns in excitable neural networks. PLoS Comput Biollgy, 14(4):e1006084.
Rollenhagen A, Ohana O, Sätzler K, Hilgetag CC, Kuhl D, Lübke J (2018) Structural Properties of Synaptic Transmission and Temporal Dynamics at Excitatory Layer 5B Synapses in the Adult Rat Somatosensory Cortex. Front Synaptic Neurosci, 10:24.
Zou Y, Zhao Z, Yin D, Fan M, Small M, Liu Z, Hilgetag C, Kurths (2018) Brain anomaly networks uncover heterogeneous functional reorganization patterns after stroke Neuroimage-Clin. 20:523-530
Ruxing Fu, Wenhan Luo, Roy Nazempour, Daxin Tan, He Ding, Kaiyuan Zhang, Lan Yin, Ji-Song Guan*, Xin Sheng*. (2018). Implantable and Biodegradable Poly(L-lactic acid) Fibers for Optical Neural Interfaces, (2018), Adv Optical Materials. DOI: 10.1002/adom.201700941
Malherbe C, Umarova R, Zavaglia M, Kaller C, Beume L, Thomalla G, Weiller C, Hilgetag C (2018). Neural correlates of visuospatial bias in patients with left hemisphere stroke: a causal functional contribution analysis based on game theory, Neuropsychologia, 115: 142-153.
Fretter C, Lesne A, Hilgetag CC, Hütt MT. (2017). Topological determinants of self-sustained activity in a simple model of excitable dynamics on graphs. Sci Rep 7:42340.
Toba M, Zavaglia M, Rastelli F, Valabrégue R, Pradat-Diehl P, Valero-Cabré A, Hilgetag CC. (2017). Game theoretical mapping of causal interactions underlying visuo-spatial attention in the human brain based on stroke lesions. Hum Brain Mapp. doi: 10.1002/hbm.23601.
Damicelli F, Hilgetag CC, Hütt MT, Messé A. (2017). Modular topology emerges from plasticity in a minimalistic excitable network model. Chaos. 27(4):047406.
Chen Y, Wang S, Hilgetag CC, Zhou C. (2017). Features of spatial and functional segregation and integration of the primate connectome revealed by trade-off between wiring cost and efficiency. PLoS Comput Biol 13(9):e1005776.
Maier-Hein KH, Neher PF, ..., Hilgetag CC, Stieltjes B, Descoteaux M. (2017). The challenge of mapping the human connectome based on diffusion tractography. Nat Commun 8(1):1349.
Stellmann J, Hodecker S, Cheng B, Wanke N, Young K, Hilgetag C, Gerloff C, Heesen C, Thomalla G, Siemonsen S (2017) Reduced rich-club connectivity is related to disability in primary progressive MS Neurol Neuroimmunol Neuroinflamm. 4(5):e375
Goulas A, Uylings H, Hilgetag C (2017) Principles of ipsilateral and contralateral cortico-cortical connectivity in the mouse BRAIN STRUCT FUNCT. 222(3):1281-1295
Ding X, Liu S, Tian M, Zhang W, Zhu T, Li D, Wu J, Deng H, Jia Y, Xie W, Xie H, Guan JS (2017). Activity-induced histone modifications govern Neurexin-1 mRNA splicing and memory preservation. Nat Neurosci. 20(5):690-699.
Zavaglia M, Forkert N, Cheng B, Gerloff C, Thomalla G, Hilgetag C (2016). Technical considerations of a game-theoretical approach for lesion symptom mapping, BMC Neurosci 17:40
Hilgetag C, Amunts K (2016) Connectivity and cortical architecture e-Neuroforum. 7(3):56-63.
Hilgetag C, Medalla M, Beul S, Barbas H (2016) The primate connectome in context: Principles of connections of the cortical visual system. Neuroimage. 134:685-702.

Publications project A3 - Multisensory integration in the aging brain — mechanisms and facilitation

Publications of the current funding period (2020-2024)

Frontzkowski L, Fehring F, Frey BM, Wróbel PP, Reibelt A, Higgen F, Wolf S, Backhaus W, Braaß H, Koch PJ, Choe CU, Bönstrup M, Cheng B, Thomalla G, Gerloff C, Quandt F, Schulz R (2024). Frontoparietal Structural Network Disconnections Correlate With Outcome After a Severe Stroke, Hum Brain Mapp. 2024 Nov;45(16):e70060. https://doi.org/10.1002/hbm.70060
Wróbel PP, Braaß H, Frey BM, Bönstrup M, Guder S, Frontzkowski LK, Feldheim JF, Cheng B, Rathi Y, Pasternak O, Thomalla G, Koerte IK, Shenton ME, Gerloff C, Quandt F, Higgen FL, Schulz R (2024). Cortical microstructure and hemispheric specialization-A diffusion-imaging analysis in younger and older adults, Eur J Neurosci. 2024, vol 60(7), 5718-5730. https://doi.org/10.1111/ejn.16518
Zhang L, Feng J, Liu C, Hu H, Zhou Y, Yang G, Peng X, Li T, Chen C, Xue G. (2024). Improved estimation of general cognitive ability and its neural correlates with a large battery of cognitive tasks, Cerebral Cortex. Jan 5:bhad510. https://doi.org/10.1093/cercor/bhad510
Braaß H, Gutgesell L, Backhaus W, Higgen FL, Quandt F, Choe CU, Gerloff C, Schulz R (2023). Early functional connectivity alterations in contralesional motor networks influence outcome after severe stroke: a preliminary analysis, Sci Rep. 2023 Jul 7;13(1):11010. https://doi.org/10.1038/s41598-023-38066-0
Sheng J, Wang S, Zhang L, Liu C, Shi L, Zhou Y, Xue G (2023). Intersubject similarity in neural representations underlies shared episodic memory content, Proceedings of the National Academy of Sciences, 120(35), e2308951120. https://doi.org/10.1073/pnas.2308951120
Shi L, Liu C, Peng X, Cao Y, Levy DA, Xue G. (2023). The neural representations underlying asymmetric cross-modal prediction of words, Human Brain Mapping, 44(6):2418-2435. https://doi.org/10.1002/hbm.26219
Biskamp J, Isla Cainzos S, Higgen FL, Gerloff C, Magnus T (2022). Normalization of Aperiodic Electrocorticography Components Indicates Fine Motor Recovery After Sensory Cortical Stroke in Mice, Stroke. 2022 Sep;53(9):2945-2953. https://doi.org/10.1161/STROKEAHA.122.039335
Feng J et al. (2022). A cognitive neurogenetic approach to uncovering the structure of executive functions, Nature Communications, (2022) 13:4588. https://doi.org/10.1038/s41467-022-32383-0
Xue G (2022). From remembering to reconstruction: The transformative neural representation of episodic memory, Progress in Neurobiology, 219, 102351. https://doi.org/10.1016/j.pneurobio.2022.102351
Backhaus W, Braaß H, Higgen FL, Gerloff C, Schulz R (2021). Early parietofrontal network upregulation relates to future persistent deficits after severe stroke—a prospective cohort study, Brain Communications (2021). https://doi.org/doi:10.1093/braincomms/fcab097
Higgen FL, Braaß H, Backhaus W, Schulz R, Xue G, Gerloff C (2021). Reduced frontal white matter microstructure in healthy older adults with low tactile recognition performance, Scientific Reports volume 11, Article number: 111689. https://doi.org/10.1038/s41598-021-90995-w
Liu Chuqi, Zhifang Y, Chen C, Axmacher N, Xue G (2021). Hippocampal Representations of Event Structure and Temporal Context during Episodic Temporal Order Memory, Cerebral Cortex, 00:1–15 (2021). https://doi.org/10.1093/cercor/bhab304
Sheng J et al., (2021). The coupling of BOLD signal variability and degree centrality underlies cognitive functions and psychiatric diseases, NeuroImage 237 118187 (2021). https://doi.org/10.1016/j.neuroimage.2021.118187
Feng J et al. (2020). Partitioning heritability analyses unveil the genetic architecture of human brain multidimensional functional connectivity patterns, Hum Brain Mapp. 2020;41:3305–331. https://doi.org/10.1002/hbm.25018
Higgen FL, Heine C, Krawinkel L, Göschl F, Engel AK, Hummel FC, Xue Gui, Gerloff C (2020). Crossmodal Congruency Enhances Performance of Healthy Older Adults in Visual-Tactile Pattern Matching, Frontiers in Aging Neuroscience 12, p. 74, 2020. https://doi.org/10.3389/fnagi.2020.00074
Higgen FL, Ruppel P, Görner M, Kerzel M, Hendrich N, Feldheim J, Wermter S, Zhang J, Gerloff C (2020). Crossmodal Pattern Discrimination in Humans and Robots: A Visuo-Tactile Case Study, Front. Robot. AI 7:540565. https://doi.org/10.3389/frobt.2020.540
Misselhorn J, Göschl F, Higgen FL, Hummel FC, Gerloff C, Engel AK (2020). Sensory capability and information integration independently explain the cognitive status of healthy older adults, Sci Rep. 2020 Dec 31;10(1):22437. https://doi.org/10.1038/s41598-020-80069-8
Xiao X et al. (2020). Individual-specific and shared representations during episodic memory encoding and retrieval, NeuroImage 217 (2020) 11690. https://doi.org/10.1016/j.neuroimage.2020.116909
Ye Z, Shi L, Li A, Chen C, Xue G (2020). Retrieval practice facilitates memory updating by enhancing and differentiating medial prefrontal cortex representations, eLife 2020;9:e57023. https://doi.org/10.7554/eLife.57023

Publications of the first funding period (2016-2019)

Higgen FL, Braaß H, Schulz R, Xue G, Gerloff C (2019). P35 A complex tactile recognition task as a marker for the aging brain. Clinical Neurophysiology 2019; 130(8): e162-e163. 10.1016/j.clinph.2019.04.688
Huang X, Zhang H, Chen C, Xue G, He Q (2019). The neuroanatomical basis of the Gambler’s fallacy: A univariate and multivariate morphometric study. Hum Brain Mapp 40, 967–975. https://doi.org/10.1002/hbm.24425
Zhu B, Chen C, Shao X, Liu W, Ye Z, Zhuang L, Zheng L, Loftus EF, Xue G (2019). Multiple interactive memory representations underlie the induction of false memory. Proceedings of the National Academy of Sciences U S A, 116, 3466-3475.
Cai, Y., Urgolites Z, Wood J, Chen C, Li S, Chen A, Xue G (2018). Distinct neural substrates for visual short-term memory of actions. Hum Brain Mapp 39, 4119–4133. https://doi.org/10.1002/hbm.24236
Xue G (2018). The Neural Representations Underlying Human Episodic Memory. Trends Cogn. Sci. (Regul. Ed.) 22, 544–561. doi: https://doi.org/10.1016/j.tics.2018.03.004
Higgen FL, Ruppel P, Görner M, Kerzel M, Magg S, Hendrich N (2018). Crossmodal pattern discrimination in humans and robots: a visuo-tactile case study. Workshop on Crossmodal Learning, IROS.2018.
Qu J, Qian L, Chen C, Xue G, Li H, Xie P, Mei L (2017). Neural Pattern Similarity in the Left IFG and Fusiform Is Associated with Novel Word Learning. Front Hum Neurosci 11, 424. doi: https://doi.org/10.3389/fnhum.2017.00424

Publications project A4 - Crossmodal joint sparse feature learning in robot tasks

Publications of the current funding period (2020-2024)

Bai K, Zhang L, Chen Z, Wan F, Zhang J (2024). Close the Sim2real Gap via Physically-based Structured Light Synthetic Data Simulation, International Conference of Robotics and Automation (ICRA 2024), Yokohama, Japan.. https://doi.org/10.1109/ICRA57147.2024.10611401
Bi X, Zhong F, Zhang Y, Zhang W, Wang Y (2023). Towards Generalizable Active Object Tracking: Reconstruct Surroundings and Predict Trajectory, Proc. AAAI-2023. https://doi.org/10.1609/aaai.v37i3.25482
Ci H, Liu M, Pan X, Zhong F, Wang Y (2023). Proactive Multi-Camera Collaboration for 3D Human Pose Estimation, Proc ICLR 2023. https://openreview.net/forum?id=CPIy9TWFYBG
Cong L, Ruppel P, Hendrich N, Zhang J (2023). Efficient Human Motion Reconstruction from Monocular Videos with Physical Consistency Loss,, Siggraph Asia 2023. https://doi.org/10.1145/3610548.3618169
Fiedler N, Jonetzko Y, Zhang J (2023). A Multimodal Pipeline for Grasping Fabrics from Flat Surfaces with Tactile Slip and Fall Detection, IEEE International Conference on Robotics and Biomimetics (IEEE ROBIO 2023). http://doi.org/10.1109/ROBIO58561.2023.10354620
Ma X, Wang C, Su J, Zhu W, Zhang J, Wang Y (2023). 3D Human Mesh Estimation via Virtual Markers, Proc. CVPR-2023. https://doi.org/10.48550/arXiv.2303.11726
Wang H, Wang Y, Zhong F, Dong M, Zhang J, Dong H, Wang Y (2023). Semantic-Agnostic and Spatial-Aware Representation for Generalizable Visual-Audio Navigation, Robotics and Automation Letters RA-L, vol 8, no 6, 3900-3907. https://doi.org/10.1109/LRA.2023.3272518
Xiao C, Yang Q, Zhou F, Zhang C (2023). From Text to Mask: Localizing Entities Using the Attention of Text-to-Image Diffusion Models, arXiv preprint, submitted to AAAI'24. https://doi.org/10.48550/arXiv.2309.04109
Xiao C, Yang Q, Xu X, Zhang J, Zhou F, Zhang C (2023). Where You Edit is What You Get: Text-Guided Image Editing with Region-Based Attention., Pattern Recognition, voi 139, 109458 )July 2023).. https://doi.org/10.1016/j.patcog.2023.109458
Ci H, Man X, Wang C, Wang Y (2022). Locally Connected Network for Monocular 3D Human Pose Estimation, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 44-3. https://doi.org/10.1109/TPAMI.2020.3019139
Li L, Hu W, Lu J, Zhang J (2022). Leaf Vein Segmentation with Self-Supervision, Computer and Electronics in Agriculture, Vol 203, 107352, 2022. https://doi.org/10.1016/j.compag.2022.107352
Lu J, Li L, Zhang C (2022). Self-reinforcing Unsupervised Matching, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 44:8, pp. 4404-4418. https://doi.org/10.1109/TPAMI.2021.3061945
Ruppel P, Zhang J (2022). Efficent gradient propagation for robot control and learning, Proceedings ICCCS 2022, Hengqin, Zhuhai, China. https://doi.org/10.1007/978-981-99-2789-0_20
Sun Y, Hu W, Zhou D, Mo B, Fu K, Che Z, Wang Z, Wang S, Zhao J, Ye J, Tang J, Zhang C (2022). Alleviating Data Sparsity Problems in Estimated Time of Arrival via Auxiliary Metric Learning, IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 12, pp. 23231-23243, Dec. 2022. https://doi.org/10.1109/TITS.2022.3200445
Wang Y, Sun Y, Lei F, Zhang C (2022). Leveraging Sparse Coding for EEG Based Emotion Recognition in Shooting, ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022. https://doi.org/10.1109/ICASSP43922.2022.9747506
Wuerkaixi A, Zhang Y, Duan Z, Zhang C (2022). Rethinking Audio-visual Synchronization for Active Speaker Detection, 2022 IEEE 32nd International Workshop on Machine Learning for Signal Processing (MLSP), 2022, pp. 01-06. https://doi.org/10.1109/MLSP55214.2022.9943352
Wuerkaixi A, Yan K, Zhang Y, Duan Z, Zhang C (2022). DyViSE: Dynamic Vision-Guided Speaker Embedding for Audio-Visual Speaker Diarization, 2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP), 2022, pp. 1-6. https://doi.org/10.1109/MMSP55362.2022.9948860
Cao Z, Cui S, Zhang C (2021). DCR: Disentangled component representation for sketch generation, Pattern Recognition Letters, vol. 145, pp. 16-22. https://doi.org/10.1016/j.patrec.2021.01.016
Guo Y, Lu M, Zuo W, Zhang C, Chen Y (2021). Deep Likelihood Network for Image Restoration with Multiple Degradation Levels, IEEE Transactions on Image Processing, vol. 30, pp. 2669-2681. https://doi.org/10.1109/TIP.2021.3051767
Ruppel P, Görner M, Hendrich N, Zhang J (2021). Detection and Reconstruction of Transparent Objects with Infrared Projection-Based RGB-D Cameras, In: Sun F, Liu H, Fang B (eds) Cognitive Systems and Signal Processing. ICCSIP 2020 Communications in Computer and Information Science, vol 1397. https://doi.org/10.1007/978-981-16-2336-3_53
Ruppel P, Hendrich N, Zhang J (2021). Direct Policy Optimization with Differentiable Physical Consistency for Dexterous Manipulation, 2021 IEEE International Conference on Robotics and Biomimetics (ROBIO), 2021, pp. 650-655. https://doi.org/10.1109/ROBIO54168.2021.9739435
Sun Y, Fu K, Wang Z, Zhang C, Ye J (2021). Road Network Metric Learning for Estimated Time of Arrival, The 25th International Conference on Pattern Recognition (ICPR) 2021. https://doi.org/10.1109/ICPR48806.2021.9412145
Sun Y, Wang Y, Fu K, Wang Z, Zhang C, Ye J (2021). Constructing Geographic and Long-term Temporal Graph for Traffic Forecasting, 25th International Conference on Pattern Recognition (ICPR), pp. 3483-3490. https://doi.org/10.1109/ICPR48806.2021.9412506
Sun Y, Wang Y, Fu K, Wang Z, Yan Z, Zhang C, Ye J (2021). FMA-ETA: Estimating Travel Time Entirely Based on FFN With Attention, 46th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3355-3359. https://doi.org/10.1109/ICASSP39728.2021.9414054
Zhong F, Sun P, Luo W, Yan T, Wang Y (2021). Towards Distraction-Robust Active Visual Tracking, Proc. 38th International Conference on Machine Learning, PMLR 139. https://doi.org/10.48550/arXiv.2106.10110
Cao Z, Lu J, Cui S, Zhang C (2020). Zero-Shot Handwritten Chinese Character Recognition with Hierarchical Decomposition Embedding, Pattern Recognition, vol. 107, 107488, 2020. https://doi.org/10.1016/j.patcog.2020.107488
Guo Y, Chen L, Chen Y, Zhang C (2020). On Connections between Regularizations for Improving DNN Robustness, IEEE Transactions on Pattern Analysis and Machine Intelligence, 43:12, 4469. https://doi.org/10.1109/TPAMI.2020.3006917
Güldenring R, Görner M, Hendrich N, Jacobsen NJ, Zhang J (2020). Learning Local Planners for Human-aware Navigation in Indoor Environments, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, pp. 6053-6060. https://doi.org/10.1109/IROS45743.2020.9341783
Sun X, Xu Y, Cao P, Kong Y, Hu L, Zhang Sh, Wang Y (2020). TCGM: An Information-Theoretic Framework for Semi-supervised Multi-modality Learning, Proceedings ECCV 2020. Lecture Notes in Computer Science, vol 12348, Springer. https://doi.org/10.1007/978-3-030-58580-8_11
Xie R, Wang C, Wang Y (2020). MetaFuse: A Pre-trained Fusion Model for Human Pose Estimation, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020), pp. 13683-13692. https://doi.org/10.1109/CVPR42600.2020.01370
Xu J, Zhong F, Wang Y (2020). Learning Multi-Agent Coordination for Enhancing Target Coverage in Directional Sensor Networks, NeurIPS 2020.

Publications of the first funding period (2016-2019)

Liang, H., Li, S., Ma, X., Hendrich, N., Gerkmann T., Sun, F., & Zhang, J. (2019). Making sense of Audio Vibration for Liquid Height Estimation in Robotic Pouring. International Conference on Intelligent Robots and Systems (IROS), Macau, China.
Yu, X., Guo, T., Fu, K, Li, L, Zhang, C. and Zhang J. (2019). Image Captioning with Partially Rewarded Imitation Learning. International Joint Conference on Neural Network (IJCNN), Budapest, Hungary.
Tang, S., Ji, Y., Lyu, J., Mi, J., Li, Q., & Zhang, J. (2019). Visual Domain Adaptation Exploiting Confidence-Samples. International Conference on Intelligent Robots and Systems (IROS), Macau, China.
Chang, D., Lin, M., & Zhang, C. (2018). On the Generalization Ability of Online Gradient Descent Algorithm Under the Quadratic Growth Condition. IEEE Transactions on Neural Networks and Learning Systems, 29(10), 1-12.
Lu R., Duan Z. and Zhang C. (2018). Listen and Look: Audio–Visual Matching Assisted Speech Source Separation. IEEE Signal Processing Letters.
Lu, R., Duan, Z., & Zhang, C. (2018). Multi-Scale Recurrent Neural Network for Sound Event Detection. 2018 In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 131-135.
Lu, J., Li, J., Yan, Z., Mei, F., & Zhang, C. (2018). Attribute-Based Synthetic Network (ABS-Net): Learning more from pseudo feature representations. Pattern Recognition, 80, 129-142.
Lu, J., Cao, Z., Wu, K., Zhang, G., & Zhang, C. (2018). Boosting Few-Shot Image Recognition Via Domain Alignment Prototypical Networks. In 2018 IEEE 30th International Conference on Tools with Artificial Intelligence (ICTAI), 260-264.
Yan, C., Wu, K., & Zhang, C. (2018). A New Anchor-Labeling Method For Oriented Text Detection Using Dense Detection Framework. IEEE Signal Processing Letters, 25(9), 1295-1299.
Yan, C., Hu, J., & Zhang, C. (2018). Deep transformer: A framework for 2D text image rectification from planar transformations. Neurocomputing, 289, 32-43.
Chu, D., Lu, R., Li, J., Yu, X., Zhang, C., & Tao, Q. (2018). Optimizing Top-k Multiclass SVM via Semismooth Newton Algorithm. IEEE Transactions on Neural Networks and Learning Systems, 29(12), 6264-6275.
Fu, K., Li, J., Jin, J., & Zhang, C. (2018). Image-text surgery: Efficient concept learning in image captioning by generating pseudopairs. IEEE Transactions on Neural Networks and Learning Systems, (99), 1-12.
Yu, N., Hu, X., Song, B., Yang, J., & Zhang, J. (2018). Topic-Oriented Image Captioning Based on Order-Embedding. IEEE Transactions on Image Processing, 28(6), 2743-2754.
Ruppel, P., Jonetzko, Y., Görner, M., Hendrich, N., & Zhang, J.(2018). Simulation of the SynTouch BioTac Sensor. International Conference on Intelligent Autonomous Systems. DOI: 10.1007/978-3-030-01370-7_30
Ruppel, P., Hendrich, N., Starke, S., & Zhang, J. (2018). Cost functions to specify full-body motion and multi-goal manipulation tasks. In 2018 IEEE International Conference on Robotics and Automation (ICRA), 3152-3159.
Wu, K., & Zhang, C. (2018). Deep Generative Adversarial Networks for the Sparse Signal Denoising. In 2018 24th International Conference on Pattern Recognition (ICPR), 1127-1132.
Chu, D., Zhang, C., & Tao, Q. (2017). A faster cutting plane algorithm with accelerated line search for linear SVM. Pattern Recognition, 67, 127-138.
Liu, L., He, B., Zhuang, J., Zhang, L., & Lv, A. (2017). Force measurement system for invisalign based on thin film single force sensor. Measurement, 97, 1-7.
Hu, W., Zhuo, Q., Zhang, C., & Li, J. (2017). Fast branch convolutional neural network for traffic sign recognition. IEEE Intelligent Transportation Systems Magazine, 9(3), 114-126.
Lu, J., Hu, J., Zhao, G., Mei, F., & Zhang, C. (2017). An in-field automatic wheat disease diagnosis system. Computers and Electronics in Agriculture, 142, 369-379.
Lu, R., Wu, K., Duan, Z., & Zhang, C. (2017, March). Deep ranking: Triplet matchnet for music metric learning. In 2017 IEEE International Confer ence on Acoustics, Speech and Signal Processing (ICASSP), 121-125.
Cui, R., Liu, H., & Zhang, C. (2017). Recurrent convolutional neural networks for continuous sign language recognition by staged optimization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 7361-7369.
Tang, S., Chen, L., Mi, J., Ye, M., Li, Q., & Zhang, J. (2017). Adaptive pedestrian detection by modulating features in dynamical environment. IEEE International Conference on Robotics and Biomimetics (ROBIO), 62-67.
Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., & Zhang, C. (2017). Learning efficient convolutional networks through network slimming. In Proceedings of the IEEE International Conference on Computer Vision（ICCV）. 2736-2744.
Fu, K., Jin, J., Cui, R., Sha, F., & Zhang, C. (2016). Aligning where to see and what to tell: image captioning with region-based attention and scene-specific contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), 2321-2334.
Guan, H., & Zhang, J. (2016). Multi-sensory based novel household object categorization system by using interactive behaviours. In 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO), 1685-1690.
Zhang, L., He, B., Zhang, J., (2016). Multimodal Information Representation Based on Approximation Theory (in Chinese). Science Press, ISBN: 978-7-03-050253-7.

Publications project A5 - Crossmodal learning in a neurobotic cortical and midbrain model

Publications of the current funding period (2020-2024)

Abawi F, Allgeuer P, Fu D, Wermter S (2024). Wrapyfi: A Python Wrapper for Integrating Robots, Sensors, and Applications across Multiple Middleware, Proceedings of the 2024 ACM/IEEE Conference on Human-Robot Interaction (HRI’24), 860-864. https://doi.org/10.1145/3610977.3637471
Cao S, Fu D, Yang X, Wermter S, Liu X, Wu H (2024). Pain Recognition and Pain Empathy from a Human-Centered AI Perspective, iScience, 27:8, 110570. https://doi.org/10.1016/j.isci.2024.110570
Qu L, Wang W, Weber C, Yue P, Li T, Wermter S (2024). Improving Speech Emotion Recognition with Unsupervised Speaking Style Transfer, Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, 10101-10105. https://doi.org/10.1109/ICASSP48485.2024.10446186
Ali H, Jirak D, Wermter S (2023). Snapture--a Novel Neural Architecture for Combined Static and Dynamic Hand Gesture Recognition, Cognitive Computation. https://doi.org/10.1007/s12559-023-10174-z
Becker D, Rueda D, Beese F, Torres BSG, Lafdili M, Ahrens K, Fu D, Strahl E, Weber T, Wermter S (2023). The Emotional Dilemma: Influence of a Human-Like Robot on Trust and Cooperation, Proceedings of the 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 1689-1696. https://doi.org/10.1109/RO-MAN57019.2023.10309321
Chen Z, Fu D, Liu X (2023). Better to Misidentify than to Miss: A Review of Occurrence Mechanisms and Applications of Face Pareidolia (in Chinese)., Advances in Psychological Science, 31(2), 240-255. https://doi.org/10.3724/SP.J.1042.2023.00240
Fu D, Abawi F, Wermter S (2023). The Robot in the Room: Influence of Robot Facial Expressions and Gaze on Human-Human-Robot Collaboration, IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 85-91. https://doi.org/10.1109/RO-MAN57019.2023.10309334
Qu L, Li T, Weber C, Pekarek-Rosin T, Ren F, Wermter S (2023). Disentangling Prosody Representations with Unsupervised Speech Reconstruction., IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32, 39-54. https://doi.org/10.1109/TASLP.2023.3320864
Cao S, Fu D, Yang X, Barros P, Wermter S, Liu X, Wu H (2022). How Can AI Recognize Pain and Express Empathy, arXiv preprint arXiv:2110.04249. (in progress). https://doi.org/10.48550/arXiv.2110.04249 (preprint)
Duczek N, Kerzel M, Allgeuer P, Wermter S (2022). Self-organised Learning from Synthetic and Real Data for a Humanoid Exercise Robot, Frontiers in Robotics and AI 9:669719. https://doi.org/10.3389/frobt.2022.6697
Eppe M, Gumbsch C, Kerzel M et al. (2022). Intelligent problem-solving as integrated hierarchical reinforcement learning, Nature Machine Intelligence 4, 11–20 (2022). https://doi.org/10.1038/s42256-021-00433-9
Fu D, Abawi F, Carneiro H, Kerzel M, Chen Z, Strahl E, Liu X, Wermter S (2022). A trained humanoid robot can perform human-like crossmodal social attention conflict resolution, International Journal of Social Robotics, 15, 1325-1340. https://doi.org/10.1007/s12369-023-00993-3
Fu D, Abawi F, Strahl E, Wermter S (2022). Judging by the look: The impact of robot gaze strategies on human cooperation., IEEE RO-MAN 2022, Workshop on Machine Learning for HRI: Bridge the Gap between Action and Perception. https://doi.org/10.48550/arXiv.2208.11647
Qu L, Weber C, Wermter S (2022). LipSound2: Self-Supervised Pre-Training for Lip-to-Speech Reconstruction and Lip, IEEE Trans Neural Netw Learn Syst. 2022. https://doi.org/10.1109/TNNLS.2022.3191677.
Schwiebert G, Weber C, Qu L, Siqueira H, Wermter S (2022). A Multimodal German Dataset for Automatic Lip Reading Systems and Transfer Learning, Proceedings of the 13th Language Resources and Evaluation Conference (LREC), pp. 6829-6836. https://aclanthology.org/2022.lrec-1.737
Zhao X, Weber C, Hafez MB, Wermter S (2022). Impact Makes a Sound and Sound Makes an Impact: Sound Guides Representations and Explorations, 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 2512-2518. https://doi.org/10.1109/IROS47612.2022.9981510
Abawi F, Weber T, Wermter S (2021). GASP: Gated Attention for Saliency Prediction, In Proceedings of The Thirtieth International Joint Conference on Artificial Intelligence (IJCAI), 584-591. https://doi.org/10.24963/ijcai.2021/81
Carneiro H, Weber C, Wermter S (2021). FaVoA: Face-Voice Association Favours Ambiguous Speaker Detection, In Proceedings of The Thirtieth International Conference on Artificial Neural Networks, 439-450. https://doi.org/10.1007/978-3-030-86362-3_36
Plüster B, Weber C, Qu L, Wermter S (2021). Hearing Faces: Target Speaker Text-To-Speech Synthesis from a Face, 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2021, pp. 757-764. https://doi.org/10.1109/ASRU51503.2021.9687866
Andriella A, Siqueira H, Fu D, Magg S, Barros P, Wermter S, Torras C, Alenya G (2020). Do I have a personality? Endowing care robots with context-dependent personality traits, International Journal of Social Robotics (2021) 13:2081–2102. https://doi.org/10.1007/s12369-020-00690-5
Fu D, Weber C, Yang G, Kerzel M, Nan W, Barros P, Wu H, Liu X, Wermter S (2020). What Can Computational Models Learn From Human Selective Attention? A Review From an Audiovisual Unimodal and Crossmodal Perspective, Front. Integr. Neurosci. 14:10. https://doi.org/10.3389/fnint.2020.00010
Qu L, Weber C, Wermter S (2020). Multimodal Target Speech Separation with Voice and Face References, Proc. Interspeech 2020, 1416-1420. http://doi.org/10.21437/Interspeech.2020-1697
Weber T, Wermter S (2020). Integrating Intrinsic and Extrinsic Explainability: The Relevance of Understanding Neural Networks for Human-Robot Interaction, The AI-HRI Symposium at AAAI-FSS, also arXiv:2010:04602. https://doi.org/10.48550/arXiv.2010.04602

Publications of the first funding period (2016-2019)

Marzouk, A., Barros, P., Eppe, M., & Wermter, S. (2019). The Conditional Boundary Equilibrium Generative Adversarial Network and its Application to Facial Attributes. Proceedings of The International Joint Conference on Neural Networks IJCNN-2019, 1–7.
Barros, P., Fliesswasser, E., Kerzel, M., & Wermter, S. (2019). Exploring Low-level and High-level Transfer Learning for Multi-task Facial Recognition with a Semi-supervised Neural Network. Proceedings of The IEEE/RSJ International Conference on Intelligent Robots and Systems IROS-2019, 1378–1384.
Mici, L., Parisi, G.I., & Wermter, S. (2019). Compositional Learning of Human Activities With a Self-Organizing Neural Architecture. Frontiers in Robotics and AI, 6.
Barros, P., Churamani, N., Lim, A., & Wermter, S. (2019). The OMG-Empathy Dataset: Evaluating the Impact of Affective Behavior in Storytelling. Proceedings of The Eighth IEEE International Conference on Affective Computing and Intelligent Interaction ACII-2019, 1–7.
Barros, P., Eppe, M., Parisi, G. I., Liu, X., & Wermter, S. (2019). Expectation Learning for Stimulus Prediction Across Modalities Improves Unisensory Classification. Frontiers in Robotics and AI, 6, 137.
Barros, P., Wermter, S., & Sciutti, A. (2019). Towards Learning How to Properly Play UNO with the iCub Robot. In Proceedings of the ICDL-EPIROB Workshop on Naturalistic Non-verbal and Affective Human-Robot Interactions. arXiv:1908.00744.
Barros, P., Parisi, G., & Wermter, S. (2019). A Personalized Affective Memory Model for Improving Emotion Recognition. In International Conference on Machine Learning (pp. 485-494).
Parisi, G., Barros, P., Fu, D., Magg, S., Wu, H., Liu, X., & Wermter, S. (2019). A Neurorobotic Experiment for Crossmodal Conflict Resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 33-36).
Lindt, A., Barros, P., Siqueira, H., & Wermter, S. (2019). Facial Expression Editing with Continuous Emotion Labels. In 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019) (pp. 1-8). IEEE.
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019). Continual lifelong learning with neural networks: A review. Neural Networks.113，54-71.
Barros, P., Parisi, G. I., Fu, D., Liu, X., & Wermter, S. (2018). Expectation learning for adaptive crossmodal stimuli association. EUCognition Meeting, arXiv preprint arXiv:1801.07654.
Fu, D., Barros, P., Parisi, G. I., Wu, H., Magg, S., Liu, X., & Wermter, S. (2018). Assessing the Contribution of Semantic Congruency to Multisensory Integration and Conflict Resolution. arXiv preprint arXiv:1810.06748.
Barros, P., Parisi, G. I., Fu, D., Liu, X., & Wermter, S. (2018). Expectation Learning and Crossmodal Modulation with a Deep Adversarial Network. In 2018 International Joint Conference on Neural Networks (IJCNN), 1-8.
Mici, L., Parisi, G. I., & Wermter, S. (2018). An incremental self-organizing architecture for sensorimotor learning and prediction. IEEE Transactions on Cognitive and Developmental Systems, 10(4), 918-928
Cruz, F., Parisi, G. I., & Wermter, S. (2018). Multi-modal Feedback for Affordance-driven Interactive Reinforcement Learning. In 2018 International Joint Conference on Neural Networks (IJCNN), 1-8.
Churamani, N., Barros, P., Strahl, E., & Wermter, S. (2018). Learning empathy-driven emotion expressions using affective modulations. In 2018 International Joint Conference on Neural Networks (IJCNN) (pp. 1-8).
Parisi, G. I., Tong, J., Barros, P., Röder, B., & Wermter, S. (2018). Towards Modeling the Interaction of Spatial-Associative Neural Network Representations for Multisensory Perception. Workshop on Computational Models for Crossmodal Learning (ICDL-EpiRob).
Parisi, G. I., Barros, P., Fu, D., Magg, S., Wu, H., Liu, X., & Wermter, S. (2018). A neurorobotic experiment for crossmodal conflict resolution in complex environments. 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2330-2335.
Parisi, G.I., Barros, P., Kerzel, M., Wu, H., Yang, G., Li, Z., Liu, X., Wermter, S. (2017). A computational model of crossmodal processing for conflict resolution. Proc. International Conference on Development and Learning and on Epigenetic Robotics (ICDL-EPIROB), pages 33-38.
Parisi, G.I., Wermter, S. (2017). Lifelong learning of action representations with deep neural self-organization. AAAI Spring Symposium Series, pages 608-612.
Parisi, G.I., Barros, P., Wu, H., Yang, G., Li, Z., Liu, X., Wermter, S. (2017). A deep neural model for emotion-driven multimodal attention. AAAI Spring Symposium Series, pages 482-485.
Barros, P., Parisi, G.I., Wermter, S. (2017). Emotion-modulated attention improves expression recognition: A deep learning model. Neurocomputing, Volume 253, pages 104-114.
Barros, P., & Wermter, S. (2017).A self-organizing model for affective memory. International Joint Conference on Neural Networks (IJCNN), pp. 31-38.
Churamani, N., Kerzel, M., Strahl, E., Barros, P., & Wermter, S. (2017). Teaching emotion expressions to a human companion robot using deep neural architectures. International Joint Conference on Neural Networks (IJCNN), pp. 627-634.
Tsironi, E., Barros, P., Weber, C., & Wermter, S. (2017). An analysis of Convolutional Long Short-Term Memory Recurrent Neural Networks for gesture recognition. Neurocomputing.
Parisi, G.I., Tani, J., Weber, C., Wermter, S. (2017). Emergence of Multimodal Action Representations from Neural Network Self-Organization. Cognitive Systems Research
Cruz, F., Parisi, G. I., and Wermter, S. (2016). Learning Contextual Affordances with an Associative Neural Architecture. Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), pp. 665-670, Bruges, Belgium, 2016.
Parisi, G.I., Wermter, S. A Neurocognitive Robot Assistant for Robust Event Detection. (2016). Trends in Ambient Intelligent Systems: Role of Computational Intelligence, Series "Studies in Computational Intelligence", pp. 1-28, Springer, 2016
Cruz, F., Parisi, G. I., & Wermter, S. (2016). Learning Contextual Affordances with an Associative Neural Architecture. Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), 665-670, Bruges, Belgium.
Speck, D., Barros, P., Weber, C. and Wermter, S. (2016). Ball Localization for Robocup Soccer using Convolutional Neural Networks. RoboCup Symposium, Leipzig, Germany, 2016. - Best Paper Award
Barros, P. , Weber, C., Wermter, S. (2016). Learning Auditory Representations for Emotion Recognition. Proceedings of International Joint Conference on Neural Networks (IJCNN/WCCI), Vancouver, Canada, July 2016.
Mousavi, N., Siqueira, H., Barros, P., Fernandes, B., Wermter, S. (2016). Understanding How Deep Neural Networks Learn Face Expressions. Proceedings of International Joint Conference on Neural Networks (IJCNN/WCCI), Vancouver, Canada, July 2016.

Publications project A6 - Brain-inspired multimodal deep learning

Publications of the current funding period (2020-2024)

Kelm AP, Hannemann N, Heberle B, Schmidt L, Rolff T, Wilms C, Yaghoubi E, Frintrop S (2024). Dynamic Inference and Top-Down Attention in a Hierarchical Classification Network, 27th International Conference on Pattern Recognition (ICPR 2024), Lecture Notes in Computer Science, vol. 15328. https://doi.org/10.1007/978-3-031-78186-5_15
Lay B, Lemercier J-M, Richter J, Gerkmann T (2024). Single and Few-step Diffusion for Generative Speech Enhancement, IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Seoul, South Korea, pp. 626-630. https://doi.org/10.1109/ICASSP48485.2024.10447860
Richter J, Wu Y-C, Krenn S, Welker S, Lay B, Watanabe S, Richard A, Gerkmann T (2024). EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberation, ISCA Interspeech, Kos, Greece, pp. 4873-4877. https://www.doi.org/10.21437/Interspeech.2024-153
Richter J, Welker S, Lemercier J-M, Lay B, Peer T, Gerkmann T (2024). Causal Diffusion Models for Generalized Speech Enhancement, IEEE Open Journal of Signal Processing, vol. 5, pp. 780-789. https://doi.org/10.1109/OJSP.2024.3379070
Yaghoubi E, Tran TK, Borza D, Frintrop S (2024). Attention-Based Fusion of Intra- and Intermodal Dynamics in Multimodal Sentiment Analysis, 2024 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), pp. 273-278. https://doi.org/10.1109/PerComWorkshops59983.2024.10502594
Borza DL, Yaghoubi E, Frintrop S, Proença H (2023). Adaptive Spatial Transformation Networks for Periocular Recognition, Sensors 2023, 23(5), 2456. https://doi.org/10.3390/s23052456
Fang H, Frintrop S, Gerkmann T (2023). Uncertainty-Driven Hybrid Fusion for Audio-Visual Phoneme Recognition, ITG Speech Communication, Aachen, Germany, Sep. 2023.
Hu Z, Chu W, Zhu X, Zhang H, Zhang B, Hu X (2023). Physically realizable natural-looking clothing textures evade person detectors via 3d modeling, 36th IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR52729.2023.01628
Kelm AP, Schmidt L, Roff T, Wilms C, Yaghoubi E, Frintrop S (2023). High-Level Features Parallelization for Inference Cost Reduction Through Selective Attention, arxiv preprint 2308.05128v3. https://doi.org/10.48550/arXiv.2308.05128
Kelm AP, Hannemann N, Heberle B, Schmidt L, Rolff T, Wilms C, Yaghoubi E, Frintrop S (2023). Select High-Level Features: Efficient Experts from a Hierarchical Classification Network, 5th Workshop on Practical ML for Limited/Low Resource Settings (PML4LRS) at ICLR, Vienna, Austria. https://openreview.net/pdf?id=A3BS8npfMV
Lay B, Welker S, Richter J, Gerkmann T (2023). Reducing the Prior Mismatch of Stochastic Differential Equations for Diffusion-based Speech Enhancement, ISCA Interspeech, pp. 3809-3813, Dublin, Ireland, Aug. 2023.. https://doi.org/10.21437/Interspeech.2023-1445
Lemercier J-M, Richter J, Welker S, Gerkmann T (2023). Analysing Diffusion-based Generative Approaches versus Discriminative Approaches for Speech Restoration, IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Rhodes, Greece, Jun. 2023.. https://doi.org/10.1109/ICASSP49357.2023.10095258
Lemercier J-M, Richter J, Welker S, Gerkmann T (2023). StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation, IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 2724-2737, 2023.. https://doi.org/10.1109/TASLP.2023.3294692
Li K, Yang R, Hu X (2023). An efficient encoder-decoder architecture with top-down attention for speech separation, 11th International Conference on Learning Representations ICLR 2023, paper 3495. http://dx.doi.org/10.48550/arXiv.2209.15200
Li X, Wang Z, Zhang B, Sun F, Hu X (2023). Recognizing Object by components with human prior knowledge enhances adversarial robustness of deep neural networks, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 45, no. 7, pp. 8861-8873. https://doi.org/10.1109/TPAMI.2023.3237935
Martel H, Richter J, Li K, Hu X, Gerkmann T (2023). Audio-Visual Speech Separation in Noisy Environments with a Lightweight Iterative Model, ISCA Interspeech, Dublin, Ireland, pp. 1673-1677, Aug. 2023.. https://doi.org/10.21437/Interspeech.2023-1753
Pegg S, Li K, Hu X (2023). TDFNet: An Efficient Audio-Visual Speech Separation Model with Top-down Fusion, 13th International Conference on Information Science and Technology (ICIST 2023). https://doi.org/10.1109/ICIST59754.2023.10367130
Richter J, Welker S, Lemercier J-M, Lay B, Gerkmann T (2023). Speech Enhancement and Dereverberation with Diffusion-Based Generative Models, IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, 2351-2364. https://doi.org/10.1109/TASLP.2023.328524
Richter J, Welker S, Lemercier J-M, Lay B, Peer T, Gerkmann T (2023). Speech Signal Improvement Using Causal Generative Diffusion Models, IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Rhodes, Greece. https://doi.org/10.1109/ICASSP49357.2023.10095126
Richter J, Frintrop S, Gerkmann T (2023). Audio-Visual Speech Enhancement with Score-Based Generative Models, ITG Conference on Speech Communication, Aachen, Germany, Sept. 2023.a. https://doi.org/10.48550/arXiv.2306.01432
Tang C, Xie L, Zhang X, Hu X, Tian Q (2023). Visual recognition by request, 36th IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR52729.2023.01465
Xu J, Zhang Z, Hu X (2023). Extracting semantic knowledge from gans with unsupervised learning, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 45, no. 8, pp. 9654-9668. https://doi.org/10.1109/TPAMI.2023.3262140
Yaghoubi E, Kelm AP, Gerkmann T, Frintrop S (2023). Acoustic and visual knowledge distillation for contrastive audio-visual localization, International Conference on Multimodal Interaction, pages 15–23, 2023..
de Oliveira D, Richter J, Lemercier J-M, Peer T, Gerkmann T (2023). On the Behavior of Intrusive and Non-intrusive Speech Enhancement Metrics in Predictive and Generative Settings, ITG Conference on Speech Communication, Aachen, Germany, pp. 260-264. https://doi.org/10.30420/456164051
Chen H, Tang C, Hu X (2022). Dense Contrastive Loss for Instance Segmentation, British Machine Vision Conference (BMVC) 2022. https://bmvc2022.mpi-inf.mpg.de/1062.pdf
Chen H, Li J, Frintrop S, Hu X (2022). The MSR-Video to Text dataset with clean annotations, Computer Vision and Image Understanding. 2022 Dec 1;225:103581. https://doi.org/10.1016/j.cviu.2022.103581
Hu X, Zeng Z (2022). Bridging the Functional and Wiring Properties of V1 Neurons Through Sparse Coding, Neural Computation. 2022 Jan 1;34(1):104-37. https://doi.org/10.1162/neco_a_01453
Hu Z, Huang S, Zhu X, Sun F, Zhang B, Hu X (2022). Adversarial Texture for Fooling Person Detectors in the Physical World., Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2022, pp. 13307-13316. https://doi.org/10.1109/CVPR52688.2022.01295
Hu Z, Zhu J, Zhang B, Hu X (2022). Amplification trojan network: Attack deep neural networks by amplifying their inherent weakness, Neurocomputing. 2022 Sep 21;505:142-53. https://doi.org/10.1016/j.neucom.2022.07.018
Hu X, Tang C, Chen H, Li X, Li J, Zhang Z (2022). Improving Image Segmentation with Boundary Patch Refinement, International Journal of Computer Vision. 2022 Nov;130(11):2571-89. https://doi.org/10.1007/s11263-022-01662-0
Hu Z, Huang S, Zhu X, Sun F, Zhang B, Hu X (2022). Adversarial Texture for Fooling Person Detectors in the Physical World, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 13307-13316. https://doi.org/10.1109/CVPR52688.2022.01295
Kuo TY, Liao Y, Li K, Hong B, Hu X (2022). Inferring Mechanisms of Auditory Attentional Modulation with Deep Neural Networks., Neural Computation 34(11):2273-93. https://doi.org/10.1162/neco_a_01537
Richter J, Liebold J, Gerkmann T (2022). Continuous Phoneme Recognition based on Audio-Visual Modality Fusion, 2022 International Joint Conference on Neural Networks (IJCNN), 2022, pp. 1-8. https://doi.org/10.1109/IJCNN55064.2022.9892053
Tang C, Xie L, Zhang G, Zhang X, Tian Q, Hu X (2022). Active Pointly-Supervised Instance Segmentation, European Conference on Computer Vision 2022, pp. 606-623. https://doi.org/10.1007/978-3-031-19815-1_35
Wang Z, Wang K, Hu X (2022). Accelerating Allen Brain Institute’s Large-Scale Computational Model of Mice Primary Visual Cortex., CAAI International Conference on Artificial Intelligence 2022, pp. 610-614. https://doi.org/10.1007/978-3-031-20503-3_57
Welker S, Richter J, Gerkmann T (2022). Speech Enhancement with Score-Based Generative Models in the Complex STFT Domain, Proc. Interspeech 2022, 2928-2932. https://doi.org/10.21437/Interspeech.2022-10653
Zhao Z, Liu Y, Zhang G, Tang L, Hu X (2022). The Winning Solution to the iFLYTEK Challenge 2021 Cultivated Land Extraction from High-Resolution Remote Sensing Image, 2022 14th International Conference on Advanced Computational Intelligence (ICACI), 2022, pp. 376-380. https://doi.org/10.1109/ICACI55529.2022.9837765
Zhu X, Hu Z, Huang S, Li J, Hu X (2022). Infrared Invisible Clothing: Hiding from Infrared Detectors at Multiple Angles in Real World, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2022, pp. 13317-13326. https://doi.org/10.1109/CVPR52688.2022.01296
Banik S, Lauri M, Knoll A, Frintrop S (2021). Object Localization with Attribute Preference Based on Top-Down Attention., International Conference on Computer Vision Systems 2021 (ICCVS), pp. 28-40. https://doi.org/10.1007/978-3-030-87156-7_3
Carbajal G, Richter J, Gerkmann T (2021). Guided Variational Autoencoder for Speech Enhancement with a Supervised Classifier, 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 681-685. https://doi.org/10.1109/ICASSP39728.2021.9414363
Carbajal G, Richter J, Gerkmann T (2021). Disentanglement Learning for Variational Autoencoders Applied to Audio-Visual Speech Enhancement, 2021 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2021, pp. 126-130. https://doi.org/10.1109/WASPAA52581.2021.9632676
Caus D, Carbajal G, Gerkmann T, Frintrop S (2021). See the Silence: Improving Visual-Only Voice Activity Detection by Optical Flow and RGB Fusion., Computer Vision Systems. ICVS 2021. Lecture Notes in Computer Science, vol 12899. https://doi.org/10.1007/978-3-030-87156-7_4
Fang H, Carbajal G, Wermter S, Gerkmann T (2021). Variational Autoencoder for Speech Enhancement with a Noise-Aware Encoder, 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2021), pp. 676-680. https://doi.org/10.1109/ICASSP39728.2021.9414060
Fang H, Carbajal G, Wermter S, Gerkmann T (2021). Joint Reduction of Ego-noise and Environmental Noise with a Partially-adaptive Dictionary, ITG Conference on Speech Communication, Kiel, Germany, Sep. 2021. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9657516
Gao G, Lauri M, Hu X, Zhang J, Frintrop S (2021). CloudAAE: Learning 6D object pose regression with on-line data synthesis on point clouds, 2021 IEEE International Conference on Robotics and Automation (ICRA) 2021 pp. 11081-11087. https://doi.org/10.1109/ICRA48506.2021.9561475
Hu X, Li K, Zhang W, Luo Y, Lemercier JM, Gerkmann T (2021). Speech separation using an asynchronous fully recurrent convolutional neural network, Advances in Neural Information Processing Systems (NeurIPS) 2021 Dec 6;34:22509-22. https://openreview.net/forum?id=SlxH2AbBBC2
Li X, Li J, Dai T, Shi J, Zhu J, Hu X (2021). Rethinking Natural Adversarial Examples for Classification Models, preprint arXiv:2102.1173. https://doi.org/10.48550/arXiv.2102.11731
Liu H, Zhang S, Lin K, Wen J, Li J, Hu X (2021). Vocabulary-Wide Credit Assignment for Training Image Captioning Models, IEEE Transactions on Image Processing, vol. 30, pp. 2450-2460, 2021. https://doi.org/10.1109/TIP.2021.3051476.
Nguyen Q, Richter J, Lauri M, Gerkmann T, Frintrop S (2021). Improving mix-and-separate training in audio-visual sound source separation with an object prior, 25th International Conference on Pattern Recognition (ICPR), 2021, pp. 5844-5851. https://doi.org10.1109/ICPR48806.2021.9412174
Tang C, Chen H, Li X, Li J, Zhang Z, Hu X (2021). Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13921-13930. https://doi.org/10.1109/CVPR46437.2021.01371
Wang J, Hu X (2021). Convolutional neural networks with gated recurrent connections, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol 44, pp 3421-3435. https://doi.org/10.1109/TPAMI.2021.3054614
Wilms C, Frintrop S (2021). DeepFH segmentations for superpixel-based object proposal refinement., Image and Vision Computing. 2021 Oct 1;114:104263. https://doi.org/10.1016/j.imavis.2021.104263
Xu J, Xiong Z, Hu X (2021). Frame difference-based temporal loss for video stylization, preprint arXiv:2102.05822. https://doi.org/10.48550/arXiv.2102.05822
Zhang G, Lu X, Tan J, Li J, Zhang Z, Li Q, Hu X (2021). RefineMask: Towards high-quality instance segmentation with fine-grained features, Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2021, pp. 6861-6869. https://doi.org/10.1109/CVPR46437.2021.00679
Zhu X, Li X, Li J, Wang Z, Hu X (2021). Fooling thermal infrared pedestrian detectors in real world using small bulbs, Proceedings of the AAAI Conference on Artificial Intelligence 2021, Vol. 35, No. 4, pp. 3616-3624. https://doi.org/10.1609/aaai.v35i4.16477
Chen H, Lin K, Maye A, Li J, Hu X (2020). A Semantics-Assisted Video Captioning Model Trained with Scheduled Sampling, Frontiers in Robotics and AI, September 30, 2020. https://doi.org/10.3389/frobt.2020.475767
Gao G, Lauri M, Wang Y, Hu X, Zhang J, Frintrop S (2020). 6D object pose regression via supervised learning on point clouds, 2020 IEEE International Conference on Robotics and Automation (ICRA) 2020, pp. 3643-3649. https://doi.org/10.1109/ICRA40945.2020.9197461
Lauri M, Pajarinen J, Peters J, Frintrop S (2020). Multi-sensor next-best-view planning as matroid-constrained submodular maximization, IEEE Robotics and Automation Letters 7;5(4):5323-30, 2020. https://doi.org/10.1109/LRA.2020.3007445
Richter J, Carbajal G, Gerkmann T (2020). Speech Enhancement with Stochastic Temporal Convolutional Networks, Proc. Interspeech 2020, 4516-4520. https://doi.org/10.21437/Interspeech.2020-2588

Publications of the first funding period (2016-2019)

Feng, W., Liu, W., Li, T., Peng, J., Qian, C., & Hu, X. (2019). Turbo Learning Framework for Human-Object Interactions Recognition and Human Pose Estimation. The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI), Honolulu, Hawaii, USA.
Yu, M., Weber, C., Hu, X. (2019). Learning Sparse Hidden States in Long Short-Term Memory. In Proc. of the International Conference on Artificial Neural Networks (ICANN2019)
Liao, F., Liang, M., Li, Z., Hu, X., & Song, S. (2019). Evaluate the Malignancy of Pulmonary Nodules Using the 3-D Deep Leaky Noisy-or Network. IEEE transactions on neural networks and learning systems.
Liu, W., Chen, J., Li, C., Qian, C., Chu, X., & Hu, X. (2018). A Cascaded Inception of Inception Network with Attention Modulated Feature Fusion for Human Pose Estimation. The Thirty-Second AAAI Conference on Artificial Intelligence.
Xiong, Z., Weber, C., & Hu, X. (2018). Frame Difference-Based Real-Time Video Stylization in Video Calls. In 2018 Ninth International Conference on Intelligent Control and Information Processing (ICICIP), 333-339.
Wang, Y., Su, H., Zhang, B., & Hu, X. (2018). Interpret neural networks by identifying critical data routing paths. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 8906-8914.
Liao, F., Liang, M., Dong, Y., Pang, T., Hu, X., & Zhu, J. (2018). Defense against adversarial attacks using high-level representation guided denoiser. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1778-1787.
Springenberg, S., Lakomkin, E., Weber, C., & Wermter, S. (2018). Image-to-Text Transduction with Spatial Self-Attention. In 2018 proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN).
Dong, Y., Liao, F., Pang, T., Su, H., Zhu, J., Hu, X., & Li, J. (2018). Boosting adversarial attacks with momentum. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 9185-9193).
Sun, T., Wang, Y., Yang, J., Hu, X., (2017). Convolutional neural networks with two pathways for image style recognition, IEEE Transactions on Image Processing, vol. 26, no. 9, pp. 4102-4113, 2017.
Wang, J., & Hu, X. (2017). Gated recurrent convolution neural network for ocr. In International Conference on Advances in Neural Information Processing Systems(NeurIPS), 335-344.
Hu, Z., Wang. T., Hu, X., (2017). An STDP-based supervised learning algorithm for spiking neural networks, Proc. of 24th International Conference on Neural Information Processing (ICONIP), Guangzhou, China, Nov. 14-18, 2017.
Hu, X., Wang, T., (2017). Training the Hopfield Neural Network for Classification Using a STDP-Like Rule, Proc. of 24th International Conference on Neural Information Processing (ICONIP), Guangzhou, China, Nov. 14-18, 2017.
Yu, N., Qiu, S., Hu, X., Li, J., (2017). Accelerating Convolutional Neural Networks by Group-wise 2D-filter Pruning, Proc. of International Joint Conference on Neural Networks (IJCNN), Anchorage, Alaska, USA, May 14–19, 2017, pp. 2502-2509.
Wu, J., Ma, L., Hu, X., (2017). Delving deeper into convolutional neural networks for camera relocalization, Proc. of IEEE International Conference on Robotics and Automation (ICRA), Singapore, May 29- June 3, 2017, pp. 5644-5651.
Zhao, Y., Jin, X., Hu, X., (2017). Recurrent convolutional neural network for speech processing, Proc. of the 42nd IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, USA, March 5-9, 2017.
Zhou, X., Weber, C., Wermter, S. (2017). Robot Localization and Orientation Detection based on Place Cells and Head-direction Cells. Proceedings of the 26th International Conference on Artificial Neural Networks (ICANN 2017), Sep 2017.
Zhuang, C., Wang, Y., Yamins, D., & Hu, X. (2017). Deep learning predicts correlation between a functional signature of higher visual areas and sparse firing of neurons. Frontiers in Computational Neuroscience, 11, 100.
Chen, X., Hu, X., Zhou, H., & Xu, N. (2017, May). Fxpnet: Training a deep convolutional neural network in fixed-point representation. In 2017 International Joint Conference on Neural Networks (IJCNN), 2494-2501.
Wu, J., Ma, L., Hu, X., (2016). Predicting world coordinates of pixels in RGB images using convolutional neural network for camera relocalization, Proc. of the Seventh International Conference on Intelligent Control and Information Processing (ICICIP), Siem Reap, Cambodia, December 1-4, 2016.
Qin H., Yan J., Li X., Hu X., (2016). Joint Training of Cascaded CNN for Face Detection, Proc. of the 28th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 3456-3465.