University of Hamburg - to research, to teach, to educate and form, to homepage
leeres logo

Photo: UHH/Denstorf

Publications - Thematic Area B

Publications project B1 - Neural dynamics in crossmodal prediction

Publications of the current funding period (2020-2024)

  • Burke R, Maye A, Misselhorn J, Fiene M, Engelhardt FJ, Schneider TR, Engel AK (2024). Delta phase-dependent modulation of temporal predictions by parietal transcranial alternating current stimulation, bioRxiv (preprint). https://doi.org/10.1101/2024.11.18.624126
  • Senkowski D, Engel AK (2024). Multi-timescale neural dynamics for multisensory integration, Nature Reviews Neuroscience 25: 625-642. https://doi.org/10.1038/s41583-024-00845-7
  • Wang P, Maye A, Daume J, Xue G, Engel AK (2024). Oscillatory multi-timescale mechanisms underlying multisensory sequence prediction, bioRxiv (preprint). https://doi.org/10.1101/778969
  • Radecke J-O, Fiene M, Misselhorn J, Herrmann CS, Engel AK, Wolters CH, Schneider TR (2023). Personalized alpha-tACS targeting left posterior parietal cortex modulates visuo-spatial attention and posterior evoked EEG activity, Brain Stimulation 16: 1047-1061. https://doi.org/10.1016/j.brs.2023.06.013
  • Fiene M, Radecke JO, Misselhorn J, Sengelmann M, Herrmann CS, Schneider TR, Schwab BC, Engel AK (2022). tACS phase-specifically biases brightness perception of flickering light, Brain stimulation. 2022 Jan 1;15(1):244-53. https://doi.org/10.1016/j.brs.2022.01.001
  • Maye A, Rauterberg R, Engel AK (2022). Instant classification for the spatially-coded BCI, Plos one. 2022 Apr 28;17(4):e0267548. https://doi.org/10.1371/journal.pone.0267548
  • Maye A, Mutz M, Engel AK (2022). Training the spatially-coded SSVEP BCI on the fly., Journal of Neuroscience Methods. 2022 Aug 1;378:109652. https://doi.org/10.1016/j.jneumeth.2022.109652
  • Daume J, Wang P, Maye A, Zhang D, Engel AK (2021). Non-rhythmic temporal prediction involves phase resets of low-frequency delta oscillations, Neuroimage. 2021 Jan 1;224:117376. https://doi.org/10.1016/j.neuroimage.2020.117376
  • Maye A, Wang T, Engel AK (2021). Neuronal Oscillatory Signatures of Joint Attention and Intersubjectivity in Arrhythmic Coaction., Frontiers in Human Neuroscience. 2021;15. https://doi.org/10.3389%2Ffnhum.2021.767208
  • Maye A, Lorenz J, Stoica M, Engel AK (2020). Subjective evaluation of performance in a collaborative task is better predicted from autonomic response than from true achievements., Frontiers in Human Neuroscience. 2020 Jul 17;14:234. https://doi.org/10.3389/fnhum.2020.00234

Publications of the first funding period (2016-2019)

  • Maye A, Wang P, Daume J, Hu X, Engel AK (2019). An oscillator ensemble model of sequence learning. Frontiers in Integrative Neuroscience 13: 43.
  • Wang P, Göschl F, Friese U, König P, Engel AK (2019). Long-range functional coupling predicts performance: oscillatory EEG networks in multisensory processing. Neuroimage 196: 114-125.
  • Rimmele JM, Gudi-Mindermann H, Nolte G, Röder B, Engel AK (2019). Working memory training integrates visual cortex into beta-band networks in congenitally blind individuals. Neuroimage 194: 259-271.
  • Chen J, Li Z, Hong B, Maye A, Engel AK, Zhang D (2019). A single-stimulus, multi-target BCI based on retinotopic mapping of motion-onset VEPs. IEEE Transactions on Biomedical Engineering 66: 464-470.
  • Nolte G, Aburidi M, Engel AK (2019). Robust calculation of slopes in detrended fluctuation analysis and its application to envelopes of human alpha rhythm. Scientific Reports 9: 6339.
  • Shahbazi Avarvand F, Bartz S, Andreou C, Samek W, Leicht G, Mulert C, Engel AK, Nolte G (2018). Localizing bicoherence from EEG and MEG. Neuroimage 174: 352-363.
  • Chen, J., Zhang, D., Engel, A. K., Gong, Q., and Maye, A. (2017). Application of a single-flicker online SSVEP BCI for spatial navigation, PloS one, 12(5), e0178385.
  • Maye A, Zhang D, Engel AK (2017). Utilizing retinotopic mapping for a multi-target SSVEP BCI with a single flicker frequency. IEEE Transactions on Neural Systems and Rehabilitation Engineering 25: 1026–1036.

Publications project B2 - Bayesian analysis of the interaction of learning, semantics and social influence
with crossmodal integration

Publications of the current funding period (2020-2024)

Publications of the first funding period (2016-2019)

  • Oganian, Y., Heekeren, H. R., & Korn, C. W. (2019). Low foreign language proficiency reduces optimism about the personal future. Quarterly Journal of Experimental Psychology, 72(1), 60-75.
  • Korn, C. W., Heekeren, H. R., & Oganian, Y. (2019). The framing effect in a monetary gambling task is robust in minimally verbal language switching contexts. Quarterly Journal of Experimental Psychology, 72(1), 52-59.
  • Liu, C., Zhuo, J., & Zhu, J. (2019). Understanding MCMC Dynamics as Flows on the Wasserstein Space. In International Conference on Machine Learning (ICML), Long Beach, CA, USA.
  • Liu, C., Zhuo, J., Chen P., Zhang, R., & Zhu, J. (2019). 135 Understand and Accelerate Particle-based Variational Inference. In International Conference on Machine Learning (ICML), Long Beach, CA, USA.
  • Shi, J., Khan, M. E., & Zhu, J. (2019). Scalable Training of Inference Networks for Gaussian-Process Models. In International Conference on Machine Learning (ICML), Long Beach, CA, USA.
  • Pang, T., Xu, K., Du, C., Chen, N., & Zhu, J. (2019). Improving Adversarial Robustness via Promoting Ensemble Diversity. In International Conference on Machine Learning (ICML), Long Beach, CA, USA.
  • Wang, Z., Ren, T., Zhu, J., & Zhang, B. (2019). Function space particle optimization for Bayesian neural networks. In International Conference on Learning Representations (ICLR), New Orleans, Louisiana, USA.
  • Dong, Y., Pang, T., Su, H., & Zhu, J. (2019). Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4312-4321.
  • Dong, Y., Su, H., Wu, B., Li, Z., Liu, W., Zhang, T., & Zhu, J. (2019). Efficient Decision-based Black-box Adversarial Attacks on Face Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 7714-7722.
  • Wei, X., Zhu, J., & Su, H. (2019). Sparse adversarial perturbations for videos. In the 33rd AAAI Conference on Artificial Intelligence (AAAI), Honolulu, Hawaii, USA.
  • Liu, C., & Zhu, J. (2018). Riemannian Stein variational gradient descent for Bayesian inference. Thirty-Second AAAI Conference on Artificial Intelligence.
  • Chen, J., Zhu, J., Teh, Y. W., & Zhang, T. (2018). Stochastic Expectation Maximization with variance reduction. Advances in Neural Information Processing Systems(NeurIPS), 7967-7977.
  • Shi, J., Sun, S., & Zhu, J. (2018). A spectral approach to gradient estimation for implicit distributions. Proc. of the 35th International Conference on Machine Learning (ICML), arXiv preprint arXiv:1806.02925.
  • Tian, T., Zhou, Y., & Zhu, J. (2018). Selective Verification Strategy for Learning from Crowds. In Thirty-Second AAAI Conference on Artificial Intelligence.
  • Du, C., Li, C., Zheng, Y., Zhu, J., & Zhang, B. (2018). Collaborative filtering with user-item co-autoregressive models. In Thirty-Second AAAI Conference on Artificial Intelligence.
  • Bayer, J., Gläscher, J., Finsterbusch, J., Schulte, L. H., & Sommer, T. (2018). Linear and inverted U-shaped dose-response functions describe estrogen effects on hippocampal activity in young women. Nature Communications, 9(1), 1220.
  • Korn, C. W., & Bach, D. R. (2018). Heuristic and optimal policy computations in the human brain during sequential decision-making. Nature Communications, 9(1), 325.
  • Rosenblau, G., Korn, C. W., & Pelphrey, K. A. (2018). A computational account of optimizing social predictions reveals that adolescents are conservative learners in social contexts. Journal of Neuroscience, 38(4), 974-988.
  • Pang, T., Du, C., Dong, Y., & Zhu, J. (2018). Towards robust detection of adversarial examples. In Proc. of Advances in Neural Information Processing Systems (NeurIPS), Montreal, Canada, 4579-4589. (Spotlight, NVIDIA Pioneering Research Award)
  • Li, C., Welling, M., Zhu, J., & Zhang, B. (2018). Graphical Generative Adversarial Networks, In Proc. of Advances in Neural Information Processing Systems (NeurIPS), Montreal, Canada
  • Luo, Y., Tian, T., Shi, J., Zhu, J., & Zhang, B. (2018). Semi-crowdsourced clustering with deep generative models. In Proc. of Advances in Neural Information Processing Systems (NeurIPS), Montreal, Canada, 3212-3222.
  • Li, J., Su, H., Zhu, J., & Zhang, B. (2018). Essay-Anchor Attentive Multi-Modal Bilinear Pooling for Textbook Question Answering. In 2018 IEEE International Conference on Multimedia and Expo (ICME), 1-6.
  • Wei, X., Zhu, J., Feng, S., & Su, H. (2018). Video-to-video translation with global temporal consistency. In 2018 ACM Multimedia Conference on Multimedia Conference, Seoul, Korea, 18-25.
  • Shi, J., Sun, S., & Zhu, J. (2017). Kernel implicit variational inference. Proc. of the 6th International Conference on Learning Representations (ICLR), arXiv preprint arXiv:1705.10119.
  • Zhuo, J., Liu, C., Shi, J., Zhu, J., Chen, N., & Zhang, B. (2017). Message passing stein variational gradient descent. Proc. of the 35th International Conference on Machine Learning (ICML), arXiv preprint arXiv:1711.04425.
  • Li, C., Zhu, J., & Zhang, B. (2017). Max-margin deep generative models for (semi-) supervised learning. IEEE transactions on pattern analysis and machine intelligence, 40(11), 2762-2775.
  • Ren, Y., Wang, Y., & Zhu, J. (2017). Spectral learning for supervised topic models. IEEE transactions on pattern analysis and machine intelligence, 40(3), 726-739.
  • Li, C., Xu, K., Zhu, J., & Zhang, B. (2017). Triple generative adversarial nets. 31st Conference on Neural Information Processing Systems(NeurIPS), Long Beach, CA, USA, 4088-4098.
  • Liu, M., Shi, J., Cao, K., Zhu, J., & Liu, S. (2017). Analyzing the training processes of deep generative models. IEEE transactions on visualization and computer graphics, 24(1), 77-87.
  • Deng, Z., Zhang, H., Liang, X., Yang, L., Xu, S., Zhu, J., & Xing, E. P. (2017). Structured generative adversarial networks. Proc. of Advances in Neural Information Processing Systems(NeurIPS), 3899-3909.
  • Zhou, Y., Li, J., & Zhu, J. (2017). Identify the Nash Equilibrium in static games with random payoffs. In Proceedings of the 34th International Conference on Machine Learning (ICML), 4160-4169.
  • Liu, S., Xiao, J., Liu, J., Wang, X., Wu, J., & Zhu, J. (2017). Visual diagnosis of tree boosting methods. IEEE transactions on visualization and computer graphics, 24(1), 163-173.
  • Dong, Y., Ni, R., Li, J., Chen, Y., Zhu, J., & Su, H. (2017). Learning accurate low-bit deep neural networks with stochastic quantization. The 28th British Machine Vision Conference (BMVC), arXiv preprint arXiv:1708.01001.
  • Ren, Y., & Zhu, J. (2017). Distributed Accelerated Proximal Coordinate Gradient Methods. Proc. of International Joint Conference on Artificial Intelligence (IJCAI), 2655-2661.
  • Hu, W., Zhu, J., Su, H., Zhuo, J., & Zhang, B. (2017). Semi-supervised Max-margin Topic Model with Manifold Posterior Regularization. Proc. of International Joint Conference on Artificial Intelligence (IJCAI), 1865-1871.
  • Liu, M., Jiang, L., Liu, J., Wang, X., Zhu, J., & Liu, S. (2017). Improving Learning-from-Crowds through Expert Validation. Proc. of International Joint Conference on Artificial Intelligence (IJCAI), 2329-2336.
  • Dong, Y., Su, H., Zhu, J., & Zhang, B. (2017). Improving interpretability of deep neural networks with semantic information. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4306-4314.
  • Tian, T., Chen, N., & Zhu, J. (2017). Learning attributes from the crowdsourced relative labels. Thirty-First AAAI Conference on Artificial Intelligence.
  • Korn, C. W., Zaiser, J., Schalk, L., Oganian, Y., & Saalbach, H. (2017). Hard-to-read fonts do not influence the framing effect. Psychonom Bull & Rev 25:2:696-703.
  • Chen, J., Zhu, J., & Song, L. (2017). Stochastic training of graph convolutional networks with variance reduction. In Proc. of the 35th International Conference on Machine Learning (ICML), Stockholm, Sweden.

Publications project B3 - Neurocognitive mechanisms for implicit learning of crossmodal predictions

Publications of the current funding period (2020-2024)

  • Jaap C, Rose M (2024). Relevance of pre-stimulus oscillatory activity for the perceived valence of emotional facial expressions, Scientific reports, 14(1), 19263. https://doi.org/10.1038/s41598-024-69433-0
  • Jaap C, Rose M (2024). Dissociable neuronal mechanism for different crossmodal correspondence effects in humans, Acta neurobiologiae experimentalis, 84(2), 136–152. https://doi.org/10.55782/ane-2024-2439
  • Ostrowski J, Rose M (2024). Increases in pre-stimulus theta and alpha oscillations precede successful encoding of crossmodal associations, Scientific reports, 14(1), 7895. https://doi.org/10.1038/s41598-024-58227-z
  • Sun Y, Sun XW, Wang YF, Fu QF (2024). Crossmodal Transfer and Its Cognitive Neural Mechanisms, Progress in Biochemistry and Biophysics, 51(1), 94–110. https://doi.org/10.16476/j.pibb.2022.0508
  • Sun X, Fu Q (2023). The visual advantage effect in comparing uni-modal and cross-modal probabilistic category learning, Journal of Intelligence, 11(12), 218.. https://doi.org/10.3390/jintelligence11120218
  • Sun X, Yao L, Fu Q, Fu X (2023). Multisensory transfer effects in implicit and explicit category learning, Psychological Research, 87(5), 1353–1369. https://doi.org/10.1007/s00426-022-01754-z
  • Sun Y, Fu Q (2023). How do irrelevant stimuli from another modality influence responses to the targets in a same-different task, Consciousness and Cognition, 107, 103455. https://doi.org/10.1016/j.concog.2022.103455
  • Yao L, Fu Q, Liu C H (2023). The roles of edge-based and surface-based information in the dynamic neural representation of objects, NeuroImage, 283, 120425. https://doi.org/10.1016/j.neuroimage.2023.120425
  • Jaap C, Maack MC, Taesler P, Steinicke F, Rose M (2022). Enriched environments enhance the development of explicit memory in an incidental learning task, Sci Rep. 2022 Nov 4;12(1):18717. https://doi.org/10.1038/s41598-022-23226-5
  • Jablonowski J, Rose M (2022). The functional dissociation of posterior parietal regions during multimodal memory formation, Hum Brain Mapp. 2022 Aug 1;43(11):3469-3485. https://doi.org/10.1002/hbm.25861
  • Wu J, Fu Q (2021). The role of working memory and visual processing in prototype category learning, Consciousness and Cognition, 94(August), 103176. https://doi.org/10.1016/j.concog.2021.103176
  • Wu J, Li Q, Fu Q, Rose M, Jing L (2021). Multisensory Information Facilitates the Categorization of Untrained Stimuli, Multisens Res. 2021 Aug 12;35(1):79-107. https://doi.org/10.1163/22134808-bja10061
  • Wu J, Fu Q, Rose M (2020). Stimulus modality influences the acquisition and use of the rule-based strategy and the similarity-based strategy in category learning, Neurobiology of Learning and Memory 168, 107152 (2020). https://doi.org/10.1016/j.nlm.2019.107152
  • Zhou X, Fu Q, Rose M (2020). The Role of Edge-Based and Surface-Based Information in Incidental Category Learning: Evidence From Behavior and Event-Related Potentials, Front. Integr. Neurosci. 14:36. https://doi.org/10.3389/fnint.2020.00036
  • Winterling SL, Shields SM, Rose M (2019). Reduced memory-related ongoing oscillatory activity in healthy older adults, Neurobiol Aging. 2019 Jul;79:1-10. https://doi.org/10.1016/j.neurobiolaging.2019.03.012

Publications of the first funding period (2016-2019)

  • Taesler, P., Jablonowski, J., Fu, Q., & Rose, M. (2019). Modeling implicit learning in a cross-modal audio-visual serial reaction time task. Cognitive Systems Research, 54, 154-164.
  • Fu, Q., Sun, H., Dienes, Z., & Fu, X. (2019). Dataset of implicit sequence learning of chunking and abstract structures. Data in brief, 22, 72-75.
  • Zhou, X., Fu, Q., Rose, M. R., & Sun, Y. (2019). Which Matters More in Incidental Category Learning: Edge-based vs. Surface-based Features. Frontiers in Psychology, 10, 183.
  • Sun, X, Sun, Y., & Fu, Q. (2019). Cross-modal learning and its cognitive and neural mechanisms. Progress in Biochemistry and Biophysics, 46(6), 565-577. (in Chinese)
  • Wu, J., Fu*, Q., Zhou, X., & Sun, X. (2018). The effect of presenting mode of different features on the acquisition of rule-based and similarity-based knowledge in category learning. aaaJournal of Psychological Science, 41(5), 1–6. (in Chinese).
  • Jablonowski, J., Taesler, P., Fu, Q., & Rose, M. (2018). Implicit acoustic sequence learning recruits the hippocampus. PloS one, 13(12), e0209590.
  • Fu, Q., Sun, H., Dienes, Z., & Fu, X. (2018). Implicit sequence learning of chunking and abstract structures. Consciousness and Cognition, 62, 42-56.
  • Clos, M., Sommer, T., Schneider, S. L., & Rose, M. (2018). Enhanced transformation of incidentally learned knowledge into explicit memory by dopaminergic modulation. PloS one, 13(6), e0199013.
  • Fu, Q., Liu, Y. J., Dienes, Z., Wu, J., Chen, W., & Fu, X. (2017). Neural correlates of subjective awareness for natural scene categorization of color photographs and line-drawings. Frontiers in psychology, 8, 210.
  • Taesler, P., & Rose, M. (2017). Psychophysically-anchored, Robust Thresholding in Studying Pain-related Lateralization of Oscillatory Prestimulus Activity. Journal of Visualized Experiments, (119), e55228.

Publications project B4 - Brain dynamics of crossmodal learning and conflict processing

Publications of the current funding period (2020-2024)

  • Kaziki D, Nolte G (2024). Semi-analytic three-shell forward calculation for magnetoencephalography, NeuroImage, Volume 299, 2024,120836, ISSN 1053-8119. https://doi.org/10.1016/j.neuroimage.2024.120836
  • Xu H, Yang G, Wu H, X J, Li Q, Liu X (2024). Distinct mechanisms underlying cross-modal semantic conflict and response conflict processing, Cerebral Cortex. 2024 Jan, 34,1–10, bhad539. https://doi.org/10.1093/cercor/bhad539
  • Xu H, Yang G, Göschl F, Nolte G, Ren Q, Li Z, Wu H, Engel AK, Li Q, Liu X (2024). Distinct and common mechanisms of cross-modal semantic conflict and response conflict in auditory relevant task, Cerebral Cortex. 2024 Feb, 34, bhae105. https://doi.org/10.1093/cercor/bhae105
  • Yang G, Wu H, Li Q, Liu X, Fu Z, Jiang J (2024). Dorsolateral prefrontal activity supports a cognitive space organization of cognitive control, eLife 2023;12:RP87126. https://doi.org/10.7554/eLife.87126
  • Luo C, Proctor RW (2022). A diffusion model for the congruency sequence effect (Review), Psychonomic Bulletin and Review (2022) 29:2034–2051. https://doi.org/10.3758/s13423-022-02119-8
  • Yang G, Fu D, Li Z, Wu H, Xu H, Liu X (2022). Independent multisensory integration and crossmodal attention processing: evidence from audiovisual gender congruency tasks., OSF Preprint. https://doi.org/10.31219/osf.io/p9x2c
  • Yang G, Wang K, Nan W, Li Q, Zheng Y, Wu H, Liu X (2022). Distinct Brain Mechanisms for Conflict Adaptation within and across Conflict Types, J Cogn Neurosci. 2022 Feb 1;34(3):445-460. https://doi.org/10.1162/jocn_a_01806
  • Li Z, Yang G, Wu H, Li Q, Xu H, Göschl F, Nolte G, Liu X (2021). Modality-specific neural mechanisms of cognitive control in a Stroop-like task, Brain Cogn. 2021 Feb;147:105662. https://doi.org/10.1016/j.bandc.2020.105662
  • Qi Y, Yang G, Fu D, Li Z, Liu X (2021). Cognitive Control Developmental Neuroscience: Future Paths and Layouts, Scientia Sinica Vitae. http://dx.doi.org/10.1360/SSV-2020-0248
  • Yang G, Xu H, Li Z, Nan W, Wu H, Li Q, Liu X (2021). The congruency sequence effect is modulated by the similarity of conflicts, J Exp Psychol Learn Mem Cogn. 2021 Oct;47(10):1705-1719. https://doi.org/10.1037/xlm0001054
  • Li Z, Göschl F, Yang G (2020). Dissociated Neural Mechanisms of Target and Distractor Processing Facilitated by Expectations. (Journal Club), J Neurosci. 2020 Mar 4;40(10):1997-1999. https://doi.org/10.1523/JNEUROSCI.2562-19.2020
  • Nolte G, Galindo-Leon E, Li Z, Liu X, Engel AK (2020). Mathematical Relations Between Measures of Brain Connectivity Estimated From Electrophysiological Recordings for Gaussian Distributed Data, Front. Neurosci. 14:577574. https://doi.org/10.3389/fnins.2020.577

Publications of the first funding period (2016-2019)

  • Avarvand, F. S., Bartz, S., Andreou, C., Samek, W., Leicht, G., Mulert, C., ... & Nolte, G. (2018). Localizing bicoherence from EEG and MEG. Neuroimage, 174, 352-363.
  • Daume, J., Graetz, S., Gruber, T., Engel, A. K., & Friese, U. (2017). Cognitive control during audiovisual working memory engages frontotemporal theta-band interactions. Scientific Reports, 7(1), 12585.
  • Li, Q.*, Yang, G.*, Li, Z., Qi, Y., Cole, M. W., & Liu, X. (2017). Conflict detection and resolution rely on a combination of common and distinct cognitive control networks. Neuroscience & Biobehavioral Reviews, 83, 123-131 (* contributed equally).
  • Li, Q., Wang, K., Nan, W., Zheng, Y., Wu, H., Wang, H., & Liu, X. (2015). Electrophysiological dynamics reveal distinct processing of stimulus-stimulus and stimulus-response conflicts. Psychophysiology, 52(4), 562-571.
  • Li, Z, Yang, G., Nan, W., Li, Q., & Liu, X. (2018). Attentional regulation mechanisms of cognitive control in conflict resolution. Advances in Psychological Science, 26(6), 966-974.
  • Nolte, G., Aburidi, M., & Engel, A. K. (2019) Robust calculation of slopes in detrended fluctuation analysis and its application to envelopes of human alpha rhythms. Scientific Reports, 9 (1), 6339.
  • Wang, P.*, Göschl, F.*, Friese, U., König, P., & Engel, A.K. (2019). Long-range functional coupling predicts performance: Oscillatory EEG networks in multisensory processing. NeuroImage, 196, 114-125 (* contributed equally).
  • Yang, G., Li, Z., Wu, H., & Liu, X. (2019). Generality and specificity of cognitive control: research logics and debates. Acta Physiologica Sinica, 71(01), 140-148.
  • Yang, G., Nan, W., Zheng, Y., Wu, H., Li, Q., & Liu, X. (2017). Distinct cognitive control mechanisms as revealed by modality-specific conflict adaptation effects. Journal of experimental psychology: human perception and performance, 43(4), 807-818.
  • Parisi, G., Barros, P., Fu, D., Magg, S., Wu, H., Liu, X., & Wermter, S. (2019). A Neurorobotic Experiment for Crossmodal Conflict Resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 33-36.
  • Zamani, M. A., Magg, S., Weber, C., Wermter, S., & Fu, D. (2018). Deep reinforcement learning using compositional representations for performing instructions. Paladyn Journal of Behavioral Robotics, 9(1), 358-373.
  • Barros, P., Parisi, G.I., Fu, D., Liu, X., Wermter, S. (2017) Adaptive crossmodal stimuli association using expectation learning. European Society for Cognitive Systems, Zurich, Switzerland.
  • 杨国春, 李政汉, 伍海燕, & 刘勋. (2019). 认知控制的一般性/特异性机制: 研究逻辑和争论. 生理学报, (1), 14.
  • 李政汉, 杨国春, 南威治, 李琦, & 刘勋. (2018). 冲突解决过程中认知控制的注意调节机制. 心理科学进展, 26(6), 966-974.

Publications project B5 - Crossmodal fusion for dexterous manipulation in proactive human-robot collaboration

Publications of the current funding period (2020-2024)

  • Chen W, Liu SC, Zhang J (2024). EHoA: A Benchmark for Task-Oriented Hand-Object Action Recognition via Event Vision, IEEE Transactions on Industrial Informatics, vol. 20, no. 8, pp. 10304-10313, Aug. 2024. https://doi.org/10.1109/TII.2024.3393007
  • Chen W, Zeng C, Liang H, Sun F, Zhang J (2024). Multimodality Driven Impedance-Based Sim2Real Transfer Learning for Robotic Multiple Peg-in-Hole Assembly, IEEE Transactions on Cybernetics, Vol 54, Issue 5, May 2024. https://doi.org/10.1109/TCYB.2023.3310505
  • Gomes M, Görner M, Riem Oliveira M, Zhang J (2024). Sensor-agnostic Visuo-Tactile Robot Calibration Exploiting Assembly-Precision Model Geometries, International Conference on Intelligent Robots and Systems, IROS 2024, Abu Dhabi. no DOI assigned yet
  • Görner M, Hendrich N, Zhang J (2024). Pluck and Play: Self-supervised Exploration of Chordophones for Robotic Playing, International Conference of Robotics and Automation (ICRA 2024), Yokohama, Japan. https://doi.org/10.1109/ICRA57147.2024.10610120
  • Liu SC, Tran VN, Chen W, Cheng WL, Huang YL, Liao IB, Li YH, Zhang J (2024). PAVLM: Advancing Point Cloud Based Affordance Understanding Via Vision-Language Model, preprint [Online]. https://doi.org/10.48550/arXiv.2410.11564
  • Wang Y, Zhang L, Tu Y, Zhang H, Bai K, Chen Z, Zhang J (2024). ToolEENet: Tool Affordance 6D Pose Estimation, International Conference on Intelligent Robots and Systems (IROS 2024), Abu Dhabi. https://doi.org/10.48550/arXiv.2404.04193
  • Wang Y, Sun F, Huang W, He F, Tao D (2024). Channel Exchanging Networks for Multimodal and Multitask Dense Image Prediction, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol 45, no 5, May 2023. https://doi.org/10.1109/TPAMI.2022.3211086
  • Zhang L, Bai K, Li Q, Chen Z, Zhang J (2024). CG-CNN: A Collision-Aware Cable Grasping Method in Cluttered Environment, International Conference of Robotics and Automation (ICRA 2024), Yokohama, Japan.. https://doi.org/10.1109/ICRA57147.2024.10610559
  • Jiang J, Tu Y, Xiao X, Fu Z, Zhang J, Chen F, Li M (2023). Improving Robotic Grasping Ability Through Deep Shape Generation, Intelligent Robotics and Applications: 15th International Conference, ICIRA 2022, Harbin, China, August 1–3, 2022, Proceedings, Part IV. https://doi.org/10.1007/978-3-031-13841-6_66
  • Shi Y, Yuan C, Tsitos A, Cong L, Hadjar H, Chen Z, Zhang J (2023). A Sim-to-Real Learning-based Framework for Contact-Rich Assembly by Utilizing CycleGAN and Force Control, IEEE Transactions on Cognitive and Developmental Systems, 2023. https://doi.org/10.1109/TCDS.2023.3237734
  • Tu Y, Jiang J, Li S, Hendrich N, Li M, Zhang J (2023). PoseFusion: Robust Object-in-Hand Pose Estimation with SelectLSTM,, International Conference on Intelligent Robots and Systems (IROS 2023). https://doi.org/10.1109/IROS55552.2023.10341688
  • Zhang H, Liang H, Cong L, Lyu J, Zeng L, Feng P, Zhang J (2023). Reinforcement Learning Based Pushing and Grasping Objects from Ungraspable Poses,, International Conference on Robotics and Automation (ICRA) 2023, ExCel London, UK. https://doi.org/10.1109/ICRA48891.2023.10160491
  • Zhang L, Pei J, Bai K, Chen Z, Zhang J (2023). A Closed-Loop Multi-perspective Visual Servoing Approach with Reinforcement Learning, IEEE International Conference on Robotics and Biomimetics (IEEE ROBIO 2023).. http://doi.org/10.1109/ROBIO58561.2023.10354958
  • Chen W, Liang H, Chen Z, Sun F, Zhang J (2022). Learning 6-DoF Task-oriented Grasp Detection via Implicit Estimation and Visual Affordance, Proc. 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 762-769. https://doi.org/10.1109/IROS47612.2022.9981900
  • Chen W, Liang H, Chen Z et al. (2022). Improving Object Grasp Performance via Transformer-Based Sparse Shape Completion, J Intell Robot Syst 104, 45 (2022). https://doi.org/10.1007/s10846-022-01586-4
  • Cong L, Liang H, Ruppel P, Shi Y, Görner M, Hendrich N, Zhang J (2022). Reinforcement Learning With Vision-Proprioception Model for Robot Planar Pushing, Front. Neurorobot. 16:829437. https://doi.org/10.3389/fnbot.2022.8294
  • Liang H, Cong L, Hendrich N, Li S, Sun F, Zhang J (2022). Multifingered Grasping Based on Multimodal Reinforcement Learning, IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 1174-1181, April 2022,. https://doi.org/10.1109/LRA.2021.3138545
  • Lyu J, Ruppel P, Hendrich N, Li S, Görner M, Zhang J (2022). Efficient and Collision-Free Human-Robot Collaboration Based on Intention and Trajectory Prediction, IEEE Transactions on Cognitive and Developmental Systems, 2022. https://doi.org/10.1109/TCDS.2022.3215093
  • Wang Y, Chen X, Cao L, Huang W, Sun F, Wang Y. (2022). Multimodal Token Fusion for Vision Transformers, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 12176-12185,. https://doi.org/10.1109/CVPR52688.2022.01187
  • Wang Y, Sun F, Huang W, He F, Tao D (2022). Channel Exchanging Networks for Multimodal and Multitask Dense Image Prediction, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, early access. https://doi.org/10.1109/TPAMI.2022.3211086
  • Wang Y, Ye TQ, Cao L, Huang W, Sun F, He F, Tao D (2022). Bridged Transformer for Vision and Point Cloud 3D Object Detection, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 12104-12113,. https://doi.org/10.1109/CVPR52688.2022.01180
  • Yu Y, Huang W, Sun F, Chen C, Wang Y, Liu X (2022). Sound Adversarial Audio-Visual Navigation, ICLR2022 link: https://openreview.net/pdf?id=NkZq4OEYN-. https://doi.org/10.48550/arXiv.2202.10910
  • Zeng C, Li S, Chen Z, Yang C, Sun F, Zhang J (2022). Multifingered Robot Hand Compliant Manipulation Based on Vision-Based Demonstration and Adaptive Force Control, IEEE Transactions on Neural Networks and Learning Systems, 2022. https://doi.org/10.1109/TNNLS.2022.3184258
  • Cong L, Shi Y, Zhang J (2021). Self-supervised Attention Learning for Robot Control, 2021 IEEE International Conference on Robotics and Biomimetics (ROBIO), 2021, pp. 1153-1158. https://doi.org/10.1109/ROBIO54168.2021.9739240
  • Jing M, Huang W, Sun F, Ma X, Kong T, Gan C, Li L (2021). Adversarial Option-Aware Hierarchical Imitation Learning, Proceedings of the 38th International Conference on Machine Learning, PMLR 139:5097-5106, 2021. http://dx.doi.org/10.48550/arXiv.2106.05530
  • Wang Y, Huang W, Fang B, Sun F, Li C (2021). Elastic Tactile Simulation Towards Tactile-Visual Perception, MM '21: Proceedings of the 29th ACM International Conference on Multimedia, October 2021 Pages 2690–2698. https://doi.org/10.1145/3474085.3475414
  • Yang F, Yang C, Guo D, Liu H, Sun F (2021). Fault-Aware Robust Control via Adversarial Reinforcement Learning., 2021 IEEE 11th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER). https://doi.org/10.1109/CYBER53097.2021.9588329
  • Zeng C, Li S, Jiang Y, Li Q, Chen Z, Yang C, Zhang J (2021). Learning compliant grasping and manipulation by teleoperation with adaptive force control, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2021 717–724. https://doi.org/10.1109/IROS51168.2021.9636832
  • Cong L, Görner M, Ruppel P, Liang H, Hendrich N, Zhang J (2020). Self-Adapting Recurrent Models for Object Pushing from Learning in Simulation, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, pp. 5304-5310. https://doi.org/10.1109/IROS45743.2020.9341076
  • Deng Z, Jonetzko Y, Zhang L, Zhang J (2020). Grasping Force Control of Multi-Fingered Robotic Hands through Tactile Sensing for Object Stabilization, MDPI Sensors, 2020, 20:1050. https://doi.org/10.3390/s20041050
  • Ruppel P, Zhang J (2020). Learning Object Manipulation with Dexterous Hand-Arm Systems from Human Demonstration, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5417-5424. https://doi.org/10.1109/IROS45743.2020.9340966
  • Wang Y, Huang W, Sun F, Xu T, Rong Y, Huang J (2020). Deep multimodal fusion by channel exchanging, Proceedings of the 34th International Conference on Neural Information Processing Systems (NeurIPS'20), Article 406, 4835–4. https://dl.acm.org/doi/abs/10.5555/3495724.3496130

Publications of the first funding period (2016-2019)

  • Michael Görner, Robert Haschke, Helge, Ritter, Jianwei Zhang (2019. Moveit! Task Constructor for Task-Level Motion Planning, IEEE International Conference on Robotics and Automation, ICRA 2019, Montreal, Canada.
  • Hongzhuo Liang, Xiaojian Ma, Shuang Li, Michael Görner, Song Tang, Bin Fang, Fuchun Sun, and Jianwei Zhang (2019). PointNetGPD: Detecting Grasp Configurations from Point Sets, IEEE International Conference on Robotics and Automation, ICRA 2019, Montreal, Canada. (pdf)
  • Shuang Li, Xiaojian Ma, Hongzhuo Liang, Michael Görner, Philipp Ruppel, Bin Fang, Fuchun Sun, and Jianwei Zhang (2019). Vision-based Teleoperation of Shadow Dexterous Hand using End-to-End Deep Neural Network, IEEE International Conference on Robotics and Automation, ICRA 2019, Montreal, Canada. (pdf)
  • Liu, C., Fang, B., Sun, F., Li, X., & Huang, W. (2019). Learning to Grasp Familiar Objects Based on Experience and Objects' Shape Affordance. IEEE Transactions on Systems, Man, and Cybernetics: Systems.
  • Wang, T., Yang, C., Kirchner, F., Du, P., Sun, F., & Fang, B. (2019). Multimodal grasp data set: A novel visual–tactile data set for robotic manipulation. International Journal of Advanced Robotic Systems, 16(1), 1729881418821571.
  • Fuchun, Sun, and Liu Huaping (2019). A Novel Multi-modal Tactile Sensor Design using Thermochromic Material. SCIENCE CHINA Information Sciences.
  • Fang, B., Wei, X., Sun, F., Huang, H., Yu, Y., & Liu, H. (2019). Skill learning for human-robot interaction using wearable device. Tsinghua Science and Technology, 24(6), 654-662.
  • Deng, Z., Guan, H., Huang, R., Liang, H., Zhang, L., & Zhang, J. (2019). Combining Model-Based Q-Learning With Structural Knowledge Transfer for Robot Skill Learning. IEEE Transactions on Cognitive and Developmental Systems, 11(1), 26-35.
  • Starke, S., Hendrich, N., & Zhang, J. (2019). Memetic Evolution for Generic Full-Body Inverse Kinematics in Robotics and Animation. IEEE Transactions on Evolutionary Computation, 23(3), 406-420.
  • Deng, Z., Gao, G., Frintrop, S., Zhang, J. (2019). Attention based visual analysis for fast grasp planning with multi-fingered robotic hand. Frontiers in Neurorobotics. doi: 10.3389/fnbot.2019.00060.
  • Jing, M., Ma, X., Huang, W., Sun, F., &amp Liu, H. (2018). ask Transfer by Preference-Based Cost Learning. arXiv preprint arXiv:1805.04686.
  • Deng, Z., Zheng, X., Zhang, L., &amp Zhang, J. (2018). A learning framework for semantic reach-to-grasp tasks integrating machine learning and optimization. Robotics and Autonomous Systems, 108, 140-152.
  • Starke, S., Hendrich, N., Zhang, J. (2018). A Forward Kinematics Data Structure for Efficient Evolutionary Inverse Kinematics, in S. Zeghloul et al. (eds.), Computational Kinematics, Mechanisms and Machine Science 50, Springer 2018. DOI: 10.1007/978-3-319-60867-9_64
  • Fang, B., Sun, F., Yang, C., Xue, H., Chen, W., Zhang, C., Guo, D., Liu, H. (2018). A Dual-Modal Vision-based Tactile Sensor for Robotic Hand Grasping, IEEE International Conference on Robotics and Automation, (ICRA-2018), Brisbane, Australia.
  • Gao, G., Lauri, M., Zhang, J., and Frintrop, S. (2018). Occlusion Resistant Object Rotation Regression from Point Cloud Segments, Proceedings of the ECCV workshop on Recovering 6D Object Pose, 2018. https://arxiv.org/abs/1808.05498.
  • Tao Kong, Fuchun Sun, et al. (2018). Deep Feature Pyramid Reconfiguration for Object Dectection. ECCV, 2018.
  • Liang, H., Li, S., Görner, M., and Zhang, J. (2018). Generating Robust Grasps for Unknown Objects in Clutter Using Point Cloud Data. Shanghai International Symposium on Human-Centered Robotics (HCR), 299-303, 2018.
  • Liang, H., and Zhao, Q. (2018). Multi-View CNNs for 3D Hand Pose Estimation, Shanghai International Symposium on Human-Centered Robotics (HCR), 255-259, 2018.
  • Sebastian Starke, Norman Hendrich, and Jianwei Zhang (2018). Memetic Evolution for Generic Full-Body Inverse Kinematics in Robotics and Animation, IEEE Transactions on Evolutionary Computation, 2018. doi: 10.1109/TEVC.2018.2867601
  • Jing M, Ma X, Sun F, et al. (2018). Learning and Inference Movement with Deep Generative Model. arXiv preprint arXiv:1805.07252, 2018.
  • Ma X, Jing M, Sun F, et al. (2018). Adversarial Task Transfer from Preference. arXiv preprint arXiv:1805.04686, 2018
  • Fuchun Sun, Wenchang Zhang, et al. (2018). Fused Fuzzy Petri Nets: A Shared Control Method for Brain Computer Interface Systems, IEEE Trans. On Cognitive and Developmental Systems, 2018.
  • Ruppel, P., Hendrich, N., Starke, S., Zhang, J. (2018). Cost Functions to Specify Full-Body Motion and Multi-Goal Manipulation Tasks, IEEE International Conference on Robotics and Automation, (ICRA-2018), Brisbane, Australia.
  • Han, D., Nie, H., Chen, J., Chen, M., Deng, Z., Zhang, J. (2018). Multi-modal haptic image recognition based on deep learning. Sensor Review.
  • Ruppel, P., Jonetzko, J., Görner, M., Hendrich, N., Zhang, J. (2018). Simulation of the SynTouch BioTac Sensor, The 15th International Conference on Intelligent Autonomous Systems, (IAS-15 2018), Baden Baden, Germany.
  • Tao Kong, Fuchun Sun, (2017). RON: Reverse Connection with Objectness Prior Networks for Object Detection, CVPR 2017.
  • Huang, W., Harandi, M., Zhang, T., Fan, L., Sun, F., & Huang, J. (2017). Efficient optimization for linear dynamical systems with applications to clustering and sparse coding. Advances in Neural Information Processing Systems (NeurIPS), 3444-3454.
  • Liu, C., Sun, F., Wang, C., Wang, F., & Yuille, A. (2017). MAT: A multimodal attentive translator for image captioning. Proc. International Joint Conference on Artificial Intelligence (IJCAI), arXiv preprint arXiv:1702.05658.
  • Huang, Z., Sun, F., Min, H., Fang, B., Zhang, W., & Hu, X. (2017). A novel wearable tactile sensor array designed for fingertip motion recognition. 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO) (pp. 165-170)
  • Li, L., Sun, F., Fang, B., Huang, Z. Yang, C., & Jing, M. (2017). Learning to detect slip for stable grasping. In 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO).
  • Wasserfall, F., Hendrich, N., & Zhang, J. (2017). Adaptive slicing for the FDM process revisited. 2017 13th IEEE Conference on Automation Science and Engineering (CASE) (pp. 49-54)
  • Starke, S., Hendrich, N., & Zhang, J. (2017). A memetic evolutionary algorithm for real-time articulated kinematic motion. 2017 IEEE Congress on Evolutionary Computation (CEC) (pp. 2473-2479).
  • Starke, S., Hendrich, N., Krupke, D., & Zhang, J. (2017). Evolutionary multi-objective inverse kinematics on highly articulated and humanoid robots. 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 6959-6966).
  • Bestmann, M., Wasserfall, F., Hendrich, N., & Zhang, J. (2017). Replacing cables on robotic arms by using serial via Bluetooth. 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO) (pp. 189-195).
  • Huaping Liu, Fuchun Sun, Di Guo, Bin Fang, (2017). Structured output-associated dictionary learning for haptic understanding, IEEE Transactions on Systems, Man and Cybernetics: Systems, vol.47, no.7, 2017, pp.1564-1574
  • Fiedler, N., Bestmann, M., Hendrich, N. (2018). ImageTagger: An Open Source Online Platform for Collaborative Image Labeling, Robocup Symposium 2018, Montreal, Canada.
  • Deng, Z., Guan, H., Huang, R., Liang, H., Zhang, L., Zhang, J. (2017). Combining Model-based Q-learning with Structural Knowledge Transfer for Robot Skill Learning. IEEE Transactions on Cognitive and Developmental Systems.
  • Huang, T., Sun, F., Fang, B., (2017). A novel wearable tactile sensor for fingertip motion recognition. IEEE Intl. Conference on Robotics and Biomimetics, (ROBIO-2017), Macau, China.
  • Li, X., Sun, F., Fang, B.,(2017). Learning to detect slip for stable grasping. IEEE Intl. Conference on Robotics and Biomimetics, (ROBIO-2017), Macau, China.
  • Wei, X., Sun, F., Fang, B., (2017). Robotic skills learning using dynamical movement primitives from a wearable device. IEEE Intl. Conference on Robotics and Biomimetics, (ROBIO-2017), Macau, China.
  • Bestmann, M., Wasserfall, F., Hendrich, N., Zhang, J. (2017). Replacing Cables on Robotic Arms by Using Serial via Bluetooth, IEEE Intl. Conference on Robotics and Biomimetics, (ROBIO-2017), Macau, China. DOI: 10.1109/ROBIO.2017.8324416
  • Starke, S., Hendrich, N., Zhang, J. (2017). Multi-Objective Evolutionary Optimisation for Inverse Kinematics on Highly Articulated and Humanoid Robots, IEEE Intl. Conference on Intelligent Robots and Systems, (IROS-2017), Vancouver, Canada. DOI: 10.1109/IROS.2017.8206620
  • Wasserfall, F., Hendrich, N., Fiedler, F., Zhang, J. (2017). 3D-Printed Low-Cost Modular Force Sensors, 20th Intl. Conference on Climbing and Walking Robots (CLAWAR-2017). In "Human-Centric Robotics", ISBN: 978-981-3231-03-0 (hardcover). ISBN: 978-981-3231-05-4 (ebook).
  • Wasserfall, F., Hendrich, N., Zhang, J. (2017). Adaptive Slicing for the FDM Process Revisited, 13th IEEE Conference on Automation Science and Engineering (CASE-2017). DOI: 10.1109/COASE.2017.8256074
  • Starke, S., Hendrich, N., Zhang, J. (2017). A Memetic Evolutionary Algorithm for Real-Time Articulated Kinematic Motion, IEEE Congress on Evolutionary Computation (CEC 2017), p 2473-2479, Sán Sebastian, Spain, 2017. DOI: 10.1109/CEC.2017.7969605
  • Starke, S., Hendrich, N., Magg, S., Zhang, J. (2016). An Efficient Hybridization of Genetic Algorithms and Particle Swarm Optimization for Inverse Kinematics, IEEE Intl. Conference on Robotics and Biomimetics (ROBIO 2016), 3-7 Dec. 2016, Qingdao, China. DOI: 10.1109/ROBIO.2016.7866587