WhenPigsFlyContext VS Put-In-Context

Hi all, We do free zoom lectures for the reddit community. This talk will cover visual recognition networks and the role of contextual information Link to event (June 24): https://www.reddit.com/r/2D3DAI/comments/mr9nlj/putting\_visual\_recognition\_in\_context/ Talk is based on the speakers' papers: - Putting visual object recognition in context (CVPR2020) - Paper: https://arxiv.org/abs/1911.07349 - Git: https://github.com/kreimanlab/Put-In-Context - When Pigs Fly: Contextual Reasoning in Synthetic and Natural Scenes - Paper: http://arxiv.org/abs/2104.02215 - Git: https://github.com/kreimanlab/WhenPigsFlyContext Talk abstract: Recent studies have shown that visual recognition networks can be fooled by placing objects in inconsistent contexts (e.g., a pig floating in the sky). This lecture covers two representative works modeling the role of contextual information in visual recognition. We systematically investigated critical properties of where, when, and how context modulates recognition. In the first work, we focused on the study of the amount of context, context and object resolution, geometrical structure of context, context congruence, and temporal dynamics of contextual modulation on real-world images. In the second work, we explored more challenging properties of contextual modulation including gravity, object co-occurrences and relative sizes in synthetic environments. In both works, we conducted a series of experiments to gain insights into the impact of contextual cues on both human and machine vision: - Psycho-physics experiments to establish a human benchmark for out-of-context recognition and then compare it with state-of-the-art computer vision models to quantify the gap between the two. - We proposed new context-aware recognition models. The models captured useful information for contextual reasoning, enabling human-level performance and significantly better robustness in out-of-context conditions compared to baseline models across both synthetic and other existing out-of-context natural image datasets. Presenters BIO: - Philipp Bomatter is a master student for Computational Science and Engineering at ETH Zurich.He is interested in artificial intelligence and neuroscience and currently works on a project concerning contextual reasoning in vision at the Kreiman Lab at Harvard University. - Mengmi Zhang completed her PhD in the Graduate School for Integrative Sciences and Engineering, NUS in 2019. She is now a postdoc in KreimanLab in Children's Hospital, Harvard Medical School.Her research interests include computer vision, machine learning, and cognitive neuroscience. In particular, she studies high-level cognitive functions in humans including attention, memory, learning and reasoning from psychophysics experiments, machine learning approaches and neuroscience. (Talk will be recorded and uploaded to youtube, you can see all past lectures and recordings in /r/2D3DAI)

Git: https://github.com/kreimanlab/WhenPigsFlyContext

Hi all, We do free zoom lectures for the reddit community. This talk will cover visual recognition networks and the role of contextual information Link to event (June 24): https://www.reddit.com/r/2D3DAI/comments/mr9nlj/putting\_visual\_recognition\_in\_context/ Talk is based on the speakers' papers: - Putting visual object recognition in context (CVPR2020) - Paper: https://arxiv.org/abs/1911.07349 - Git: https://github.com/kreimanlab/Put-In-Context - When Pigs Fly: Contextual Reasoning in Synthetic and Natural Scenes - Paper: http://arxiv.org/abs/2104.02215 - Git: https://github.com/kreimanlab/WhenPigsFlyContext Talk abstract: Recent studies have shown that visual recognition networks can be fooled by placing objects in inconsistent contexts (e.g., a pig floating in the sky). This lecture covers two representative works modeling the role of contextual information in visual recognition. We systematically investigated critical properties of where, when, and how context modulates recognition. In the first work, we focused on the study of the amount of context, context and object resolution, geometrical structure of context, context congruence, and temporal dynamics of contextual modulation on real-world images. In the second work, we explored more challenging properties of contextual modulation including gravity, object co-occurrences and relative sizes in synthetic environments. In both works, we conducted a series of experiments to gain insights into the impact of contextual cues on both human and machine vision: - Psycho-physics experiments to establish a human benchmark for out-of-context recognition and then compare it with state-of-the-art computer vision models to quantify the gap between the two. - We proposed new context-aware recognition models. The models captured useful information for contextual reasoning, enabling human-level performance and significantly better robustness in out-of-context conditions compared to baseline models across both synthetic and other existing out-of-context natural image datasets. Presenters BIO: - Philipp Bomatter is a master student for Computational Science and Engineering at ETH Zurich.He is interested in artificial intelligence and neuroscience and currently works on a project concerning contextual reasoning in vision at the Kreiman Lab at Harvard University. - Mengmi Zhang completed her PhD in the Graduate School for Integrative Sciences and Engineering, NUS in 2019. She is now a postdoc in KreimanLab in Children's Hospital, Harvard Medical School.Her research interests include computer vision, machine learning, and cognitive neuroscience. In particular, she studies high-level cognitive functions in humans including attention, memory, learning and reasoning from psychophysics experiments, machine learning approaches and neuroscience. (Talk will be recorded and uploaded to youtube, you can see all past lectures and recordings in /r/2D3DAI)

Git: https://github.com/kreimanlab/Put-In-Context