Seminar: Inference of Mechanical Properties of Cloth from Videos: Human Vision and Machine Learning.

Abstract

My work combines human psychophysics, computer vision, and graphics to investigate how humans estimate material properties from dynamic scenes as well as developing computer vision algorithm to achieve the same. In everyday life, we can easily distinguish cheese from butter, silk from linen, and snow from ice just by looking. Material perception is a central component of human intelligence because it helps us decide how to interact with an object. Even though great strides have been made in object recognition, robust material perception in both human and computer vision are under developed. My talk mainly focuses on one common deformable material: cloth. First, I will discuss how humans estimate mechanical properties of cloth in dynamic scenes. I will discuss the effect of spatiotemporal information across multiple frames (multi-frame motion) on estimating the bending stiffness. I will present both human psychophysics results as well as machine learning model on how long-range motion information could be used to estimate stiffness of cloth. Second, I will discuss how textures and movement cues are combined in judgement of stiffness of cloth and demonstrate that changing the pattern of dynamic deformation alone can alter the perceived stiffness of cloth in a variety of dynamic scenes. Finally, I will briefly discuss my newest work on developing deep learning models for cloth perception and using GANs for cloth video synthesis and how the model results are connected to human perception.

Speaker's Biography

Dr. Bei Xiao is an Assistant Professor in the Department of Computer Science at American University and she leads the Visual Intelligence Laboratory at AU. She received her PhD from University of Pennsylvania in Neuroscience and postdoctoral training at CSAL at MIT. Her research includes human vision, tactile perception, computer vision, perception-driven Computer Science Seminar Series National Capital Region graphics, and VR/AR. Her recent work focuses on human and machine inference of material properties of deformable objects in dynamic scenes and multisensory rendering and perception of material properties in VR. More information about her work can be found: https://sites.google.com/site/beixiao/.