Recognition of complex activities from surveillance video requires tracking individuals and maintaining their identities across gaps in observation – for example, we might see one person placing a package down on a desk; if at a later time a different person takes the package away, then it is a possible theft; but if the same person takes it away, it does not represent a security violation. Identity maintenance has been traditionally addressed using appearance matching approaches. I will discuss computer vision and artificial intelligence problems associated with tracking people through gaps in observation – for example, a person seen in one camera at time t₁ and then (possibly) in a distant camera at a later time t₂; or, a person is viewed entering a closed space and after some time leaving it, all within the field of view of one camera.

There is a variety of visual "soft" biometrics that can be used to address this matching problem, including face recognition, gait analysis, and clothing appearance. Any measurement process intended to capture these biometrics has to cope with variations due to lighting and pose, as well as occlusion. I will describe a clothing appearance model that combines an intrinsic geometric parameter based on path length (geodesic length between a distinguished body point like the head and any other body point), and color features that provide a measure of illumination invariance (based on ranking). I will then describe a sequence to sequence matching algorithm that attempts to overcome local errors in segmentation and intrinsic variability in the appearance model due to pose changes, and present the results of matching experiments on a database of two camera sequences.

However, these appearance-based approaches, by themselves, still make errors. What additional information might be available to an observer to reduce these errors? I describe how to augment traditional appearance matching with contextual information about the environment and self identifying traits of certain actions. This is accomplished using a prioritized, multi-valued, default logic that can be employed to reason about the identities of individuals. This framework also encodes qualitative confidence for the identity decisions it takes and uses this information to reason about the occurrence of activities in video.

Biography of Larry S Davis:
Larry S. Davis received his B.A. from Colgate University in 1970 and his M. S. and Ph. D. in Computer Science from the University of Maryland in 1974 and 1976 respectively. From 1977-1981 he was an Assistant Professor in the Department of Computer Science at the University of Texas, Austin. He returned to the University of Maryland as an Associate Professor in 1981. From 1985-1994 he was the Director of the University of Maryland Institute for Advanced Computer Studies. He is currently a Professor in the Institute and the Computer Science Department, as well as Chair of the Computer Science Department. He was named a Fellow of the IEEE in 1997.