the visual computing center ensures that · the visual computing center ensures that king abdullah...

The Visual Computing Center ensures that King Abdullah University of Science and Technology (KAUST) is at the forefront of petascale computing, modeling, visualization & immersive environments.

The Center serves as a focal point for interdisciplinary research, encompassing modeling, analysis, algorithm development and simulation for problems arising throughout various fields....”

Wolfgang Heidrich | Director, Professor of Computer Science

Helmut Pottmann | Adjunct Named Professor, Applied Mathematics and Computational Science

Peter Wonka | Associate Professor of Computer Science

Niloy J. Mitra | Adjunct Associate Professor of Computer Science

Ph.D., University of Erlangen, GermanyDr. Heidrich’s research interests are in computational photography and display, an emerging research area within visual computing, which combines methods from computer graphics, machine vision, imaging, inverse methods, optics and perception to develop new sensing and display technologies.

Ph.D., Vienna University of Technology, AustriaDr. Pottmann’s research interests are in applied geometry and visual computing, in particular geometric modeling, geometry processing, geometric computing for architecture and manufacturing, robot kinematics and 3D computer vision.

Ph.D., Vienna University of Technology, AustriaProfessor Wonka’s research interests lie in visualization, remote sen-sing and computer graphics with a focus on modeling and analysis of urban and geospatial data.

Ph.D., Stanford University, United StatesProfessor Mitra’s research interests lie in shape capture, shape analysis, shape editing, global and local alignment of shapes and Applications of geometry processing in architectural geometry and other art forms.

Visual Computing Center provides a unique interactive environment that cultivates multidisciplinary collaborations among KAUST researchers and students... ”

Faculty Members

Markus Hadwiger | Associate Professor of Computer Science

Bernard Ghanem | Assistant Professor of Electrical Engineering

Ganesh Sundaramoorthi | Assistant Professor of Electrical Engineering

Ph.D., Vienna University of Technology, AustriaDr. Hadwiger’s research interests are in scientific visualization spe-cifically, focusing in the following areas: Petascale visualization and scientific computing, volume visualization, medical visualization, interactive segmentation and Image processing, GPU-based algo-rithms, and general-purpose computations on GPUs.

Ph.D., University of Illinois at Urbana-Champaign, USADr. Ghanem’s research interests are in representation and analysis of dynamic objects in video sequences with applications to object tracking, action/activity recognition, video registration, motion esti-mation, video segmentation, sparse and low-rank representation in computer vision and machine learning.

Ph.D., Georgia Institute of Technology, USADr. Sundaramoorthi’s research interests are in Computer Vision and Medical Image Understanding. His recent interests are Visual object tracking, Visual object recognition, Shape Modeling/Analysis, and Medical Image Segmentation/Registration from modalitites such as MRI and DT-MRI.

Wolfgang Heidrich | Director, Professor of Computer Science

CVPR 2015

«Robust Manhattan Frame Estimation from a Single RGB-D Image»

«Structural Sparse Tracking»

«lo TV: A New Method for Image Restoration in the Presence of Impulse Noise»

«On the Relationship between Visual Attributes and Convolutional Networks»

«ActivityNet: A Large-Scale Video Benchmark for Human Activity Understanding»

«Coarse-to-Fine Region Selection and Matching»

«Shape-Tailored Local Descriptorsand their Application to Segmentation and Tracking»

«Defocus Deblurring and Superresolution for Time-of-Flight Depth Cameras»

«Rolling Shutter Motion Deblurring»

«Fast and Flexible Convolutional Sparse Coding»

Bernard GhanemKing Abdullah University of Science and Technology (KAUST)

Ali ThabetKing Abdullah University of Science and Technology (KAUST)

Juan C. NieblesUniversidad del Norte, Colombia

Fabian CabaKing Abdullah University of Science and Technology (KAUST)

«Robust Manhattan Frame Estimation from a Single RGB-D Image»

AbstractThis paper proposes a new framework for estimating the Manhattan Frame (MF) of an indoor scene from a singleRGB-D image. Our technique formulates this problem as the estimation of a rotation matrix that best aligns the normals of the captured scene to a canonical world axes. By introducing sparsity constraints, our method can simultaneously estimate the scene MF, the surfaces in the scene that are best aligned to one of three coordinate axes, and the outlier surfaces that do not align with any of the axes. To test our approach, we contribute a new set of annotations to determine ground truth MFs in each image of the popular NYUv2 dataset. We use this new benchmark to experimentally demonstrate that our method is more accurate, faster, more reliable and more robust than the methods used in the literature. We further motivate our technique by showing how it can be used to address the RGB-D SLAM problem in indoor scenes by incorporating it into and improving the performance of a popular RGB-D SLAM method.

«On the Relationship between Visual Attributes and Convolutional Networks»

One of the cornerstone principles of deep models is their abstraction capacity, i.e. their ability to learn abstractconcepts from ‘simpler’ ones. Through extensive experiments,we characterize the nature of the relationship between abstract concepts (specifically objects in images) learned by popular and high performing convolutional networks (conv-nets) and established mid-level representationsused in computer vision (specifically semantic visual attributes).We focus on attributes due to their impact on several applications, such as object description, retrieval and mining, and active (and zero-shot) learning. Among thefindings we uncover, we show empirical evidence of the existence of Attribute Centric Nodes (ACNs) within a conv-net, which is trained to recognize objects (not attributes) in images.These special conv-net nodes (1) collectively encode information pertinent to visual attribute representation and discrimination, (2) are unevenly and sparsely distributionacross all layers of the conv-net, and (3) play an importantrole in conv-net based object recognition.

Victor EscorciaKing Abdullah University of Science and Technology (KAUST) and Universidad del Norte, Colombia



«Structural Sparse Tracking»

Sparse representation has been applied to visual trackingby finding the best target candidate with minimal reconstruction error by use of target templates. However, most sparse representation based trackers only consider holistic or local representations and do not make full use of the intrinsic structure among and inside target candidates, thereby making the representation less effective when similar objects appear or under occlusion. In this paper, we propose a novel Structural Sparse Tracking (SST) algorithm, which not only exploits the intrinsic relationship among target candidates and their local patches to learn their sparse representations jointly, but also preserves the spatial layout structure among the local patches inside each target candidate. We show that our SST algorithm accommodates most existing sparse trackers with the respective merits. Both qualitative and quantitative evaluations on challenging benchmark image sequences demonstrate that the proposed SST algorithm performs favorably against several state-of-the-art methods.

Tianzhu ZhangAdvanced Digital Sciences Center and Institute of Automation, CAS

Si LiuInstitute of Information Engineering, CAS

Changsheng XuInstitute of Automation, CAS and China-Singapore Institute of Digital Media

Shuicheng YanNational University of Singapore

Bernard GhanemAdvanced Digital Sciences Center andKing Abdullah University of Science and Technology (KAUST)

Narendra AhujaAdvanced Digital Sciences Center and University of Illinois at Urbana-Champaign Ming-Hsuan YangUniversity of California at Merced

«ActivityNet: A Large-Scale Video Benchmark for Human ActivityUnderstanding»

In spite of many dataset efforts for human action recognition, current computer vision algorithms are still severely limited in terms of the variability and complexity of the actions that they can recognize. This is in part due to the simplicity of current benchmarks, which mostly focus on simple actions and movements occurring on manually trimmed videos. In this paper we introduce ActivityNet, a new largescale video benchmark for human activity understanding.Our benchmark aims at covering a wide range of complex human activities that are of interest to people in their daily living. In its current version, ActivityNet provides samples from 203 activity classes with an average of 137 untrimmed videos per class and 1.41 activity instances per video, fora total of 849 video hours. We illustrate three scenarios in which ActivityNet can be used to compare algorithms for human activity understanding: untrimmed video classification, trimmed activity classification and activity detection.

Fabian CabaUniversidad del Norte, ColombiaandKing Abdullah University of Science and Technology (KAUST)

Victor EscorciaKing Abdullah University of Science and Technology (KAUST) and Universidad del Norte, Colombia



«lo TV: A New Method for Image Restoration in the Presence ofImpulse Noise»

Total Variation (TV) is an effective and popular prior model in the field of regularization-based image processing. This paper focuses on TV for image restoration in the presence of impulse noise. This type of noise frequently arises in data acquisition and transmission due to many reasons, e.g. a faulty sensor or analog-to-digital converter errors. Removingthis noise is an important task in image restoration.State-of-the-art methods such as Adaptive Outlier Pursuit(AOP) [42], which is based on TV with l-20 norm data fidelity, only give sub-optimal performance.In this paper, we propose a new method, called loTV -PADMM, which solves the TV-based restoration problem with lo- norm data fidelity. To effectively deal with the resulting non-convex nonsmooth optimization problem, we first reformulate it as an equivalent MPEC (Mathematical Program with Equilibrium Constraints), and then solve it using aproximal Alternating Direction Method of Multipliers (PADMM). Our loTV -PADMM method finds a desirable solution to the original lo-norm optimization problem and is proven to be convergent under mild conditions. We apply loTV -PADMM to the problems of image denoising and deblurring in the presence of impulse noise. Our extensive experiments demonstrate that loTV -PADMM outperforms state-of-the-art image restoration methods.

Ganzhao YuanSouth China University of Technology (SCUT), P.R. China


«Coarse-to-Fine Region Selection and Matching»

We present a new approach to wide baseline matching.We propose to use a hierarchical decomposition of the image domain and coarse-to-fine selection of regions to match.In contrast to interest point matching methods, which sample salient regions to reduce the cost of comparing all regions in two images, our method eliminates regions systematically to achieve efficiency. One advantage of our approach is that it is not restricted to covariant salient regions, which is too restrictive under large viewpoint and leads to few corresponding regions. Affine invariant matching of regions in the hierarchy is achieved efficiently by a coarse-to-fine search of the affine space. Experiments on two benchmark datasets shows that our method finds more correct correspondence of the image (with fewer false alarms) than other wide baseline methods on large viewpoint change.

Yanchao YangKing Abdullah University of Science and Technology (KAUST)andUniversity of California, Los Angeles, USA

Zhaojin LuKing Abdullah University of Science and Technology (KAUST) andInstitute of Automation, Chinese Academy of Sciences

Ganesh Sundaramoorthi King Abdullah University of Science and Technology (KAUST)

«Shape-Tailored Local Descriptors and their Application to Segmentation and Tracking»

We propose new dense descriptors for texture segmentation.Given a region of arbitrary shape in an image, these descriptors are formed from shape-dependent scale spaces of oriented gradients. These scale spaces are defined by Poisson-like partial differential equations. A key property of our new descriptors is that they do not aggregate image data across the boundary of the region, in contrast to existing descriptors based on aggregation of oriented gradients.As an example, we show how the descriptor can be incorporatedin a Mumford-Shah energy for texture segmentation. We test our method on several challenging datasets for texture segmentation and textured object tracking. Experiments indicate that our descriptors lead to more accurate segmentation than non-shape dependent descriptors and the state-of-the-art in texture segmentation.

Naeemullah KhanKing Abdullah University of Science and Technology (KAUST)

Marei AlgarniKing Abdullah University of Science and Technology (KAUST)

Anthony YezziSchool of Electrical & Computer Engineering, Georgia Institute of Technology, USA

Ganesh SundaramoorthiKing Abdullah University of Science and Technology (KAUST)

«Rolling Shutter Motion Deblurring»

Although motion blur and rolling shutter deformationsare closely coupled artifacts in images taken with CMOSimage sensors, the two phenomena have so far mostly beentreated separately, with deblurring algorithms being unableto handle rolling shutter wobble, and rolling shutter algorithms being incapable of dealing with motion blur.We propose an approach that delivers sharp and undistortedoutput given a single rolling shutter motion blurred image. The key to achieving this is a global modeling of the camera motion trajectory, which enables each scanline of the image to be deblurred with the corresponding motion segment. We show the results of the proposed framework through experiments on synthetic and real data.

Shuochen SuUniversity of British Columbia andKing Abdullah University of Science and Technology (KAUST)

Wolfgang Heidrich King Abdullah University of Science and Technology (KAUST) andUniversity of British Columbia

«Defocus Deblurring and Superresolution for Time-of-Flight Depth Cameras»

Continuous-wave time-of-flight (ToF) cameras show great promise as low-cost depth image sensors in mobile applications. However, they also suffer from several challenges, including limited illumination intensity, which mandates the use of large numerical aperture lenses, and thus results in a shallow depth of field, making it difficult to capture scenes with large variations in depth. Another shortcoming is the limited spatial resolution of currently available ToF sensors.In this paper we analyze the image formation model for blurred ToF images. By directly working with raw sensor measurements but regularizing the recovered depth and amplitude images, we are able to simultaneously deblur and super-resolve the output of ToF cameras. Our method outperforms existing methods on both synthetic and real datasets. In the future our algorithm should extend easily to cameras that do not follow the cosine model of continuouswave sensors, as well as to multi-frequency or multi-phase imaging employed in more recent ToF cameras.

Lei XiaoUniversity of British Columbia andKing Abdullah University of Science and Technology (KAUST)

Felix HeideUniversity of British Columbia andKing Abdullah University of Science and Technology (KAUST)

Matthew O’TooleUniversity of Toronto

Andreas KolbUniversity of Siegen

Matthias B. HullinUniversity of Bonn

Kyros KutulakosUniversity of Toronto

Wolfgang HeidrichKing Abdullah University of Science and Technology (KAUST) and University of British Columbia

«Fast and Flexible Convolutional Sparse Coding»

Convolutional sparse coding (CSC) has become an increasingly important tool in machine learning and computer vision. Image features can be learned and subsequently used for classification and reconstruction tasks. As opposed to patch-based methods, convolutional sparse coding operates on whole images, thereby seamlessly capturing the correlation between local neighborhoods. In this paper, we propose a new approach to solving CSC problems and show that our method converges significantly faster and also finds better solutions than the state of the art. In addition, the proposed method is the first efficient approach to allow for proper boundary conditions to be imposed and it also supports feature learning from incomplete data as well as general reconstruction problems.

Shuochen SuUniversity of British Columbia andKing Abdullah University of Science and Technology (KAUST)

Wolfgang Heidrich King Abdullah University of Science and Technology (KAUST) andUniversity of British Columbia

«Learning Shape Placements by Example»

We present a method to learn and propagate shape placements in 2D polygonal scenes from a few examples provided by a user. The placement of a shape is modeled as an oriented bounding box. Simple geometric relationships between this bounding box and nearby scene polygons define a feature set for the placement. The feature sets of all example placements are then used to learn a probabilistic model over all possible placements and scenes. With this model, we can generate a new set of placements with similar geometric relationships in any given scene. We introduce extensions that enable propagation and generation of shapes in 3D scenes, as well as the application of a learned modeling session to large scenes without additional user interaction. These concepts allow us to generate complex scenes with thousands of objects with relatively little user interaction.

Paul Guerrero∗Vienna University of TechnologyandKing Abdullah University of Science and Technology (KAUST)

Stefan JeschkeIST Austria

Michael WimmerVienna University of Technology

Peter WonkaKing Abdullah University of Science and Technology (KAUST)

«Doppler Time-of-Flight Imaging»

Over the last few years, depth cameras have become increasingly popular for a range of applications, including human-computer interaction and gaming, augmented reality, machine vision, and medical imaging. Many of the commercially-available devices use the time-of-flight principle, where active illumination is temporally coded and analyzed on the camera to estimate a per-pixel depth map of the scene. In this paper, we propose a fundamentally new imaging modality for all time-of-flight (ToF) cameras: per-pixel velocity measurement. The proposed technique exploits the Doppler effect of objects in motion, which shifts the temporal frequency of the illumination before it reaches the camera. Using carefully coded illumination and modulation frequencies of the ToF camera, object velocities directly map to measured pixel intensities. We show that a slight modification of our imaging system allows for color, depth, and velocity information to be captured simultaneously. Combining the optical flow computed on the RGB frames with the measured metric axial velocity allows us to further estimate the full 3D metric velocity field of the scene. We believe that the proposed technique has applications in many computer graphics and vision problems, for example motion tracking, segmentation, recognition, and motion deblurring.

Felix HeideUniversity of British Columbia andKing Abdullah University of Science and Technology (KAUST)

Gordon WetzsteinStanford University

Matthias B. HullinUniversity of Bonn

Wolfgang HeidrichKing Abdullah University of Science and Technology (KAUST) and University of British Columbia

the visual computing center ensures that · the visual computing center ensures that king abdullah...

Documents