Research interests / Current Research

Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS

Research Interests

2D/3D VisionRange SensorsMotion SensingMassive Parallel ProcessingGeneral Purpose Computation on Graphics Processing Unit (GPGPU)Psychology/Neurobiology of Visual PerceptionArtificial Intelligence

Current Research

 

 

Current research activities comprise the processing of real-time capable 3D sensors, like Time-of-Flight (ToF) cameras (also called 3D cameras) and the necessary measures to achieve the needed computational speed. For this purpose I employ General Purpose Computation on Graphics Processing Units (GPGPU). Some of currently achieved results can be found in the following subsections.

 

ToF Camera Data Processing

Unfilterd ToF camera data

ToF cameras have attracted attention in many fields, e.g. automotive engineering, industrial engineering, mobile robotics and surveillance. So far, 3D laser scanners and stereo camera systems are mostly used for these tasks due to their high measurement range and precision. Stereo vision requires the matching of corresponding points from two images to obtain depth information, which is directly provided by laser scanners but with the drawback of a lower frame rate. In contrast to laser scanners, ToF cameras allow for higher frame rates and thus enable the consideration of motion. However, the high framerate has to be balanced with measurement precision. Depending on external interfering factors (e.g. sunlight) and scene configurations, e.g. distances, orientations and reflectivities, the same scene entails large fluctuations in distance measurements from different perspectives. These influences cause systematic errors besides noise and have to be handled by the application.

 

    What you see in this video is a rotating 3D view of data acquired with a Swissranger SR-3k device. For the first rotation, data is shown unfiltered. Systematic and non-systematic errors can be seen clearly. The second rotation is performed while applying several filters (cf. Additional Informations in section 3D Map Construction).

     

    3D Map Construction

    Scene used for mapping
    Generated 3D map

    (Kopie 1)

    What you see in this video is from a bird's view the map construction of 616 captures taken with a Swissranger SR-3k device along a path in our robotic pavillon. The sensor's trajectory is painted in green color. After loop-closure the accumulated error (determined by matching the first and the last capture) is distributed among all captures. You can see this effect especially for the stairs on the right side of the scene. After error relaxation these measurements are fitting exactly.

    The resulting map is then refined in a two step process. First, sparse points are removed and second a principle component analysis (PCA) is performed to detect surface normals. Related pixels are shifted along this vector towards the detected surface.

     

    This work focuses on precise 3D environment mapping relying on time-of-flight (TOF) cameras as the only sensing modality. The pose is estimated using visual odometry. Imprecision of depth measurements caused by external interfering factors, e.g. sunlight or reflectivities are properly handled by several filters. Pose tracking and mapping is performed on-the-fly during exploration and allows even for hand-guided operation. A final refinement step, comprising the error distribution after loop-closure and surface smoothing, results in a precise 3D map.

     

      What you see in this video is from a bird's view the map construction of 616 captures taken with a Swissranger SR-3k device along a path in our robotic pavillon. The sensor's trajectory is painted in green color. After loop-closure the accumulated error (determined by matching the first and the last capture) is distributed among all captures. You can see this effect especially for the stairs on the right side of the scene. After error relaxation these measurements are fitting exactly.

      The resulting map is then refined in a two step process. First, sparse points are removed and second a principle component analysis (PCA) is performed to detect surface normals. Related pixels are shifted along this vector towards the detected surface.

       

        GPGPU Vocus

        Salient regions detected by GPU-VOCUS. The most salient region (MSR) moves from the quadcopter to the red car passing by

         

        This work aimes to achieve real-time capability for a visual attention system. It utilizes GPGPU (General Purpose computation on GPU). The system is derived from the CPU version of VOCUS.

         

           

          What you see in this video is the detection of the most salient region. This is comparable to where humans might look first. The region is marked with a red rectangle. Mostly the rectangle keeps its position on the quadcopter until a red car (which is more salient in this scene) is passing by. The GPU version achieved a speedup of 6~9 and reached real-time capability (more than 30 frames per second) for videos in VGA resolution (640x480). The approach was presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems in San Diego (IROS'2007). For details please refer my publications list.