PointSemSeg+ (SPP 100+)

  two images: Point cloud of bridge and segmented bridge Copyright: © gia  

"Bayesian Neural Networks for Semantic Point Cloud Segmentation".

The SPP "Hundred plus" aims at extending the usability of complex structures through intelligent digitization in the sense of a digital twin. To this end, the priority program includes three core research topics:

  1. Digital Models,
  2. Digital Linking, and
  3. State Indicators,

with topics (1) and (2) being the focus of the current first funding phase. A real demonstrator structure is available for testing as well as validating the developed methods for model generation, digital linkage and derivation of condition indicators.
The modeling of geometry and semantics from acquisition data (for example laser scanning, photogrammetry) or other existing data sources (for example as-built documents) represents an essential basis for the development of digital twins of existing building structures as part of research topic (1) of the priority program. The subproject of gia "Semantic Segmentation of point clouds through Bayesian neural networks for uncertainty estimation” (PointSemSeg+) is assigned to this funding priority and investigates methods for the automatic derivation of semantically-rich digital building inventory models from 3D point clouds (scan to BIM) based on machine learning methods, in particular deep neural networks.

  Graphic of 4 bridges with different segmentation Copyright: © gia

However, the uncertainty quantification of the result, in particular of the semantic segmentation as a central process step in the scan-to-BIM process, represents a central challenge and remains an unsolved problem. Unavoidable imperfections of the input point clouds, such as measurement deviations, non-homogeneous point densities, gaps as well as obscuring objects may influence the resulting semantic segmentation in an unexpected way. The result is therefore incomplete or in parts incorrectly segmented and classified point cloud, which inevitably affects the quality downstream automated model generation in a negative way. However, especially for the derivation of digital twins for engineering applications, the knowledge of the quality or uncertainty of the underlying digital model is of immense importance. In the present project, therefore, research into the uncertainty of semantic point cloud segmentation based on Deep Learning is to be carried out. Specifically, it will be investigated on the input data level whether uncertainty of in the semantic segmentation can be reduced by supplementary feature information and quantified on the model level by the use of Bayesian neural networks. For this purpose, 3D point clouds are generated with geodetic laser scanners - also supported by unmanned aerial vehicles (UAV) - which contain additional radiometric information besides the 3D point information.

Logo of the German Research Foundation