Overview GitHub

Learning Regional Attraction for Line Segment Detection

Nan Xue, Song Bai, Fudong Wang, Gui-Song Xia, Tianfu Wu, Liangpei Zhang, Philip H.S. Torr
LIESMARS CAPTAIN, Wuhan University, Wuhan, China
University of Oxford, Oxford, United Kingdom
Dept. Electrical & Computer Engineering, NC State University, USA

The source code will be released soon!

Abstract

This paper presents regional attraction of line segment maps, and hereby poses the problem of line segment detection (LSD) as a problem of region coloring. Given a line segment map, the proposed regional attraction first establishes the relationship between line segments and regions in the image lattice. Based on this, the line segment map is equivalently transformed to an attraction field map (AFM), which can be remapped to a set of line segments without loss of information. Accordingly, we develop an end-to-end framework to learn attraction field maps for raw input images, followed by a squeeze module to detect line segments. Apart from existing works, the proposed detector properly handles the local ambiguity and does not rely on the accurate identification of edge pixels. Comprehensive experiments on the Wireframe dataset and the YorkUrban dataset demonstrate the superiority of our method. In particular, we achieve an F-measure of 0.831 on the Wireframe dataset, advancing the state-of-the-art performance by 10.3 percent.

Introduction

Line segment detection (LSD) is an important yet challenging low-level task in computer vision. LSD aims to extract visible line segments in scene images (See Figure 1 (a) and (e)). Most of the existing line segment detectors are built upon edge pixel identification with two main drawbacks: such work lacks elegant solutions to solve the issues caused by inaccurate or incorrect edge detection results (e.g., local ambiguity, high false positive detection rates and incomplete line segments) and requires carefully designed heuristics or extra contextual information to infer line segments from identified edge pixels.




Figure 1. Illustrative examples of different methods on an image. In the top row: (a) shows an example test image in the Wireframe dataset; (b)-(e) shows the corresponding local edge map, gradient magnitude and deep edge map respectively. In the bottom row: (e) shows our detected line segments; (f) - (h) display the results of MCMLSD [1], LSD [2] and DWP [3] respectively.

In this paper, we focus on a deep learning based LSD framework and propose a single-stage method that rigorously addresses the drawbacks of existing LSD approaches. Our method is motivated by the following observations:

  • The duality between regions and the contour (or the surface) of an object is well-known in computer vision.
  • All pixels in the image lattice should be involved in the formation of line segments in an image.
  • The recent remarkable progress led by deep learning based methods (e.g., U-Net and DeepLab V3+) in semantic segmentation.
Thus, the intuitive idea of this paper is that when bridging a line segment map and its spatial proximate regions, we can pose the problem of LSD as the problem of region coloring, and thus open the door to leveraging the best practices developed in state-of-the-art deep ConvNet based semantic segmentation methods to improve the performance of LSD. Following this idea, we exploit the spatial relationship between pixels in the image lattice and line segments, and propose a new formulation termed regional attraction for line segment detection. By learning the regional attraction, our proposed line segment detector eliminates the limitations of edge pixel identification.

Experimental Results

Results on Wireframe Dataset

Figure 2. Some results of line segment detection of different approaches on the Wireframe dataset. From top to bottom: LSD [4], MCMLSD [3], Linelet [6], DWP [5], AFM [2] with the a-trous Residual U-Net and AFM++ proposed in this paper.

Results on YorkUrban Dataset

Figure 3. Some results of line segment detection of different approaches on the YorkUrban dataset. From top to bottom: LSD [4], MCMLSD [3], Linelet [6], DWP [5], AFM [2] with the a-trous Residual U-Net and AFM++ proposed in this paper.

References

  1. Learning Regional Attraction for Line Segment Detection [paper]
    N. Xue, S. Bai, F.-D. Wang, G.-S. Xia, T. Wu, L. Zhang, P.H.S. Torr
    IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), 2019.
  2. Learning Attraction Field Representation for Robust Line Segment Detection [paper]
    N. Xue, S. Bai, F.-D. Wang, G.-S. Xia, T. Wu, L. Zhang
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  3. MCMLSD: A Dynamic Programming Approach to Line Segment Detection [paper]
    E. J. Almazan, R. Tal, Y. Qian, J. H. Elder
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
  4. LSD: A Fast Line Segment Detector with a False Detection Control [paper]
    R. G. von Gioi, J. Jakubowicz, J. M. Morel, G. Randall
    IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), 2010.
  5. Learning to Parse Wireframes in Images of Man-Made Environments [paper]
    K. Huang, Y. Wang, Z. Zhou, T. Ding, S. Gao, Y. Ma
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
  6. A Novel Linelet-Based Representation for Line Segment Detection [paper]
    N.-G. Cho, A. Yuille, S.-W. Lee
    IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), 2018.

© 2019 by Nan Xue at CAPTAIN