PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space: Difference between revisions

From statwiki
Jump to navigation Jump to search
Line 1: Line 1:
= Introduction =
= Introduction =
This paper builds off of ideas from PointNet (Qi et al., 2017).
This paper builds off of ideas from PointNet (Qi et al., 2017). The name PointNet is derived from the network's input - a point cloud. A point cloud is a set of three dimensional points that each have coordinates <math> (x,y,z) </math>. These coordinates usually represent the surface of an object. For example, a point cloud describing the shape of a torus is shown below.


PointNet and PointNet++ are deep neural network architecture that take point clouds as their input and perform the tasks such as classification or part segmentation or scene segmentation.  
[[File:Point cloud torus.gif|thumb|center|Point cloud torus]]




PointNet differs from other point cloud processing networks by directly taking the unordered points from the point cloud as input, instead of re-representing the point cloud by voxelization or 2D images.
Processing point clouds is important in applications such as autonomous driving where point clouds are collected from an onboard LiDAR sensor. These point clouds can then be used for object detection. However, point clouds are challenging to process because:


# They are unordered. If <math> N </math> is the number of points in a point cloud, then there are <math> N! </math> permutations that the point cloud can be represented.
# The spatial arrangement of the points contains useful information, thus it needs to be encoded.
# The function processing the point cloud needs to be invariant to transformations such as rotation and translations of all points.


[[File:Point cloud torus.gif|thumb|Point cloud torus]]
Previously, typical point cloud processing methods handled the challenges of point clouds by transforming the data with a 3D voxel grid or by representing the point cloud with multiple 2D images. When PointNet was introduced, it was novel because it directly took points as its input. PointNet++ improves on PointNet by using a hierarchical method to better capture local structures of the point cloud.
 


[[File:point_cloud.png | 400px|thumb|center|Examples of point clouds and their associated task. Classification (left), part segmentation (centre), scene segmentation (right) ]]
[[File:point_cloud.png | 400px|thumb|center|Examples of point clouds and their associated task. Classification (left), part segmentation (centre), scene segmentation (right) ]]

Revision as of 20:42, 16 March 2018

Introduction

This paper builds off of ideas from PointNet (Qi et al., 2017). The name PointNet is derived from the network's input - a point cloud. A point cloud is a set of three dimensional points that each have coordinates [math]\displaystyle{ (x,y,z) }[/math]. These coordinates usually represent the surface of an object. For example, a point cloud describing the shape of a torus is shown below.

Point cloud torus


Processing point clouds is important in applications such as autonomous driving where point clouds are collected from an onboard LiDAR sensor. These point clouds can then be used for object detection. However, point clouds are challenging to process because:

  1. They are unordered. If [math]\displaystyle{ N }[/math] is the number of points in a point cloud, then there are [math]\displaystyle{ N! }[/math] permutations that the point cloud can be represented.
  2. The spatial arrangement of the points contains useful information, thus it needs to be encoded.
  3. The function processing the point cloud needs to be invariant to transformations such as rotation and translations of all points.

Previously, typical point cloud processing methods handled the challenges of point clouds by transforming the data with a 3D voxel grid or by representing the point cloud with multiple 2D images. When PointNet was introduced, it was novel because it directly took points as its input. PointNet++ improves on PointNet by using a hierarchical method to better capture local structures of the point cloud.

Examples of point clouds and their associated task. Classification (left), part segmentation (centre), scene segmentation (right)

Review of PointNet

PointNet architecture

]

PointNet++

Problem Statement

Method

PointNet++ architecture

]

Sampling Layer

Grouping Layer

Example of the two ways to perform grouping

]

PointNet Layer

Robust Feature Learning under Non-Uniform Sampling Density

Experiments

Sources

1. Charles R. Qi, Li Yi, Hao Su, Leonidas J. Guibas. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space, 2017

2. Charles R. Qi, Hao Su, Kaichun Mo, Leonidas J. Guibas. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation, 2017