We dream a magic button for 3-D point cloud processing

[Joon]

A monumental video

Only Google ARCore's motion tracking made this possible and was confirmed as such until recently.

Lee JaeHyo Gallery Large Ring - Google ARCore

youtu.be/ugnvhvVwqaw

[Joon]

I have reviewed the visionOS, the OS for Apple Vision Pro.

Apple wants to open/close the provision of 2D imagery and 3D point cloud sensor data. Apple wants to control everything.

Close everything for now. Apple then has the opportunity to open them selectively.

In today's visionOS we see that Apple drew this picture.

The technology is already available and legal controls will follow sooner or later.

Joon

[Joon]

ISO 10360-6 was published in 2001.

It serves to test the accuracy of software that processes 2D/3D measurement data, without giving any indication of how an optimal SW can be realized.

Why FindSurface 2023 CurvSurf.png

[Joon]

What you see may not be what it is.

We need to have and process both 2D image and 3D depth data of a scene. This allows us to increase accuracy and processing speed and reduce misunderstanding rates.

[Joon]

What new applications are possible using consumer products?

By processing Apple LiDAR's 576 points in real time, you could determine the shape, size, location, and orientation of a real-world object's surface front of you, even in total darkness. Not only objects but also you are allowed to move.

[Joon]

Night vision enables the visually impaired to see. In other words, it provides information about the shape, size, position and posture of objects in front of a visually impaired person. FindSurface can provide real-time information about this.

[Joon]

The FindSurface SDK software library has a binary file size of about 300 KB for Windows 64-bit DLL or 1 MB (due to the module name texts) for iOS/iPadOS/macOS/Linux/Android framework. It is a really tiny runtime library that contains many algorithms for region growing, segmentation, model fitting (orthogonal distance fitting), optimization, geometry, curvature analysis, statistics and probability theory without any commercial library components or ML/DL algorithms. It is a simple container with optimized abstract mathematical formulas and strategies.

[Joon]

We want to maximize the use of low-cost filtering and reduce costly fittings.

We measured FindSurface's desktop PC operating speed through experiments and obtained an operating speed model. According to the model,
- Extraction of a cylinder with 1,000 points from 100,000 point data is done 87 times/sec.
- Extraction of a cylinder with 10,000 points from 1 million point data is performed 11 times/sec.
- Extraction of a cylinder with 10,000 points from 10 million point data is performed 2 times/sec.

The operating speed model of FindSurface on an average desktop PC:
Base load = 3 ms
Fitting load = 4.5 ms/k
Filtering load = 0.04 ms/k.

[Joon]

There is a big gap between 2D image processing and 3D point cloud processing.

- 2D image processing:
2D Camera -> 2D Image -> 'Edge Detection' -> Boundaries of 2D Regions -> 2D Curves.

- 3D point cloud processing:
3D Camera -> 3D Point Cloud -> Boundaries of 3D Volume Regions -> 3D Surfaces.

“Edge detection” data processing techniques applied to 2D images have virtually nothing to do with 3D point clouds and should not be applied to them. The 3D point cloud already represents the boundary points of 3D volume regions.

Boundary points of boundary points are unstable and unreliable.

For example, the two endpoints of chained edgels (edge pixels) in a 2D image are unstable and unreliable, depending on the image resolution, image processing algorithm, etc (boundary points of boundary points).

As the same, the boundary points of 3D point cloud are unstable and unreliable, depending on the sensor cell resolution and S/N ratio.

Boundary points of boundary points are unstable, unreliable, and useless.

[Joon]

From a mathematical perspective, the only difference between 2- and 3-dimensional is the numbers 2 and 3.

The problem of processing measurement points to obtain information about the measurement object could be the same for 2- and 3-dimensional.

For example, we want to obtain SHAPE, SIZE, POSITION, and ORIENTATION information of the measured object in 2 or 3 dimensional space.

SHAPE and SIZE can be expressed as a formula, e.g., ||x||^2 - r^2 = 0 representing a 2-dimensional circle or a 3-dimensional sphere.

In addition, the formula of x = R(X - Xo) representing the POSITION and ORIENTATION of object is also the same between 2- and 3-dimensional.

This means that the definition of problem is the same between 2- and 3-dimensional and the mathematical equations derived during the solution process are also the same.

Therefore, the source code of the computer program that implements the problem-solving result is also the same between 2- and 3-dimensional. The only difference is the number 2 or 3.

My monograph “Least Squares Orthogonal Distance Fitting of Curves and Surfaces in Space,” Springer, 2004, describes in detail how to solve the problem and implement the solution.