We dream a magic button for 3-D point cloud processing

aurivus|REVIT, 3D Studio MAX, 3D Coat, Lightwave, ZBrush, VRMesh, TopoGun, Blender, Rhino, Trimble Sketchup Etc.
Post Reply
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 351
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 38 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

The elapse time model of FindSurface for extracting an object cylinder has been determined experimentally on Mac Book Pro M1, 64GB RAM.

FindSurface is a real-time 3D object/geometry recognition, measurement, and extraction software library developed by CurvSurf.

CurvSurf/FindSurface
https://github.com/CurvSurf/FindSurface

Model of elapse time [msec] = 3 + 3.5 * A + 0.015 * B, with
A: number of inlier points [1000],
B: number of input points [1000].

Examples of elapse time calculation:
- Extraction of 1,000 inlier points cylinder from 10,000 input points
= 3 + 3.5 * 1 + 0.015 * 10 = 6.65 [msec] = 150 times/sec.

- Extraction of 1,000 inlier points cylinder from 100,000 input points
= 3 + 3.5 * 1 + 0.015 * 100 = 8.00 [msec] = 125 times/sec.

- Extraction of 10,000 inlier points cylinder from 100,000 input points
= 3 + 3.5 * 10 + 0.015 * 100 = 39.50 [msec] = 25 times/sec.

- Extraction of 10,000 inlier points cylinder from 1,000,000 input points
= 3 + 3.5 * 10 + 0.015 * 1,000 = 53 [msec] = 19 times/sec.

- Extraction of 100,000 inlier points cylinder from 10,000,000 input points
= 3 + 3.5 * 100 + 0.015 * 10,000 = 503 [msec] = 2 times/sec.

- Extraction of 1,000 inlier points cylinder from 10,000,000 input points
= 3 + 3.5 * 1 + 0.015 * 10,000 = 156.5 [msec] = 6.4 times/sec.

It’s fast!
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 351
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 38 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Measurement chart of elapsed times in milliseconds, extracting an object cylinder with 1,000 inlier points from up to 65,536,000 input points.
Up to 256,000 input points, the elapsed time remains less than around 10 milliseconds. After them, the filtering out of input points dominates with 0.015 milliseconds per 1,000 input points.

Image
2023-11-14-측정 - AnalysisGraph.png
You do not have the required permissions to view the files attached to this post.
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 351
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 38 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

The current runtime model of CurvSurf FindSurface on MacBook Pro M1, 64GB RAM, will soon be deprecated.

The current model is:
Elapsed time [ms] = 3 + 3.5 * A + 0.015 * B, with
A: Number of inlier points [1000],
B: Number of input points [1000].

The model says that filtering out an outlier takes 15 nanoseconds (45 clocks on a 3 GHz CPU). The current CurvSurf FindSurface runs in a single thread without GPU usage.

Recently, CurvSurf found a possibility of approximately x 2 speedup based on new algorithms running in a single thread without GPU usage.

There are still many open possibilities for additional accelerations: multi-thread, GPU, mathematics, logic, ...
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 351
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 38 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

How to Pick a 3D Point in 2D Screen (https://github.com/CurvSurf/ARKitDepthFindSurfaceWeb):

- You aim your device at object surface. Your touch point is in the screen center.
- The probe radius [pixel] on screen is converted to physical radius at unit distance from ray_pos. unitRadius represents the half vertex angle of the “view cone”.
- Squared slope length: UR_SQ_PLUS_ONE = unitRadius * unitRadius + 1.0 .
- len1 = distance of list[idx] from ray_pos in the direction of ray_dir .
- Squared slope length at distance len1 is UR_SQ_PLUS_ONE * (len1 * len1).
- If the squared distance of list[idx] from ray_pos is smaller than the squared slope length at len1, the point list[idx] is viewed inside the “view cone”.
- Orthographic version “view cylinder (pipe)” is used by https://developers.curvsurf.com/WebDemo/

// ViewController.swift --- func _pickPoint()
pickIdx = pickIdxExt = -1; // initialization
if (there is at least one point inside the "view cone") the closest point to ray_pos with the smallest len1 is picked as pickIdx = idx; // pickIdx >= 0
else if (there is at least one point outside the "view cone") the closest point to the "view circle" on screen with the largest cosine value is picked as pickIdxExt = idx; // pickIdxExt >= 0
return pickIdx < 0 ? pickIdxExt : pickIdx; // return (-1) means that no point is picked or list[] is empty.
How to Pick a 3D Point in 2D Screen _ CurvSurf 2023.PNG
You do not have the required permissions to view the files attached to this post.
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 351
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 38 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Solving problems in processing 3D spatial information is not easy.

Can Apple, Google and Microsoft do it? Of course not.
If it was possible, it should have already happened.

Before robots, this is a problem that everyone has been trying to solve since the invention of photography.

This is not a problem that can be solved overnight.
User avatar
gord
I have made 30-40 posts
I have made 30-40 posts
Posts: 32
Joined: Tue Dec 22, 2020 9:30 pm
3
Full Name: Gord Anderson
Company Details: tiyuti
Company Position Title: developer
Country: Australia
Linkedin Profile: No
Has thanked: 39 times
Been thanked: 10 times

Re: We dream a magic button for 3-D point cloud processing

Post by gord »

Joon wrote: Sun Dec 31, 2023 2:23 am Solving problems in processing 3D spatial information is not easy.
...
Before robots, this is a problem that everyone has been trying to solve since the invention of photography.
You have an impressive body of work, Joon - I think well see a lot more practical use of VR & AR in 2024.

The virtual interview with Lex Fridman and Mark Zuckerberg was pretty cool - very lifelike animated 3D avatars : https://www.youtube.com/watch?v=MVYrJJNdrEg

Its an exciting time to see all these innovations in AI/ML, gaussian splats, NeRFs, sensors, photo-grammetry and more efficient compute, as they converge into useful applications.

Wishing you a wonderful New Year !
gord anderson
quato.xyz
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 351
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 38 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

gord wrote: Wed Jan 03, 2024 5:33 am
Joon wrote: Sun Dec 31, 2023 2:23 am Solving problems in processing 3D spatial information is not easy.
...
Before robots, this is a problem that everyone has been trying to solve since the invention of photography.
You have an impressive body of work, Joon - I think well see a lot more practical use of VR & AR in 2024.

The virtual interview with Lex Fridman and Mark Zuckerberg was pretty cool - very lifelike animated 3D avatars : https://www.youtube.com/watch?v=MVYrJJNdrEg

Its an exciting time to see all these innovations in AI/ML, gaussian splats, NeRFs, sensors, photo-grammetry and more efficient compute, as they converge into useful applications.

Wishing you a wonderful New Year !
Hi Gord,

Thanks.

In implementing and improving the performance of CurvSurf FindSurface, all minimum neccessary public and private mathematical and engineering knowledges required for 2D image processing, 3D measurement, and 3D point cloud processing have been mobilized and applied over the past 30 years.

In next weeks, I would like to compare them comprehensively, including the latest new technologies, ML/DL, NerF/Gaussian Splatting, etc., in simple PPT slides.

There is nothing new under the sun.
It just gets discovered/unearthed.

Joon
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 351
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 38 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

As the latest artificial intelligence technologies, we have seen that machine learning (ML) and deep learning (DL) show excellent performance in many fields. However, it is lagging behind in 3D measurement information processing. Let's find out why, and I'll show you how FindSurface can solve many problems.

The operating principles and conditions of ML/DL are as follows:
- ML/DL solves tasks through learning.
- Learning and assignment data should be arranged at regular intervals, like a checkerboard.
- Humans must analyze large-scale learning data and prepare correct answers in advance.
- ML/DL learns such materials/answers prepared.
- ML/DL analyzes new assignments and provides answers.

When the above processes and conditions are met, ML/DL solves various tasks relatively satisfactorily. However, the weaknesses of ML/DL are seen in the process and conditions:
- ML/DL can't solve tasks it hasn't learned.
- Unarranged data cannot be read.
- A human must prepare the correct answer.
- And so on, there are many problems.

Areas where ML/DL excels are where all the above processes and conditions are met:
- Text/voice recognition/translation (1D arranged data, series).
- Image/video analysis/recognition (2D arranged data, pixels).
- MRI/CT data analysis/recognition (3D arranged data, voxels).

The problem is that developing ML/DL that handles 3D measurement points (point cloud) is not easy:
- The 3D points show a large non-alignment (density, accuracy, outliers).
- The size of the data is large.
- It is difficult for humans to come up with the correct answer.

Therefore, many researchers have begun developing ML/DL that deals with 3D measurement points, but good research results cannot be expected if the fundamental problems are not resolved.

If you ask Google Bard and MS Bing the following two questions, they will point out the problem:
- What are the difficulties in obtaining object information, e.g., shape, size, and 6DOF pose by analyzing and processing a 3D point cloud with ML/DL?
- How costly is it to annotate and label 3D point clouds for use by ML/DL?

FindSurface solves that problem:
- Obtain the exact shape, size, position, and orientation of an object inexpensively from 3D measurement points.
- Classify points belonging to the object (inliers, segmentation).
- It is not significantly affected by the size of the 3D measurement point data.

If you wish to solve the above problem manually, a fee of 0.2 - 1 USD per object will be charged.

The application market size of FindSurface in ML/DL is huge.
FindSurface and ML DL.png
You do not have the required permissions to view the files attached to this post.
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 351
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 38 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

The secret to FindSurface's behavioral adaptability (I)

The secret is understanding what and how to measure it. FindSurface behaves accordingly.

Firstly, the accuracy of the measuring devices used and the distance between the measuring points are given.

Next, the measurement object must be outlined. This is done by selecting a point on the measurement object surface whose approximate size range is known.

Finally, the degree of region growing refines FindSurface's results.

The above information is intuitive, simple and represents the minimum requirement.
CurvSurf FindSurface Parameters.png
You do not have the required permissions to view the files attached to this post.
Last edited by Joon on Sat Jan 20, 2024 3:30 pm, edited 2 times in total.
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 351
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 38 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

The secret to FindSurface's behavioral adaptability (II)

The secret is understanding what happened.

Since the process of coordinate measurement goes through several HW/SW steps, the probabilistic statistical model for coordinate measurement is the Gaussian model. ISO 10360-6 is an international standard based on these facts.

Therefore, using the maximum likelihood estimation (MLE) is the best way to determine the shape, size, position, orientation of an object curve/surface from coordinate measurement points. The a posteriori error of the maximum likelihood estimation should approximately correspond to the a priori error of the coordinate measurement.

FindSurface complies with them.
CurvSurf FindSurface MLE.png
You do not have the required permissions to view the files attached to this post.
Post Reply

Return to “Modeling Software”