We dream a magic button for 3-D point cloud processing

3D Studio MAX, 3D Coat, Lightwave, ZBrush, VRMesh, TopoGun, Blender, Rhino, Trimble Sketchup Etc.
Post Reply
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 225
Joined: Wed Aug 21, 2013 8:01 pm
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: CEO_CTO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 1 time
Been thanked: 7 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Point cloud is a set of measurement points, subject to accuracy and density.

C2C (cloud to cloud) is based on ICP (iterative closest point) algorithm and suffers from the accuracy and density of the point clouds measured. C2C doesn’t consider the object geometries at all.

The goal of coordinate metrology is to determine accurately the geometries of object surfaces, including shapes, sizes, positions and orientations. The influence of measurement accuracy and density should be minimized as possible.

C2C is folding the point clouds, not the object geometries measured.

The future of registration is F2F (feature to feature), or G2G (geometry to geometry). In 3-5 years, F2F (or G2G) might be the standard of registration, determining simultaneously the object geometries.

Examples of the minimal conditions of F2F (or G2G):
- 3 not parallel object planes
- 2 not parallel object cylinders
- 3 not colinear object spheres
- Etc.

The object surfaces are the ground truth of geometric reality for applications, e.g., coordinate metrology, augmented reality, robotics, autonomous vehicle, ...

Point clouds and mathematical geometries are only guessing and representing approximately the unknown real object surfaces.

How to accurately estimate and recognize the object geometries are the Holy Grail of Coordinate Metrology.
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 225
Joined: Wed Aug 21, 2013 8:01 pm
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: CEO_CTO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 1 time
Been thanked: 7 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

FindSurface extracts fast and accurately the geometric features from a point cloud.

The YouTube video below shows how 3 flange cylinders are extracted from a partial point cloud.

The estimated radii of the three coaxial cylinders are 198.4, 198.2, and 197.5 mm. They are accurate enough to pin-point MEP elements.

The ability of geometric feature extraction from a partial point cloud is crucial to a single scan (no registration necessary) job or to a hand-held mobile 3-D camera application under the condition of limited accessibility.


youtu.be/MqqkWsfClmc
Last edited by Joon on Thu Mar 12, 2020 10:44 pm, edited 1 time in total.
User avatar
Daniel Wujanz
I have made 90-100 posts
I have made 90-100 posts
Posts: 97
Joined: Fri Aug 24, 2018 11:26 am
Full Name: Daniel Wujanz
Company Details: technet GmbH
Company Position Title: 3D Laser Scanning Specialist
Country: Germany
Linkedin Profile: Yes
Location: Berlin
Has thanked: 39 times
Been thanked: 45 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Daniel Wujanz »

Dear Sung,

first of all thank you for your book! We have recommended it to many of our students.

I absolutely agree with you on the future of registration - and not just because we also use this strategy in Scantra.

Yes, computers are becoming faster - but the same applies to scanners! Hence, substituting the point cloud by primitives breaks down the complexity and consequently allows to process huge datasets by deploying non-rocket-scientist computers in much shorter periods of time. We use Dell XPS 15 laptops with 16 GB of RAM and a 2.8 GHz i7 processor to digest projects with e.g. 6000+ scans.

All the best and hope to meet you at Intergeo

Daniel
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 225
Joined: Wed Aug 21, 2013 8:01 pm
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: CEO_CTO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 1 time
Been thanked: 7 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Daniel Wujanz wrote: Thu Mar 12, 2020 3:30 pm first of all thank you for your book! We have recommended it to many of our students.
Least Squares Orthogonal Distance Fitting of Curves and Surfaces in Space

Thank you!
Daniel Wujanz wrote: Thu Mar 12, 2020 3:30 pm Hence, substituting the point cloud by primitives breaks down the complexity and consequently allows to process huge datasets by deploying non-rocket-scientist computers in much shorter periods of time.
The statistics and probability theory guarantee that about 1,000 inlier points are enough to accurately and economically determine the parameters of a primitive object. 2,000 points are of no advantage to 1,000 points, only increasing the cost. Random sampling is much cheaper than heavy data processing. For a large scale project, we may have to include only the large scale objects first, e.g., large pipes, tanks, containers, etc.
Daniel Wujanz wrote: Thu Mar 12, 2020 3:30 pm All the best and hope to meet you at Intergeo
We will be showcasing our software at AWE USA 2020. But the current global Corona outbreak makes all uncertain.

Joon
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 225
Joined: Wed Aug 21, 2013 8:01 pm
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: CEO_CTO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 1 time
Been thanked: 7 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

We are planning and working on some industrial AR applications. One of them is identifying steel plates. They were plasma-cut from large steel plates.

The material cost for shipbuilding industry is up to 70 % of the total cost. Saving only 1 % of the material cost is simply a lot.

The steel plates are destined to be transported to various assembly sites. From cutting sites to assembly sites.

If the steel plates could be identified automatically at each transporting stages, they could be transported automatically to the destined assembly sites, saving time and material loss.
Steel Plates.png
You do not have the required permissions to view the files attached to this post.
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 225
Joined: Wed Aug 21, 2013 8:01 pm
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: CEO_CTO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 1 time
Been thanked: 7 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Planning application demos for Apple iPad Pro with LiDAR Scanner

The development history of CurvSurf’s AR application demos:

1. Motion tracking without 3-D camera

a. Microsoft HoloLens (2016)
HoloLens Spatial Mapping Points with Sphere Extraction, 2016
https://youtu.be/C7mLH_5QzvU

b. Apple ARKit (2017)
3-D Augmented Reality - Apple ARKit, 2018
https://youtu.be/FzdrxtPQzfA

c. Google ARCore (2017)
Being Shaken and then Remembering - Google ARCore, 2019
https://youtu.be/SX43xEgN_2k


2. Motion tracking with 3-D camera

a. Intel RealSense with Scene Perception (Metaio) (2014)
Real-Time Automatic Object Detection and Measurement, 2015
https://youtu.be/NRTWMBLQZ_w

b. Google Tango (2014)
Geometric Feature Extraction with Google Project Tango Tablet, 2015
https://youtu.be/98znvIIg_HY

c. Microsoft HoloLens 2 (Feb. 2019)
https://www.microsoft.com/en-us/hololens/buy

d. Apple iPad Pro with Lidar Scanner (Mar. 2020)
https://www.apple.com/newsroom/2020/03/ ... in-ipados/

e. Apple iPhone 12 Pro with Lidar Scanner (Sept. 2020)


As of now, Apple iPad Pro with LiDAR Scanner is available. CurvSurf will develop some application demos utilizing the advantages of Apple’s 3-D camera and CurvSurf’s FindSurface.

The 1st demo in short-term will be based on the current ARKit demo of
Apple ARKit: Augmented Reality Based on Curved Object Surfaces, 2019
https://youtu.be/4U4FlavRKa4

The demo will utilize the Apple’s 3-D camera and show the measurement of small 'moving' objects, e.g., balls and pipes of small radii 1 ~ 2 cm.

The 2nd demo in mid-term will be based on the previous Intel RealSense demo of single object recognition:
Augmented Reality: Real time geometric feature extraction and annotation, 2016
https://youtu.be/HGwojt6Gqrc

The demo will show a ‘3-D Scene Recognition’. First, image segmentation will determine the locations and the approximate sizes of objects in the RGB-image captured. Next, CurvSurf FindSurface will determine the shapes, sizes, positions & orientations of all the obtainable objects in scene by processing the point cloud captured. The expected frame rate is 1 ~ 3 scenes per second with FindSurface on device.

They will open a small window to the next AR.

Eventually, the source codes of the application demos will be available at GitHub CurvSurf.

Joon
Last edited by Joon on Sat Apr 04, 2020 1:42 am, edited 1 time in total.
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 225
Joined: Wed Aug 21, 2013 8:01 pm
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: CEO_CTO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 1 time
Been thanked: 7 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Apple iPad Pro LiDAR (iPad Pro 4)

The ToF 3-D camera of iPad Pro 4 has a resolution of 24 x 24 dots with ca. 50-degree FOV (i.e., the dot-spacing angle is ca. 2 degree).

Here is iPad Pro’s LiDAR under infrared
https://twitter.com/i/status/1243071065946910721

Ca. 600 (24 x 24 = 576) laser-dots with 2 deg. dot-spacing are,
enough/good to
- save the battery power
- measure relatively long-distance points
- mesh the points (dots), and
- detect fast a plane from the current frame,
short/negative to
- detect a moving object shape
- detect a small object shape.

Apple has not solved the problem of large-scale SLAM but small tabletop or room scan.

Indoor mapping:
iPad Pro LiDAR meshing using ARKit
https://youtu.be/fRUsV-esskM

Outdoor mapping:
Apple iPad Pro Lidar Scanner for Augmented Reality - Superba AR
https://youtu.be/ipcq02PEX5c

Meshing the point cloud is an integral element of AR platform:
- Microsoft HoloLens 'Spatial Mapping' (2016)
- Google ARCore 'Depth API' (Jan. 2020)
- Apple ARKit 'Scene Geometry API' (Mar. 2020).

Anyway, Apple iPad Pro LiDAR shows the direction AR software developers are looking to:
functionality, availability, supporting API, etc.
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 225
Joined: Wed Aug 21, 2013 8:01 pm
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: CEO_CTO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 1 time
Been thanked: 7 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

We have tested the potential use of Apple iPad Pro LiDAR.

The current and future HW/SW techniques of iPad Pro LiDAR will be applied to Apple iGlass in years.

In weeks (Late May 2020), a demo video of CurvSurf FindSurface utilizing iPad Pro LiDAR will be released.

Once the raw depth data stream of 576 ( = 24 x 24 ) laser dots rather than the ARMeshAnchor could be available, real-time object detection, recognition and measurement would be made possible. Not only the device self but also the object is allowed to move and be measured in real time.


youtu.be/oUUodYjnxfM
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 225
Joined: Wed Aug 21, 2013 8:01 pm
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: CEO_CTO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 1 time
Been thanked: 7 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 225
Joined: Wed Aug 21, 2013 8:01 pm
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: CEO_CTO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 1 time
Been thanked: 7 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Tools used:
- Apple iPad Pro LiDAR
- Apple ARKit 3.5
- Apple Metal API
- CurvSurf FindSurface.


youtu.be/sMRfX334blI
Post Reply

Return to “Modeling Software”