We dream a magic button for 3-D point cloud processing
- Joon
- V.I.P Member
- Posts: 335
- Joined: Wed Aug 21, 2013 8:01 pm
- 10
- Full Name: Sung Joon Ahn
- Company Details: CurvSurf
- Company Position Title: Founder + CEO
- Country: Republic of Korea
- Linkedin Profile: Yes
- Location: Seongnam-si, Korea
- Has thanked: 7 times
- Been thanked: 35 times
- Contact:
Re: We dream a magic button for 3-D point cloud processing
Point cloud is a set of measurement points, subject to accuracy and density.
C2C (cloud to cloud) is based on ICP (iterative closest point) algorithm and suffers from the accuracy and density of the point clouds measured. C2C doesn’t consider the object geometries at all.
The goal of coordinate metrology is to determine accurately the geometries of object surfaces, including shapes, sizes, positions and orientations. The influence of measurement accuracy and density should be minimized as possible.
C2C is folding the point clouds, not the object geometries measured.
The future of registration is F2F (feature to feature), or G2G (geometry to geometry). In 3-5 years, F2F (or G2G) might be the standard of registration, determining simultaneously the object geometries.
Examples of the minimal conditions of F2F (or G2G):
- 3 not parallel object planes
- 2 not parallel object cylinders
- 3 not colinear object spheres
- Etc.
The object surfaces are the ground truth of geometric reality for applications, e.g., coordinate metrology, augmented reality, robotics, autonomous vehicle, ...
Point clouds and mathematical geometries are only guessing and representing approximately the unknown real object surfaces.
How to accurately estimate and recognize the object geometries are the Holy Grail of Coordinate Metrology.
C2C (cloud to cloud) is based on ICP (iterative closest point) algorithm and suffers from the accuracy and density of the point clouds measured. C2C doesn’t consider the object geometries at all.
The goal of coordinate metrology is to determine accurately the geometries of object surfaces, including shapes, sizes, positions and orientations. The influence of measurement accuracy and density should be minimized as possible.
C2C is folding the point clouds, not the object geometries measured.
The future of registration is F2F (feature to feature), or G2G (geometry to geometry). In 3-5 years, F2F (or G2G) might be the standard of registration, determining simultaneously the object geometries.
Examples of the minimal conditions of F2F (or G2G):
- 3 not parallel object planes
- 2 not parallel object cylinders
- 3 not colinear object spheres
- Etc.
The object surfaces are the ground truth of geometric reality for applications, e.g., coordinate metrology, augmented reality, robotics, autonomous vehicle, ...
Point clouds and mathematical geometries are only guessing and representing approximately the unknown real object surfaces.
How to accurately estimate and recognize the object geometries are the Holy Grail of Coordinate Metrology.
- Joon
- V.I.P Member
- Posts: 335
- Joined: Wed Aug 21, 2013 8:01 pm
- 10
- Full Name: Sung Joon Ahn
- Company Details: CurvSurf
- Company Position Title: Founder + CEO
- Country: Republic of Korea
- Linkedin Profile: Yes
- Location: Seongnam-si, Korea
- Has thanked: 7 times
- Been thanked: 35 times
- Contact:
Re: We dream a magic button for 3-D point cloud processing
FindSurface extracts fast and accurately the geometric features from a point cloud.
The YouTube video below shows how 3 flange cylinders are extracted from a partial point cloud.
The estimated radii of the three coaxial cylinders are 198.4, 198.2, and 197.5 mm. They are accurate enough to pin-point MEP elements.
The ability of geometric feature extraction from a partial point cloud is crucial to a single scan (no registration necessary) job or to a hand-held mobile 3-D camera application under the condition of limited accessibility.
youtu.be/MqqkWsfClmc
The YouTube video below shows how 3 flange cylinders are extracted from a partial point cloud.
The estimated radii of the three coaxial cylinders are 198.4, 198.2, and 197.5 mm. They are accurate enough to pin-point MEP elements.
The ability of geometric feature extraction from a partial point cloud is crucial to a single scan (no registration necessary) job or to a hand-held mobile 3-D camera application under the condition of limited accessibility.
youtu.be/MqqkWsfClmc
Last edited by Joon on Thu Mar 12, 2020 10:44 pm, edited 1 time in total.
- Daniel Wujanz
- V.I.P Member
- Posts: 227
- Joined: Fri Aug 24, 2018 11:26 am
- 5
- Full Name: Daniel Wujanz
- Company Details: technet GmbH
- Company Position Title: 3D Laser Scanning Specialist
- Country: Germany
- Linkedin Profile: Yes
- Location: Berlin
- Has thanked: 206 times
- Been thanked: 188 times
- Contact:
Re: We dream a magic button for 3-D point cloud processing
Dear Sung,
first of all thank you for your book! We have recommended it to many of our students.
I absolutely agree with you on the future of registration - and not just because we also use this strategy in Scantra.
Yes, computers are becoming faster - but the same applies to scanners! Hence, substituting the point cloud by primitives breaks down the complexity and consequently allows to process huge datasets by deploying non-rocket-scientist computers in much shorter periods of time. We use Dell XPS 15 laptops with 16 GB of RAM and a 2.8 GHz i7 processor to digest projects with e.g. 6000+ scans.
All the best and hope to meet you at Intergeo
Daniel
first of all thank you for your book! We have recommended it to many of our students.
I absolutely agree with you on the future of registration - and not just because we also use this strategy in Scantra.
Yes, computers are becoming faster - but the same applies to scanners! Hence, substituting the point cloud by primitives breaks down the complexity and consequently allows to process huge datasets by deploying non-rocket-scientist computers in much shorter periods of time. We use Dell XPS 15 laptops with 16 GB of RAM and a 2.8 GHz i7 processor to digest projects with e.g. 6000+ scans.
All the best and hope to meet you at Intergeo
Daniel
- Joon
- V.I.P Member
- Posts: 335
- Joined: Wed Aug 21, 2013 8:01 pm
- 10
- Full Name: Sung Joon Ahn
- Company Details: CurvSurf
- Company Position Title: Founder + CEO
- Country: Republic of Korea
- Linkedin Profile: Yes
- Location: Seongnam-si, Korea
- Has thanked: 7 times
- Been thanked: 35 times
- Contact:
Re: We dream a magic button for 3-D point cloud processing
Least Squares Orthogonal Distance Fitting of Curves and Surfaces in SpaceDaniel Wujanz wrote: ↑Thu Mar 12, 2020 3:30 pm first of all thank you for your book! We have recommended it to many of our students.
Thank you!
The statistics and probability theory guarantee that about 1,000 inlier points are enough to accurately and economically determine the parameters of a primitive object. 2,000 points are of no advantage to 1,000 points, only increasing the cost. Random sampling is much cheaper than heavy data processing. For a large scale project, we may have to include only the large scale objects first, e.g., large pipes, tanks, containers, etc.Daniel Wujanz wrote: ↑Thu Mar 12, 2020 3:30 pm Hence, substituting the point cloud by primitives breaks down the complexity and consequently allows to process huge datasets by deploying non-rocket-scientist computers in much shorter periods of time.
We will be showcasing our software at AWE USA 2020. But the current global Corona outbreak makes all uncertain.
Joon
- Joon
- V.I.P Member
- Posts: 335
- Joined: Wed Aug 21, 2013 8:01 pm
- 10
- Full Name: Sung Joon Ahn
- Company Details: CurvSurf
- Company Position Title: Founder + CEO
- Country: Republic of Korea
- Linkedin Profile: Yes
- Location: Seongnam-si, Korea
- Has thanked: 7 times
- Been thanked: 35 times
- Contact:
Re: We dream a magic button for 3-D point cloud processing
We are planning and working on some industrial AR applications. One of them is identifying steel plates. They were plasma-cut from large steel plates.
The material cost for shipbuilding industry is up to 70 % of the total cost. Saving only 1 % of the material cost is simply a lot.
The steel plates are destined to be transported to various assembly sites. From cutting sites to assembly sites.
If the steel plates could be identified automatically at each transporting stages, they could be transported automatically to the destined assembly sites, saving time and material loss.
The material cost for shipbuilding industry is up to 70 % of the total cost. Saving only 1 % of the material cost is simply a lot.
The steel plates are destined to be transported to various assembly sites. From cutting sites to assembly sites.
If the steel plates could be identified automatically at each transporting stages, they could be transported automatically to the destined assembly sites, saving time and material loss.
You do not have the required permissions to view the files attached to this post.
- Joon
- V.I.P Member
- Posts: 335
- Joined: Wed Aug 21, 2013 8:01 pm
- 10
- Full Name: Sung Joon Ahn
- Company Details: CurvSurf
- Company Position Title: Founder + CEO
- Country: Republic of Korea
- Linkedin Profile: Yes
- Location: Seongnam-si, Korea
- Has thanked: 7 times
- Been thanked: 35 times
- Contact:
Re: We dream a magic button for 3-D point cloud processing
Planning application demos for Apple iPad Pro with LiDAR Scanner
The development history of CurvSurf’s AR application demos:
1. Motion tracking without 3-D camera
a. Microsoft HoloLens (2016)
HoloLens Spatial Mapping Points with Sphere Extraction, 2016
https://youtu.be/C7mLH_5QzvU
b. Apple ARKit (2017)
3-D Augmented Reality - Apple ARKit, 2018
https://youtu.be/FzdrxtPQzfA
c. Google ARCore (2017)
Being Shaken and then Remembering - Google ARCore, 2019
https://youtu.be/SX43xEgN_2k
2. Motion tracking with 3-D camera
a. Intel RealSense with Scene Perception (Metaio) (2014)
Real-Time Automatic Object Detection and Measurement, 2015
https://youtu.be/NRTWMBLQZ_w
b. Google Tango (2014)
Geometric Feature Extraction with Google Project Tango Tablet, 2015
https://youtu.be/98znvIIg_HY
c. Microsoft HoloLens 2 (Feb. 2019)
https://www.microsoft.com/en-us/hololens/buy
d. Apple iPad Pro with Lidar Scanner (Mar. 2020)
https://www.apple.com/newsroom/2020/03/ ... in-ipados/
e. Apple iPhone 12 Pro with Lidar Scanner (Sept. 2020)
As of now, Apple iPad Pro with LiDAR Scanner is available. CurvSurf will develop some application demos utilizing the advantages of Apple’s 3-D camera and CurvSurf’s FindSurface.
The 1st demo in short-term will be based on the current ARKit demo of
Apple ARKit: Augmented Reality Based on Curved Object Surfaces, 2019
https://youtu.be/4U4FlavRKa4
The demo will utilize the Apple’s 3-D camera and show the measurement of small 'moving' objects, e.g., balls and pipes of small radii 1 ~ 2 cm.
The 2nd demo in mid-term will be based on the previous Intel RealSense demo of single object recognition:
Augmented Reality: Real time geometric feature extraction and annotation, 2016
https://youtu.be/HGwojt6Gqrc
The demo will show a ‘3-D Scene Recognition’. First, image segmentation will determine the locations and the approximate sizes of objects in the RGB-image captured. Next, CurvSurf FindSurface will determine the shapes, sizes, positions & orientations of all the obtainable objects in scene by processing the point cloud captured. The expected frame rate is 1 ~ 3 scenes per second with FindSurface on device.
They will open a small window to the next AR.
Eventually, the source codes of the application demos will be available at GitHub CurvSurf.
Joon
The development history of CurvSurf’s AR application demos:
1. Motion tracking without 3-D camera
a. Microsoft HoloLens (2016)
HoloLens Spatial Mapping Points with Sphere Extraction, 2016
https://youtu.be/C7mLH_5QzvU
b. Apple ARKit (2017)
3-D Augmented Reality - Apple ARKit, 2018
https://youtu.be/FzdrxtPQzfA
c. Google ARCore (2017)
Being Shaken and then Remembering - Google ARCore, 2019
https://youtu.be/SX43xEgN_2k
2. Motion tracking with 3-D camera
a. Intel RealSense with Scene Perception (Metaio) (2014)
Real-Time Automatic Object Detection and Measurement, 2015
https://youtu.be/NRTWMBLQZ_w
b. Google Tango (2014)
Geometric Feature Extraction with Google Project Tango Tablet, 2015
https://youtu.be/98znvIIg_HY
c. Microsoft HoloLens 2 (Feb. 2019)
https://www.microsoft.com/en-us/hololens/buy
d. Apple iPad Pro with Lidar Scanner (Mar. 2020)
https://www.apple.com/newsroom/2020/03/ ... in-ipados/
e. Apple iPhone 12 Pro with Lidar Scanner (Sept. 2020)
As of now, Apple iPad Pro with LiDAR Scanner is available. CurvSurf will develop some application demos utilizing the advantages of Apple’s 3-D camera and CurvSurf’s FindSurface.
The 1st demo in short-term will be based on the current ARKit demo of
Apple ARKit: Augmented Reality Based on Curved Object Surfaces, 2019
https://youtu.be/4U4FlavRKa4
The demo will utilize the Apple’s 3-D camera and show the measurement of small 'moving' objects, e.g., balls and pipes of small radii 1 ~ 2 cm.
The 2nd demo in mid-term will be based on the previous Intel RealSense demo of single object recognition:
Augmented Reality: Real time geometric feature extraction and annotation, 2016
https://youtu.be/HGwojt6Gqrc
The demo will show a ‘3-D Scene Recognition’. First, image segmentation will determine the locations and the approximate sizes of objects in the RGB-image captured. Next, CurvSurf FindSurface will determine the shapes, sizes, positions & orientations of all the obtainable objects in scene by processing the point cloud captured. The expected frame rate is 1 ~ 3 scenes per second with FindSurface on device.
They will open a small window to the next AR.
Eventually, the source codes of the application demos will be available at GitHub CurvSurf.
Joon
Last edited by Joon on Sat Apr 04, 2020 1:42 am, edited 1 time in total.
- Joon
- V.I.P Member
- Posts: 335
- Joined: Wed Aug 21, 2013 8:01 pm
- 10
- Full Name: Sung Joon Ahn
- Company Details: CurvSurf
- Company Position Title: Founder + CEO
- Country: Republic of Korea
- Linkedin Profile: Yes
- Location: Seongnam-si, Korea
- Has thanked: 7 times
- Been thanked: 35 times
- Contact:
Re: We dream a magic button for 3-D point cloud processing
Apple iPad Pro LiDAR (iPad Pro 4)
The ToF 3-D camera of iPad Pro 4 has a resolution of 24 x 24 dots with ca. 50-degree FOV (i.e., the dot-spacing angle is ca. 2 degree).
Here is iPad Pro’s LiDAR under infrared
https://twitter.com/i/status/1243071065946910721
Ca. 600 (24 x 24 = 576) laser-dots with 2 deg. dot-spacing are,
enough/good to
- save the battery power
- measure relatively long-distance points
- mesh the points (dots), and
- detect fast a plane from the current frame,
short/negative to
- detect a moving object shape
- detect a small object shape.
Apple has not solved the problem of large-scale SLAM but small tabletop or room scan.
Indoor mapping:
iPad Pro LiDAR meshing using ARKit
https://youtu.be/fRUsV-esskM
Outdoor mapping:
Apple iPad Pro Lidar Scanner for Augmented Reality - Superba AR
https://youtu.be/ipcq02PEX5c
Meshing the point cloud is an integral element of AR platform:
- Microsoft HoloLens 'Spatial Mapping' (2016)
- Google ARCore 'Depth API' (Jan. 2020)
- Apple ARKit 'Scene Geometry API' (Mar. 2020).
Anyway, Apple iPad Pro LiDAR shows the direction AR software developers are looking to:
functionality, availability, supporting API, etc.
The ToF 3-D camera of iPad Pro 4 has a resolution of 24 x 24 dots with ca. 50-degree FOV (i.e., the dot-spacing angle is ca. 2 degree).
Here is iPad Pro’s LiDAR under infrared
https://twitter.com/i/status/1243071065946910721
Ca. 600 (24 x 24 = 576) laser-dots with 2 deg. dot-spacing are,
enough/good to
- save the battery power
- measure relatively long-distance points
- mesh the points (dots), and
- detect fast a plane from the current frame,
short/negative to
- detect a moving object shape
- detect a small object shape.
Apple has not solved the problem of large-scale SLAM but small tabletop or room scan.
Indoor mapping:
iPad Pro LiDAR meshing using ARKit
https://youtu.be/fRUsV-esskM
Outdoor mapping:
Apple iPad Pro Lidar Scanner for Augmented Reality - Superba AR
https://youtu.be/ipcq02PEX5c
Meshing the point cloud is an integral element of AR platform:
- Microsoft HoloLens 'Spatial Mapping' (2016)
- Google ARCore 'Depth API' (Jan. 2020)
- Apple ARKit 'Scene Geometry API' (Mar. 2020).
Anyway, Apple iPad Pro LiDAR shows the direction AR software developers are looking to:
functionality, availability, supporting API, etc.
- Joon
- V.I.P Member
- Posts: 335
- Joined: Wed Aug 21, 2013 8:01 pm
- 10
- Full Name: Sung Joon Ahn
- Company Details: CurvSurf
- Company Position Title: Founder + CEO
- Country: Republic of Korea
- Linkedin Profile: Yes
- Location: Seongnam-si, Korea
- Has thanked: 7 times
- Been thanked: 35 times
- Contact:
Re: We dream a magic button for 3-D point cloud processing
We have tested the potential use of Apple iPad Pro LiDAR.
The current and future HW/SW techniques of iPad Pro LiDAR will be applied to Apple iGlass in years.
In weeks (Late May 2020), a demo video of CurvSurf FindSurface utilizing iPad Pro LiDAR will be released.
Once the raw depth data stream of 576 ( = 24 x 24 ) laser dots rather than the ARMeshAnchor could be available, real-time object detection, recognition and measurement would be made possible. Not only the device self but also the object is allowed to move and be measured in real time.
youtu.be/oUUodYjnxfM
The current and future HW/SW techniques of iPad Pro LiDAR will be applied to Apple iGlass in years.
In weeks (Late May 2020), a demo video of CurvSurf FindSurface utilizing iPad Pro LiDAR will be released.
Once the raw depth data stream of 576 ( = 24 x 24 ) laser dots rather than the ARMeshAnchor could be available, real-time object detection, recognition and measurement would be made possible. Not only the device self but also the object is allowed to move and be measured in real time.
youtu.be/oUUodYjnxfM
- Joon
- V.I.P Member
- Posts: 335
- Joined: Wed Aug 21, 2013 8:01 pm
- 10
- Full Name: Sung Joon Ahn
- Company Details: CurvSurf
- Company Position Title: Founder + CEO
- Country: Republic of Korea
- Linkedin Profile: Yes
- Location: Seongnam-si, Korea
- Has thanked: 7 times
- Been thanked: 35 times
- Contact:
Re: We dream a magic button for 3-D point cloud processing
An in-depth report about Apple iPad LiDAR chip
With the Apple iPad LiDAR chip, Sony landed on the moon without us knowing.
With the Apple iPad LiDAR chip, Sony landed on the moon without us knowing.
- Joon
- V.I.P Member
- Posts: 335
- Joined: Wed Aug 21, 2013 8:01 pm
- 10
- Full Name: Sung Joon Ahn
- Company Details: CurvSurf
- Company Position Title: Founder + CEO
- Country: Republic of Korea
- Linkedin Profile: Yes
- Location: Seongnam-si, Korea
- Has thanked: 7 times
- Been thanked: 35 times
- Contact:
Re: We dream a magic button for 3-D point cloud processing
Tools used:
- Apple iPad Pro LiDAR
- Apple ARKit 3.5
- Apple Metal API
- CurvSurf FindSurface.
youtu.be/sMRfX334blI
- Apple iPad Pro LiDAR
- Apple ARKit 3.5
- Apple Metal API
- CurvSurf FindSurface.
youtu.be/sMRfX334blI