We dream a magic button for 3-D point cloud processing

aurivus|REVIT, 3D Studio MAX, 3D Coat, Lightwave, ZBrush, VRMesh, TopoGun, Blender, Rhino, Trimble Sketchup Etc.
Post Reply
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Physical ads displays will be superfluous.
Physical object surfaces will provide virtual ads display surfaces.
------------------------------------

Chungmuro station Line 3 - iPhone Pro 12

youtu.be/BmKNmZCiMkw

https://en.wikipedia.org/wiki/Chungmuro_station

Chungmuro Station is on Line 3 and line 4 of the Seoul Subway system.
Platforms for both Line 3 and Line 4 are in Chungmuro-4-ga, Jung-gu, Seoul.

This station is named after the road under which it passes, in honor of the Chosun general Yi Sunsin, who was also known by the title of Chungmugong.

---------------------------------------
Once, the shape, size, position, orientation of an object surface is known, there are a variety of AR applications.

YouTube CurvSurf, GitHub CurvSurf, FindSurface Web Demo.

The source code for AR overlaying/rendering virtual image/video around/on the real geometric object primitives extracted from the LiDAR point cloud is available.
https://github.com/CurvSurf/FindSurface ... ARDemo-iOS

#spatialcomputing #realtime #automation #robotics #ar #computervision #surface #fitting #pointcloud #curvature

Tools used:
- iPhone 12 Pro
- Apple ARKit
- Apple SceneKit
- Apple Metal API
- CurvSurf FindSurface runtime library.

3-D, least squares, analytic geometry, geometric primitive, differential geometry, principal curvature, principal direction, fitting, curve, surface, orthogonal distance fitting, ODF, robust fitting, point cloud, segmentation, region growing, outlier elimination, object recognition, reverse engineering, CAD CAM, BIM, augmented reality, occlusion, curve fitting, surface fitting, plane fitting, sphere fitting, cylinder fitting, cone fitting, torus fitting, piping, reducer, elbow, box, cuboid, radius, length, size, position, location, orientation, pose, rotation, axis, Software, ISO 10360-6, ARKit, ARCore, HoloLens, RealSense, FindSurface, LiDAR, ToF, laser scanner, iPad Pro, iPhone Pro
Last edited by Joon on Fri Mar 10, 2023 11:32 am, edited 1 time in total.
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

As far as I observed, Apple has been improving the SOFTWARE performance of LiDAR and ARKit since their release in 2020 and 2017, respectively.

The overall performance of the combination of Apple's LiDAR and ARKit can be checked by using CurvSurf's FindSurface runtime library:
1. 3D measurement accuracy of LiDAR
2. Robustness of DepthMap against darkness
3. Accuracy of the motion tracking of ARKit
4. Robustness of the motion tracking of ARKit against darkness and device's shaking.

CurvSurf's FindSurface runtime library determines the shape, size, position, and orientation of an object surface by processing the 3D measurement points according to the orthogonal distance fitting.

The accuracy of the orthogonal distance fitting algorithms adopted by CurvSurf's FindSurface runtime library is approved by the German PTB according to ISO 10360-6 (see my PhD thesis, ISBN 3540239669, 2004).

If there are somewhat shortages in the overall performance of Apple's LiDAR and ARKit mentioned above, we will observe some misalignment between the real object surfaces and the virtual ads.

In the same way, we can check the overall performance of '3D measurement and motion tracking' of Google ARCore and Microsoft HoloLens.

Virtual Ads inside Chungmuro station Line 3 - iPhone Pro 12:
https://youtu.be/BmKNmZCiMkw

The source code of the App producing the above video is available on GitHub CurvSurf:
FindSurface-SceneKit-ARDemo-iOS
https://github.com/CurvSurf/FindSurface ... ARDemo-iOS
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Why FindSurface?

A solution to the problem of 3D object detection, measurement and extraction from 3D point cloud data:
- FindSurface can detect and fit various geometric primitives such as planes, spheres, cylinders, cones and tori from noisy and incomplete 3D point clouds in real time.
- FindSurface does not require any prior knowledge or assumptions about the shape, size, orientation or location of the objects in the scene.
- FindSurface also does not require any preprocessing or segmentation of the point cloud data.
- FindSurface can handle point clouds from any source, such as LiDAR, structured light, stereo vision or photogrammetry.
- FindSurface can also handle point clouds with varying density, resolution and scale.
- FindSurface can provide accurate and robust results even in challenging scenarios such as occlusion, clutter, outliers and overlapping objects.
- FindSurface can be used for various applications such as 3D scanning, reverse engineering, inspection, quality control, augmented reality and virtual reality.
- FindSurface is a breakthrough solution that solves the long-standing problem of 3D object recognition, measurement and extraction from 3D point cloud data in a fast, easy and reliable way.

CurvSurf FindSurface is a solution to the problem of 3D object recognition, measurement and extraction from 3D point cloud data.

A well-known problem in computer vision, robot vision, machine vision, and coordinate metrology is how to recognize, measure and extract 3D object geometries including their parameters from 3D point cloud data.

A measurement point is a probable observation of an unknown real object point next to the measurement point. The unknown measurement error is the distance between the measurement point and an unknown real object point.

The maximum likelihood estimation is minimizing the unknown measurement error down to zero by substituting an unknown real object point with the measurement point.

For 3D object surface measurement, the maximum likelihood estimation is minimizing the sum of the squared distances between a 3D model surface and the 3D measurement points.

FindSurface SDK as a solution to this problem is available at GitHub CurvSurf.
https://github.com/CurvSurf/FindSurface

A web demo is also available.
https://developers.curvsurf.com/WebDemo .
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Apple Vision Pro: Spatial Monitor, Not Computer!
john-newbegin
V.I.P Member
V.I.P Member
Posts: 141
Joined: Wed Apr 29, 2015 4:06 pm
8
Full Name: John Newbegin
Company Details: Travis County Sheriff Engineering
Company Position Title: Engineering Specialist
Country: USA
Linkedin Profile: Yes
Has thanked: 9 times
Been thanked: 7 times

Re: We dream a magic button for 3-D point cloud processing

Post by john-newbegin »

does any of this work with Android, phones, tablets etc?

great stuff your working on each time I check in I am more impressed with your progress, what a time to be alive!

John
nyda
I have made 10-20 posts
I have made 10-20 posts
Posts: 19
Joined: Wed Jul 11, 2018 8:08 pm
5
Full Name: Giovanni Scocchera
Company Details: New York Design Architects
Company Position Title: New York Design Architects
Country: United States
Has thanked: 34 times
Been thanked: 1 time

Re: We dream a magic button for 3-D point cloud processing

Post by nyda »

Joon wrote: Sun Jun 11, 2023 1:30 am Apple Vision Pro: Spatial Monitor, Not Computer!
Do you believe the M2 chip inside is for visual computation and not regular PC tasks?
The R1 should do that part, so that the Vision Pro is a Macbook Mini Air attached to your eyes :lol:

Gio
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Apple Vision Pro: Spatial Monitor, Not Computer

An unspoken reality. Apple doesn't want anyone to know.


Apple unveiled Vision Pro at WWDC2023. The screens are anchored in space and the users are sitting or standing without movement. Such conditions are required for, because the screens are blocking the space so you can't see beyond them. Otherwise, Apple Vision Pro may cause a safety accident, especially outdoors.

The safest way to place a screen in space is to place it inside a real surface. It not only ensures the user's safety, but also provides comfort. To do that, we need to know exactly the shape, size, position, and orientation of the real surface.

CurvSurf’s FindSurface SDK provides the solution.
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

john-newbegin wrote: Wed Jun 21, 2023 2:54 pm does any of this work with Android, phones, tablets etc?

great stuff your working on each time I check in I am more impressed with your progress, what a time to be alive!

John
Thanks John,

the last stuff we created using Android devices is dated back late 2020.
https://www.youtube.com/CurvSurf
https://github.com/CurvSurf

As dToF LiDAR 3D camera was added to iPad Pro and iPhone Pro (Max), we concentrated our works in iOS/iPadOS.

Once an XR headset from Samsung, Google, and Qualcomm possibly with 3D camera, we will show some applications running on the device using FindSurface SDK.

Joon
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

The main motion tracking difference between Apple ARKit and Google ARCore is the distance range used by SLAM to collect and store feature points.

While Apple ARKit collects feature points up to 10m from the device, it does not store them. Google ARCore collects feature points even from clouds in the sky (i.e. at infinite distance) and stores them (SW techniques inherited from Google Tango's Area Learning).

As soon as devices return to a previous location, Google ARCore remembers this, but Apple ARKit does not. This may be one reason Apple Vision Pro restricts the device's range of motion within 5 feet of the home position. Otherwise, the misalignments between the anchored screens and the real object surfaces will be noticeable/visible (e.g., at 1m35sec of videos below).

Solar Power Generator - Apple ARKit

youtu.be/4cp7Qg6WCFw
https://youtu.be/4cp7Qg6WCFw?t=95

Solar Power Generator - Google ARCore

youtu.be/FMytXWXQ6wg
https://youtu.be/FMytXWXQ6wg?t=95
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Another series of video showing the performance differences in indoor motion tracking between Apple ARKit, Google ARCore and Microsoft HoloLens.

Motion tracking performance:
Google ARCore > Microsoft HoloLens > Apple ARKit.

It will not change today.

Extraction of plane and spheres

youtu.be/QhBtGHmfBOg

youtu.be/MPw__oryLm4

youtu.be/5sEnrZW3yks
Post Reply

Return to “Modeling Software”