We dream a magic button for 3-D point cloud processing

aurivus|REVIT, 3D Studio MAX, 3D Coat, Lightwave, ZBrush, VRMesh, TopoGun, Blender, Rhino, Trimble Sketchup Etc.
Post Reply
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Advance notice
https://github.com/CurvSurf/FindSurface ... nsDemo-UWP

User Interface

This app supports eye-tracking features and voice commands, both built-in features of HoloLens2. During the holographic experiences using this app, Users gaze at target places in the space and activate FindSurface API functions using voice commands.

Gazing at target surfaces (Eye-tracking)

The gazing point follows where users' eyes gaze. If the eye-tracking feature isn't available for reasons like an inconvenience with eye-tracking or calibration failures, etc., disable the eye-tracking features in the settings panel. The app will allow users to use their hands as an alternative to eye-tracking. If no hands are detected, the traditional way (head tracking like what users do in HoloLens 1) is used.
Using the gazing point, users point to a place to detect surfaces and how large the target surface is. The circle around the gazing point implies the size of the target object to detect. Users can change the size using the voice commands.

Eye Tracking - HoloLens 2

youtu.be/2nDnWzOr0lo
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Now, the source codes of the demo App 'HolographicFindSurfaceDemo' are public.
FindSurface-HoloLensDemo-UWP

Overview
This demo is a Holographic UWP application that demonstrates FindSurface API with raw depth stream (research mode). The FindSurface library for UWP is included in the ext directory in this demo project.

Prerequisites
HoloLens 2 device (HoloLens 1 is not supported).
Visual Studio 2019 or higher.
Install the latest Mixed Reality tools.

Setup
Enable Device Portal and Research Mode.

Joon
Last edited by Joon on Sun Feb 13, 2022 9:55 pm, edited 1 time in total.
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Augmented Reality on Site

We have the use cases of:
1. Creation of as-built 3D CAD information for Digital maps, Advertisements, Game environment, ...
2. Overlaying 3D CAD information on hand onto the real objects.
3. ...

We can overlay the 3D CAD information on hand onto the buildings and structures, if we know the sizes, positions & orientations of real:
- 3 balls
- 3 non-parallel planes
- 2 non-parallel cylinders
- 2 parallel cylinders and a plane (the case of the video recorded)
- ...

Practically, the existing man-made stationary structures of planes, spheres, cylinders, cones, or tori can be used as permanent anchors with 6DoF info.

FindSurface Outdoors - HoloLens 2

youtu.be/cH1jKGrOP6Y
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Every real space is a potential advertising space.
It becomes more immersive with AR glasses.

The stakes are huge.

Apple, Google, Meta (Facebook), Microsoft are striving to ...

AR Smart Signage - Apple iPhone 12 Pro

youtu.be/eSGY49ND6dY
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

AR Smart Signage on Cylindrical Surface

youtu.be/fEQa1nTNJqQ

#spatialcomputing #virtualadvertising #virtualsignage #smartsignage #augmentedreality #iphonepro #ipadpro #lidar #arkit #pointcloud

Most man-made object surfaces consist of planes, spheres, cylinders, cones and tori.

Why? Because the shapes and movements of the tool are combinations of lines and circles.

The shapes and poses of man-made object surfaces obey Mother Nature's law; gravitation and dynamics.

Man-made object surfaces are ubiquitous.

The future market for AR displays on man-made object surfaces is huge.

The FindSurface runtime library detects and measures the shape, size, position and orientation of artificial object surfaces.

Tools used:
- Apple iPhone 12 Pro
- Apple ARKit 4
- Apple Metal API
- CurvSurf's FindSurface runtime library .

The FindSurface runtime libraries for Android, iOS, Windows, UWP, Linux along with the source codes of demo apps are available on GitHub CurvSurf.
https://github.com/CurvSurf/FindSurface
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Bill Gates personally reviewed the state-of-the-art AR glasses & HMD hardware. I bet the next consideration will relate to the software.
https://twitter.com/BillGates/status/15 ... 2900017154
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

The source codes of the demo App 'FindSurface-SceneKit-ARDemo-iOS' are public.

Overview
This demo app demonstrates a real-time AR SceneKit application using FindSurface to search point clouds, which ARKit provides, for geometry shapes. The source code of this app is based on the Xcode Augmented Reality (SceneKit, iOS) template project. FindSurfaceFramework is required to build the source code into a program. Download the framework here and refer to here for an instruction about how to setup your project to build it with the framework.

Prerequisites
This demo app requires the LiDAR sensor to get a point cloud from ARKit:
  • Apple iPad Pro LiDAR
  • Apple iPhone 12 Pro
  • Apple iPhone 12 Pro Max
  • Apple iPhone 13 Pro
  • Apple iPhone 13 Pro Max
Output Example
FindSurface-SceneKit-ARDemo-iOS LiDAR

youtu.be/7zSUrrm_RiU

Joon
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

FindSurface Apple Myeongdong - iPhone 13 Pro Max


youtu.be/dzDXzQXSFOM

Apple Store Myeongdong Seoul

#spatialcomputing #virtualadvertising #virtualsignage #smartsignage #augmentedreality #iphonepro #ipadpro #lidar #arkit #pointcloud

The source code of the App is public on GitHub CurvSurf.

Once you know the shape, size, position and orientation of an object surface, you can augment/render anything onto/around it.

1) Your AR device measures in real-time:
- 6DoF motion
- object point cloud.

2) CurvSurf FindSurface determines/measures in real-time:
- shape
- size
- position
- orientation of the object surface by processing the point cloud.

Every real space is a potential advertising space.
It becomes more immersive with AR glasses.

Sculpture:
Lee Jaehyo
0121-1110=122 (Wood) 2022
http://leejaehyo.com
http://www.leeart.name
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

NVIDIA Research Turns 2D Photos Into 3D Scenes in the Blink of an AI

Now, scanning is real-time in 2D and 3D space.
In 2D or 3D space, scanning is seeing.
2D scanning results are photos and 3D scanning results are point clouds.
Understanding the geometric contents in photos or point clouds is the same: shape, size, position & rotation of objects in 2D/3D space.

2D-line segments or 3D meshes are artifacts for visualizing the geometric contents understood by machine/computer/human.

Real time understanding the 2D/3D geometric contents can be accomplished only by processing directly the 2D/3D point cloud and RGB data (raw measurement data) before 2D-line segments or 3D meshes.

Joon
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

'RoomPlan' is released on Apple WWDC 2022.

Introducing RoomPlan
Create parametric 3D room scans with RoomPlan

The object surfaces shown are planes/boxes.

CurvSurf are planning to develop and release 'SpacePlan' including source code.
The object surfaces will be curved.

Due End 2022.
Post Reply

Return to “Modeling Software”