Software for finding repeating items/patterns in point cloud

3D Studio MAX, 3D Coat, Lightwave, ZBrush, VRMesh, TopoGun, Blender, Rhino, Trimble Sketchup Etc.

Re: Software for finding repeating items/patterns in point c

Postby Scott » Wed Apr 19, 2017 4:30 pm

neeravbm wrote:Scott, from what I understood, you want to measure dimensions within point cloud. For e.g. wall thickness, floor to ceiling height, etc. Am I correct? As I see it, the problem is that point cloud by itself doesn't know what a wall vs floor vs ceiling is. So there has to be a higher layer which segments and classifies point clouds and says that this group of points is a wall, this group of points is a floor, etc. Then you can measure distances between these higher order entities. As of now, I can't think of any way to directly measure distances between these higher order entities without actually creating these higher order entities. Please let me know if your reply was something totally different and I misunderstood it. Thanks!

Neerav,
Well stated -I appreciate your response. I guess we are stuck with vectorization and solid modeling for the moment (for dimensioning purposes). Point cloud visualization has improved immensely, so it would be nice to use the point cloud itself, as a stand alone deliverable. Visualization promotes and clarifies the project, dimensions get it built. Segmentation will likely be part of that process. Thanks!
Scott
V.I.P Member
V.I.P Member
 
Posts: 451
Joined: Tue Mar 29, 2011 7:39 pm
Full Name: Scott Page
Company Details: Scott Page Design- Architectural service
Company Position Title: Freelance
Country: USA
Linkedin Profile: No

Re: Software for finding repeating items/patterns in point c

Postby jedfrechette » Wed Apr 19, 2017 4:41 pm

neeravbm wrote:If you had this software which could be instructed what to recognize, how often will you use it?

Our use case might be a little unique since we rely heavily on automatic model generation from point clouds rather than hand modeling, but classification regularly accounts for 50% or more of our post-processing time. With better tools I could easily see that time being cut in half.

neeravbm wrote:Can they provide separate photographs as output as well? If yes, are these 360 photographs or regular ones? With each photograph, do we also get the scanner position from where it was captured?

In some cases yes, but I think the challenge is that having separate photos (or even a colored point cloud) isn't guaranteed. Due to the extra time it takes on-site we rarely scan with color enabled, so most of our point clouds are x, y, z, intensity. Data collected with mobile platforms like GeoSLAM also commonly lack color information.

In addition to color, there are a bunch of other data channels that could potentially be used as classification features. For example, although we typically just work with the coordinates of each point, those coordinates actually represent a vector from the scanner to the measurement point and that vector should not pass through solid surfaces like walls. With static scans it is also pretty easy to estimate good surface normals for each point to provide even more information about its geometry.

neeravbm wrote:When you first start using the software, it will have no intelligence. You will have to start teaching it. So you'll select an object a few times in the point cloud and classify it as Object 1. This will be stored in your personal library now. You'll create many such objects from multiple point cloud files. Once the tool has been taught, any time you open a new point cloud file, it will find those objects for you automatically. Any machine learning system is going to have a few errors. So you'll need to validate the object that this tool finds. But in effect, you are creating a library of objects that you personally are interested in and that you personally come across in many of your different projects. Am I correct?

Exactly. I think the Training->Classification->Validation->Training feedback loop is critical and something that is currently being ignored at the user level. Yes the software should come with some pre-trained classifiers for common universal features, but I think the real value would start being realized once users can start training the system to detect features the developers never thought of. Going one step beyond that advanced users should be able to build their own classifiers. My ideal software would essentially be a classification platform that comes with batteries included. New users would be able to push a button and get a result using builtin pretrained classifiers. Intermediate users would be able to train the built in classifiers using their own data. Advanced users would be able to build their own classifiers from the ground up.

A couple examples of similar software for image classification are ERDAS Imagine for remote sensing data:

https://youtu.be/N6rjV2k-s18

and at a smaller scale ilastik:

http://ilastik.org/index.html
Jed
jedfrechette
V.I.P Member
V.I.P Member
 
Posts: 722
Joined: Mon Jan 04, 2010 8:51 pm
Location: Albuquerque, NM
Full Name: Jed Frechette
Company Details: Lidar Guys
Company Position Title: Lidar Supervisor
Country: USA
Linkedin Profile: Yes

Re: Software for finding repeating items/patterns in point c

Postby Scott » Wed Apr 19, 2017 7:26 pm

Jed, Nice post and links -a good discussion overall... +1
Scott
V.I.P Member
V.I.P Member
 
Posts: 451
Joined: Tue Mar 29, 2011 7:39 pm
Full Name: Scott Page
Company Details: Scott Page Design- Architectural service
Company Position Title: Freelance
Country: USA
Linkedin Profile: No

Re: Software for finding repeating items/patterns in point c

Postby neeravbm » Thu Apr 20, 2017 4:50 pm

Jed, I have a few quick questions for you. You mentioned that you rely on automatic model generation from point cloud and classification accounts for 50% of your post-processing time. What kind of models do you generate automatically from point cloud? Is it pipes using ClearEdge3D or is it more than pipes? Also what do you exactly expect the software to do in terms of classification? From what I understand, software will find locations of different objects in your point cloud. But how you want to classify that object, for e.g. call it a pump, or a valve, or a chair, is up to you, right?
neeravbm
I have made 10-20 posts
I have made 10-20 posts
 
Posts: 13
Joined: Thu Mar 16, 2017 4:29 pm
Full Name: Neerav Mehta
Company Details: Red Crackle
Company Position Title: Founder
Country: USA
Linkedin Profile: No

Re: Software for finding repeating items/patterns in point c

Postby jedfrechette » Wed Apr 26, 2017 12:21 am

neeravbm wrote:What kind of models do you generate automatically from point cloud? Is it pipes using ClearEdge3D or is it more than pipes?

When I said modeling I should have clarified that I was referring to lower level surface modeling, rather than object modeling. If we have different levels of data processing, e.g.:

Level 1: Measurements (e.g. point clouds)
Level 2: Surface (e.g. 3D polygonal mesh, terrain model)
Level 3: Features (e.g. valve, pump, chair, etc.)

Software like ClearEdge attempts to go from Level 1 to Level 3, which is a perfectly valid approach, but Level 3 features don't necessarily provide the same functionality as a Level 2 surface model. Most of our work requires Level 2 surfaces, which means that it is highly sensitive to the quality of the point cloud. If you have a person walking through your scan ClearEdge doesn't care because that person doesn't look like a pipe. If we build a surface model from that scan though, that person will be part of the model, which is usually not desireable.

neeravbm wrote:Also what do you exactly expect the software to do in terms of classification? From what I understand, software will find locations of different objects in your point cloud. But how you want to classify that object, for e.g. call it a pump, or a valve, or a chair, is up to you, right?

I would like software that I can use to efficiently assign points to arbitrary classes. I see creating the actual model, whether that is a Level 2 surface or a Level 3 feature as a separate process (although in many cases tightly coupled to the classification step). Exactly what objects are interesting can and does vary on a day-to-day and project-by-project basis. Also, as alluded to in the person example above, in many cases classification is as much about identifying points that represent things I want to exclude from the model as it is about identifying the points representing objects I want to model.

To give more concrete examples, on Monday, I might be scanning a city street with cars parked along the curb. I'm only interested in the street and the surrounding buildings so I need a classifier that can identify cars so that they can be excluded from my model. On Tuesday, I'm scanning a car to reconstruct it, so now I need a classifier tuned to identify the types of noise common in car scans, e.g. specular reflections off the headlights, mixed pixels and bogus distance measurements around the edges of the windshield, etc. On Wednesday I'm scanning a golf course and need to locate sprinkler heads marked with pink flags, so now I need a classifier that can identify pink flags. Those are all very specific scenarios and I could easily come up with many more. I think most people will have their own equally unique requirements, which is why I'm more interested in good tools rather than inflexible pre-built solutions that can't adapt to changing and diverse user needs.

To be generally useful I think any classification software would need to be a hybrid system, with both good manual tools that help humans work efficiently and perform the tasks machines can't do and automatic tools that can provide efficiencies wherever possible. The manual tools probably need to come first, after all if you want to train a classifier to identify sprinklers, first you're going to need to identify 1000 sprinklers by hand so you have labeled training data to build your classifier with.
Jed
jedfrechette
V.I.P Member
V.I.P Member
 
Posts: 722
Joined: Mon Jan 04, 2010 8:51 pm
Location: Albuquerque, NM
Full Name: Jed Frechette
Company Details: Lidar Guys
Company Position Title: Lidar Supervisor
Country: USA
Linkedin Profile: Yes

Re: Software for finding repeating items/patterns in point c

Postby neeravbm » Thu Apr 27, 2017 2:16 pm

Thanks, Jed, for the reply!

You mentioned that you want to efficiently assign points to arbitrary classes so that you can either convert points to a model or remove those points from your cloud. Do you spend more time finding the location of these classes in point cloud or actually assigning points to these classes? The reason I ask is finding the location of classes is much easier than assigning all the points to them. In other words, if the software gave you a 3D bounding box where most of the points belonged to your class, will it be useful to you?

It is interesting that you gave example of a car. Car is a category and is a higher-level construct than a specific model. To a machine, a Tesla S looks very different than a BMW. It's much easier to highlight a Tesla S in a point cloud and ask the software to find all other Tesla S's rather than highlight a Tesla S and ask the software to find all other cars. Latter requires lot more training than former. As first version of the software, is it still useful to you if the software finds exactly the same object that you highlighted rather than all the objects that fall under that category?
neeravbm
I have made 10-20 posts
I have made 10-20 posts
 
Posts: 13
Joined: Thu Mar 16, 2017 4:29 pm
Full Name: Neerav Mehta
Company Details: Red Crackle
Company Position Title: Founder
Country: USA
Linkedin Profile: No

Re: Software for finding repeating items/patterns in point c

Postby Scott » Sat Apr 29, 2017 10:35 pm

I don't want to hijack this thread -the following links do relate to the topic, though a separate thread for 'Image Processing' may also be needed if there is sufficient interest. I started looking for software used in medical imaging (MRI's CT scans), curious to see what was out there, and stumbled upon MATLAB and other resources. It was clear that the scanning world was, of course, a small branch of a much larger tree. 2D images and 3D constructs (point clouds & solid models) often overlap, so it's important to know the terminology in order to ask the right questions. To extract dimensions from point clouds and organize them in a sensible way, it's helpful to understand some of the processes needed to get you there: 'edge detection', 'segmentation', vectorization, and 2D image processing. The following videos (and chapter) do an excellent job explaining these terms:

IMAGE PROCESSING MADE EASY - MATLAB Video:
https://www.youtube.com/watch?v=1-jURfDzP1s
EDGE DETECTION WITH MATLAB -Video:
https://www.youtube.com/watch?v=Z4msmuJNpI4
CANNY EDGE DETECTOR - Computerphile Video:
https://www.youtube.com/watch?v=sRFM5IEqR2w
EDGE DETECTION AND SEGMENTATION- Book, Chapter 5:
https://pixhawk.ethz.ch/_media/eth_only/books/6-embedded-image-processing-dsp-edge-detection-segmentation.pdf
Scott
V.I.P Member
V.I.P Member
 
Posts: 451
Joined: Tue Mar 29, 2011 7:39 pm
Full Name: Scott Page
Company Details: Scott Page Design- Architectural service
Company Position Title: Freelance
Country: USA
Linkedin Profile: No

Previous

Return to Modeling Software

Who is online

Users browsing this forum: Google [Bot] and 1 guest

 
Stay in touch    RSS feed    Twitter    Youtube    Linkedin

LSF SPONSOR

Accessibility | Terms of Use | Privacy Policy
DouZa Webdesign