Help- Point Cloud creation with larger than our normal data set

Discuss FARO SCENE software here.
User avatar
smacl
Global Moderator
Global Moderator
Posts: 1409
Joined: Tue Jan 25, 2011 5:12 pm
13
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 627 times
Been thanked: 657 times
Contact:

Re: Help- Point Cloud creation with larger than our normal data set

Post by smacl »

Justin Richards wrote: Fri Apr 28, 2023 8:23 pmHere is mine. This was at 54% and went on for almost a week before completing.
Image
That's unusual insofar as none of the resources are maxxed out and the CPU is barely above idle, certainly less than 100% of one core. Given the highest resource usage listed is disk, and Scene is only getting 1.5MB/s of this, I'd guess something else is interrupting and messing with disk access, again possibly some security software. For reference, top NVMe drives write at about 7GB/s, but even slower ones will get 1GB/s which is about 600 times faster than what Scene is getting in this graph. Something like a virus scanner can interrupt each write access and cause a massive slowdown here. Could also be backup or disk-mirroring software (e.g. OneDrive, DropBox, Acronis etc...) that needs to be told to avoid certain disks, folders and filetypes. The other possibility is memory exhaustion, but even with paging taking place, the figure of 47% in the graph suggests this isn't the problem in this case.

The problem with using task manager to look at these types of problems is that it does not log resource usage over time, it just gives you a snap shot. For more detailed analysis, have a look at perfmon.
Shane MacLaughlin
Atlas Computers Ltd
www.atlascomputers.ie

SCC Point Cloud module
User avatar
robjlg
I have made 50-60 posts
I have made 50-60 posts
Posts: 54
Joined: Mon Aug 19, 2019 11:18 am
4
Full Name: Roberto João Lopes Garcia
Company Details: Laser scanning services
Company Position Title: Owner
Country: BRAZIL
Skype Name: robjlg
Linkedin Profile: Yes
Has thanked: 24 times
Been thanked: 9 times
Contact:

Re: Help- Point Cloud creation with larger than our normal data set

Post by robjlg »

This very slow PPC generations process is happen to me too. The task manager show very small use of CPU and DISK (SSD) only the memory use is at 50%.

After 16 hours of processing it was only at 12% of the job.

I wonder if it would be doe to some windows problem or doe to the SCENE itself.

I think SCENE should use most of the resources available to have the job done fast. It does not appear to be the case.
badam
V.I.P Member
V.I.P Member
Posts: 916
Joined: Tue May 11, 2021 5:36 pm
2
Full Name: Adam Berta
Company Details: InnoScan 3D Hungary Kft
Company Position Title: unknown
Country: Hungary
Linkedin Profile: No
Has thanked: 51 times
Been thanked: 297 times
Contact:

Re: Help- Point Cloud creation with larger than our normal data set

Post by badam »

these two topic 100% the same issue.

viewtopic.php?f=44&t=20278

What i would recommend you is the same, just create "project pointcloud with other softwares, like cloud comapre, pdal.

I've just exported a project with 521 rtc 360 setup most of them are small res (12mm@10m) and maybe a 100 setup middle res (6mm@10m) yesterday, exported separated structured e57 from reg 360.

export started: 20:03
export finished: 23:44
our subsample process finished at 3:05, where we run a SOR filter on the whole dataset, create subsampled PPC for each floor (currently 9) and one "overview cloud". All of which are created such as it can be imported to archicad (around 120m point max). Also generated potree viewer, with a higher res project point cloud with 10mm raster and images/separated image cloud with 5mm raster. And of course the same subsampled clouds imported to recap as well.

processing computer: SSD only for tempfiles. project exported from nas to nas (10G). subsampling through nas. temp storage only used for recap and potree generation. CPU ryzen 9 3900X, 128GB ram.
All is done with scripting/pdal/cloud compare. Those are the best tools for pointcloud processing. everything is paralellized by the script if possible. 90% of the time it is pegged at 100% cpu utilization. network is not heavily used but i can see sometimes 4-5 gbit/s. especially when pdal reads files with 24 thread, or when register 360 writes the final files to the nas.
User avatar
ProCro
V.I.P Member
V.I.P Member
Posts: 151
Joined: Thu Jun 27, 2019 2:00 pm
4
Full Name: Nino Skupnjak
Company Details: SKIMI64
Company Position Title: procurator
Country: Croatia
Linkedin Profile: Yes
Location: Croatia
Has thanked: 7 times
Been thanked: 30 times
Contact:

Re: Help- Point Cloud creation with larger than our normal data set

Post by ProCro »

Many times, just restarting the Scene helps. I'm guessing some kind of bug in Scene...
tipwynn
I have made 10-20 posts
I have made 10-20 posts
Posts: 17
Joined: Thu Feb 21, 2019 9:18 pm
5
Full Name: thiep nguyen
Company Details: government
Company Position Title: scanner
Country: usa
Linkedin Profile: No
Has thanked: 12 times
Been thanked: 4 times

Re: Help- Point Cloud creation with larger than our normal data set

Post by tipwynn »

So after trying a couple variations. Removing McAfee and switching the scratch disk from our GRAID to an internal drive we were able to process the point cloud in 3 hours. This is much better than 3 days.
User avatar
smacl
Global Moderator
Global Moderator
Posts: 1409
Joined: Tue Jan 25, 2011 5:12 pm
13
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 627 times
Been thanked: 657 times
Contact:

Re: Help- Point Cloud creation with larger than our normal data set

Post by smacl »

tipwynn wrote: Wed May 03, 2023 6:33 pm So after trying a couple variations. Removing McAfee and switching the scratch disk from our GRAID to an internal drive we were able to process the point cloud in 3 hours. This is much better than 3 days.
Well done, be interesting to know if McAfee or the GRAID was the main culprit. Guessing McAfee, but just a guess
Shane MacLaughlin
Atlas Computers Ltd
www.atlascomputers.ie

SCC Point Cloud module
Post Reply

Return to “FARO SCENE”