cyclone to Potree

Discuss all Leica Cyclone, Cyclone REGISTER 360 & Cyclone FIELD 360 software here.
Post Reply
MalteHC
V.I.P Member
V.I.P Member
Posts: 200
Joined: Mon May 05, 2014 11:19 am
9
Full Name: Malte Holm
Company Details: TLS
Company Position Title: Surveyor
Country: Norway
Linkedin Profile: Yes
Has thanked: 2 times
Been thanked: 1 time

cyclone to Potree

Post by MalteHC »

Hi All

I have done a lot of road scannings.
Lately my company has gotten Potree, where we want to upload a lot of this date, to share with our customers.
My regular procedure is to export an e57, use lastools to convert it into las and finally use the potree converter.
When i do this procedure, the computer runs i somekind of a loop in the tileing process, where it never ends. I can see in the commandopromt that the tileing process runs slower, as it gets further into the process and after around 24 hours, it almost doesnt calculate anymore.
I have around 100 gb of data, i try to convert.

Does anyone have experienced this kind of issue?
User avatar
LeicaUK_RC
V.I.P Member
V.I.P Member
Posts: 777
Joined: Sun Jul 17, 2011 9:30 am
12
Full Name: Phil Marsh
Company Details: Leica Geosystems part of Hexagon
Company Position Title: Reality Capture Director UK
Country: UK
Skype Name: Phil.H.Marsh
Linkedin Profile: Yes
Location: Barnsley, UK
Been thanked: 52 times
Contact:

Re: cyclone to Potree

Post by LeicaUK_RC »

Why not get PUBLISHER Pro and go directly to LAS to avoid the e57?
Leica Geosystems
M: 07540 201383
E: [email protected]
W: www.leica-geosystems.com
Ioannis
V.I.P Member
V.I.P Member
Posts: 627
Joined: Sun Jul 24, 2016 6:19 am
7
Full Name: Ioannis Papas
Company Details: techical office
Company Position Title: CEO
Country: EU
Skype Name: -
Linkedin Profile: Yes
Has thanked: 99 times
Been thanked: 62 times

Re: cyclone to Potree

Post by Ioannis »

Leica dealers all the time buy software from Leica plus ccp plus $$$
User avatar
smacl
Global Moderator
Global Moderator
Posts: 1409
Joined: Tue Jan 25, 2011 5:12 pm
13
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 627 times
Been thanked: 657 times
Contact:

Re: cyclone to Potree

Post by smacl »

MalteHC wrote: Tue May 26, 2020 7:05 am Hi All

I have done a lot of road scannings.
Lately my company has gotten Potree, where we want to upload a lot of this date, to share with our customers.
My regular procedure is to export an e57, use lastools to convert it into las and finally use the potree converter.
When i do this procedure, the computer runs i somekind of a loop in the tileing process, where it never ends. I can see in the commandopromt that the tileing process runs slower, as it gets further into the process and after around 24 hours, it almost doesnt calculate anymore.
I have around 100 gb of data, i try to convert.

Does anyone have experienced this kind of issue?
100gb in one LAS file or in a number of smaller LAS files? I'd expect the potree converted to take a few hours for this but 24 hours seems very excessive. Like any slow process, you need to figure out where the bottleneck is. The potree converter is basically building an octree comprising of lots of small files so really benefits from a very fast SSD. Also look at increasing the --flush-limit parameter to use more memory and reduce the number of disk writes. Potree is a great package but the documentation isn't so good and you really need to experiment a bit with it to get the best out of it.
Shane MacLaughlin
Atlas Computers Ltd
www.atlascomputers.ie

SCC Point Cloud module
User avatar
geomontgomery
V.I.P Member
V.I.P Member
Posts: 186
Joined: Thu Sep 13, 2018 3:23 pm
5
Full Name: George Montgomery
Company Details: ECE Design
Company Position Title: Support Manager
Country: USA
Linkedin Profile: Yes
Has thanked: 20 times
Been thanked: 48 times

Re: cyclone to Potree

Post by geomontgomery »

I used the same procedure you did to play around with potree. My cloud was 1/4 the size of yours though It did work in the end. Perhaps you're running out of temp space when the cloud is trying to convert? Have you tested with smaller clouds first just to see if you can get any product out?
MalteHC
V.I.P Member
V.I.P Member
Posts: 200
Joined: Mon May 05, 2014 11:19 am
9
Full Name: Malte Holm
Company Details: TLS
Company Position Title: Surveyor
Country: Norway
Linkedin Profile: Yes
Has thanked: 2 times
Been thanked: 1 time

Re: cyclone to Potree

Post by MalteHC »

It is in a single LAS.
It is quite a lot, especially doing to i have a very powerfull computer, dedicated to this kind of work.
My computer has 128 gb of ram and 2TB m.2 disk drive.

Where do i control the flush limit?

smacl wrote: Tue May 26, 2020 12:10 pm
MalteHC wrote: Tue May 26, 2020 7:05 am Hi All

I have done a lot of road scannings.
Lately my company has gotten Potree, where we want to upload a lot of this date, to share with our customers.
My regular procedure is to export an e57, use lastools to convert it into las and finally use the potree converter.
When i do this procedure, the computer runs i somekind of a loop in the tileing process, where it never ends. I can see in the commandopromt that the tileing process runs slower, as it gets further into the process and after around 24 hours, it almost doesnt calculate anymore.
I have around 100 gb of data, i try to convert.

Does anyone have experienced this kind of issue?
100gb in one LAS file or in a number of smaller LAS files? I'd expect the potree converted to take a few hours for this but 24 hours seems very excessive. Like any slow process, you need to figure out where the bottleneck is. The potree converter is basically building an octree comprising of lots of small files so really benefits from a very fast SSD. Also look at increasing the --flush-limit parameter to use more memory and reduce the number of disk writes. Potree is a great package but the documentation isn't so good and you really need to experiment a bit with it to get the best out of it.
User avatar
smacl
Global Moderator
Global Moderator
Posts: 1409
Joined: Tue Jan 25, 2011 5:12 pm
13
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 627 times
Been thanked: 657 times
Contact:

Re: cyclone to Potree

Post by smacl »

--flush-limit is on the potreeconverter command line so just add something like --flush-limit 10000000 to increase it to ten million points. You could also try playing around with the spacing to see if that improves things. Working with SCC, ff the data is from a very high density scanner such as an RTC360 I tend to reduce so that points within 3mm of one another are deleted which can cut the file size down by about 60%. Very easy to get masses of points you don't want or need very close to the scanner if you don't do something like this.
po.jpg
You do not have the required permissions to view the files attached to this post.
Shane MacLaughlin
Atlas Computers Ltd
www.atlascomputers.ie

SCC Point Cloud module
mschuetz
I have made 40-50 posts
I have made 40-50 posts
Posts: 40
Joined: Fri Nov 01, 2013 7:03 am
10
Full Name: Markus Schuetz
Company Details: potree.org
Company Position Title: developer
Country: Austria
Linkedin Profile: No
Been thanked: 5 times

Re: cyclone to Potree

Post by mschuetz »

PotreeConverter can be a bit slow and unstable for >10 billion points right now but here are some possible suggestions:
* You can consider using Entwine instead of PotreeConverter to build up the octree, since entwine is a bit better at creating an octree out of massive data sets. You can still use Potree to visualize the results:
** https://entwine.io/
** Some demos: https://potree.entwine.io/ (there is even one with 640 billion points)
* Check if the points inside the las file are in some way sorted by tile, or if they are spread all around. If they're spread randomly, then memory consumption will skyrocket and once you run out of memory, performance will drop to zero. It's advantageous to partition large point clouds into tiles of 1 to 50 million points, each, and then specify the folder containing the tiles as the input of the converter.
I was able to build an octree out of 18 billion points in around 16 hours, so 1 billion points / hour should be realistic. However, it also depends on whether the data has high depth complexity (buildings with interiors) or low depth complexity (aerial lidar scans). For the former, it's strongly recomended to partition it into smaller tiles since it improves memory consumption and reduces the amount of reads/writes that are necessary to process all the data.

In any case, I'm currently working on a complete rewrite of the converter which will be around 10x faster and produce 3 files instead of thousands to millions of files. It's still 1-3 months until it's ready, and it was mainly intented for the new WebGPU based rewrite of Potree but some sponsor is already helping out with some funding to bring it back to Potree 1.7 (next WebGL based release) as well. :) On that note, it'd be greatly appreciated if people/companies who've got good value out of Potree up until now could help out getting these future versions done by becoming sponsors at github! https://github.com/potree/potree
bne
I have made <0 posts
I have made <0 posts
Posts: 1
Joined: Mon May 18, 2020 8:32 pm
3
Full Name: Bogdan Negrea
Company Details: LE34
Company Position Title: LE34
Country: Denmark
Linkedin Profile: Yes

Re: cyclone to Potree

Post by bne »

Hi everyone

I am a collegue of MalteHC and I have also tried to use potree converter with the above mentioned file and I can come with a bit of extra information.

First I dont think that the problem is potree as we have tiled single files of up to 225 gb without any problems.

The present .las file is created as an Cyclone export to .e57 which is then converted to .las.
I have tried to convert each individual .e57 file to .las(with sizes between 2 gb and 15 gb) and then use potree to tile them individually.
While some of the .las files ran fine, for some of them less than half the points were written to the tile output. If I merge these files and then run potree on the merged file it becomes extremly slow and looks like a never ending process.

I have also tried to import the e57 files to Recap and then export them back to .e57 and i could see that only 1/5 of the points were exported which is somehow corelated with the
previous experiment where potree wrote to the output a small amount of points.

I believe there is something wrong with the files coming out of Cyclone as we have no problems processing files exported from Trimble Real Works or Autodesk Recap but we only have problems with the files exported from Cyclone.

If anyone has any ideea of what is going wrong in the process would be much appreciated :)
User avatar
smacl
Global Moderator
Global Moderator
Posts: 1409
Joined: Tue Jan 25, 2011 5:12 pm
13
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 627 times
Been thanked: 657 times
Contact:

Re: cyclone to Potree

Post by smacl »

bne wrote: Tue Jun 02, 2020 3:34 pmIf anyone has any idea of what is going wrong in the process would be much appreciated :)
Don't know what is going wrong, but if you want to send through one of the LAS files that is failing to me via wetransfer or similar I can run it through SCC and out to potree to see how it behaves. This would let you know if the problem with the LAS file produced by Cyclone or the potree converter application.
Shane MacLaughlin
Atlas Computers Ltd
www.atlascomputers.ie

SCC Point Cloud module
Post Reply

Return to “Leica Cyclone, Cyclone REGISTER 360 & Cyclone FIELD 360”