What Hardware Makes the Biggest Difference in Processing

To chat about anything else.
Bill-Wallace
V.I.P Member
V.I.P Member
Posts: 140
Joined: Fri Oct 30, 2009 12:14 am
Full Name: Bill Wallace
Company Details: Leica Geosystems
Company Position Title: HDS Sr. Software Product Manager

Re: What Hardware Makes the Biggest Difference in Processing

Post by Bill-Wallace » Fri Sep 09, 2016 8:13 pm

[Vendor commercial]
Hmmm, I wonder how much time it would take to send the data to JetStream instead of Recap, and then use CloudWorx inside AutoCAD. Would you be willing to:

a. Let us have a copy of the *imp data so we can benchmark it?
or
b. Use some demo licenses to try this workflow yourself?

I think writing to JetStream format is at least 10 faster than dumping to just about any other generic format. If you haven't seen the display/rendering characteristics of JetStream based data using CloudWorx for AutoCAD (or Revit or NavisWorks, or MicroStation) you should take a look. Most people are pretty impressed and the instantaneous nature of the loading and rendering really improves productivity.

JWFARO
I have made 40-50 posts
I have made 40-50 posts
Posts: 45
Joined: Thu Feb 06, 2014 11:58 am
Full Name: Julian Weisser
Company Details: FARO Europe
Company Position Title: 3D Software Developer
Country: Germany
Linkedin Profile: No

Re: What Hardware Makes the Biggest Difference in Processing

Post by JWFARO » Fri Sep 09, 2016 8:34 pm

All this computer power to read an ASCII file for 9 hours... :?

Toulinash
I have made 80-90 posts
I have made 80-90 posts
Posts: 89
Joined: Thu Feb 09, 2012 4:01 pm
Full Name: Cornelis Kolbeek
Company Details: 3d modeling
Company Position Title: Scan specialist
Country: Netherlands
Linkedin Profile: Yes
Location: NL

Re: What Hardware Makes the Biggest Difference in Processing

Post by Toulinash » Tue Sep 13, 2016 11:16 am

danielgadowski wrote:This reply should be permanently bolted to the top of the list under Hardware topic :)
Nah, all these components and configurations will be outdated by the end of the week :D

Rikore
V.I.P Member
V.I.P Member
Posts: 168
Joined: Sun Aug 16, 2015 6:48 pm
Full Name: Bruce Yager
Company Details: Rikore Geomatics
Company Position Title: Department Head
Country: USA
Linkedin Profile: No

Re: What Hardware Makes the Biggest Difference in Processing

Post by Rikore » Tue Sep 13, 2016 11:33 am

I find it interesting that we will spend a lot of money on scanners and the software, but not the hardware to process the scans. I know some some folks that buy off the shelf laptops then are let down that they can't perform very well. Spend the money for the one computer that really works. Now if the software would only use the high performance of those computers.

User avatar
TommyMaddox
V.I.P Member
V.I.P Member
Posts: 179
Joined: Mon Apr 25, 2016 7:15 pm
Full Name: Tommy R Maddox III
Company Details: BC Engineering and Design
Company Position Title: Engineer I
Country: USA
Linkedin Profile: Yes
Location: Columbia, South Carolina

Re: What Hardware Makes the Biggest Difference in Processing

Post by TommyMaddox » Tue Sep 13, 2016 3:23 pm

One of the biggest issues I face (I build our computers as well as doing field work and other engineering), is the widespread lack of support for advanced technologies like CUDA and large core counts or multiple physical processors, cluster computing, etc...

Pretty much the only software package that seems to really utilize those technologies (atleast in the 20-ish software packages that are in our workflows) is ANSYS. And they charge a fortune for the opportunity for you to use the expensive hardware you've purchased. HPC (High-Performance-Computing) licenses are sold on a per-core or per-card license in the case of GPU acceleration. Otherwise, they limit you to using 2 cores, which is asinine in an era where the 32 cores powering one of our machines can be had for all of $200.

User avatar
JoshK
V.I.P Member
V.I.P Member
Posts: 105
Joined: Tue Mar 31, 2015 12:44 pm
Full Name: Josh K
Company Details: Complete 3D Concepts
Company Position Title: Director
Country: Australia
Linkedin Profile: No
Location: Brisbane, Australia
Contact:

Re: What Hardware Makes the Biggest Difference in Processing

Post by JoshK » Thu Sep 15, 2016 3:31 am

Bill-Wallace wrote:[Vendor commercial]
Hmmm, I wonder how much time it would take to send the data to JetStream instead of Recap, and then use CloudWorx inside AutoCAD. Would you be willing to:

a. Let us have a copy of the *imp data so we can benchmark it?
or
b. Use some demo licenses to try this workflow yourself?

I think writing to JetStream format is at least 10 faster than dumping to just about any other generic format. If you haven't seen the display/rendering characteristics of JetStream based data using CloudWorx for AutoCAD (or Revit or NavisWorks, or MicroStation) you should take a look. Most people are pretty impressed and the instantaneous nature of the loading and rendering really improves productivity.
mate i would love to hand it over to see if there are better ways, but this particular job is under one of the strictest NDAs i have ever seen. and i have no idea why. the place is ancient and has nothing of particular interest in it as far as i can tell. i am guessing it is because of the large company it was for. but i have asked if i can do this, just don't hold your breath. ;)
Toulinash wrote:
danielgadowski wrote:This reply should be permanently bolted to the top of the list under Hardware topic :)
Nah, all these components and configurations will be outdated by the end of the week :D
Laughed at both of these, thanks gents. :lol:
Rikore wrote:I find it interesting that we will spend a lot of money on scanners and the software, but not the hardware to process the scans.
we do this because we are broke from spending the money on the scanners! :lol: seriously though i have always run reasonably high end machines for the drafting work i have been doing for the last decade. when i started scanning, they handled it ok. yes not great, but there are pretty low expectations when it comes to deliverable times of scan data at the moment, so i was able to get away with it until now.
Rikore wrote:I know some some folks that buy off the shelf laptops then are let down that they can't perform very well. Spend the money for the one computer that really works. Now if the software would only use the high performance of those computers.
You are dead right, this mentality has always dumbfounded me. especially as even when i was a low level draftie i spent nearly half my time waiting for my computer to work because my boss was to tight to buy decent computers, but was happy to rock up to work every day in his new high end cars that he got a new one each year...go figure...
TommyMaddox wrote:One of the biggest issues I face (I build our computers as well as doing field work and other engineering), is the widespread lack of support for advanced technologies like CUDA and large core counts or multiple physical processors, cluster computing, etc...
^^^THIS^^^ i would LOVE to know why this is. i was gob smacked when i started looking into all this and founf out that AutoCAD was still only single thread! i had known it for a few years, but thought surely by now it would have changed. i get that not a lot of people would be taking advantage of some of this stuff yet, but for programs like 3DS Max not to natively use CUDA?? it makes no sense!
TommyMaddox wrote:the 32 cores powering one of our machines can be had for all of $200.
:shock: say what now? :shock: where and how is that possible? i paid over $625 for my 6 core...

User avatar
JoshK
V.I.P Member
V.I.P Member
Posts: 105
Joined: Tue Mar 31, 2015 12:44 pm
Full Name: Josh K
Company Details: Complete 3D Concepts
Company Position Title: Director
Country: Australia
Linkedin Profile: No
Location: Brisbane, Australia
Contact:

Re: What Hardware Makes the Biggest Difference in Processing

Post by JoshK » Thu Sep 15, 2016 3:39 am

missed this one.
JWFARO wrote:All this computer power to read an ASCII file for 9 hours... :?
HEY! i can do more than just that now! :lol: yesterday i was rendering an image from 3DS Max, had a point cloud open in Recap, had 2015 and 2016 Inventor open both with huge point clouds open in them, was listening to music on youtube and had email, word and excel open as well as a few explorer windows for good measure. (I know that sounds wierd, but it was more a result of being able to do it than my actual work flow. :lol: ) I thought that was a pretty good indication of the power of the machine. :D

User avatar
JoshK
V.I.P Member
V.I.P Member
Posts: 105
Joined: Tue Mar 31, 2015 12:44 pm
Full Name: Josh K
Company Details: Complete 3D Concepts
Company Position Title: Director
Country: Australia
Linkedin Profile: No
Location: Brisbane, Australia
Contact:

Re: What Hardware Makes the Biggest Difference in Processing

Post by JoshK » Thu Sep 15, 2016 4:37 am

so an update for anyone interested.

the missing RAM issue was that a couple of the sticks weren't seated properly. :oops: i am blaming my mate that was helping me with the build for that as he put them in. :lol: I just unseated them and clicked them back in turned it on and they were there. so i have a full 128Gb of ram now. not had to process any more huge jobs since this one, and didn't think to monitor the system when doing the 40-60 scan jobs since, but that is looking like it will change next week hopefully, so will keep you posted.

i have been doing some renders out of 3DS Max the last couple of days, (hence the strange programs running as mentioned above and now being on here :lol: ) and they are the one thing i have found that can "kill" the machine. that said i am still writing this from said machine and i have been working fine on it the last few days catching up on all sorts of other things. It just has a moment every now and again where it stops working for about 30secs or so. it is interesting to note that while rendering alone, the CPU sits somewhere between 95 and 100% and ram at about 50-70%. i was a bit surprised by that because everyone says the graphics card does all the work, but apparently not. i would almost say that this machine is slower at processing the renders that i am doing, but i haven't done enough of them to know for sure, and i haven't timed them previously. But i know a graphics card(s) upgrade is coming as soon as i have the cash!

I have been doing a shite load of work on this thing and cleared a huge back log of stuff in the last few weeks! some interesting finds, transferring 50Gb between SSDs (saved and export to the wrong place :oops: ) 9mins 30sec. :shock: transferring stuff to USB 3.0 sticks and external drives is lightning fast. this mother board came with a PCIe USB mounted thingy and it is called USB 3.1, i tried it a couple of times, but couldn't see a noticeable difference between it and the USB 3 so ditched it for a PCIe mounted sound card. the sound card i had to get as the on board sound card wasn't working with my mic. that is a saga in itself, but suffice to say if anyone is having low volume mic issues on windows 7, just go and get a sound card! Mine cost about $50 from memory and the difference in sound is astounding. i had always heard they make little to no difference, but that is a LIE! :lol:

I got the sound card because i want the mic to work for some video tutorials i wanted to do. i do screen recordings with a video capture card (was in the PC when i brought it off my mate) and then chop it all up add sound etc in adobe premier elements then render it out. a 20min video takes about 30minutes or less to render (long enough that i can't be arsed to sit and watch it render, but not all that long, usually done within the time it takes to watch one TV show on netflix. :lol: ). a 10min video i can comfortably sit and watch the status bar (about 5mins maybe), a 5 minute video i can do in a few seconds (at least it feels that fast).

i can't begin to express the difference this machine has made to my workflows and the speed at which i can get work done across the board now. i have a high end gaming laptop for field work and it is pretty quick by all standards. now it is like using a rock by comparison. if anyone reading this because they are thinking about upgrading their machine, i say don't mess around, go the biggest and fastest machine you can afford to buy! If you can't afford all the high end bits you want now, get a good motherboard, a power supply that will power it with max components and the best CPU you can afford and upgrade it as you can. RAM, while there are advantages to getting matched sets, don't have to be, so you can upgrade a couple of sticks at a time. mine aren't matched as they weren't available in time for my build and i had to do something. Mine are all the same brand/type though. i am still way more than happy. CPU you can upgrade later, just get a motherboard with the higher end socket. you will never regret it for one minute.

JWFARO
I have made 40-50 posts
I have made 40-50 posts
Posts: 45
Joined: Thu Feb 06, 2014 11:58 am
Full Name: Julian Weisser
Company Details: FARO Europe
Company Position Title: 3D Software Developer
Country: Germany
Linkedin Profile: No

Re: What Hardware Makes the Biggest Difference in Processing

Post by JWFARO » Thu Sep 15, 2016 8:09 am

JoshK wrote:missed this one.
JWFARO wrote:All this computer power to read an ASCII file for 9 hours... :?
HEY! i can do more than just that now! :lol: yesterday i was rendering an image from 3DS Max, had a point cloud open in Recap, had 2015 and 2016 Inventor open both with huge point clouds open in them, was listening to music on youtube and had email, word and excel open as well as a few explorer windows for good measure. (I know that sounds wierd, but it was more a result of being able to do it than my actual work flow. :lol: ) I thought that was a pretty good indication of the power of the machine. :D
Sure you can. Because it probably uses only one of your cores and is throttled by the disk speed or even worse by a slow parser spending many cycles to read one coordinate at a time. That is why I'm wondering if this there is no smaller (binary) file format to get the points from A to B? My guess would be e57?! You already mentioned the speed up when going from hard disks to SSDs. But you should experience an even bigger speed up when going from ASCII file formats to binary file formats.

User avatar
JoshK
V.I.P Member
V.I.P Member
Posts: 105
Joined: Tue Mar 31, 2015 12:44 pm
Full Name: Josh K
Company Details: Complete 3D Concepts
Company Position Title: Director
Country: Australia
Linkedin Profile: No
Location: Brisbane, Australia
Contact:

Re: What Hardware Makes the Biggest Difference in Processing

Post by JoshK » Thu Sep 15, 2016 8:22 am

JWFARO wrote:Sure you can. Because it probably uses only one of your cores and is throttled by the disk speed or even worse by a slow parser spending many cycles to read one coordinate at a time. That is why I'm wondering if this there is no smaller (binary) file format to get the points from A to B? My guess would be e57?! You already mentioned the speed up when going from hard disks to SSDs. But you should experience an even bigger speed up when going from ASCII file formats to binary file formats.
you are correct in that the file format makes a big difference to import speeds. i have been exporting to PTS to go to Recap for the last couple of projects i have done and they are MUCH faster to bring in. (admittedly they have been much smaller jobs, but still faster.) i haven't timed it, but we are talking maybe half the time. i have been experimenting with other formats too, but until i have the time to actually sit and watch or think to set a timer or something i won't be able to say which is best. have you experimented with this much at all? file types i think will be the next thing i spend a bit of time to get to know. i was thinking today of doing some testing on a smallish data set in every format and doing a post about it. but that will take a while to get together.

Post Reply

Return to “General Chat”

Who is online

Users browsing this forum: No registered users and 1 guest