Thanks for all the replies folks. I have been ultra-busy and not been able to get back since posting this for long enough to reply. (Hopefully my phone stays quite while I type this out.)
So to give some context and answer a few things raised here;
I have a job I am processing (and was
struggling with a bit) and it has taken WAY longer than expected and my computer was taking days to process small chunks of it. I wanted to do something to speed up the unify/export/import times and I didn't want to spend a fortune on it if I could get away with it, or spend the money on the wrong bits. Hence the original post. Knowing that SSDs make a big difference, and with no replies to the thread at that stage, I went and got 2 x 2Tb Samsung EVO 850 SSDs. This cut the processing time out of cyclone from about 60hrs per section to 22hrs. The time to get the files into Recap stayed about the same though (roughly 48hrs), so I was pretty happy with that.
However during all the copy and pasting of the data bases to get the new drives in, one of the ones I was working on became corrupted. So I spent days trying to work out how to fix this (or if I could fix it) due to the computer still taking 12+ hrs to run processes like compress and optimize. All the while my deliverable deadline moving further into the past. Had I known the processing was going to take so long I would have just redone it. but you know what they say about hindsight. When I finally realized the database was not going to be able to be recovered and that the backup was also obviously corrupt I thought "well I still have 10+ days to process this job" and that is after I re-import the scans and get back to the stage I was at! God knows how long that was going to take...
So after doing a lot of testing of imports/export/unifying and so forth while monitoring the system processes, reading more than I care to think about and watching a heap of videos, I worked out the best hardware to upgrade and went to the computer store. (Sorry, a few of you had replied by then, but this thread was the last thing on my mind by this stage.) So for anyone finding this thread in the future
, or those interested now, what I worked out was this;Disclaimer:
Before I get into this I am not a computer person by any stretch of the imagination. I had a better than average comprehension of how things worked before all this started, and have learned a LOT in the last week or so, but please correct me on anything I have wrong here. And please don't laugh too hard if I have it totally wrong. This is just my understanding of what I have read, watched and talked to people about since starting this thread.
The read/write speed of your hard drives, be it solid state drives (SSDs) or hard disk drives (HDDs), is very critical when doing anything on a computer due to the way the data is processed, hence SSDs make a huge difference. The latest SSD drives can take advantage of 6Ghz Sata 3 connections and work up to 5 times faster than existing Sata 3 connections. I can't begin to express how big of a difference this alone makes to EVERYTHING
you do on a computer. The speed is mind blowing! However you need a motherboard that has these connections to take advantage of this.
The CPU is doing that processing, but cyclone only uses 1 core, so large Ghz numbers help more than number of cores. However the CPU architecture makes an even bigger difference, so smaller Ghz more current processors can do more than larger Ghz older ones. Keep that in mind if thinking about purchasing an older computer or computer parts.
If you are using Autodesk "flagship" products in your workflow most of them use multiple cores so the more cores you can get the better. But check this for your specific product. For example, AutoCAD is still (2017) a single thread program meaning it only uses one core. So again, the bigger the Ghz numbers the better. That said all multiple cores do is use all the cores to do the same amount of work. So if you are doing something that uses 100% CPU on a dual core processor it is using 50% per core. If you are doing a 100% CPU usage tasks on a 4 core processor, than you are only using 25% of each core. If I understand things correctly, the computer doesn’t slow down to much until you are using 100% of one or more cores. So it basically allows you to do more concurrent tasks as opposed to doing individual tasks faster.
When processing point clouds in cyclone, RAM, RAM, RAM and more RAM! While the quantity makes a big difference, the clock speed (Ghz number) makes a big difference too. I had 32Gb of 2400Ghz ram in the old setup and could max that without trying too hard. So the more ram you can throw at it the better. But the higher the clock speed the better also. So if you are limited by your motherboard to the amount of ram you can have, get the highest clock speed you can. I have got 128Gb of 2666Ghz RAM plugged in but something is amiss and it is only seeing 96Gb. I have to look into this more, but it is common apparently. Something to do with needing to up the voltage to the RAM, but could just as likely be a faulty ram set or motherboard. But that is something for a later date. It is worth noting that when processing a large job in cyclone the ram usage slowly builds as the job progresses. So when the ram is full, the system needs to dump bits to add new bits and this is where things slow down. This is also where the RAM clock speed comes in. when I was exporting a 34 scan section of this job I am working on the RAM usage peaked at 75-76Gb. Bigger jobs will need more obviously.
Interestingly Autodesk Recap uses a single core of the CPU as opposed to ram. It only used about 15-20Gb of ram while importing the above mentioned export, but frequently used 100% processor on a single core.
When it comes to graphics cards they don't matter as much as you would think. When they talk "render times" for a graphics card it is better to think of it as frames per second (FPS) speed than an actual time to render a JPG scene out of a program. The reason for this is that, that is how the graphics cards are "rated" due to the fact most of them are used for gaming and not displaying 3D models. In games the speed which a card can render a scene changes how smooth the game is and how high the definition of the game is. With 3D models though the bigger the models the higher the frame rate you need. Most high end 3D modelling programs recommend using Quadro cards. While there is a host of reasons for this, which I won’t go into, in my opinion they are massively overrated. In my 10+years of 3D modelling I have only used 2 computers that used the Quadro graphics cards that everyone says you "have to have". They both sucked as far as graphics were concerned when compared to my computers using gaming cards. This is a really well debated topic and yes sometimes you will get a driver issue when working with some 3D modelling programs, but not enough issues that it is worth the ridiculous asking prices for the Quadro cards. Most issues are pretty well documented and easy to find solutions for online. Also if you update your driver and your 3D modelling program starts playing up it is a fair guess what caused it and you can easily rollback drivers for popular gaming graphics cards. So don't waste the money on a Quadro card.
If you are doing actual renders of scenes (outputting JPGs/movies from a program) then Graphics Processing Unit (GPU or graphics card) memory is king! This is the bit where people say "I have 2Gb graphics card". The 2Gb refers to the memory (RAM) on the graphics card itself. All programs use the GPU to do renders and now some programs will let you split or change that to CPU processing if desired. So if you are doing large, detailed or complex renders, than GPU memory is what you need. The higher the better! If you are doing this in a serious way, than multiple graphics cards is where it’s at. However you need 2 things to take advantage of multiple graphics cards, 1) a motherboard that supports them with multiple PCIe slots and support for crossfire or SLi. 2) a program that takes advantage of them. So check this out in detail if this is a path you want to go down. From what I have seen/heard some programs say they support it, but when put to the test 1 large card can outperform multiple linked cards. This is something I need to look into more, but the take away for me is that it isn’t THAT important when building a system. A half decent gaming card will do a good enough job for most people.
3DS Max uses crossfire or SLi graphics with CUDA (I still don't fully understand this bit, so please don't laugh too hard if I have that wrong somehow
) but only if you have the iRay render engine (additional $350ish plugin). I thought from what I read that it may work with mental ray rendering, but have to look into it more.
Don’t skimp on the power supply. The cheaper ones can be a bit dodgy apparently and they can and will fry your newly built system the first time you flick the on switch. Also always buy something that is a higher capacity than what you need. If you are working fine for 12months and a graphics card that can’t be replaced dies or you need to add more ram, you don’t want to have to fork out for a new power supply as well because you were a tight arse at the start. They are one of the cheaper parts of a build, so do it right the first time. The rule of thumb should be buy for what your motherboard will support not the components you are buying.So all that said, this is what I have ended up with;
Motherboard = ASRock X99 Extreme6/3.1
CPU = I7 6800K 3.4Ghz 6 core
CPU Cooling = Cooler Master Nepton 140XL Liquid Cooler (I only "need" this as I intend to over clock it at some point. Apparently it was not needed as the case I have has enough cooling to keep everything under control for a “stock” build.)
Ram = 128Gb Corsair DDR4 2666mhz (I couldn’t afford to get the 3000Ghz RAM, but it would be interesting to see how much faster it would be. One day maybe…)
Power Supply = Corsair HX1200-i Platinum 1200W 80+ ATX (this is overkill for what I have now, but will allow me to get multiple graphics cards if I want/need to go down that path.)
Graphics = (Existing) Asus Nvidia GTX 680
SSDs = 2 x 2Tb Samsung EVO 850 (I can not begin to describe how much of a difference these things make! Get them and get them now!
HDD = (Existing) 2 x 3Tb Toshiba (storage only)
Total cost for the new bits approximately = $4100So what does this translate to?
Export 1 360 degree scan @ 3.2mm with colour = 4 minutes!
Export 34 scan registration to PTX (scans @ 3.2mm with colour) = 4hrs
Import PTX into recap = 9hrs
So I can comfortably do in 13hrs what was previously taking me nearly a week.
As far as my personal processing time is concerned, once the scans are into cyclone I can get a 30-40 scan registration done in about 1-1.5hrs with cloud to cloud and depending on how many targets I shot on site etc.. Obviously if I have shot all targets on site and let Cyclone do the cloud to cloud bit it is minutes. I doubt that will ever get too much better, but it sure as hell is smoother since the upgrade.
Again, no matter how much more training I do, I doubt those times will get much better. I did spend a day with the Leica Cyclone Guru (Australian) after doing a half a dozen jobs and it was one of the best days I have ever spent with training in a program. My processing times did drop a LOT and I would highly recommend it. He REALLY knows his stuff and is an excellent help. (Has been this past week or so too.) He obviously takes pride in his work and keeps a good eye on how Cyclone progresses. Can’t speak highly enough about the guy! That said, it doesn't matter what program you are using, spend some time/money on some proper training courses. They are worth every penny and every second of your time!
While I know and get the point about having a well trained user of the software (it’s not just cyclone that applies to), in this case it was purely finding out how to drop the time the computer takes to “crunch the numbers”.
By the same token, this was not intended as a “will this computer do” thread. (I HATE those threads too and stopped reading/replying to them years ago.) I was trying to be a tight arse and find out what individual components would give me the best bang for buck. Now to answer my own question
In case anyone finds this in the future, it will depend greatly what motherboard you have as to what will work best for your system. You need to know what your motherboard will support before you can say what will be the most effective upgrade. It will also depend on what process you are trying to speed up and what the program uses to do that process.
So assuming you only have standard SATA 3 connections and HDD. SSDs can be a great start, but the big ones still aren’t cheap. My 2Tb ones cost $830 each. For most people not processing large point clouds, a smaller one will do the job. Just buy a size to suit your needs. If your motherboard has 6Ghz SATA connections RUN, don’t walk, to get yourself some SSDs that use them! You will not regret it! It is probably worth stipulating that they should be made your C (main) drive for overall performance upgrades. If you just put it in as a secondary drive, you will only see mild increases in performance when reading/writing to/from that drive. It won’t improve anything else.
If you are already running SSDs than RAM is the next cheapest upgrade. Again, it is worth making sure your RAM is the key component your program uses before you go and spend the money on this, but it never hurts to have more RAM. Also make sure your operating system can support it. windows 7 “only” supports 190 something Gb from memory. If you are going down this path, buy the highest clock speed and as much of it as you can afford. Clock speed apparently makes a bigger difference than quantity if it comes to it.
While it is possible to upgrade your CPU, it is only possible to do so within the same socket type and manufacturer without upgrading your motherboard. This means unless you are already using a high end chip (1151 or 2011 socket) it is unlikely you will get much of a performance upgrade from this. And the CPU is the single most expensive bit in the computer. So be careful before spending the money. To find out what motherboard you have you have to run your command prompt in administrator mode and type in wmic baseboard get product,Manufacturer,version,serialnumber The biggest thing I have taken away from all this is
If you are going to be processing large (100+ scan) jobs, there is no substitute for a fast computer. (or as the car guys say, no replacement for displacement!) Unfortunately that doesn’t come cheap. Like anything you want to do professionally, spend the money for the right tools to do the job. You won’t regret it.