AMD THREADRIPPER PRO 3995WX

To chat about anything else.
dhirota
V.I.P Member
V.I.P Member
Posts: 704
Joined: Sun Nov 01, 2009 11:18 pm
Full Name: Dennis Hirota
Company Details: Sam O. Hirota Inc.
Company Position Title: President
Country: USA
Linkedin Profile: Yes
Location: Hawaii, USA
Has thanked: 26 times
Been thanked: 127 times

Re: AMD THREADRIPPER PRO 3995WX

Post by dhirota »

Still working on PC88, which is the workstation consisting of the AMD ThreadRipper Pro 3995WX with the ASUS Pro WS WRX80E-SAGE-SE MB. I decided to get the UBUNTU version installed first since there are some problems running both W10 Pro and UBUNTU on this original BIOS on the ASUS MB and there has been only one person at ASUS level one support who knew more than anyone at level two support in the last week.

Below is a view of the case showing the following from the top down:

20210314_112256.jpg

1. DeepCool Castle 360EX Cooler (3X 120mm fans)
2. ASUS Pro WS WRX80E-SAGE-SE MB (US$90 ASUS Hyper M.2 X16 PCIe 4.0 X16 NVMe card, included in box)
3. 4X128 GB ECC RDIMM=512GB RAM (will fit anywhere in the 8 RAM slots. I have 1 on top and 3 on bottom)
4. AMD 3995WX CPU (under cooler)
5. LSI AVAGO 9361-8i SAS RAID Controller, with 8X Seagate 12TB = 72TB (all not included in pricing table)
6. ASUS Hyper M.2 X16 PCIe 4.0 X16 NVMe card with only Samsung 980 Pro M.2 shown below
7. NVIDIA GTX 1050Ti (decided to use this since it fits works for now and I do not need to steal a RTX2080Ti from another workstation)
8. EVGA 1300W G2 power supply
9. Not shown, Dual Intel 10GbE ports on ASUS MB.

20210314_111644.jpg

The table below shows the cost of building the 3 workstations that we use to process our NavVis, other scanning/imaging projects, and other experimental visualizations.

NEWEGG_COST_3995WX -210314.jpg

Costs do not include Hawaii tax and freight (Free from Amazon and $34 two day from NEWEGG).

The cost difference between the two 3990X and 3995WX workstations is in the CPUs, RAM, and MBs. It is hard to believe that 256GB ECC RDIMM would be that much of a cost difference, but if you do not have it and you crash or have to break up you project in to smaller pieces, what is that cost? We can expand the RAM to 1.5TB with enough $$ and 4 empty RAM slots.
You do not have the required permissions to view the files attached to this post.
Dennis Hirota, PhD, PE, LPLS
[email protected]
dhirota
V.I.P Member
V.I.P Member
Posts: 704
Joined: Sun Nov 01, 2009 11:18 pm
Full Name: Dennis Hirota
Company Details: Sam O. Hirota Inc.
Company Position Title: President
Country: USA
Linkedin Profile: Yes
Location: Hawaii, USA
Has thanked: 26 times
Been thanked: 127 times

Re: AMD THREADRIPPER PRO 3995WX

Post by dhirota »

Finally got most of the pieces together for my AMD 3995WX Threadripper Pro + ASUS Pro WS WRX80E SAGE MB + Raid 6 8X12TB=72 TB + W10 Pro + Ubuntu 18.04 + 512GB ECC DDR4 RAM with everything working. It is not every day that you see 128threads working at 100% using 300+GB of RAM.

Jed: I am still going to wait to see if anything better comes out on the GPU market without taking 3 PCIe slots, especially since I only have 4 empty PCIe-4 slots left. I could take the last (7) PCIe slot since the case has two empty positions below it.

Unfortunately, ASUS support + knowledge base needs significant help since this MB is nothing like the previous ones, especially if you have any legacy devices. You will have problems using the ASUS Hydrid NVMe card without reading the manuals in detail which do not relate with each other. It was trial and error for a week. I am using the NVMe card to boot W10 Pro off a Samsung 980 Pro 1TB and boot UBUNTU 18.04 off of a Samsung 980 Pro 2TB, which is easier to move NVMe sticks around since the MB NVMe have 3 locations, but only 2 can be active and are difficult to move around.

LESSON LEARNED: If you are going to use these type of multi-core CPUs with MB having many PCI-e-4 slots, it is best to check the MB manual for the number of power plugs to be used and counting power plugs to be used on the number of GPUs before deciding to purchase the MB and power supply (which both are expensive or difficult to find).

Here is the ASUS manual showing a minimum of 5 power plugs needed on the MB + what ever you will need for your GPUs, and the other image showing how many power plugs are on the EVGA 1300Watt PSU. With this configuration, you will only have 3 cables left for VGA/GPU.

3-25-2021 2-14-06-POWER.jpg
3-25-2021 2-35-59-EVGA.jpg
You do not have the required permissions to view the files attached to this post.
Dennis Hirota, PhD, PE, LPLS
[email protected]
User avatar
smacl
Global Moderator
Global Moderator
Posts: 589
Joined: Tue Jan 25, 2011 5:12 pm
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 184 times
Been thanked: 179 times
Contact:

Re: AMD THREADRIPPER PRO 3995WX

Post by smacl »

dhirota wrote: Fri Mar 26, 2021 1:32 am Finally got most of the pieces together for my AMD 3995WX Threadripper Pro + ASUS Pro WS WRX80E SAGE MB + Raid 6 8X12TB=72 TB + W10 Pro + Ubuntu 18.04 + 512GB ECC DDR4 RAM with everything working. It is not every day that you see 128threads working at 100% using 300+GB of RAM.
Glad to hear you're up and running, Dennis. I remember being gleeful in the early 80s when I got the 16k RAM pack for my ZX81 with 3.25mhz CPU, being delighted to have all that memory to wallow in. You have 19,660,800 times the amount of memory and about 62,828,257 the integer OPS (very many more FLOPS) :D

PCs and parts are getting scarce enough around this part of the world. Needed to update an old testing rig yesterday and found it is now cheaper to buy than to build, LENOVO P340 i9-10900K with 64gb cost me €1,450 on amazon.de. Self build would cost €1,650 using available parts delivered to Ireland in the same time frame. I'm lucky enough to have a spare GPU, these are silly prices right now. Planning to replace my main rig at some point soon but reckon I'll hold off for a few months more.
dhirota
V.I.P Member
V.I.P Member
Posts: 704
Joined: Sun Nov 01, 2009 11:18 pm
Full Name: Dennis Hirota
Company Details: Sam O. Hirota Inc.
Company Position Title: President
Country: USA
Linkedin Profile: Yes
Location: Hawaii, USA
Has thanked: 26 times
Been thanked: 127 times

Re: AMD THREADRIPPER PRO 3995WX

Post by dhirota »

dhirota wrote: Wed Mar 03, 2021 6:58 pm Jed

We have 5 Nvidia RTX2080Ti GPUs in the office that take up 3 slots. I will steal the one on our 18Core i9-9980XE until I figure out how to best use the 7 PCIe-Gen-4 slots with 128 lanes on the 3995WX and which GPUs make the most sense after the Bit-coin guys go broke.
I decided that running a GTX 1050Ti is OK some of the time to visualize 100Mpt, but if you have to visualize and render 500Mpts to 1.3Bpts you need something better. I did some adjustment on the 72TB SAS disk arrangement to get better cooling of the disks and gave me some extra room for the two blank spaces below the motherboard for the RTX2080Ti, which takes up three slots and 2 VGA (8pin cables, leaving me with only one VGA cable left)
20210331_123321.jpg

Still waiting to see what/when NVIDIA will have beyond the RTX A6000.
You do not have the required permissions to view the files attached to this post.
Dennis Hirota, PhD, PE, LPLS
[email protected]
dhirota
V.I.P Member
V.I.P Member
Posts: 704
Joined: Sun Nov 01, 2009 11:18 pm
Full Name: Dennis Hirota
Company Details: Sam O. Hirota Inc.
Company Position Title: President
Country: USA
Linkedin Profile: Yes
Location: Hawaii, USA
Has thanked: 26 times
Been thanked: 127 times

Re: AMD THREADRIPPER PRO 3995WX

Post by dhirota »

This is probably the last add-on to this system, unless I can get a better NVIDIA GPU at MSRP.

20210402_105749.jpg

I call this my GOLD 22 KT XBOX wireless controller. It took a little while to figure out how to get it running on some of the visualization software. It was straight forward once you get a game person in the room to help figure it out. It makes a significant difference in real time movement of 1.35Bpts on a 82 inch 4K screen versus a mouse to control everything.

ONLY "BENCHMARK"
1. NavVis VLX scan outdoors
2. Area =80,600 sq.meters
3. post-processed on 5mm grid, 1.35 billion points
4. importing 20GB E-57 file = 3 minutes
5. pan/zoom in real time for entire point cloud
You do not have the required permissions to view the files attached to this post.
Dennis Hirota, PhD, PE, LPLS
[email protected]
User avatar
gsisman
V.I.P Member
V.I.P Member
Posts: 492
Joined: Fri Oct 07, 2016 1:51 pm
Full Name: Steve Long
Company Details: Montgomery County DOT _ MD
Company Position Title: Land Survey Supervisor
Country: United States
Skype Name: gsisman1
Linkedin Profile: Yes
Has thanked: 224 times
Been thanked: 49 times

Re: AMD THREADRIPPER PRO 3995WX

Post by gsisman »

dhirota wrote: Fri Apr 02, 2021 11:01 pm This is probably the last add-on to this system, unless I can get a better NVIDIA GPU at MSRP.


20210402_105749.jpg


I call this my GOLD 22 KT XBOX wireless controller. It took a little while to figure out how to get it running on some of the visualization software. It was straight forward once you get a game person in the room to help figure it out. It makes a significant difference in real time movement of 1.35Bpts on a 82 inch 4K screen versus a mouse to control everything.

ONLY "BENCHMARK"
1. NavVis VLX scan outdoors
2. Area =80,600 sq.meters
3. post-processed on 5mm grid, 1.35 billion points
4. importing 20GB E-57 file = 3 minutes
5. pan/zoom in real time for entire point cloud
Nice Dennis. I'd love to see a little video clip of the navigation of that puppy on that 82" screen 1.35bpts. That is a LOT of points for the VLX, My 3500' linear Street scape only had 531 million
dhirota
V.I.P Member
V.I.P Member
Posts: 704
Joined: Sun Nov 01, 2009 11:18 pm
Full Name: Dennis Hirota
Company Details: Sam O. Hirota Inc.
Company Position Title: President
Country: USA
Linkedin Profile: Yes
Location: Hawaii, USA
Has thanked: 26 times
Been thanked: 127 times

Re: AMD THREADRIPPER PRO 3995WX

Post by dhirota »

gsisman wrote: Mon Apr 05, 2021 4:03 am
Nice Dennis. I'd love to see a little video clip of the navigation of that puppy on that 82" screen 1.35bpts. That is a LOT of points for the VLX, My 3500' linear Street scape only had 531 million
Steve

I have not at least posted some screen shots because of the client requirements of not showing images to the public. We have other public VLX scans that we scanned last year that we need to re-process to improve color balance and visualization soon.
Dennis Hirota, PhD, PE, LPLS
[email protected]
dhirota
V.I.P Member
V.I.P Member
Posts: 704
Joined: Sun Nov 01, 2009 11:18 pm
Full Name: Dennis Hirota
Company Details: Sam O. Hirota Inc.
Company Position Title: President
Country: USA
Linkedin Profile: Yes
Location: Hawaii, USA
Has thanked: 26 times
Been thanked: 127 times

Re: AMD THREADRIPPER PRO 3995WX

Post by dhirota »

dhirota wrote: Mon Apr 05, 2021 8:01 pm
gsisman wrote: Mon Apr 05, 2021 4:03 am
Nice Dennis. I'd love to see a little video clip of the navigation of that puppy on that 82" screen 1.35bpts. That is a LOT of points for the VLX, My 3500' linear Street scape only had 531 million
Steve

I have not at least posted some screen shots because of the client requirements of not showing images to the public. We have other public VLX scans that we scanned last year that we need to re-process to improve color balance and visualization soon.
Steve

Here is a partial resort scan of 700 million points. I am not organized yet to generate great color balance and improved visualization, the "BENCHMARK" below is interesting to see a NVIDIA RTX 2080Ti working.

4-6-2021 12-39-56 PM.jpg

"BENCHMARK"
NavVis VLX scan, post-processed on 5mm grid, 700 million points
E-57 file = 10.2 GB
50% of 82 inch Samsung 4K screen displaying 30million pixels
NVIDIA RTX 2080Ti running at 80% to 97% with XBOX wireless controller

4-6-2021 1-33-36 PM.jpg

full screen of 82 inch Samsung 4K screen displaying 30million pixels
You do not have the required permissions to view the files attached to this post.
Dennis Hirota, PhD, PE, LPLS
[email protected]
dhirota
V.I.P Member
V.I.P Member
Posts: 704
Joined: Sun Nov 01, 2009 11:18 pm
Full Name: Dennis Hirota
Company Details: Sam O. Hirota Inc.
Company Position Title: President
Country: USA
Linkedin Profile: Yes
Location: Hawaii, USA
Has thanked: 26 times
Been thanked: 127 times

Re: AMD THREADRIPPER PRO 3995WX

Post by dhirota »

After searching the entire weekend trying to find another NVIDIA GPU, I just purchased a NVIDIA A6000 with 48GB ECC VRAM for MSRP US$4,650 and it will be delivered next week.
Dennis Hirota, PhD, PE, LPLS
[email protected]
User avatar
TommyMaddox
V.I.P Member
V.I.P Member
Posts: 364
Joined: Mon Apr 25, 2016 7:15 pm
Full Name: Tommy R Maddox III
Company Details: ONSITE3D
Company Position Title: Director of US Technical Operations
Country: USA
Linkedin Profile: Yes
Location: The Woodlands, Texas
Has thanked: 27 times
Been thanked: 31 times

Re: AMD THREADRIPPER PRO 3995WX

Post by TommyMaddox »

Eager to see how RiScan Pro performs on the 48GB VRAM A6000 GPU. I know on our end the 1080Ti 11GB units we have perform better than the newer RTX series cards that have lower vram amounts just due to how RSP handles point display. Heavily considering getting that once prices become more justifiable.
Post Reply

Return to “General Chat”