I need a serious workhorse computer for complex simulations
 

I am running simulations that involve around 6-10 million calculations. My files (10 of them) are each over 500 MBs large (all data and formulas), if that's any indication of the size of the project i'm dealing with. I am currently bogging down a Dell PC with 2.2 Core Duo processor and 2 gigs of RAM. It literally won't run the program in less than one hour.

I have about $6000 in my lab budget for a computer to run this simulation, but i want to make sure i get the most bang for the buck. I priced the following Mac Pro setup at around $6000 (education price).

Mac Pro
Both Bluetooth 2.0+EDR and AirPort Extreme
One 16x SuperDrive
Apple Keyboard and Mighty Mouse - U.S. English
Accessory kit
Two 3.0GHz Quad-Core Intel Xeon
Mac OS X - U.S. English
8GB (4 x 2GB)
750GB 7200-rpm Serial ATA 3Gb/s
ATI Radeon X1900 XT 512MB (2 x dual-link DVI)


Can i beat that system elsewhere for the same price? Please keep in mind that i don't want to use it for daily work... just these, and future, simulations.

TIA

Re: I need a serious workhorse computer for complex simulations
 

Possibly, but a MacPro 8 core is indeed a beast [img]/images/graemlins/smile.gif[/img]

Your simulations are CPU limited, right?

EDIT: And your simulations can run multi-process/multi-threaded, right?

Re: I need a serious workhorse computer for complex simulations
 

is there much disk IO once the projects are loaded?

Re: I need a serious workhorse computer for complex simulations
 

We just bought a new Dell Precision 690 in my lab for CFD simulations. It's essentially the same as that, except we also got a dual 750GB Raid 1 HD array and a 24" LCD monitor for about $6000 (education price).

Re: I need a serious workhorse computer for complex simulations
 

Try to get 10K RPM drives if you can.

Re: I need a serious workhorse computer for complex simulations
 

Do 10K RPM 750 GB hird drives even exist?

Re: I need a serious workhorse computer for complex simulations
 

[ QUOTE ]
Do 10K RPM 750 GB hird drives even exist?

[/ QUOTE ]

I honestly have no idea. Probably, but not as SATA, only SCSI or SAS.

Re: I need a serious workhorse computer for complex simulations
 

It's something to look into. If you can get a hard drive big enough for your needs at 10k RPM and if that's faster/more cost-effective than a Raid 1 array at 7200 RPM. Of course, if your problem is CPU limited that won't matter at all anyway.

Re: I need a serious workhorse computer for complex simulations
 

[ QUOTE ]
Possibly, but a MacPro 8 core is indeed a beast [img]/images/graemlins/smile.gif[/img]

Your simulations are CPU limited, right?

EDIT: And your simulations can run multi-process/multi-threaded, right?

[/ QUOTE ]
Yes, they are CPU limited and can run multi-threaded.

Re: I need a serious workhorse computer for complex simulations
 

[ QUOTE ]
is there much disk IO once the projects are loaded?

[/ QUOTE ]
I'm sorry, i honestly don't understand this question. I'm not familiar with the term "disk IO". I'm not an expert in this area, which is why i'm posting here, so please forgive my ignorance. [img]/images/graemlins/smile.gif[/img]

Re: I need a serious workhorse computer for complex simulations
 

After your data is loaded from the files on the HD, is there a lot of reading and writing to the HD during the simulation?

Re: I need a serious workhorse computer for complex simulations
 

Ok, i get what you're asking. It doesn't seem to be the case based on the lack of noise in the hard drive during the simulation.

Re: I need a serious workhorse computer for complex simulations
 

Not to be offensive, but you don't seem to be able to determine the hardware you need (obv why you're asking). Let me see if I can figure it out.

What exactly does the simulation do? What operating system are you running? Do you have access to the source code? Do you know the parallelization (eg. MPI, OpenMP)?

Re: I need a serious workhorse computer for complex simulations
 

You're absolutely right, in that i don't know what i need. The more i think about it, the more i realize that i'm lost and need more guidance before purchasing the right computer.

I'll try to answer your questions.

The simulation is mimicking 200,000 typical statistical "experiments" (well, each permutation is). To be specific, i believe that the statistic P-replication (P-rep) underestimates the true probability of replication. Thus, i've created 200,000 "experiments" using various population parameters (permutations of effect sizes) to determine what the actual probability of replication is, and then compare the mean/median of those to what P-rep predicts. The experiments are basically t-tests with an n = 10, 20, 40, and 100. They are all running simultaneously in the program. The program is written in C++. I didn't write (my programmer did), but i do have access to the source code if necessary.

Sorry, i don't know what "parallelization (eg. MPI, OpenMP)" means.

Thanks for your help in this. I do appreciate it.

Re: I need a serious workhorse computer for complex simulations
 

wonder if it's worth it to overclock with a badass cooling system?

Re: I need a serious workhorse computer for complex simulations
 

[ QUOTE ]
You're absolutely right, in that i don't know what i need. The more i think about it, the more i realize that i'm lost and need more guidance before purchasing the right computer.

I'll try to answer your questions.

The simulation is mimicking 200,000 typical statistical "experiments" (well, each permutation is). To be specific, i believe that the statistic P-replication (P-rep) underestimates the true probability of replication. Thus, i've created 200,000 "experiments" using various population parameters (permutations of effect sizes) to determine what the actual probability of replication is, and then compare the mean/median of those to what P-rep predicts. The experiments are basically t-tests with an n = 10, 20, 40, and 100. They are all running simultaneously in the program. The program is written in C++. I didn't write (my programmer did), but i do have access to the source code if necessary.

Sorry, i don't know what "parallelization (eg. MPI, OpenMP)" means.

Thanks for your help in this. I do appreciate it.

[/ QUOTE ]
Okay, first let me see if I understand this. You're running a monte carlo simulation with new inputs for each "experiment"?

It shouldn't be likely that disk IO is a limitation, unless each experiment depends on the results of the previous experiment and the simulation has to write then read the result to disk each time (if it does, fire your programmer).

That means disk IO isn't likely to be a limiting factor and you'd best be served by spending your money on the processor. The questions you can't/didn't answer aren't likely to change that. The computer you posted would likely be good for what you are doing, but I don't know if you could get something comparable for cheaper (the one I described in my first post would likely be several hundred cheaper).

Re: I need a serious workhorse computer for complex simulations
 

I will add this. Given the computational parameters in your original post, it seems to me that a dual quad-core system would be overkill. You could probably get by with something smaller for much less.

That's just semi-informed personal opinion though. I can't provide any evidence to back that up.

Re: I need a serious workhorse computer for complex simulations
 

If you can talk to the programmer why not just ask him what would be best?

Re: I need a serious workhorse computer for complex simulations
 

[ QUOTE ]
[ QUOTE ]
You're absolutely right, in that i don't know what i need. The more i think about it, the more i realize that i'm lost and need more guidance before purchasing the right computer.

I'll try to answer your questions.

The simulation is mimicking 200,000 typical statistical "experiments" (well, each permutation is). To be specific, i believe that the statistic P-replication (P-rep) underestimates the true probability of replication. Thus, i've created 200,000 "experiments" using various population parameters (permutations of effect sizes) to determine what the actual probability of replication is, and then compare the mean/median of those to what P-rep predicts. The experiments are basically t-tests with an n = 10, 20, 40, and 100. They are all running simultaneously in the program. The program is written in C++. I didn't write (my programmer did), but i do have access to the source code if necessary.

Sorry, i don't know what "parallelization (eg. MPI, OpenMP)" means.

Thanks for your help in this. I do appreciate it.

[/ QUOTE ]
Okay, first let me see if I understand this. You're running a monte carlo simulation with new inputs for each "experiment"?

It shouldn't be likely that disk IO is a limitation, unless each experiment depends on the results of the previous experiment and the simulation has to write then read the result to disk each time (if it does, fire your programmer).

That means disk IO isn't likely to be a limiting factor and you'd best be served by spending your money on the processor. The questions you can't/didn't answer aren't likely to change that. The computer you posted would likely be good for what you are doing, but I don't know if you could get something comparable for cheaper (the one I described in my first post would likely be several hundred cheaper).

[/ QUOTE ]
Yes, there are new inputs for each experiment. None of the experiments depend on prior experimental results. The only thing that relies on prior experimental results in the final tally, which comes after all the experiments have run.

[ QUOTE ]
That means disk IO isn't likely to be a limiting factor and you'd best be served by spending your money on the processor. The questions you can't/didn't answer aren't likely to change that. The computer you posted would likely be good for what you are doing, but I don't know if you could get something comparable for cheaper (the one I described in my first post would likely be several hundred cheaper).

[/ QUOTE ]
Thank you for your advice. I very much appreciate it.

Re: I need a serious workhorse computer for complex simulations
 

[ QUOTE ]
If you can talk to the programmer why not just ask him what would be best?

[/ QUOTE ]
He doesn't know anything about hardware. He didn't even know how to hook up a bluetooth printer adapter. [img]/images/graemlins/frown.gif[/img] But, on the bright side, he's supposed to be the best grad student in the CS dept when it comes to programming C++ and Java. Go figure.

Re: I need a serious workhorse computer for complex simulations
 

[ QUOTE ]
[ QUOTE ]
If you can talk to the programmer why not just ask him what would be best?

[/ QUOTE ]
He doesn't know anything about hardware. He didn't even know how to hook up a bluetooth printer adapter. [img]/images/graemlins/frown.gif[/img] But, on the bright side, he's supposed to be the best grad student in the CS dept when it comes to programming C++ and Java. Go figure.

[/ QUOTE ]
Dear lord, he should at least know what the memory usage is. They don't teach kids anything these days. *grumble grumble*

Anyway you'll need a system with enough memory to contain the program (if you don't have disk activity while running than whatever you have is good enough). Which leaves just getting the fastest processor you can get. With your budget you might be able to swing dual quad core Xeons.

Re: I need a serious workhorse computer for complex simulations
 

[ QUOTE ]
I will add this. Given the computational parameters in your original post, it seems to me that a dual quad-core system would be overkill. You could probably get by with something smaller for much less.

That's just semi-informed personal opinion though. I can't provide any evidence to back that up.

[/ QUOTE ]

Would depend on how many threads are being run or how many individual processes are being started.

If its a truly multi-threaded application, the more core's the better...though I don't know how well Mac OS scales for CPUs. Windows tends to stop scaling decently after 4, though it still utilizes the processors, just less effectively as Windows doesn't pin the scheduler to a particular CPU. But since Mac is built off of Unix, it might do better with up to 8.

Re: I need a serious workhorse computer for complex simulations
 

Ganjasaurus, he has to know if it's multithreaded or not. Find out! It matters!

Re: I need a serious workhorse computer for complex simulations
 

I'd probably go with server hardware and get a high quality motherboard that can take 8-16GB of FB memory.

Raid 1 is probably a good idea also, since I'm guessing losing all your data would be bad. Raid 1 means it's constantly writing the same data to both harddrives so if one fails you don't lose any data. This does not replace the necessity of backups, however.

The most important thing to figure out directly from the programmer is whether or not it is multithreaded, like nuclear500 said.

Multithreaded means that there are different 'parts' of the program that can be assigned to multiple processors. If it's not multithreaded, you need the best 'core' of a processor you can afford; you won't benefit from dual-core, quad-core, or dual processors unless you are running your simulation many times at once.

If it IS multithreaded, then you should look at good multi-core systems.

Personally I would try to get quad-core xeons on a server board, and probably get 2 10k or 15k RPM SAS drives in Raid 1. A good raid controller is also probably important.

Re: I need a serious workhorse computer for complex simulations
 

[ QUOTE ]
Ganjasaurus, he has to know if it's multithreaded or not. Find out! It matters!

[/ QUOTE ]
I asked the programmer, and he says it is multi-threaded.

Re: I need a serious workhorse computer for complex simulations
 

Thanks Freakin!


[ QUOTE ]
Personally I would try to get quad-core xeons on a server board, and probably get 2 10k or 15k RPM SAS drives in Raid 1. A good raid controller is also probably important.

[/ QUOTE ]
Sorry to sound ignorant again, but is this something i can show our computer tech dept so they can order it? Or do i need more details for them?

Re: I need a serious workhorse computer for complex simulations
 

[ QUOTE ]
Thanks Freakin!


[ QUOTE ]
Personally I would try to get quad-core xeons on a server board, and probably get 2 10k or 15k RPM SAS drives in Raid 1. A good raid controller is also probably important.

[/ QUOTE ]
Sorry to sound ignorant again, but is this something i can show our computer tech dept so they can order it? Or do i need more details for them?

[/ QUOTE ]

If your computer tech department has a guy responsible for building and configuring servers, then he should be able to shop for what you need.

It's been a while since I looked around, but I think the Intel Xeon Clovertown are still the best.

Something like the 5355 or 5345 are probably the best pricepoint for you

http://www.newegg.com/Product/ProductLis...name=Clovertown

You'd also want to get fully buffered (FB) dimms for your memory.

I honestly can't recommend a good motherboard since it's been too long since I shopped for one. Your computer department guys should be able to.

So tell them you want 1 or 2 Xeon Clovertown X5355's, 8-16GB of Fully Buffered memory.

I honestly don't know if you want high-speed storage or more spacious storage. If you're getting 10-15k RPM drives then the biggest drive you'll be getting is 74-150GB, and it will be expensive. Expect to pay about $300-350 for 74GB of 15kRPM Serial Attached SCSI (SAS). This will be faster performance, but possibly not necessary for your uses.

From the way you've described it, it sounds like you don't need high speed storage. You'd probably benefit from having your OS on a 10k RPM drive, but you can get a Western Digital Raptor drive for $180 or so that is 150GB of 10kRPM storage. This would be easy and cost-effective.

Re: I need a serious workhorse computer for complex simulations
 

Awesome. Thanks for your help!

Re: I need a serious workhorse computer for complex simulations
 

[ QUOTE ]
Awesome. Thanks for your help!

[/ QUOTE ]
Just to give you an idea this is a server I spec'd out at dell for $1,765. I did an Apple as well but that came out to $4,754 (to be fair the apple did use better processors). Needless to say if you can get the programmer to make it work on linux you can save yourself a ton of money. Also the machine displayed below neglects memory/disks since it sounds like these are not critical for your current application. If you intend to do anything with this after your current research is complete it would almost certainly be a good idea to upgrade both following Freakin's suggestions.


# PowerEdge SC1430


Date 11/20/2007 2:51:18 PM Central Standard Time
Catalog Number 4 Retail 04
Catalog Number / Description Product Code SKU Id
PowerEdge SC1430:
Quad Core Intel® Xeon® E5335; 2X4MB Cache, 2.0GHz, 1333MHZ FSB 143E20 [223-1438] 1
2nd Processor:
Quad Core Intel® Xeon® E5335; 2X4MB Cache, 2.0GHz, 1333MHZ FSB 2PE20 [311-7416] 2
Operating System:
No Operating System NOOS [420-6320] 11
Memory:
2GB 667MHz (2x1GB), Dual Ranked Fully Buffered DIMMs 2G2D6S [311-6152] 3
Hard Drive Configuration:
Onboard SATA Controller supports 1-2 Hard Drives - No RAID MSTNR [341-3825] 27
Hard Drive Controller:
Onboard SATA Controller - No RAID OBSATA [430-1949] 9
Primary Hard Drive:
80GB 7.2K RPM Serial ATA 3Gbps 3.5-in Cabled Hard Drive, Primary 80S272 [341-3757] 8
2nd Hard Drive:
80GB 7.2K RPM Serial ATA 3Gbps 3.5-in Cabled Hard Drive, Additional 80S2A72 [341-3763] 23
Floppy Drive:
No Floppy Drive NOFD [341-4035] 10
CD/DVD Drive:
48X CD-RW/DVD ROM Drive CDRWDVD [313-4337] 16
Network Adapter:
On-Board Single Gigabit Network Adapter OBNIC [430-0488] 13
System Documentation:
Electronic Documentation and OpenManage CD Kit EDOCS [310-8146] 21
Keyboard:
No Keyboard Option NOKYB [310-5017] 4
Mouse:
No Mouse Option NOMSE [320-2346] 12
Monitor:
No Monitor Option NOMON [320-0058] 5
Hardware Support Services:
3Yr STANDARD SUPPORT: 7x24 Tech HW/SW Support, NBD Onsite After Diagnosis 3STDU [970-6242][970-6282][980-2590][985-0848][985-0857] 29
Installation Support Services:
No Installation Assessment NOINSTL [900-9997] 32