32bit vs 64bit, and what about multicores?

[seanb]

I've been following temporary's guide, and WOW do i love him/her for it. This is more a learning experience for me than anything. I dont expect to get rich or popular running a private wow server

It does leave me with a question though. I have a 64bit processor, and i'm using 64bit windows 7. When i downloaded everything, i was never given any options for 32 or 64 bit. Which did i compile, and does it really matter?

Also, what about multi-cores? Does this emulator benefit from them at all? Or if i was building a machine to run a server dedicated, would i just find the fastest single core system i could find? (probably 3ghz)

[xeross155]

At the moment the server mightly slightly benefit from multiple cores, however the biggest CPU sucker is the map code, however this isn't running on multiple threads yet.

And for 64 bit you can choose that in the configuration manager when compiling I think.

[darkstalker]

you need 64bit if you plan to use 4gb or more memory

[rilex]

you need 64bit if you plan to use 4gb or more memory

More accurately, when you need more than 3GB VAS (for Windows Server 32bit systems with /3GB) or 2GB VAS (for Windows Client 32bit systems). Of course, 64bit Windows Server/Client provide 4GB VAS to each process.

[freghar]

you need 64bit if you plan to use 4gb or more memory

To be more exact - most 32bit systems can use up to 64GB of RAM, they just can't address it directly.

More accurately, when you need more than 3GB VAS (for Windows Server 32bit systems with /3GB) or 2GB VAS (for Windows Client 32bit systems). Of course, 64bit Windows Server/Client provide 4GB VAS to each process.

Even more accurately, 32bit VAS is always 2^32. No more, no less. However a 32bit app can't use entire 4GB, because it needs already-mapped (loaded) system/user libraries, it needs addresses for various system calls (like mmap()) and so on. That gives 3GB of RAM on newer (AFAIK post-2.6.13) Linux systems for in-process user data (ie. 3GB can be malloc()'ed).. Windows have 2GB by default, but it can be changed to 3GB via some /3GB switch in boot.ini or so, it never worked for me.

The situations is the same on 64bit system using 32bit userspace application. The app stil needs to access all those shared/system calls, it needs _some_ way to talk to the kernel and since the app itself uses 32bit pointers, it can't use addresses larger than 2^32-1 for it. This is not true for 64bit userspace applications, obviously.

[caeruleaus]

freghar: On windows for the /3GB boot switch to work, you need to compile mangos with the large address aware flag. I don't think it's set by default though I could be wrong.

EDIT: NVM It seems that it's set by default now. Did you try the /3GB flag in conjunction with the /PAE boot flag?

[rilex]

Yes, it has been set with LAA flag by default for some time now.

[freghar]

freghar: On windows for the /3GB boot switch to work, you need to compile mangos with the large address aware flag. I don't think it's set by default though I could be wrong.

EDIT: NVM It seems that it's set by default now. Did you try the /3GB flag in conjunction with the /PAE boot flag?

Yes, I've tried it with /PAE as well, though PAE shouldn't be related to 2/2 or 1/3 "switch".

Is there any particular reason why 2/2-compiled applications shouldn't run on 1/3 systems? Aside from some LAA thing (which is just a windows-specific binary flag as I googled out), there's no particular reason I can think of - assuming the NT kernel puts the shared 1GB stuff on top, leaving lower 3GB of VAS for the application itself. It would make sense for 1/3 -> 2/2 "move" when the binary could have hardcoded addresses above 2GB boundary, but not the other way around. Though I have to admit I'm no expert in NT kernel memory management.

PS: On a related note - I found out that winXP is unable to address more than 4G of RAM even with /PAE - because of some *possible* 3rd party driver issues. I understand that.

[rilex]

XP 32bit can't address more than 4GB of physical (and typically < 3.5GB; a limit introduced with XP SP2 due to 3rd party driver manufactures creating defective drivers) due to licensing issues.

Applications not compiled with the LAA flag in a Windows OS that supports /3GB will continue to use a 2/2 split.

/PAE is only related to using >4GB RAM but <16GB RAM on a 32bit system, however it was introduced with XP SP2 by default because DEP (NX bit) requires it. It doesn't enable support of additional physical memory in XP 32bit, however.

[freghar]

XP 32bit can't address more than 4GB of physical (and typically < 3.5GB; a limit introduced with XP SP2 due to 3rd party driver manufactures creating defective drivers) due to licensing issues.

Applications not compiled with the LAA flag in a Windows OS that supports /3GB will continue to use a 2/2 split.

/PAE is only related to using >4GB RAM but <16GB RAM on a 32bit system, however it was introduced with XP SP2 by default because DEP (NX bit) requires it. It doesn't enable support of additional physical memory in XP 32bit, however.

Yes, I was able to read all that on the web, that LAA flag is there to *not* negatively affect performance for apps that use less than 2G of RAM (and thus don't need 3/1). It's even possible to have 4/4G "switch", in fact there's been a patch for 2.3/2.4 linux kernel doing that, but removing the shared part (and thus needed context switches, TLB flushes, ...) caused about 30% slowdown. There are also ways to access more than 4G RAM from within a single process, either using some windows fancy-name feature or mmap() hooks.

What I didn't understand were crashing apps with forced LAA flag even when they were compiled without it - IIRC there was (on linux) also 2/2->3/1 switch around 2.2-2.3 kernel and no apps needed recompilation. I'm not talking about LAA flag itself, the flag (as I found out) is just a "allow 3/1 for me" thing, it has nothing to do with actual binary code (ie. it can be switched using HEX editor). However since most of the crash-sensitive applications were/are big ones like certain games, I guess NT kernel puts the shared part (2GB) in the lower section of VAS, leaving upper 2GB for app data. When the kernel try to run the binary in 3/1 mode, there's a 1GB gap and a missing 1GB on top, which probably causes those crashes. That's just my theory, as said earlier, I'm no NT kernel memory specialist.

Anyway, it all boils down to using 64bit OS whenever possible, all major OSes are able to run both 64bit and 32bit code (given proper libraries, which is - again - automated). Having 4GB on 32bit CPU is pretty rare, though my IBM R60 is a living proof of it. So there's no reason to use 32bit system on 64bit capable CPU ... other than saving a few MBs of used memory and some GBs of disk space.

[rilex]

that LAA flag is there to *not* negatively affect performance for apps that use less than 2G of RAM (and thus don't need 3/1).

LAA has nothing to do with RAM. Please don't confuse RAM and VAS.

There are also ways to access more than 4G RAM from within a single process, either using some windows fancy-name feature

Windows uses AWE. But AWE can only address certain data types. It isn't worth it to implement AWE at this point.

I guess NT kernel puts the shared part (2GB) in the lower section of VAS, leaving upper 2GB for app data

No, lower bound is app, upper bound is kernel.

There are no "gaps".

[faramir118]

XP 32bit can't address more than 4GB of physical (and typically < 3.5GB; a limit introduced with XP SP2 due to 3rd party driver manufactures creating defective drivers) due to licensing issues.

I had always assumed that other devices used the same address space, and Win32 just addressed RAM last, such as graphic adapter memory taking up a considerable chunk of address space.

Never would have guessed it was intentionally crippled like this.

Do you have any recommended reading on the subject?

[rilex]

The 4GB limit is intentional (so they can divide up the space, although 64bit Windows Vista/Windows 7 can address something like 128GB of RAM). The 3.5-ish limit was not originally in place. It was done because of poor driver manufactures (i.e. nVidia, ATi, Creative). Yes, device address space is also placed at 4GB and working on down (that is why with the more graphics card RAM, the less available physical RAM there is to the OS).

Windows Internals, 5th Edition: http://www.amazon.com/Windows%C2%AE-Internals-Including-Windows-PRO-Developer/dp/0735625301/ref=sr_1_1?ie=UTF8&s=books&qid=1273894225&sr=8-1

[Patman128]

If you are using Visual C++, it is compiling 32-bit. If you are using Visual Studio, you can select either Win32 or x64 mode from the drop down build menu at the top.

The main difference is that in 32-bit mode, it can only use up to ~4 GBs of RAM, where in 64-bit mode it can use much much more.

Mangos is multi-threaded, so I would assume it would benefit from a multicore processor, but I'm not sure about it, I've never seen benchmarks done. The best processor you can get would probably be a Gulftown Xeon. Any Nehalem based processor is good. NEVER judge a processor based on it's GHz EVER, unless you are comparing 2 processors of the same type. 3GHz means basically nothing.

[rilex]

VC++ is a language (Microsoft's implementation of C++). Visual Studio is an IDE. When you're using Visual Studio and MaNGOS, you're using MS VC++. VC++ can compile in either 32bit or 64bit modes (extra work needed with VS Express to be able to do so).

While MaNGOS does have multiple threads, it doesn't have it where it "counts". Map code, IIRC, is not multithreaded. While a dual core processor will help, it is doubtful a Nehalem would have a significant performance gain over, say, a Core 2 Duo when it comes to running MaNGOS. If you're packing MaNGOS, MySql, and IIS or Apache on one machine, then it would probably help to have > 2 cores.

[Patman128]

VC++ is a language (Microsoft's implementation of C++).

Sorry, but what? Since when dd Microsoft create their own C++ variant? It has certain compiler features, yeah, but the core language is still the same, it's fully ANSI/Ecma compatible.

http://msdn.microsoft.com/en-us/library/3bstk3k5.aspx

This reference explains the C++ programming language as implemented in Microsoft Visual C++. The organization is based on The Annotated C++ Reference Manual by Margaret Ellis and Bjarne Stroustrup and on the ANSI/ISO C++ International Standard (ISO/IEC FDIS 14882). Microsoft-specific implementations of C++ language features are included.

So yeah, Microsoft did their own implementation, but only of standard C++ features, except for Visual C++ .NET 2003, which broke away from the standards in a few ways.

And Visual C++ is an IDE.

Microsoft Visual C++ (often abbreviated as MSVC) is a commercial integrated development environment (IDE) product engineered by Microsoft for the C, C++, and C++/CLI programming languages. It has tools for developing and debugging C++ code, especially code written for the Microsoft Windows API, the DirectX API, and the Microsoft .NET Framework.

Visual Studio just combines the Visual C++ IDE with Visual C#, Visual Basic, and some other stuff.