Windows HPC

[abachler]

Has anyone evaluated or used Windows HPC? I am downloadign ti now to test on my cluster, it looks interesting from the fluff they write, but I was wondering how it performs in actual use. In particular does it work with existing applications withotu having to recompile? Am I correct in assuming that it will automatically migrate threads or do I have to use explicit API calls to get a thread to migrate to a new node?

[abachler]

OK, I foudn this -

http://msdn.microsoft.com/en-us/libr...80(VS.85).aspx

which details teh API. Apparently it isnt as simple as the fluff claims. Applications have to be written specifically to take advantage of the HPC nodes.

[matsp]

OK, I foudn this -

http://msdn.microsoft.com/en-us/libr...80(VS.85).aspx

which details teh API. Apparently it isnt as simple as the fluff claims. Applications have to be written specifically to take advantage of the HPC nodes.

It would be rather magical otherwise, since applications can't really KNOW that they would be spread across multiple machines.

--
Mats

[Codeplug]

You will have better luck find existing code that has been written for MPI - which MS has an implementation of for HPC. Will most like be some porting involved as well

http://msdn.microsoft.com/en-us/libr...31(VS.85).aspx

gg

[abachler]

well, thats not entirely true. There are several mechanisms for distributed memory and execution, they require kernel level support, and applications that routinely write to the same memory locations from multiple threads on different nodes can impact performance, but this is usually handled by the kernel migrating such threads to the same node. These have been used in supercomputing for some time, although I'm not sure if MS is ready to commit to movignteh windows kernel in this direction, as they usually wait for patents to expire first.

[Codeplug]

Yeah there's been a history of research on kernels running on DSM. There are some interesting SSI's for linux too:
http://openssi.org/cgi-bin/view?page=openssi.html
http://www.faqs.org/docs/Linux-HOWTO/SSI-UML-HOWTO.html

gg

[abachler]

Essentially it treats paged memory differently depending on whether its system/code/data. System memory is of course present on every node, and code pages are kept on every node, but restricted to read access, if a code page is written to, it must be updated on all other systems, while data pages are handled essentially like any other paging, except they are only ever present on one node, and paged out if another node requests access to them. The reason why code pages must be handled differently is due to multiple threads accessign reentrant code simultaneously, as in the CreateThread call. Usually seperate threads will access different data though, preventing network thrashing, although that is a performance consideration. Some devices such as MUTEX and CRITICAL_SECTION's have to be handled slightly differently, but nothing unusual. Such tight integration between teh kernel and the network requires kernel level support for networking though, which uually means its restricted to a subset of available NIC's. In practice the kernels are usualy written fro one particular model of NIC and all the nodes are identical, but that can be changed with custom kernels for each NIC.

I can feel the urge coming on to write another OS, it's been so long...

[matsp]

The reason why code pages must be handled differently is due to multiple threads accessign reentrant code simultaneously, as in the CreateThread call.

This doesn't sound right to me. The rare instance of self-modifying code would be a good reason for that - although self-modifying code is a BAD IDEA at all times.

--
Mats

[CornedBee]

Is it self-modifying if a JIT compiler compile bytecode once, and later (after gaining some profiling data) again?

[matsp]

Is it self-modifying if a JIT compiler compile bytecode once, and later (after gaining some profiling data) again?

No, I'm sure it allocates new memory for the new code.

--
Mats

[CornedBee]

It does, but it has to modify existing code to call the new code instead of the old.
(A group at my university is the main force behind the Cacao JVM, and they're researching this stuff. Not across nodes, just across cores in a multi-CPU Alpha system. Those CPUs have absolutely no cache coherency promises in the code cache.)

[abachler]

This doesn't sound right to me. The rare instance of self-modifying code would be a good reason for that - although self-modifying code is a BAD IDEA at all times.

--
Mats

That is why its set to read only, which raises an exception on write, allowing the kernel to temporarily gain exclusive use of that page, so self modifying code stiull works, it just suffers a performance hit while modifying itself, but regular code does not suffer a performance penalty when it is only executing unless some other node is writing to the code page at the same time. I am pretty sure that any feasable self modifying code is not reentrant and is unlikely to end up with a race condition with multiple threads. Alternatively, modifying a code page could induce the kernel to mark it as a data page instead, although that woudl have long term consequences for code that is both self modifying and reentrant after the modifications are done. It would be better to keep it marked as a code page, but to

1. gain exclusive access by marking the page as non-present on other nodes
2. marking the page as full access locally
3. if the page is requested by another node, mark the local page as read only locally and transfer a copy of the page to the requesting node

Ultimately code pages have to be handled differently for performance reasons. Code is much more likly than data to be simultaneously accessed by multiple threads which may be execting on different nodes. Code should thus have some mechanism to allow it to be executed in parallel. Data on the other hand has a greater need for atomic writes, which is only feasable if there is a single valid copy of the page in the system. Since data routinely experiences writes as well as reads, while code rarely is written and the paging mechanism supports a differentiation between code and data pages, it becomes obvious that we should take advantage of this to improve system performance for each type of memory.

[matsp]

Abachler, I do agree with that. But you originally stated a completely different reason for the code-pages being handled this way - and that is what I objected to. By the way, Linux and Windows generally treat code as READ ONLY in the first place, so I don't see how this is different for HPC shared memory across networked systems.

CornedBee: Yes, I suppose some modifications will happen in such a case. I doubt that Java will ever be a HPC solution for any larger scale projects, no matter how good the JIT is, it is unlikely to beat C or C++ (or Fortran) code compiled with a good compiler.

There are several problems with self-modifying code that are just simply processor related: The processor itself needs to identify that the code has changed, and not execute instructions in the pipeline that has been modified later on. It gets even worse when the modification is done on code that has already been speculatively executed. It's a bit like a worse case of branch-mispredict.

--
Mats

[abachler]

Abachler, I do agree with that. But you originally stated a completely different reason for the code-pages being handled this way - and that is what I objected to.

how so, I'm nto seign how I gave a different reason, maybe you took something differently than I meant it?

[matsp]

how so, I'm nto seign how I gave a different reason, maybe you took something differently than I meant it?

Sorry, reading your post again, I can't see what I think I saw then - time for my medication I think...

--
Mats