What is NX?

NX has been a common message-passing interface on multicomputers for about a decade.Its implementations, however, require substantial software overhead. For example, one-way user-to-user latency for a four-byte message on the Intel Paragon is about 50 microseconds, of which only about 1 microsecond is in hardware.Over 98% of the overhead is in software.This software provides protected communication, buffer management, translation between virtual and physical addresses, DMA setup, message receiving,d ispatching and delivery. It is clear that reducing the software overhead is the key to achievinglow-latency message passing.

Shrimp NX

We have implemented the NX message-passing library and measured its performance on our Shrimp multicomputer prototype. For a four-byte message, our implementation achieves a user-to-user latency of 12 microseconds, a significant improvement over the the Intel Paragon implementation which relies on a special message-passing processor. For large messages, our implementation quickly approaches about 23Mbytes/second, which is close to the bandwidth limit imposed bythe Shrimp hardware.The software overhead of NX message-passing primitives in our implementation is an order-of-magnitute smaller than that of commercial systems such as CM-5, Intel Paragon and Meiko CS-2.

As the Shrimp machine grows and matures, it can meet the needs of more demanding applications.We currently are collecting and running NX message passing applications on the Shrimp hardware, including Gaussian Elimination, LU Decomposition, FFT, EM3D,Ocean, and Barnes-Hut N-Body.

Last Updated : March 22, 1999

skumar@cs.princeton.edu