Fig. 2. Incorporation of the proposed active messages prototype into the OpenSHMEM reference implementation
The prototype implementation was designed as part of the Open- SHMEM reference implementation  that in turn uses GASNet  for inter-process communication. Our prototype is built on top of the existing support of Active Messages that is offered by GASNet.
The incorporation of the prototype within the OpenSHMEM reference implementation is illustrated in Fig. 2. It must be noted that fine- tuned implementations of Active Messages in OpenSHMEM should take advantage of network hardware capabilities (if any) and the exploration of different design approaches is out-of-scope of this paper.
The experimental results presented in the following sections were obtained using a cluster with AMD Opteron processors (model 6174) and Infiniband interconnect (Mellanox MT26418). Each compute node comprises of a total of 48 cores (4 sockets/node, 12 cores/socket) with approximately 5 MB shared L3 cache and 16 GB main memory. The OS distribution on each compute node is OpenSUSE Linux (ver. 3.11).
Process Layout. The results from the bandwidth and message rate tests, microbenchmarks were obtained by binding each process (PE) to a specific core on different compute nodes. The results for the token-ring based tests and the miniapp (Traveling Salesman Problem) were obtained by launching multiple number of PEs - 2 through 512 and 256 respectively, each bound to a specific core across multiple nodes.
Fig. 3. Communication line diagrams and performance results for bandwidth and message rates
-  The Active Message prototype implementation is available as a fork of the Open-SHMEM reference implementation and is available as a git repository at https://github.com/openshmem-org/openshmem-am.