Context Implementation Details
In terms of Contexts-Domains implementation, in spite of nomenclature based similarities, context-based domains and DMAPP-based domains are two separate entities. In this paper, context-based domain is referred as domain and DMAPP-based resource domain is referred as CDM.
Fig. 5. Thread-safe design in cray SHMEM
N. Namashivayam et al.
Fig. 6. Contexts-domains prototype: design-1 domain-based resource sharing
Domain and Context features are used to split up message injection resources from remote completion tracking. There are two different ways to track event completion. Based on remote completion tracking, we have designed two different Context prototypes.
- - Design:1 Domain-based Resource Sharing; and
- - Design:2 Context-based Resource Sharing
Design:1 Domain-Based Resource Sharing. Each domain has its own
CDM, and all contexts from that particular CDM shares same CQ. Now, CQ tracks completion of all events on that particular domain, and this is of little use because all memory ordering operations are contexts-based. Hence, completion of all events from a context is tracked separately with a unique reference called sync_id. sync_id is generated for each context-based events. This sync_id is tracked separately in SHMEM library-level, rather than on the DMAPP-level. shmem_ctx_quiet operation on a particular context waits for completion of all sync_ids associated with that context.
Figure 6, shows basic design for domain-based resource sharing. In Fig. 6, we have two domains created for a particular PE, with multiple contexts per domain. Each domain has its registered CDM, with an allocated CQ. All events from contexts that shares the same domain is tracked by a single CQ. Different shades of the CQs represent events from multiple different contexts, and events tracked using their sync_id in the SHMEM library-level is not shown in Fig. 6.
Design:2 Context-Based Resource Sharing. In this Design, each context has its own CDM and all memory ordering operations will be tracked through CQ mapped to that CDM. Here, domain is just a group of CDMs with similar properties.
Figure 7, shows design for context-based resource sharing. There are two domains, each with different thread-levels. Domain-1 has SHMEM_THREAD-SINGLE as thread-level with 2 contexts associated with it. Domain-2 has thread-level as SHMEM_THREAD_MULTIPLE, with 2 contexts associated with it. There are only 3 CDMs available to share within that PE. Due to thread-levels, all contexts in Domain-1 gets a unique CDM, while Contexts in Domain-2 share a common CDM.
Fig. 7. Contexts-domains prototype: design-2 context-based resource sharing