Why atomics have integrated ordering constraints

The interfaces for atomic objects that we have been considering provide ordering constraints as part of the atomic operations themselves. This is consistent with the Java and C# volatile-based approach, and our atomic_ops package, but inconsistent with some other atomic operation implementations, such as that in the Linux kernel, which often require the use of explicit memory fences..

Some rationale for this choice is provided as part of N2145 and N2047. Here we provide a somewhat expanded and updated version of that rationale.

Note that N2153 argues that explicit fences are still needed for maximum performance on certain applications and architectures. The arguments here do not preclude providing them as well.

Here we list our reasons for explicitly associating ordering semantics with atomic operations, and correspondingly providing different variants with different ordering constraints:

On architectures such as X86 and Itanium, it can lead to substantially faster code, at least in the absence of significant compiler analysis. For example, on X86, a lock is often released with a simple store instruction, which is widely believed to effectively have implicit "release" semantics. If this appears in the source as store_release it is easy to simply map that to a store instruction. If it is expressed as a fence followed by a store, the compiler would have to deduce that the fence is redundant.
On X86 processors, the fence is redundant only if precisely the right kind of fence (for a store, one that prevents "LoadStore" and "StoreStore" reordering) is used. (N2153 does suggest such a fence.)
On Itanium, the fence, once requested, can generally not be optimized back to an st.rel instruction. To see this, consider the hypothetical lock release sequence:
```
x = 1;	// lock protect assignment
LoadStore_and_StoreStore_fence(); // hypothetical optimal fence
lock.store_relaxed(0);	// atomically clear spinlock
y.store_relaxed(42);  // Unrelated atomic assignment
```
Note that this does not allow the assignments to x and y to be reordered. However, on Itanium, we really want to transform this to the equivalent of
```
x = 1;	// lock protect assignment
lock.store_release(0);	// atomically clear spinlock
y.store_relaxed(42);  // Unrelated atomic assignment
```
This is not safe, since this version does allow the assignments to x and y to be reordered. In most realistic contexts, it would be hard to determine that there is no subsequent assignment like the store to y. Hence I would not expect this transformation to be generally feasible.
This issue is admittedly largely Itanium specific. But note that the above reordering should be allowed; the fence-based implementation adds a useless constraint. If we only have fences, we can't express the abstractly correct ordering constraint for even a spin-lock release.
Acquire/release ordering constraints associated with atomic operations only affect the threads accessing those atomics. Hence, if the optimizer combines those threads (or if there is only one such thread to start with), not only can the atomic operations become ordinary memory operations, but any need for fences or similar mechanisms in the generated code disappears. Fences, on the other hand, order an open-ended set of memory accesses, which are unlikely to all be sufficiently visible to the optimizer to ever optimize out a fence. (This is closely related to the preceding point, and the detailed benefits depend on details of the memory model that we are currently still revising.)
In many cases, it seems to be more convenient. For example, double-checked locking can be easily written with load_acquire and store_release, with no explicit barriers. (The semantics of the fence version are also unnecessarily stronger, causing unnecessary overhead on Itanium. We are not aware of common examples where the reverse is true for other architectures.)
It gives us an easy way to express that atomic loads and stores "normally" have acquire and release semantics, but that weaker options may exist. Acquire loads and release stores enjoy the highly desirable property that they do not need dependency-based memory ordering. They also seem to be the minimal ordering properties that somewhat naive programmers expect without realizing it.
It makes it harder to ignore ordering issues. Ignoring them is almost always a mistake at this level. A number of existing atomics interfaces suffer from either sloppiness in dealing with memory ordering, or from making it impossible to state some useful ordering constraints. Empirically, this has resulted in many bugs.
Explicitly ordered atomics integrate cleanly with "higher level" atomics that guarantee sequential consistency if all "races" involve only atomic operations. These again provide a consistent programming model across C++ and Java. (I believe C# volatiles are essentially acquire/release atomics. Thus we should also get consistency with C#.)