In a message passing system there may be times when mutable data must be shared amongst many tasks. In traditional programming this would be handled by a read-writer block, which would allow one writer thread to block all other threads while it updates the shared data. With high performance systems one doesn’t want to ever block threads, so an alternate solution is needed.
One interesting technique is the use of a message passing API and a set of task schedulers. Actions that require only reading data would be processed using a task scheduler that allows as many concurrent threads as there are available cores. As each message is processed its thread is immediately reused for the next available message.
Messages that may need to update the shared data use a separate task scheduler that only allows one message to be processed at a time. This is known as an exclusive scheduler.
The magic occurs with the use of what TPL Dataflow calls the “Concurrent-Exclusive Scheduler Pair”. This object combines the two task schedulers, one for the concurrent tasks and one for the exclusive tasks. When there is a message waiting to be processed on the exclusive scheduler no new messages are sent to the concurrent scheduler. Any threads that would have been used by the concurrent scheduler are returned to the thread pool just as if there were no more messages to be processed. Once the exclusive scheduler is done processing all its messages the concurrent scheduler is woken up again and may resume processing when a thread becomes available.
While this technique eliminates the problem with blocking threads on a read-writer lock, care must still be taken to use this technique correctly. Just as with a reader-writer lock, too many write messages can prevent the read messages from being processed. There is also a risk that something that isn’t under the control of the scheduler could update the shared data in an unsafe manner.