Spawn or Not

Separate concerns of code structure from runtime structure in Elixir. Use functional decomposition for the former, and processes for the latter. link

my main points: * Use functions and modules to separate thought concerns. * Use processes to separate runtime concerns. * Do not use processes (not even agents) to separate thought concerns.

The construct “thought concern” here refers to ideas which exist in our mind, such as order, order item, and product for example. If those concepts are more complex, it’s worth implementing them in separate modules and functions to separate different concerns and keep each part of our code focused and concise.

Using processes (e.g. agents) for this is a mistake I see people make frequently. Such approach essentially sidesteps the functional part of Elixir, and instead attempts to simulate objects with processes. The implementation will very likely be inferior to the plain FP approach (or even an equivalent in an OO language). Keep in mind that there is a price associated with processes (memory and communication overhead). Therefore, reach for processes when there are some tangible benefits which justify that price. Code organization is not among those benefits, so that’s not a good reason for using processes.

Processes are used to address runtime concerns - properties which can be observed in a running system. For example, you’ll want to reach for multiple processes when you want to prevent a failure of one job to affect other activities in the system. Another motivation is when you want to introduce a potential for parallelism, allowing multiple jobs to run simultaneously. This can improve your performance, and open up potential for scaling in both directions. There are some other, less common cases for using processes, but again - separation of thought

# Blackjack

we need to keep track of different types of states which change over time: a deck of cards, hands of each player, and the state of the round. A naive take on this, would be use multiple processes. We could have one process per each hand, another process for the deck of cards, and the “master” process that drives the entire round.

[Choosing processes to model this is a mistake.] The game is in its nature highly synchronized. Things happen one by one in a well defined order: I get my cards, I make one or more moves, and when I’m done, you’re next. At any point in time, there’s only one activity happening in a single round.

Using multiple processes to power a single round is therefore going to do more harm than good. With multiple processes, everything is concurrent, so you need to make additional effort to synchronize all the actions. You’ll also need to pay attention to proper process termination and cleanup. If you stop the round process, you need to stop all the associated processes as well. The same should hold in the case of a crash: an exception in a round, or a deck process should likely terminate everything (because the state is corrupt beyond repair). Maybe a crash of a single hand could be isolated, and that might improve fault-tolerance a bit, but I think this is a too fine level to be concerned about fault isolation.

(Here, the author shows a nice example of functional decomposition for separation of concerns. We skip ahead to capture details about the round.)

# Blackjack round

This abstraction, powered by the Blackjack.Round module, ties everything together. It has following responsibilities: * keeping the state of the deck * keeping the state of all the hands in a round * deciding who’s the next player to move * accepting and interpreting player moves (hit/stand) * taking cards from the deck and passing them to current hand * computing the winner, once all the hands are resolved

The round abstraction will follow the same functional approach as deck and hand. However, there’s an additional twist here, which concerns separation of the temporal logic. A round takes some time and requires interaction with players. For example, when the round starts, the first player needs to be informed about the first two card they got, and then they need to be informed that it’s their turn to make a move. The round then needs to wait until the player makes the move, and only then can it step forward.

My impression is that many people, experienced Erlangers/Elixorians included, would implement the concept of a round directly in a GenServer or :gen_statem. This would allow them to manage the round state and temporal logic (such as communicating with players) in the same place.

However, I believe that these two aspects need to be separated, since they are both potentially complex. The logic of a single round is already somewhat involved, and it can only get worse if we want to support additional aspects of the game, such as betting, splitting, or dealer player. Communicating with players has its own challenges if we want to deal with netsplits, crashes, slow or unresponsive clients. In these cases we might need to support retries, maybe add some persistence, event sourcing, or whatnot.

I don’t want to combine these two complex concerns together, because they’ll become entangled, and it will be harder to work with the code. I want to move temporal concerns somewhere else, and have a pure domain model of a blackjack round.

So instead I opted for an approach I don’t see that often. I captured the concept of a round in a plain functional abstraction.

(Here we skip ahead again. If you enjoy reading code, dear Reader, we recommend the full article.)

# Sending notifications

When functions from the `Round` module return the instruction list to the server process, it will walk through them, and interpret them.

The notifications themselves are sent from separate processes. This is an example where we can profit from extra concurrency. Sending notifications is a task which is separate from the task of managing the state of the round. The notifications logic might be burdened by issues such as slow or disconnected clients, so it’s worth doing this outside of the round process. Moreover, notifications to different players have nothing in common, so they can be sent from separate processes. However, we need to preserve the order of notifications for each player, so we need a dedicated notification process per each player.

When the round server plays a move, it will get a list of instructions from the round abstraction. The server will then forward each instruction to the corresponding notifier server which will interpret the instruction and invoke a corresponding M/F/A to notify the player.

Hence, if we need to notify multiple players, we’ll do it separately (and possibly in parallel). As a consequence, the total ordering of messages is not preserved.

It might happen that `player_2` messages arrives before `player_1` is informed that it’s busted. But that’s fine, since those are two different players. The ordering of messages per each player is of course preserved, courtesy of player-specific notifier server process.

Before parting, I want to drive my point again: owing to the design and functional nature of the `Round` module, all this notifications complexity is kept outside of the domain model. Likewise, notification part is not concerned with the domain logic.

# Conclusion

Simple functional abstractions such as `Deck` and `Hand` allowed me to separate concerns of a more complex round state without needing to resort to agents.

That doesn’t mean we need to be conservative with processes though. Use processes wherever they make sense and bring some clear benefits. Running different rounds in separate processes improves scalability, fault-tolerance, and the overall performance of the system. The same thing applies for notification processes. These are different runtime concerns, so there’s no need to run them in the same runtime context.

If temporal and/or domain logic are complex, consider separating them. The approach I took allowed me to implement a more involved runtime behaviour (concurrent notifications) without complicating the business flow of the round. This separation also puts me in a nice spot, since I can now evolve both aspects separately. Adding the support for dealer, split, insurance, and other business concepts should not affect the runtime aspect significantly. Likewise, supporting netsplits, reconnects, player crashes, or timeouts should not require the changes in the domain logic.

.

I'm super excited to see _code structure_ delineated from _runtime structure_. The software industry needs language and good examples to discuss those things separately. See BEAMs of Insight