Udi Dahan on increasing scalability by making things asynchronous

The alternative and probably more scalable approach would be a network attached cache like IBM ObjectGrid or one of the gigoherence competitors. The network attached cache can hold millions if not hundreds of millions of such pairs in the collective memory of the web farm and then provide a login service to the farm. You're adding a little latency because of the network hop to fetch the data but the benefit is that you are no longer limited to what fits in a single address space in terms of how much you can store. As the farm scales out, the grid scales out in parallel to keep up. You're now limited to what fits in the memory of the farm, not a single process.

I don't see where there is any asynchronous activity here. The user shows up on the login screen, which blocks until he provides credentials, which are then sent on to the authentication mechanism (cached or DB, either way), which blocks, and then the results are returned and the user is routed to either a welcome screen, or an error. There is no asynchronous activity here is there?

Now the idea of caching credential data in memory is great, and will certainly speed that up. But how does it make it asynchronous?

What I thought the article was going to be about was an AJAXy sort of thing, where users are allowed into the app as soon as they have provided their credentials. Initially they have no more access than a guest, but at least the UI has options they can start using right away. Meanwhile, in the background, asynchronously, their credentials are being authenticated (however that happens) and the results are applied against their first activity that requires authentication. So they don't wait for authentication up front, they only wait for it after they have initiated an activity that requires it.

I enjoyed reading this, but was just confused about exactly what the technique brings to the table.

I agree. I too enjoyed the article. The approach, whilst not new, is elegant and has clearer options for scalability than what you might call "traditional" options. But if you have a user at one end, sending authentication data to an authenticator at the other end, waiting for a yes/no result that is elemental to what can happen next, then that to me is synchronous in nature.

The Ajax model could work well in handling the to and fro over two asynchronous steps, but you'd need timeouts and retries because ultimately you need that authentication yes/no to get to the secured stuff.

Nice article though.

John,

Glad you enjoyed it.

The asynchronous part of the solution deals with registering new users and the "long-running" workflow involved. Keeping the cache updated across the farm is also handled asynchronously/push-based with respect to servers that didn't have the user register there.

Does that make sense?

Julian,

Thank you for your kind words.

From the user perspective, logging in is a blocking process. Asynchrony is not a concept found in human-computer interaction design. Users care about blocking and speed (particularly for blocking processes).

The speed that is achieved is by having the entire login process occur on the web server. From an overall system perspective, the load on the DB decreases thus increasing the scalability of other aspects of the system.

Does the make it any clearer?

Billy,

The choice of technology you bring up is an interesting one. I could definitely see changing the implementation of the Cache object on the web server to store its data in a distributed, in-memory cache rather than just locally in memory. However, the overall solution could still look the same.

You do bring up an interesting architectural trade-off - an extra network hop vs using less memory. I'll definitely be looking at that in greater detail in my consulting engagements.

Of course there is asynchrony in the example. But it is only used to implement a distributed cache. I would really not try to implement such a scheme myself. It is much better to simply use a mature cache like billy mentioned.

By using this you can even simplify the registry of users. When a new user is registered you simply add it to the distributed cache. So all machines know the user. This way you do not need explicit message passing.

Christian,

"only" :-)

I agree that this isn't necessarily the end of the line for even the architectural analysis. Given that we have such a cache that efficiently utilizes memory, we'd want an intelligent load/data partitioning scheme such that requests go to servers that have the data needed by that request locally (in the distributed cache), so that we can save the extra network hop (which is critical in some environments).

I posted on this topic a while ago here,
and more recently here.

I'm really not trying to be argumentative here. But I believe words have meaning, and that their meanings are important, especially in technology.

The asynchronous part of the solution deals with registering new users and the "long-running" workflow involved.

Keeping the cache updated across the farm is also handled asynchronously/push-based with respect to servers that didn't have the user register there.

From the user perspective, logging in is a blocking process.

Asynchrony is not a concept found in human-computer interaction design. Users care about blocking and speed (particularly for blocking processes).

The speed that is achieved is by having the entire login process occur on the web server. From an overall system perspective, the load on the DB decreases thus increasing the scalability of other aspects of the system.

The alternative and probably more scalable approach would be a network attached cache like IBM ObjectGrid or one of the gigoherence competitors.

John,

I don't view your comment as argumentative - just a friendly technological discussion. Please keep it coming :-)

The long-running part is user registration - starting with the submission on the site, followed up with clicking on the email validation link. The asynchronous part is the use of one-way messaging.

Actually, at this point I'd like to invoke one of my betters - Pat Helland: "Asynchronous and Synchronous are subjective terms":

blogs.msdn.com/pathelland/archive/2007/08/23/as...

You're correct, though, about the login process blocking on the local cache - and, indeed, for much less time than when working with the DB.

And if the cache is stale, the caller will block for as long as it takes to refresh the cache.

When I rename a folder in Lotus Notes, and the entire application blocks, instead of just the one folder I am renaming, I very much understand that is asynchrony at play.