BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Articles Polymorphism of MVC-esque Web Architecture: Real Time Reactive Fulfillment

Polymorphism of MVC-esque Web Architecture: Real Time Reactive Fulfillment

Bookmarks

Key takeaways

  • MVC’s observable event-driven synchronization between the user’s views and their related real or virtual world had been largely absent in web applications over the first two decades of the web age.
  • Recent advancements have revitalized this fundamental ideal in the web development community.
  • The dWMVC and pWMVC architectural paradigms can be leveraged to fulfil and close end-to-end change-observing “event loops” to create seamless and efficient real time reactive application behaviors.
  • Both traditional middleware infrastructures and new breed of server runtime environments can be applied to fulfil these real time reactive behaviors.
  • Non-traditional server runtime environment and data repository enjoy the architectural freedom to use non-standardized technology to create innovative solutions.

Previously in Polymorphism of MVC-esque Web Architecture: Classification, we illustrated and discussed three categories of WMVC (web-based MVC) architectural paradigms. They were Server-side WMVC (sWMVC), Dual WMVC (dWMVC), and Peer-to-Peer WMVC (pWMVC). The sWMVC is generally static in nature, but the other two architectural paradigms can be utilized to build real time reactive web application components. In this follow-up paper, we will leverage these two architectural paradigms to design and demonstrate fully dynamic and reactive modern web components.

At the heart of MVC architectural approach is observable event-driven fulfilments of synchronization between the user’s views and their related real or virtual world that the views reflect upon. The views are expected to, with or without additional commands from the user, react to the changing world. This ideal has been reflected in many MVC realizations from the original desktop GUI to modern augmented and virtual reality (AR and VR). As discussed in the WMVC classification article, this fundamental idea had been largely absent in web applications over the first two decades of the web age. During this time, web applications are dominated by sWMVC-based approach. It has been somewhat revitalized in WUI (web user interface) application development community in recent years. This new movement has been driven by a number of recent technological offerings and standardized protocols.

In this paper, we will put some of these new advancements into action to fulfill asynchronous, natural, seamless, and efficient change-observing reactive “event loop” from WUI to backend SoR (source of record) repository. The key enabling technologies applied herein are:

The source codes related to the following discussions can be found here at GitHub.

The User Story

Let’s presume that our client wants a browser-based blog commenting system. This web application should allow users to view and post comments on a blog topic.

Below is a screenshot of a conceptual web page design, consisting of three subviews. The top block displays a blog topic, which is followed by a comment entry and submission field. The last section is the area displaying comments entered by all users.

Figure 1. A blog comment screenshot design.

This blog system shall include two distinctive applications:

  1. The first application shall take ownership of the blog comments and store them in a centralized repository.
  2. The second application will not store any user comments in a centralized database to ensure user privacy and client liability.

A third component of this system is integration and aggregation of blog comments from other sources into the centralized repository, which is to be developed in the future.

All the application users shall have an always up-to-date view of the blog comments.

As a user is reading the comments, new comments added by other users or by integration shall automatically and instantly display on all the viewing user’s web page without his or her manual command.

System Architecture

The blog web application with centralized repository will be designed and developed using the dWMVC paradigm. Overall, the communication among the application components will be realized with AngularJS, SSE, InSoR, and CDC. These technologies will allow the system to react to any modifications (through this web application or future integration module) to the records in the centralized repository and deliver the changes in real-time to end users, as outlined in Figure 2.

Figure 2. System architecture of centralized real time blog web application. The communication between the client and server sides is based on HTTP and SSE protocols, while InSoR and CDC complete the round trip between application server and data repository.

The second web application will be implemented with the pWMVC scheme (Figure 3). It will act as an enabler to bring the users together, without taking ownership of the contents exchanged.

Figure 3. System architecture of peer-to-peer real time blog web application.

Centralized Web App by dWMVC

Figure 4 below outlines the designs of the dWMVC-based blog web application. On the browser side, the view and controller components are based on AngularJS. Two different combinations of server-side technological stacks are used herein to implement the dWMVC model components. On the left is a traditional stack of Java and Java EE infrastructure, along with a relational database, PostgresSQL. NodeJS and RethinkDB are applied to illustrate the architectural paradigm of JavaScript-based server-side runtime environment and NoSQL data repository. These different server-side designs and implementations present two different approaches to the same functionality.  Aside from NodeJS’s asynchronous nature, the difference is particularly evident in InSoR and CDC, in which NoSQL database providers possess the architectural freedom to use non-standardized technology to create innovative solution (such as lazy evaluation and lazy loading).  The two implementations also provide technological flavors to meet variable interests of the web development community from traditional middleware practitioners to NodeJS/NoSQL enthusiasts.

Figure 4. An architectural diagram of dWMVC design scheme for the blog application. The client-side WMVC view and controller are based on AngularJS. Two flavors of server-side model components are: Java-RDBMS (left) and NodeJS-NoSQL (right).

View and Controller of dWMVC

The blog web page is implemented with AngularJS partial templates. It is a composite view used to serve both post and display of blog comments.

<div class="blocker1">
    <h3>Topic: WMVC Real Time Reactive Fulfillment</h3>    
</div>

<div id="castingId" class="blocker2">
    <div>
        <h4>Post a Comment:</h4>
    </div>
    <form id="commentFormId">
        <div>
            <input type="text" style="width: 30%" name="newCommentId" placeholder="What is in your mind?" ng-model="newComment"/>
            <button role="submit" class="btn btn-default" ng-click="addComment()"><span class="glyphicon glyphicon-plus"></span>Send</button>  
        </div>
    </form>
</div>

<div>
    <h4>All Comments:</h4>
</div>

<div class="view" ng-switch on="comments.comments.length" ng-cloak>
    <ul ng-switch-when="0">
        <li>
            <em>No comments yet available.  Be the first to add a comment.</em>
        </li>
    </ul>
    <ul ng-switch-default>
        <li ng-repeat="comment in comments.comments">
            <span>{{comment.comment}} </span>
        </li>
    </ul>				
</div>

The HTML page depends on a dWMVC controller (Figure 5) to communicate with the server side to add new comments and to be refreshed with new blog posts by other users.

Figure 5. View and controller components of the blog comment application.

To display and refresh blog comments to the user, the controller:

  1. Connects to the backend server through SSE over HTTP.
  2. Asynchronously retrieves and displays all existing blog comments, if any.
  3. Keeps the connection alive and listens for future SSE events, which carries updated comments as event payloads.
  4. Pushes and binds updated blog comments with the view page of the user when a SSE event occurs.

All these interaction and reaction are accomplished with the following code snippet:

var dataHandler = function (event) 
{
	var data = JSON.parse(event.data);
	console.log('Real time feeding => ' + JSON.stringify(data));
	$scope.$apply(function () 
	{
		$scope.comments = data;
	});
};
var eventSource = new EventSource('/wmvcapp/svc/comments/all');
eventSource.addEventListener('message', dataHandler, false);

When a user adds a new comment, it is simply passed onto the server side for processing:

$scope.addComment = function () 
{
	var newInput = $scope.newComment.trim();
	
	if (!newInput.length) 
	{
		return;
	}

	var url = '/wmvcapp/svc/comments/cast';
	$http.post(url, newInput);

	$scope.newComment = '';
};

Then, its related data change is captured and handled by server-side model components as discussed below.

Java and PostgreSQL Model Components of dWMVC

The major components involved in this traditional technological stack, a combination of Java-based middleware application libraries and a relational database, are shown in Figure 6.

Figure 6. The dWMVC model components based on Java and PostgreSQL.

The interactive and reactive events among these model components are illustrated in Figure 7 below. It shows that two users access the blog application.

Figure 7. A sequence of interactions providing real-time lazy updates to blog comment viewing users, based on Java and PostgreSQL relational database.

When a user brings up the blog web page, the dWMVC controller immediately instantiates a SSE instance, which initiates communication with the server to retrieve blog comments. The related server component, as shown below, is annotated to honor the SSE request and deliver SSE-based outputs. When the server-side component receives the SSE-based request from the dWMVC controller, it first queries the database for existing comments and broadcast an asynchronous EventOutput to the controller for displaying the comments to the user browser. In the meantime, the server-side component adds a listener to keep an ear on the “topics_observer” of the PostgreSQL database in order to receive continuous notifications of any subsequent changes to the blog topic in the PostgreSQL.

@GET
@Path("/all")
@Produces(SseFeature.SERVER_SENT_EVENTS)
public EventOutput getAllComments() throws Exception
{
	final EventOutput eventOutput = new EventOutput();
	Statement sqlStatement =  null;
	
	//Query and return current data
	String comments = BlogByPostgreSQL.getInstance().findComments(ConfigStringConstants.TOPIC_ID);       
	this.writeToEventOutput(comments, eventOutput); 
	
	//Listen to future change notifications
	PGConnection conn = (PGConnection)BlogByPostgreSQL.getInstance().getConnection();
	sqlStatement = conn.createStatement();
	sqlStatement.execute("LISTEN topics_observer");             
	conn.addNotificationListener("topics_observer", new PGNotificationListener() 
	{
		@Override
		public void notification(int processId, String channelName, String payload) 
		{
			JSONObject plJson = new JSONObject(payload);
			String plComments = plJson.getJSONObject("topic_comments").toString();
			writeToEventOutput(plComments, eventOutput);
		}
	});    
		
	return eventOutput;
}
  
private void writeToEventOutput(String comments, EventOutput eventOutput)
{
	OutboundEvent.Builder eventBuilder = new OutboundEvent.Builder();
	eventBuilder.mediaType(MediaType.APPLICATION_JSON_TYPE);
	
	if(comments == null || comments.trim().equals(""))
	{
		comments = NO_COMMENTS;
	}
	eventBuilder.data(String.class, comments);

	OutboundEvent event = eventBuilder.build();
	
	eventOutput.write(event); 
}

PostgreSQL is an open source relational database. One of its recently-added features is capture and delivery of full record changes as inbound payloads to connected application components. This InSoR capability is configured as a pair of database trigger and function. The following are the setups for our blog topic table.

CREATE OR REPLACE FUNCTION proc_topics_notify_trigger() RETURNS trigger AS $$
  DECLARE
  BEGIN
    PERFORM pg_notify('topics_observer', json_build_object('topic_id', NEW.topic_id, 'topic_comments', NEW.comments)::text);
    RETURN new;
  END;
  $$ LANGUAGE plpgsql
  
DROP TRIGGER trigger_topics_notify ON topics;
CREATE TRIGGER trigger_topics_notify AFTER INSERT OR UPDATE OR DELETE ON topics
  FOR EACH ROW EXECUTE PROCEDURE proc_topics_notify_trigger()

Assuming, while the users are reading the blog comments, one of them decides to add a new comment.

@POST
@Path("/cast")
@Consumes(MediaType.APPLICATION_JSON) 
public void addComment(String newComment) throws Exception 
{
	if(newComment != null && !newComment.trim().equals(""))
	{
		ObjectMapper mapper = new ObjectMapper();
		TopicComments topicComments;
		String comments = BlogByPostgreSQL.getInstance().findComments(ConfigStringConstants.TOPIC_ID);

		if(comments == null || comments.trim().equals(""))
		{
			topicComments = new TopicComments();
			topicComments.addComment(newComment);
			String topicCommentsStr = mapper.writeValueAsString(topicComments);
			BlogByPostgreSQL.getInstance().addTopic(topicCommentsStr);
		}
		else
		{     
			if(!comments.contains(newComment))
			{
				topicComments = mapper.readValue(comments, TopicComments.class);
				topicComments.addComment(newComment);
				String topicCommentsStr = mapper.writeValueAsString(topicComments);
				BlogByPostgreSQL.getInstance().updateTopic(topicCommentsStr);
			}
		}
	}
}

Then, as soon as the topic record in the database is modified with the new comment, the database “trigger_topics_notify” trigger will invoke its related function “proc_topics_notify_trigger” to initiate a change event notification for the “topic_observer”. The “topic_observer” notification will be immediately pushed inbound to the listener of the “topic_observer”, along with the updated JSON-formatted comments as data payload. The application component associated with the listener, in turn, processes and writes another SSE EventOutput to the controller to refresh the updated comments to both the users’ views. This is all accomplished without a necessity for the users to initiate new requests (Figure 7).

Node and RethinkDB Model Components of dWMVC

Over the past few years, NodeJS has become a new and prominent alternative server-side runtime environment for building web applications. Its core architecture is event-driven, asynchronous processing. RethinkDB is an open source NoSQL database, which takes heavy consideration of developing real-time web applications into its architecture and design. One of its built-in unique features is providing notifications of change events to calling application components.

In comparison with Figure 6, one of the key differences in Figure 8 below is that database trigger and procedure function are no longer required setups with RethinkDB. The notification of its database change events is accomplished with its chainable query language, ReQL

Figure 8. The dWMVC model components based on NodeJS and RethinkDB database.

Figure 9 below shows the sequence of interactive and reactive behaviors among application and database components.

Figure 9. A sequence of interactions providing real-time updates to blog comment viewing users, based on NodeJS and RethinkDB database.

When the server side component blogApp.js receives a SSE-based getAllComments request, it first prepares to respond accordingly by adding a special HTTP header, as shown below, to it’s initial response to establish a SSE handshake with the dWMVC controller. This allows the controller to continue listening for subsequent SSE streaming events.

function setUpSSE(res)
{
    res.writeHead(200, 
    {
        'Content-Type': 'text/event-stream',
        'Cache-Control': 'no-cache',
        'Connection': 'keep-alive',
        'Transfer-Encoding': 'chunked'
    });
    res.write('\n');
}

Next, it executes a chained ReQL query through BlogByRethinkDB.js to inform the database that it wants to observe and receive any future changes to the data record. This observable query allows the database to lazily stream the changes back into the application component immediately after the changes occur.

BlogByRethinkDB.prototype.observeComments = function(process)
{
    connectToRDB(function(err, rdbConn) 
    {
        if(err) 
        {
            if(rdbConn) rdbConn.close();
            return console.error(err);
        }

        //Listen for blog comment change events
        r.table(config.wmvcBlog.dbTable).filter(r.row('topicId').eq(config.wmvcBlog.wmvcBlogTopicId))
                .changes({includeInitial: false}).run(rdbConn, function(err, cursor) 
        {
            if(err) 
            {
                if(rdbConn) rdbConn.close();
                return console.error(err);
            }

            //Retrieve all the comments in an array.
            cursor.each(function(err, row) 
            {      
                if(err) 
                {
                    if(rdbConn) rdbConn.close();
                    return console.error(err);
                }

                if(row) 
                {
                    return process(null, row.new_val);
                }
            });
        });
    });
};

Then, it retrieves the existing comments of the requested topic.

BlogByRethinkDB.prototype.getAllComments = function(process)
{
    connectToRDB(function(err, rdbConn) 
    {    
        if(err) 
        {
            if(rdbConn) rdbConn.close();
            return console.error(err);
        }

        //Query for comments
        r.table(config.wmvcBlog.dbTable).filter(r.row('topicId').eq(config.wmvcBlog.wmvcBlogTopicId))
              .run(rdbConn, function(err, cursor) 
        {
            if(rdbConn) rdbConn.close();

            if(err) 
            {
                return console.error(err);
            }

            //Retrieve all the comments in an array.
            cursor.toArray(function(err, result) 
            {        
                if(err) 
                {
                    return console.error(err);
                }

                if(result && result.length > 0) 
                {
                    return process(null, result[0]);
                }
                else
                {
                    return process(null, null);
                }
            });
        });
    });
};

Finally, the server side object composes and returns a HTTP response with SSE-compatible formatted data.

function handleSSEResponse(err, blogComments, res, next)
{
    if(err) 
    {
        return next(err);
    }
    
    if(blogComments) {
        var id = new Date().getTime();
        res.write('id: ' + id + '\n');        
        res.write('data: ' + JSON.stringify(blogComments) + '\n\n');
    }
    else
    {
        var empty = new Array();
        var noResult = { comments: empty };
        var id = new Date().getTime();
        res.write('id: ' + id + '\n');        
        res.write('data: ' + JSON.stringify(noResult) + '\n\n');
    }
}

Subsequently, when a new comment is added to the system, the observeComments function will react to its database change event asynchronously and broadcast the updated comments to all viewing users, as shown in Figure 9.

Peer-To-Peer Web App by pWMVC

The backbone of the pWMVC architectural scheme is the WebRTC protocol. Specifically, one of its major components, RTCDataChannel, is used in this blog app implementation. This component provides the capability of bidirectional peer-to-peer (P2P) data transfers between browsers, without requiring an additional plugin installed.

A JavaScript wrapper library for the RTCDataChannel, DataChannelJS, is used to avoid low level complexity and simplify the implementation. For the same reason, PusherJS is selected to provide signaling service. A WebRTC-aware application requires a signaling channel for the participated clients to exchange information about their session description and network reachability. The entire application is glued together for deploying to a NodeJS web server.

It shall be noted that neither NodeJS server nor PusherJS signaler persists or retains any data exchanged among the browsers. As shown in Figure 10, the information exchanged among the participants is stored in the browser localStorage of each user. Along with the localStorage, all major application components are located at and executed from the browser at runtime. The NodeJS component is only to relay the blog comments between browsers, maintain the group connection status, and keep communication channel open.

Figure 10. An architectural diagram of pWMVC implementations for the blog application. All application comments and data repository are on the user browser. The main role of the Node.js server is for signaling between all the participant browsers.

Figure 11 illustrates the sequential process flow in which two users establish and form a blog topic group. As the first user accesses and initializes the pWMVC application on his browser, the p2pController goes through a number of steps to open a DataChannelJS instance, bind to a PusherJS signaling channel, and start sending out communication signals. At this point, since there is no other peer participant(s), the application displays a default page to this first user. Next, another user brings up the blog web page. The p2pCcontroller detects that the blog group has been opened, thus it simply connects the DataChannelJS of this second user and binds it to the PusherJS signaler. Immediately afterward, these two browsers engage in a series of ICE (Interactive Connectivity Establishment) communication and negotiation to complete a p2p handshake. This process is represented by chunks after chunks of outputs on the browser console window; thus the details are not shown for brevity. After this handshake, these two users are ready to exchange information privately and only between themselves as the DataChannelJS is now open.

Figure 11. A sequence of interactions between two user browsers to establish WebRTC-based communication, based on Pusher.js, DataChannel.js, and Node.js (continued in Figure 10).

As soon as the DataChannelJS is open between the two users (Figure 12), the application first retrieves and displays existing comments, if any, on this topic from the browser localStorage, so they know where they left off from last conversation.

webRTCDatachannel.onopen = function (userId) 
{
	p2pModel.getAllComments(groupName)
	.success(function(updatedComments) 
	{
		if(updatedComments === null)
		{
			updatedComments = { comments: new Array() };
		}
		
		$scope.comments = updatedComments;
	})
	.error(function(error) 
	{
		alert('Failed to save the new comment' + error);
	});
}


getAllComments: function (groupName) 
{
	var savedComments = $window.localStorage.getItem(groupName);

	if(savedComments !== null)
	{
		savedComments = JSON.parse(savedComments);
	}

	var updatedComments = aggregateComments("", null, savedComments);

  return handlePromise($q, updatedComments);
}

Figure 12. Continued from Figure 11, a sequence of interactions between two user browsers to exchange WebRTC-based messages, based on Pusher.js, DataChannel.js, and Node.js. The messages are retained in the localStorage of individual user browser.

While both of these users are reviewing the comments, their browsers continue signaling to each other to keep the communication channel open. Therefore, the users can post additional new comments as shown in Figure 12 and the following code snippet.

$scope.addComment = function () 
{        
	var newInput = $scope.newComment.trim();
	if (!newInput.length) 
	{
		return;
	}
	
	var currentComments = $scope.comments;
	
	p2pModel.aggregateAndStoreComments(groupName, newInput, currentComments)
	.success(function(updatedComments) 
	{
		webRTCDatachannel.send(updatedComments);
		$scope.comments = updatedComments;
	})
	.error(function(error) 
	{
		alert('Failed to save the new comment' + error);
	});

	$scope.newComment = '';
}

When a new comment is posted, the p2pController first uses the p2pModel to aggregate and update the localStorage for the topic (shown below). Then, the updated comments are sent to other participants through the DataChannelJS.

aggregateAndStoreComments: function (groupName, comment, currentComments) 
{
	var savedComments = $window.localStorage.getItem(groupName);

	if(savedComments !== null)
	{
		savedComments = JSON.parse(savedComments);
	}

	var updatedComments = aggregateComments(comment, currentComments, savedComments);

	storeComments(groupName, updatedComments, $window);

	return handlePromise($q, updatedComments);
}

When the other participants receive the updated comments, the comments are displayed on the web page and also stored in their localStorage.

webRTCDatachannel.onmessage = function (newInput, userId) 
{
	p2pModel.aggregateAndStoreComments(groupName, "", newInput)
	.success(function(updatedComments) 
	{
		$scope.comments = updatedComments;
	})
	.error(function(error)
	{
		alert('Failed to save the new comment' + error);
	});
}

Summary

Although the interactive and reactive ideal of the MVC architectural approach was diminished in web applications during the first two decades of the world wide web, recent advancements have revitalized this fundamental theory in the web development community. Standardized communication protocols and proprietary InSoR capabilities allow information change events to dynamically and asynchronously loop across infrastructural boundaries of web application systems in real time. These enable modern web application developers to leverage the dWMVC and pWMVC architectural paradigms to fulfil and close MVC-esque real time change-observing “event loops” to create seamless and efficient reactive application behaviors in variable fashions. These facilities are available not only in modern new breed of server-side runtime environments but also in traditional middleware infrastructures.

About the Authors

Victor Chen aspires to pursue innovation and transformation in computing. He has experienced in design and implementation of virtual reality, mobile, and web applications. He had also successfully designed and constructed a number of competitive robots.

 

 

Brent Chen has experienced in system architecture and application developments since the 90s. His solution deliveries have covered a range of subject domains, including payroll, human resources management,employee benefits management, regulatory compliance, health care, and governmental affairs.  He has affiliated with a number of leading solution and service providers, such as Computer Sciences Corp, Northrop Grumman, and ADP, LLC.  One of his research interests is exploring the fresh opportunities and emerging frontiers of current and developing web architecture and technologies.

Rate this Article

Adoption
Style

BT