Many of the presumptions of how slow and resource-demanding "Fat” XML is compared to JSON’s lightweight payload do not hold up to a test David Lee, lead engineer at Marklogic, states after running an experiment with 33 different documents and almost 1200 tests on the most commonly used browsers and operating systems. David found the performance for the total user experience, (transfer, parsing and querying of a document), to be nearly identical for both XML and JSON formats.
For his experiment, David has created a publicly available test environment mimicking a use case with XML and JSON documents delivered by a web server and parsed and queried in a web browser. The server provides the client with source data and collecting results submitted by the client. The client is a browser based JavaScript application with the measuring part of the tests in handwritten JavaScript, except for a part that measures jQuery performance.
Besides using seven different documents, with size from 100kB to 1MB, and each document in two JSON and three XML variants, David also tries to cover a range of devices, browsers, operating systems and networks in his test. He has done that by "crowd sourcing", i.e. making the test environment URL public and distributed to a range of mailing lists and social media sites. Up till now almost 1200 distinct and successful tests results has been collected covering the most commonly used browsers and operating systems. In his article David has documented all test data as well as the results for the different tests.
Some of the conclusions David has drawn after the experiment are:
- Parsing speed varies with the technique used. Pure JavaScript parsing generally performs better with XML than with JSON, while query speed generally is faster for JSON. Both with exceptions though where the contrary is true.
- Using the JavaScript library jQuery imposes a steep penalty on JSON and even worse on XML.
- Compressed documents in all formats, even very large representations of JSON or XML, compress to nearly identical size which indicates that they contain approximately the same information content.
- Transferring documents to a wide variety of devices takes effectively the same time per device irrespective of mark-up format.
Based on his experiment David has some suggestions for architects and developers, including:
- Use HTTP Compression which most often is the single most important factor in total performance.
- Optimise the mark-up for transmission and query.
- Do not try to optimise unless the data transmission, parsing and querying is a significant problem comparing to other issues.
Finally David has one piece of advice:
Don’t Trust Anyone.
Don’t believe blindly what you are told. Perform experiments, test your own data and code with your own users and devices. What "seems obvious" is not always true.