BT

Adding Flexibility to your REST Implementation with Yoga

Posted by Corby Page on Jun 27, 2013 |

REST APIs are extremely attractive for their design elegance. You get something that Google’s Adam Bosworth describes as “simple, relaxed, sloppily extensible,” but you do not get something built for consistent performance.

Existing REST frameworks do a great job of converting your domain model to a JSON or XML response. However, they operate under the assumption that each resource has a single document view. Every request for a resource returns the entire document, even if only a subset of the data is required for the request.

More importantly, each GET call in a REST API returns only one resource type. Therefore, a request that needs to aggregate data from multiple resource types will need to perform at least one request for each resource that comprises the desired result data. Think of a join query from the relational database world. Under a pure REST model, each row returned from the join would require its own GET request issued over the network. This level of chattiness quickly becomes a performance bottleneck in an Internet application.

What’s the Yoga Solution?

Yoga is an open-source toolkit that integrates with your existing REST server implementation, and allows you to customize your published API’s. It adds a selector to your GET requests which specifies exactly what data you expect to get from your request, like a column projection clause in the relational world. It also includes the ability to specify relational expressions that can aggregate multiple documents from different related resource types into a single response.

This is not a new concept. We’ve seen it before in published proprietary API’s, such as earlier versions of the LinkedIn API and Google’s GData specification. But unlike these API’s, Yoga allows you to introduce this syntax into your own Java REST application.

The remainder of this section will demonstrate use-cases where Yoga will improve application performance, while preserving the ease-of-use of REST syntax.

Specifying Resource Fields

Here is a typical REST request, for retrieving an instance of a resource. It will return all of the fields associated with the User resource type:

GET /user/1.json

What if, for security or performance concerns you want to generate a request that only returns name and location data for a given user? For development purposes, or publishing to trusted clients, you can append a selector to your request:

GET /user/1.json?selector=(id,name,city,state,country)

For publishing public API’s, you probably don’t want to give end-users unlimited selector capabilities. Here, you would simply publish an alias to the defined selector:

GET /user/1.json?selector=$locationView

Retrieving Multiple Resource Types

Now, consider a situation where you want to navigate your domain model’s object graph, and return data from multiple classes in a single API call:

The above graph reflects the data entity model utilized by a mobile or Javascript client which aggregates a number of entities on a single informational screen. One such API request issued by the client should return data about a user, his friends, which musical artists his friends like, and the albums and songs produced by those artists (this path in the graph is highlighted in maroon).

Concepts like User, Artist, and Song are distinct REST resources, so using the standard REST approach requires many distinct network calls:

GET /user/1.json (Get user)  

GET /user/2.json (Get detailed friend entities)     
GET /user/3.json     
...     
GET /artist/1.json (Get favorite artists)     
GET /artist/2.json     
...     
GET /album/1.json (Get albums for artists)     
GET /album/2.json     
...     
GET /song/1.json (Get songs for albums)     
GET /song/2.json     
...

Clearly, this does not scale as we traverse the depth of the object graph. Network latency will quickly become a performance bottleneck.

With selectors, we can specify all of the data we need, and retrieve it in a single request. For development and trusted clients, we can specify the selector explicitly:

GET /user/1.json?selector=friends(favoriteArtists(albums(songs)))

For production deployment of public APIs, we can publish an Alias:

GET /user/1.json?selector=$friendsFavoriteMusic

Implementing Yoga

Adding Yoga to your existing application requires minimal configuration changes. Yoga cleanly integrates with Spring MVC REST, Jersey, and RESTEasy. In the example below, we will implement Yoga in a Spring MVC REST application.

Our REST application uses Spring’s MappingJacksonJsonView to serialize responses:

        <property name="defaultViews">
            <list>
                <bean class="org.springframework.web.servlet.view.json.MappingJacksonJsonView"/>
            </list>
        </property>

Our User Controller uses parameterizable URIs to process GET requests for the User resource, and Spring’s @ResponseBody annotation to render the output with the MappingJacksonJsonView:

@RequestMapping("/user/{id}")
    public @ResponseBody User get( @PathVariable long id )
    {
        return _userRepository.fetchUser( id );
    }

If we need more control over how we render the User documents, we can migrate from a REST application to a Yoga application. First, we import our Maven dependencies:

        <dependency>
            <groupId>org.skyscreamer</groupId>
            <artifactId>yoga-core</artifactId>
            <version>1.0.0</version>
        </dependency>
        <dependency>
            <groupId>org.skyscreamer</groupId>
            <artifactId>yoga-springmvc</artifactId>
            <version>1.0.0</version>
        </dependency>

Next, we replace Spring’s MappingJacksonJsonView with the YogaSpringView, which knows how to parse selectors:

        <property name="defaultViews">
            <list>                 <bean class="org.skyscreamer.yoga.springmvc.view.YogaSpringView"p:yogaView-ref="jsonView"/>
<!--<beanclass="org.springframework.web.servlet.view.json.MappingJacksonJsonView"/>-->
            </list>
        </property>

The injected dependency of jsonView tells SpringMVC that Yoga will be processing JSON requests, and rendering JSON output. We define the jsonView in our application context:

    <bean name="jsonView" class="org.skyscreamer.yoga.view.JsonSelectorView"
          p:selectorParser-ref="selectorParser" />
    <bean id="selectorParser" class="org.skyscreamer.yoga.selector.parser.GDataSelectorParser"/>

Here, we also specify that the GData specification’s syntax will be used for the selectors. Yoga ships with an alternate SelectorParser implementation, LinkedInSelectorParser, which can be used by developers who prefer the LinkedIn API’s selector format.

Finally, we remove the @ResponseBody annotations from our UserController. The @ResponseBody annotations are dependent on MappingJacksonJsonView, which is now replaced with the YogaSpringView.

@RequestMapping("/user/{id}")
    public User get( @PathVariable long id )
    {
        return_userRepository.fetchUser( id );
    }

At this point, the developer can launch the web application, and append the appropriate selectors to the resource requests.

GET /user/1.json?selector=id,name

renders the output:

{
    "name": "Carter Page",
    "id": 1  
}

The developer can add favoriteArtists to the selector:

GET /user/1.json?selector=id,name,favoriteArtists(id,name)

and navigate down the object graph to view instances of the Artist resource:

{
    "id": 1,
    "name": "Carter Page",
    "favoriteArtists": [
          {
                "id": 1,
                "name": "Arcade Fire"           
          },
          { 
                "id": 3,
                "name": "Neutral Milk Hotel"           
          }
     ]
}

Core Fields

In our previous example, let’s assume the id and name fields of User are mandatory and must be returned for every User resource request. These are cheap, small, integral fields, and naming them explicitly each time we create a selector could get verbose.

Yoga provides a @Core annotation that can be applied to your serialized domain model (or DTO) to identify fields that will always be returned on a Yoga request. Here, we annotate the getter methods for our User domain object:

    @Core
    public long getId()
    {
        return _id;
    }
    @Core
    public String getName()
    {
        return _name;
    }

Now, we no longer have to explicitly request id and name in our selector. The following request:

GET /user/1.json?selector=favoriteArtists(id,name)

will return id, name, and anything else specified in the selector:

{
     "id": 1,
     "name": "Carter Page",
     "favoriteArtists": [
           {
                 "id": 1,
                 "name": "Arcade Fire"              
           },
           {
                 "id": 3,
                 "name": "Neutral Milk Hotel"            
           }
     ]
}

Aliases

Iteratively refining your selectors makes for a rapid development/debug cycle. However, when you get to a Production deployment of your API, you may not want external users composing arbitrary selectors to run in your environment. Unrestricted navigation of your object graph can quickly lead to security and performance issues.

Yoga allows you to define aliases for the selectors you want to publish, and only allow users to invoke selectors that have defined aliases. Let’s say we are happy with the following selector and want to publish it:

?selector=id,name,favoriteArtists(id,name)

First, in our Production configuration we will disable the use of explicit selectors, so that users in that environment will not be able to compose selectors using the GData (or LinkedIn) syntax.

    <bean id="selectorParser" class="org.skyscreamer.yoga.selector.parser.GDataSelectorParser"
            p:disableExplicitSelectors="true"/>

Next, we define the alias. Yoga provides several mechanisms for specifying aliases; in this case, we will define them in a properties file.

    <bean id="aliasSelectorResolver" class="org.skyscreamer.yoga.selector.parser.DynamicPropertyResolver"
          p:propertyFile="classpath:selectorAlias.properties"/>

In the properties file, we set up the alias and name it. By convention, Yoga aliases begin with a $:

$userFavoriteArtists=id,name,favoriteArtists(id,name)

Now, in the Production environment, users are able to invoke the alias, whose behavior is defined in our API docs:

GET /user/1.json?selector=$userFavoriteArtists

Conclusion

For many developers, the REST model is sufficient for providing web API access to your application’s domain. If you need more fine-grained control over the structure of your document responses, Yoga will integrate with your existing REST application, and allow you to add selectors to your web requests.

This week, Skyscreamer Software has released version 1.0 of Yoga. Yoga integrates directly with Java applications that run Spring MVC REST, Jersey, or RESTEasy implementations. You can find a live video demonstration of the material in this article at this link.

About The Author

Corby Page has been writing software for cash and fun for 20 years. He focuses on Java solutions through his company ASP Methods, and he collaborates with Carter Page and Solomon Duskis as part of the open source initiative Skyscreamer Software.

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

I think OData is a cleaner approach by Faisal Waris

With OData you can expose a coherent data model to the clients (entities and relationships). The clients can query / update the data in arbitrary ways by understanding the model metadata.

See www.odata.org/

OData is an open standard with support from IBM, SAP, Microsoft, Red Hat, and more.
www.oasis-open.org/committees/tc_home.php?wg_ab...

SAP plans to implement OData directly on top of its HANA platform.

Yoga and OData solve different problems by Carter Page

Microsoft's OData solution is a very credible approach for users who are fluent in the .NET ecosystem.

Yoga is different from OData in a few important ways. OData is a heavy protocol; Yoga is a Java toolkit that adds lightweight semantics to REST. OData is a 310-page specification designed for sophisticated enterprise integration requirements; Yoga is for developers who like the simplicity of REST and want to add some additional flexibility without learning Yet Another Framework.

Yoga could actually implement the relational query subsection of the OData spec if there were enough demand, but it is really targeted at a different type of developer.

using dozer (bean mapper) framework by ali akbar azizkhani

i have performance problem for domain model’s object graph when get json
i am using dozer bean mapper framework and solve this problem
in ui i have viewmodel that declared and mapped from domain model
your solution is not for big business application that have more than 1000 form and domain models

Re: using dozer (bean mapper) framework by Solomon Duskis

Ali,

If I understand you correctly, you're suggesting that enterprise applications with large domain models inherently need data transfer objects a.k.a. DTOs a.k.a. forms (a la Struts). Dozer sure helps in those cases, however...

I would say that I consider DTOs to be an anti-pattern. They cause duplicate work and subtle bugs. IMHO, the ideal is a single model used throughout the various application layers. I was really happy when I moved away from Struts to a Web MVC approach that I was able to use the ORM capabilities to create objects that more resembled the view layer than the database tables. If I really needed to, I could always use a DTO, if I really wanted to, but I only used that approach in extreme situations. The approach happened to be Spring MVC, but the lack of DTOs is not unique to that framework.

REST servers now force you to use DTOs more often than not. I saw that forcing DTOs in REST stacks as a significant step back, which is why I made the argument to stay away from DTOs to Corby and Carter, my fellow Yoga contributors; it wasn't originally intuitive to them, but they agreed after some debate. I believe that the ability to keep a single domain hierarchy and to have different context dependent renderings is an intentional unique feature in java REST implementations. The decision was not an oversight. I'm certainly looking for additional opinions on this.

To the InfoQ team at large: Is DTOs a great pattern to always use in MVC, or is it something to use sparingly?

Re: using dozer (bean mapper) framework by ali akbar azizkhani

see this Martinfowler article (slide 4)martinfowler.com/articles/gap-scms-po/

Re: using dozer (bean mapper) framework by Solomon Duskis

Aziz,

Thanks for the article and the discussion. I have the greatest respect for Mr. Fowler's work

I read through Mr. Fowler's article. I saw that he used DTOs; however I didn't see anything that really described why he used them. Yoga allows you to achieve the same goals with less code by eliminating the need for DTOs. That change reduces application complexity and increases performance.

In addition, Yoga provides an additional enhancement. Mr. Fowler describes a lot of the goals of his approach: martinfowler.com/articles/gap-scms-po/#service which talks about "The web service uses a Resource API." Yoga would allow him to increase his Richardson Maturity Model level to 3. We produce and expose URLs as part of every request about relationships between model objects expressed as URIs.

In theory, our approach should work with morphia/MongoDB in the same way that it works with JPA/Relational objects, since both approaches provide a means of rich java object domain model relationships to map relationships between persisted artifacts. Yoga adds on rich hypermedia controls to those object relationships (i.e. converts a java reference to a URL in the serialization phase).

I certainly don't want to put words into Mr. Fowler's mouth, but I would venture to say that Yoga would enhance Mr. Fowler's approach rather than contradict it.

I would definitely appreciate additional feedback on both the high level goals of Yoga as well as implementation!

Never expose your domain model by Paulo Trecenti

I like things that's make my life more easy, but I think with that solution we can have problems in the future. As Fowler say in his book "Never expose your domain model" there is a reason for that. When I make a change in my domain I do not broken the interface and I never will expose unnecessary things. I like REST, but use REST by that way I think it's dirty, after read a lot of article about REST anti pattern I think that solution is a BIG REST anti-pattern.

Re: Never expose your domain model by Corby Page

Yoga works at the DTO layer (if you don't want to expose your domain model), or at the domain model layer (if you do). In this article, I showed Yoga directly serializing the domain objects because that makes for a simple example.

Yoga will work with whichever architectural approach you have chosen. The rule of thumb is, whatever you are serializing out of your controllers for your REST interface, that is what you will annotate with the Yoga annotations, and you are good to go!

Re: Never expose your domain model by Faisal Waris

In true REST, domain changes are handled 'magically' and we never run into any versioning issues because there is no explicit or implicit contract :)

Re: Never expose your domain model by Solomon Duskis

Is this a criticism of Yoga or REST solutions at large?

Re: Never expose your domain model by Faisal Waris

This is a criticism of using the term "REST" when one is really talking about Web API.

With APIs versioning is important so all this talk of exposing or not exposing the domain model is irrelevant.

By definition API is interface; whether it's based on domain model or DTO is an implementation detail.

In most versioning scenarios backward compatible changes are labeled as minor and breaking changes as major.

Re: using dozer (bean mapper) framework by Mark N

I have to agree with Solomon about DTOs.

Every time someone says or uses DTO an angel loses its wings.

The DTO pattern is mainly misunderstood and is typically not what people are actually doing. DTOs are values of domain entities without all the extra stuff (aka methods). They were used with things like EJBs because, in the past, they were not serializable and you ran into network performance issues is you access them. If your domain entities are serializable, you don't need DTO's. Struts, unfortunately, used something they called DTOs but were not and they also perpetuated the myth that you needed to "separate the layers" with "DTOs". If your "DTO' contains more than values and basic getters/setters, it is not a DTO.

What people typically call DTOs are really anemic domain objects or view/presentation/form objects. I think that (view/presentation) is what Martin is referring to. Sadly he does not show how or why. Where i have run into using these is with something like JavaScript where I am using an UI API that expects to bind to the object in a certain way. I have also had to use them with GWT because certain things that exist in Java cannot be represented in JS. These are effectively domain entities that have been modified or dumbed down.

print preview by pradeep b

I dont see print preview option.

Re: using dozer (bean mapper) framework by Tamjeed Ahamed

I hope we are in the context called large scale. And when this is the context, definitely the decoupled approach is preferred for all obvious reasons.

I put my money on the fact that domain entities should never be exposed to other layers.

Let UI handling (view layer on the server) just confine itself to processing the form data with it's bean. And the end points in view layer requiring services should have a mapper (UI bean to Service Bean) both to and fro, and hence making the system distributed in all aspects viz., pliable, development, design, architecture, deployment and maintainability too.

Most importantly a single responsibility design rule fits aptly here.

I have adopted the same design / architecture in our application's framework, meant to deal Large Scale applications, with all kinds of separation of concerns, with polyglot persistence layer, and most importantly domains preserved under repositories. NOTE: The domain is either drawn as class diagram or imported (in case of reverse engineering).

My UI never ever had issues to point to another system (with a different domain model for the same business), as the UI was only handling it's FormBean and the services now had to point to another system.

FYI; we used ModelMapper; as it was better over Dozer in terms of features and performance.

Data Filtering by Robert Krier

I haven't seen anywhere that Yoga as some neat filtering capabilities. I really like the selector approach to allow the user to control how much of the object graph is returned.

Is there anything that supports filtering? I need to be able to combine selection with attribute filtering, like ((user.age > 30 and user.sex = Female) or user.hairColor = Brown). I know that is a stupid example, but hopefully you get the idea.

Does anyone have a recommendation? Also I don't want to expose our domain model directly either.

Thanks

Doesn't Jackson Filters provide the same functionality by Travis De Silva

We use Jackson FilterProviders to achieve this. What does Yoga provide that cannot be done using the inbuilt Jackson filters?

Our REST API's (or WEB APIs) are developed using pure JAX-RS running on JBoss App server which by default using the ResyEasy as the JAX-RS implementation.

ResyEasy in turn uses Jackson for JSON Marshalling and Unmarshalling.

So why would we want to introduce another Java Library if we can achieve what we want out of the box.

Or have I misunderstood what Yoda does?

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

16 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT