BT

Fetching strategy implementation in a J2EE application using AOP

Posted by Manjunath R Naganna on Mar 30, 2009 |

A typical J2EE application using an O/R Mapping tool is faced with the task of fetching the required data using a minimal set of SQL queries. But this is not always an easy task. By default the O/R Mapping tool loads data on demand unless otherwise instructed to. This behavior known as Lazy Loading ensures that dependents are only loaded as they are specifically requested and hence makes it possible to avoid unnecessary creation of objects. Lazy loading is useful when the dependent components are not used for a business use case we are trying to achieve and solves the issue of unnecessarily loading components.

Typically our business use case knows what data is required. Because of the lazy loading, DB performance reduces when a large number of Select queries are executed, as all the required data for a business case are not fetched at once. This might prove a bottleneck for an application which needs to support a large number of requests (a scalability issue).

Let's look at an example, a business use case that requires a Person with his Address information to be fetched. As the Address component is configured to be lazily loaded more SQL queries are run to fetch the required data i.e. first the Person and then his Address. This increases the communication between the database and the application. This could have been avoided by loading both the Person and the Address component in a single query because we knew that our business use case required both components.

If we start writing business use case specific Fetching-APIs in the DAO/Repository and lower level service layer we need to write different APIs to fetch and populate the same domain object with different set of data. This will bloat the repository Layer or the low level service Layer and will become a maintenance nightmare.

Another issue with lazy fetching is that the database connection has to be retained until all the required data is fetched else the application will throw a lazy loading exception.

Note: The above issue resolution is not solvable if we are eagerly fetching the second level cached data in our query. In case of Hibernate O/R mapping tool if we are using eager fetching for second level cached data it will retrieve that data from database instead of retrieving it from cache even if the data is already available in the second level cache. Hibernate has no solution for the above issue. This in turn indicates that we should never use eager fetching in our query for second-level cached objects.  

An O/R mapping tool which allows us to tune the queries for the objects that are configured to be cached, will read from cache if the object is already cached otherwise fetch it eagerly. This will resolve the above transaction/DB connection issue since cached data is also fetched during query execution instead of allowing it to read on demand (i.e. lazy loading)

Let us take a sample code to see the issues faced using lazy loading and a solution to overcome it. Let us consider a domain with three entities Employee, Department and Dependent.

Relationships among these entities are:

  • Employee has zero or more dependents.
  • Department has zero or more employees.
  • Employee belongs to zero or one department.

We take three uses cases to execute:

  1. Fetch employee details.
  2. Fetch employee with dependent details.
  3. Fetch employee with department details.

The above use cases require different data to be fetched and rendered. Using lazy loading we have following disadvantages:

  • If we use lazy loading feature on employee entity for dependent and department entities then in use case 2 and 3, more SQL queries are fired to fetch the required data.
  • Database connection needs to be maintained for the duration of the multiple query cycle else a lazy loading exception is thrown, resulting in inadequate use of resources.

On the other hand using Eager-fetching has the following disadvantages.

  • Eager fetching of dependents and department entities for an employee will result in fetching unnecessary data.
  • Queries cannot be tuned for a specific use case.

Addressing the above issues, using use case specific APIs in the Repository/DAO or lower level service layer will result in:

  • Code bloat - of both Service and Repository/DAO classes.
  • Maintenance nightmare - any new use case for both service and Repository/DAO Layers, new APIs need to be added.
  • Code duplication - If some business logic needs to be applied on a retrieved entity on the lower level service layer. Similarly in DAO/Repository layer query response needs to be checked for data availability before returning the data...

In order to overcome the above conundrum, the Repository/DAO layer needs to be aware of the query that needs to be executed to retrieve the entity based on the use case. To do this the default fetching pattern in the Repository/DAO class is overridden by a different fetching pattern based on the specific use case as defined in the Aspect class. All fetching pattern classes implement the same interface.

a

 

 



The repository class uses the above fetching pattern to retrieve the query that needs to be executed as shown in the sample code below:

public Employee findEmployeeById(int employeeId) {
	List employee = hibernateTemplate.find(fetchingStrategy.queryEmployeeById(),
		new   Integer(employeeId));
  if(employee.size() == 0)
  return null;
  return (Employee)employee.get(0);
}

The employee fetching strategy in the repository class needs to be changed based on the use case required. Decision about changing fetching strategy in repository layer is kept outside the repository and service layers in an aspect class, hence adding any new business use cases just needs modification only at aspect and one more fetching strategy implementation for the repository’s use. Here we use Aspect Oriented Programming for deciding which fetching strategy needs to used on business use case basis.

So what is Aspect Oriented Programming?

Aspect Oriented Programming (AOP) enables modularized implementation of crosscutting concerns that abound in practice: logging, tracing, dynamic profiling, error handling, service-level agreement, policy enforcement, pooling, caching, concurrency control, security, transaction management, business rules, and so forth. Traditional implementation of these concerns requires you to fuse their implementation with the core concern of a module. With AOP, you can implement each of the concerns in a separate module called aspect. The result of such modular implementation is simplified design, improved understandability, improved quality, reduced time to market, and expedited response to system requirement changes.

Then reader may refer AspectJ in Action by Ramnivas Laddad for a detailed treatment of the AspectJ concepts and programming and AspectJ Development Tools for aspectj tools.

Aspect plays a very important role in fetching strategy implementation. Fetching strategy is a business level cross cutting concern and it changes based on the business use case. Aspect helps in deciding which fetching strategy needs to be used in a particular business use case. Here fetching strategy decision management is kept out side the lower level service or repository layer. As any new business use case might require a different fetching strategy and can be applied with out modifying the lower level service or repository API.

FetchingStrategyAspect.aj

/**
     Identify the getEmployeeWithDepartmentDetails flow where you need to change the fetching 
     strategy at repository level 
*/
pointcut empWithDepartmentDetail(): call(* EmployeeRepository.findEmployeeById(int))
&& cflow(execution(* EmployeeDetailsService.getEmployeeWithDepartmentDetails(int)));

/**
    When you are at the specified poincut before continuing further update the fetchingStrategy in   
    EmployeeRepositoryImpl to EmployeeWithDepartmentFetchingStrategy
*/
before(EmployeeRepositoryImpl r): empWithDepartmentDetail() && target(r) { 
r.fetchingStrategy = new EmployeeWithDepartmentFetchingStrategy(); 
}

/** 
   Identify the getEmployeeWithDependentDetails flow where you need to change the fetching 
   staratergy at repository level 
*/
pointcut empWithDependentDetail(): call(* EmployeeRepository.findEmployeeById(int))
&& cflow(execution(* EmployeeDetailsService.getEmployeeWithDependentDetails(int)));

/** 
   When you are at the specified poincut before continuing further update the fetchingStrategy in 
    EmployeeRepositoryImpl to EmployeeWithDependentFetchingStrategy 
*/
before(EmployeeRepositoryImpl r): empWithDependentDetail() && target(r) { 
r.fetchingStrategy = new EmployeeWithDependentFetchingStrategy(); 
}

Thus  decisions about the particular query that needs to be executed by the repository is kept out side the service and repository layers, such that new use cases do not require modification to the lower level Service or Repository layers. The logic of deciding which query needs to be executed is a crosscutting concern that is retained in an Aspect. Aspect decides which fetching strategy needs to be injected in the repository before the service layer executes the API on the repository based on business use case. By this we can use the same service and repository layer APIs to satisfy different business use case requirements.

Let us discuss this with an example business use case fetching both Department and Dependent details of an employee. Here we need to make changes at our business service layer by adding the use case getEmployeeWithDepartmentAndDependentsDetails(int employeeId). Implement the new Fetching strategy class EmployeeWithDepartmentAndDependentFetchingStaratergy that implements EmployeeFetchingStrategy and overrides the API queryEmployeeById that returns the optimized query that helps in fetching the required data in one shot.

The decision of injecting the above fetching strategy for the required business use case is placed in aspect as shown below.

pointcut empWithDependentAndDepartmentDetail(): call(* EmployeeRepository.findEmployeeById(int))
&& cflow(execution(* EmployeeDetailsService.getEmployeeWithDepartmentAndDependentsDetails(int)));

before(EmployeeRepositoryImpl r): empWithDependentAndDepartmentDetail() && target(r) { 
     r.fetchingStrategy = new EmployeeWithDepartmentAndDependentFetchingStaratergy(); 
}

As you can see we haven't modified our lower level service or repository layers to achieve the above new use case. We used aspect and a new FetchingStrategy implementation to achieve our new business use case.

Now let us discuss the issue with the query optimization for the objects which are configured for second level cache. In our example code let us modify department entity to be configured in the second level cache. If we try to eager fetch the department entity then for every fetch of same department the database is hit for department information even though it is available in our second level cache. If we don't fetch the department entity in our query then our Business layer (use case layer) will participate in the transaction because the department entity is not cached and it will be fetched by lazy loading.

Thus transaction demarcation is moved to the business layer from lower layer even though we know what data is needed for our business use case, but only if the O/R mapping tool doesn’t provide a mechanism to solve the above issue I.e. eager fetching of cached data.

This approach works well for all non-cached data items but for cached data items it depends on the O/R mapping tool which resolves the cached data issue.

See the attached source code for complete working example illustrating the fetching strategy. The zip file contains a working example illustrating all the scenarios discussed above. You can use any IDE or run it from command prompt using aspectj compiler to execute and test the attached source code. Before executing make sure you have edited jdbc.properties and created required tables for demo application.

You can try it using the Eclipse IDE and AJDT plugin by following these steps:

  1. Unzip the downloaded source code and Import the project into eclipse.
  2. Configure the database configuration in jdbc.properties file under Resources/dbscript.
  3. Execute the resources\dbscript\tables.sql on the database configured above which creates the tables required for the demo application.
  4. Run the Main.java as AspectJ/Java application to create default data and test the above fetching strategy implementation.

Conclusion

This article showed how a fetching strategy can optimize the data retrieval process from a back end system on use case basis in a modular way without bloating your lower level service or repository layers.

Bio: Manjunath R Naganna is working in Alcatel Lucent as Senior Software Engineer with a specialization in the design and implementation of enterprise applications using Java/J2EE. He is mostly interested in Spring framework, Domain Driven Design, Event Driven Architecture and Aspect Oriented Programming. Manjunath would like to thank Hemraj Rao Surlu for editing and formatting of the content. He would also like to thank his leads Ramana and Saurabh Sharma for reviewing the content and providing important feedback.

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

So what for is repository? by Sławomir Sobótka

In this approach the repository is just for executing strategy. For me it's not enough justification for existence of repository class:/
There is also additional code that checks list details (size) but of course this clumsy design (returning from strategy list instead of object) is just to mask odd idea i guess.

Sounded good... by Branden Root

The first part of the article sounded good - I can attest to having a repository's code size explode as more and more specific use case queries are added. But it seemed like the article ran out of steam when it came to the execution of the AOP strategy. Maybe flush it out with more concrete examples?

Is AOP used inappropriately here? by Wei Wei

It is a brilliant idea to inject different fetching strategies during the run time by taking advantages of AOP. But to me, aspects are not use cases specific. My understanding is that they are meant to address more general concerns in the application, e.g. logging.

Refactored Bloat by bruce b


  • Code bloat - the only bloat is compared to the naive eager or lazy approaches. The use case specific code still exists in the AOP solution.

  • Maintenance nightmare -Splitting the implementation of ORM fetching between inline code and aspect driven code is the real maintenance nightmare. The concerns implemented with AOP should be orthogonal to the use case driven service/repository code. This isn't the case here.

  • Code duplication - The same duplication occurs in the sibling FetchingStrategy impl classes

    Missing the role interface by Udi Dahan

    I did something like this two years ago in .NET and wrote it up here:

    www.udidahan.com/2007/04/23/fetching-strategy-d...

    A year later is was rolled into the open source Rhino suite Ayende Rahien is leading as written about here:

    ayende.com/Blog/archive/2008/03/27/Adaptive-Dom...

    The main thing necessary for this to work is for there to be use-case-specific role interfaces for the domain which the service layer uses, rather than using the specific domain classes themselves.

    This allows us to hook the fetching strategy on the use case, rather than on any implementation detail.

    Hope that helps.

    Re: Is AOP used inappropriately here? by Bill Poitras

    Although AOP has traditionally used to address infrastructure cross cutting concerns like logging and transactions, its meant to address all cross-cutting concerns. I saw an AOP talk that gave a good example of using AOP for application functionality. I think in this case it was exception mapping between a service and its caller.

    Re: Missing the role interface by Manjunath Naganna

    Hi Udi,

    I have a question here regarding how do you hook-up the use-case specific role interface to the service layer at run-time?

    As you see currently from my example code i am hooking it up using aspect. Can you give me a code snippet that explains it?

    -Manjunath

    Re: Refactored Bloat by Manjunath Naganna

    Code Bloat-
    If we start writing use case specific fetching api's we will have lot more code bloating then here since here aspect is handling of injecting the actual query that needs to be executed based on use case.

    And if there is any common processing needs to be done after fetching domain object at repository level then this code needs to be present at all the use case specific fetching api's.

    Maintenance Nightmare - Regarding this how come this becomes more maintenance nightmare than having multiple fetching use case specific api's at repository and lowel level service api's. You can check my sample code and evaluate it by adding new use case specific api needs changes at Business level service interface and aspect than changes to the lower level layers.

    Code duplication - Here i don't see any code duplication since each FetchingStratergyImpl will have fine tuned queries specific to use case.

    -Manjunath

    Re: Missing the role interface by Udi Dahan

    The service layer specifically asks for the role interface. Rather than:

    Employee e = ORM.Get<Employeee>(id);
    e.Fire();

    It calls:

    IAmGoingToFireEmployee e = ORM.Get<IAmGoingToFireEmployee>(id);
    e.Fire();

    Then the ORM internally uses a service locator to find an implementation of IFetchingStrategy<IAmGoingToFireEmployee>. If found, the ORM uses it to find out which other objects to eagerly load.

    Does that make sense?

    Re: Missing the role interface by Manjunath Naganna

    Hi Udi,

    I still have a bit of confusion regarding missing role interface. I have gone through your talk Making Roles Explicit on InfoQ where you mention about role interface.

    But as part of my example code i have Business Layer above Service layer to retrieve different set information of information to be displayed based on use case how do i notify my repository that i need to use particular fetching strategy. How do i pass this information from lower level service api's to repository.

    For ex:-

    Business Layer has a service which needs to run some Specification(Business Rule validation) on the retrieved entity

    <Pseudo-Code>

    BusinessLayer

    void makeHimAValuableEmployee(int employeeId) {
    Employee emp = lowerLevelServiceLayer.findEmplyee(employeeId);
    ISpecification.isValuableEmployee(emp);
    //Some more operations on domain object.
    }

    <Pseudo-Code>

    Can you just go through my source code. Can your provide me a pseudo-code how would you have achieved it by making role explicit?

    -Manjunath

    </pseudo-code></pseudo-code>

    Re: Missing the role interface by Udi Dahan

    Manjunath,

    Apparently there are more architectural differences in our approaches than was originally visible. In your case, your business layer would call:

    lowerLevelServiceLayer.MakeHimAValuableEmployee(employeeId);

    And the lower level service would call:

    IWantToMakeEmployeeValuable e = repository.Get<IWantToMakeEmployeeValuable>(employeeId);
    e.MakeValuable();

    Is that any clearer?

    Re: Refactored Bloat by bruce b

    Sorry for the delayed response, just got back from vacation. Downloaded the code and see that that each fetching strategy is a one liner returning the HQL for a use case. So for each entity that may have multiple fetching options, you need to define an interface, implement the interface for each fetching option, then you have to define pointcuts to intercept calls to determine the correct fetching option. That's a lot of boilerplate to support a simple lookup algorithm for a database query.

    As I'm writing this, I realize it's not a fetching strategy, so much as a fetching option. Options can usually be managed using low-tech means such as method parameters. The repository method could accept a parameter passed by the service method indicating which fetch mode to use when fetching an entity and its children. At this point mapping the option parameter to the HQL statement is a straightforward lookup that can be implemented in numerous ways. If a new use case arises requiring a new fetch mode, all you have to do is add a new name => HQL mapping to whatever lookup mechanism is in use.

    If a need arises to execute common code after a fetch, then introducing AOP might be warranted, but only if it leads to less inline code boilerplate and duplication.

    Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

    Email me replies to any of my messages in this thread

    Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

    Email me replies to any of my messages in this thread

    12 Discuss

    Educational Content

    General Feedback
    Bugs
    Advertising
    Editorial
    InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
    Privacy policy
    BT