BT

Using AWS Cloud Search

Posted by Boris Lublinsky on Dec 11, 2012 |

Many of today’s applications rely heavily on their search functionality. The range of such applications spans everything from the e-commerce web site, leveraging search for finding appropriate products, to social networks, leveraging search to find people, to mapping sites, leveraging search for finding POIs and addresses.

Amazon’s new CloudSearch service provides a viable alternative to  custom search implementations or using custom installations of such popular products as Apache Lucene, Apache Solr or elasticsearch. They describe it as:

“a fully-managed search service in the cloud that allows customers to easily integrate fast and highly scalable search functionality into their applications.[It] enables customers to offload the administrative burden of operating and scaling a search platform. Customers don't have to worry about hardware provisioning, data partitioning, or software patches.”

The only drawback that we have found in this implementation is an absence of Java APIs for uploading data and implementing search. Although Amazon provides REST APIs that can be used for these purposes, using them from Java code is not always easy. In this article we will present simple Java APIs, which we have developed to simplify invocation of search and data upload functionality from Java.

Data definition

Although Amazon provides data definitions for data uploads in both JSON and XML and Search Response (again both XML and JSON), data upload documentation specifies only a Relax NG schema, while search response does not specify any schema at all.

For our implementation we decided to use XML data formats rather than the JSON one, due to the simpler implementation of data marshaling – XML uses a consistent data format, whilst JSON is based on a dynamic one (JSON tags are defined dynamically and change from request to request). We are using the following two schemas (Listing 1 - Listing 2) for data upload and search results respectively.

<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"  elementFormDefault="qualified">
         ………………………………………………………………………………………….
	 <xsd:complexType name="fieldType">
	         <xsd:simpleContent>
	                 <xsd:extension base="xsd:string">
 	                           <xsd:attribute name="name" type="field_nameType" />
	                 </xsd:extension>
	         </xsd:simpleContent>
	</xsd:complexType>

        <xsd:complexType name="addType">
	         <xsd:sequence>
	                 <xsd:element name="field" type="fieldType" maxOccurs="unbounded" />
	         </xsd:sequence>
	         <xsd:attribute name="id" type="IDType" />
	         <xsd:attribute name="version" type="versionType" />
	         <xsd:attribute name="lang" type="xsd:language" />
        </xsd:complexType>

	<xsd:complexType name="deleteType">
	         <xsd:attribute name="id" type="IDType" />
	         <xsd:attribute name="version" type="versionType" />
	</xsd:complexType>

	<xsd:complexType name="batchType">
	         <xsd:sequence>
	                 <xsd:element name="add" type="addType" minOccurs="0" maxOccurs="unbounded" />
	                 <xsd:element name="delete" type="deleteType" minOccurs="0" maxOccurs="unbounded" />
	         </xsd:sequence>
	</xsd:complexType>

	<xsd:element name="batch" type="batchType" />

	<xsd:simpleType name="statusType">
	         <xsd:restriction base="xsd:string">
	                 <xsd:enumeration value="success"/>
	                 <xsd:enumeration value="error" />
	         </xsd:restriction>
</xsd:simpleType>
<xsd:complexType name="errorsType">          <xsd:sequence>              <xsd:element name="error" type="xsd:string" maxOccurs="unbounded" />          </xsd:sequence>
</xsd:complexType>
<xsd:complexType name="warningsType">          <xsd:sequence>              <xsd:element name="warning" type="xsd:string" maxOccurs="unbounded" />          </xsd:sequence>
</xsd:complexType>
     <xsd:complexType name="responseType">          <xsd:sequence>
<xsd:element
name="errors" type="errorsType" minOccurs="0" />            <xsd:element name="warnings" type="warningsType" minOccurs="0" />          </xsd:sequence>
<xsd:attribute
name="status" type="statusType"/>          <xsd:attribute name="adds" type="xsd:int"/>          <xsd:attribute name="deletes" type="xsd:int"/> </xsd:complexType>

<xsd:element
name="response" type="responseType" /> </xsd:schema>       Listing 1 Upload data schema <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" targetNamespace="http://cloudsearch.amazonaws.com/2011-02-01/results" xmlns="http://cloudsearch.amazonaws.com/2011-02-01/results" elementFormDefault="qualified"> <xsd:complexType name="constraintType"> <xsd:attribute name="value" type="xsd:string"/> <xsd:attribute name="count" type="xsd:int"/> </xsd:complexType> <xsd:complexType name="facetType"> <xsd:sequence>
<xsd:element name="constraint" type="constraintType" maxOccurs="unbounded"/> </xsd:sequence>
<xsd:attribute
name="name" type="xsd:string" /> </xsd:complexType>
<xsd:complexType
name="facetsType"> <xsd:sequence>
        
<xsd:element name="facet" type="facetType" maxOccurs="unbounded"/> </xsd:sequence>
</xsd:complexType>
<xsd:complexType name="infoType"> <xsd:attribute name="rid" type="xsd:string" /> <xsd:attribute name="time-ms" type="xsd:int" /> <xsd:attribute name="cpu-time-ms" type="xsd:int" /> </xsd:complexType> <xsd:complexType name="dType">        <xsd:simpleContent>
               
<xsd:extension base="xsd:string">                  <xsd:attribute name="name" type="xsd:string" />              </xsd:extension>
      
  </xsd:simpleContent>
</xsd:complexType>

<xsd:complexType
name="hitType"> <xsd:sequence>
<xsd:element name="d" type="dType" maxOccurs="unbounded"/> </xsd:sequence>
<xsd:attribute
name="id" type="xsd:string" /> </xsd:complexType> <xsd:complexType name="hitsType"> <xsd:sequence>
         
<xsd:element name="hit" type="hitType" maxOccurs="unbounded"/> </xsd:sequence>
<xsd:attribute
name="found" type="xsd:int" /> <xsd:attribute name="start" type="xsd:int" />
</xsd:complexType>
<xsd:complexType name="resultsType">
<xsd:sequence>
         
<xsd:element name="rank" type="xsd:string" />      <xsd:element name="match-expr" type="xsd:string" />
             <xsd:element
name="hits" type="hitsType" minOccurs="0"/>
             <xsd:element
name="facets" type="facetsType" minOccurs="0"/>              <xsd:element name="info" type="infoType" />
</xsd:sequence>
</xsd:complexType>
  <xsd:element name="results" type="resultsType"/> <xsd:complexType name="messageType">
         <xsd:attribute
name="severity" type="xsd:string" />
         <xsd:attribute
name="code" type="xsd:string" />
         <xsd:attribute
name="message" type="xsd:string"/>
</xsd:complexType>
<xsd:complexType name="errorType">
         <xsd:sequence>
         
<xsd:element name="error" type="xsd:string" />
             <xsd:element
name="rid" type="xsd:string" />
             <xsd:element
name="time-ms" type="xsd:int" />
           <xsd:element
name="cpu-time-ms" type="xsd:int" />          <xsd:element name="messages" type="messageType" maxOccurs="unbounded" /> </xsd:sequence> </xsd:complexType>

<xsd:element
name="error" type="errorType" /> </xsd:schema> Listing 2 Search results data schema

With these two schemas in place, we used the xjc binding compiler to generate Java classes that can be automatically marshaled/unmarshaled using Java Architecture for XML Binding (JAXB).

Query definition

With the data definition in place, the next component that is required to implement the search APIs is a query definition. We have created a set of classes, implementing Amazon’s query definition.

At the heart of the search query is a filter. We have introduced SearchQueryFilter interface for which we have two implementations – Search Query Value Filter (Listing 3) and Search Query Filter Operation (Listing 4).

public class SearchQueryValueFilter implements SearchQueryFilter{

    private String _field;
    private String _value;
    private boolean _isExclude;
    private boolean _isNumeric;
	  
    public SearchQueryValueFilter(){}

    public SearchQueryValueFilter(String field, String value, boolean isNumeric, boolean isExclude){
        _field = field;
        _value = value;
        _isExclude = isExclude;
	_isNumeric = isNumeric;
    }

	    public String getField() {
	        return _field;
	    }

	    public void setField(String field) {
	        _field = field;
	    }

	    public String getValue() {
	        return _value;
	    }

	    public void setValue(String value) {
	        _value = value;
	    }

	    public boolean isExclude() {
	        return _isExclude;
	    }

	    public void setExclude(boolean isExclude) {
	        _isExclude = isExclude;
	    }

	    public boolean isNumeric() {
	        return _isNumeric;
	    }

	    public void setNumeric(boolean isNumeric) {
	        _isNumeric = isNumeric;
	    }

	    @Override
	    public String toString(){
	        StringBuffer sb = new StringBuffer();
	        if(_isExclude){
	            sb.append("(not ");
	        }
	        if(_field != null){
	            sb.append(_field);
	            sb.append(":");
	        }
	        if(!_isNumeric){
	            sb.append("'");        
	        }
	        sb.append(_value);
	        if(!_isNumeric){
	            sb.append("'");        
	        }
	        if(_isExclude){
	            sb.append(")");
	        }
	        return sb.toString();
        }
}

Listing 3 Value filter implementation

public class SearchQueryFilterOperation implements SearchQueryFilter {
	    
	List<SearchQueryFilter> _filters;
	FilterOperation _operation;
	    
	public SearchQueryFilterOperation(){
	        _operation = FilterOperation.and;
	        _filters = new LinkedList<SearchQueryFilter>();
	}    
	    
	public List<SearchQueryFilter> getFilters() {
	        return _filters;
	}

	public void setFilters(List<SearchQueryFilter> filters) {
	        _filters = filters;
	}

	public void addFilters(SearchQueryFilter filter) {
	        _filters.add(filter);
	}

	public FilterOperation getOperation() {
	        return _operation;
	}

	public void setOperation(FilterOperation operation) {
	        _operation = operation;
	}

        @Override
	public String toString() {
	        StringBuffer sb = new StringBuffer();
	        sb.append("(");
	        sb.append(_operation);
	        for(SearchQueryFilter f : _filters){
	                 sb.append(" ");
	                 sb.append(f);
	        }
	        sb.append(")");
	        return sb.toString();
	}

	public enum FilterOperation{
	        and, or
	}
}

Listing 4 Operation filter implementation

Search Query Value Filter allows a developer to set limits on an individual field using operators including equal to, less than, greater than and between (it also supports negative comparison).  Search Query Filter Operation also lets a developer  combine several Search Query Value Filters and Search Query Filter Operations using AND/OR operators. The combination of these two classes allows the expression of arbitrary query filters supported by Amazon CloudSearch.

Amazon CloudSearch supports  faceted classification:

“A faceted classification system allows the assignment of an object to multiple characteristics (attributes), enabling the classification to be ordered in multiple ways, rather than in a single, predetermined, taxonomic order. A facet comprises "clearly defined, mutually exclusive, and collectively exhaustive aspects, properties or characteristics of a class or specific subject".[1] For example, a collection of books might be classified using an author facet, a subject facet, a date facet, etc.”

Faceted classification is used in faceted search systems that enable a user to navigate information along multiple paths corresponding to different orderings of the facets.

AWS allows the use of facets for controlling the search execution, and sorting the search results. It also allows a developer to control how many facets to return with search results. All facet operations are implemented by a single class Search Query Facet (Listing 5).

public class SearchQueryFacet {
	    
          private String _name;
	  private int _maxFacets;
	  private List<String> _constraints;
          private FacetSort _sort;
	    
	  public SearchQueryFacet(String name){
	           _name = name;
	           _maxFacets = -1;
	           _constraints = null;
	           _sort = FacetSort.none;
	  }

	  public SearchQueryFacet(String name, int maxFacets){
	           _name = name;
	           _maxFacets = maxFacets;
	           _constraints = null;
	           _sort = FacetSort.none;
	  }

	  public SearchQueryFacet(String name, int maxFacets, FacetSort sort){
	           _name = name;
	           _maxFacets = maxFacets;
	           _constraints = null;
	           _sort = sort;
	  }

	  public SearchQueryFacet(String name, int maxFacets, List<String> constraints){
	           _name = name;
	           _maxFacets = maxFacets;
	           _constraints = constraints;
	           _sort = FacetSort.none;
	  }

	  public SearchQueryFacet(String name, List<String> constraints){
	           _name = name;
	           _maxFacets = -1;
	           _constraints = constraints;
	           _sort = FacetSort.none;
	  }

	  public SearchQueryFacet(String name, FacetSort sort, List<String> constraints){
	           _name = name;
	           _maxFacets = -1;
	           _constraints = constraints;
	           _sort = sort;
	  }

	  public SearchQueryFacet(String name, FacetSort sort){
	           _name = name;
	           _maxFacets = -1;
	           _constraints = null;
	           _sort = sort;
	  }

	  public String getName() {
	           return _name;
	  }

	  public void setName(String name) {
	           _name = name;
	  }

	  public int getMaxFacets() {
	           return _maxFacets;
	  }

	  public void setMaxFacets(int maxFacets) {
	           _maxFacets = maxFacets;
	  }

	  public FacetSort getSort() {
	           return _sort;
	  }

	  public void setSort(FacetSort sort) {
	           _sort = sort;
	  }

	  public int get_maxFacets() {
	           return _maxFacets;
	  }

	  public void set_maxFacets(int _maxFacets) {
	           this._maxFacets = _maxFacets;
	  }

	  public List<String> getConstraints() {
	           return _constraints;
	  }

	  public void setConstraints(List<String> constraints) {
	           _constraints = constraints;
	  }

	  public void addConstraint(String constraint) {
	           if(_constraints == null)
	                      _constraints = new LinkedList<String>();
	           _constraints.add(constraint);
	  }

	  @Override
	  public String toString(){
	           StringBuffer sb = new StringBuffer();
	           sb.append("&facet=");
	           sb.append(_name);
	           if(_maxFacets > 0){
	                      sb.append("&facet-");
	                      sb.append(_name);
	                      sb.append("-top-n=");
	                      sb.append(_maxFacets);
	           }
	           if((_constraints != null) && (_constraints.size() > 0)){
	                      sb.append("&facet-");
	                      sb.append(_name);
	                      sb.append("-constraints=");
	                      boolean first = true;
	                      for(String c : _constraints){
	                                if(!first)
	                                           sb.append("%2C");
	                                else
	                                           first = false;
	                                sb.append("%27");
	                                sb.append(c);
	                                sb.append("%27");
	                     }
	           }
	           if(!_sort.equals(FacetSort.none)){
	                      sb.append("&facet-");
	                      sb.append(_name);
	                      sb.append("-sort=");
	                      sb.append(_sort);
	           }
	        
	           return sb.toString();
	  }
	    
	  public enum FacetSort{
	           none, alpha, count, max, sum
	  }
}
Listing 5 Facets control class

Finally Search Query Sort class (Listing 6) allows a developer to control results sorting.

public class SearchQuerySort {
	  
	  private List<SearchRank> _ranks;
	    
	  public SearchQuerySort(){
	           _ranks = new LinkedList<SearchRank>();
	  }
	   
	  public void addRank(SearchRank rank){
	           _ranks.add(rank);
	  }
	  @Override
	  public String toString(){
	           if(_ranks.size() == 0)
	                      return null;
	           StringBuffer sb = new StringBuffer();
	           sb.append("&rank=");
	           boolean first = true;
	           for(SearchRank r : _ranks){
	                      if(!first)
	                              sb.append("%2C");
	                      else
	                              first = false;
	                      sb.append(r);
	           }
	           return sb.toString();
	  }

	  public static class SearchRank{
	           private String _name;
	           private boolean _ascending;
	        
	           public SearchRank(){
	                      _ascending = true;
	           }
	           public SearchRank(String name){
	                      _ascending = true;
	                      _name = name;
	           }
	           public SearchRank(String name, boolean ascending){
	                      _ascending = ascending;
	                      _name = name;
	           }
	        
	           @Override
	           public String toString(){
	                      if(_ascending)
	                                return _name;
	                      return "-" + _name;
	           }
	  }
}

Listing 6 Sort control class

All these parameters are brought together by a CloudSearch query, which also adds pagination information and a set of returned fields.

The query class also provides a method - convert to HTTP Query (Listing 7), that brings all the components of the Search query together and generates an appropriate HTTP string that can be processed by search.

	public String toHttpQuery() throws Exception{
	        StringBuffer sb = new StringBuffer();
	        sb.append("?results-type=xml");
	        if(_size > 0){
	                   sb.append("&size=");
	                   sb.append(_size);
	        }

	        if(_start > 0){
	                   sb.append("&start=");
	                   sb.append(_start);
	        }

	        if((_fields != null) && (_fields.size() > 0)){
	                   sb.append("&return-fields=");
	                   boolean first = true;
	                   for(String f : _fields){
	                              if(!first)
	                                          sb.append("%2C");
	                              else
	                                          first = false;
	                              sb.append(f);
	                   }
	        }

	        if(_filter != null){
	                   if(_filter instanceof SearchQueryValueFilter)
	                                          sb.append("&q=");
	                   else
	                                          sb.append("&bq=");
	                   sb.append(URLEncoder.encode(_filter.toString(), "UTF8"));
	        }

	        if((_facets != null) && (_facets.size() > 0)){
	                   for(SearchQueryFacet f : _facets){
	                                          sb.append(f);
	                   }
	        }

	        if((_sorts != null) && (_sorts.size() > 0)){
	                   for(SearchQuerySort s : _sorts){
	                                          sb.append(s);
	                   }
	        }

	        return sb.toString();
	    }

Listing 7 Convert to HTTP query method

We used Apache HttpComponents for implementing the actual communications with Amazon CloudSearch endpoint.

Testing our implementation

For our testing we were using the IMDB example provided by Amazon. The first unit test (Listing 8) exercises our search API implementation.

public class SearchAPITester extends TestCase {

	    private static final String SearchURL = "search-imdb-movies-ab4fpqw4eocczpgsnrtlu4rn7i.us-east-
1.cloudsearch.amazonaws.com"
;     private CloudSearchClient client; protected void setUp() throws Exception {         client = new CloudSearchClient(SearchURL);     }     protected void tearDown() {         client.close();     }     public void testSearch() throws Exception{         SearchQueryValueFilter f1 = new SearchQueryValueFilter("title", "star", false, false);         SearchQueryValueFilter f11 = new SearchQueryValueFilter("title", "war", false, true);         SearchQueryValueFilter f2 = new SearchQueryValueFilter("year", "..2000", true, false);         SearchQueryFilterOperation f12 = new SearchQueryFilterOperation();         f12.setOperation(FilterOperation.or);         f12.addFilters(f1);         f12.addFilters(f11);                 SearchQueryFilterOperation f3 = new SearchQueryFilterOperation();         f3.addFilters(f12);         f3.addFilters(f2);                  CloudSearchQuery query = new CloudSearchQuery(f3);         query.addField("actor");         query.addField("director");         query.addField("title");         query.addField("year");                  SearchQueryFacet sf = new SearchQueryFacet("genre", 5, FacetSort.alpha);         sf.addConstraint("Drama");         sf.addConstraint("Sci-Fi");         query.addFacet(sf);                  SearchQuerySort sort = new SearchQuerySort();         SearchRank r1 = new SearchRank("title");         SearchRank r2 = new SearchRank("year", false);         sort.addRank(r1);         sort.addRank(r2);         query.addSort(sort);                  try {             System.out.println("Test 1 ");             SearchResults result = client.search(query);             System.out.println(result);         } catch (Exception e) {             e.printStackTrace();         }     } } Listing 8 Search API tester

The results of this test (Listing 9), correspond to the results that we got by using Amazon REST APIs directly.

SearchResults
[ID=6ddcaa561c05c4cc3dae0f2d67b89419fbfea467ac6292b612dfb3a4a547692c6bea0194d6d37630b171b100197578dc, hitcount=1942, start=0, expression=(and (or title:'star' (not title:'war')) year:..2000), execution time=35ms, cpu execution time=0ms
Hit [ID=tt0092493, values={title:['Crocodile' Dundee II]year:[1988]actor:[Blinco, Maggie,Dingo, Ernie,Hogan, Paul,Holt, Jim,Kozlowski, Linda,Meillon, John,Mercurio, Gus,Rackman, Steve,Scavone, Anthony,Skilton, Gerry,Wilson, Alec]director:[Cornell, John]}]
Hit [ID=tt0078718, values={title:[...And Justice for All.]year:[1979]actor:[Bryggman, Larry,Christian, Robert,Forsythe, John,Lahti, Christine,Levene, Sam,Pacino, Al,Strasberg, Lee,Tambor, Jeffrey,Waites, Thomas G.,Warden, Jack,Williams, Jonathan]director:[Jewison, Norman]}]
Hit [ID=tt0078721, values={title:[10]year:[1979]actor:[Andrews, Julie,Crosby, Denise,Daly, Rad,Dennehy, Brian,Derek, Bo,Haven, Annette,Jones, Sam J.,LeMay, Dorothy,Money, Constance,Moore, Dudley,Royalle, Candida,Serena,Showalter, Max,Volz, Nedra,Wallace, Dee,Webber, Robert]director:[Edwards, Blake]}]
Hit [ID=tt0147800, values={title:[10 Things I Hate About You]year:[1999]actor:[Babin, Michelle,Bennett, Tim,Blake, Shelsie,Gordon-Levitt, Joseph,Junger, Gil,Keegan, Andrew,Kountz, Daniel,Krumholtz, David,Ledger, Heath,Magnuson, Katy,Matthews, Amber,Miller, Larry,Mitchell, Daryl,O'Neill, Bridget,Oleynik, Larisa,Pratt, Susan May,Snider, Tommy,Stiles, Julia,Union, Gabrielle,Zorich, Jay]director:[Junger, Gil]}]
Hit [ID=tt0214388, values={title:[100 Girls]year:[2000]actor:[Billman, Ange,Chriqui, Emmanuelle,DeBello, James,Graham, Aimee,Grant, Tanisha,Green, Johnny,Heigl, Katherine,Hiraizumi, Gina,Musiala, Agnieszka,Oleynik, Larisa,Pressly, Jaime,Ribisi, Marissa,Tucker, Jonathan]director:[Davis, Michael]}]
Hit [ID=tt0115433, values={title:[101 Dalmatians]year:[1996]actor:[Close, Glenn,Daniels, Jeff,Fielder, Harry,Fraser, Hugh,Laurie, Hugh,McInnerny, Tim,Mullard, Arthur,Plowright, Joan,Richardson, Joely,Richardson, Laurence,Shrapnel, John,Weiss, Zohren,Welker, Frank,Williams, Mark]director:[Herek, Stephen]}]
Hit [ID=tt0050083, values={title:[12 Angry Men]year:[1957]actor:[Balsam, Martin,Begley, Ed,Binns, Edward,Bond, Rudy,Cobb, Lee J.,Fiedler, John,Fonda, Henry,Kelly, James,Klugman, Jack,Marshall, E.G.,Nelson, Billy,Savoca, John,Sweeney, Joseph,Warden, Jack]director:[Lumet, Sidney]}]
Hit [ID=tt0103594, values={title:[1492: Conquest of Paradise]year:[1992]actor:[Assante, Armand,Dean, Loren,Depardieu, Gérard,Dunn, Kevin,Karyo, Tchéky,Langella, Frank,Molina, Ángela,Montero, Silvia,Rey, Fernando,Weaver, Sigourney,Wincott, Michael]director:[Scott, Ridley]}]
Hit [ID=tt0078723, values={title:[1941]year:[1979]actor:[Aykroyd, Dan,Beatty, Ned,Belushi, John,Caan, James,Cheshire, Denise,Gary, Lorraine,Hamilton, Murray,Lassick, Sydney,Lauren, Mo,Lee, Christopher,Marshall, Penny,Matheson, Tim,Mifune, Toshirô,Moriarty, Steve,Oates, Warren,Robinson, Hank,Rothstein, Debbie,Stack, Robert]director:[Spielberg, Steven]}]
Hit [ID=tt0046672, values={title:[20000 Leagues Under the Sea]year:[1954]actor:[Cooper, Ted,Daheim, John,Douglas, Kirk,Gargan, Jack,Graham, Fred,Harvey, Harry,Helton, Percy,Kerrigan, J.M.,Lorre, Peter,Lukas, Paul,Lummis, Dayton,Marr, Eddie,Mason, James,Mitchell, Laurie,Pall, Gloria,Pennick, Jack,Vigran, Herb,Wilke, Robert J.,Young, Carleton,de Corsia, Ted]director:[Fleischer, Richard]}]
Facet [name=genre, values={(Sci-Fi,237},(Drama,1063}}]
]

Listing 9 Search APIs test results

The second test (Listing 10) was exercising the adding and deleting of a document

public class DocumentAPITester extends TestCase {

	    private static final String DocumentURL = "doc-imdb-movies-ab4fpqw4eocczpgsnrtlu4rn7i.us-east-
1.cloudsearch.amazonaws.com"
;     private CloudSearchDocumentClient client;     private BatchType batch;          protected void setUp() throws Exception {         client = new CloudSearchDocumentClient(DocumentURL);                  FieldType title = new FieldType();         title.setName("title");         title.setValue("The Seeker: The Dark Is Rising");         FieldType director = new FieldType();         director.setName("director");         director.setValue("Cunningham, David L.");         FieldType genrea = new FieldType();         genrea.setName("genre");         genrea.setValue("Adventure");         FieldType genred = new FieldType();         genred.setName("genre");         genred.setValue("Drama");         FieldType genref = new FieldType();         genref.setName("genre");         genref.setValue("Fantasy");         FieldType genret = new FieldType();         genret.setName("genre");         genret.setValue("Thriller");         FieldType actor1 = new FieldType();         actor1.setName("actor");         actor1.setValue("McShane, Ian");         FieldType actor2 = new FieldType();         actor2.setName("actor");         actor2.setValue("Eccleston, Christopher");         FieldType actor3 = new FieldType();         actor3.setName("actor");         actor3.setValue("Conroy, Frances");         FieldType actor4 = new FieldType();         actor4.setName("actor");         actor4.setValue("Conroy, Frances");         FieldType actor5 = new FieldType();         actor5.setName("actor");         actor5.setValue("Ludwig, Alexander");         FieldType actor6 = new FieldType();         actor6.setName("actor");         actor6.setValue("Crewson, Wendy");         FieldType actor7 = new FieldType();         actor7.setName("actor");         actor7.setValue("Warner, Amelia");         FieldType actor8 = new FieldType();         actor8.setName("actor");         actor8.setValue("Cosmo, James");         FieldType actor9 = new FieldType();         actor9.setName("actor");         actor9.setValue("Hickey, John Benjamin");         FieldType actor10 = new FieldType();         actor10.setName("actor");         actor10.setValue("Piddock, Jim");         FieldType actor11 = new FieldType();         actor11.setName("actor");         actor11.setValue("Lockhart, Emma");         AddType add = new AddType();         add.setId("tt0484562");         add.setVersion(1l);         add.setLang("en");         add.getField().add(title);         add.getField().add(director);         add.getField().add(genrea);         add.getField().add(genred);         add.getField().add(genref);         add.getField().add(genret);         add.getField().add(actor1);         add.getField().add(actor2);         add.getField().add(actor3);         add.getField().add(actor4);         add.getField().add(actor5);         add.getField().add(actor6);         add.getField().add(actor7);         add.getField().add(actor8);         add.getField().add(actor9);         add.getField().add(actor10);         add.getField().add(actor11);                  DeleteType delete = new DeleteType();         delete.setId("tt0301199");         delete.setVersion(1l);                  batch = new BatchType();         batch.getAdd().add(add);         batch.getDelete().add(delete);     }     protected void tearDown() {         client.close();     }     public void testSearch() throws Exception{         try {             System.out.println("Test 1 ");             ResponseType result = client.index(batch);             System.out.println("Status " + result.getStatus() + " Added " + result.getAdds() + " Deleted " + result.getDeletes());         } catch (Exception e) {             e.printStackTrace();       }   } } Listing 10 Document upload tester

It also produced the expected result (Listing 11)

Status SUCCESS Added 1 Deleted 1

Listing 11 Document upload test results

Conclusion

The simple Java APIs for Amazon CloudSearch can significantly simplify usage of Amazon CloudSearch within existing Java applications and consequently increase their reach.

About the Author

Dr Boris Lublinsky is a Principal Architect at Nokia, where he is actively participating in Big Data, SOA, BPM and middleware implementations. Prior to this Boris was a Principal Architect at Herzum Software, where he was designing large-scale SOA systems for the clients, an Enterprise Architect at CNA Insurance where he was involved in design and implementation of CNA’s integration and SOA strategy, building application frameworks and implementing service-oriented architectures, Boris has over 25 years’ experience in enterprise and technical architecture and software engineering. He is an active member of OASIS SOA RM committee, coauthor of Applied SOA: Service-Oriented Architecture and Design Strategies book and author of numerous articles on Architecture, programming, Big Data, SOA and BPM.

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

API SOurce COde by skaushik6 .

Hi,
Is the API open source?
From where can I download the source code of the same.

Regards.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

1 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT