MainOverviewWikiIssuesForumBuildFisheye

Chapter 9. RSEM - Resource/Search Engine Mapping

9.1. Introduction

Compass provides OSEM technology for use with an applications Object domain model or XSEM when working with xml data structures. Compass also provides Resource Mapping technology for resources other than Objects/XML (that do not benefit from OSEM). The benefits of using Resources can be summarized as:

  • Your application does not have a domain model (therefore cannot use OSEM), but you still want to use the functionality of Compass.

  • Your application already works with Lucene, but you want to add Compass additional features (i.e. transactions, fast updates). Working with Resources makes your migration easy (as it is similar to working with Lucene Document).

  • You execute a query and want to update all the meta-data (Resource Property) with a certain value. You use OSEM in your application, but you do not wish to iterate through the results, performing run-time object type checking and casting to the appropriate object type before method call. You can simply use the Resource interface and treat all the results in the same abstracted way.

9.2. Mapping Declaration

In order to work directly with a Resource, Compass needs to know the alias and the primary properties (i.e. primary keys in data-base systems) associated with the Resource. The primary properties are also known as id properties. This information is declared in Resource Mapping XML documents, so that Compass knows how to manage the Resource internally (this is needs especially for update/delete operations).

Here is an example of a Resource Mapping XML document:

<?xml version="1.0"?>
<!DOCTYPE compass-core-mapping PUBLIC
    "-//Compass/Compass Core Mapping DTD 2.0//EN"
    "http://www.compass-project.org/dtd/compass-core-mapping-2.1.dtd">

<compass-core-mapping>

  <resource alias="a">
    <resource-id name="id" />
  </resource>

  <resource alias="b">
    <resource-id name="id1" />
    <resource-id name="id2" />
  </resource>

  <resource alias="c">
    <resource-id name="id" />
    <resource-property name="value1" />
    <resource-property name="value2" store="yes" index="analyzed" />
    <resource-property name="value3" store="compress" index="analyzed" />
    <resource-property name="value4" store="yes" index="not_analyzed" />
    <resource-property name="value5" store="yes" index="no" converter="my-date" />
  </resource>
</compass-core-mapping>

And here is the same example using JSON mapping configuration:

{
  "compass-core-mapping" : {
    "resource" : [
      {
        alias : "a",
        id : {name : " id"}
      },
      {
        alias : "b",
        id : [
          {name : "id1"},
          {name : "id2"}
        ]
      },
      {
        alias : "c",
        property : [
          { name : "value1"},
          { name : "value2", store : "yes", index : "analyzed"},
          { name : "value3", store : "compress", index : "analyzed"},
          { name : "value4", store : "yes", index : not_analyzed""},
          { name : "value5", store : "yes", index : "no", converter : "my-date"}
        ]
      }
    ]
  }
}

Now that the Resource Mapping has been declared, you can create the Resource in the application. In the following code example the Resource is created with an alias and id property matching the Resource Mapping declaration.

ResourceFactory resourceFactory = compass.getResourceFactory();
Resource r = resourceFactory.createResource("a");
Property id = resourceFactory.createProperty("id", "1",
    Property.Store.YES, Property.Index.NOT_ANALYZED);
r.addProperty(id);
r.addProperty(resourceFactory.createProperty("mvalue", "property test",
    Property.Store.YES, Property.Index.ANALYZED));

session.save(r);

The Resource Mapping file example above defines mappings for three resource types (each identified with a different alias). Each resource has a set of resource ids that are associated with it. The value for the resource-id tag is the name of the Property that is associated with the primary property for the Resource.

The third mapping (alias "c"), defines resource-property mappings as well as resource-id mappings. The resource-property mapping works with the Resource#addProperty(String name, Object value) operation. It provides definitions for the resource properties that are added (index, store, and so on), and they are then looked up when using the mentioned add method. Using the resource-property mapping, helps clean up the code when constructing a Resource, since all the Property characteristics are defined in the mapping definition, as well as auto conversion from different objects, and the ability to define new ones. Note that the resource-property definition will only work with the mentioned addProperty method, and no other addProperty method.

Here is an example of how resource-property mappings can simplify Resource construction code:

ResourceFactory resourceFactory = compass.getResourceFactory();
Resource r = resourceFactory.createResource("c");
r.addProperty("id", 1);
r.addProperty("value1", "this is a sample value");
r.addProperty("value5", new Date()); // will use the my-date converter (using the format defined there)
session.save(r);

All XML mappings should declare the doctype shown. The actual DTD may be found at the URL above or in the compass core distribution. Compass will always look for the DTD in the classpath first.

There are no compass-core-mapping attributes that are applicable when working with resource mappings.

9.2.1. resource

You may declare a resource mapping using the resource element:

<resource
      alias="aliasName"
      sub-index="sub index name"
      extends="a comma separated list of aliases to extend"
      analyzer="name of the analyzer"
/>
    all?,
    sub-index-hash?,
    resource-id*,
    (resource-analyzer?),
    (resource-boost?),
    (resource-property)*

Table 9.1. 

AttributeDescription
aliasThe name of the alias that represents the Resource.
sub-index (optional, defaults to the alias value)The name of the sub-index that the alias will map to.
extends (optional)A comma separated list of aliases to extend. Can extend a resource mapping or a resource-contract mapping. Note that can extend more than one resource/resource-contract
analyzer (optional, defaults to the default analyzer)The name of the analyzer that will be used to analyze ANALYZED properties. Defaults to the default analyzer which is one of the internal analyzers that comes with Compass. Note, that when using the resource-analyzer mapping (a child mapping of resource mapping) (for a resource property value that controls the analyzer), the analyzer attribute will have no effects.

9.2.2. resource-contract

You may declare a resource mapping contract using the resource-contract element:

<resource-contract
        alias="aliasName"
        extends="a comma separated list of aliases to extend"
        analyzer="name of the analyzer"
  />
      resource-id*,
      (resource-analyzer?),
      (resource-property)*
  

Table 9.2. 

AttributeDescription
aliasThe name of the alias that represents the Resource.
extends (optional)A comma separated list of aliases to extend. Can extend a resource mapping or a resource-contract mapping. Note that can extend more than one resource/resource-contract
analyzer (optional, defaults to the default analyzer)The name of the analyzer that will be used to analyze ANALYZED properties. Defaults to the default analyzer which is one of the internal analyzers that comes with Compass. Note, that when using the resource-analyzer mapping (a child mapping of resource mapping) (for a resource property value that controls the analyzer), the analyzer attribute will have no effects.

9.2.3. resource-id

Mapped Resource's must declare at least one resource-id. The resource-id element defines the Property that identifies the Resource for the specified alias.

<resource-id
      name="idName"
/>

Table 9.3. 

AttributeDescription
nameThe name of the Property (known also as the name of the meta-data) that is the id of the Resource.

9.2.4. resource-property

Declaring and using the resource-property element.

<resource-property
      name="property name"
      store="yes|no|compress"
      index="analyzed|not_analyzed|no"
      boost="boost value for the property"
      analyzer="name of the analyzer"
      reverse="no|reader|string"
      override="true|false"
      exclude-from-all="no|yes|no_analyzed"
      converter="converter lookup name"
>
</resource-property>

Table 9.4. 

AttributeDescription
nameThe name of the Property (known also as the name of the meta-data).
store (optional, defaults to yes)If the value of the resource property is going to be stored in the index.
index (optional, defaults to analyzed)If the value of the resource property is going to be indexed (searchable). If it does, than controls if the value is going to be broken down and analyzed (analyzed), or is going to be used as is (not_analyzed).
boost (optional, defaults to 1.0f)Controls the boost level for the resource property.
analyzer (optional, defaults to the resource mapping analyzer decision scheme)The name of the analyzer that will be used to analyze ANALYZED resource property mappings defined for the given property. Defaults to the resource mapping analyzer decision scheme based on the analyzer set, or the resource-analyzer mapping.
exclude-from-all (optional, default to false)Excludes the property from participating in the "all" meta-data. If set to no_analyzed, not_analyzed properties will be analyzed when added to the all property (the analyzer can be controlled using the analyzer attribute).
override (optional, defaults to true)If there is another definition with the same mapping name, if it will be overridden or added as additional mapping. Mainly used to override definitions made in extended mappings.
reverse (optional, defaults to no)The meta-data will have it's value reversed. Can have the values of no - no reverse will happen, string - the reverse will happen and the value stored will be a reversed string, and reader - a special reader will wrap the string and reverse it. The reader option is more performant, but the store and index settings will be discarded.
converter (optional)The global converter lookup name registered with the configuration.

Defines the characteristics of a Resource Property identified by the name mapping. The definition only applies when using the Resource#addProperty(String name, Object value) operation, and the operation can only be used with the resourcde-property mapping.

Note that other Resource Property can be added that are not defined in the resource mapping using the createProperty operation.

9.2.5. resource-analyzer

Declaring an analyzer controller property using the resource-analyzer element.

<resource-analyzer
      name="property name"
      null-analyzer="analyzer name if value is null"
      converter="converter lookup name"
>
</resource-analyzer>

Table 9.5. 

AttributeDescription
nameThe name of the Property (known also as the name of the meta-data).
null-analyzer (optional, defaults to error in case of a null value)The name of the analyzer that will be used if the property has the null value.
converter (optional)The global converter lookup name registered with the configuration.

The analyzer resource property mapping, controls the analyzer that will be used when indexing the Resource. If the mapping is defined, it will override the resource mapping analyzer attribute setting.

If, for example, Compass is configured to have two additional analyzers, called an1 (and have settings in the form of compass.engine.analyzer.an1.*), and another called an2. The values that the resource property can hold are: default (which is an internal Compass analyzer, that can be configured as well), an1 and an2. If the analyzer will have a null value, and it is applicable with the application, a null-analyzer can be configured that will be used in that case. If the resource property has a value, but there is not matching analyzer, an exception will be thrown.

9.2.6. resource-boost

Declaring a dynamic property to control the resource boost value using the resource-boost element.

<resource-boost
      name="property name"
      default="the boost default value when no property value is present"
      converter="converter lookup name"
>
</resource-boost>

Table 9.6. 

AttributeDescription
nameThe name of the Property (known also as the name of the meta-data).
default (optional, defaults to 1.0)The default value if the property has a null value.
converter (optional)The global converter lookup name registered with the configuration.

The boost resource property mapping, controls the boost associated with the Resource created based on the mapped property. The value of the property should be allowed to be converted to float.