Summary

Adds rich search functionality to Grails domain models. Built on Compass (http://www.compass-project.org/) and Lucene (http://lucene.apache.org/) This version is recommended for JDK 1.5+

Installation

Use the above dependency declaration in your BuildConfig.groovy file and then add

mavenRepo "http://repo.grails.org/grails/core"

to the repositories section of that file.

Description

The Searchable Plugin brings rich search features to Grails applications with minimum effort, and gives you power and flexibility when you need it.

It is built on the fantastic Compass Search Engine Framework and Lucene and has the same license as Grails (Apache 2).

The roadmap and issues are tracked in JIRA.

Only versions 0.6.5+ work with Grails 2.3+.
Only versions 0.6.4+ work with Grails 2.2+.

Initially the plugin will focus on exposing Groovy domain models.

Support for Java domain classes is on the roadmap but until then, if your Grails application uses Java domain classes, using Compass’s native annotations or XML mapping config and setting up Compass yourself may be a better fit at this stage.

Features

  • Maps searchable domain classes to the underlying search index

  • Performs a bulk index of all searchable class instances in the database

  • Automatically synchronises any changes made through GORM/hibernate with the index

  • Provides a SearchableService for easy cross-domain-class searching and index management

  • Adds domain class methods for easy per-domain-class searching and index management

  • Provides a SearchableController and view for testing queries and demonstrating the Searchable Plugin API

  • Follows a Convention over Configuration approach with overridable behaviour where necessary

Quick start

Install the plugin

JDK 1.5+ users:

grails install-plugin searchable

JDK 1.4 users:

grails install-plugin searchable14

Define Searchable classes

Add a static searchable property to the domain classes you want to be stored in the search index, for example:

class Post {
    static searchable = true              // <-- Make Posts searchable
    static hasMany = [comments: Comment]
    String category, title, post
    User author
    Date createdAt
}
class Comment {
    static searchable = true              // <-- Make Comments searchable
    static belongsTo = [post: Post]
    String comment
    Post post
    User user
    Date createdAt
}
class User {
    static searchable = true              // <-- Make Users searchable
    static hasMany = [posts: Post]
    String username, password
}

Declaring static searchable = true is the simplest way to define your [searchable class mappings|Searchable Plugin - Mapping].

In some cases during implementation with Grails 2.3, Searchable Plugin 0.6.6, i had end up with Issue unable tp locate org.compass-project:compass:2.2.1. Refer to this tutorial for the set up in case you are facing such issues with 2.3 Grails version. [http://jolorenz.wordpress.com/2013/10/02/how-to-use-searchable-plugin-with-grails-2-3-0-and-hibernate-4-x/]

Try it!

Fire up your app…​ during startup the plugin will build the search index all for searchable class instances in your database .

Navigate to [http://localhost:8080/YOUR-APP-NAME-HERE/searchable], and try a few queries.

You just made your domain searchable !

Next steps

Hack the controller and view to your own tastes.

Find what you’re looking for with the simple and powerful search API methods.

Customize the mappings between your classes and the search index.

See how to manage the index yourself if you need to; normally you do not, since changes made through Hibernate/GORM are mirrored to the index automatically.

Override defaults and more with configuration: for example, you could increase the search result page size from 10 to 20.

Trouble-shoot problems with the debugging tips and faq.

\\ \\ Happy searching !

SearchableController and view

Searchable Plugin comes with a controller SearchableController and view searchable/index.gsp. Try these with your application - they can help to test queries and you can probably copy some of the code.

You could use the SearchableController as it comes, but you will probably want a different URL and HTML results page.

To change the URL add entries to your grails-app/conf/UrlMappings.groovy.

To change the view, you can simply keep the plugin’s controller and copy myapp/plugins/searchable-x.x/grails-app/views/searchable/index.gsp to myapp/grails-app/views/searchable/index.gsp where it will override the plugin’s version.

Or of course you can create your own dedicated search controller and view.

Under the covers

Here’s the implementation of the search action from SearchableController:

import org.compass.core.engine.SearchEngineQueryParseException

// ...

class SearchableController {
    def searchableService

    /**
     * Index page with search form and results
     */
    def index = {
        if (!params.q?.trim()) {
            return [:]
        }
        try {
            return [searchResult: searchableService.search(params.q, params)]
        } catch (SearchEngineQueryParseException ex) {
            return [parseException: true]
        }
    }

    // ...
}

Notice that params.q is the search query string and params is also given to the search method as the second argument. This is a Map of options and are things like page size, start result number, etc.

Any String arguments are parsed, so you can use request parameters directly, even when they have string values. For example, params may be \[escape: "true", offset: "20", q: "toast"\] , but you won’t get a ClassCastException.

Pagination of search results in the view

Search results can be paginated using Grails' standard <g:paginate /> tag.

Here it is in action in the Searchable Plugin’s own search results page, grails-app/views/searchable/index.gsp. The searchResult is an object returned by either SearchableService#search or DomainClass#search:

<g:if test="${haveResults}"> <!-- or you could use test="${searchResult?.results}" -->
    Page:
    <g:set var="totalPages" value="${Math.ceil(searchResult.total / searchResult.max)}" />
    <g:if test="${totalPages == 1}">
        <span class="currentStep">1</span>
    </g:if>
    <g:else>
        <g:paginate controller="searchable" action="index" params="[q: params.q]"
                    total="${searchResult.total}" prev="&lt; previous" next="next &gt;"/>
    </g:else>
</g:if>

And here’s some CSS to style the generated HTML:

.paging a.step {
    padding: 0 .3em;
}

.paging span.currentStep {
    font-weight: bold;
}

Suggested queries

You can easily highlight the difference between the original and a [suggested query|Searchable Plugin - Methods - suggestQuery] with the following in a GSP:

<%@ page import="org.codehaus.groovy.grails.plugins.searchable.util.StringQueryUtils" %>

<p>Did you mean
  <g:link controller="searchable" action="index" params="[q: searchResult.suggestedQuery]">
${StringQueryUtils.highlightTermDiffs(params.q.trim(), searchResult.suggestedQuery)}
  </g:link>?
</p>

Which results in something like:

Did you mean space invader?

See this in action in the plugin’s view.

Searching

If you’ve outgrown the simple controller and view, you need the search API.

The plugin provides search methods with its SearchableService and also adds some to your searchable domain classes. The available methods are the same but the domain class methods restrict the search space to only that class (or hierarchy).

The methods are:

  • search - Find objects matching a query

  • countHits - Find the number of hits for a query

  • moreLikeThis - Finds similar objects to the indicated searchable domain class instance

  • suggestQuery - Suggest a new search query based on spelling

  • termFreqs - Returns term frequencies for the terms in the index

Unresolved directive in index.adoc - include::searching_query_strings.adoc[]

Query Builder

Searchable Plugin comes with a builder, which makes programmatic queries Groovy, baby.

When you pass a closure to one of the search methods, you are using the query builder.

TODO the query builder needs a reference section

It builds on Compass

The query builder syntax mirrors Compass’s own CompassQueryBuilder: to get the most out of the Searchable Plugin’s query builder, you should take a moment to familiarise yourself with it.

And makes it better

The plugin’s Groovy query builder improves on the raw Compass experience in a few ways. First it relieves you of the burden to call toQuery() when using the various specific builders obtained from calling some CompassQueryBuilder methods, and it allows you to easily nest these specific builders using closures.

It also simplifies the job of constructing boolean queries by not requiring you to explicitly create a boolean builder and add "should" clauses. You still need to explicitly add must and must not clauses, but other clauses in a boolean context are assumed to be should clauses.

The rules

The methods you can invoke depend on the current context. In the outer-most context you can invoke CompassQueryBuilder methods.

Within nested contexts (which are created by nested closures) you can call both CompassQueryBuilder methods and whatever methods the current nested builder supports.

The builder also allows you to call Compass’s various query builders' options "setters" using a literal options Map as the last argument, instead of requiring method invocations.

The builder shortens a few method names too, so the CompassBooleanQueryBuilder addMust, addShould, and addMustNot methods can be shortened to must, should, and mustNot (and as already mentioned you typically don’t need to use should because any clause that is not a must or mustNot is assumed to be should) and the CompassQuery addSort method can be shortedned to sort.

And finally because it’s Groovy, you have the language at your disposal so you can use control flow, loops, variables etc.

Enough theory, let’s explore some examples.

By Example

Say you want to search for items in the index where "pages" is less than 50 and "type" is "poetry". A String query for this might look like "pages:\[\ TO 50\] type:poetry"*.

Using the builder you would do

search {                    // <-- create an implicit boolean query
    lt("pages", 50)         // <-- uses CompassQueryBuilder#lt, and adds a boolean "should" clause
    term("type", "poetry")  // <-- uses CompassQueryBuilder#term, and adds a boolean "should" clause
}

We just built a boolean query\! It has two clauses: the search must match EITHER "pages" < 50 OR "type" == "poetry". Not bad but this query will match when either condition is true, and not necessarily both.

So let’s improve the search results and make sure that matches DO have "type" == "poetry".

search {                          // <-- creates an implicit boolean query
    lt("pages", 50)               // <-- uses CompassQueryBuilder#lt, and adds a boolean "should"
    must(term("type", "poetry"))  // <-- uses CompassQueryBuilder#term, adds a boolean "must"
}

Ok let’s assume we’re getting matches we don’t want, and so we add another "mustNot" clause:

search {                           // <-- creates an implicit boolean query
    lt("pages", 50)                // <-- uses CompassQueryBuilder#lt, and adds a boolean "should"
    must(term("type", "poetry"))   // <-- uses CompassQueryBuilder#term, adds a boolean "must"
    mustNot(term("theme", "war"))  // <-- uses CompassQueryBuilder#term, adds a boolean "mustNot"
}

Great, so now we’re matching any non-war poetry under 50 pages\!

Now we want to search for a specific phrase "all hands on deck", let’s add it to the query. First we try it as a nested String query:

search {                                   // <-- creates an implicit boolean query
    must(queryString("all hands on deck")) // <-- uses CompassQueryBuilder#queryString, and adds a boolean must
    lt("pages", 50)                        // <-- uses CompassQueryBuilder#lt, and adds a boolean "should"
    must(term("type", "poetry"))           // <-- uses CompassQueryBuilder#term, adds a boolean "must"
    mustNot(term("theme", "war"))          // <-- uses CompassQueryBuilder#term, adds a boolean "mustNot"
}

The nested String query has introduced the possibility for the query to matching anything containing any of the words "all", "hands", "on", or "deck" and maybe in any searchable property. But we wanted to search for this exact text, hmmm.

Check the CompassQueryBuilder API and you’ll notice that CompassQueryBuilder#queryString returns a CompassQueryStringBuilder, so in fact we can create a context for that builder with a closure and in that closure call any methods the CompassQueryStringBuilder exposes to tighten up the query:

search {                                      // <-- creates an implicit boolean query
    must(queryString("all hands on deck") {   // <-- creates a nested CompassQueryStringBuilder context
        useAndDefaultOperator()               // <-- calls CompassQueryStringBuilder#useAndDefaultOperator
        setDefaultSearchProperty("body")      // <-- calls CompassQueryStringBuilder#setDefaultSearchProperty
    })                                        // <-- added as boolean must to surrounding boolean
    lt("pages", 50)                           // <-- uses CompassQueryBuilder#lt, and adds a boolean "should"
    must(term("type", "poetry"))              // <-- uses CompassQueryBuilder#term, adds a boolean "must"
    mustNot(term("theme", "war"))             // <-- uses CompassQueryBuilder#term, adds a boolean "mustNot"
}

Ok, now the query requires that ALL of the words "all hands on deck" are present in matches. And we have set the defaultSearchProperty to "body", so the string query will now match terms in the searchable "body" property.

But we can do a little better. First let’s use an options Map instead of calling those setters:

search {                                    // <-- creates an implicit boolean query
    must(queryString("all hands on deck", [useAndDefaultOperator: true, defaultSearchProperty: "body"]))
        // ^^ add a "must" nested query string, calling useAndDefaultOperator() and setDefaultSearchProperty("body")
        //    on the CompassQueryStringBuilder
    lt("pages", 50)                         // <-- uses CompassQueryBuilder#lt, and adds a boolean "should"
    must(term("type", "poetry"))            // <-- uses CompassQueryBuilder#term, adds a boolean "must"
    mustNot(term("theme", "war"))           // <-- uses CompassQueryBuilder#term, adds a boolean "mustNot"
}

This is the same query as before, only fewer lines of code.

So now let’s use CompassQueryBuilder#multiPhrase instead of queryString, since a multi-phrase query can require that the words appear in order, whereas a query string generally just requires the words appear somewhere.

search {                                    // <-- creates an implicit boolean query
    must(multiPhrase("body", [slop: 2]) {   // <-- creates a nested CompassMultiPhraseQueryBuilder context, calling setSlop(2)
        add("all")                          // <-- calls CompassMultiPhraseQueryBuilder#add
        add("hands")                        // <-- calls CompassMultiPhraseQueryBuilder#add
        add("on")
        add("deck")
    })                                      // <-- adds multiPhrase as boolean "must"
    lt("pages", 50)                         // <-- uses CompassQueryBuilder#lt, and adds a boolean "should"
    must(term("type", "poetry"))            // <-- uses CompassQueryBuilder#term, adds a boolean "must"
    mustNot(term("theme", "war"))           // <-- uses CompassQueryBuilder#term, adds a boolean "mustNot"
}

Let’s go all out for the final example and add three new features: (i) move the number pages clause into a nested boolean with a new clause for items with 50 or more pages, and a "boost" (meaning higher score/relevance) for the smaller number, (ii) a clause for "publishedDate" being within the last 4 weeks (and note the use of a Date object) and (iii) a sort first by relevance then author surname.

search {                                    // <-- creates an implicit boolean query
    must(multiPhrase("body", [slop: 2]) {   // <-- creates a nested CompassMultiPhraseQueryBuilder context, and calls setSlop(2)
        add("all")                          // <-- calls CompassMultiPhraseQueryBuilder#add
        add("hands")                        // <-- calls CompassMultiPhraseQueryBuilder#add
        add("on")
        add("deck")
    })                                      // <-- adds multiPhrase as boolean "must"
    must {                                  // <-- creates an nested boolean query, implicitly
        ge("pages", 50)                     // <-- uses CompassQueryBuilder#ge, adds a boolean "should"
        lt("pages", 50, [boost: 1.5])       // <-- uses CompassQueryBuilder#lt, calls setBoost(1.5f), adds a boolean "should"
    }                                       // <-- adds nested boolean as "must" clause to outer boolean
    must(term("type", "poetry"))            // <-- uses CompassQueryBuilder#term, adds a boolean "must"
    mustNot(term("theme", "war"))           // <-- uses CompassQueryBuilder#term, adds a boolean "mustNot"
    ge("publishedDate", new Date() - 28)
    sort(CompassQuery.SortImplicitType.SCORE)                  // <-- uses CompassQuery#addSort
    sort("authorSurname", CompassQuery.SortPropertyType.STRING) // <-- uses CompassQuery#addSort
}

Sorting

Search results are sorted by relevance by default, the most relevant being the first.

You can specify a custom sort when using either String queries or a query builder Closure using the sort and order/direction method options:

  • sort - Either "SCORE" (ie, relevance) or the name of a mapped class property

  • order or direction - One of "auto", "reverse", "asc", "desc".

Examples

// Get Property instances matching "Riverside Apartment"
// sorted by price in ascending order
def searchResult = Property.search(
    "Riverside Apartment",
    sort: "price", order: "asc"
)
// Find the least relevant for a fuzzy term search
def leastRelevant = searchableService.search({
        fuzzy("word", "roam")
    },
    result: "top", sort: "SCORE", direction: "reverse"
)

The values "auto" and "reverse" are symbols for Compass API constants: when sorting by SCORE an order of "auto" means highest scoring first and "reverse" means lowest scoring first. When sorting by anything else, "auto" is the natural ascending order and "reverse" is the natural descending order.

The Searchable Plugin also gives you the opton to use "asc" or "desc" for the order or direction value.

Why are there two options for sort order - order and direction - and are they different? Well, they both control the sort order, and you can mix the option names and values, so they are actually just synonyms. But both are provided to satisfy those people more familiar with the GORM style parameters (order with asc/desc) and those more familiar with Compass and search engine queries (direction with auto/reverse). Choose whichever you prefer.

The following summaries the behaviour of these option combinations. Remember that order and direction are interchangable:

sort order or direction Sorting behavoir

SCORE

auto

The results are ordered most-relevant (highest scoring) first. This is the default

SCORE

reverse

The results are ordered least-relevant (lowest scoring) first.

SCORE

"asc

The results are ordered least-relevant (lowest scoring) first.

SCORE

"desc

The results are ordered most-relevant (highest scoring) first. This is the same as SCORE + auto, which is the default

someProperty

asc

The results are ordered in natural order for the someProperty field value, eg, String ascending, Number ascending

someProperty

desc

The results are order in revserse natural order for the someProperty field value, eg, String descending, Number descending

someProperty

auto

The results are ordered in natural order for the someProperty field value, eg, String ascending, Number ascending

someProperty

reverse

The results are order in revserse natural order for the someProperty field value, eg, String descending, Number descending

It is also possible to add sorting using a query builder Closure, and in fact with that technique you can add multiple sorts, eg:

import org.compass.core.*

// Sort first by score (relevance),
// then by most votes (when the score is equal)
def hits = searchableService.searchEvery {
    queryString("reality tv")
    sort(CompassQuery.SortImplicitType.SCORE)
    sort("votes", CompassQuery.SortDirection.REVERSE)
}

When using a query building Closure you can also combine sorts in the closure with the above sort/order options. If you do this, the sort/order options are added as the last sort in the chain and therefore are applied last.

Mapping

A searchable class mapping describes how the class instance appears in the index and the data is searched.

This includes things like:

  • Which properties are searchable

  • How a property is processed during indexing - whether it is "analyzed" or not, for example

  • How a property influences the search - whether it has a "boost", for example

  • How associated searchable classes are linked and/or embed one another’s searchable data

When you declare

static searchable = true

the plugin maps the class with built-in conventions.

You can override these conventions in a number of ways:

The Mapping DSL

The mapping DSL, is a bit like GORM’s mapping DSL.

You can selectively override the built-in conventions for specific properties (or the class itself), and inherit the default behaviour for any properties you do no explicitly map.

Native Compass Mappings

You can also use Compass annotations and Compass XML to map your classes.

With either Compass XML or Compass annotations, you need to map every aspect of the searchable class yourself. In other words, the built-in conventions no longer apply to classes you map with Compass mappings.

Not All Properties

Using only and except

You can limit the properties that are made searchable like:

class Post {
    static searchable = [only: ['category', 'title']]
    // ...
}

or

class Post {
    static searchable = [except: 'createdAt']
    // ...
}

The value of except or only can be a String or List<String> as shown above.

The String can contain shell-like-wildcards:

  • * means any number of characters

  • ? means any single character

except and only wildcard examples
// map, eg, 'addressLine1', 'addressLine2', 'addressPostcode', etc...
static searchable = [only: 'address*']
// do not map, eg, 'screenX' and 'screenY' and 'version'
static searchable = [except: ['screen?', 'version']]

However read this [caveat and alternative approach|Searchable Plugin - Mapping - Not All Properties#Excluding certain fields from being searched].

Using except or only with class property mapping

You can combine [class property mappings|Searchable Plugin - Mapping - Class Property Mapping] with only and except:

class Post {
    static searchable = {
        except = ["version", "createdAt"]   // version and createdAt will not be mapped to the index
        category index: 'not_analyzed', excludeFromAll: true
        title boost: 2.0
        comments component: true
    }
    static hasMany = [comments: Comment]

    User author
    String title, post, category
    Date createdAt
}

Excluding certain fields from being searched

Note that using only or except means that the index will only contain those properties as indicated, so when an object is returned from the index, some properties may be null, even if they have non-null values in the database.

You can do better than this by still mapping those properties to the index, but having them not actually indexed for search purposes. To do this add a [mapping for the property|Searchable Plugin - Mapping - Class Property Mapping] with an index: 'no' option.

This method may become default behavior for properties excluded from mapping with only and except in future.

Mapping conventions

First please familiarise yourself with some essential Compass mapping concepts.

The easiest way to map a class to the search index is by declaring

static searchable = true

which maps all "mappable" properties in the class using built-in rules:

  • Simple property types* like numbers, dates and strings, or a collection of, are mapped as [searchable-properties|Searchable Plugin - Mapping - Compass concepts#Searchable Property], which becomes the searchable text in the index.

  • Non-embedded domain class properties* or element type if a collection are mapped as [searchable-references|Searchable Plugin - Mapping - Compass concepts#Searchable Reference] as long as they are searchable too.

  • Embedded domain class properties* are mapped as [searchable-components|Searchable Plugin - Mapping - Compass concepts#Searchable Component] as long as they are not specifically non-searchable.

INFO: The mapping DSL gives you more flexibility. For example you can limit the searchable properties and override the conventions a per-class-property basis, and define class-level mapping options.

Example

Let’s take this class to explain the built-in mapping conventions:

class Post {
    static searchable = true
    static embedded = ['metadata']
    static hasMany = [comments: Comment]
    String category, title, post
    User author
    Metadata metadata
    Date createdAt
}
Simple type properties become the searchable text in the index

The simple type properties category, title, post and createdAt are mapped as [searchable-properties|Searchable Plugin - Mapping - Compass concepts#Searchable Property].

As is the case with [searchable-properties|Searchable Plugin - Mapping - Compass concepts#Searchable Property], the values of these properties becomes searchable text in the index, in fields named after the property.

Additionally an "all" field is created which combines all searchable text for the object.

Therefore you can search for "title:grails" or "category:javascript" to target those specific fields, whereas if you search for just "grails" it will match any text in any searchable field.

Non-embedded domain class properties become searchable references

As long as associated domain classes are searchable too, they are normally mapped as [searchable-references|Searchable Plugin - Mapping - Compass concepts#Searchable Reference].

Here, if User and Comment are searchable, then the user and comments properties are both mapped as [searchable-references|Searchable Plugin - Mapping - Compass concepts#Searchable Reference].

This means that when an object is returned from the index as a search hit, it can be returned along with it’s associated domain objects (and domain object collections) set on it.

Embedded domain class properties become searchable components

The embedded domain class property metadata is mapped as a [searchable-component|Searchable Plugin - Mapping - Compass concepts#Searchable Component] so any search query that matches the Post 's metadata will return that Post instance.

You have the choice of whether to declare the Metadata class as searchable:

  • If you don’t declare it as searchable, the class is automatically mapped as a [non-root|Searchable Plugin - Mapping - Class Mapping#Options] class with the built-in conventions

  • To make Metadata non-searchable you can add static searchable = false

  • Otherwise whatever value of Metadata’s static searchable property (or one of native the Compass mapping choices) defines how the class is mapped

Compass concepts

Properties of your domain classes will be mapped with one of the following strategies:

Searchable Property

This is used for simple types (or collections of), basically anything that can be represented as a string in the index, eg, strings, numbers, dates, enums, etc.

We might use the phrase "searchable property" to refer to the class property that is mapped as a searchable property, as well as the mapping concept.

Within the index, a Lucene field named after the property is created, and the value of that field contains the searchable text. You can then target that field with a prefixed term in the query.

Additionally Compass creates a field in the index called "all" which contains all searchable text for a class instance. If you do not prefix search query terms, they hit the "all" field.

INFO: You can and probably should map your own immutable/value-types as searchable-properties.

INFO: You can provide a custom Converter implementation to convert the object to/from String form.

Example

Given the following class:

class Book {
    String title, text
}
`title` and `text` can be mapped as a searchable-properties.

You can then search specifically for text in title with a query string like "title:attack".

The query string "attack" would match Book s with that word in their title or text fields (or both), because in fact it hits the "all" field which combines all searchable text for the class instance.

Searchable Reference

This is used for complex types (or collections of); typically associated domain classes.

We might use the phrase "searchable reference" to refer to the class property that is mapped as a searchable reference, as well as the mapping concept.

A searchable-reference’s class must be searchable itself.

When indexing an object its searchable-references are stored in their own indexes (Compass calls these "sub-indexes").

The relationship is also stored in the index belonging to the class with the searchable reference mapping. This is simply the class type and id of the other object(s), so it’s efficient.

Therefore:

  • Objects on either side of the relationship are first-class citizens in the index and can be search for in their own right.

  • When searching, hits will only occur for text present within the searchable class instance itself

  • When re-creating an object from the index (for search results), Compass can re-create its relationships too.

Example

Given the following two classes

class Post {
    Set<Comment> comments
    // ..
}
class Comment {
    Post post
    // ....
}

lets says both classes are searchable and both Post#comments and Comment#post are mapped as searchable-references .

Now when a search matches a Post then the Post will be returned from the index with the comments populated from data saved in the index - it doesn’t hit the database.

Likewise if a search finds a Comment it can be returned along with it’s associated Post .

Searchable Component

This is used for complex types (or collections of); typically associated domain classes.

We might use the phrase "searchable component" to refer to the class property that is mapped as a searchable component, as well as the mapping concept.

A searchable-component’s class must be searchable itself.

Unlike [searchable-references|Searchable Plugin - Mapping - Compass concepts#Searchable Reference], when indexing searchable-components, those searchable-component objects are not stored in their own indexes (at least from the owning side). Instead the data of the searchable-component is stored within the owning object’s own searchable data. In other words, the searchable data of a searchable-component is added to the searchable data of the class instance declaring the property.

Therefore:

  • You can search for data that is embedded into a top-level object from a searchable component, this counts as a match for the top-level object

  • If the searchable component’s class is not mapped as a [root|Searchable Plugin - Mapping - Class Mapping#Options] class, then you will never find instances with search

  • When re-creating an object from the index (for search results), Compass can re-create its associated searchable-components too.

Example

Given the following two classes

class Post {
    Set<Comment> comments
    // ..
}
class Comment {
    Post post
    // ....
}

lets says both classes are searchable and both Post#comments and Comment#post are mapped as searchable-components .

Now the user searches for a string that happens to match a Comment , they will get both Post and Comment hits. Additionally search result objects will have their associations populated, and this doesn’t hit the database.

More on Searchable Reference and Searchable Component

You cannot map the same class property as both a searchable reference and a searchable component.

However if you have a bidirectional relationship, it is possible to map one side of a relationship as a searchable-reference and the other as searchable-component.

Mapping DSL

The mapping DSL allows you customize various aspects of the class mapping, for example:

Class Property Mapping

If you want to override the [conventional mapping|Searchable Plugin - Mapping - Conventions] for one or more properties, make the value of searchable a Closure and add method calls named after the class property:

class Post {
    static searchable = {
        category index: 'not_analyzed', excludeFromAll: true
        title boost: 2.0
        comments component: true
    }
    static hasMany = [comments: Comment]
    User author
    String title, post, category
    Date createdAt
}

The mapping options available depend on the kind of class property.

Mapping the id

Compass needs to know the object’s identifier property, so that needs to be mapped too.

Since Grails has the well defined and generated id property, the plugin generates an id mapping.

You can customize this with an explicit [id mapping|Searchable Plugin - Mapping - Searchable Id].

Mapping simple types

For simple types, you can declare any of the [searchable property mapping options|Searchable Plugin - Mapping - Searchable Property].

Example

Given the following class:

class Article {
    static searchable = true
    static hasMany = [keywords: String]
    String title, body
}

we currently have a [conventional mapping|Searchable Plugin - Mapping - Conventions] because it declares simply static searchable = true. This maps both title and body as standard [searchable properties|Searchable Plugin - Mapping - Compass concepts#Searchable Property].

Let’s define a few additional [searchable property mapping options|Searchable Plugin - Mapping - Searchable Property] to customize the [conventional mapping|Searchable Plugin - Mapping - Conventions] to our needs:

class Article {
    static searchable = {
        title boost: 2.0
        keywords index: 'not_analyzed'
    }
    static hasMany = [keywords: String]
    String title, body
}

In this case we’ve given title a [boost|Searchable Plugin - Mapping - Searchable Property#Options], which means hits in the title property score higher than other properties, and we’ve declared keywords as [not_analyzed|Searchable Plugin - Mapping - Searchable Property#Options], meaning it’s value is stored verbatim in the index.

Mapping associated domain classes

For associated domain classes you have two choices:

  • You can inherit and add to the [conventional mapping|Searchable Plugin - Mapping - Conventions] - whether a [searchable reference|Searchable Plugin - Mapping - Compass concepts#Searchable Reference] or [searchable component|Searchable Plugin - Mapping - Compass concepts#Searchable Component] - and add to it with the supported [searchable reference mapping options|Searchable Plugin - Mapping - Searchable Reference] or [searchable component mapping options|Searchable Plugin - Mapping - Searchable Component].

  • Or you can override the [conventional mapping|Searchable Plugin - Mapping - Conventions] - of [searchable reference|Searchable Plugin - Mapping - Compass concepts#Searchable Reference] or [searchable component|Searchable Plugin - Mapping - Compass concepts#Searchable Component] to be the other.

Example

Given the following classes:

class News {
    static searchable = true
    static hasMany = [comments: Comment]
    String text
}
class Comment {
    static searchable = true
    String text
    News news
}

We start out with the [conventional mapping|Searchable Plugin - Mapping - Conventions], since Comment and News declare static searchable = true.

This [maps the classes by convention|Searchable Plugin - Mapping - Conventions] meaning:

  • News#comments becomes a [searchable reference|Searchable Plugin - Mapping - Compass concepts#Searchable Reference]

  • Comment#news becomes another [searchable reference|Searchable Plugin - Mapping - Compass concepts#Searchable Reference]

Inherit and add

To inherit the Comment#news [searchable reference|Searchable Plugin - Mapping - Compass concepts#Searchable Reference] mapping and add [searchable reference mapping options|Searchable Plugin - Mapping - Searchable Reference]:

class Comment {
    static searchable = {
        news cascade: ['create', 'delete']
    }
    String text
    News news
}

If you like you can also make the conventional mapping explicit, ie, declare it as a [searchable reference|Searchable Plugin - Mapping - Compass concepts#Searchable Reference] or [searchable component|Searchable Plugin - Mapping - Compass concepts#Searchable Component]:

class Comment {
    static searchable = {
        news reference: [cascade: ['create', 'delete']]
    }
    String text
    News news
}

The value of reference or component may be true or a Map of options.

Override

To override the News#comments [searchable reference|Searchable Plugin - Mapping - Compass concepts#Searchable Reference] mapping and make it a [searchable component|Searchable Plugin - Mapping - Compass concepts#Searchable Component] instead:

class News {
    static searchable = {
        comments component: true
    }
    static hasMany = [comments: Comment]
    String text
}

You can also specify [searchable reference mapping options|Searchable Plugin - Mapping - Searchable Reference] or [searchable component mapping options|Searchable Plugin - Mapping - Searchable Component] options with a Map value:

class News {
    static searchable = {
        comments component: [cascade: 'all', accessor: 'field']
    }
    static hasMany = [comments: Comment]
    String text
}

Searchable Id

Summary

Used to map the searchable class’s identifier.

Syntax

static searchable = {
    id options
}

Description

Maps the searchable id with the given [options|#Options].

Parameters

  • options - A Map of [options|#Options]

Options

  • name - The name of the searchable id field in the index. Default is id. This allows you to target the id field in a query, eg, if you decided to name your id "_\_the\_id\__" you could query it with "\__the\_id\_\_:10".

  • accessor - How the property is accessed. One of "field" or "property". Default is "property"

  • converter - The name of a configured converter to use to convert the property value to/from text

Examples

// Obscure the name of the id property
id name: "\_\_the\_id\_\_"

Searchable Property

Summary

Used to map [searchable properties|Searchable Plugin - Mapping - Compass concepts#Searchable Property].

Syntax

static searchable = {
    propertyName options
}

Description

Maps the class’s propertyName property with the given [options|#Options].

Supports regular properties (those you declare the field for) and synthetic properties (those with no class field but a "getter" method).

It is possible to map the same property with multiple different mappings (as long as the mappings are semantically valid), see the [examples|#Examples].

Parameters

  • options - a Map of [options|#Options]

Options

  • accessor - How the property is accessed. One of "field" or "property". Default is "property"

  • analyzer - The name of a configured analyzer used to analyze this property. Default is "default" which is a built-in analyzer (Lucene’s StandardAnalyzer)

  • boost - A decimal boost value. With a positive value, promotes search results for hits in this property; with a negative value, demotes search results that hit this property. Default is 1.0

  • excludeFromAll - Whether the property should be excluded from the generated "all" searchable text field in the index. One of "yes", "no" or "no_analyzed"

  • format - How the property is formatted when made into searchable text. Applies to objects like `Date`s and `Number`s usually for the purposes of range searches. Value is a format string for the appropriate formatter.

  • index - How or if the property is made into searchable text. One of "'no", "not_analyzed" or "analyzed".

  • name - The name of the field in the search index. Can be used with multiple mappings for the same property, each with their own name. Default is propertyName. This becomes the name of the field in the index, so if name is "title", you can target that field with a query like "title:grails"

  • nullValue - The value to use if the property is null when indexed.

  • propertyConverter - The name of a configured ResourcePropertyConverter which converts the property from/to searchable text.

  • reverse - Whether the property should be reversed when made searchable. One of "no", "reader" or "string`". Default is `"no"

  • spellCheck - Should the values of the property be included in the spell-check index? Either "include" or "exclude". If not defined then inherits the class’s own spell-check mapping.

  • store - Should the value be stored in the index? One of "yes", "no" or "compress". If "no" then the property may still be searchable (depending on the index option), but when re-creating the object for search results this property will always be null. "compress" is useful for large or binary property values. Default is "yes"

  • termVector - Should the term-vector data be collected for the property in the index? One of "yes", "no", `"with\_positions`", `"with\_offsets`", `"with\_positions\_offsets`". If not defined inherits the class’s term-vector mapping. This is required for [more-like-this|Searchable Plugin - Methods - moreLikeThis] searches.

Examples

// Give matches in the title field a boost
title boost: 2.0
// Format a date property
// so it's easy to search with range queries
createdAt format: "yyyyMMdd"
// Format a number property padding with zeroes
// so it's easy to search with range queries
numVotes format: "0000"
// Multiple mappings for the same class property, one analyzed the other not
// This means we can search for, say, "tagsExact:Miami" and it would only match the
// exact text "Miami", not "MiAmi" or "miami" etc.
// And we can search for "tags:Miami" and match any variation
tags index: 'not\_analyzed', name: 'tagsExact'
tags index: 'analyzed'

Searchable Reference

Summary

Used to map [searchable references|Searchable Plugin - Mapping - Compass concepts#Searchable Reference].

Syntax

static searchable = {
    propertyName options
}
static searchable = {
    propertyName reference: true
}
static searchable = {
    propertyName reference: options
}

Description

Maps the class’s propertyName property with the given [options|#Options].

You can use all three syntaxes when a [conventional mapping|Searchable Plugin - Mapping - Conventions] would give you a [searchable reference|Searchable Plugin - Mapping - Compass concepts#Searchable Reference].

When you want to change a [conventional searchable component mapping|Searchable Plugin - Mapping - Conventions] (which can happen when the associated class is embedded), you can use either of the last two syntaxes to make it a [searchable reference|Searchable Plugin - Mapping - Compass concepts#Searchable Reference] instead.

Parameters

  • options - a Map of [options|#Options]

Options

  • accessor - How the property is accessed. One of "field" or "property". Default is "property"

  • cascade - The operations to cascade to the target association. A String comma-delimited-list of values in "all", "create", "save" and "delete".

  • converter - The name of a configured converter to use to convert the property value to/from text

  • lazy - If true the association is lazy when recreating the owing instance for search results. Only applicable to Collections or arrays. Default is false

Examples

// Provide a custom converter
metadata converter: "metadata\_converter"
// Map metadata as a reference, overriding any conventional mapping
metadata reference: true
// Map metadata as a reference, overriding any conventional mapping
// and define some options
metadata reference: [converter: "metadata\_converter"]

Searchable Component

Summary

Used to map [searchable components|Searchable Plugin - Mapping - Compass concepts#Searchable Component].

Syntax

static searchable = {
    propertyName options
}
static searchable = {
    propertyName component: true
}
static searchable = {
    propertyName component: options
}

Description

Maps the class’s propertyName property with the given [options|#Options].

You can use all three syntaxes when a [conventional mapping|Searchable Plugin - Mapping - Conventions] would give you a [searchable component|Searchable Plugin - Mapping - Compass concepts#Searchable Component].

When you want to change a [conventional searchable reference mapping|Searchable Plugin - Mapping - Conventions] (which is the normally how domain class associations are mapped), you can use either of the last two syntaxes to make it a [searchable component|Searchable Plugin - Mapping - Compass concepts#Searchable Component] instead.

Parameters

  • options - a Map of [options|#Options]

Options

  • accessor - How the property is accessed. One of "field" or "property". Default is "property"

  • cascade - The operations to cascade to the target association. A String comma-delimited-list of values in "all", "create", "save" and "delete".

  • converter - The name of a configured converter to use to convert the property value to/from text

  • maxDepth - The depth of cyclic component references allowed. Default is 1

  • prefix - A String prefix to apply to the component’s properties. A prefix can be used to distinguish between multiple components of the same type. Without a prefix you can search within a component’s properties with their property names, eg, "city:london", with a prefix like 'homeAddress$' you can search for 'homeAddress$city:london'

Examples

// Provide a custom converter
metadata converter: "metadata\_converter"
// Map metadata as a component, overriding any conventional mapping
metadata component: true
// Map metadata as a component, overriding any conventional mapping
// and define some options
metadata component: [converter: "metadata\_converter"]

Class Mapping

Summary

Defines the class mapping

Syntax

static searchable = {
    option value
    option value
}
static searchable = {
    mapping options
}
static searchable = {
    mapping {
        option value
        option value
    }
}
static searchable = {
    mapping options, {
        option value
        option value
    }
}

Description

A number of mapping options affect the class itself or provide defaults for property mappings.

The syntaxes are interchangeable. Use whichever you prefer.

One of the last three syntaxes (introduced by the mapping keyword) may be necessary if you need to use an option that clashes with one of your domain class property names.

Parameters

  • options - A Map of [options|#Options]

  • option - The name of an [option|#Options]

  • value - An [option|#Options] value

Options

all

Controls the behaviour of the automatically created "all" field in the index for instances of this class. If not defined uses built-in defaults. More

analyzer

The name of a configured Analyzer used to analyze the searchable data of instances. Overridable on a per property basis. Default is "default", a built-in Analyzer.

alias

A name for documents of this class type in the index. Default is the simple class name, eg, the class org.boom.User would have an alias of "User" by default. It’s essentially a piece of constant text added for every object of this class type, and allows you to query by class, eg, "alias:User maurice"

boost

A decimal value which increases or decreases the ranking of hits for this class if positive or negative respectively. Default is 1.0

constant

Allows you to define constant searchable data for every instance of the class. Requires a Map value, which defines the constant’s `name and value or values and other options.

converter

The name of configured Converter used to convert this class to/from searchable text. A Compass default is used if not defined.

root

Defines whether class instances are kept in their own index. When false instances should be searchable components of another class, otherwise they are not saved in any index and cannot be searched for. Default is true.

spellCheck

Whether to include the instance searchable data is added to the spell check index. Either "`include"` or "exclude" to include or exclude from the spell-check index respectively. Default is "exclude".

subIndex

The name of the Lucene index for this class. By default, each searchable class (or hierarchy) gets its own complete Lucene index, with the same name as alias. (Compass calls these "sub-indexes".) This name is then used as the index’s on-disk directory name or as the table-table name, for example, depending on your index storage strategy. This option allows you to override the default behaviour. For example you could use the same sub-index for multiple different classes.

supportUnmarshall

Whether you need to get the data in the index back out again as domain classes. If false class instances are "marshalled" to the index, but they cannot be "unmarshalled" back into classes (ie the reverse). May be useful if you want to easily index your data but are accessing the index directly with Compass/Lucene yourself. Default is true.

Examples

// Define class mapping options the simple way
static searchable = {
    alias "foobar"
    subIndex "fb"
    constant name: "type", value: "some foobar"
    constant name: "noise", values: ["squawk", "shriek"]
}
// Define class mapping options with the "mapping" keyword;
// some options are defined within a Map option to "mapping"
// and others are given in a nested Closure
static searchable = {
    mapping alias: "foobar", subIndex: "fb", {
        constant name: "type", value: "some foobar"
        constant name: "noise", values: ["squawk", "shriek"]
    }
}
// Define class mapping options within the nested "mapping" Closure
static searchable = {
    mapping {
        boost 2.0
        spellCheck "include"
    }
}

Compass annotations

In order to use Compass annotations just define them in your classes, eg:

import org.compass.annotations.*

@Searchable(alias = 'user')
class User {
    static hasMany = [friends: User]

    @SearchableId
    Long id

    @SearchableProperty
    String name

    @SearchableReference(refAlias = 'user')
    Set friends
}

Normally with Compass you need a master XML config file - typically compass.cfg.xml - in which you declare these mapped classes. With the Searchable Plugin you can choose whether you want a compass.cfg.xml; if it is not found, it detects annotated domain classes automatically.

Note that when using annotations, Compass (well, Java really) needs actual properties for the annotations to annotate, so you might find you have to add things like the id property of the class and any Collection properties that are normally created dynamically by Grails.

Additionally, relationships in Grails are lazy by default, meaning that direct property access does not always work as you might expect (you might be accessing the property of a lazy proxy rather than the associated domain class) and getter/setter methods should be used instead. You may find you need to need to add an accessor = 'property' to your annotations like so:

@SearchableId(accessor = 'property')
Long id

which tells Compass to use getter/setter access rather than field access. Or you can just annotate the getter instead of the field.

Refer to the Compass docs [from here|http://www.compass-project.org] for more.

=n Compass Mapping XML

To use Compass Mapping XML, add a DomainClassName.cpm.xml file to your classpath.

Normally with Compass you need a master XML config file - typically compass.cfg.xml - in which you declare these mapped classes. With the Searchable Plugin you can choose whether you want a compass.cfg.xml; if it is not found, it detects your mapping XML files automatically.

Refer to the Compass docs from here for more.

INFO: The plugin currently configures the Compass mappings by generating this XML at runtime, so if you would like to see it, enable [debug logging|Searchable Plugin - Debugging].

Searchable Plugin - Managing the index

Thanks to Compass::GPS the Searchable Plugin mirrors changes made through Hibernate/GORM to the index, and this means that you normally don’t have to think about maintaining the index yourself.

But perhaps when an object is created/saved/updated/deleted with GORM/Hibernate is not when you want it to be indexed. Perhaps your application updates the DB with raw SQL or JDBC, which Compass::GPS isn’t aware of, or there are other applications updating the DB. Maybe you don’t like the mirroring feature and have disabled it (see Configuration).

In these cases you will need to maintain the index yourself, and Searchable Plugin provides a few methods to help with this.

Like the search methods, they come in SearchableService and searchable domain class varieties. The domain class methods restrict the operation to instances of that class (or hierarchy).

The methods are:

  • index - Indexes searchable class instances (adds them to the search index

  • unindex - Un-indexes searchable class instances (removes them from the search index)

  • reindex - Re-indexes searchable class instances (refreshes them in the search index)

There are also a few SearchableService-only methods:

Configuration

By default the plugin configures the search engine for you.

If you want to override any of these settings, or simply prefer the settings declared in your own codebase rather than within the plugin’s, install the configuration file:

grails install-searchable-config

from your project dir, then edit the generated file, myproject/grails-app/conf/Searchable.groovy.

It uses the same tech as Config.grooy so it supports per-environment config.

The name and syntax of this file has chaged in 0.5; it was previously called SearchableConfiguration.groovy. You can migrate your settings by just installing the newer config file and copy/pasting across.

With this file you can configure such things as

  • The index location (eg, file-system or RAM index)

  • Default search options (max, escape, reload, etc)

  • Default property mapping exclusions and formats

  • Enabling/disabling the bulk index and mirror changes features

  • Compass settings

See the docs in the [config file itself|https://svn.codehaus.org/grails-plugins/grails-searchable/trunk/src/conf/Searchable.groovy] for more.

Spring beans

Since 0.5.1, custom analyzers and converters can be defined as Spring beans.

Native Compass XML config

Since 0.4 the plugin also looks for a native Compass XML configuration file at the root of the classpath called compass.cfg.xml. If this file is present, Compass is configured with the settings defined in it, in addition to any that may be defined by the plugin’s own Searchable.groovy.

See the [XML configuration section|http://www.opensymphony.com/compass/versions/1.1/html/core-configuration.html#core-configuration-xml] of the Compass manual for more info and examples.

Analyzers

In order to make your data searchable it is typically analyzed.

When text is analyzed using Lucene’s StandardAnalyzer, for example, white-space and other irrelevant characters (eg punctation) are discarded, as are un-interesting words (eg, 'and', 'or', etc) and the remaining words are lower-cased. The input text is effectively normalized for the search index.

Additionally when you search with a query string, that too is analyzed. This process means that the terms you search on are normalized in the same way as the terms in the index.

Lucene includes many analyzers out of the box and you can also provide your own.

What we get with Compass

Compass acts as a registry of Analyzers, each identified by a name.

Compass provides two analyzers: "default" which is used for indexing and "search" which is used for searching (analyzing query strings).

They are both instances of Lucene’s StandardAnalyzer (or equivalent).

You can re-define both of these or define additional analyzers with new names.

Defining Analyzer implementations

You can define an analyzer with [#Compass settings] and (since 0.5.1) as a [#Spring bean].

Compass settings

The Compass settings can either be defined in the plugin’s configuration or in a native Compass configuration file.

Compass actually provides shortcut names for some of the standard Lucene analyzers, and this is a simple way to define them, eg:

Map compassSettings = [
    'compass.engine.analyzer.german.type': 'German'
]

Here "German" is a synonym provided by Compass for one of the standard Lucene analyzers and it has been named "german".

But you can also define your own implementations this way with a fully qualified class name:

Map compassSettings = [
    'compass.engine.analyzer.swedishChef.type': 'com.acme.lucene.analysis.SwedishChefAnalyzer'
]

See the Compass settings reference and "general discussion with XML examples" for the complete range of options.

Spring bean

Since 0.5.1

If you define a Spring bean in resources.xml or resources.groovy that is an instance of org.apache.lucene.analysis.Analyzer then it wil be automatically registered with Compass using the Spring bean name as it’s name.

This allows you to inject your analyzer with other Spring beans and configuration, eg

import com.acme.lucene.analysis.MyHtmlAnalyzer

beans = {
    htmlAnalyzer(MyHtmlAnalyzer) {
        context = someContext
        includeMeta = true
    }
}

defines an analyzer called "htmlAnalyzer", while

import org.apache.lucene.analysis.standard.StandardAnalyzer

beans = {
    'default'(StandardAnalyzer, new HashSet()) // there are now no stop words
}

re-defines the "default" analyzer so that it has no stop-words (and will not discard 'and', 'or', etc).

Using Analyzers

Indexing

For indexing purposes you define the analyzer in the mapping, either at the class level

class Book {
    static searchable = {
        analyzer 'bookAnalyzer'
    }
    String title
}

and/or at the property level

class Book {
    static searchable = {
        title analyzer: 'bookTitleAnalyzer'
    }
    String title
}

Property-level analyzers override class-level analyzers just for that property.

Note you can also use native Compass XML or annotations to map with custom analyzers.

Searching

You can say which analyzer you want to use on a per-query basis

def sr = Song.search("only the lonely", analyzer: 'songLyricsAnalyzer')

or with the plugin’s configuration you can choose a search analyzer for all search queries (unless overriden on a per-query basis).

    defaultMethodOptions = [
        search: [reload: false, escape: false, offset: 0, max: 10, defaultOperator: "and", analyzer: 'myAnalyzer'],
        suggestQuery: [userFriendly: true]
    ]

You could also simply redefine the "search" analyzer to achieve the same effect.

Converters

In order to transform your objects and their properties into searchable text, and from data stored in the index back into objects, Compass has the notion of converters.

Compass acts as a registry of converters, each identified by name.

Compass includes many converters itself, some responsible for converting entire class instances and some for individual properties.

Additionally the plugin includes a custom converter itself which supports Map<String, String> class property types, since Grails supports this as a persistent class property type.

Defining Converter implementations

You can define a converter with Compass settings and (since 0.5.1) as Spring beans.

Compass settings

The "Compass settings":http://www.compass-project.org/docs/2.1.0/reference/html/core-settings.html#config-converter can either be defined in the plugin’s configuration or in a native Compass configuration file.

In the plugin’s config you might do:

Map compassSettings = [
    'compass.converter.funkyConverter.type': 'com.acme.my.converters.MyFunkyConverter',
    'compass.converter.funkyConverter.registerClass': 'com.acme.my.special.classes.MyCoolDateTime'
]

which registers the class MyFunkyConverter as a converter with the name "funkyConverter" which can be configured to be used on properties of type MyCoolDateTime in a domain class.

See the "Compass settings":http://www.compass-project.org/docs/2.1.0/reference/html/core-settings.html#config-converter for the complete range of options.

Spring beans

If you define a Spring bean in resources.xml or resources.groovy that is an instance of org.compass.core.converter.Converter then it wil be automatically registered with Compass using the Spring bean name as it’s name.

This allows you to inject your analyzer with other Spring beans and configuration, eg

import com.acme.compass.converter.MyHtmlConverter
beans = {
    htmlConverter(MyHtmlConverter) {
        context = someContext
        includeMeta = true
    }
}

defines a converter called "htmlConverter".

Using Converters

Converters are defined in the class mapping, at either the class level

class Book {
    static searchable = {
        converter 'bookConverter'
    }
    String title
}

and/or at the property level

class Book {
    static searchable = {
        title converter: 'bookTitleConverter'
    }
    String title
}

Note you can also use native Compass XML or annotations to map with custom converters.

Debugging

Logging

Enable Searchable Plugin debug logging by adding the following to Config.groovy’s log4j section:

// log4j configuration
log4j = {
    ....

    debug 'org.codehaus.groovy.grails.plugins.searchable'

    // more detailed debug logging
    // trace 'org.compass'
}

Luke - for inspecting the index

[Luke|http://code.google.com/p/luke/] is a Swing GUI for Lucene indexes.

Use Luke to inspect the index and test queries.

FAQ

INFO: Contribute! This FAQ is a work in progress! Please add stuff that you think others may find helpful.

Negative numbers are indexed as positive

This is probably because the number is being "analyzed" during the index process.

This process is usually applied to searchable text to normalize it and remove what are normally useless words and characters (punctuation for example).

If you want to store the value of any searchable property exactly as it appears map that field with index: "not_analyzed".

class Idea {
    static searchable = {
        votes index: "not_analyzed"
    }
    int votes
    // ...
}

Number Range searches don’t return the expected results

Searching for objects with values for a numeric property in a certain range may return unexpected results.

Say you have the following domain class:

class Bug {
    static searchable = {
        votes index: "not_analyzed"
    }
    int votes = 0
    // ...
}

And you want to search for Bugs with between 0 and 20 votes. With the Searchable plugin you could do it this way

def results = Bug.search("votes:[0 TO 20]")

But you might get back bugs with votes with 100 votes! What gives?

Data in the search index is not typed like a database, everything is text. So comparisons are done lexicographically, rather than numerically as you would expect in this case.

The solution is to apply some formatting that pads the number with zeros. The "format" mapping option defines how the number is converted to searchable text in the index:

class Bug {
    static searchable = {
        votes index: "not_analyzed", format: "000000000"
    }
    int votes = 0
    // ...
}

With this mapping the search above works as you would expect.

Note: search indexes are like databases in some respects and you need to think about the storage of every piece of data in there. Giving something too much room may result in a larger than necessary search index (on disk). Compare the search index mapping to the database column to fine the right amount of padding for number formats.

Startup is slow

The plugin performs a synchronous bulk-index of all the searchable domain class instances in your app by default.

You have a few options:

Bulk index at startup

You can change the default bulk-index-on-startup setting with configuration: you can disable it or fork a new thread.

Even if you disable it, you can always call searchableService.index() to perform a complete index at any time

You can also perform a bulk-index in a separate thread easily like:

Thread.start {
    println "forked bulk index thread"
    searchableService.index()
    println "bulk index thread finished"
}

Which is a good thing since whilst performing a bulk-index Compass does not actually destroy the current index until the bulk-index is finished, so if you have a previous index you can do searches even while the index is being refreshed.

Disable mirroring during bootstrap

If mirroring is enabled (default is on) and you are creating domain classes in your BootStrap#init you can temporarily disable mirroring while the bootstrap process runs then enable it and perform a complete index:

class BootStrap {
    def searchableService

    def init = { servletContext ->
        searchableService.stopMirroring()

        for (i in 0..<10000) {
            def thingie = new Thingie(description: "this right here is a thingie and it's number ${i}")
            assert thingie.validate(), thingie.errors
            thingie.save()
        }

        searchableService.startMirroring()
        searchableService.indexAll()
    }

    def destroy = {
    }
}

Does Searchable work with multiple datasources?

It does, but you will need to add a few things to your application to get it working.

You will likely get the following error if you have multiple datasources, or a datasource with a non-default name (like dataSource_users ):

GrailsContextLoader Error initializing the application: No entities listed to be indexed, have you defined your entities correctly?
java.lang.IllegalArgumentException: No entities listed to be indexed, have you defined your entities correctly?

To correct this issue, add the following to resources.groovy:

import org.compass.gps.device.hibernate.HibernateGpsDevice
import grails.plugin.searchable.internal.compass.config.SessionFactoryLookup

beans = {
    compassGpsDevice(HibernateGpsDevice) { bean ->
        bean.destroyMethod = "stop"
        name = "hibernate"
        sessionFactory = { SessionFactoryLookup sfl ->
          sessionFactory = ref('sessionFactory_datasourceName')
        }
        fetchCount = 5000
    }
}

To add other dataSources with indexable classes, add the following:

anotherUniquecompassGpsDevice(HibernateGpsDevice) { bean ->
    bean.destroyMethod = "stop"
    name = "unqiueHibernateName"
    sessionFactory = { SessionFactoryLookup sfl ->
        sessionFactory = ref('sessionFactory_uniquedatasource')
    }
    fetchCount = 5000
  }

And finally add…​

import org.compass.gps.impl.SingleCompassGps

compassGps(SingleCompassGps) {
    compass = ref('compass')
    gpsDevices = [compassGpsDevice, anotherUniqueCompassGpsDevice]
}

(Thanks to mydigitalbricks.blogspot.com for this fix. Read the original blog post here.)

Methods

search

Summary

Finds domain object hits for a query.

Syntax

searchableService.search(String query)
searchableService.search(String query, Map options)
searchableService.search(Map options, String query) // same as previous
searchableService.search(Closure builder)
searchableService.search(Closure builder, Map options)
searchableService.search(Map options, Closure builder) // same as previous
DomainClass.search(String query)
DomainClass.search(String query, Map options)
DomainClass.search(Map options, String query) // same as previous
DomainClass.search(Closure builder)
DomainClass.search(Closure builder, Map options)
DomainClass.search(Map options, Closure builder) // same as previous

Description

Issues a query to the search engine and returns the result.

Normally that result ([return value|Searchable Plugin - Methods - search#Returns]) is a "search result" object, which contains a subset of hits and other data, but you can ask for a different result with the result option.

The query can be specified as either a [String|Searchable Plugin - Searching - String Queries] or [Closure|Searchable Plugin - Searching - Query Builder] parameter.

[Options|Searchable Plugin - Methods - search#Options] can be provided to modify the query or result.

Parameters

  • query - A [query String|Searchable Plugin - Searching - String Queries]

  • builder - A [query-buiding Closure|Searchable Plugin - Searching - Query Builder]

  • options - A Map of [options|Searchable Plugin - Methods - search#Options]

Options
Options affecting the search query

  • sort - The field to sort results by (default is 'SCORE'). [More|Searchable Plugin - Searching - Sorting]

  • order or direction - The sort order, only used with sort (default is 'auto'). [More|Searchable Plugin - Searching - Sorting]

Options for String queries

  • escape - Should special characters be escaped? Default is false. [More|Searchable Plugin - Searching - String Queries#Advanced String Query Options]

  • defaultProperty or defaultSearchProperty - The searchable property for un-prefixed terms. Default is "all". [More|Searchable Plugin - Searching - String Queries#Advanced String Query Options]

  • properties - The names of the class properties in which to search. [More|Searchable Plugin - Searching - String Queries#Advanced String Query Options]

  • defaultOperator - Either "and" or "or". Default is "and" unless set otherwise elsewhere. [More|Searchable Plugin - Searching - String Queries#Advanced String Query Options]

  • analyzer - The name of a query analyzer. [More|Searchable Plugin - Searching - String Queries#Advanced String Query Options]

  • parser or queryParser - The name of a query parser. [More|Searchable Plugin - Searching - String Queries#Advanced String Query Options]

Options affecting the return value

  • result - What should the method return? If defined, one of "searchResult", "top", "every" or "count". Default is "searchResult". (See [Returns|Searchable Plugin - Methods - search#Returns] below)

  • offset - The 0-based start result offset (default 0)

  • max - The maximum number of results to return (default 10). Only used with result: "searchResult"

  • reload - If true, reloads the objects from the database, attaching them to a Hibernate session, otherwise the objects are reconstructed from the index. Default is false

  • withHighlighter - A Closure instance that is called for each search result hit to support highlighting. [More|Searchable Plugin - Searching - Highlighting]

  • suggestQuery - Do you want a suggested query with that search result? If true or Map and result is undefined or "searchResult", adds a suggestedQuery property to the search result object. Use a Map value to define nested [suggestQuery options|Searchable Plugin - Methods - suggestQuery#options]. Use true to use the default [suggestQuery options|Searchable Plugin - Methods - suggestQuery#options].

Returns

By default (if result is undefined) or if result is "searchResult", returns a "search result" object containing a subset of objects matching the query, with the following properties:

  • results - A List of matching domain objects

  • scores - A List of scores: one for for each entry in results

  • total - The total number of hits

  • offset - The 0-based hit object offset

  • max - The maximum number of results requested. Note that this may be higher than results.size() if there were fewer hits than requested

  • suggestedQuery - An alternative query String, based on spelling suggestions. Only if suggestQuery is non-false

If result is "every", returns every domain object hit.

If result is "top", returns the first domain object hit.

If result is "count", returns the number of hits (as if you had used [countHits|Searchable Plugin - Methods - countHits]).

The order of the hits is either by relevance (the default) or [a sort you define|Searchable Plugin - Searching - Sorting].

If you prefer, you can use * searchTop(…​) instead of search(result: 'top', …​) * searchEvery(…​) instead of search(result: 'every', …​) * countHits(…​) instead of search(result: 'count', …​)

Examples

// Get the first page of up to 20 domain objects
// matching the query 'Chelsea Florist'
def searchResult = searchableService.search(
    "Chelsea Florist",
    [offset: 0, max: 20]
)
assert searchResult instanceof Map
println "${searchResult.total} hits:"
for (i in 0..<searchResult.results.size()) {
    println "${searchResult.offset + i + 1}: " +
        "${searchResult.results[i].toString()} " +
        "(score ${searchResult.scores[i]})"
}
// Find the lowest priced product matching the query
// '(laser OR L.A.S.E.R.) beam'
def product = Product.search(
    "(laser OR L.A.S.E.R.) beam",
    [sort: 'price', order: 'asc', result: 'top']
)
assert product instanceof Product
// Get all Articles that contain the terms
// 'police' OR 'doughnut', giving a higher score
// to items matching 'doughnut', and load from DB
def articles = Article.search(
    "Police doughnut^2.0",
    [reload: true, result: 'every']
)
assert articles.each { it instanceof Product }
// Count the number of domain objects matching 'cow pie'
def count = searchableService.countHits("cow pie")
println "There are ${count} hits for query 'cow pie'"
// Find other objects like the identified Book
def searchResult = searchableService.moreLikeThis(
    class: Book, id: 2l
)
assert searchResult instanceof Map
println "${searchResult?.results?.size()} similar items found"
// Check the spelling in a query
def suggestedQuery = Song.suggestQuery("living on a preyer")
println "Did you mean ${suggestedQuery}?"
// Get a "search result" object for the given
// closure-defined query across all
// searchable class instances
def searchResult = searchableService.search {
    fuzzy('name', 'lundon')
}
// Get every Book relevant to the closure-defined query and
// highlight the "title" and "summary" properties,
// keeping the highlights in a separate List
def highlights = []
def bookHighlighter = { highlighter, index, sr ->
    highlights[index] = [
        title: highlighter.fragment("title"),
        summary: highlighter.fragment("summary")
    ]
}
def books = Book.search(result: 'every', withHighlighter: bookHighlighter) {
    gt('averageReview', 3)
    queryString('learning techniques')
}
// With typical suggested query
def searchResult = searchableService.search("lambda", suggestQuery: true)
println "did you mean ${searchResult.suggestedQuery}?"
// With suggested query, non-default options
def searchResult = Recipie.search("bacon", suggestQuery: [escape: true, userFriendly: false])
println "did you mean ${searchResult.suggestedQuery}?"

countHits

Summary

Get the number of hits for a query.

Syntax

searchableService.countHits(String query)
searchableService.countHits(String queryString, Map options)
searchableService.countHits(Map options, String queryString) // same as previous
searchableService.countHits(Closure builder)
searchableService.countHits(Closure builder, Map options)
searchableService.countHits(Map options, Closure builder) // same as previous
DomainClass.countHits(String query)
DomainClass.countHits(String queryString, Map options)
DomainClass.countHits(Map options, String queryString) // same as previous
DomainClass.countHits(Closure builder)
DomainClass.countHits(Closure builder, Map options)
DomainClass.countHits(Map options, Closure builder) // same as previous

Description

Issues a query to the search engine and returns the number of hits.

This method is just like [search|Searchable Plugin - Methods - search], except that it always return the hit count rather than relevant class instances. In fact you can use search(…​, result: 'count') and countHits(…​) interchangably.

[Options|Searchable Plugin - Methods - countHits#Options] can be provided to modify the query.

Parameters

  • queryString - A [query String|Searchable Plugin - Searching - String Queries]

  • builderClosure - A [query-buiding Closure|Searchable Plugin - Searching - Query Builder]

  • options - A Map of [options|Searchable Plugin - Methods - countHits#Options]

Options
Options for String queries

  • escape - Should special characters be escaped? Default is false. [More|Searchable Plugin - Searching - String Queries#Advanced String Query Options]

  • defaultProperty or defaultSearchProperty - The searchable property for un-prefixed terms. Default is all. [More|Searchable Plugin - Searching - String Queries#Advanced String Query Options]

  • properties - The names of the class properties in which to search. [More|Searchable Plugin - Searching - String Queries#Advanced String Query Options]

  • defaultOperator - Either "and" or "or". Default is "and" unless set otherwise elsewhere. [More|Searchable Plugin - Searching - String Queries#Advanced String Query Options]

  • analyzer - The name of a query analyzer. [More|Searchable Plugin - Searching - String Queries#Advanced String Query Options]

  • parser or queryParser - The name of a query parser. [More|Searchable Plugin - Searching - String Queries#Advanced String Query Options]

Returns

The number of hits for the query

Examples

// count hits across all searchable class instances
def count = searchableService.countHits("samuri")
// count hits within Show instances,
// escaping reserved string query characters
def count = Show.countHits("CSI [Las Vegas]", escape: true)
// count hits across all searchable class instances
def count = searchableService.countHits("CSI [Miami]", [escape: true])
// count hits across all searchable class instances
// defining the query with a builder closure
def count = searchableService.countHits {
    term("format", "MP3")
    multiPhrase("title") {
        add("Wrecking")
        add("Ball")
    }
}
// count hits for Show instance with a query
// built with a closure
def count = Show.countHits {
    term("keywords", "crime")
    term("keywords", "drama")
    queryString("ongoing love interest subplot", [defaultSearchProperty: "notes"])
}

moreLikeThis

Summary

Finds similar objects to the indicated searchable domain class instance.

Syntax

domainObject.moreLikeThis()
domainObject.moreLikeThis(Map options)
DomainClass.moreLikeThis(Map options)
DomainClass.moreLikeThis(Serializable id)
DomainClass.moreLikeThis(Serializable id, Map options)
DomainClass.moreLikeThis(DomainClass domainObject)
DomainClass.moreLikeThis(DomainClass domainObject, Map options)
searchableService.moreLikeThis(Map options)
searchableService.moreLikeThis(Class domainClass, Serializable id)
searchableService.moreLikeThis(Class domainClass, Serializable id, Map options)
searchableService.moreLikeThis(DomainClass domainObject)
searchableService.moreLikeThis(DomainClass domainObject, Map options)

Description

Issues a "more-like-this" query to the search engine and returns the result.

Note that you do not create the more-like-this query, it is created by the search engine for a given searchable class instance. You do need to provide or identify the class instance though.

Conceptually (apart from the difference in the query) this method is like [search|Searchable Plugin - Methods - search], and you could think of it like a specialised "search" for more-like-some-object.

Unsurprisingly then they share code and this method has many options in common with [search|Searchable Plugin - Methods - search] itself, and also adds [more of its own|Searchable Plugin - Methods - moreLikeThis#Options].

To use moreLikeThis you need to add termVector to the searchable domain class’s ["all" property mapping|Searchable Plugin - Mapping - all] and rebuild the index. See the [example here|Searchable Plugin - Mapping - all#Examples].

Parameters

  • domainClass - The searchable domain class of the object to find more like

  • id - The id of the searchable domain object to find more like

  • domainObject - A searchable domain class instance to find more like

  • options - A Map of [options|Searchable Plugin - Methods - moreLikeThis#Options]

Options
Options affecting the more-like-this query

  • sort - The field to sort results by (default is 'SCORE'). [More|Searchable Plugin - Searching - Sorting]

  • order or direction - The sort order, only used with sort (default is 'auto'). [More|Searchable Plugin - Searching - Sorting]

If you prefer you can identify the domain class instance with options alone:

  • class - The searchable domain class to find more like. Not required for DomainClass.moreLikeThis(…​)

  • id - The searchable domain object identifier to find more like

These options tweak the more-like-this query:

  • aliases - Exposes the Compass aliases option allowing you to narrow the search space by alias. A List of aliases, eg \['post', 'comment'\]

  • boost - Sets whether to boost terms in query based on "score" or not. true or false

  • maxNumTokensParsed - The maximum number of tokens to parse in each example doc field that is not stored with TermVector support

  • maxQueryTerms - The maximum number of query terms that will be included in any generated query.

  • maxWordLen - The maximum word length above which words will be ignored. Set this to 0 for no maximum word length. The default is 0.

  • minResourceFreq - The frequency at which words will be ignored which do not occur in at least this many resources. Defaults to 5

  • minTermFreq - The frequency below which terms will be ignored in the source doc. Defaults to 2.

  • minWordLen - The minimum word length below which words will be ignored. Set this to 0 for no minimum word length. Default is 0.

  • properties - Limits the search to these class properties. A List of property names, eg \['post', 'title'\].

  • stopWords - A List of words that are not considered when comparing items for similarity. This is used in addition to any stop-words your analyzer has

  • subIndexes - A List of sub-indexes to search in, eg, \['bookmark'\]

Options affecting the return value

  • result - What should the method return? If defined, one of "searchResult", "top", "every" or "count". Default is "searchResult". (See [Returns|Searchable Plugin - Methods - moreLikeThis#Returns] below)

  • offset - The 0-based start result offset (default 0)

  • max - The maximum number of results to return (default 10). Only used with result: "searchResult"

  • reload - If true, reloads the objects from the database, attaching them to a Hibernate session, otherwise the objects are reconstructed from the index. Default is false

  • withHighlighter - A Closure instance that is called for each search result hit to support highlighting. [More|Searchable Plugin - Searching - Highlighting]

There are also some [additional options for string queries|Searchable Plugin - Searching - String Queries#Advanced String Query Options].

Returns

By default (if result is undefined) or if result is "searchResult", returns a "search result" object containing a subset of similar objects, with the following properties:

  • results - A List of domain objects

  • scores - A List of scores: one for for each entry in results

  • total - The total number of hits

  • offset - The 0-based hit object offset

  • max - The maximum number of results requested. Note that this may be higher than results.size() if there were fewer hits than requested

If result is "every", returns every similar object.

If result is "top", returns the first similar domain object.

If result is "count", returns the number of similar domain objects.

The order of the hits is either by relevance (the default) or [a sort you define|Searchable Plugin - Searching - Sorting].

Examples

// Get more like the identified Artist
def searchResult = searchableService.moreLikeThis(Artist, 101l)

assert searchResult instanceof Map
println "${searchResult.total} hits:"
for (i in 0..<searchResult.results.size()) {
    println "${searchResult.offset + i + 1}: " +
        "${searchResult.results[i].toString()} " +
        "(score ${searchResult.scores[i]})"
}
// Find similar items to a Product
def products = product.moreLikeThis(result: 'every')
println "We can also recommend ${products.size()} other items"
// Get a third page of more-like-this results for the
// Catalogue instance defined by options (rather than
// formal parameters)
def searchResult = searchableService.moreLikThis(
    class: Catalogue, id: 312l, offset: 20, max: 10
)
// Get the most similar item to the given searchable class instance
def mostSimilar = searchableService.moreLikeThis(item, result: 'top')
// Get similar books to the instance identified with the
// the *id* option (rather than formal parameters),
// having the method return *all* hits, and
// saving highlights in an external List
def highlights = []
def bookHighlighter = { highlighter, index, sr ->
    highlights[index] = [
        title: highlighter.fragment("title"),
        summary: highlighter.fragment("summary")
    ]
}
def books = Book.moreLikeThis(
    id: 20l, result: 'every', withHighlighter: bookHighlighter
)

suggestQuery

Summary

Suggest a new search query based on spelling

Syntax

searchableService.suggestQuery(String query)
searchableService.suggestQuery(String query, Map options)
searchableService.suggestQuery(Map options, String query) // same as previous
DomainClass.suggestQuery(String query)
DomainClass.suggestQuery(String query, Map options)
DomainClass.suggestQuery(Map options, String query) // same as previous

Description

Uses the spelling index to suggest an alternative query.

You need to add some sort of spellCheck mapping for domain classes/properties that you wish to include in spelling index, either at the [class level|Searchable Plugin - Mapping - Class Mapping] or [property level|Searchable Plugin - Mapping - Searchable Property].

You can programmatically [re-build the spelling index|Searchable Plugin - Methods - rebuildSpellingSuggestions] if you like. This might be useful for testing, but otherwise don’t worry, it is periodically refreshed automatically by Compass.

INFO: You can also call [search|Searchable Plugin - Methods - search] with a suggestQuery [option|Searchable Plugin - Methods - search#Options], which returns search results for the original query, along with a suggested query.

The plugin provides a helper class to highlight the different terms in the suggested vs original query, a la Google, etc. [More|Searchable Plugin - SearchableController and view#Suggested queries]

Parameters

  • query - A [query String|Searchable Plugin - Searching - String Queries]

  • options - A Map of [options|Searchable Plugin - Methods - suggestQuery#options]

options
Options affecting the search query

  • escape - Should special characters be escaped? Default is false. [More|Searchable Plugin - Searching - String Queries#Advanced String Query Options]

Options affecting the return value

  • userFriendly - Should the suggested query look like a user-query? When false returns queries as they are re-written and suggested by the search engine. Default is true.

  • emulateCapitalisation - When false returns a query as re-written and suggested by the search engine, which is usually all lowercase, when true tries to emulate the upper-casing of the original query. Default is true

  • allowSame - Can the method return the same query as the given query? This is possible if no better suggestions are found. If false, the method returns null instead of allowing the same query. Default is true

Returns

A suggested query String or null

Examples

// Get a suggested query, using all available class instances
// for spelling suggestions
def suggestedQuery = searchableService.suggestQuery("grate briton")
println "Did you mean ${suggestedQuery}"
// Get a suggested query using only searchable text from "Player"
// instances for spelling suggestions, allowing for bad
// characters and disabling the user-friendly options
def suggestedQuery = Player.suggestQuery(
    "kartoon",
    escape: true, userFriendly: false, emulateCapitalisation: false
)
println "Did you mean ${suggestedQuery}"

termFreqs

Summary

Get term frequencies for the terms in the search index.

Syntax

searchableService.termFreqs()
searchableService.termFreqs(String propertyNames...)
searchableService.termFreqs(Map options, String propertyNames...)
searchableService.termFreqs(Map options)
DomainClass.termFreqs()
DomainClass.termFreqs(String propertyNames...)
DomainClass.termFreqs(Map options, String propertyNames...)
DomainClass.termFreqs(Map options)

Description

{info:title=What’s a term frequency?}

A term frequency represents a term in the index (where a term is normally a word) and its frequency in the index (number of occurances).

Term frequencies (often abbreviated to "term freqs") is just the plural of that, ie, a collection of terms and their respective frequencies.

With term frequencies you can discover which terms are more common and which are less common, or find out the exact number of occurances of each. {info}

Term frequencies can be limited to the subset of class hierarchy, if called as a domain class static method or with a class option, and/or class properties, if the propertyNames parameter or properties option is provided.

You can also normalize the returned frequencies, which can make tag clouds easy, for example.

TODO show/link to tag cloud example

Parameters

  • propertyNames - An arbitrary number of domain class property names. If defined, only terms for these properties are returned

  • options - A Map of [options|Searchable Plugin - Methods - termFreqs#options]

options

  • properties - A List of property names; use this to get term freqs for multiple properties

  • size - The maximum number of term freqs to return. Default is all

  • normalise or normalize - A Groovy Range used to normalise the frequencies. Without this option the frequencies of the returned term freqs are the actual number of occurences for the term in the index.

  • class - The class to restrict the term freqs to

  • sort - Sorts the term frequencies; either "term" to sort alphabetically by term or "freq" to sort by highest frequency first. Default is "freq"

Returns

An array of CompassTermFreq, each with the following methods:

  • getTerm - returns the term

  • getFreq - returns the frequency

  • getProperty - returns the searchable property from which the term comes

Examples

// print all Book term frequencies
def termFreqs = Book.termFreqs()
termFreqs.each {
    println "${it.term} occurs ${it.freq} times in the index for Book instances"
}
// get Book term frequencies for Book#title
def termFreqs = Book.termFreqs("title")
// get Book term frequencies for Book#title,
// limiting the size to 100 and normalising the frequencies
// between 0 (minimum) and 1 (maximum)
def termFreqs = Book.termFreqs("title", size: 100, normalize: 0..1)
// get terms from all properties in the index,
// sorting by term and limited to the Author class
def termFreqs = searchableService.search(class: Author, sort: "term")
// get terms from searchable "title" and "description" properties
// in the index, limiting and normalsing
def termFreqs = searchableService.search(
    properties: ["title", "description"], size: 1000, normalise: 0..1
)

index

Summary

Indexes searchable class instances (adds them to the search index)

Syntax

domainObject.index()
DomainClass.index()
DomainClass.index(DomainClass instances...)
DomainClass.index(Serializable ids...)
searchableService.index()
searchableService.index(Map options)
searchableService.index(DomainClass instances...)
searchableService.index(Serializable ids..)
searchableService.index(Map options, Serializable ids...)

Description

When called as a domain class instance method, indexes the instance.

When called as a domain class static method with no arguments, indexes all instances for that class (or hierarchy).

When called as a SearchableService method with no arguments, indexes all searchable class instances.

In other invocations, indexes the searchable class instances you provide or identify.

Parameters

  • instances - One or more searchable class instances

  • ids - One or more searchable class instance ids

  • options - A Map of [options|Searchable Plugin - Methods - index#options]

options

  • class - a searchable class; use this option with the SearchableService method and ids parameter or as an alternative to the domain-class method

Returns

No meaningful value

Examples

// Index a Post instance
post.index()
// Add some Menus to the index
Menu.index(m1, m2)
// Add all Countries to the index
Country.index()
// Index everything
searchableService.index()
// Index the identified Book
searchableService.index(class: Book, 1l)
// Index the given Tool
searchableService.index(tool)

unindex

Summary

Un-indexes searchable class instances (removes them from the search index)

Syntax

domainObject.unindex()
DomainClass.unindex()
DomainClass.unindex(DomainClass instances...)
DomainClass.unindex(Serializable ids...)
searchableService.unindex()
searchableService.unindex(Map options)
searchableService.unindex(DomainClass instances...)
searchableService.unindex(Serializable ids..)
searchableService.unindex(Map options, Serializable ids...)

Description

When called as a domain class instance method, un-indexes the instance.

When called as a domain class static method with no arguments, un-indexes all instances for that class (or hierarchy).

When called as a SearchableService method with no arguments, un-indexes all searchable class instances.

In other invocations, un-indexes the searchable class instances you provide or identify.

Parameters

  • instances - One or more searchable class instances

  • ids - One or more searchable class instance ids

  • options - A Map of [options|Searchable Plugin - Methods - unindex#options]

options

  • class - a searchable class; use this option with the SearchableService method and ids parameter or as an alternative to the domain-class method

Returns

No meaningful value

Examples

// Unindex a Post instance
post.unindex()
// Add some Menus to the index
Menu.unindex(m1, m2)
// Add all Countries to the index
Country.unindex()
// Unindex everything
searchableService.unindex()
// Unindex the identified Book
searchableService.unindex(class: Book, 1l)
// Unindex the given Tool
searchableService.unindex(tool)

reindex

Summary

Re-indexes searchable class instances (refreshes them in the search index)

Syntax

domainObject.reindex()
DomainClass.reindex()
DomainClass.reindex(DomainClass instances...)
DomainClass.reindex(Serializable ids...)
searchableService.reindex()
searchableService.reindex(Map options)
searchableService.reindex(DomainClass instances...)
searchableService.reindex(Serializable ids..)
searchableService.reindex(Map options, Serializable ids...)

Description

When called as a domain class instance method, re-indexes the instance.

When called as a domain class static method with no arguments, re-indexes all instances for that class (or hierarchy).

When called as a SearchableService method with no arguments, re-indexes all searchable class instances.

In other invocations, re-indexes the searchable class instances you provide or identify.

Parameters

  • instances - One or more searchable class instances

  • ids - One or more searchable class instance ids

  • options - A Map of [options|Searchable Plugin - Methods - reindex#options]

options

  • class - a searchable class; use this option with the SearchableService method and ids parameter or as an alternative to the domain-class method

Returns

No meaningful value

Examples

// Reindex a Post instance
post.reindex()
// Add some Menus to the index
Menu.reindex(m1, m2)
// Add all Countries to the index
Country.reindex()
// Reindex everything
searchableService.reindex()
// Reindex the identified Book
searchableService.reindex(class: Book, 1l)
// Reindex the given Tool
searchableService.reindex(tool)

rebuildSpellingSuggestions

Summary

Re-builds the spelling suggestions index

Syntax

searchableService.rebuildSpellingSuggestions()
searchableService.rebuildSpellingSuggestions(options)

Description

A spelling suggestions index is required to use [query suggestions|Searchable Plugin - Methods - suggestQuery].

This method allows you to re-build the spelling suggestions index(es) on demand.

The spelling suggestions feature is provided by Compass and normally takes care of itself. In other words, you do not need to re-build the spelling index in normal circumstances.

However it may be useful in certain scenarios, eg, in tests.

Parameters

  • options - A Map of [options|#Options]

Options

  • fork - If true, forks a new thread and returns immediately. May not be used with subIndex. Default is false, in which case the method only returns when the re-build is finished

  • subIndex - The sub-index of a searchable class to re-build the spelling suggestions for. May not be used with fork. Default is to re-build the spelling suggestions indexes for all classes.

Examples

// Rebuild in a backgroudn thread
searchableService.rebuildSpellingSuggestions(fork: true)
// Rebuild for just one sub-index (maps to one class hierarchy)
searchableService.rebuildSpellingSuggestions(subIndex: "House")

startMirroring

Summary

Starts the mirror-changes service

Syntax

searchableService.startMirroring()

Description

By default changes made through GORM/Hibernate are mirrored to the search index.

If you have disabled this feature in the configuration, or if you previously [stopped the mirror-changes service|Searchable Plugin - Methods - stopMirroring], you can start it with this method.

Examples

searchableService.startMirroring()

stopMirroring

Summary

Stops the mirror-changes service

Syntax

searchableService.stopMirroring()

Description

By default changes made through GORM/Hibernate are mirrored to the search index.

This feature can be controlled by configuration, but this method allows you to stop mirroring changes.

You can start mirroring changes again with the [startMirroring|Searchable Plugin - Methods - startMirroring] method.

Examples

searchableService.stopMirroring()

Releases

0.5.5

Lastest version, recommended if you are using Grails 1.1.1.

0.5.1

This release fixes some bugs and includes new features. It is a recommended upgrade for all users.

It has been tested with Grails 1.0.3, 1.0.4 and 1.1-beta1.

JIRA release notes are "here":http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=11450&styleName=Html&version=14751

0.5

This release includes several bug fixes, adds few new search methods, exposes more Compass mapping options in the mapping DSL, uses the standard Grails config notation, and is now better tested :-)

This release has been tested and confirmed working on Grails 1.0.4 and 1.0.3. It may or may not work on earlier versions.

If you are upgrading from an older version of the plugin, please read the [#Deprecated] and [#Bundles libs] sections below.

New domain class/SearchableService methods and options

moreLikeThis

Finds similar objects to the indicated searchable domain class instance. [Docs|Searchable Plugin - Methods - moreLikeThis]

suggestQuery

Suggest a new search query based on spelling. [Docs|Searchable Plugin - Methods - suggestQuery]

search

As an alternative to the search, searchTop, searchEvery and countHits methods you can pass a new result option to the [search|Searchable Plugin - Methods - search] method to indicate what to return.

Additionally the search method now supports a suggestQuery option, which returns a suggested query along with search results for the original query. [Docs|Searchable Plugin - Methods - search]

Mapping DSL

The [mapping DSL|Searchable Plugin - Mapping - Mapping DSL] has better support for Compass [Searchable Property|Searchable Plugin - Mapping - Searchable Property], [Reference|Searchable Plugin - Mapping - Searchable Reference] and [Component|Searchable Plugin - Mapping - Searchable Component].

Improved [Searchable Id|Searchable Plugin - Mapping - Searchable Id] support.

Support for [class mapping|Searchable Plugin - Mapping - Class Mapping] including [constant|Searchable Plugin - Mapping - constant] meta data and the [all|Searchable Plugin - Mapping - all] field.

Config improvements

The plugin’s configuration file now uses the same tech as your Config.groovy, so it now supports per-environment settings.

Bugs fixed

GRAILSPLUGINS-254 - Trying to marshall null id \[id\]

GRAILSPLUGINS-353 - org.hibernate.AssertionFailure collection was not processed by flush after calling Domain.properties = params

Other New and Noteworthy

GRAILSPLUGINS-464 - Automatically unlocks locked indexes on startup

GRAILSPLUGINS-299 - The [index|Searchable Plugin - Methods - index] method can be used for all instances of a class

This release splits the plugin into two versions, one called "searchable" for JDK 1.5+, and another called "searchable14" for JDK 1.4 users. They are almost identical except for a few files and will be maintained and released in tandem as long as Grails itself supports JDK 1.4.

The plugin itself now has its own functional test suite, which better reflects the multitude of Grails domain class possibilities for mapping, searching etc.

And it is now built on a (private) Continuous Integration server against the current production and development branches of Grails.

Deprecated

This release deprecates a few features which will be removed in the next point release (0.6).

The index management methods indexAll, unindexAll and reindexAll are deprecated; please use index, unindex, reindex instead, which provide the same functionality.

The SearchableConfiguration.groovy file is deprecated. If you currently have a SearchableConfiguration.groovy, run grails install-searchable-config to add a copy of the new config file to your project, place your settings in the new file and delete the old.

The defaultSearchOptions confg setting allowed you to define default options for your domain class/SearchableService search method. This idea has been expanded to accomodate the other Searchable methods, and you now define these defaults on a per-method basis in the defaultMethodOptions config setting. (See the new config file for details.)

Bundles libs

This release includes Compass 2.1.0 and Lucene 2.4.

Indexes based on an older version of the plugin/Compass/Lucene should be rebuilt.

If you have your own Lucene/Compass extensions, you wish to consult their release notes here [Compass|http://svn.compass-project.org/svn/compass/branches/2\_1/upgrade.txt] and here [Lucene|http://svn.apache.org/repos/asf/lucene/java/tags/lucene\_2\_4\_0/CHANGES.txt].

0.4.1

This is a maintenance release that fixes a few bugs - see JIRA for details: http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=11450&styleName=Html&version=14142

It bundles a patched version of Compass 1.2.2 specifically for GRAILSPLUGINS-254. I (Maurice) will give the code to the Compass project and hope they will include it in future versions of Compass.