org.compass.core.lucene.engine.transaction.async
Class AsyncTransactionProcessorFactory

java.lang.Object
  extended by org.compass.core.lucene.engine.transaction.async.AsyncTransactionProcessorFactory
All Implemented Interfaces:
CompassConfigurable, SearchEngineFactoryAware, TransactionProcessorFactory

public class AsyncTransactionProcessorFactory
extends Object
implements TransactionProcessorFactory, CompassConfigurable, SearchEngineFactoryAware

A transaction processor that created AsyncTransactionProcessor instances. Supports async execution of transactions against the index. A transaction (which includes several dirty operations) is packaged into a single operation which is then applied to the index asynchronously.

Note, when several instances of Compass are running using this transaction processor, order of transactions is not maintained, which might result in out of order transaction being applied to the index.

The number of transactions that have not been processed (backlog) are bounded and default to 10. If the processor is falling behind in processing transactions, commit operations will block until the backlog lowers below its threshold. The backlog can be set using the LuceneEnvironment.Transaction.Processor.Async.BACKLOG. Commit operations will block by default for 10 seconds in order for the backlog to lower below its threshold. It can be changed using the LuceneEnvironment.Transaction.Processor.Async.ADD_TIMEOUT setting.

Processing of transactions is done by a background thread that waits for transactions. Once there is a transaction to process, it will first try to wait for additional transactions. It will block for 100 milliseconds (configurable using LuceneEnvironment.Transaction.Processor.Async.BATCH_JOBS_TIMEOUT), and if one was added, will wait again up to 5 times (configurable using LuceneEnvironment.Transaction.Processor.Async.BATCH_JOBS_SIZE). Once batch jobs based on timeout is done, the processor will try to get up to 5 more transactions in a non blocking manner (configurable using LuceneEnvironment.Transaction.Processor.Async.NON_BLOCKING_BATCH_JOBS_SIZE).

When all transaction jobs are accumulated, the processor starts up to 5 threads (configurable using LuceneEnvironment.Transaction.Processor.Async.CONCURRENCY_LEVEL) in order to process all the transaction jobs against the index. Hashing of actual operation (create/update/delete) can either be done based on uid (of the resource) or sub index. By default, hashing is done based on uid and can be configured using LuceneEnvironment.Transaction.Processor.Async.HASHING.

When the transaction processor closes, by default it will wait for all the transactions to finish. In order to disable it, the LuceneEnvironment.Transaction.Processor.Async.PROCESS_BEFORE_CLOSE setting should be set to false.

Author:
kimchy

Constructor Summary
AsyncTransactionProcessorFactory()
           
 
Method Summary
 void add(TransactionJobs jobs)
          Adds the TransactionJobs to be processed asynchronously.
 void close()
          Closes the transaction processor.
 void configure(CompassSettings settings)
          Configure using the given settings.
 TransactionProcessor create(LuceneSearchEngine searchEngine)
          Creates a new AsyncTransactionProcessor.
protected  boolean isClearCacheOnCommit()
           
 boolean isThreadSafe()
          Async transaction processor is not thread safe.
 boolean remove(TransactionJobs jobs)
          Removed (if still pending) the given TransactionJobs from being processed.
 void setSearchEngineFactory(SearchEngineFactory searchEngineFactory)
          Injects the search engine factory.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

AsyncTransactionProcessorFactory

public AsyncTransactionProcessorFactory()
Method Detail

setSearchEngineFactory

public void setSearchEngineFactory(SearchEngineFactory searchEngineFactory)
Description copied from interface: SearchEngineFactoryAware
Injects the search engine factory.

Specified by:
setSearchEngineFactory in interface SearchEngineFactoryAware

configure

public void configure(CompassSettings settings)
               throws CompassException
Description copied from interface: CompassConfigurable
Configure using the given settings.

Specified by:
configure in interface CompassConfigurable
Parameters:
settings - The settings for the configured object
Throws:
CompassException

close

public void close()
Closes the transaction processor. Will wait for ongoing transactions if the LuceneEnvironment.Transaction.Processor.Async.PROCESS_BEFORE_CLOSE is set to true (the default).

Specified by:
close in interface TransactionProcessorFactory

create

public TransactionProcessor create(LuceneSearchEngine searchEngine)
Creates a new AsyncTransactionProcessor.

Specified by:
create in interface TransactionProcessorFactory

isThreadSafe

public boolean isThreadSafe()
Async transaction processor is not thread safe.

Specified by:
isThreadSafe in interface TransactionProcessorFactory

remove

public boolean remove(TransactionJobs jobs)
               throws SearchEngineException
Removed (if still pending) the given TransactionJobs from being processed.

Throws:
SearchEngineException

add

public void add(TransactionJobs jobs)
         throws SearchEngineException
Adds the TransactionJobs to be processed asynchronously. If a procesing threads has not started, will start it (it is started lazily so if the async transaction processor is not used, it won't incur any overhead).

The addition of TransactionJobs is "offered" to a blocking queue, waiting until the queue if cleared in case it is full. This will cause a transaction commit to block if the backlog is full. The time to wait can be controlled using LuceneEnvironment.Transaction.Processor.Async.ADD_TIMEOUT and defaults to 10 seconds.

Throws:
SearchEngineException

isClearCacheOnCommit

protected boolean isClearCacheOnCommit()


Copyright (c) 2004-2009 The Compass Project.