JBoss Cache can use a
CacheLoader
to back up the in-memory cache to a backend datastore.
If JBoss Cache is configured with a cache loader, then the following features are provided:
- Whenever a cache element is accessed, and that element is not in the cache (e.g. due to eviction or due to server restart), then the cache loader transparently loads the element into the cache if found in the backend store.
- Whenever an element is modified, added or removed, then that
modification is persisted in the backend store via the cache loader. If
transactions are used, all modifications created within a transaction
are persisted. To this end, the
CacheLoadertakes part in the two phase commit protocol run by the transaction manager, although it does not do so explicitly.
...
<!-- Cache loader config block -->
<!-- if passivation is true, only the first cache loader is used; the rest are ignored -->
<loaders passivation="false" shared="false">
<preload>
<!-- Fqns to preload -->
<node fqn="/some/stuff"/>
</preload>
<!-- if passivation is true, only the first cache loader is used; the rest are ignored -->
<loader class="org.jboss.cache.loader.JDBCCacheLoader" async="false" fetchPersistentState="true"
ignoreModifications="false" purgeOnStartup="false">
<properties>
cache.jdbc.driver=com.mysql.jdbc.Driver
cache.jdbc.url=jdbc:mysql://localhost:3306/jbossdb
cache.jdbc.user=root
cache.jdbc.password=
</properties>
</loader>
</loaders>
The currently available implementations shipped with JBoss Cache are as follows.
- Due to the way the FileCacheLoader represents a tree structure on disk (directories and files) traversal is inefficient for deep trees.
- Usage on shared filesystems like NFS, Windows shares, etc. should be avoided as these do not implement proper file locking and can cause data corruption.
- Usage with an isolation level of NONE can cause corrupt writes as multiple threads attempt to write to the same file.
- File systems are inherently not transactional, so when attempting to use your cache in a transactional context, failures when writing to the file (which happens during the commit phase) cannot be recovered.
BdbjeCacheLoader, which is a cache loader implementation based on the Oracle/Sleepycat's BerkeleyDB Java Edition .JdbmCacheLoader, which is a cache loader implementation based on the JDBM engine , a fast and free alternative to BerkeleyDB.
Note that the BerkeleyDB implementation is much more efficient than the filesystem-based implementation, and provides transactional guarantees, but requires a commercial license if distributed with an application (see http://www.oracle.com/database/berkeley-db/index.html for details).
See this wiki page for configuration tips with specific database systems.
Table and column names as well as column types are configurable with the following properties.
- cache.jdbc.table.name
- the name
of the table. Can be prepended with schema name for the given table:
{schema_name}.{table_name}. The default value is 'jbosscache'. - cache.jdbc.table.primarykey - the name of the primary key for the table. The default value is 'jbosscache_pk'.
- cache.jdbc.table.create - can be true or false. Indicates whether to create the table during startup. If true, the table is created if it doesn't already exist. The default value is true.
- cache.jdbc.table.drop - can be true or false. Indicates whether to drop the table during shutdown. The default value is true.
- cache.jdbc.fqn.column - FQN column name. The default value is 'fqn'.
- cache.jdbc.fqn.type - FQN column type. The default value is 'varchar(255)'.
- cache.jdbc.node.column - node contents column name. The default value is 'node'.
- cache.jdbc.node.type - node contents column type. The default value is 'blob'. This type must specify a valid binary data type for the database being used.
You can also set any c3p0 parameters in the same cache loader properties section but don't
forget
to
start the property name with 'c3p0.'. To find a list of available properties, please check the
c3p0 documentation for the c3p0 library version distributed in
c3p0:JDBC DataSources/Resource Pools
.
Also, in order to provide quick and easy way to try out different pooling
parameters, any of these properties can be set via a System property overriding any values these
properties might have in the JBoss Cache XML configuration file, for example:
-Dc3p0.maxPoolSize=20
.
If a c3p0 property is not defined in either the configuration file or as a System property,
default
value, as indicated in the c3p0 documentation, will apply.
<loaders passivation="false" shared="false">
<preload>
<node fqn="/some/stuff"/>
</preload>
<!-- if passivation is true, only the first cache loader is used; the rest are ignored -->
<loader class="org.jboss.cache.loader.JDBCCacheLoader" async="false" fetchPersistentState="true"
ignoreModifications="false" purgeOnStartup="false">
<properties>
cache.jdbc.table.name=jbosscache
cache.jdbc.table.create=true
cache.jdbc.table.drop=true
cache.jdbc.table.primarykey=jbosscache_pk
cache.jdbc.fqn.column=fqn
cache.jdbc.fqn.type=VARCHAR(255)
cache.jdbc.node.column=node
cache.jdbc.node.type=BLOB
cache.jdbc.parent.column=parent
cache.jdbc.driver=oracle.jdbc.OracleDriver
cache.jdbc.url=jdbc:oracle:thin:@localhost:1521:JBOSSDB
cache.jdbc.user=SCOTT
cache.jdbc.password=TIGER
</properties>
</loader>
</loaders>
<loaders passivation="false" shared="false">
<preload>
<node fqn="/some/stuff"/>
</preload>
<!-- if passivation is true, only the first cache loader is used; the rest are ignored -->
<loader class="org.jboss.cache.loader.JDBCCacheLoader" async="false" fetchPersistentState="true"
ignoreModifications="false" purgeOnStartup="false">
<properties>
cache.jdbc.datasource=java:/DefaultDS
</properties>
</loader>
</loaders>
Cconfiguration example for a cache loader using c3p0 JDBC connection pooling:
<loaders passivation="false" shared="false">
<preload>
<node fqn="/some/stuff"/>
</preload>
<!-- if passivation is true, only the first cache loader is used; the rest are ignored -->
<loader class="org.jboss.cache.loader.JDBCCacheLoader" async="false" fetchPersistentState="true"
ignoreModifications="false" purgeOnStartup="false">
<properties>
cache.jdbc.table.name=jbosscache
cache.jdbc.table.create=true
cache.jdbc.table.drop=true
cache.jdbc.table.primarykey=jbosscache_pk
cache.jdbc.fqn.column=fqn
cache.jdbc.fqn.type=VARCHAR(255)
cache.jdbc.node.column=node
cache.jdbc.node.type=BLOB
cache.jdbc.parent.column=parent
cache.jdbc.driver=oracle.jdbc.OracleDriver
cache.jdbc.url=jdbc:oracle:thin:@localhost:1521:JBOSSDB
cache.jdbc.user=SCOTT
cache.jdbc.password=TIGER
cache.jdbc.connection.factory=org.jboss.cache.loader.C3p0ConnectionFactory
c3p0.maxPoolSize=20
c3p0.checkoutTimeout=5000
</properties>
</loader>
</loaders>
The
S3CacheLoader
uses the
Amazon S3
(Simple Storage Solution)
for storing cache data.
Since Amazon S3 is remote network storage and has fairly high latency,
it is really best for caches that store large pieces of data, such as media
or files.
But consider this cache loader over the JDBC or
file system based cache loaders if you want remotely managed, highly reliable
storage. Or, use it for applications running on Amazon's EC2 (Elastic Compute Cloud).
If you're planning to use Amazon S3 for storage, consider using it with JBoss Cache. JBoss Cache itself provides in-memory caching for your data to minimize the amount of remote access calls, thus reducing the latency and cost of fetching your Amazon S3 data. With cache replication, you are also able to load data from your local cluster without having to remotely access it every time.
Note that Amazon S3 does not support transactions. If transactions are used in your application then there is some possibility of state inconsistency when using this cache loader. However, writes are atomic, in that if a write fails nothing is considered written and data is never corrupted.
Data is stored in keys based on the Fqn of the Node and Node data is
serialized as a java.util.Map using the
CacheSPI.getMarshaller()
instance.
Read the javadoc on how data is structured and stored.
Data is stored using Java serialization.
Be aware this means data is not readily accessible over HTTP to
non-JBoss Cache clients. Your feedback and help would be appreciated
to extend this cache loader for that purpose.
With this cache loader, single-key operations such as
Node.remove(Object)
and
Node.put(Object, Object)
are the slowest as data is stored in a single Map instance.
Use bulk operations such as
Node.replaceAll(Map)
and
Node.clearData()
for more efficiency.
Try the
cache.s3.optimize
option as well.
If you do not use Maven, you can still download the amazon-s3 library by navigating the repository or through this URL.
<repository>
<id>e-xml.sourceforge.net</id>
<url>http://e-xml.sourceforge.net/maven2/repository</url>
</repository>
...
<dependency>
<groupId>net.noderunner</groupId>
<artifactId>amazon-s3</artifactId>
<version>1.0.0.0</version>
<scope>runtime</scope>
</dependency>
cache.s3.accessKeyId- Amazon S3 Access Key, available from your account profile.cache.s3.secretAccessKey- Amazon S3 Secret Access Key, available from your account profile. As this is a password, be careful not to distribute it or include this secret key in built software.cache.s3.secure- The default isfalse: Traffic is sent unencrypted over the public Internet. Set totrueto use HTTPS. Note that unencrypted uploads and downloads use less CPU.cache.s3.bucket- Name of the bucket to store data. For different caches using the same access key, use a different bucket name. Read the S3 documentation on the definition of a bucket. The default value isjboss-cache.cache.s3.callingFormat- One ofPATH,SUBDOMAIN, orVANITY. Read the S3 documentation on the use of calling domains. The default value isSUBDOMAIN.cache.s3.optimize- The default isfalse. If true,put(Map)operations replace the data stored at an Fqn rather than attempt to fetch and merge. (This option is fairly experimental at the moment.)cache.s3.parentCache- The default istrue. Set this value tofalseif you are using multiple caches sharing the same S3 bucket, that remove parent nodes of nodes being created in other caches. (This is not a common use case.)JBoss Cache stores nodes in a tree format and automatically creates intermediate parent nodes as necessary. The S3 cache loader must also create these parent nodes as well to allow for operations such as
getChildrenNamesto work properly. Checking if all parent nodes exists for everyputoperation is fairly expensive, so by default the cache loader caches the existence of these parent nodes.cache.s3.location- This choses a primary storage location for your data to reduce loading and retrieval latency. Set toEUto store data in Europe. The default isnull, to store data in the United States.
The configuration looks as follows:
<loaders passivation="false" shared="false">
<preload>
<node fqn="/"/>
</preload>
<!-- if passivation is true, only the first cache loader is used; the rest are ignored -->
<loader class="org.jboss.cache.loader.TcpDelegatingCacheLoader">
<properties>
host=myRemoteServer
port=7500
timeout=10000
reconnectWaitTime=250
</properties>
</loader>
</loaders>
A cache loader can be used to enforce node passivation and activation on eviction in a cache.
The following figure shows 2 JBoss Cache instances sharing the same backend store:
Both nodes have a cache loader that accesses a common shared
backend store. This could for example be a shared filesystem (using
the FileCacheLoader), or a shared database. Because both nodes access
the same store, they don't necessarily need state transfer on
startup.
[3]
Rather, the
FetchInMemoryState
attribute could be set to false, resulting in a 'cold' cache, that
gradually warms up as elements are accessed and loaded for the first
time. This would mean that individual caches in a cluster might have
different in-memory state at any given time (largely depending on
their preloading and eviction strategies).
When storing a value, the writer takes care of storing the change in the backend store. For example, if node1 made change C1 and node2 C2, then node1 would tell its cache loader to store C1, and node2 would tell its cache loader to store C2.
