Authors : Georges Goebel, Marc Calvisi, Rod Schultz, Jerome Pioux
April 3, 2006
Version : 3.0
Load balancing at web level with mod_jk
Session Replication at web level
Load Balancing and Session Replicatoin at EJB level
Clustering databases with C-JDBC
SampleCluster2 Practical Example
Installing Java, Ant and JOnAS / Tomcat
Description of the sampleCluster2 Web Interface
This guide describes how to configure Apache, Tomcat, and JOnAS to install a cluster.
Clustering with JOnAS uses the following:
This document describes one architecture with all the clustering functionalities available in JOnAS, the configuration of architectures integrating one of those functionalities, and other possible configurations.
For hands- on practice, this document uses the sampleCluster2 JOnAS example.
The architecture shown in the following illustration, provides all the clustering functionality available in JOnAS: 1) Apache as the front-end HTTP server, 2) JOnAS/Tomcat as the J2EE Container, and 3) a shared database.
This architecture provides:
At the Servlet / JSP level, the mod_jk plug-in
provides Load Balancing / High Availability and the
Tomcat-Replication module provides Failover.
At the EJB level, the CMI protocol and JNDI replication provides Load Balancing / High Availability.
Replication of Stateful EJBs provides failover.
The database is shared by the JOnAS servers.
sampleCluster2 presented later in this document will use the same architecture.
A node (computer) that hosts one or more servers |
|
| |
A web container |
An ejb container | ||
A JOnAS instance that hosts a web container |
A JOnAS instance that hosts an ejb container | ||
A JOnAS instance that hosts a web container and an ejb container |
|
| |
An Apache server with the mod_jk module |
|
|
This following illustration describes how Apache, mod_jk, and JOnAS/Tomcat Containers interact:
This example uses mod_jk, but an alternative configuration using Enhydra Director is also possible. (see config)
Mod_jk is a plug-in (module) that handles the communication and load balancing between Apache and Tomcat.
Mod_jk uses the concept of worker. A worker is a Tomcat instance that is running to perform servlet requests coming from the web server. Each worker is identified to the web server by the host on which it is located, the port where it listens, and the communication protocol that is used to exchange messages.
AJP13 is the preferred TCP/IP sockets protocol that mod_jk uses for communicating between the web server and Tomcat workers.
In this configuration there is one worker for each Tomcat instance and one worker (this is a specific worker with no host and no port number) that will handle the load balancing. All workers are defined in a file called worker.properties.
Note: this module can also be used for site partitioning.
This following illustration shows that the HTTP session replication feature has been added to the architecture:
The term "session replication" is used when the current service state is being replicated across multiple application instances. Session replication occurs when the information stored in an HttpSession is replicated from, in this example, one servlet engine instance to another. This could be almost any type of data, such as items contained in a shopping cart or information being entered on an insurance application. Anything being stored in the session must be replicated for the service to failover without a disruption.
The solution chosen for achieving Session replication is called all-to-all replication. Tomcat uses a proprietary protocol based on TCP for the all-to-all replication.
This following illustration shows that load balancing and session replication at EJB level is now included in the architecture:
CMI is a new ORB used by JOnAS to provide clustering for load balancing and high availability. It provides replication of the JNDI registries among all EJB containers. Several instances of JOnAS can be started together in a cluster to share their EJBs. It is possible to start the same EJB on each JOnAS, or to distribute their load. A URL referencing several JOnAS instances can be provided to the clients. At lookup time, a client randomly chooses one of the available servers to request the required bean. Each JOnAS instance has the knowledge (through Javagroups) of the distribution of the Beans in the cluster. An answer to a lookup is a special clustered stub, containing stubs to each instance known in the cluster. Each method call on the Home of the bean can be issued by the stub to a new instance in order to balance the load on the cluster. The default algorithm used for load distribution is currently a weighted round robin.
With C-JDBC it is possible to cluster databases. Refer to HowTo setup a C-JDBC Database.
sampleCluster2 is a cluster deployed on a single machine.
As shown in the illustration below, there are four JOnAS instances. Two
have WEB containers and the other two have EJB containers.
The application is composed of three EJBs: a stateful session bean, a stateless session bean, and an entity bean. These EJBs are instantiated by the web interface. When the user connects to the web interface for the first time, an HTTP session is created. During this process a stateful session bean is created in one of the JOnAS nodes where the EJB container is started. The handle of this stateful session bean is added to the HTTP session (see note). Next, the web interface creates a new instance of the stateless session bean. The first time, no entity bean is created. The useful information from the stateless session bean (JOnAS node name, number of instances) is set in the stateful session bean.
The user can see the result in the web interface where all the necessary information is displayed. The user should now execute the same servlet several times in order to see the changes made by the cluster. Each time the servlet is executed, it may be executed on a different node (JOnAS with web container).
The servlet gets a reference to the stateful session bean (through the handle). After that, a new stateless session bean is created and its information is passed to the stateful session bean. On every tenth instantiation of the stateless session bean, an entity bean is created.
However, the stateless session bean may be instantiated in a different EJB container than the stateful session bean. This sample uses local interfaces between the stateless session bean and the entity bean. As a result, the entity bean is created in the same EJB container as the stateless session bean.
The entity bean writes the time and the name of the node in the database.
Note: The handle is added to the http session and not the stateful session bean. This is because the handle is able to get a reference to the stateful session bean when the JVM has changed.
See also description of the SampleCluster2 Web interface for a description of using sampleCluster2 to demonstrate cluster features.
This chapter provides information about installing the packages JOnAS /Tomcat, Apache, mod_jk.
The versions assumed here are: JOnAS 4.7.x /Tomcat 5.5.x, Apache 2.2.x, and mod_jk 1.2.15.
Perform steps one through three in Installing JOnAS with a web container from scratch.
Linux Notes:
export JAVA_HOME=<location where java was installed>
export ANT_HOME=<location where ant was installed>
export JONAS_ROOT==$HOME/cluster/JONAS_4_7_2
export PATH=$PATH:$JAVA_HOME/bin:$ANT_HOME/bin
Windows Notes:
JAVA_HOME <location where java was installed>
ANT_HOME <location where ant was installed>
JONAS_ROOT c:\cluster\JONAS_4_7_2
Add to PATH ;$JAVA_HOME/bin;$ANT_HOME/bin
For this sample, Apache will run as <user> (rather than
'root').
Below, <prefix> refers to the install folder for Apache.
This sample uses prefix = $APACHE_HOME=
$HOME/cluster/apache2.
Run and Stop Apache HTTP server to verify the installation.
Notes:
Run and Stop Apache HTTP server:
For further information about installing Apache HTTP server please refer to Apache Site.
Mod_jk will be installed in the same <user> account as Apache.
A mod_jk.so file should now have been created under $APACHE_HOME/modules.
The process for making apache aware of this new file will be performed in the
next chapter.
Note: For some distributions, a binary version may 1 be
available at the Tomcat site.
For further installation information, refer to the Tomcat Site.
For further installation information, refer to the Tomcat Site.
newjc is a tool included in
JOnAS that constructs and configures the JOnAS bases (JOnAS
instances) for the cluster. It also creates the configuration
files needed for mod_jk.
This tool is located under
$JONAS_ROOT/examples/sampleCluster2/newjc.
newjc can be used to configure cluster configurations other than sampleCluster2. These configuration files are basic and can be edited as needed to fit site requirements.
newjc can customize some of the characteristics of the cluster using the build-jc.properties file.
For sampleCluster2, this file is located at $JONAS_ROOT/examples/sampleCluster2/newjc/build-jc.properties. This file should not require modifications for sampleCluster2, unless there are port conflicts. Port assignments for sampleCluster2 are also summarized in $JONAS_ROOT/examples/sampleCluster2/PortConfiguration.txt. This file is for documentation only and is not used by newjc.
Before newjc is run for the first time on Linux, create the following directory: $HOME/jc.config/lib
mkdir –p $HOME/jc.config/lib
Run newjc.
cd $JONAS_ROOT/examples/sampleCluster2/newjc
./newjc.sh
Here is an example output. Script output is in gray. User inputs are black and bold.
Buildfile: ./build-jc.xml
init:
create_jonas_cluster:
[input] Choose your output directory : [default : ${myenv.JONAS_BASE}] :
/home/<user>/cluster
[input] Choose your prefix for the JONAS_BASE directories : [default : jb] :
wk
[input] Choose your protocol : [default : cmi] :(jrmp,jeremie,iiop,cmi,)
cmi
[input] Choose your database : [default : hsql] :
hsql
[input] Choose your WebContainer : [default : tomcat] :(tomcat,)
tomcat
[input] Choose your cluster architecture [bothWebEjb/diffWebEjb] : [default : diffWebEjb] :(bothWebEjb,diffWebEjb,)
diffWebEjb
[input] Choose the number of web instances : [default : 2] :
2
[input] Choose the number of ejb instances : [default : 2] :
2
/// newjc output deleted for brevity
Files created by newjc will be discussed in the following section.
Note: On windows the steps are the same, except;
create the folder C:\Documents and Settings\<user>\jc.config\lib)
Run the newjc.cmd
For more information about newjc refer to the ObjectWeb site.
Newjc created tomcat_jk.conf
and
workers.properties files. For sampleCluster2 they are located in
$APACHE_HOME/conf/jk.
The #Web section of build-jc.properties defines configuration
parameters set in these files by newjc.
Refer to workers how to documentation on the Tomcat site.
tomcat_jk.conf contains apache directives supported in
httpd.conf. They are isolated in a separate file for modularity,
then included in
httpd.conf.
newjc also generated four JOnAS instances
(JONAS_BASE) in directories wk1, wk2, wk3 and wk4 under
/home/<user>/cluster. Configuration files are all in the conf
directory of each JOnAS instance. The files pertinent to cluster
are described below.
wk1 and wk2 are sampleCluster2 JOnAS web container instances. wk3 and wk4 are sampleCluster2 JOnAS EJB container instances.
The #Carol section of build-jc.properties defines configuration parameters set in the carol.propetries file and jgroups-cmi.xml by newjc.
This allows JNDI replication to support load balancing at the EJB level using the CMI protocol.
Note: The multicast address and port must be identically configured for all JOnAS/Tomcat instances.
The #Services section of build-jc.properties defines configuration parameters set in the jonas.propetries file by newjc.
The #Web section of build-jc.properties defines some
configuration parameters set in the server.xml file by newjc.
In particular, a connector XML element for AJP1.3 protocol is
defined, as well as a jvmRoute to support load balancing at the
web level, via this connector.
A Cluster XML element was also added. It defines parameters for achieving Session replication at the web level with JOnAS.
Refer to the AJP Connector and the Engine Container documentation.
Note: The jvmRoute name generated by newjc is the same as the name of the associated worker defined in worker.properties. This will ensure the Session affinity.
To make apache aware of this new file, edit <prefix>/conf/httpd.conf.
LoadModule jk_module modules/mod_jk.so
Include
conf/jk/tomcat_jk.conf
Note some UNIX distributions may locate the module in the folder libexec instead of the folder modules.
Note: For sampleCluster2, in the JOnAS-specific deployment descriptor, tag shared is set it to true for the entity beans involved. When this flag is set to true, multiple instances of the same entity bean in different JOnAS servers can access a common database concurrently.
The following is an example of a deployment descriptor with the flag shared:
<jonas-ejb-jar>
<jonas-entity>
.
. .
<shared>true</shared>
.
. .
</jonas-entity>
</jonas-ejb-jar>
sampleCluster2 is located under $JONAS_ROOT/examples/sampleCluster2.
/lib librairies needed by the aplication to run
/output generated ear,jar,war files
/src source code
Note: sampleCluster2 contains both a war file for the web application and a jar file for the EJB. JOnAS will only deploy the war file to instances with a web container. Similarly, it will only deploy the jar file on instances with an EJB container.
The following graphic shows an example "SessionServlet Output" page.
The following is an explanation of the information shown in the "SessionServlet Output" page above.
Home of sampleCluster2
After click on open session link
After click on check session link