Middleware ( Oracle / IBM ): Jboss Setup

Sample JBOSS environment setup

Naming Conventions

The following abbreviations are used throughout this document:

JBOSS_HOME - /home/jboss/jboss-4.0.3SP1/
SERVER_HOME - $JBOSS_HOME/server/test

Logfiles

Location	Purpose	Policy
$JBOSS_HOME/log/wrapper.log	Logs standard out from the JBoss server	Rolls at 5MB, 10 saved
$SERVER_HOME/log/server.log	Log4J logging	Rolls daily at midnight, date stamped
$SERVER_HOME/log/localhost_access.log	Access log for UserPortal	Rolls at midnight, date stamped
$SERVER_HOME/log/rc_abc_access.log	Access log for Response Center	Rolls at midnight, date stamped

Installation – System Setup

The following system level steps were performed:

Created on both application servers a user jboss with the password jboss1024.
Installed an updated Java runtime and development kit – jdk-1_5_0_06-linux in the location /usr/java/jdk1.5.0_06

JBoss Installation and Configuration

The following setup and configuration was performed on the JBoss Application server

Installed the JBoss/Tomcat bundle from jboss.com version 4.0.3SP1 to the directory $JBOSS_HOME
Created a server called test by copying the JBoss profile all to a new profile.
Configure Tomcat to server http to both the public and private interfaces. Primary traffic will be routed to the private interface via the load balancer, but the public interface will be open for testing and monitoring. This is accomplished by adding the following to the file $SERVER_HOME/deploy/jbossweb-tomcat55.sar/server.xml. Note – being bound to 2 interfaces could make the output from HttpRequest.getHost() unpredictable.

<Connector port="8080" address="ip1"

maxThreads="250" strategy="ms" maxHttpHeaderSize="8192"

minSpareThreads="25" maxSpareThreads="75"

emptySessionPath="true"

enableLookups="false" redirectPort="8443" acceptCount="100"

connectionTimeout="20000" disableUploadTimeout="true"/>

o Slim the installation of JBoss – in order to improve the JBoss server memory profile, performance, and reduce complexity, the following services were removed from the JBoss test server

Remove test/deploy/hsqldb-ds.xml (Hypersonic database)
Remove test/lib/hsql*.jar (Hypersonic database JARs)
Remove test/deploy/iiop-service.xml (CORBA services)
Remove test/deploy/jms (JMS Server)
Remove test/lib/jbossmq.jar (JMS JARs)
Remove test/deploy/uuid-key-generator.sar (Unique key generator)
Remove test/lib/autonumber-plugin.jar (Unique key generator JAR)
Remove server/test/deploy-hasingleton/jms (JMS server)
Remove test/server/deploy/jboss-ha-xa-jdbc.rar (HA JDBC
Remove test/server/deploy/jboss-xa-jdbc.rar
Edit server/slim/conf/jboss-service.xml and removed the following XML fragments from the from under the <mbean code="org.jboss.management.j2ee.LocalJBossServerDomain" Mbean

<attribute name="JMSService"> jboss.mq:service=DestinationManager </attribute>
<attribute name="RMI_IIOPService"> jboss:service=CorbaORB</attribute>

In jboss-service.xml, alter <attribute name="ScanEnabled">true</attribute> to be <attribute name="ScanEnabled">false</attribute>

Configure JBoss for Mysql RAC

The following steps were taken to configure JBoss to work with Mysql RAC.

Add the Mysql JDBC driver from technet.mysql.com to the directory $SERVER_HOME/lib.
Configure the driver by creating the file $SERVER_HOME/deploy/mysql-ds.xml. That file appears as follows. It differs from the installation guide in the connect string, which is significantly different for RAC.

<?xml version="1.0" encoding="UTF-8"?>

<local-tx-datasource>

<jndi-name>DefaultDS</jndi-name>

<connection-url></connection-url>

<driver-class>mysql.jdbc.driver.MysqlDriver</driver-class>

<user-name>jboss</user-name>

<password>jboss1024</password>

<min-pool-size>1</min-pool-size>

<max-pool-size>50</max-pool-size>

<blocking-timeout-millis>5000</blocking-timeout-millis>

<idle-timeout-minutes>15</idle-timeout-minutes>

<exception-sorter-class-name>org.jboss.resource.adapter.jdbc.vendor.MysqlExceptionSorter</exception-sorter-class-name>

<type-mapping>Mysql9i</type-mapping>

</metadata>

</local-tx-datasource>

<local-tx-datasource>

<jndi-name>MysqlDAODS</jndi-name>

<use-java-context>false</use-java-context>

<connection-url></connection-url>

<driver-class>mysql.jdbc.driver.MysqlDriver</driver-class>

<user-name>jboss</user-name>

<password>jboss1024</password>

<min-pool-size>5</min-pool-size>

<max-pool-size>50</max-pool-size>

<blocking-timeout-millis>5000</blocking-timeout-millis>

<idle-timeout-minutes>15</idle-timeout-minutes>

<exception-sorter-class-name>org.jboss.resource.adapter.jdbc.vendor.MysqlExceptionSorter</exception-sorter-class-name>

<type-mapping>Mysql9i</type-mapping>

</metadata>

</local-tx-datasource>

</datasources>

Secure Existing JBoss Resources

The following steps were taken to secure existing JBoss resources. The username admin with password -------- is used to login to these resources.

Secure the JMX Console

in $SERVER_HOME/deploy/jmx-console.war/WEB-INF/web.xml, uncomment the security-constraint block
in $SERVER_HOME/deploy/jmx-console.war/WEB-INF/jboss-web.xml, uncomment the security-domain block
in $SERVER_HOME/conf/props/jmx-console-users.properties change the password for admin to -----

Secure the web console

in $SERVER_HOME/deploy/management/console-mgr.sar/web-console.war/WEB-INF/web.xml, uncomment the security-constraint block
in $SERVER_HOME/deploy/management/console-mgr.sar/web-console.war/WEB-INF/jboss-web.xml, uncomment the security-domain block
in $SERVER_HOME/conf/login-config.xml, change the path to the web-console-users.properties and the web-console-roles.properties as follows (add props/ to the front of the path)
cp $SERVER_HOME/conf/props/jmx-console-users.properties to $SERVER_HOME/conf/props/web-console-users.properties
cp $SERVER_HOME/conf/props/jmx-console-roles.properties to $SERVER_HOME/ /conf/props/web-console-roles.properties

Install JBoss As A System Service

JBoss is installed as a system service using the Java Service Wrapper (http://wrapper.tanukisoftware.org). This software allows any Java process to be treated as a Unix or Windows system service. The Java Service Wrapper was installed as follows:

Download and unzip wrapper-linux-x86-32-3.2.0
Copy $WRAPPER_HOME/bin/wrapper to $JBOSS_HOME/bin
Copy $WRAPPER_HOME/src/bin/sh.script.in to $JBOSS_HOME/bin/jboss
In $JBOSS_HOME/bin/jboss, set APP_NAME to jboss and APP_LONG_NAME to JBoss Application Server
Create the directories $JBOSS_HOME/logs and $JBOSS_HOME/conf
Copy $WRAPPER_HOME/lib/libwrapper.so and $WRAPPER_HOME/lib/wrapper.jar to $JBOSS_HOME/lib
Place the following configuration file in $JBOSS_HOME/conf. Note the classpath entries, the additional Java parameters, and the application parameters.

Memory setting are provided in this file. They are currently set to Xmx 1600m and Xms 1600m. Both of these should be adjusted downward for actual expected application usage.
–Djava.net.preferIPv4Stack=true – configures JBoss to ignore the Linux IPv6 stack. This is needed for clustering support.
--host ip1 – sets the bind address for JBoss. This binds JBoss services by default to the private IP address. Services that need to be bound to the public IP address are manually configured to do this (e.g. HTTP).

#********************************************************************

# Wrapper Properties

#********************************************************************

# Java Application

wrapper.java.command=/usr/java/jdk1.5.0_06/bin/java

# Java Main class. This class must implement the WrapperListener interface

# or guarantee that the WrapperManager class is initialized. Helper

# classes are provided to do this for you. See the Integration section

# of the documentation for details.

wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp

# Java Classpath (include wrapper.jar) Add class path elements as

# needed starting from 1

wrapper.java.classpath.1=../lib/wrapper.jar

wrapper.java.classpath.2=run.jar

wrapper.java.classpath.3=/usr/java/jdk1.5.0_06/lib/tools.jar

# Java Library Path (location of Wrapper.DLL or libwrapper.so)

wrapper.java.library.path.1=../lib

# Java Additional Parameters

wrapper.java.additional.1=-server

wrapper.java.additional.2=-Dprogram.name=run.sh

wrapper.java.additional.3=-Djava.net.preferIPv4Stack=true

# Initial Java Heap Size (in MB)

wrapper.java.initmemory=1600

# Maximum Java Heap Size (in MB)

wrapper.java.maxmemory=1600

# Application parameters. Add parameters as needed starting from 1

wrapper.app.parameter.1=org.jboss.Main

wrapper.app.parameter.2=-c

wrapper.app.parameter.3=test

wrapper.app.parameter.4=--host

wrapper.app.parameter.5=ip1.25.30

#********************************************************************

# Wrapper Logging Properties

#********************************************************************

# Format of output for the console. (See docs for formats)

wrapper.console.format=PM

# Log Level for console output. (See docs for log levels)

wrapper.console.loglevel=INFO

# Log file to use for wrapper output logging.

wrapper.logfile=../logs/wrapper.log

# Format of output for the log file. (See docs for formats)

wrapper.logfile.format=LPTM

# Log Level for log file output. (See docs for log levels)

wrapper.logfile.loglevel=INFO

# Maximum size that the log file will be allowed to grow to before

# the log is rolled. Size is specified in bytes. The default value

# of 0, disables log rolling. May abbreviate with the 'k' (kb) or

# 'm' (mb) suffix. For example: 10m = 10 megabytes.

wrapper.logfile.maxsize=5m

# Maximum number of rolled log files which will be allowed before old

# files are deleted. The default value of 0 implies no limit.

wrapper.logfile.maxfiles=10

# Log Level for sys/event log output. (See docs for log levels)

wrapper.syslog.loglevel=NONE

#********************************************************************

# Wrapper Windows Properties

#********************************************************************

# Title to use when running as a console

wrapper.console.title=@app.long.name@

#********************************************************************

# Wrapper Windows NT/2000/XP Service Properties

#********************************************************************

# WARNING - Do not modify any of these properties when an application

# using this configuration file has been installed as a service.

# Please uninstall the service before modifying this section. The

# service can then be reinstalled.

# Name of the service

wrapper.ntservice.name=@app.name@

# Display name of the service

wrapper.ntservice.displayname=@app.long.name@

# Description of the service

wrapper.ntservice.description=@app.description@

# Service dependencies. Add dependencies as needed starting from 1

wrapper.ntservice.dependency.1=

# Mode in which the service is installed. AUTO_START or DEMAND_START

wrapper.ntservice.starttype=AUTO_START

# Allow the service to interact with the desktop.

wrapper.ntservice.interactive=false

o Create a link from $SERVER_HOME/bin/jboss to /etc/tc.d/init.d/jboss

o The following links were created to identify at which run levels JBoss should be started and stopped.

ln -s /etc/rc.d/init.d/jboss /etc/rc3.d/S84jboss

ln -s /etc/rc.d/init.d/jboss /etc/rc5.d/S84jboss

ln -s /etc/rc.d/init.d/jboss /etc/rc4.d/S84jboss

ln -s /etc/rc.d/init.d/jboss /etc/rc6.d/K15jboss

ln -s /etc/rc.d/init.d/jboss /etc/rc0.d/K15jboss

ln -s /etc/rc.d/init.d/jboss /etc/rc1.d/K15jboss

ln -s /etc/rc.d/init.d/jboss /etc/rc2.d/K15jboss

Starting and Stopping JBoss

JBoss can be started and stopped as a standard system service. It is configured to start at system start, and restart if it fails.

/etc/rc.d/init.d/jboss {start|stop|restart|console}

Configure Logging

System out and system error from the JBoss process are captured by the Java Service Wrapper. This output should be limited. It is logged in $JBOSS_HOME/log/wrapper.log. This file is rolled at 5MB and 10 files are preserved. JBoss logging to the file is set to INFO level.

Log4J is configured to roll files nightly at midnight. Files are created in $SERVER_HOME/log/server.log. Rolled files are stamped with the log date. INFO level and above events are also logged to the console.

Setup mod_jk

[Mod_jk proved unstable, so the the load balancer was pointed directly at Tomcat.]

In order to pass traffic from Apache to Tomcat, mod_jk was installed and configured. The following steps were performed.

Download the latest mod_jk from Jakarta Tomcat (http://tomcat.apache.org/connectors-doc/)
Rename the library to mod_jk.so and put it in /etc/httpd/modules on the Apache server machine
Add the following lines to httpd.conf

# Include mod_jk configuration file

Include conf/mod-jk.conf

Create the file mod-jk.conf

# Load mod_jk module

# Specify the filename of the mod_jk lib

LoadModule jk_module modules/mod_jk.so

# Where to find workers.properties

JkWorkersFile conf/workers.properties

# Where to put jk logs

JkLogFile logs/mod_jk.log

# Set the jk log level [debug/error/info]

JkLogLevel info

# Select the log format

JkLogStampFormat "[%a %b %d %H:%M:%S %Y]"

# JkOptions indicates to send SSK KEY SIZE

JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories

# JkRequestLogFormat

JkRequestLogFormat "%w %V %T"

# Mount your applications

JkMount /application/* loadbalancer

# You can use external file for mount points.

# It will be checked for updates each 60 seconds.

# The format of the file is: /url=worker

# /examples/*=loadbalancer

JkMountFile conf/uriworkermap.properties

# Add shared memory.

# This directive is present with 1.2.10 and

# later versions of mod_jk, and is needed for

# for load balancing to work properly

JkShmFile logs/jk.shm

# Add jkstatus for managing runtime data

JkMount status

Order deny,allow

Deny from all

Allow from 127.0.0.1

</Location

Create the file worker.properties. This file has the specific IP addresses for the HTTP server. Also relevant here is the setting local_worker=1 and local_worker_only – these settings force mod_jk to prefer the ip1 server, and only to use ip2 if the first server fails.

# Define list of workers that will be used

# for mapping requests

worker.list=loadbalancer,status

# Define Node1

# modify the host as your host IP or DNS name.

worker.node1.port=8009

worker.node1.host=ip1

worker.node1.type=ajp13

worker.node1.lbfactor=1

worker.node1.local_worker=1

worker.node1.cachesize=10

# Define Node2

# modify the host as your host IP or DNS name.

worker.node2.port=8009

worker.node2.host=ip2

worker.node2.type=ajp13

worker.node2.lbfactor=1

#worker.node2.local_worker=1 (1)

worker.node2.cachesize=10

# Load-balancing behaviour

worker.loadbalancer.type=lb

worker.loadbalancer.balance_workers=node1,node2

worker.loadbalancer.sticky_session=1

worker.loadbalancer.local_worker_only=1

#worker.list=loadbalancer

# Status worker for managing load balancer

worker.status.type=status

Create the file uriworkermap.properties. This file will need to be updated with the appropriate applications.

# Mount the Servlet context to the ajp13 worker

/jmx-console=loadbalancer

/jmx-console/*=loadbalancer

/web-console=loadbalancer

/web-console/*=loadbalancer

Modify the tomcat serverl.xml as follows on appserver1:

Modify the tomcat serverl.xml as follows on appserver2:

In jboss-service.xml set the following to true:

Tune Tomcat

Tomcat was tuned by editing the following file: $SERVER_HOME/deploy/jbossweb-tomcat55.sar/server.xml. Since the primary route for traffic is on the private HTTP connector, this connector is the only one tuned.

<Connector port="8009" address="${jboss.bind.address}"

maxThreads="250" minSpareThreads="25" maxSpareThreads="75"

emptySessionPath="true" enableLookups="false" redirectPort="8443"

protocol="AJP/1.3"/>

Install applications

The User Portal and Response Center application were installed in $SERVER_HOME/deploy according to instructions. Both were deployed in an exploded configuration.

Virtual Hosts

The ResponseCenter WEB-INF/jbossweb.xml was set to use a virtual host as follows:

<?xml version="1.0" encoding="ISO-8859-1"?>

<jboss-web>

<security-domain flushOnSessionInvalidation="true">java:/jaas/rSecurity</security-domain>

<context-root>ResponseCenter</context-root>

<virtual-host>abc.com </virtual-host>

</jboss-web>

A virtual host was added to Tomcat for the reponse center by modifying the file $SERVER_HOME/deploy/jbossweb-tomcat55.sar/server.xml as follows:

<Alias>rc.abonlive.com</Alias>

<Valve className="org.apache.catalina.valves.FastCommonAccessLogValve"

prefix="rc_abonlive_access_log." suffix=".log"

pattern="common" directory="${jboss.server.home.dir}/log"

<Context path=""

docBase="${jboss.server.home.dir}/deploy/redirect.war" debug="0"

reloadable="false"/>

</Host>

Also, the default host in the same file was set to use the same context. This application, redirect.war, is used to forward traffic from the root context to the front page of the application.

Tomcat Sesison Replication

Somcat session replication has been configured as follows. The following addition was made to the web.xml file for UserPortal and Response Center.

<web-app>

</web-app>

Also in jboss-web.xml, the following options were configured:

<jboss-web>

<replication-config>

<replication-trigger>SET_AND_NON_PRIMITIVE_GET</replication-trigger>

<replication-granularity>SESSION</replication-granularity>

</replication-config>

</jboss-web>

Finally, jboss-service.xml under jbossweb-tomcat5.0/META-INF directory was edited to the following:

<jboss-service>

<attribute name="SnapshotMode">instant</attribute>

<attribute name="UseJK">false</attribute>

</jboss-service>

Middleware ( Oracle / IBM )

Sunday, July 20, 2014

Jboss Setup

Naming Conventions

Logfiles

Installation – System Setup

JBoss Installation and Configuration

Configure JBoss for Mysql RAC

Secure Existing JBoss Resources

Secure the web console

Install JBoss As A System Service

Starting and Stopping JBoss

Configure Logging

Setup mod_jk

Tune Tomcat

Install applications

Virtual Hosts

Tomcat Sesison Replication

No comments:

Post a Comment