The Cluster Membership objectTable of Contents
Introduction
The membership component in the Apache Tribes Channel is responsible
for dynamic discovery of other members(nodes) in the cluster.
There are currently two different membership service, the org.apache.catalina.tribes.membership.McastService
and the org.apache.catalina.tribes.membership.StaticMembershipService
.
The McastService
builds a multicast based membership service
that sends UDP packets to multicast IP addresses.
The StaticMembershipService
builds a unicast based membership
service that sends TCP packets to predefined member address.
Default Implementation
The default implementation of the cluster group notification is built on top of multicast heartbeats
sent using UDP packets to a multicast IP address.
Cluster members are grouped together by using the same multicast address/port combination.
Each member sends out a heartbeat with a given interval (frequency
), and this
heartbeat is used for dynamic discovery.
In a similar fashion, if a heartbeat has not been received in a timeframe specified by dropTime
ms. a member is considered suspect and the channel and any membership listener will be notified.
Attributes
Common Attributes
Attribute
Description
className
The implementation of the membership component.
Two implementations available, org.apache.catalina.tribes.membership.McastService
and org.apache.catalina.tribes.membership.StaticMembershipService
.
Multicast Attributes
Attribute
Description
className
The value is org.apache.catalina.tribes.membership.McastService
.
This implementation uses multicast heartbeats for member discovery.
address
The multicast address that the membership will broadcast its presence and listen
for other heartbeats on. The default value is 228.0.0.4
Make sure your network is enabled for multicast traffic.
The multicast address, in conjunction with the port
is what
creates a cluster group. To divide up your farm into several different group, or to
split up QA from production, change the port
or the address
Previously known as mcastAddr.
port
The multicast port, the default value is 45564
The multicast port, in conjunction with the address
is what
creates a cluster group. To divide up your farm into several different group, or to
split up QA from production, change the port
or the address
frequency
The frequency in milliseconds in which heartbeats are sent out. The default value is 500
ms.
In most cases the default value is sufficient. Changing this value, simply changes the interval in between heartbeats.
dropTime
The membership component will time out members and notify the Channel if a member fails to send a heartbeat within
a give time. The default value is 3000
ms. This means, that if a heartbeat is not received from a
member in that timeframe, the membership component will notify the cluster of this.
On a high latency network you may wish to increase this value, to protect against false positives.
Apache Tribes also provides a TcpFailureDetector
that will
verify a timeout using a TCP connection when a heartbeat timeout has occurred. This protects against false positives.
bind
Use this attribute if you wish to bind your multicast traffic to a specific network interface.
By default, or when this attribute is unset, it tries to bind to 0.0.0.0
and sometimes on multihomed hosts
this becomes a problem.
ttl
The time-to-live setting for the multicast heartbeats.
This setting should be a value between 0 and 255. The default value is VM implementation specific.
domain
Apache Tribes has the ability to logically group members into domains, by using this domain attribute.
The org.apache.catalina.tribes.Member.getDomain()
method returns the value specified here.
soTimeout
The sending and receiving of heartbeats is done on a single thread, hence to avoid blocking this thread forever,
you can control the SO_TIMEOUT
value on this socket.
If a value smaller or equal to 0 is presented, the code will default this value to frequency
recoveryEnabled
In case of a network failure, Java multicast socket don't transparently fail over, instead the socket will continuously
throw IOException upon each receive request. When recoveryEnabled is set to true, this will close the multicast socket
and open a new socket with the same properties as defined above.
The default is true
.
recoveryCounter
When recoveryEnabled==true
this value indicates how many
times an error has to occur before recovery is attempted. The default is
10
.
recoverySleepTime
When recoveryEnabled==true
this value indicates how long time (in milliseconds)
the system will sleep in between recovery attempts, until it recovers successfully.
The default is 5000
(5 seconds).
localLoopbackDisabled
Membership uses multicast, it will call java.net.MulticastSocket.setLoopbackMode(localLoopbackDisabled)
.
When localLoopbackDisabled==true
multicast messages will not reach other nodes on the same local machine.
The default is false
.
Static Membership Attributes
Attribute
Description
className
The value is org.apache.catalina.tribes.membership.StaticMembershipService
.
connectTimeout
Timeout for attempting a TCP connection to address of predefined static member.
Default is 500
ms.
expirationTime
If members have failed to update their alive time within the given time,
this membership will notify the memberDisappeared event to cluster.
Default is 5000
ms.
rpcTimeout
Timeout for messages that used for member notification to/from othee nodes.
Default is 3000
ms.
useThread
If set to true, this membership service will start a local thread for
sending a ping message. if set to false
, channel heartbeat
will send a ping message. Default is false
.
pingInterval
If useThread
== true
, defines the interval of
sending a ping message. Default is 1000
ms.
Nested Components
Static Membership Service allows nesting of a <LocalMember>
and <Member>element.
StaticMember Attributes
LocalMember:
Static member that is the local member of the static cluster group.
Attribute
Description
className
Only one implementation available:org.apache.catalina.tribes.membership.StaticMember
port
There is no need to set.
The value of this attribute inherits from the cluster receiver setting.
securePort
There is no need to set.
The value of this attribute inherits from the cluster receiver setting.
host
There is no need to set.
The value of this attribute inherits from the cluster receiver setting.
domain
The logical cluster domain for that this static member listens for cluster messages.
Two different type of values are possible:
1. Regular string values like "staging-domain" or "tomcat-cluster" will be converted into bytes
using ISO-8859-1 encoding.
2. byte array in string form, for example {216,123,12,3}
uniqueId
A universally uniqueId for this static member.
The values must be 16 bytes in the following form:
1. byte array in string form, for example {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
Member:
Static member that add to the static cluster group.
Attribute
Description
className
Only one implementation available:org.apache.catalina.tribes.membership.StaticMember
port
The port that this static member listens to for cluster messages
securePort
The secure port this static member listens to for encrypted cluster messages
default value is -1
, this value means the member is not listening on a secure port
host
The host (or network interface) that this static member listens for cluster messages.
Three different type of values are possible:
1. IP address in the form of "216.123.1.23"
2. Hostnames like "tomcat01.mydomain.com" or "tomcat01" as long as they resolve correctly
3. byte array in string form, for example {216,123,12,3}
domain
The logical cluster domain for that this static member listens for cluster messages.
Two different type of values are possible:
1. Regular string values like "staging-domain" or "tomcat-cluster" will be converted into bytes
using ISO-8859-1 encoding.
2. byte array in string form, for example {216,123,12,3}
uniqueId
A universally uniqueId for this static member.
The values must be 16 bytes in the following form:
1. byte array in string form, for example {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
The membership component in the Apache Tribes Channel is responsible
for dynamic discovery of other members(nodes) in the cluster.
There are currently two different membership service, the org.apache.catalina.tribes.membership.McastService
and the org.apache.catalina.tribes.membership.StaticMembershipService
.
The McastService
builds a multicast based membership service
that sends UDP packets to multicast IP addresses.
The StaticMembershipService
builds a unicast based membership
service that sends TCP packets to predefined member address.
The default implementation of the cluster group notification is built on top of multicast heartbeats
sent using UDP packets to a multicast IP address.
Cluster members are grouped together by using the same multicast address/port combination.
Each member sends out a heartbeat with a given interval (frequency
), and this
heartbeat is used for dynamic discovery.
In a similar fashion, if a heartbeat has not been received in a timeframe specified by dropTime
ms. a member is considered suspect and the channel and any membership listener will be notified.
Common Attributes
className
The implementation of the membership component.
Two implementations available, org.apache.catalina.tribes.membership.McastService
and org.apache.catalina.tribes.membership.StaticMembershipService
.
Multicast Attributes
className
The value is org.apache.catalina.tribes.membership.McastService
.
This implementation uses multicast heartbeats for member discovery.
address
The multicast address that the membership will broadcast its presence and listen
for other heartbeats on. The default value is 228.0.0.4
Make sure your network is enabled for multicast traffic.
The multicast address, in conjunction with the port
is what
creates a cluster group. To divide up your farm into several different group, or to
split up QA from production, change the port
or the address
Previously known as mcastAddr.
port
The multicast port, the default value is 45564
The multicast port, in conjunction with the address
is what
creates a cluster group. To divide up your farm into several different group, or to
split up QA from production, change the port
or the address
frequency
The frequency in milliseconds in which heartbeats are sent out. The default value is 500
ms.
In most cases the default value is sufficient. Changing this value, simply changes the interval in between heartbeats.
dropTime
The membership component will time out members and notify the Channel if a member fails to send a heartbeat within
a give time. The default value is 3000
ms. This means, that if a heartbeat is not received from a
member in that timeframe, the membership component will notify the cluster of this.
On a high latency network you may wish to increase this value, to protect against false positives.
Apache Tribes also provides a TcpFailureDetector
that will
verify a timeout using a TCP connection when a heartbeat timeout has occurred. This protects against false positives.
bind
Use this attribute if you wish to bind your multicast traffic to a specific network interface.
By default, or when this attribute is unset, it tries to bind to 0.0.0.0
and sometimes on multihomed hosts
this becomes a problem.
ttl
The time-to-live setting for the multicast heartbeats. This setting should be a value between 0 and 255. The default value is VM implementation specific.
domain
Apache Tribes has the ability to logically group members into domains, by using this domain attribute.
The org.apache.catalina.tribes.Member.getDomain()
method returns the value specified here.
soTimeout
The sending and receiving of heartbeats is done on a single thread, hence to avoid blocking this thread forever,
you can control the SO_TIMEOUT
value on this socket.
If a value smaller or equal to 0 is presented, the code will default this value to frequency
recoveryEnabled
In case of a network failure, Java multicast socket don't transparently fail over, instead the socket will continuously
throw IOException upon each receive request. When recoveryEnabled is set to true, this will close the multicast socket
and open a new socket with the same properties as defined above.
The default is true
.
recoveryCounter
When recoveryEnabled==true
this value indicates how many
times an error has to occur before recovery is attempted. The default is
10
.
recoverySleepTime
When recoveryEnabled==true
this value indicates how long time (in milliseconds)
the system will sleep in between recovery attempts, until it recovers successfully.
The default is 5000
(5 seconds).
localLoopbackDisabled
Membership uses multicast, it will call java.net.MulticastSocket.setLoopbackMode(localLoopbackDisabled)
.
When localLoopbackDisabled==true
multicast messages will not reach other nodes on the same local machine.
The default is false
.
Static Membership Attributes
className
The value is org.apache.catalina.tribes.membership.StaticMembershipService
.
connectTimeout
Timeout for attempting a TCP connection to address of predefined static member.
Default is 500
ms.
expirationTime
If members have failed to update their alive time within the given time,
this membership will notify the memberDisappeared event to cluster.
Default is 5000
ms.
rpcTimeout
Timeout for messages that used for member notification to/from othee nodes.
Default is 3000
ms.
useThread
If set to true, this membership service will start a local thread for
sending a ping message. if set to false
, channel heartbeat
will send a ping message. Default is false
.
pingInterval
If useThread
== true
, defines the interval of
sending a ping message. Default is 1000
ms.
Static Membership Service allows nesting of a <LocalMember> and <Member>element.
StaticMember Attributes
LocalMember:
Static member that is the local member of the static cluster group.
className
org.apache.catalina.tribes.membership.StaticMember
port
securePort
host
domain
1. Regular string values like "staging-domain" or "tomcat-cluster" will be converted into bytes using ISO-8859-1 encoding. 2. byte array in string form, for example {216,123,12,3}
uniqueId
1. byte array in string form, for example {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
Member:
Static member that add to the static cluster group.
className
org.apache.catalina.tribes.membership.StaticMember
port
securePort
-1
, this value means the member is not listening on a secure port
host
1. IP address in the form of "216.123.1.23"
2. Hostnames like "tomcat01.mydomain.com" or "tomcat01" as long as they resolve correctly
3. byte array in string form, for example {216,123,12,3}
domain
1. Regular string values like "staging-domain" or "tomcat-cluster" will be converted into bytes using ISO-8859-1 encoding.
2. byte array in string form, for example {216,123,12,3}
uniqueId
1. byte array in string form, for example {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}