This document will discuss the different possibilities one has when setting up an InterWorx Cluster, and the general procedure one should follow when doing so. We recommend that you completely read over this document prior to make a decision.
Figure 1: Basic single server hosting
Figure 1 shows a basic single server hosting setup. All hosting services are handled by this single server. We break the services down into the following Roles: AppServer, (web, mail, dns, etc), File Server, and Database. In this setup, there is one network device on the server, and it is connected directly to the public network, which is then connected to the greater internet. This is not a clustered setup, it is a baseline from which we can expand into a clustered setup.
Figure 2: InterWorx Clustering with 1 NIC per AppServer
The first thing to note in Figure 2 is the addition of an extra Role to first server - Load Balancer. This server can now load balance application requests to the other AppServers, or Nodes, in the cluster. Also, note the addition of the blue directional arrows, indicating intra-cluster communication. Since we are still using just the public network for this cluster, this means that one IP address on the Load Balancer server must be reserved for intra-cluster communication - this IP is called the quorum IP. At least one other public IP address must be assigned to the Load Balancer, which will be the IP(s) that hosting services utilize.
Figure 3: InterWorx Clustering with 2 NIC per AppServer
Figure 3 shows the addition of a second, private network for the cluster to utilize. The cluster will use the private network exclusively for intra-cluster communication. The quorum, that is, the IP address on the load balancer that the other servers use to communicate to the load balancer, is now a private IP address, rather than a public one. This removes the requirement of two public IP addresses for cluster load balancing to work.
Figure 4: Interworx Clustering with 2 NIC per AppServer, and separate File and Database Servers
Figure 4 shows a more advanced InterWorx Cluster setup, where the File Server and Database Roles have been moved to two separate servers connected to the rest of the cluster only via the private network. You can add a separate MySQL server by following the instructions found in our MySQL Server Pooling documentation.
Intra-cluster communication refers to the sharing of site and email data between servers, commands being sent through the InterWorx command queue, and the routing of load balanced requests from cluster manager to nodes for processing. Before setting up your InterWorx Cluster, you must decide if you will use a private or public network for intra-cluster communication.
If you want a separate partition for /home on the InterWorx Cluster Node servers, you should name this partition /chroot instead, and symlink /home → /chroot/home. This will allow the shared storage mounts to mount cleanly.
The default setup will be to use the Cluster Manager server as the shared storage device, however, it is possible to set up the cluster using an external storage device. You need to decide prior to setting up the cluster because switching later will require a complete cluster re-setup.
If you want to use an external device for shared storage, mount that device via NFS so that /home on the Cluster Manager points to this mounted device. Make sure the cluster nodes are allowed to mount the shared storage device as well. Then, proceed with the cluster setup in NodeWorx. InterWorx will make the cluster nodes mount the correct location - the same location that the Cluster Manager was setup with for /home.
If using an external shared storage device (not the cluster manager)
Create the NFS export on the fileserver or appliance. The NFS project website has documentation for this. Ensure that every intra-cluster communication IP in the cluster has access to this NFS share.
Configure /home on the cluster manager to point to this storage device via NFS.
Note: If this shared storage device is mounted after InterWorx is installed, you may need to create the following symlinks:
(Optional) Add all additional public IPs used to host servers to the same NIC which contains the primary public IP on the IP Management Page (documentation). This can be done after the cluster is setup if you wish.
Add your cluster nodes using the private IPs (if you use the two network device scenario).
Test SiteWorx account creation by creating a temporary SiteWorx account to make sure that the clustering works properly.
Check: API Keys, if servers are virtualized that NFS or quotas are permitted in virtual environment. Also ensure that /home isn't its own paritition and that the files in /home actually reside in /chroot/home and are symlinked to /home.
SiteWorx account fails to add when using a separate NFS server.
Check: That the symlinks are in place, that you can set quotas on the NFS share from the Cluster Manager.
The command queue gets stuck when adding a SiteWorx account.
Check: That you can set quotas on the NFS share from the node, that the IP the SiteWorx account is added on is on the node on the loopback (lo) device.
You can always contact our support department should you run into issues.
How-To: Setup an Interworx Load Balanced Cluster
Related Documentation
This document will discuss the different possibilities one has when setting up an InterWorx Cluster, and the general procedure one should follow when doing so. We recommend that you completely read over this document prior to make a decision.
Figure 1 shows a basic single server hosting setup. All hosting services are handled by this single server. We break the services down into the following Roles: AppServer, (web, mail, dns, etc), File Server, and Database. In this setup, there is one network device on the server, and it is connected directly to the public network, which is then connected to the greater internet. This is not a clustered setup, it is a baseline from which we can expand into a clustered setup.
The first thing to note in Figure 2 is the addition of an extra Role to first server - Load Balancer. This server can now load balance application requests to the other AppServers, or Nodes, in the cluster. Also, note the addition of the blue directional arrows, indicating intra-cluster communication. Since we are still using just the public network for this cluster, this means that one IP address on the Load Balancer server must be reserved for intra-cluster communication - this IP is called the quorum IP. At least one other public IP address must be assigned to the Load Balancer, which will be the IP(s) that hosting services utilize.
Figure 3 shows the addition of a second, private network for the cluster to utilize. The cluster will use the private network exclusively for intra-cluster communication. The quorum, that is, the IP address on the load balancer that the other servers use to communicate to the load balancer, is now a private IP address, rather than a public one. This removes the requirement of two public IP addresses for cluster load balancing to work.
Figure 4 shows a more advanced InterWorx Cluster setup, where the File Server and Database Roles have been moved to two separate servers connected to the rest of the cluster only via the private network. You can add a separate MySQL server by following the instructions found in our MySQL Server Pooling documentation.
Public vs. Private Network For Intra-cluster Communication
Intra-cluster communication refers to the sharing of site and email data between servers, commands being sent through the InterWorx command queue, and the routing of load balanced requests from cluster manager to nodes for processing. Before setting up your InterWorx Cluster, you must decide if you will use a private or public network for intra-cluster communication.
Public Intra-cluster Communication
Pros
Cons
Private Intra-cluster Communication
Pros
Cons
Disk Partition Considerations
If you want a separate partition for /home on the InterWorx Cluster Node servers, you should name this partition /chroot instead, and symlink /home → /chroot/home. This will allow the shared storage mounts to mount cleanly.
Cluster Shared Storage Setup
The default setup will be to use the Cluster Manager server as the shared storage device, however, it is possible to set up the cluster using an external storage device. You need to decide prior to setting up the cluster because switching later will require a complete cluster re-setup.
If you want to use an external device for shared storage, mount that device via NFS so that /home on the Cluster Manager points to this mounted device. Make sure the cluster nodes are allowed to mount the shared storage device as well. Then, proceed with the cluster setup in NodeWorx. InterWorx will make the cluster nodes mount the correct location - the same location that the Cluster Manager was setup with for /home.
General InterWorx Cluster Installation
Requirements
Procedure
Troubleshooting
Some typical issues are:
You can always contact our support department should you run into issues.