Installing and Maintaining CA AppLogic › Advanced Maintenance Guide › Network Configuration Details

Network Configuration Details

This section is provided to assist network architects and engineers in deploying CA AppLogic grids in existing data centers.

This section contains the following topics:

Introduction

Key Terms

Network Addresses

Ports and Access

Introduction

CA AppLogic requires the use of two separate and independent networks:

• Private network (that is, the backbone), and
• public network (that is, the external network)

For illustration, see the Network Diagrams. There are two sample configurations:

• Single grid per backbone
• Multiple grids per backbone

Important:

• Each server has at least two physical NICs, one for connecting to the private network and one for connecting to the public network. It is highly recommended that the private network is not connected to anything outside the grid servers and the BFC (if this is violated, the consequences include negative impact on at least stability, performance, and security).
• The server NIC connected to the private network MUST be a Gigabit Ethernet; the private network switch MUST be Gigabit Ethernet and cannot be cascaded or uplinked -- all servers on the same backbone MUST be connected to the same private switch (if this is violated, the consequences include negative impact on stability and performance).

Note: The recommended NIC configuration is: eth0 is the server NIC connected to the private network and eth1 is the server NIC connected to the public network. Contact Technical Support if you need to change this default assignment (usually the only reason to do so is if eth0 is a 100Mpbs NIC).

Key Terms

The following CA AppLogic terms are used in this document and are consistent with their use elsewhere in the CA AppLogic documentation. The definitions here are given with the specific meanings used in this document.

backbone

a set of servers connected to the same private network switch

grid

a set of servers on the same backbone which make up a single logical unit (aka grid)

server

a physical server belonging to a backbone and to at most one grid

appliance

a virtual server environment running on a grid

application

a functionally complete service provided by one or more appliances running on a grid

grid maintainer

a network administrator with full access rights on a grid

grid user

a system admin or developer with access to a grid only through the CA AppLogic user interface (CA AppLogic shell and GUI)

Network Addresses

This section contains the following topics:

Grid ID

Private Network

Public Network

MAC Address Assignment

Using 10.0.0.0/8 and 192.168.0.0/16 Addresses on the Public Network

Grid ID

When a grid is installed, it is assigned a grid ID. This parameter is specified as two integer numbers, in the form M/N. The M value (backbone number) should be between 1..254 and the N value (grid number) should be between 1..31.

Grids installed on the same backbone (private network) must have different values for N (regardless of the value of M). For any two grids that have a common Ethernet (layer-2) network connected to them -- private or public -- the combined value M/N must be unique.

Although this is not required, it is recommended that all grids on the same backbone have the same M (backbone number) and grids on different backbones have different values of M.

Private Network

The private network is used by CA AppLogic for the following key purposes:

• Control communication between the servers and the grid controller
• Storage access for appliances (similar to a storage network)
• Private inter-appliance communications

CA AppLogic uses the following IP subnets on the private network:

• 192.168.0.s/24, where s is the server number in the backbone (1..254). Network addresses are assigned by the Backbone Fabric Controller during the discovery process and are no longer manually assigned.
• 192.168.N.i/24, where N is the grid number of the grid in the backbone and i is the server number within the grid. The range of N is 1..31; the range of i is 1..254 (currently, 1..32, as the max. number of servers in a grid is 32). Addresses in this subnet are automatically assigned by CA AppLogic to a server when a server boots up as part of a grid. These addresses should not be manually assigned or changed by the grid administrator.
• 192.168.255.0/24. This subnet is used by the grid installer to perform various tests when creating new grids and adding servers to the grid. No servers or appliances should use addresses from this subnet in order not to interfere with the installer.
• 10.A.B.C/13, where A is calculated as N*8 and N is the grid number within the backbone (as above). The remainder of A, B, C is freely assigned by CA AppLogic to appliances running on the grid, with the last valid address being reserved for the grid controller appliance (10.(N*8+7).255.254). Optionally, this address can be changed to 1.A.B.C/13 on a per-grid basis in CA AppLogic.
• CA AppLogic versions

The routes on servers and appliances are properly configured with the subnet masks and NIC devices, so that these addresses will always be routed to the private network (unless otherwise reconfigured explicitly by the grid administrator or CA AppLogic user with root access -- the same way as if they were configured on physical servers).

Each grid uses only the private addresses belonging to it; this helps ensure that multiple grids (with different grid numbers N) can coexist on the same backbone.

Public Network

On the public network CA AppLogic grids use the following IP addresses (all as assigned by the grid administrator):

• Server public IP addresses - The server public IP addresses are automatically assigned by the Backbone Fabric Controller during the discovery process. These addresses are used for obtaining the time from external time source (NTP) and for troubleshooting grid problems (may be disabled altogether in a future release of CA AppLogic)
• Application IP addresses - This is a range of freely available public IP addresses that can be used by applications running on the grid. If these applications running on the grid are to be accessible outside of the Ethernet segment to which the servers' public NICs are connected, these addresses should be in the range that is routed into the public NICs' segment. All public NICs in a grid should be on the same Ethernet segment, as appliances can be moved from server to server (and servers from the same backbone may be added and removed from a grid). These addresses must be available and designated for use only by applications on the grid. The application addresses don't need to be a proper subnet; however, it is recommended that they are a contiguous range in a single subnet.
• controller IP address. An IP address through which grid users access the CA AppLogic grid's user interface. The address is assigned by the grid administrator during grid install and is automatically configured by CA AppLogic during grid startup. Same as for the application IP addresses, if the controller is to be accessed outside the 'public' Ethernet segment, this IP address must be routed to that segment from outside. The address must not be in use prior to creating the grid. Note: Usually the grid controller public IP address is in the same subnet as the application IP addresses.

CA AppLogic is also configured with an IP gateway and one or more DNS server IP addresses.

The controller's public IP address can be changed in the grid parameters page.

MAC Address Assignment

CA AppLogic generates MAC addresses for appliances in a predictable, computable fashion.

MAC addresses of virtual network interfaces for appliances are generated as follows: F2:M:N:a:b:c, where:

• F2 is a constant, designating the MAC address as a locally administered address (so it does not overlap with any hardware NIC MAC address)
• M and N are the backbone number and network number, respectively (as described above)
• a, b, c are assigned by CA AppLogic, in the range of 0..255 each

Using 10.0.0.0/8 and 192.168.0.0/16 Addresses on the Public Network

CA AppLogic installer will reject public IP addresses that overlap with the private network addresses reserved by CA AppLogic. If you need to use those (For example, some addresses in the 10.249.0.0/16 range), contact Technical Support for a patch or field engineering code to disable the check, and see the limitations on their use described below.

By default, the TCP stack in Linux sends ARP response for an IP address the server has, regardless of whether the IP address is on the same NIC on which the ARP request was received. This may create problems when the same non-routable addresses are used on two independent networks.

CA AppLogic has the following (version 1.2.3+):

• grid controller appliance:
  • ARP response is tied to the interface on which the request was received (/proc/sys/net/ipv4/conf/all/arp_ignore is set to 1).
• catalog appliances:
  • ARP response is tied to the interface on which the request was received (/proc/sys/net/ipv4/conf/all/arp_ignore is set to 1).
• physical server:
  • For CA AppLogic prior to 1.2.12: ARP response is left to the default mode
  • For CA AppLogic 1.2.12+: ARP responses are tied to the interface on which the ARP request was received (/proc/sys/net/ipv4/conf/all/arp_ignore is set to 1).

In short, this means that:

• the 192.168.{0-31}.n/24 subnets can be safely used on the public network, as long as: (A) CA AppLogic version is 1.2.12 or above, (B) addresses that are actually in use by CA AppLogic internally on the backbone are not assigned as the grid's controller address or as an external address used by an application on the grid; (C) CA AppLogic doesn't need to talk with any of the conflicting addresses. For example:
  • If addresses in the 192.168.1.0/24 subnet are assigned to the controller or applications or accessed by them, they would overlap with the 192.168.1.0/24 range used by CA AppLogicgrid number 1, so this grid number should not be configured; in this case CA AppLogic will be able to use or access the 192.168.1.0/24 addresses as public IP addresses.
  • If the public network has a subnet 192.168.2.0/24 which doesn't need to interact with applications on the grid, CA AppLogic grid 2 can be configured and will not interfere with the operation of the 192.168.2.0/24 subnet (but 192.168.2.0/24 addresses cannot be used as public addresses for the grid and connections from these addresses to the grid will not be possible).
  • In CA AppLogic releases prior to 1.2.12, the 192.168.{0-31}.n/24 subnets cannot be safely used on the public network (CA AppLogic servers may interfere with those addresses). If this is needed, upgrade to CA AppLogic 1.2.14b or later.
• The 10.A.B.C/xx subnets can safely be used on the public network, as long as: (A) addresses that are actually in use by CA AppLogic internally on the backbone are not assigned as the grid's controller address or as an external address used by an application on the grid; (B) CA AppLogic doesn't need to talk with any of the conflicting addresses. For example:
  • If addresses in the 10.249.0.0/16 subnet are assigned to the controller or applications or accessed by them, they would overlap with the 10.248.0.0/13 range used by CA AppLogic grid number 31, so this grid number should not be configured; in this case CA AppLogic will be able to use or access the 10.249.0.0/16 addresses as public IP addresses.
  • If the public network has a subnet 10.8.0.0/16 which doesn't need to interact with applications on the grid, CA AppLogic grid 1 can be configured and will not interfere with the operation of the 10.8.0.0/16 subnet (but 10.8.0.0/16 addresses cannot be used as public addresses for the grid and connections from these addresses to the grid will not be possible).

Ports and Access

CA AppLogic requires the following types of incoming connections:

• incoming SSH (port 22) to the controller public IP address
• incoming https (port 443) to the controller public IP address
• incoming SSH (port 22) to the public IP addresses of servers (for troubleshooting)
• incoming IP traffic for applications' public IP addresses

CA AppLogic requires the following types of outgoing connections:

• outgoing SSH (port 22) from the controller's public IP address
• outgoing SMTP (port 25) from the controller's public IP address
• outgoing DNS (udp/tcp 53) and NTP (123) from the controller's public IP address
• outgoing DNS (udp/tcp 53) and NTP (123) from the servers' public IP addresses
• outgoing IP traffic for application's public IP addresses