Configuring a multiple-machine deployment

This document explains how to configure ArcGIS Server when its components have been installed across multiple machines. This is sometimes known as a distributed installation.

LicenseLicense:

Distributed installations of ArcGIS Server are only available at the Enterprise level. The Workgroup level of ArcGIS Server is licensed for deployment on one machine only.

Contents

Introduction

Choose a configuration

Install the appropriate features on each machine

Run the postinstallation -- Windows only

Add users to the agsadmin and agsusers groups

Log off or restart each machine—Windows only

Prepare any server directories that your server will use

Associate virtual directories with your server directories

Grant permissions to data directories

Connect to the GIS server

Add server directories

Add SOC machines

Add services

Expose services to external users

Step by step—ArcGIS Server distributed setup on Linux/Solaris

Troubleshooting

Summary

Appendix A: Accounts and permissions diagram

Appendix B: Directory diagram

Appendix C: Common problems and error messages

Introduction

ArcGIS Server has a scalable architecture that allows deployment sizes ranging from one to many machines. When you first install ArcGIS Server, you will likely choose to install all components on one machine for development and testing purposes. Once you are ready to deploy your ArcGIS Server application, you will need to consider a distributed installation of ArcGIS Server so that you can achieve an acceptable level of performance for the number of users accessing the system.

A distributed installation of ArcGIS Server is when components of one ArcGIS Server system reside on multiple machines in the same local network. For example, the graphic below depicts a distributed installation because the server object manager (SOM), server object containers (SOCs), and Web server reside on separate machines.

A distributed installation of ArcGIS Server allows you the flexibility to scale out your deployment by adding more machines. Since the container processes do the GIS work and typically consume the most CPU resources, each SOC machine that you add to your system increases the number of users that your GIS server can accommodate.

Wisely distributing the ArcGIS Server components among multiple machines can help you make the most efficient use of your hardware resources. For example, if you have a limited number of machines available, you might consider installing the SOM on the same machine as the Web server, since the SOM uses relatively little memory. Your remaining hardware can then be put to use as SOC machines to increase the processing loads that your GIS server can handle.

Performing a distributed installation of ArcGIS Server differs from a regular ArcGIS Desktop or ArcGIS Engine installation because you must configure multiple machines to correctly communicate with each other. The SOM must be able to send a request for a service to any SOC machine in the system. Since each machine is working with the same set of data and directories, a common naming convention (such as Universal Naming Convention [UNC] paths or NFS paths) must be used so that each machine can refer to the data and directories in the same way.

Security mechanisms can also pose a challenge in communication between machines. For example, a SOC account can require permissions to read or write data on a number of different machines. Because of the open communication required by the ArcGIS Server architecture, firewalls are not recommended between the components of ArcGIS Server (such as between the Web server and the SOM or between the SOM and a SOC). This topic contains an alternative recommendation to securing your system with firewalls.

Configuring a distributed installation of ArcGIS Server requires you to perform a series of important administrative tasks in the correct order. The purpose of this topic is to help you through the process of a distributed installation.

Choose a configuration

The first step in deploying a distributed ArcGIS Server system is designing the configuration. The SOM, SOC, and Web Application Developer Framework are ArcGIS Server features that can be installed on the same machine or distributed among multiple machines.

Your data must be available on the same local area network as your GIS server. If you are not using Manager to administer your server, ArcGIS Desktop must also be available on the network. However, your data and ArcGIS Desktop do not have to reside on the same machine as any other components of your GIS server.

The System Design Strategies white paper at www.esri.com/systemdesign also contains diagrams of recommended distributed installations of ArcGIS Server. Most of this information is in section 4: GIS Product Architecture. Additionally, this paper contains information about deploying ArcSDE with ArcGIS Server.

A note about firewalls

ESRI does not recommend or support firewalls between ArcGIS Server components. The recommended technique for protecting an ArcGIS Server system with firewalls is to configure a reverse proxy Web server within a perimeter network (also known as a demilitarized zone [DMZ], or screened subnet). In this scenario, the reverse proxy Web server receives incoming HTTP requests through a firewall that restricts traffic to a known port (usually port 80). It then sends the request through another firewall--using a port unknown to the end user--to the ADF Web server. This Web server hosts your ArcGIS Server Web applications and services and resides in a secure internal network. The ADF Web server is then free to establish unrestricted Distributed Component Object Model (DCOM) communications with the other ArcGIS Server components. In this way, the entire GIS server operates within a secure internal network and does not require firewalls between its components.

The above graphic shows an ArcGIS Server system protected by firewalls. The reverse proxy Web server receives the client request through a known port on the first firewall, then directs the request through a different port on the second firewall to the ADF Web server. The ADF Web server then forwards DCOM requests to the GIS server. The second firewall restricts access through any other port.

If your ArcGIS Server is running on Windows, in some cases, the firewalls included with Windows XP Service Pack 2 and Windows Server 2003 Service Pack 1 may need to be configured to work with ArcGIS Server. For more information, see ESRI Knowledge Base article 27798.

Install the appropriate features on each machine

The ArcGIS Server Installation Guide contains detailed information about the software installation process. It also contains a list of system requirements and prerequisites for ArcGIS Server. You can open the guide by clicking the Install Guide button on the first panel of the setup wizard when you install on the Windows platform.

After installing any necessary prerequisites as detailed in the system requirements, you can begin installing the ArcGIS Server software on each machine in your system. As you navigate through the setups, you will see a panel that allows you to choose which components, or features, of ArcGIS Server to install.For each machine, choose only the features necessary for the machine to perform its function in the system.

Run the postinstallation—Windows only

On Linux/Solaris, the installation includes the authorization and configuration that postinstallation on Windows does. You don't need to run them separately.

On Windows, you need to run the GIS Server Post Install on all machines in the GIS server. This is any machine that has the SOM or the SOC component installed. Note that you can re-run the postinstallation from the Start menu at any time.

The GIS Server Post Install has two parts: Configure ArcGIS Server and Authorize ArcGIS Server. The sections of the GIS Server Post Install you need to complete on each machine may vary. For example, you will only need to authorize ArcGIS Server on machines that will function as SOCs. For a machine on which you installed the SOM only, the authorization portion of the GIS Server Post Install will be disabled.

When using the GIS Server Post Install, you will be prompted to provide names and passwords for the accounts used by the GIS server, namely the SOM, SOC, and ArcGIS Web Services accounts. To understand what these accounts do and best practices for configuring them, see Accounts used by the GIS server.

You should enter the same SOM, SOC, and ArcGIS Web Services account information on each machine on which you run the postinstallation. The accounts must have the same name and password on each machine. The GIS Server Post Install gives you the option to save a configuration file that contains the names and passwords that you entered for the accounts. When you run the GIS Server Post Install on other machines, you can use the configuration file to quickly load the same name and password information.

This panel of the GIS Server Post Install prompts you to specify the SOM and SOC accounts.

For security purposes, ESRI recommends that you make the SOM and SOC accounts local instead of specifying domain accounts. This ensures that a malicious user could not use the accounts to acquire administrative privileges on other machines in your network.

Note that in Windows Computer Management, the full names of the SOM and SOC accounts are ArcGIS Server Object Manager Account and ArcGIS Server Object Container Account, respectively. When granting permissions to these accounts in Windows, the full name will appear.

For additional information on the GIS Server Post Install, see step 4a of the ArcGIS Server Install Guide.

Add users to the agsadmin and agsusers groups

On Linux/Solaris, you manage the local GIS Server accounts through Manager. The installation creates one local GIS server user, admin, with the default password admin. You can log on to Manager using admin/admin for the first time. Then you can add/edit/delete local GIS server users in Manager. See Managing local GIS server users in Manager to learn how to do it.

During the installation, you can specify the installation owner, which is also the SOM and/or SOC account. It is recommended that you use the same user name, password and UID on every machine.

You must add the installation owner with the same password to the local GIS server user list. If you use a different user name as the installation owner, all installation users on all the SOM and SOC machines must be added to the ArcGIS Server Users list on the SOM machine.

On Windows, after running the appropriate postinstallations on each machine, you need to specify which users will have administrative and regular-usage access to your server. The GIS Server Post Install creates two operating system groups on the SOM: agsadmin and agsusers. The agsadmin group is for administrators, for example, those who will add SOC machines and services to the server. You need to add yourself, and including anyone else who will be administering the server, to the agsadmin group on the SOM machine.

The agsusers group is for those who will be making local connections to the GIS server but do not need administrative access. You need to populate the agsusers group with a list of authorized users. Accounts already in the agsadmin group do not need to be added to agsusers.

You do not need to add the SOM and SOC accounts to the agsadmin and agsusers groups. These accounts are only used internally by the GIS server.

Log off or restart each machine—Windows only

For the account settings created by the postinstallations to take effect, you need to log off and back on to each machine in your system before continuing to configure your ArcGIS Server system.

You don't need to do this on Linux/Solaris.

Prepare any server directories that your server will use

The GIS server uses five special types of directories: cache, index, jobs, output, and input. The server uses these directories to store map and globe caches, store search indexes, manage geoprocessing results, write temporary files and output map images, and store map service definition files. See About server directories to learn more about these different types of server directories.

Each SOC machine in your system needs to be able to access the server directories. To make this possible, you can configure the folders representing your server directories to be accessible for certain users from other machines, so that other machines on the network can access them.

Prepare server directories on Linux/Solaris

You need to make the directory path accessible from other machines. For example, on Linux/Solaris, you have a folder on your machine named myserver at the path /opt/gisdir/server_output that you want to make available to any computer on the network. You can share this folder through NFS so that other machines can access it either by mounting this file system or through /net/myserver/opt/gisdir/server/serveroutput using automount.

You can also mount this directory on other machines using the mount command. You need to be a superuser to do so.

You also need to give the SOC user (which is also the installation owner) from other machines read (r) and write (w) permissions to this directory. You need to grant permissions to every directory down to this folder. For the example above, you need to grant permissions to all the folders below:

  • /opt
  • /opt/gisdir
  • /opt/gisdir/server_output

Grant permissions to data directories

The services created by ArcGIS Server rely on existing GIS resources, which are items such as map documents, geodatabases, and toolboxes that you create in ArcGIS Desktop and that you intend to publish to ArcGIS Server. All SOC machines must have access to this data. There are two options for ensuring that all SOC machines can access the data:

Option 1: Maintain one copy of the data in a shared folder. All SOC machines will access the data in this folder using a UNC path or NFS path or mounted path.

Option 2: Maintain identical copies of the data on each SOC machine using an identical folder structure. You can then use local paths to refer to the data. This configuration is potentially faster, since a SOC will never have to retrieve data from a different machine; however, it can be difficult to maintain if the data changes often. Also, this option is impractical for large datasets, map and globe caches, and data that will be edited.

For both options, you need to give the SOC account permissions for each data folder in the same way that you granted the SOC account access to your server directories.

If a folder contains data that will be used by a service, you need to do the following:

Steps:
  1. On Linux/Solaris
    1. Grant Read and Write permissions to the SOC account for the data.
    2. Grant Read and Write permissions to the SOC account for the folder. You need to grant permissions to every directory down to this folder. For the example above, you need to grant permissions to all the folders below:
      • /opt
      • /opt/gisdir
      • /opt/gisdir/server_output
  2. On Windows
    1. If the folder is shared, grant Change share permissions to the SOC account for the folder.
    2. Grant Read and Write file permissions to the SOC account for the folder.

Steps on how to grant permissions on Windows are included earlier in this document.

These steps apply not only to folders that contain the source documents but also to folders that contain data referenced within the document. Suppose you have a map document that displays two data layers. If the map document and the data reside in different directories, you should grant permissions as described in the steps above for both the folder containing the map document and the folder containing the data.

A simpler solution would be to store the map document and its data in the same folder. You could then use relative paths within the map document to reference the data. This way you would only have to grant permissions for one folder.

Associate virtual directories with your server directories

A virtual directory allows Internet users to access the contents of a folder on your computer via a URL. When you associate a virtual directory with your server directory, you allow your Web applications to use the server directory's contents.

You create a virtual directory on your Web server machine using your Web server's administration software; however, the server directory itself does not have to be on the same machine as the Web server. Remember the following points when creating a virtual directory:

Accessing ArcSDE data

If your data is accessed through ArcSDE, you need to make sure that your name and password are saved in the database connection. For detailed help with accessing ArcSDE data through ArcGIS Server, see the topic Preparing resources for publishing. This topic also discusses what to do if your data is stored on a machine where no other components of ArcGIS Server are installed.

Connect to the GIS server

At this point, you are ready to connect to the GIS server. You can use either Manager or ArcCatalog (including the Catalog window in ArcMap) to connect to and administer the server. ArcCatalog does not have to be installed on the same machine as any of your other ArcGIS Server components; it just has to be on the same local network and not behind any firewalls.

If you are using Manager to administer your server, see Logging in to Manager.

If you are using ArcCatalog to administer your server, see Making an administrative connection to ArcGIS Server in ArcCatalog.

Add server directories

Once you've connected to the server, you can specify one or more server directories that it can access. The Server Properties window contains a Directories tab where you can add server directories. Before you add a server directory, you should have created it in the file system and configured its sharing and permissions as explained earlier in this topic. When you type the name of the directory, be sure to use a UNC path or NFS path or mounted path.

Learn more about creating server directories

When you first set up your server, it may be easiest to create one of each type of directory. However, the directories you will actually need depend on what you do with the server. For example, if you won't ever be configuring a search service, there's no need for an index directory.

Add SOC machines

ArcGIS Server cannot function without machines that host SOC processes. Before you can create and use services, you need to provide the SOM with a list of the machines that will function as SOCs.

Before adding the SOC machines to your server, on Windows make sure that you have run the GIS Server Post Install on each machine, and on Linux/Solaris, make sure you add the installation owner to the local GIS server user list. Make sure that you have granted the SOC account permissions to the data and server directories that it will need to access.

Learn more about adding SOC machines

Add services

With a connection to the server established and server directories, data directories, and SOC machines in place, you can begin publishing GIS resources on your server as services.

Learn more about adding services

When you specify the GIS resource for the service (for example, a map document for a map service), you can use either of the following:

Expose services to external users

If you'll be exposing ArcGIS Server to the Internet, you need to perform a few configuration tasks before external users can access your services. For more information, see the topic Exposing your services to external users.

If you'll only be exposing your services locally (for example, on your internal network), no further action is required.

Step by step—ArcGIS Server distributed setup on Linux/Solaris

If you need more help getting a distributed setup of ArcGIS Server running on Linux/Solaris, a full walkthrough is available: Step by step: ArcGIS Server distributed setup on Linux/Solaris.

Troubleshooting

With the many deployment possibilities of ArcGIS Server, the multiple steps involved in setting up the system, and the uniqueness of environments in each organization, troubleshooting is often a necessary part of the installation process. An important resource for troubleshooting is the log files. These are found in <ArcGIS Server installation directory>\Server\user\log on Windows and <ArcGIS Server installation directory>/arcgis/server<ArcGIS version number>/server/user/log on Linux/Solaris. For information on how to set the logging level and interpret the log files, see How log files work.

Appendix C contains a list of problems or error messages that you might encounter when working with ArcGIS Server--especially during the distributed installation process--and suggested solutions. If you don't see the problem there, also consult Common problems and solutions.

Summary

ArcGIS Server has a scalable architecture that allows a distributed installation among any number of machines. The most direct way to add computing power to your GIS server is by adding SOC machines. The machines in an ArcGIS Server system must be able to freely communicate with each other, unhindered by firewalls, file-naming conflicts, or restricted permissions to data and folders.

To facilitate a successful distributed installation of ArcGIS Server, remember the following key points:

On Linux/Solaris

On Windows

Appendix A: Accounts and permissions diagram

Setting up a distributed installation involves running postinstallations, creating users, and managing operating system groups on multiple machines. Below is a guide that shows what you'll need to do on each machine. Each machine in the diagram contains some green text denoting the postinstallation you must run on that machine. Items in blue are accomplished by the install or postinstallation. Items in red are things that you must do.

On Linux/Solaris

On Windows

Appendix B: Directory permissions reference

ArcGIS Server uses various directories for reading and writing information. These directories

The sections below explain the steps you need to take to enable each type of directory in a distributed installation:

Server directories

The items below need to be completed to create a server cache, index, input, jobs, or output directory.

  • Create the directory.
  • Give the SOC account Read and Write-level file permissions. Give the SOM account Full Control.
  • Share the directory.
  • Give the SOC account Change-level share permissions. Give the SOM account Full Control.
  • Create an associated virtual directory (optional depending on the type of server directory and your ArcGIS Server system design).
  • Configure as an ArcGIS Server directory in ArcCatalog or Manager.

Log directory

By default, the log files are configured to be written to local paths on the SOM and SOC machines. In a distributed installation, you can keep this configuration or modify it such that the log files are written to one shared location. Make sure the items below are completed if you want to write logs to a shared location:

  • Create the directory.
  • Give the SOC account Read and Write-level file permissions.
  • Share the directory.
  • Give the SOC account Change-level share permissions.
  • Set as the ArcGIS Server log directory in theServer Properties dialog box.

Data directories

Make sure the items below are completed for all the directories in which you store source data for your services:

  • Create the directory.
  • Give the SOC account Read-level file permissions (and Write-level file permissions if users will edit data using the service).
  • Share the directory.
  • Give the SOC account Read-level share permissions (Change-level share permissions if users will edit data using the service).

Linux/Solaris permissions diagram

This diagram displays the directories used by the GIS server on Linux/Solaris and the steps you need to take to configure them properly:

Appendix C: Common problems and error messages

Below is a list of common problems and error messages you might encounter when configuring ArcGIS Server. Error messages are shown in quotes.

Usually these appear when you first attempt to create a service. Error messages may appear in the log files, on your screen, or both.

When previewing a service in ArcCatalog, you see a white screen. The coordinates adjust correctly when you move the mouse, but nothing is visible.

This problem can occur if your map document contains invalid data layers. Open the document in ArcMap to verify that the data source paths are correct for all layers.

You may also see a blank screen when the server output directory is configured correctly but its associated virtual directory is not. Following are several ways the virtual directory could be configured incorrectly that would yield this result:

  • The virtual directory previously existed but was deleted.
  • The virtual directory is pointing to a valid directory, but it is not the directory containing the data for this service.
  • The virtual directory does not have the correct permissions applied. At a minimum, Read permissions are required to view the data.

"Add host machine <machine name> failed. Check if ArcGIS Server owner is added to the users list."

This error message appears when you try to add a SOC machine without adding the installation owner(s) to the local GIS server user list first. If you use different user names for the SOC and SOM machines, all installation users on all the SOM and SOC machines must be added to the ArcGIS Server Users list on the SOM machine.

Server configuration and/or services are lost when ArcGIS Server Object Manager service is restarted.

If you need to restart the SOM service, an error may occur with the MSXML parser version. ESRI Knowledge Base article 29524 gives more details.

"Access denied" or "The connection could not be made."

These messages sometimes appear as parts of the more detailed messages listed below. You can find general support for these errors at ESRI Knowledge Base article 29042.

On Linux/Solaris, run the Server Diagnostic tool to check whether the GIS server is running properly.

"Server object instance creation failed on all SOC machines. Server Object instance creation failed on machine <machine name>."

This message occurs when the server fails to create a service. Usually this message is followed by additional information. Check the other errors in this section to see if one of them matches the additional information in your error message. The log files usually contain a reason for the failure, especially when using the more detailed log levels.

"Machine <machine name> is not a valid server container. Error: (-8001) You are not licensed for ArcGIS Server."

This message appears when you attempt to use a SOC machine that has not been properly authorized. For Windows, verify that you have run the Authorize GIS Server portion of the GIS Server Post Install on all SOC machines. On Linux/Solaris, make sure your license is valid. You can also open your keycodes file (commonly located in C:\Program Files\ESRI\License<ArcGIS version number>\sysgen on Windows and <Install owner's home directory>/.esriprogramfiles/cup/esri/License<ArcGIS version number>/sysgen/keycodes on Linux/Solaris) with a text editor and verify that the following are true:

  • arcgisserver, 100 is listed.
  • The date listed has not expired.

If either of the above two conditions is not met, you will need to request a new file from ESRI Customer Service.

"Access to output directory is denied."

This error can appear when you forget to give the SOC account appropriate access to your server output directory. You can either give the SOC account at least Read and Write permissions to the output directory (remember to give it both share and file permissions) or reconfigure the service so that it does not use an output directory.

Also see ESRI Knowledge Base article 26554.

"The connection could not be made. Access denied: The SOM service on machine <machine name> is not started and does not allow startup by this user."

This error can appear on Windows if you fail to log out and back on to your machine after running the postinstallation. Certain DCOM permissions settings related to the SOM and SOC accounts are modified during the postinstallation. You need to log out and log back in for these to take effect.

"Access denied: The SOM service is not registered on machine <machine name>."

This error appears on the Windows platform when the SOM component is not installed on the machine that you are attempting to connect to. If you've installed the SOM, verify that you have run the GIS Server Post Install, followed by logging off and back on to your machine.

"The request method (GET, POST, etc.) was not allowed for this particular resource."

One cause of this error is attempting to make an ArcGIS Server Internet connection to a machine that is not configured correctly for Web services. For example, this message can appear if the SOM component is not installed on the machine you are connecting to.

This message will also appear if you type an invalid instance name in the URL text box. For example, if you're trying to connect to Server1 with the default instance name of ArcGIS and you type http://Server1/AArcGIS/services, you will likely get this error message.

"Couldn't resolve host. The given remote host was not resolved."

This error can appear when you type an invalid URL when attempting to make an ArcGIS Server Internet connection. It is most likely due to an error in the machine name part of the URL.

"Machine <machine name> is not a valid server container. Error: 0x80070005 (Access is denied.)."

This error appears when you are attempting to add a SOC machine to your server. Verify that you have installed the SOC component of ArcGIS Server on the machine you are attempting to add. This error message can also appear on the Windows platform if you have not run or completed the GIS Server Post Install on the SOC machine, or if the SOC machines do not have identical names and passwords for the SOC account.

On Linux/Solaris, run the Server Diagnostic tool to check whether the GIS server is running properly.

"Server context creation failed on machine <machine name>. File <file name> does not exist."

The most likely cause of this error is that the server cannot find the file you specified as the GIS resource for your service, such as a map document. Check the Service Properties to make sure that a valid path to the data was entered. Then check the data itself to make sure that the SOC account has appropriate permissions to access it, as described earlier in this topic. You can also try checking your server output directory to make sure that the SOC account has appropriate permissions for it and that any virtual directories pointing to the server output directory are configured correctly.

"Your selection cannot be displayed in the current view."

This error appears on the Preview tab of ArcCatalog. This can occur on the Windows platform when you fail to specify the same user name and password for the SOM and SOC accounts across all machines in your system. As a result, the server cannot establish a connection to one or more SOC machines.

This message will also appear when you attempt to preview a map service that is stopped or paused. Right-click the service and click Start. The preview should appear on the Preview tab.

"Machine <machine name> is not a valid server container. Error: 0x80040154 (Class not registered)."

The probable cause of this error is that a machine you are trying to add as a SOC host does not have the SOC component of ArcGIS Server installed. For Windows GIS Server, to correctly configure the SOC machine, install the SOC software and run the postinstallation, ensuring that the SOM and SOC accounts have the same names and passwords as they do on the other machines in your configuration. Then log out and back in to the machine.

On Linux/Solaris, run the Server Diagnostic tool to check whether the GIS server is running properly.

"The connection could not be made. GIS server is not running on machine <machine name>."

This error may actually signify that ArcCatalog or Manager cannot locate machine X. Make sure that you have entered the name of a valid machine on your network that would be accessible to the machine running ArcCatalog or Manager.

"The selection cannot be previewed."

This message appears in the Preview window of ArcCatalog if an irrelevant node is clicked, such as the GIS server name or a GIS server folder. It will also appear if you attempt to preview a geodata or geoprocessing service. These services have no direct graphic output and cannot be previewed in ArcCatalog.


11/18/2013