Buffer size initialization parameters

ArcSDE provides three transport buffer initialization parameters: MINBUFSIZE, MAXBUFSIZE, and MINBUFOBJECTS.

Data is transferred between an ArcSDE service (application server) and client application through a stream. A stream serves as the communication mechanism for querying a table or layer. Streams use a memory structure (a transport buffer) as a container prior to passing the structure to the underlying network stack.

When an ArcSDE stream is created, the ArcSDE process allocates transport buffers on both the client and the server. Transport buffers reduce input/output (I/O) and improve performance by accumulating vector data records and sending them across the network in batches rather than as individual records.

The records are collected in the ArcSDE application server process’ transport buffer and sent to the ArcSDE client transport buffer when the application is querying the database. Alternatively, the records are collected in the client’s transport buffer and sent to the ArcSDE server process transport buffer when the application is writing data to the database.

The transfer buffer parameters

These parameters together form the lower thresholds that prevent buffers from being transmitted in response to a client's request. While the data requested by the query is being retrieved, the batch of data in the buffer does not get sent until either MINBUFOBJECTS or MINBUFSIZE is attained. Once all the fetched data has been placed in the buffer, the data is transferred to the client even if these thresholds have not been breached. MINBUFOBJECTS is examined first.

MINBUFOBJECTS is the minimum number of records stored in a transmission buffer before it is sent across the stream. The default value is 512 records. This is also the minimum value. ESRI tests have demonstrated no performance benefits to increasing this parameter beyond 512.

MINBUFSIZE should be no more than one-half of the MAXBUFSIZE. Setting MINBUFSIZE too high can increase the length of time the client must wait before the buffer is transmitted. If MAXBUFSIZE is 64 kilobytes and MINBUFSIZE is 56 kilobytes, the client waits until the 56-kilobyte threshold is reached before sending the transport buffer. The minimum value is 4,096 bytes.

The MAXBUFSIZE parameter represents the total amount of memory that is allocated to each transport buffer. The transport buffer stops accumulating records once MAXBUFSIZE is reached and waits for the request to send the records to the client. Before increasing the size of the buffer, verify there is sufficient physical memory on the server or performance may suffer from excessive paging. For example, the default value allocates 64 kilobytes per buffer. If there are 100 users connected, each gsrvr process would contain one buffer consuming a total of 6.4 MB of system memory. If the MAXBUFSIZE is doubled, 12.8 MB of memory would be consumed.

Data transfer sequence

While data is being fetched in response to a query, ArcSDE checks if the number of rows in the data batch meets the MINBUFOBJECTS parameter. If the query is still being processed, the MINBUFOBJECTS threshold is reached (the required number of rows are present), and the client is waiting for data, data is sent. If MINBUFOBJECTS is not met and data is still being retrieved, ArcSDE checks the size of the data to see if MINBUFSIZE has been reached. If it hasn't, the buffer continues to fill.

When MINBUFSIZE is reached, if the client is waiting, data is sent to the client.

If the client had not been waiting and data retrieval was still in progress, the buffer would continue to fill past MINBUFSIZE.

The buffer continues to fill until either all requested data has been placed in the buffer, the client is waiting for data, or MAXBUFSIZE is reached. Once MAXBUFSIZE is met, buffer loading stops.