Inside the compact cache storage format

ArcGIS Server 10.0 introduced the compact cache storage format, a new way of grouping map cache tiles in large files, rather than storing the tiles as individual files. The advantages of storing tiles in groups include the following:

• Caches are easier to copy because the number of files is reduced.
• The total size on disk of the cache is reduced.
• Tiles are generally created more quickly because disk I/O is reduced during tile creation.
• Scalability is improved when creating tiles with multiple-machine deployments, because of reduced network traffic.

How a compact cache works

The compact cache groups many tiles together in one large file called a bundle. A bundle contains up to 16,384 tiles. The result is a cache with dozens or hundreds of files, instead of thousands or millions. If you look at a compact cache on disk, you can see the bundle files with the .bundle extension. You'll also see some corresponding index files with the extension .bundlx.

It's possible to have a small cache with just one bundle at each level. More common is that you have a bundle boundary crossing some portion of the geography, so you get multiple bundles in a level (although the bundles might not contain the full 16,000 tiles if the geography is small). Large caches encompass many bundles.

The bundle boundaries are determined by the tiling scheme origin and are not adjustable. For reference, at the neighborhood/street level scale of 1:4096, a full bundle covers about the area of a mid-size county in the eastern United States.

Tile creation with bundles

During tile creation, each bundle is allocated to an ArcSOC.exe process (running service instance), which creates the tiles for the bundle. There can never be two ArcSOC.exe's working on the same bundle at a time.

If you have a small cache, or you are caching based on small feature boundaries, you may not see full utilization of the available ArcSOC.exe processes. This is because the number of ArcSOC.exe's exceeds the number of available bundles, so some ArcSOC.exe's must sit idle.

Tip:

If underutilization of ArcSOC.exe's is causing prolonged tile creation times for a large cache, you need to use larger features to divide up the caching job.

How updates occur on a compact cache

When you update tiles in a compact cache, the entire bundle is not re-created. Instead, a finer-grained area of 4096 x 4096 pixels (no antialiasing) or 2048 x 2048 pixels (antialiasing) is updated. In ArcGIS documentation this unit of area is sometimes referred to as a "supertile".

Relationship between bundles, supertiles, and ArcSOC.exe processes

Getting tiles from the bundle

ArcGIS clients, including the Web APIs, know how to read the bundle files produced by the compact cache format. In a Web situation, the client issues a call to the server for the specific level, row, and column of the tile. ArcGIS Server receives the request and returns the appropriate tile from the bundle.

The internal architecture of the bundle is not publicly documented by ESRI. If you've coded your own logic to pull tiles out of a virtual directory, you should continue to use the "exploded" format which stores each tile as a single file and was the only option at ArcGIS Server versions 9.3.1 and previous.