Types of geodatabases

The geodatabase is a "container" used to hold a collection of datasets. There are three types:

1. File geodatabases—Stored as folders in a file system. Each dataset is held as a file that can scale up to 1 TB in size. The file geodatabase is recommended over personal geodatabases.
2. Personal geodatabases—All datasets are stored within a Microsoft Access data file, which is limited in size to 2 GB.
3. ArcSDE geodatabases—Stored in a relational database using Oracle, Microsoft SQL Server, IBM DB2, IBM Informix, or PostgreSQL. These multiuser geodatabases require the use of ArcSDE and can be unlimited in size and numbers of users.

Comparing the three types of geodatabases

Key characteristics	ArcSDE geodatabase	File geodatabase	Personal geodatabase
Description	A collection of various types of GIS datasets held as tables in a relational database (This is the recommended native data format for ArcGIS stored and managed in a relational database.)	A collection of various types of GIS datasets held in a file system folder.(This is the recommended native data format for ArcGIS stored and managed in a file system folder.)	Original data format for ArcGIS geodatabases stored and managed in Microsoft Access data files.(This is limited in size and tied to the Windows operating system.)
Number of users	Multiuser: many readers and many writers	Single user and small workgroups:many readers or one writer per feature dataset, stand-alone feature class, or table. Concurrent use of any specific file eventually degrades for large numbers of readers.	Single user and small workgroups with smaller datasets: some readers and one writer. Concurrent use eventually degrades for large numbers of readers.
Storage format	Oracle Microsoft SQL Server IBM DB2 IBM Informix PostgreSQL	Each dataset is a separate file on disk. A file geodatabase is a file folder that holds its dataset files.	All the contents in each personal geodatabase are held in a single Microsoft Access file (.mdb).
Size limits	Up to DBMS limits	One TB for each dataset. Each file geodatabase can hold many datasets. The 1 TB limit can be raised to 256 TB for extremely large image datasets. Each feature class can scale up to hundreds of millions of vector features per dataset.	Two GB per Access database. The effective limit before performance degrades is typically between 250 and 500 MB per Access database file.
Versioning support	Fully supported across all DBMSs; includes cross-database replication, updates using checkout and check-in, and historical archiving	Only supported as a geodatabase for clients who post updates using checkout and check-in and as a client to which updates can be sent using one-way replication.	Only supported as a geodatabase for clients who post updates using checkout and check-in and as a client to which updates can be sent using one-way replication.
Platforms	Windows, UNIX, Linux, and direct connections to DBMSs that can potentially run on any platform on the user's local network	Cross-platform.	Windows only.
Security and permissions	Provided by DBMS	Operating file system security.	Windows file system security.
Database administration tools	Full DBMS functions for backup, recovery, replication, SQL support, security, and so on	File system management.	Windows file system management.
Notes	Requires the use of ArcSDE technology; ArcSDE for SQL Server Express included with ArcEditor and ArcInfo ArcGIS Engine ArcGIS Server Workgroup ArcSDE for all other DBMSs included with ArcGIS Server Enterprise	You can optionally store data in a read-only compressed format to reduce storage requirements.	Often used as an attribute table manager (via Microsoft Access). Users like the string handling for text attributes.

A comparison of the three types of geodatabases

Learn about creating geodatabases

File geodatabases and personal geodatabases

File and personal geodatabases, which are freely available to all users of ArcView, ArcEditor, and ArcInfo, are designed to support the full information model of the geodatabase, which comprises topologies, raster catalogs, network datasets, terrain datasets, address locators, and so on. File and personal geodatabases are designed to be edited by a single user and do not support geodatabase versioning. With a file geodatabase, it is possible to have more than one editor at the same time provided they are editing in different feature datasets, stand-alone feature classes, or tables.

The file geodatabase was a new geodatabase type released in ArcGIS 9.2. Its goals are to do the following:

• Provide a widely available, simple, and scalable geodatabase solution for all users.
• Provide a portable geodatabase that works across operating systems.
• Scale up to handle very large datasets.
• Provide excellent performance and scalability, for example, to support individual datasets containing well over 300 million features and datasets that can scale beyond 500 GB per file with very fast performance.
• Use an efficient data structure that is optimized for performance and storage. File geodatabases use about one-third of the feature geometry storage required by shapefiles and personal geodatabases. File geodatabases also allow users to compress vector data to a read-only format to reduce storage requirements even further.
• Outperform shapefiles for operations involving attributes and scale the data size limits way beyond shapefile limits.

Personal geodatabases have been used in ArcGIS since their initial release in version 8.0 and have used the Microsoft Access data file structure (the .mdb file). They support geodatabases that are limited in size to 2 GB or less. However, the effective database size is smaller, somewhere between 250 and 500 MB, before the database performance starts to slow down. Personal geodatabases are also only supported on the Microsoft Windows operating system. Users like the table operations they can perform using Microsoft Access on personal geodatabases. Many users really like the text-handling capabilities in Microsoft Access for working with attribute values.

ArcGIS will continue to support personal geodatabases for numerous purposes. However, in most cases, ESRI recommends using file geodatabases for their scalability in size, significantly faster performance, and cross-platform use. The file geodatabase is ideal for working with file-based datasets for GIS projects, personal use, and in small workgroups. It has strong performance and scales well to hold extremely large data volumes without requiring the use of a DBMS. Plus, it is portable across operating systems.

Typically, users will employ multiple file or personal geodatabases for their data collections and access these simultaneously for their GIS work.

ArcSDE geodatabases

When you need a large, multiuser geodatabase that can be edited and used simultaneously by many users, the ArcSDE geodatabase provides a good solution. It adds the ability to manage a shared, multiuser geodatabase as well as support for a number of critical version-based GIS workflows. The ability to leverage your organization's enterprise relational databases is a key advantage of the ArcSDE geodatabase.

ArcSDE geodatabases work with a variety of DBMS storage models (IBM DB2, Informix, Oracle, PostgreSQL, and SQL Server). ArcSDE geodatabases are primarily used in a wide range of individual, workgroup, department, and enterprise settings. They take full advantage of underlying DBMS architectures to support the following:

• Extremely large, continuous GIS databases
• Many simultaneous users
• Long transactions and versioned workflows
• Relational database support for GIS data management (providing the benefits of a relational database for scalability, reliability, security, backup, integrity, and so forth)
• SQL types for Spatial in all supported DBMSs (Oracle, SQL Server, PostgreSQL, Informix, and DB2)
• High performance that can scale to a very large number of users

Through many large geodatabase implementations, it has been found that DBMSs are efficient at moving in and out of tables the type of large binary objects required for GIS data. In addition, GIS database sizes can be much larger and the number of supported users greater than with file-based GIS datasets.

For information about the ArcSDE geodatabase architecture and how ArcSDE geodatabases leverage relational database technology, see Architecture of the geodatabase.

ArcSDE provides long and short transaction management on the DBMS transaction framework

One of the primary roles for ArcSDE is to support the geodatabase versioning framework in each DBMS.

Quite frequently, an individual editing transaction in a GIS can involve changes to multiple rows in multiple tables. For example, updating a parcel may require that you change the polygon's representation along with changing the corresponding boundary lines and parcel corners. In addition, attribute records for each of these features must be updated as well. This edit involves making changes to multiple records in many tables. In these cases, users want to treat this collection of edits as a single transaction. When the changes are committed or rolled back, they are managed together as a unified operation.

At the same time, users want the ability to undo and redo individual edit operations within an edit session. To further complicate this situation, the edits may need to be performed in a system that is disconnected from the central, shared database.

Furthermore, during these specialized GIS data maintenance processes, the GIS database must remain continuously available for daily operations, where each user might have a personal view or state of the shared GIS database.

The ArcSDE geodatabase supports management and updates for these and many other data management scenarios in a multiuser environment by using a method called versioning. Versioning is a mechanism in which all database changes are recorded as rows in tables. For example, each time you update a value in a row, the old row is "retired" and a new, updated row is added.

In this way, ArcSDE technology manages such high-level, complex GIS transactions on the simple DBMS transaction framework by storing change information as delta records in the database.

Note:

This mechanism for maintaining records of all changes, along with their metadata, is the genesis of the term versioning.

ArcSDE uses metadata about versions to isolate multiple edit sessions, support complex transactions, share replicas,synchronize contents across multiple databases, perform automatic archiving, and support historical queries.

See An overview of editing and maintaining data for more information.

How is ArcSDE technology included in ArcGIS?

In the past, ArcSDE was sold as a separate Esri product. Starting with ArcGIS 9.2, ArcSDE technology is included as capabilities within ArcGIS rather than being offered separately. ArcSDE technology has been integrated into ArcGIS Desktop, ArcGIS Engine, and ArcGIS Server so that users can scale their DBMS-based geodatabases across their organizations.

Beginning at ArcGIS 9.2, ArcEditor and ArcInfo includes an installation of Microsoft SQL Server Express. These software products also include ArcSDE capabilities to support ArcSDE geodatabases in SQL Server Express for up to three simultaneous desktop users.

Beginning with ArcGIS 9.3, ArcGIS Engine includes an installation of SQL Server Express. The optional Geodatabase Update extension for ArcGIS Engine is required for ArcGIS Engine applications that edit geodatabases.

Microsoft limits the use of SQL Server Express to one CPU (or core within a socket) and 1GB of RAM. The maximum database size for SQL Server 2005 or 2008 Express is limited to 4GB. The maximum database size for SQL Server 2008 Express R2 is 10GB. (Consult your Esri license agreement for specific information on the instance size limitations for your implementation.)

Within ArcEditor and ArcInfo, the ArcCatalog application provides the ability for you to fully administer and manage ArcSDE geodatabases using SQL Server Express. This provides full ArcSDE geodatabase capabilities for up to a few users at a time. You set up and manage these ArcSDE geodatabases within ArcCatalog or the Catalog window. No extra software or database administration expertise is required.

ArcGIS Server Workgroup also includes ArcSDE support for SQL Server Express. With this level of ArcSDE, you can use SQL Server Express for up to 10 simultaneous Windows desktop users and editors (users of ArcView, ArcEditor, ArcInfo, a custom ArcGIS Engine application, AutoCAD, MicroStation, and so on) plus any number of additional server connections from Web applications. (Consult your license agreement for specific information on the number of connections for your implementation.)

For ArcGIS Server Workgroup, you can use ArcEditor or ArcInfo to create, administer, and manage ArcSDE geodatabases for SQL Server Express within ArcCatalog or the Catalog window. No extra database administration expertise is required.

ArcGIS Server Enterprise includes full enterprise ArcSDE technology with no limits. You can still run the traditional ArcSDE technology for Oracle, SQL Server, PostgreSQL, IBM DB2, and IBM Informix. ArcSDE support at the enterprise server level can scale to databases of any size and number of users and runs on computers of any size and configuration. With ArcGIS Server Enterprise, you provide your own DBMS license for this level of ArcSDE use. Choices include the following:

• Oracle
• Microsoft SQL Server
• IBM Informix
• IBM DB2
• PostgreSQL

The DBMS is typically administered and managed by a database administrator (DBA).

Summary points

You do not need to monitor and manage the use of your memory and CPUs for SQL Server Express. The SQL Server Express software will automatically limit computer use to 1 GB RAM on a single CPU and a database size of 4GB or 10GB, depending on the SQL Server Express version you use.

Unlike the Microsoft Access database, SQL Server Express performance does not degrade as the database size approaches its maximum size limit.

ArcSDE for SQL Server Express is included free as part of ArcEditor and ArcInfo. If you choose to install SQL Server Express, you can administer SQL Server Express databases using ArcCatalog.

When you install ArcGIS Server Workgroup, you can install and use SQL Server Express. After installing SQL Server Express, you can use ArcCatalog in your ArcEditor or ArcInfo seat to administer SQL Server Express databases for your workgroup server.

These various levels enable users to take full advantage of ArcSDE geodatabases for any number of users, large or small. This allows organizations to have one scalable data architecture that works across their single user systems up into their large enterprise systems.

Summary of ArcSDE levels