How to build effective data models for field data collection

The ArcGIS Mobile editing framework, combined with the power of the geodatabase data and transaction model, caters to a wide variety of field editing solutions and workflows. It is important that you carefully consider what is and is not supported within the geodatabase information and transactional models when determining how best to support field editing applications with ArcGIS Mobile.

Operational and basemap layers

When designing maps that will be edited in the field using ArcGIS Mobile, it is important to define which map layers will be synchronized with a GIS server (operational layers) and which layers will provide supportive background information (basemap layers). If there are layers in your map document that simply make the map look nicer and do not help field-workers complete their tasks, you should consider removing them before publishing the map. Try to keep your map simple. Avoid using definition queries, selection layers, joined tables, and other more advanced ArcMap behavior. Consider the constraints of the device and conditions where a mobile map will be used. You might already have existing mobile applications that connect directly to a map service or use an existing cache extracted from a mobile service.

Geodatabase support

ArcGIS Mobile supports editing of data within both file-based and multiuser relational geodatabases. Your mobile map can contain layers that reference other data sources, but if you want to capture new features or update existing features, the layer's data source must be a feature class stored within a geodatabase.

These are ArcGIS Mobile feature class requirements:

Each field that you add to a feature class in the geodatabase has certain properties that define its behavior when editing or querying against it. The ArcGIS Mobile applications fully leverage those properties throughout the user experience.

NoteNote:

Feature classes that are z and/or m enabled can be edited, but z- and/or m-values will not be maintained. For newly collected features, z- and m-values will not be set.

ArcGIS Mobile will not allow you to create a new map layer or alter the schema of attribute fields on a mobile device. The map schema must already exist within your geodatabase, and this implies that you must create an appropriate database design to accommodate any planned data capture or editing. If you anticipate the need to capture features that do not exist inside your current data model, or if you need to capture unstructured information (field notes, for example), it is recommended that you create an additional feature class inside your geodatabase schema to handle the storage of this information.

NoteNote:

It is extremely important, after you create a mobile cache, that you do not alter the data and/or map schema. Any and all interaction between the mobile cache and the map/geodatabase requires that the schema be checked for consistency. This is called a checksum. If the map or geodatabase is altered after the cache has been created, you will receive a checksum error when attempting to synchronize changes.

Geodatabase design considerations

An important step in building your field solution is to design the geodatabase information model that will best support your field workflows. The ArcGIS Mobile field applications and SDK take full advantage of both the intelligence you build into your data model and how you display that content using feature layers in ArcMap.

If you are planning to repurpose an existing data model for field use, you can do one of the following:

The best way to discover the approach you need to take is to consider how field-workers describe the information that they need to capture/update in the field. The way that you describe information that is collected in the field can be significantly different than the way that you model it in your enterprise geodatabase. Quite often, spatial information that is collected in the field is captured in pieces. For example, when collecting information contained within a city park, field-workers will start at one corner and work their way across the park. They may collect part of a lake shoreline, then stop and collect a park bench, then resume collecting the lake shoreline. In the enterprise data model, lakes are represented as polygon features; however, they are captured as shoreline line features in the field. It may therefore be more beneficial to model lakes as shorelines in a separate geodatabase designed for capture of data in the field.

You can define processes that transform your enterprise data model into a field data model. These processes are called ETL. For this example, you could define a process that would transform and stitch together shoreline features into lake polygons when synchronizing changes between your mobile geodatabase and the enterprise geodatabase.

Some additional geodatabase design questions you need to consider might include these:

The following section describes geodatabase replication and ETL as two separate models for managing field editing applications using multiuser geodatabases.

Geodatabase replication model

You can simply publish the contents of your production geodatabase for use in the field and use versioning to isolate the edits that come in from the field from other edits that are made in the office. However, there are a number of challenges with this approach. For example, if you need to be able to synchronize updates from the field, your production geodatabase will need to be accessible outside your company firewall. For most organizations, this is not possible. A better approach would be to use geodatabase replication and isolate the information that is captured in the field from the information that is continually updated in the office.

Using geodatabase replication, you can create a separate enterprise geodatabase instance that will store edits made in the field and periodically synchronize updates with the parent geodatabase you created the replica from. Using this approach, you only need to replicate the information that you need to take to the field (not the entire information model), and you can isolate mobile Web services from your master production geodatabase. Some examples of where replication can be very useful include a distributed system where remote offices have field-workers and do not have connectivity to the main office or when a vehicle-mounted laptop contains a replicated geodatabase and field edits are synchronized when the device is docked in the vehicle. In either example, you might also want to store metadata for each field editing task that does not belong in the enterprise geodatabase (for example, GPS measurement data required for differential correction of collected data).

Editing a replicated geodatabase created for managing field edits
Editing a replicated geodatabase created for managing field edits

ETL model

Quite often, the way that you represent spatial information in an enterprise database is different than how you create and update it in the field. Modeling lake polygons as shoreline line features so that they can be captured in pieces is one example. Another is to join normalized data tables or attribute fields stored in the enterprise geodatabase into one table or attribute in the field geodatabase. Another example is the way that street attribute information is stored. Often the proper street name is stored in multiple attributes (number, prefix, name, suffix, and so on). In ArcMap, you would use an expression to label the street name. If you want to display the street name on a mobile device, you need to join those attributes together into one field that you can label against.

You can use geoprocessing models to manage the ETL processes between the mobile and enterprise geodatabases. You can also use the ArcGIS Data Interoperability extension to visually design these transformation processes. It is important to note that the processes you define might not be bidirectional. You will need to define one set of geoprocessing models or custom spatial ETL tools for transforming your data model to a mobile device and a second set of models to reassemble field data to fit the enterprise schema.

Mobile transaction model

Once you have defined the geodatabase data model that best suits your field editing tasks, you then need to define an appropriate transaction model to manage updates coming in from the field. In part, this is defined by the data model decisions you have made, but it is also defined by the number of field editors you need to support and how isolated you want their edits to be.

Some of the transaction model considerations that you need to make include the following:

You will also need to have an understanding of how a specific transaction model works from the geodatabase perspective before building the transactional geodatabase or publishing mobile map services and how the mobile editing application synchronizes changes.

The following discussion outlines some of the key functionality you should consider when designing your geodatabase for field use.

Editing a nonversioned geodatabase

If you have few field editors that are performing simple editing tasks (updating attribute fields, for example), and there is little to no chance that they will update the same feature in the field, a nonversioned transaction model may best fit your needs. One potential drawback of this approach is that direct update is the only option available to the field editor. If for some reason updates need to be synchronized but are not complete, synchronization becomes problematic. With versioning, you have more flexibility in how your field-workers can synchronize updates.

Field edits directly update the geodatabase

Using a versioned transaction model, you can isolate field edits and add additional postprocessing and quality assurance checking before reconciling updates. Depending on how you want to isolate field edits, you could create a single version to store all field edits or you could create a version for each field editor. If you create a version for each editor, you will need to publish a mobile map service for each editor. Once the editors have completed their data capture or maintenance and synchronized their updates with the geodatabase, ArcGIS Desktop is needed to reconcile the version edits with the parent version when in the office.

Diagram of reconciling the version edits

ArcGIS Mobile client editing framework

ArcGIS Mobile applications do not have a concept of start, stop, and save edits as in other ArcGIS applications. Each edit that is made is stored within the mobile service cache on the device until the time that you decide to synchronize updates with the server. You can choose to cancel an edit that you have made within the application, which will roll back all updates made in the field and restore the original state of the feature prior to editing, but you cannot undo one edit to a feature at a time.

Updating the feature layer does not synchronize your changes with the geodatabase directly.

Posting changes

The updates that you perform in the field are stored locally in the mobile service cache on the mobile device. This is important because your field-workers may not be connected in the field at all or they may need to shut down and charge their devices, and it is important that updates are not lost. When connectivity with the server is established, you can then synchronize updates stored in the cache with the server.

When posting changes from the mobile device, only deltas are sent to the server. For example, if you change an attribute on a feature, only the change to that specific field is recorded rather than marking the entire row as edited. This is done so that when synchronizing changes, only the information that has actually changed is sent back to the server. When synchronizing updates in the field, bandwidth and storage must be conserved whenever possible.

Depending on the number of edits you expect and the type of server connection you have (GPRS, for example), you might want to enable posting features only when the application and device are back in the office to ensure a stable high-speed connection.


9/20/2011