Exercise 3b: Transforming data and using Visualizer

Complexity: Advanced Data Requirement: ArcGIS Tutorial Data Setup Data Path: C:\arcgis\ArcTutor\Data Interoperability\ Goal: Learn the fundamentals of transforming features and attributes in FME Workbench and using Visualizer to verify output.

A transformer is an FME Workbench object that performs a particular restructuring of features or schema. It is a visual object that you add to your workspace canvas and connect between the source and destination features, or to other transformers.

This exercise introduces you to the fundamentals of using FME transformers. You create a spatial ETL tool that transforms MGE parcel data, evaluates the new features with a test clause, and generates output as failed or passed features. Finally, you inspect the output of your workspace without generating data using Visualizer. You complete the exercise by exporting the workspace as a custom format.

Using FME Workbench, you perform the following tasks:

• Add an AreaBuilder transformer to create polygons from the LOT_LINES source feature type.
• Add a PointOnAreaOverlayer transformer to overlay the LOT_POINTS features onto the polygon features and merge their attributes.
• Add a Tester transformer to evaluate and count the point features inside each polygon feature.
• Redirect the output of the workspace to Visualizer and inspect the output of your workspace.
• Export the workspace as a custom format.

Start ArcMap

Prerequisite:

The Data Interoperability extension must be enabled.

Before starting this exercise, you must have completed the previous exercise, Exercise 3a: Getting started with spatial ETL.

Steps:

1. Start ArcMap.
2. Click the Catalog Window button on the Standard toolbar.
3. Type C:\arcgis\ArcTutor\Data Interoperability\parcels\ in the Location text box and press ENTER.

   The location is added to the Catalog tree under the Folders Connection heading.

Copy and rename the spatial ETL tool

Steps:

1. Right-click the Import MGE Parcel tool and click Copy on the shortcut menu.
2. Right-click the Parcels toolbox and click Paste on the shortcut menu.
3. Right-click the Import MGE Parcel (2) tool, click Rename on the shortcut menu, then type Custom Parcels.
4. Right-click the Custom Parcels tool and click Edit.

   Workbench starts.

Add an AreaBuilder transformer

The AreaBuilder transformer creates topologically correct polygon features from lines that form closed shapes.

Connect an AreaBuilder transformer to the LOT_LINES GeometryFilter.

Steps:

1. Confirm that Transformer Gallery is available.

   To open the Transformer window from the main menu, click View > Windows > Transformer Gallery.

2. In the Transformers window, type areabuilder in the search text box and press ENTER.
3. Double-click AreaBuilder.

   An AreaBuilder transformer is added to the canvas.

4. Click the line connecting the LOT_LINES GeometryFilter and the LOT_LINES destination feature and press DELETE.

   The GeometryFilter is disconnected from the destination feature type.

5. On the LOT_LINES GeometryFilter, click the yellow arrow beside the LINE attribute output and drag the arrow to the red arrow beside the AreaBuilder input.

   The GeometryFilter output is connected to the AreaBuilder input, and the arrow turns green.

Add a PointOnAreaOverlayer transformer

The PointOnAreaOverlayer transformer overlays point features onto area features and merges their attributes. As a result, each polygon feature acquires attributes from the point feature contained within its boundary. Attributes with the same name are not replaced.

Connect a PointOnAreaOverlayer transformer to the LOT_LINES AreaBuilder transformer and the LOT_POINTS GeometryFilter.

Steps:

1. In the Transformers window, type PointOnArea in the search text box and press ENTER.
2. Double-click PointOnAreaOverlayer.

   A PointOnAreaOverlayer transformer is added to the canvas.

3. Click the Properties button and view the parameters.

   The PointOnAreaOverlayer Parameters dialog box opens.

   Overlap Count Attribute counts the number of point features contained in each polygon. This is useful for validating data and sending features to separate outputs for further processing. In the next section, you will connect a Tester transformer to evaluate this number and direct output accordingly.

4. Click OK.

   The PointOnAreaOverlayer Parameters dialog box closes.

5. Click the line connecting the LOT_POINTS GeometryFilter and the LOT_POINTS destination feature type and press DELETE.

   The GeometryFilter is disconnected from the destination feature type.

6. On the LOT_POINTS GeometryFilter, click the yellow arrow beside the POINT output and drag the line to the red arrow beside the PointOnAreaOverlayer POINT input.

   The GeometryFilter POINT output is connected to the PointOnAreaOverlayer POINT input, and the arrow turns green.

Organizing the canvas layout

At this time, the connectors may begin to cross over each other and appear confusing. There are several ways to organize the canvas layout:

• Click Tools > Auto Layout, then drag the objects on the canvas.
• Right-click a connector line, click Insert Vertex on link on the shortcut menu to add one more vertex, then drag each vertex to route the connector around other transformers.

Add a Tester transformer

The Tester transformer evaluates one or more tests on a feature and routes the feature according to the outcome of the test(s).

Connect a Tester transformer to the PointOnAreaOverlayer transformer. Add a test clause that evaluates the Overlap Count attribute generated by the PointOnAreaOverlayer transformer. Send polygon features containing one point feature to the PASSED output and all others to the FAILED output.

Steps:

1. In the Transformers window, type Tester in the search text box and click Find.
2. Double-click Tester.

   A Tester transformer is added to the canvas.

3. Connect the PointOnAreaOverlayer AREA output to the Test INPUT.

4. Click the Properties button on the Tester transformer.

   The Tester Parameters dialog box opens.

5. Double-click the first cell in the Left Value field, click the down arrow, then choose _overlaps.
6. Double-click the first cell in the Op. field, click the arrow, then choose =.
7. Click the first cell in the Right Value field and type 1.
8. Click OK.

   The Tester Parameters dialog box closes.

Redefine the LOT_LINES destination feature

Redefine the LOT_LINES destination feature to generate the polygon features that failed the test clause specified in the Tester transformer. Also, add a user attribute to capture the _overlay value.

Steps:

1. Connect the FAILED output on the Tester transformer to the INPUT on the LOT_LINES destination feature.

2. Click the Properties button on the LOT_LINES destination feature.

   The Feature Type Properties dialog box opens.

3. Click the General tab.
4. In the Feature Type Name text box, type problem_polys.
5. Click the Allowed Geometries arrow and choose geodb_polyline.
6. Click the User Attributes tab.
7. Select all attributes and press DELETE.

   The LOT_LINES attributes are not needed.

8. Click the first empty cell in the Attribute Name field and type num_points.

   The num_points attribute is used to capture the _overlay attribute value generated by the Tester transformer. This value represents the quantity of overlapping points for each failed feature.

9. Click the Data Type arrow and choose integer.
10. Click OK.

    The Feature Type Properties dialog box closes.

11. Expand the Tester transformer and connect the _overlay output to num_points on the LOT_LINES destination feature.

Redefine the LOT_POINTS destination feature

Redefine the LOT_POINTS destination feature to generate the polygon features that passed the test clause specified in the Tester transformer.

Steps:

1. Connect the PASSED output on the Tester transformer to the INPUT on the LOT_POINTS destination feature.

2. Click the Properties button on the LOT_POINTS destination feature.

   The Feature Type Properties dialog box opens.

3. Click the General tab.
4. In the Feature Type Name text box, type parcels.
5. Click the Allowed Geometries arrow and choose geodb_polyline.
6. Click the User Attributes tab.
7. Select only the attributes with names that start with igds_ and press DELETE.

   The igds_ attributes are symbolic names created by the FME reader to simplify the translation of the IGDS source element types to FME feature types. There is no need to include them with the destination features.

8. Click OK.

   The Feature Type Properties dialog box closes.

Verify the output of your workspace with Visualizer

Visualizer is an in-memory viewer that enables you to inspect the output of your workspace without generating data or possibly overwriting existing data.

Redirect the output of your workspace to Visualizer and verify your results.

Steps:

1. In the Navigator window, expand Tool Settings, right-click Destination Redirect, then click Edit Parameter from the shortcut menu.

   The Edit Parameter dialog box opens.

2. Click the Destination Redirect arrow and choose Redirect to Visualizer.
3. Click OK.

   The Edit Parameter dialog box closes.

4. On the main menu, click File > Run Translation.

   Visualizer starts.

5. Expand the datasets in Viewspace.
6. Click the Identify tool on the toolbar, select features, then explore their attribute values displayed in the Information window.
Observe that the workspace creates polygons from line elements stored in the MicroStation DGN (.par) files and acquires the point feature attributes stored in the parcels_attr.mdb database. Also observe that the Tester transformer routed features that did not pass the test clause to the problem_polys destination features.
7. Exit Visualizer.
8. On the Workbench menu, click File > Save.

Save the spatial ETL tool as a custom format

A custom format is a translation workspace that is exported as a format and utilized in the FME reader gallery. It is saved to an external (.fds) file and can be imported to reader galleries on other desktops.

Creating a custom format adds the format to the FME reader gallery. Once created, subsequent uses of the gallery allow you to edit the format in FME Workbench or delete it from the list and remove the (.fds) file from the default folder location.

Steps:

1. On the Workbench menu, click File > Export as Custom Format.

   The Set Custom Format Name and Description dialog box opens.

2. Type Custom_Parcels in the New Custom Format Name text box.
3. Type Custom Parcels in the New Custom Format Description text box.
4. Click OK.

   A second Workbench session starts as a custom format (.fds) workspace.

5. Click File > Exit to close the custom format workspace.
6. Click File > Exit to close the transformation workspace.

This completes the exercise.

Continue to the next exercise: Exercise 3c: Using attribute values to control output.