Minimizing noise from lidar for contouring and slope analysis

Lidar is promoted as an accurate form of elevation data. To a large degree, this comes from its dense sample interval. Some refer to lidar as "painting the ground with elevation measurements." Indeed, it is common to acquire submeter sampling with lidar. This facilitates canopy penetration, which improves the accuracy of ground models in forested areas. The high sample density also improves results for certain applications such as floodplain delineation. However, lidar is not always the most optimal source for all surface modeling activities. Two areas that tend to be problematic are contour derivation and slope analysis.

Messy contours and steep slopes

Getting a computer to produce nice-looking contours from traditional data sources is hard enough. It is even more difficult to produce them from lidar. Contours produced from full-resolution lidar tend to be jagged with many twists and turns and isolated closed rings.

Slope assessment with lidar data is also problematic. If you examine the average slope from full-resolution lidar, you will see that it is unusually high. This is true even over relatively flat ground, as shown in the image below depicting the slope generated from full-resolution lidar. Green is little to no slope, yellow is moderate slope, and red is steepest. Note the high amounts of yellow and red sprinkled throughout the surface.

Some people misinterpret these issues with contours and slope as simply a result of lidar being more accurate than other forms of surface data. At large enough scales, almost any surface is fairly rough. While there is some truth to this statement, a significant part of the problem really stems from the relationship between horizontal sample density and vertical accuracy. As with any measurement technology, lidar is not perfectly accurate. Its vertical accuracy generally ranges from 12 to 15centimeters. That provides a random height difference between two adjacent points of 24 to 30centimeters. When the points are only 1 meter or less apart horizontally, that height differential becomes significant. From a signal processing perspective, this is called high frequency noise.

Techniques to reduce noise

To get less jagged contours and more reasonable slope estimates, you have to remove the noise from the lidar and lose as little real information as possible while doing so. Although there is no way to prevent all loss, harm can be minimized. The terrain dataset offers a couple of tools to facilitate this. One is intelligent point thinner; the other is a high-quality interpolator.

Point thinning occurs during the terrain pyramid building process. Some people think pyramids are strictly a visualization tool, used solely to speed up drawing. While this is true for raster pyramids, the same does not apply to terrain datasets. Terrains were designed knowing that lidar is better suited for some applications when it has been generalized.

There are two terrain dataset point thinning algorithms available in ArcGIS: z-tolerance pyramiding and window size pyramiding. Both algorithms have something specific to offer and are arguably better than random point filters used by other solutions that, while fast and reasonable for visualization, are not what you would want to use for analysis.

The z-tolerance filter employs a TIN-based algorithm to find a subset of points sufficient to create a surface that is within a given vertical distance to the full-resolution surface. The use of this filter is recommended when a quantifiable measure of vertical accuracy is needed.

The window size filter selects points within a given horizontal sample distance. Every so many units in x,y (the sampling window size), the points within that area are examined and one or two are selected depending on which option you chose. The point selection method can be the point closest to the mean of the other points in the sample window, the highest or lowest of the points in the sample window, or both the highest and lowest. As stated earlier, much of the noise problem results from a bad ratio of high horizontal sample density to vertical accuracy. The window size filter enables you to improve that ratio. While it is hard to quantify the accuracy of the thinned data, empirical evidence has shown this approach works well.

Thinning the data

Lidar points are thinned when a terrain dataset is built. The filter algorithm applied is based on the type of pyramid selected for the terrain dataset. Fortunately, the same name is used for pyramid type as filter algorithm so there is no ambiguity. The pyramid type is selected through the Terrain wizard found on the feature dataset context menu in ArcCatalog or the Catalog window.

Next, you would need to specify the pyramid levels. For the sake of noise reduction, you are interested in that first pyramid level, which is one step removed from full resolution. A z-tolerance equal to the vertical accuracy of the data is reasonable for that level. This will eliminate as much noise as possible while decreasing the accuracy of the result as little as possible. If you start out with 15 centimeters vertical accuracy, you end up with approximately 30 centimeters accuracy; if you go on to produce contours, set their interval to at least double this, namely 60 centimeters or approximately 2 feet. When building a window size pyramid, a sample distance equal to twice the nominal point spacing is reasonable.

After having built a terrain dataset, the thinned points reside in its pyramid. Use the Surface Contour and Surface Slope geoprocessing tools to generate contours and slope. The results will be less noisy than if made from the unthinned points. One drawback is that contours and slope estimates made from these TIN-based surfaces will still be more angular and discontinuous than necessary. A recommended alternative is to go through a raster where the results will be smoother.

Raster interpolation

Contour and slope outputs can be improved by deriving them from a raster that is made from a terrain dataset. Engineers have a traditional bias toward working directly on TINs, so they may not like the idea of going through a raster, but when dealing specifically with contour and slope production from lidar data, this bias is unwarranted. For one thing, lidar points are essentially collected in a random distribution, and the resulting triangles are not handpicked or guaranteed to fit some mathematical model made inside a CAD package (for example, for road design).

Additionally, the linear piecewise surface defined by planar triangle faces is not smooth. A smoother surface can be made using the Terrain To Raster geoprocessing tool with the natural neighbors interpolation option. An added benefit of using Terrain To Raster is that you can rasterize an entire terrain dataset at once and avoid the size constraint associated with TIN extraction.

Once you have generated a raster surface model of the terrain dataset, you can then use the Contour and Slope raster geoprocessing tools on the derived raster. The following images illustrate the difference between full resolution and generalized contour and slope derivatives.

Contours made from full-resolution lidar on the left and a rasterized version of the thinned set on the right. The result, while obviously not cartographic quality, still represents a significant improvement.

Slope made from full-resolution lidar is on the left and slightly thinned and rasterized is on the right.

Lidar is considered noisy due to a low vertical accuracy relative to horizontal sample distance. This noise translates into poor-quality contours and excessively high average slope rates. Noise can be reduced through point thinning and smoothing without losing much of the accuracy and detail. The terrain dataset provides these tools via pyramiding and natural neighbor interpolation.