SRTM 30m and NED 30m Digital Elevation Models:



SRTM 30m Digital Elevation Models:

Downloading and processing from seamless.Run.htm

Nathan A. Toké

01/22/04

Table of Contents

1. Introduction

2. Downloading data sets using seamless

3. Unzipping and organizing seamless files

4. Importing files into ArcMap

5. Projecting grid files

6. Reducing grid file size

7. Merging grid files

8. Using hillshade and color ramp visualization

9. Comparing SRTM and NED data sets in terms of slope tendencies

• Step 1 – a common DEM projection

• Step 2 – calculating slope files

• Step 3 – slope files integerized

• Step 4 – clipping slope files to one another

• Step 5 – converting raster (grid) to features (shapefile)

• Step 6 – exporting slope points as .txt

• Step 7 – using Vi to clean .txt files

• Step 8 – using Mat Lab scripts to compare the data

Introduction

This document is a tutorial describing how to download and process DEM data from the USGS web site (seamless.) using the ESRI products: ArcMap, ArcCatalogue, and ArcToolbox. I downloaded the Shuttle Radar Topography Mission (SRTM) 30m data for the Rocky Mountain GEON test bed; which extends in latitude from 31 to 49 degrees north and in longitude from 103 to 115 degrees west. In this document I will describe how I downloaded, unzipped, and organized these files. Then I will discuss how to import grid files into ArcMap, reduce file size, re-project, merge, rename, and hillshade the SRTM 30m grid files. Finally, I will describe how to calculate slope files from the DEM data and one way in which I compared SRTM slopes to NED slopes over the same geographical space.

Downloading files from seamless

SRTM 30m, NED, and many other useful GIS datasets are available for download from seamless.Run.htm. The data are available by several methods of selection. The selection options are on the left side of the seamless viewer. I chose to use the define area by coordinates option because I needed to download a specific area (figure 1).

[pic][pic]

Figure 1: Define area by coordinates allows you to select a specific area for download (right window).

When selecting an area for download, seamless returns multiple zipped files that make up the area of selection. The amount and the size of these files vary depending upon the area you select, so it is best to select an area of consistent size and range, this way the zipped files will be the same size and their boarders will align. I consistently selected areas of 1 degree of latitude by 12-degree strips of longitude, from 103 to 115 degrees west (the width of GEON). This returned 8 zipped files for download, each file had dimensions of 1 degree of latitude by 1.5 degrees of longitude (figure 2). I limited the selection area to these increments because seamless requests that you pay for the data to be sent by CD if the selection area is greater than 100mb of zipped data (about 30 square arc seconds for the SRTM 30m data).

[pic]

Figure 2: After defining and area by coordinates, seamless breaks the download area into zipped files. Each file covers a discrete area. Using the same coordinate dimensions for each selection will provide consistent file dimensions. In this case 8 files (each 1 degree of latitude by 1.5 degrees of longitude) completed the area of selection that was one degree of latitude by 12 degrees of longitude.

To complete the download click download on the desired zip file. Seamless opens a pop up window that displays the data retrieval progress. Eventually a file download window will appear (figure 3), seamless will select a random numerical file name. At this point it is important to change the file name to one corresponding to the geographical area of the file, I chose to name files according to their range in latitude and longitude. It is also important to save the zip files to folders corresponding to that name, I chose to bin the 8 zipped files for each 12-degree strip of longitude into a folder named according to that strip’s latitudinal range.

[pic]

Figure 3: Upon clicking download for one of the desired files (figure 2) a window will appear that shows you the file retrieval process from seamless (background). Once the file is ready for download the above option will appear (foreground). Remember to name the files and store them in a systematic way that makes them easily identifiable with their geographical location.

Unzipping and organizing files

The zip files downloaded from seamless can be unzipped using WinZip or a similar program. Double click the zipped file and WinZip will unzip it. Then extract each grid file to a predetermined location, maintaining file organization (keep each file in a folder corresponding to its geographical location). This is important because the unzipped files will have a random numerical name. To deal with this problem I stored the files in folders corresponding to their latitudes. I then renamed each file in ArcCatalogue according the lower bounding latitude and upper bounding longitude. Example: An SRTM30m grid file from 31-32 degrees north and 110.5-112 degrees west would be named srtm_31_112. (*Note: file and folder names should not have spaces since ESRI scripts do not deal with spaces well) Once unzipped, the grid files should always be manipulated (renamed, moved, or deleted) using ArcCatalogue because ArcCatalogue will optimize file organization, keeping track of and protecting changes that may affect the files’ ability and efficiency for use in the Arc products.

Importing files into ArcMap

Open ArcMap. Under the file menu and select add data, or select the add data icon on the ArcMap display. In the add data dialogue box, navigate to the folder containing the grid file to import, select this file and click add. You may add multiple files to your map. It is useful to add a reference (shape file) layer under the grid files. Most useful are political boundary maps or maps displaying natural boundaries such as water bodies. These types of files can be downloaded from sites similar to seamless. I used a US states shape file from the national atlas digital GIS warehouse:

Projecting files

The files you downloaded from seamless may have a different projection than you would like to use when viewing the files in ArcMap. The usefulness of a projection depends upon several factors including what you will use it to view, the geographical location of the file, and the projection of any other files to be used in conjunction with the file you are adding. To change a file’s projection first open ArcToolbox. In ArcToolbox open data management tools, under that toolbox open the projections toolbox and double click on project wizard (coverages, grids) (figure 4). (Do not use define projection wizard because this option only renames the projection, it does not actually change it).

[pic]

Figure 4: Opening the Project Wizard in ArcToolbox.

In the project wizard you may choose to project a data set to a specified coordinate system or you may project the data set to match existing data. For the first file you should select Project my data to a specified coordinate system and click next (figure 5a). Now you must select the dataset for which you would like to project a coordinate system. Click the folder icon (figure 5b) and navigate to the SRTM dataset. ArcToolbox will list the parameters of the SRTM file’s current projection (SRTM30m grid files come projected in WGS 84). Click next. The next dialogue box (figure 5c) asks you what projection you want your data to have. In this example, I would like to project the file in UTM coordinates. To do this scroll down the projections list (they are in alphabetical order) and select UTM, click next. In the next dialogue box you must choose several UTM parameters (figure 5d). Make sure the units = meters and the leave the x and y shift at zero. You must choose a UTM zone for your SRTM data. This is based upon the geographical location of your dataset. To select the correct zone find your dataset location on a UTM zone map, one such map can be found at . Now a datum must be applied to the dataset (figure 5e). In this case I used NAD 27 – CONUS (North American Dataset 1927, Continental US). Scroll down and select which datum you desire and click next. You must now specify an output dataset (figure 5f). When specifying an output file make sure to save the file as a grid type and maintain organization by naming the file geographically. Click next, if the projection information you desired matches the summary click finish. The dataset with its new projection is now complete. If the grid file is large the final step may take several minutes, be patient!

At this point it is useful to make sure that the projection has been defined correctly. One way to check this is through ArcCatalogue. Open ArcCatalogue and navigate to the file that you defined the projection for. Right click on this file and select properties. Under spatial reference the projection reads Transverse_Mercator because I selected a UTM projection, the projection should match whatever you defined. It should not read GCS_WGS_1984, which is the projection that the SRTM data comes with.

To change the projections of your remaining SRTM data you may now choose project the data set to match existing data in the define projections wizard (figure 6a). This may be done as long as the datasets are within the same UTM zone, if this is true click next. Now select a grid to project and click next (figure 6b). Now you must select a grid that you wish to match the projection to, the coordinate system’s parameters should appear below and match the projection you desire for your dataset (figure 6c), click next. You must specify an output dataset (figure 6d). Remember to save the file as grid type and proceed. Now a summary of your projection input is displayed (figure 6e). If everything is in order, click finish and a new grid file with this projection will be created. Continue this process to define the new projection for each of your data sets remembering to define the correct zone for each of the datasets. Periodical checking of projections in ArcCatalogue is a good idea.

a) [pic] b) [pic]

c) [pic] d)[pic]

e) [pic] f)[pic]

g) [pic]

Figure 5 (a-g): Steps to follow in defining a coordinate system interactively.

a) [pic] b) [pic]

c) [pic] d) [pic]

e) [pic]

Figure 6 (a-e): Steps to follow in projecting a grid to match existing data.

Reducing SRTM30m Grid File Size

Each point of the SRTM30m grid files is assigned an elevation value (in meters) with many insignificant digits (figure 7). The presence of these extraneous decimal places over the entire grid file increases each grid file by several orders of magnitude. To reduce the file size the grids can be manipulated in raster calculator to produce copies consisting of only integer point values. This maintains precision at the 1-meter level, which is still more precise than the 10-meter vertical confidence interval for the SRTM data.

[pic]

Figure 7: SRTM_48_115 grid file, projected in ArcMap. Notice the highlighted layer’s range of elevation values. The SRTM data is assigned too many insignificant digits. This increases the file’s size by several orders of magnitude. I fixed this disk space problem by using raster calculator and the integer function to produce grid file copies without any decimal places. The range in confidence of the SRTM data is 10 vertical meters.

In ArcMap, add an SRTM30m grid file: example srtm_49_115 (figure 7). This 1-degree of latitude by 1.5-degree longitude file is roughly 80mb in size (with pyramids drawn). Under the “Tools” menu, open “extensions”. Make sure that the spatial analyst tool is available (figure 8a). Then open the spatial analyst toolbar: under the “View” menu, select toolbars and then spatial analyst to open the toolbar (figure 8b).

a)[pic] b)[pic]

Figure 8: Steps for opening the spatial analyst toolbar: a) turning on spatial analyst, b) opening the toolbar.

Under the spatial analyst toolbar (figure 9), select “Options”, at the bottom of the menu. In the “General” tab of “Options” Pick a working directory where you want to store files created by spatial analyst (figure 10a). Select the “Extent” tab and pick “Intersection of Inputs” since you will be using raster calculator to make a copy of each grid value (figure 10b), click ok.

[pic]

Figure 9: The spatial analyst toolbar contains the functions shown in the above menu.

a)[pic] b)[pic]

Figure 10: Choose a working directory for files created in spatial analyst/raster calculator to be stored (a) and set the analysis extent as intersection of inputs (b).

Now open the “Raster Calculator” under the spatial analyst toolbar (figure 9). Type newsmallfile = int([oldbigfile]) into the raster calculator (make sure your new file name corresponds to its geographical location, there is one space on each side of the equals sign, and that there are parenthesis and brackets as shown) and click evaluate (figure 11). ArcMap will then build a table and evaluate the solution. Once this process is done a new grid file should appear. The output file should be exactly the same as the input file less many megabytes in size. This can be assured by subtracting the input file from the output file using raster calculator and the following equation: Difference = [oldbigfile] - [newsmallfile]. The difference file will produce a grid from the values produced by subtracting each of corresponding points of the two grids; all points should be zero if everything has been entered correctly. The integer truncated grid files are about 20mb, one-fourth the size of the original SRTM30m files. Because there is no difference between the grid files and since the vertical error in the SRTM 30m data is greater than the 1-meter integer function truncation used to reduce the grid size, we can be confident that reducing the grid size is a valuable task for the interest of data management.

[pic]

Figure 11: “Raster Calculator” interface with an example for how to reduce grid file size for the SRTM 30m data.

Merging SRTM30m grid files

When working with larger regions covered by the SRTM data you will want to merge grid files. This can be done in raster calculator. Before merging it will be useful to reduce the file size as described above because it will take less time for raster calculator to evaluate the merge equation and you will be able to merge more files. Merging files is limited by your computer system’s memory (if you have 2Gb of memory then you will be able to merge only as many SRTM grids that add up to less than 2Gb).

With two files added to ArcMap open “Options” under the spatial analyst toolbar: under the “General” tab choose a location for the merged output file (figure 10a) and under extent (figure 10b) choose union of inputs since you are merging two files which do not overlap. Now open “Raster Calculator” under the spatial analyst toolbar (figures 9 and 11). Type in the following expression, replacing the hypothetical file names with geographically significant ones and evaluate: fileAC = merge([fileAB] , [fileBC]). The merged files will be continuous because the SRTM data were downloaded using the selection by coordinates option.

Hill shade and color ramp visualization

The visual appeal of the digital elevation models can be greatly improved by applying a stretched color ramp to highlight regions of elevation differences and using the “Hillshade” function to visualize changes in relief. The “Hillshade” function can be found under “Surface Analysis” in the spatial analyst toolbar (figure 9). Enter the desired parameters for the azimuth and altitude and select an output raster file, then click ok (figure 12).

[pic]

Figure 12: Parameters for creating a Hillshade grid file.

Hillshade files are excellent for viewing drainages and patterns of relief (slopes) over a region (figure 13a), but they do not provide information about the magnitude of elevation differences over those regions. DEMs provide this information. To make a really nice figure where both elevations and the patterns of relief can be seen one must overlay these two maps. Hillshading looks the best if it is done over a DEM in UTM projection. Hillshading does not work well with units of decimal degrees because as you move away from the equator the degrees of latitude and longitude are not equivalent and because the x/y coordinates are much longer than the z units. This results in small differences in relief appearing as vast hill shadowing effects. In order to see the DEM through a Hillshade map (or vise versa) one must make the top layer (the Hillshade) transparent. This can be done by right clicking on the top layer, selecting properties and under properties selecting the display tab. Now make the topmost layer transparent, somewhere between 40 and 70% transparent should work well. Now you would be able to see the information provided in both layers at once if they did not both have the same color scale. To fix this problem change the color ramp of the DEM layer. This option is also found under the layer’s properties, this time in within the symbology tab. Make sure that the values are shown as stretched and then pick a color ramp which displays elevation differences well. It is best to tinker with this until you find a color scheme that brings out the regions of elevation best for you (figure12b). Now both the regions of elevation as well as the patterns of relief are visible (figure 12c).

[pic]

Figure 12a: An example of a Hillshade map created from SRTM30m data. Notice that you can see the patterns of relief very well. Drainages can be distinguished and other topographic features are evident.

[pic]

Figure 12b: A color ramped DEM clearly shows regions of high elevation and low lands based upon color schemes. White are mountain tops and pale greens represent the low lands.

[pic]

Figure 12c: A Hillshade map made 40% transparent with a color ramped DEM underneath allows hill slopes, drainages, and other features of relief to be seen along with the different regions of elevation.

Many other manipulations are available in the Arc products that allow for improved visualization and can be used to study the geomorphology, hydrology, and tectonic features of a region using digital elevation models and other GIS data.

Comparing SRTM and NED data sets in terms of slope tendencies

An interesting comparison of the SRTM 30m DEM data to the NED 30m data is to analyze them in terms of their differences in slope. This comparison reveals differences between the two data sets answering questions such as: how do SRTM slopes in regions of high relief compare to NED slopes of high relief? This analysis is important because it is important to know the tendencies of each data set over different regions of relief. Do SRTM data sets provide reliable data over regions of high relief? What about the NED data? As long as this grid point by grid point analysis is done over the exact same geographical area this analysis is possible. The following steps outline this process. Refer to the accompanying SRTM and NED comparison report for analysis. File organization is important, especially with the number of files created for each slope comparison

Step 1: Project SRTM and NED into a common projection. UTM works well for this analysis. Follow the projection steps from earlier in this report.

Step 2: In ArcMap, calculate slope files for both of these UTM projected files. To do this first open the UTM files, then select surface analysis from the spatial analyst toolbar, and choose the slope option. The output measurement should be in degrees, z factor = 1, cell size should equal the cell size of the input file, and remember to specify an output file.

Step 3: The slope files must be integerized and multiplied by 1000 in Raster calculator (Figure 13) (will divide by 1000 in Mat lab). Remember to select the extent of calculation and the working directory options under the spatial analyst toolbar.

[pic] Fig 13: Sample integer calculation.

Step 4: For point-by-point analysis in Mat Lab the slope files need to have exactly the same geographical extent, they must have the same number of rows, columns, and the same cell size. It is likely that the SRTM and NED files you are working with do not meet these requirements because the SRTM has many areas of data voids also; the files may differ in their number of rows or columns (check under the file’s general properties). Voids, extra rows, and columns can be removed by clipping the files to one another using the extent option and raster calculator under the spatial analyst toolbar of ArcMap. First load the SRTM and NED integerized slope files. Next, it is important to set the spatial analyst extent to intersection of inputs, remember to specify your work directory as well (file management). Now use raster calculator to created clipped files for both the SRTM and NED slope files using the following general equations:

Equation 1 will replicate the location of SRTM data voids on the NED file so the two can be compared point-by-point in Mat Lab.

NEDclippedslope = Con( SRTMslope >= 0, NEDslope)

Equation 2 will remove any extra columns/rows on the SRTM data making the NED and SRTM clipped slope files equal in extent.

SRTMclippedslope = Con(NEDslope >= 0, SRTMslope)

Step 5: The clipped grid files must now be converted to point files. Under spatial analyst toolbar select the convert: raster to feature tool. In the dialogue box specify one of the clipped slope files as the input raster, select the output geometry type as point, and specify a location for the slope point file (Figure 14). Repeat the process for the other clipped file (NED or SRTM). Do not let the point files load (un click them from visible extent) because they are very large and will use up CPU memory.

[pic]Figure 14: Converting raster to features.

Step 6: Point files must now be exported to .txt files for use in Mat Lab. Right click the point files and open the attribute table. Under options, select export. Export all records. Make sure to select the output file type as .txt. This process will take a very long time. Typically these text files are more than 100mb. So be prepared to wait.

Step 7: Cleaning the .txt slope files. The exported text file contains extraneous information in the first line as well as commas that Mat Lab cannot deal with, so they must be cleaned. This can be done using the text-editing environment of VI. Open the point file in VI through SSH Secure Shell Client by typing vi(space)filename. To do this you will need to navigate to between folders in secure shell, to do this use the following commands in bold:

1. Print working directory (show which folder you are in) = pwd

2. To move up one folder = cd(space)..

3. To list the files within the working directory = ls

4. To open a folder (remember not to follow the folder name with the file extension) = cd(space)foldername

Then, once the file is loaded in vi, perform the following operations (enter what is typed in bold)

1. Delete the first line that contains erroneous entries = dd

2. Replace commas with spaces = :1,$s/,/(space)/g

3. Finish = wq

Complete this process for both files. This process can also be accomplished in Microsoft Word using search and replace, but this would be much more time consuming because it takes many repeated commands due to the MS program’s memory limits.

Step 8: Use Mat Lab to compare the NED and SRTM slope data, it is useful to create scripts for easy repetition and analysis alteration.

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download