Time Series in Arc Hydro

 

CE 394K GIS in Water Resources

University of Texas at Austin

Fall 2002

 

Prepared by: Tim Whiteaker and David R. Maidment

 

Time Series in Arc Hydro

1.Introduction. 2

2. Retrieving and Displaying NWIS Data. 3

A.  Familiarizing Yourself with the Data. 3

B.  Loading the NWIS tool into ArcMap. 4

C.  Using the NWIS Tool 5

D.  Displaying Time Series in Excel 10

3. Viewing Time Series in a GIS. 18

A.  Familiarizing Yourself with the Data. 18

B.  Loading the Time Series Viewer into ArcMap. 19

Adding the Arc Hydro Tools. 19

Adding the Time Series Tools. 19

C.  Viewing NEXRAD Data. 20

D.  Transferring NEXRAD data to Catchments. 22

E.  Performing Rainfall Analyses with the Results. 26

Preparing the Data for the Accumulate Tool 26

Running the Accumulate Tool 28

 

 

 

1.1. Introduction

 

Arc Hydro integrates spatial and temporal data in a hydrologic context within a GIS.  In this exercise, we look at different ways of utilizing Arc Hydro’s time series format for time series display and analysis.

 

TimeSeries.zip contains the files used for this exercise. The data folder for this part of the exercise can be downloaded from Prometheus or from the web site timeseries.zip

 

The files included in TimeSeries.zip are:

 

Guadalupe.mdb: An Arc Hydro geodatabase for the Guadalupe River basin

NWIS.zip: A zip file containing NWIS dll and documentation (source files and additional sample data are included for your benefit, but are not required for this exercise).   These files are also stored on the LRC Class server at class/maidment/giswr/timeseries.

 

This exercise assumes a basic working knowledge of ArcMap, the ArcGIS Hydro Data Model, and Microsoft Excel.

 

·               To begin, unzip the contents of TimeSeries.zip to your computer. 

·               To start this exercise, open ArcMap, and click on the Add Data button.

·               Browse to Guadalupe.mdb, and add the data in the Hydrography, Drainage, and Network feature datasets.  Also add the ts_P_NEXRAD, ts_Q_15min, and TSTypeInfo tables.

·               Explore the data in ArcMap.  This database contains data for the Guadalupe River basin in Texas.  The database also contains more detailed data (such as Catchments) for the San Marcos River basin, which is located in the northern portion of the Guadalupe River basin.  The Basin feature class represents the four USGS hydrologic cataloging units that are within the Guadalupe River basin.  You can change the legend for Basin so that each of the four HUC units is labeled and has a different color, for visualization purposes.

 

 

1.2. Retrieving and Displaying NWIS Data

 

The USGS publishes real-time and historical water quantity and quality data for monitoring stations in the United States on its National Water Information System (NWIS) web site.  Normally, one can retrieve the data by going to the USGS web page and navigating through various web pages.  But for this exercise, we will retrieve the data automatically from within an ArcMap application, without ever having to open up a web browser.  We will do this with the NWIS tool developed by the Center for Research in Water Resources, CRWR at UT Austin.

 

A.  Familiarizing Yourself with the Data

 

·               For this part of the exercise, turn off all map layers except MonitoringPoint, HydroEdge, and Basin. 

 

 

·               Open the attribute table for MonitoringPoint.  There are 29 MonitoringPoints in this dataset.  Each MonitoringPoint represents a USGS stream gage.  The 8-digit numbers in HydroCode represent USGS stream gage IDs.  The NWIS tool uses the HydroCode to locate the appropriate data online for each gage.

 

 

B.  Loading the NWIS tool into ArcMap

 

·              Extract NWIS.dll from the Nwis.zip file included with this exercise.  You can place the dll in C:\Program Files\ESRI\ArcHydro\bin, or another directory if you wish.  Note that the NWIS tool can also be downloaded from http://www.crwr.utexas.edu/gis/archydrobook/ArcHydroTools/Tools.htm.

·              Click ToolsàCustomize.  Click the Commands tab in the Customize window, and then click Add from file.

 

 

·              Navigate to NWIS.dll (it is inside the folder NWIS/dll/as NWIS.dll), and click Open.  A window appears showing that clsNWIS was added.  Click OK.  Note:  If you get a message saying ‘No new objects added’, then you may need to have administrator privileges to install the tool.  See your network administrator for help.

 

 

·              If necessary, click on ‘NWIS’ in the Categories pane of the Customize window.  The Get NWIS Data command appears in the Commands pane.  To add this button to ArcMap, click on ‘Get NWIS Data’ with the left mouse button.  While holding down the left mouse button, drag the Get NWIS Data button next to another button in the gray area where all the commands and tools are in ArcMap.  When you see the insertion cursor (looks like a capital I,) that means that tool can be placed in that location.  Release the left mouse button to drop the tool in that location. 

 

 

·              Close the Customize window.  The Get NWIS Data button is now ready for use.

 

 

C.  Using the NWIS Tool

 

We will download daily streamflow data for the Guadalupe River at Victoria.  To do this, we will select the MonitoringPoint at that location and activate the NWIS tool.  The NWIS tool can download data for any number of selected gages, but we’ll just use one in the interest of time for this exercise.

 

·              In the attribute table for MonitoringPoint, select the record with Name = ‘Guadalupe River at Victoria TX’.  This record is now selected in the table as well as the map.

 

 

·              Click Get NWIS Data .  In the NWIS Data Retrieval window, select MonitoringPoint as the layer, HydroCode as the USGS Site Number field, and HydroID as the Feature ID field.  Then click OK.

 

 

·              In the NWIS Inputs window, enter 10/1/1999 as the Start Date, and 9/30/2000 as the end date, then click OK.  These dates encompass the water year beginning in 1999. 

 

 

·              In the Select TimeSeries Table dialog box, navigate to our Guadalupe geodatabase and type ts_Q_Daily.  Then click Save.  This is the table where we will store the USGS daily streamflow data.

 

 

·              Input 1 as the TSTypeID.  The TSTypeID identifies categories of time series data, e.g. rainfall data might have a TSTypeID of 7, while daily streamflow data might have a TSTypeID of 1.  The TSTypeID will link each time series record to a row in the TSTypeInfo table, which provides descriptive information about categories of time series data.   It may occur that you are not requested to provide the TSTypeID, and in that event, you’ll have to modify the resulting table to add it later.

 

 

The tool will now go to the NWIS web site and attempt to retrieve the requested time series data.  A window displays the tool’s progress. 

 

 

The tool ignores IDs for which no USGS gage exists.  If the available data on the USGS web site isn’t sufficient to cover the entire period of record requested by the user, the tool will only retrieve the data that is available from the web site.  When the tool is finished, a message box appears providing some information about the tool’s processing time.    Pretty cool!  You’ve just access the USGS data server in Reston, VA, at an internet address programmed into this NWIS tool, and retrieved the streamflow data without going through any html web interface at all.

 

 

·              In the ArcMap table of contents, note that the TimeSeries (Ts_Q_Daily) and TSTypeInfo tables have been added to the map document.  Open these tables and inspect the data.  They should look similar to this:

 

If your time series table does not show TsTypeID = 1 (e.g. it has a value of 0 for this field), using the Table Calculator to reset TsTypeID = 1 for the daily streamflow data you’ve just downloaded.

 

 

Note how the TSTypeInfo table describes the time series data as Daily Streamflow, in units of cfs, at regular intervals, etc.

 

You can use the Statistics tools in ArcGIS table display to define the statistics of a dataset.   Right click on the field you want to summarize and select the S Statistics option.

 

 

You’ve now loaded one water year of daily streamflow data into a table in ArcGIS.  In the next part of the exercise, we will use Excel’s capabilities to create a graph of time series data for the Guadalupe River at Victoria.

 

To be turned in: What is are the mean, minimum and maximum daily flows (cfs) for the Guadalupe River at Victoria in the 1999-2000 water year (October 1999 – September 2000) ? What is the volume of water (ft3) which passes through the gaging station in this year?

 

 

D.  Displaying Time Series in Excel

 

While ArcGIS is a powerful tool for displaying spatial data, the graphing capabilities of Excel provide a useful tool for displaying certain time series data.  Because ArcGIS uses a relational database format to store its data, this data can be imported into Excel.  In this portion of the exercise, we will import daily streamflow data for the Guadalupe River at Victoria and plot a graph.

 

·              Save the ArcMap document, and then close ArcMap.

·              Open Microsoft Excel. 

·              Click DataàGet External DataàNew Database Query…

 

 

·              In the Choose Data Source dialog box, click MS Access Database* and then click OK.

 

 

·              Navigate to Guadalupe.mdb and click Open.

 

 

·              In the Choose Columns window, locate the ts_Q_Daily table, and expand its entry by clicking the + sign next to it.  This displays the fields in ts_Q_Daily.

·              To make the plot, we will use the TSDateTime and TSValue fields.  Highlight TSDateTime by clicking on it, and then click the > button to move that field to the Columns in your query: text box. 

·              Follow the same steps to place TSValue in the window. 

·              Click Next.

 

 

·              We will not filter the data.  In the Filter Data window, just click Next to continue.

 

 

·              We will sort the data in chronological order.  Choose TSDateTime in the Sort by combo box, and make sure Ascending is selected in the options on the right.  Then click Next.

 

 

·              In the next window, choose Return Data to Microsoft Excel option, and then click Finish.

 

 

·              The next window prompts you for the location where the data will be placed.  Cell A1 in the existing worksheet is fine.  Click OK.

 

 

The data are loaded into Excel.  Date fields in ArcGIS store date information down to the millisecond, although date values are displayed according to the precision of the value.  Thus, even though the data in ts_Q_Daily represent daily records, Excel detects a higher precision, hence the MM/DD/YYYY 0:00 format in the TSDateTime field.

 

 

·              To change the TSDateTime field to display daily precision, highlight all of the values in that field, and then click FormatàCells. 

 

 

·              Choose Date as the Category, and 3/14/98 as the type.  Then click OK.

 

 

The data are now displayed with daily precision.

 

 

·              To make the plot, select all of the data, and then click the Chart Wizard button.

 

 

·              In Step 1 of the Chart Wizard, choose XY (Scatter) as the chart type and select the graph option for just lines (the default is for just points):

 

 

·              In Step 2, make sure Columns is selected as the Series in: option.  Click Next.

 

 

·              In the next two windows, you can customize the look of the plot as you see fit.  The finished result may look something like this.

 

 

·              You may now save your results and close Excel.

 

Congratulations!  You have loaded daily streamflow data from the Internet into ArcGIS, and created a plot of the data in Excel.  You are a multi-application master!

 

To be turned in: Make a layout showing the basin, streamflow station, and the excel plot of daily streamflow data (make your own design).

 

 

1.3. Viewing Time Series in a GIS

 

Graphs of time series data for a single feature can be plotted quite elegantly in Excel, but when viewing time series for multiple features simultaneously, the GIS may be the better tool for the job.  ESRI has developed a time series viewer for accomplishing such a task.  The viewer creates a pseudo-animation of time series data by rendering spatial features according to the magnitude of their time series value for a given timestamp while stepping through each timestamp.  The time series tool can also transfer time series data from one feature to another.  For example, in rainfall data from NEXRAD, Next Generation Radar (a Doppler Radar), all rainfall values are located in NEXRAD cells that can be transferred to catchment polygons, so that each catchment knows how much rain it’s getting.  In this portion of the exercise, we will use the ESRI time series viewer to display rainfall data for NEXRAD cells, and then transfer this data to catchments for the San Marcos River basin.

 

A.  Familiarizing Yourself with the Data

 

·               If necessary, open Guadalupe.mxd.

·               For this part of the exercise, turn off all map layers except HydroResponseUnit and Basin.  HydroResponseUnit represents the NEXRAD rainfall cells for the San Marcos River basin.  Each cell has an hourly rainfall time series associated with it over the period 10/12/2001 11:00 pm to 10/13/2001 2:00 pm.

 

 

·               Turn off the Basin map layer.

·               Open the ts_P_NEXRAD table.  This table contains the hourly rainfall data for the NEXRAD cells.  Each record is linked to a particular cell through the FeatureIDàHydroID association.  The given rainfall data cover a storm that occurred in October of 2001.  Close the table.

 

B.  Loading the Time Series Viewer into ArcMap

 

Adding the Arc Hydro Tools

 

·               You will need both the Arc Hydro toolbar and the Time Series toolbar for this portion of the exercise.  If the Arc Hydro toolbar is not present on the ArcMap user interface, click ViewàToolbars, and then select Arc Hydro Tools.  This displays the Arc Hydro toolbar.  You may want to reposition the toolbar for optimum viewing of the data.

 

Arc Hydro Toolbar

 

·               If Arc Hydro Tools was not one of the options in the toolbars list, you will have to add the toolbar to ArcMap.  To do so, click ToolsàCustomize.  Then click Add from file.

·               Navigate to C:\Program Files\ESRI\ArcHydro\bin, or wherever the ArcHydroTools.dll was installed.  Highlight ArcHydroTools.dll and click Open.  After a moment, you see a list of new objects added to ArcMap.  Click OK to continue.  Note:  If you get a message saying ‘No new objects added’, then you may need to have administrator privileges to install the tool.  See your network administrator for help.

·               Place a check by Arc Hydro Tools to activate the toolbar, and then close the Customize window.

 

Adding the Time Series Tools

 

·               If the time series toolbar  is not present on the ArcMap user interface, click ViewàToolbars, and then select TimeSeriesTools.  This displays the Time Series toolbar.  You may want to reposition the toolbar for optimum viewing of the data.

·               If TimeSeriesTools was not one of the options in the toolbars list, you will have to add the toolbar to ArcMap.  To do so, click ToolsàCustomize.  Then click Add from file.

·               Navigate to C:\Program Files\ESRI\ArcHydro\bin, or wherever the TimeSeriesManager.dll was installed.  Highlight TimeSeriesManager.dll and click Open.  After a moment, you see a list of new objects added to ArcMap.  Click OK to continue.  Note:  If you get a message saying ‘No new objects added’, then you may need to have administrator privileges to install the tool.  See your network administrator for help.

·               Place a check by TimeSeriesTools to activate the toolbar, and then close the Customize window.

 

 

 

C.  Viewing NEXRAD Data

 

·               Before we run the time series viewer, we need to set the target locations for storing time series data.  To do this, in the Arc Hydro toolbar click ApUtilitiesàSet Target Locations.

 

 

·               Select HydroConfig as the node to edit, and click OK.

 

 

·               Choose Guadalupe.mdb as the database to store vector and time series data, and click OK.

 

 

·               To display the time series of rainfall for the NEXRAD cells, click TSProcessingàDisplay Time Series.

 

 

·               Choose HydroResponseUnit as the layer with time series to display, and ts_P_NEXRAD as the time series table.  The tool will then update features in order to efficiently render the features for each timestamp.  This may take a few minutes.

 

 

During this step, the tool adds a field in the HydroResponseUnit table for each timestamp.  Each field is populated with the time series value for that timestamp.  To display the time series, the tool will create a color ramp for HydroResponseUnit’s legend.  For a given timestamp, each feature’s value in the field corresponding to that timestamp is matched up with the appropriate color from the color ramp.  This controls the color of the feature in the display.  As the time series tool steps through time, the values for each feature change, and thus each feature’s color changes, depending on the magnitude of the value.

 

When the tool is finished updating the features, the Time Series Viewer window opens, and the legend for HydroResponseUnit is automatically updated.

 

 

·               You may want to zoom so that you can see the entire San Marcos River basin.  Click Start to begin the time series animation.  The timestamp is displayed in the top of the map.  As time progresses, you can see the storm move from west to east across the basin.  If you are working on a slow computer, it may not be able to render the graphics fast enough, resulting in a choppy animation.  If this is the case, then pause the animation, zoom in to a smaller area (so that there are less features to render) and start the animation again.

 

  

 

 

 

D.  Transferring NEXRAD data to Catchments

 

It’s neat to watch the storm move across the basin, but we also want to know how much rain we’re getting on a catchment by catchment basis.  Thus, we need to transfer the time series data from the NEXRAD cells onto our Catchment polygons. 

 

·               Close the Time Series Viewer

·               Turn off the HydroResponseUnit layer. 

·               Turn on the Catchment layer.  These are the catchments for the San Marcos River basin.

 

 

In order to transfer the time series from the NEXRAD cells to the catchments, we first need to intersect the Catchment layer with the HydroResponseUnit layer.  This creates an individual polygon for each spatial combination of NEXRAD cells and catchment polygons. 

 

 

The time series tool uses these polygons to determine the average rainfall over a given catchment.  For example, at a given timestamp, if 30% of a catchment’s area is covered by a NEXRAD cell with a rainfall value of 1.2 inches, and 70% of the catchment’s area is covered by a cell with a rainfall value of 0.7 inches, then the average rainfall over that catchment for that timestamp is calculated as

 

0.3*1.2 inches + 0.7*0.7 inches = 0.85 inches.

 

The time series tool processes the NEXRAD cells and catchments in this manner until each catchment has a composite time series value for each time stamp.  Once each catchment has its composite value, the intersected layer is no longer needed.

 

·               To intersect the catchments with the NEXRAD cells, click ToolsàGeoProcessing Wizard.

·               In the GeoProcessing Wizard, click Intersect two layers.

·               Click Next.

·               Click the Input layer dropdown arrow and click Catchment.

·               Click the Polygon overlay layer dropdown arrow and click HydroResponseUnit.

·               Save the output in the Hydrography feature dataset of the Guadalupe geodatabase.  Call the output intCatchNEXRAD.

·               Click Finish.  It may take a few minutes to intersect the features.  Now is a good time to stand up, stretch, exercise your eyes, and relax a bit.

 

 

·               The intersected layer appears in the map.  Next, click TSProcessingàIDTransfer. 

·               Enter HydroResponseUnit as the From Layer, Catchment as the To Layer, and intCatchNEXRAD as the Intersect Layer. 

·               Click OK.

 

 

·               Enter HydroID as the Key Field for each layer.

·               Click OK.

 

 

It may take a few minutes for the tool to process the features.  In