Introducing OpenMI

Jon Goodall, Duke University

 

Table of Contents

·         Introduction

·         The Big Picture: Why use OpenMI?

·         Developing an OpenMI Linkable Component: Case Study of Wrapping Arc Hydro

·         Running OpenMI 

·         Contact Information

Introduction

 

OpenMI (short for Open Model Integration Environment) is a data and model integration framework.  The primary objective is to take otherwise independent data and computing systems, for example a database storing rainfall time series, a rainfall-runoff model, and a river hydraulics model, and provide a standard means for describing how time series are communicated between the systems.  OpenMI was developed by HaromIT, which consists of a number European environmental software companies and university researchers.  Software companies like DHI are working to make their models “OpenMI-compliant”, thereby allowing environmental modelers the ability to couple different DHI models into modeling frameworks. The following links provide additional background information on OpenMI.

 

 

In this document, I’ll cover two main topics.  The first is how to create and OpenMI component.  The case study is wrapping an Arc Hydro geodatabase to supply boundary conditions for a model written in FORTRAN that accumulates river flows going downstream.  The FORTRAN code is included as part of the OpenMI install package as an example of how to wrap legacy code (typically C or FORTRAN) and make that code OpenMI-compliant.  What I show is how modify this example so that instead of input flow time series for the model coming from a text file, it is instead stored in an Arc Hydro geodatabase (in the TimeSeries and TSType tables).  The second topic covers how to link OpenMI components using the OmiEd application.  It assumes that you have Visual Studio .Net installed and that you are familiar with developing DLL projects, implementing interfaces, etc.  If you have created buttons or toolbars in ArcGIS using ArcObjects, creating OpenMI components is done is a similar manner.   The code is written in C# but could be written in any .Net language (VB, C++, J#).  A Java version of OpenMI is in the works, but has not been released to date.  Check the OpenMI site above for more information.  I have provided the necessary files to complete the second topic without having to work through building an OpenMI component. 

 

The Big Picture: Why use OpenMI?

 

It’s easy to get lost in the details of creating an OpenMI component, so let me first start with a big picture view.  One way to think of OpenMI is as a flexible, extensible modeling engine.  It’s flexible and extensible because developers can make their databases, text files, spreadsheets, FORTRAN models, C models, etc Open-MI compliant.  This basically means that developers tell OpenMI what the values the data source or model is capable of supplying and how to extract these values from the source or model.  OpenMI handles all of the coordination issues between different data source and models (i.e. if a groundwater model needs recharge from a watershed model, OpenMI knows when and how to get the value from the watershed model).  As you would imagine, one of the most difficult issues of model and data integration is misalignment in space and time.  The computational mesh for a watershed, for example, may not match up with the computational mesh for a groundwater model.  OpenMI is designed to handle these misalignments through user defined linkages (watershed node 1 links to groundwater node 2) or spatial linkages (linkages are derived by the spatial overlap and intersection of features). 

 

The issues that arise in model and data integration can become quite complex and OpenMI is a valiant attempt to create a system capable of facilitating the integration of heterogeneous data sources and models.  That said, it still requires a fairly significant amount of work for a model developer to make his or her model “OpenMI compliant”.  The most significant change that needs to be made to make a model OpenMI-compliant is “stripping out” the time looping component of the model so that OpenMI can the model more interactively through time. Depending on the complexity of your model, this process could take weeks to months to complete (as estimated by the OpenMI group themselves).  Thus, OpenMI might not be the way to go for all modeling activities, but as we become more interdisciplinary and need to integrate models capable of computing different components of the water and biochemical cycles, the applicability of a modeling framework like OpenMI becomes more attractive. 

 

Developing an OpenMI Linkable Component: Case Study of Wrapping Arc Hydro

 

There are a many pages of documentation for OpenMI available from their website.  In this example, I will not go into details for implementing an OpenMI Linkable Component interface, but instead keep the discussion more focused on the specific example of wrapping Arc Hydro.  If you are interested in creating an OpenMI Linkable Component, this tutorial, along with the documentation available from the OpenMI site (specifically the guidelines document http://www.openmi.org/documents/B_Guidelines.pdf), should walk you through the steps. 

 

Implementing ILinkableComponent

 

Although there are a number of methods and properties of the class ILinkableComponent, when you implement this interface for a data source, most of the code will be devoted to four methods and properties (TimeHorizon, GetValues, GetOutputExchangeItem, and Initiate).  I briefly describe the purpose of each method/property below and then provide the source code for implementing each method/property for an Arc Hydro geodatabase.

 

TimeHorizon – will return an ITimeSpan object that stores the time span over which the component has time series to share.  In this case, we are making a database an OpenMI component, so the TimeHorizen would be the time of the first and last observation within the database.

 

GetValues – will return an IValueSet object that stores the values for a particular time and link.  A link is the connection between models that describes what time series are being exchanged between them.   

 

GetOutputExchangeItem – will return an IOutputExchangeItem object that stores an element set and quantity, given the index of the output exchange item.  

 

Initialize – sets up the component by setting the time horizon and populating a list of available exchange items

 

The Initialize Method

 

In the Initialize method, we are reading the database and learning what data is available for use by other OpenMI components.  I have separated most of “heavy lifting” in the code to a function named readArcHydro().  In the Initiative method itself, I just create two new arrays to store time stamps and links and set the culture variable to the current culture of the machine (this is meant to handle situations where different cultures might use commas instead of periods as decimal places in floating point values, for example). 

 

 

Text Box: public void Initialize(org.OpenMI.Standard.IArgument[] properties) { _timeStamps = new ArrayList(); _culture = CultureInfo.CurrentCulture.NumberFormat; _links = new Hashtable(); readArcHydro(); } The readArcHydro() function contains the logic for extracting contains three main sections: (1) create an array of quantities, (2) create an array of output exchange items, (3) create an array of time stamps within the database.  I’m using an OleDB connection to link to a personal geodatabase (i.e. Access database) in which the Arc Hydro data is stored.  From there, the task is to simply formulate SQL queries to extract the necessary information from Arc Hydro (e.g. quantity information comes from the TSType table).  The global variable _geodatabasePath is a string that stores the path to the ArcHydro database on file (e.g. “C:/myArcHydro.mdb”)

 

Text Box: private void readArcHydro() { OleDbConnection conn = new OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + _geodatabasePath) ; // create quantities conn.Open(); OleDbCommand cmd = new OleDbCommand("SELECT [Variable], [Units] FROM [TSType]", conn); cmd.CommandType = CommandType.Text; OleDbDataReader rdr = cmd.ExecuteReader(); Quantity quantity; _quantities = new ArrayList(); while (rdr.Read()) { Dimension flowDimension = new Dimension(); //NOTE: unit conversion is hard code for now, should come from database Unit flowUnit = new Unit((string) rdr["Units"],1,0, (string) rdr["Units"]); quantity = new Quantity(flowUnit,Convert.ToString(rdr["Variable"]), "InFlow",org.OpenMI.Standard.ValueType.Scalar,flowDimension); _quantities.Add(quantity); } conn.Close(); // create output exchange items OutputExchangeItem output; ElementSet elementSet; _outputs = new ArrayList(); for (int i = 0; i < _quantities.Count; ++i) { // create element set conn.Open(); // not directly using the shape to avoid including ArcObjects cmd = new OleDbCommand("SELECT HydroID, LocalX, LocalY FROM MonitoringPoint", conn); rdr = cmd.ExecuteReader(); int j = 0; while (rdr.Read()) { elementSet = new ElementSet("description","Node: " + j, ElementType.XYPoint, new SpatialReference("ref")); Element e = new Element(Convert.ToString(rdr["HydroID"])); e.AddVertex(new Vertex(Convert.ToDouble(rdr["LocalX"]),Convert.ToDouble(rdr["LocalY"]),0)); elementSet.AddElement (e); output = new OutputExchangeItem(); output.Quantity = (Quantity) _quantities[i]; output.ElementSet = (ElementSet) elementSet; _outputs.Add(output); ++j; } conn.Close(); } // create time stamps conn.Open(); cmd = new OleDbCommand("SELECT [TSDateTime] FROM [TimeSeries] GROUP BY [TSDateTime]", conn); rdr = cmd.ExecuteReader(); _timeStamps.Clear(); double modifiedJulianDateTime; while (rdr.Read()) { DateTime timestamp = Convert.ToDateTime (rdr["TSDateTime"], _culture); modifiedJulianDateTime = CalendarConverter.Gregorian2ModifiedJulian (timestamp); _timeStamps.Add (modifiedJulianDateTime); } conn.Close(); Console.WriteLine("done reading db"); } The OutputExchangeItem Method

 

We’ve already created an array of output exchange items in the readArcHydro() function, so the OutputExchangeItem simply returns a single output exchange item based on the index passed to the method. 

 

Text Box: public org.OpenMI.Standard.IOutputExchangeItem GetOutputExchangeItem(int outputExchangeItemIndex) { return (OutputExchangeItem) _outputs[outputExchangeItemIndex]; } The GetValues Method

 

Keep in mind that the OutputExchangeItems contain only metadata about the available time series within the database and do not contain the values.  To retrieve time series values, OpenMI will call the components GetValues method which returns data one time-slice per request.  Again, this code creates a connection to the database, performs a SQL query to extract the values from the TimeSeries table based on a particular time, and then returns these values as an OpenMI IValueSet object.  NOTE: this code assumes there is only one TSType within the database.  To make the code more generic, the variable name could be obtained from the linked, and then that variable name could be included in the SQL query statement. 

 

Text Box: public org.OpenMI.Standard.IValueSet GetValues(org.OpenMI.Standard.ITime time, string linkID) { OleDbConnection conn = new OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + _geodatabasePath) ; ArrayList locationValues = new ArrayList(); TimeStamp timestamp = (TimeStamp) time; OleDbCommand cmd = new OleDbCommand("SELECT [TSValue] FROM [TimeSeries]" + " WHERE [TSDateTime] = #" + CalendarConverter.ModifiedJulian2Gregorian((double) timestamp.ModifiedJulianDay) + "#", conn); // Access needs #'s around date, SQL Server requires the date to be a string [format: '1/1/1990'] conn.Open(); OleDbDataReader rdr = cmd.ExecuteReader(); while (rdr.Read()) { locationValues.Add( (double) rdr["TSValue"]); } conn.Close(); double[] results = (double[]) locationValues.ToArray(typeof(double)); return new ScalarSet(results); } The TimeHorizon Property

 

Like the OutputExchangeItem method, this property is pretty simple because we’ve already created an array of all time stamps within the database in the readArcHydro() function. 

 

Text Box: public org.OpenMI.Standard.ITimeSpan TimeHorizon { get { double simulationStartTime = (double) _timeStamps[0]; double simulationEndTime = (double) _timeStamps[_timeStamps.Count -1]; return new org.OpenMI.Backbone.TimeSpan(new TimeStamp(simulationStartTime),new TimeStamp(simulationEndTime)); } } The OMI File

 

Once you have created the class to wrap Arc Hydro by implementing the ILinkableComponent, the final step to integrate this class with OpenMI is to create an OMI file.  An OMI file (i.e. *.omi) is an XML file that stores the location of a .Net dll containing a class that implement ILinkableComponet) and other information necessary for displaying an OpenMI component within the Configuration Editor. 

 

For the ArcHydro linkable component we created in the first part of the tutorial, the OMI would contain the following text. 

 

Text Box: <?xml version="1.0"?> <LinkableComponent Type="CUAHSI.OpenMI.ArcHydro" Assembly="bin\debug\OpenMI_Example.dll"> <Arguments> <Argument/> <Argument Key="Model" ReadOnly="true" Value="NA" /> </Arguments> </LinkableComponent>  

 

 

 

 

 

 

 

The Type attribute is the namespace of the linkable component class.  The assembly is a relative path to the dll file containing that class.  As it currently exists, the component has no arguments.  One possible enhancement to the current model design would be to add an argument for the path to the Arc Hydro geodatabase.

 

That’s it for the development work.  The next section will describe how to take what you’ve produced here and integrate it with the simple river model using OpenMI’s OmiEd application.

 

Running OpenMI

 

 

Now that we’ve covered the “behind the scenes” issues of making a database OpenMI-compliant, we’ll now walk through coupling the Arc Hydro component with the Simple River Model component.

 

Before getting started, you need to …

  1. download and install OpenMI.  You can do so by using an installer available on this page: http://sourceforge.net/projects/openmi.  I’m using OpenMI version 1.0 in this tutorial. 
  2. download the tutorial files to C:\temp\OpenMI.  This includes the source code described above and the OpenMI omi and opr files for the project.

 

 

  1. Open the OmiEd application (Start à All Programs à OpenMI à OmiEd)

 

 

  1. Add the Arc Hydro model to OmiEd
    1. Composition à Add Model

 

 

    1. Browse to the ArcHydro.omi file and open it (this will be in the root folder of the tutorial files available above).

 

 

    1. View the properties of the component by right clicking on the yellow rectangle and choosing properties. 

 

 

 

From this dialog we can see that Arc Hydro has output exchange items of Flows (with units m3/sec) for four nodes.  If you’d like, open the Arc Hydro geodatabase to verify this information.

 

  1. Add the Simple River Model to OmiEd
    1. Composition à Add Model
    2. Browse to the SimpleRhineRiver.omi file and open it (this will be in the \SimpleRiver\Wrapper\UnitTest\ folder of the tutorial files available above).

 

 

    1. View the properties of the river model component (Right-click on component à properties).

 

 

From this dialog we can see that the river model can accept input exhange items for each branch and node and that it produces output flows for each branch.

 

  1. Add a connection between the Arc Hydro component and then Simple River Model component
    1. Composition à Add Connection
    2. Left-click on the Arc Hydro component and then left-click on the Simple River Model component.  You should then see a blue line linking the two components with the arrow pointing from Arc Hydro (the data source) to the Simple River Model.

 

 

Here we are telling OpenMI that the river model is dependent on input values from Arc Hydro. 

 

  1. Edit the connection properties
    1. Right-click on the connection to view it’s properties
    2. From the output exchange item list, choose Node 0 and from the input exchange item list, choose Node 0.  Then click Apply to create this link.  Do the same for Node 1 so that your dialog looks the same as the one shown below.  Close the dialog.

 

 

Here we are telling OpenMI how Arc Hydro and the river model are connected, specifically that Node 0 within Arc Hydro supplies an input exchange item for node 0 of the River Model, and likewise for node 1 of each component.

 

  1. Add a trigger to the model
    1. Composition à Add Trigger

 

  1. Add a connection between the trigger and the river model
    1. Add a connection between the Simple River Model component and the trigger the same why you connected the Arc Hydro component with the Simple River Model component.

 

 

  1. Edit the connection’s properties
    1. Right-click on the connection from the river model to the trigger and view it’s properties
    2. Under output exchange items, click Branch:2 and under input exchange items choose TriggerElementID.  Click Apply and then Close.

 

 

You’ve just told OpenMI what it should calculate.  The model will run until the trigger value (flow on Branch:2) is known.

  1. Run the model [note that you may want to save the project before running]
    1. Composition à Run

 

 

    1. Choose the message you’d like to view as the model runs, click the “Latest overlapping” button to make the model run for the maximum time within the database, and then click run. The code should only take a second to complete.

 

 

    1. Close the messages dialog and you’ll get a warning like what’s shown below.  Choosing Yes will allow you to run the model again, choosing no will let you view the results.  Choose no this time so we can see the results file.

 

 

    1. Open and view the results \SimpleRiver\Wrapper\UnitTest\SimpleRiver.out

 

 

[note: if you have any trouble running the model, I have included a project file in the tutorial file package: ArcHydro2SimpleRiverModel.opr]

 

The results show the time (Julian days) and flow in m3/second for each branch.  Again, the model itself is not terribly interesting, but the integration of a FORTRAN model with an Arc Hydro geodatabase being facilitated through OpenMI standards is a very powerful concept.  It is far simpler to manipulate this example to have the input data come from a text file, or to have Arc Hydro feed some other modeling routine because of OpenMI.  Yes, it may take more time to make use of OpenMI standards, but in many cases, the flexibility it affords will justify the additional development effort.

 

Primary Contact:

Jon Goodall
Duke University

e-mail: jon.goodall@duke.edu
Phone: (919) 613-8716
webpage: http://www.nicholas.duke.edu/faculty/goodall.html

These materials may be used for study, research, and education, but please credit the authors and the Center for Research in Water Resources, The University of Texas at Austin. All commercial rights reserved. Copyright 2006 Center for Research in Water Resources.