TSTool-Vol2-CommandReference - Laserfiche WebLink

columns. If both DateColumn and TimeColumn are specified, their contents are merged with a joining colon character and are then treated as if DateTimeColumn had been specified. Required if DateColumn is specified and the interval requires time. LocationID Used with multiple data column table. The location identifier(s) to assign to time series, separated by columns if more than one column is read from the table. Column names can be specified as literal strings or as TC[start:stop] to match table column names, where start is 1+ and stop is blank to read all columns or a negative number to indicate the offset from the end column. None – must be specified for multiple column data tables. LocationType Column Used with single data column table. The name of the column containing the location type. Do not assign a location type. LocationColumn Used with single data column table. The name of the column containing the location identifier. None – must be specified for single column data tables. DataSource Column Used with single data column table. The name of the column containing the data source. Use the DataSource parameter, which can be blank. DataType Column Used with single data column table. The name of the column containing the data type. Use the DataType parameter, which can be blank. ScenarioColumn Used with single data column table. The name of the column containing the scenario. Use the Scenario parameter, which can be blank. UnitsColumn Used with single data column table. The name of the column containing the data units. Use the Units parameter, which can be blank. ValueColumn The name(s) of column(s) containing data values. None – must be 485 TableToTimeSeries () Command TSTool Documentation Command Reference – TableToTimeSeries () -8 Parameter Description Default Separate column names with commas. The TC[start:stop] notation discussed for LocationID can be used. Only one column should be specified for single data column table. specified. FlagColumn The name(s) of column(s) containing the data flag. Separate column names with commas. The TC[start:stop] notation discussed for LocationID can be used. If specified, the number of columns must match the ValueColumn parameter, although specifying blank column names is allowed to indicate that a value column does not have a corresponding flag column.. Flags are not read. DataSource The data source (provider) identifier to assign to time series for each of the value columns (or specify one value to apply to all columns). No provider will be assigned. DataType The data type to assign to time series for each of the value columns (or specify one value to apply to all columns). Use the value column names for the data types. Interval The interval for the time series. Only one interval is recognized for all the time series in the table. Interval choices are provided when editing the command. If it is possible that the date/times are not evenly spaced, then use the Irregular interval (this is difficult to do for multiple data column tables). None – must be specified. Scenario The scenario to assign to time series for each of the value columns (or specify one value to apply to all columns). No scenario will be assigned. Units The data units to assign to time series for each of the value columns (or specify one value to apply to all columns). No units will be assigned. Missing Strings that indicate missing data in the table (e.g., “m”), separated by commas. Interpret empty column values as missing data. Alias The alias to assign to time series, as a literal string or using the special formatting characters listed by the command editor. The alias is a short identifier used by other commands to locate time series for processing. No alias will be assigned. InputStart The date/time to start reading data. All data or global input start. InputEnd The date/time to end reading data. All data or global input end. 486 Command Reference – TimeSeriesToTable () -1 Command Reference: TimeSeriesToTable() Copy one or more time series into a table Version 10.21.00, 2013-06-26 The TimeSeriesToTable() command copies one or more time series into a table. This command is useful when performing table analysis processing and outputting table formats (e.g., with the WriteTableToDelimitedFile() or WriteTableToHTML() commands). The command can be configured to output one of two table forms: • Each time series in a separate column, with shared date/time column: o The time series must be regular interval (no irregular interval time series) and the intervals must match in order to allow alignment of the date/times. o Do not specify the TableTSIDColumn or TableTSIDFormat parameters. • All time series values in a single column (useful for converting time series to a stream of data for loading into a database) o Any interval is allowed although mixing time series of varying precision is discouraged. o Specify the TableTSIDColumn and optionally TableTSIDFormat parameters. Currently the command can only be used to create a new table but in the future the command is envisioned to write into an existing table. The following dialog is used to edit the command and illustrates the syntax of the command when writing a multi-column data table while also outputting data flags. Note that the value columns can be specified using time series properties. TimeSeriesToTable TimeSeriesToTable() Command Editor to Create Multi-Column Data Table 487 TimeSeriesToTable() Command TSTool Documentation Command Reference – TimeSeriesToTable() -2 The command syntax is as follows: TimeSeriesToTable(Parameter=Value,…) Command Parameters Parameter Description Default TSList Indicates the list of time series to be processed, one of: • AllMatchingTSID – all time series that match the TSID (single TSID or TSID with wildcards). • AllTS – all time series before the command. • EnsembleID – all time series in the ensemble. • FirstMatchingTSID – the first time series that matches the TSID (single TSID or TSID with wildcards). • LastMatchingTSID – the last time series that matches the TSID (single TSID or TSID with wildcards). • SelectedTS – the time series are those selected with the SelectTimeSeries() command. AllTS TSID The time series identifier or alias for the time series to be modified, using the * wildcard character to match multiple time series. Required when TSList=*TSID EnsembleID The ensemble to be modified, if processing an ensemble. Required when TSList=EnsembleID. TableID The identifier for the table to copy data into (or the identifier for the new table to create if IfTableNotFound=Create). None – must be specified. DateTimeColumn The table column name to receive date/time information. None – must be specified. TableTSIDColumn For single-column output, the name of the column in the table for time series identifier information. The format of the identifier can be specified using the TableTSIDFormat parameter. Optional – if specified will indicate single-column output. TableTSIDFormat For single-column output, indicates how to format the time series identifier that is inserted in the column specified by the TableTSIDColumn parameter. Optional – if not specified the alias or full TSID will be used. Include MissingValues For single-column output, indicates whether missing values should be transferred to the table. This is useful to screen out missing values from sparse time series. True ValueColumn The data value column name(s) to receive time None – must be specified. 488 TSTool Documentation TimeSeriesToTable () Command Command Reference – TimeSeriesToTable () -3 Parameter Description Default series data, specified as follows: • Multiple names separated by a comma. • Time series property format specifiers, available in a list of choices. These specifiers are consistent with other commands and the legend formatter in the graphing tool. • If a literal string is specified with multicolumn output, names for columns 2+ will be generated by adding a sequential number to ValueColumn. FlagColumn The data flag column name(s) to receive time series flags, specified using the same syntax as ValueColumn. A blank in the list will result in no transfer of flags for the specific time series. Do not output flags to the table. DataRow1 First table row for data (1+), where the row number is data only (column names are not considered a data row). None – must be specified. OutputStart The starting date/time for the copy. Available period. OutputEnd The ending date/time for the copy. Available period. OutputWindowStart The calendar date/time for the output start within each year. Specify using the format MM, MM-DD, MM-DD hh, or MM-DD hh:mm, consistent with the time series interval precision. A year of 2000 will be used internally to parse the date/time. Use this parameter to limit data processing within the year, for example to output only a single month or a season. Output the full year. OutputWindowEnd Specify date/time for the output end within each year. See OutputWindowStart for details. Output the full year. IfTableNotFound Indicate action if the table identifier is not matched, one of: • Create – create a new table • Warn – warn that the table was not matched Warn A sample command file is as follows (this command file is used to verify the command during testing): # Test copying annual time series to a table, and also create the table StartLog(LogFile="Results/Test_TimeSeriesToTable_Year_Create.TSTool.log") RemoveFile(InputFile="Results/Test_TimeSeriesToTable_Year_Create_out.csv", IfNotFound=Ignore) NewPatternTimeSeries(Alias=”ts1”,NewTSID="ts1..Flow.Year",SetStart="1960", SetEnd="2000",Units="ACFT",PatternValues="1,2,5,8,,20") NewPatternTimeSeries(Alias=”ts2”,NewTSID="ts2..Fl ow.Year",SetStart="1950", SetEnd="2005",Units="ACFT",PatternValues="2,4,10,16,,40") TimeSeriesToTable(TableID=TestTable,DateTimeColumn=Year,ValueColumn=%L-%T, FlagColumn=”%L-%T-flag”,DataRow=1,IfTabl eNotFound="Create") # Generate the results. WriteTableToDelimitedFile(TableID="TestTable", 489 TimeSeriesToTable() Command TSTool Documentation Command Reference – TimeSeriesToTable() -4 OutputFile="Results\Test_TimeSeriesToTable_Year_Create_out.csv") # Uncomment the following to recreate expected results # WriteTableToDelimitedFile(TableID="TestTable", # OutputFile="ExpectedResults\Test_TimeSeriesToTable_Year_Create_out.csv") CompareFiles(InputFile1="ExpectedResults/Test_ TimeSeriesToTable_Year_Create_out.csv", InputFile2="Results/Test_TimeSeriesToTable_Year_Create_out.csv",IfDifferent=Warn) The resulting table will be listed in the Tables area of the TSTool interface and clicking on the TestTable identifier will display the table similar to the following: TimeSeriesToTable2 Multi-Column Data Table 490 TSTool Documentation TimeSeriesToTable () Command Command Reference – TimeSeriesToTable () -5 The following example illustrates how to create a single data column table. Because a single column is being used for data, the data value and corresponding data flag column names are specified literally (not as time series properties). The column and format for the TSID also must be specified. TimeSeriesToTable_Single TimeSeriesToTable() Command Editor to Create Single Data Column Table 491 TimeSeriesToTable() Command TSTool Documentation Command Reference – TimeSeriesToTable() -6 The resulting table is as shown in the following figure: TimeSeriesToTable_Single2 Single Data Column Table 492 Command Reference: VariableLagK() Lag and attenuate (route) a time series with parameters that vary by rate Version 10.00.01, 2011-05-15 The VariableLagK() command can be used to lag and attenuate an input time series, resulting in a new time series. The command is commonly used to route an instantaneous flow time series through a stretch of river (reach). Lag and K routing is a common routing method that combines the concepts of: 1. Lagging the inflow to simulate travel time in a reach and, 2. Attenuating the wave to simulate the storage-outflow relationship for the reach (see Figure 1). Inflows Lagged Inflows Outflows (lagged and attenuated) Q Time Figure 1: Lag and K Routing At its fundamental level, the method solves the continuity equation using an approach similar to Muskingum routing (assuming that the Muskingum parameter representing wave storage is negligible). The governing equation for this routing method is given as: t Q Q S in out ΔΔ − = where: Qin = instantaneous inflow [rate] lagged appropriately, Qout = instantaneous outflow [rate] lagged appropriately, ΔS = change in storage in the reach [volume], Δt = time difference. 493 Command Reference – VariableLagK() -1 VariableLagK() Command TSTool Documentation The relationship assumes an outflow-storage relationship of the form: S = k ⋅ Qout, where: k = attenuation for the outflow [time]. To ensure accurate results, k should be larger or equal to Δt/2. For discrete time steps these relationships translate into: 2 , 2 1 2 1 1 1 2 2 k t t k O t I I S O Δ ≥ + Δ − Δ + + = where: I1 and I2 are the lagged inflows into the reach at the previous and current time step, respectively, O1 and O2 are the outflows out of the reach at the previous and current time step, respectively, S1 is the storage within the reach at the previous time step, defined as S1 = k⋅O1, and Δt is the time difference between the two time steps. Values for Lag and K can usually be established by comparing routed flows to downstream observations. Alternatively, the Lag can be estimated using the reach length and wave speed in the reach. Without any other information, K can be set to Lag/2. The above discussion applies where the Lag and K parameters are single values (as implemented in the LagK() command). However, there are cases where the values vary by flow, which is handled by this command. The approach that is implemented is an adaptation of that described in National Weather Service River Forecast System LAG/K documentation: http://www.nws.noaa.gov/oh/hrl/nwsrfs/users_manual/part2/_pdf/24lagk.pdf. Command Reference – VariableLagK() -2 494 TSTool Documentation VariableLagK() Command The following dialog is used to edit the command and illustrates the syntax for the command: ViriableLagK VariableLagK() Command Editor The command syntax is as follows: VariableLagK(Parameter=Value,…) Command Parameters Parameter Description Default TSID Identifier or alias for the time series to be routed. It is assumed that this series describes an instantaneous flow. Due to the lagging, the first data values required for the computation of O2 are not available within this time series and are therefore set to values set in the InflowStates parameter. None – must be specified. NewTSID Identifier for the new (routed) time series. This is required to ensure that the internal identifier for the time series is unique and accurate for the data. The interval of the identifier must be the same as for the time series specified by TSID. None – must be specified. 495 Command Reference – VariableLagK() -3 VariableLagK() Command TSTool Documentation Parameter Description Default FlowUnits The units of the flow data specified in the Lag and K tables. These units must be compatible with the time series units. The table values will be converted to the time series units if the units are not the same. None – must be specified. LagInterval The base interval for the time data specified in the Lag and K tables. The interval must be compatible with the time series base interval. The table values will be converted to the time series time interval if the intervals are not the same. For example, table data specified in Hour base interval will be converted to Minute if the time series being routed contains NMinute data. None – must be specified. Lag Flow value and lag time pairs to control routing. The units of the data values are as specified by the FlowUnits parameter (see above). The units of the lag are time as specified by the LagInterval parameter. The Lag value is not required to be evenly divisible by the time step interval; values in the time series between time steps will be linearly interpolated. Use commas and semi-colons to separate values, for example: 100.0,10;200.0,20 None – must be specified. K Flow value and K time pairs to control routing. The attenuation factor K is applied to the wave. The units of K are time as specified by the LagInterval parameter. Use commas and semi-colons to separate values, for example: 100.0,5;200.0,10 None – must be specified. InflowStates Comma-delimited list of default inflow values prior to the start of the time series. The order of the values is earliest to latest. The array must specify (Lag/multiplier) + 1 values; i.e., a 10 minute interval with a LAG of 30 must be provided with 30/10 + 1 = 4 inflow carryover values. Note: Specifying values that are not consistent with the Lag and K parameters will result in oscillation! 0 for each value CURRENTLY ALWAYS DEFAULT OutflowStates Comma-delimited list of default outflow values prior to the start of the time series. See InflowStates for details. 0 for each value CURRENTLY ALWAYS DEFAULT Alias The alias that will be assigned to the new time series. No alias will be assigned. Command Reference – VariableLagK() -4 496 TSTool Documentation VariableLagK() Command A sample command file is as follows (commands to read time series are omitted): # Test routing at 3 hour interval StartLog(LogFile="Results/Test_VariableLa gK_3hr.TSTool.log") # Read NWSCard input file ReadNwsCard(InputFile="Data\3HR_INPUT.SQIN",Alias=”Inflow”) # # Route using the same routing parameters used in the mcp3 input deck # (metric units: Lag(hrs) K(hrs) Q(cms) # Lag # K # 24.0 200.0 12.0 600.00 9.0 1500.0 42.0 3000.0 # 24.0 200.0 12.0 600.00 9.0 1500.0 42.0 3000.0 # VariableLagK(TSID="Inflow",NewTSID="TestLoc..SQIN.3Hour.route d",DataUnits=CMS, LagInterval=Hour,Lag="200,24.0;600,12.0;1500,9.0;3000,42.0", K="200,24.0;600,12.0;1500,9.0;3000,42.0",Alias="3Hr") 497 Command Reference – VariableLagK() -5 VariableLagK() Command TSTool Documentation This page is intentionally blank. Command Reference – VariableLagK() -6 498 Command Reference – WebGet() -1 Command Reference: WebGet() Retrieve a file from a website Version 10.01.00, 2011-11-15 The WebGet() command retrieves content from a website and writes the content to a local file. The transfer occurs using binary characters and the local copy is the same as that shown by View…Source (or View…Page Source) in the web browser. This command is useful for mining time series data and other content from data provider web sites. The local file can then be processed with additional commands such as ReadFromDelimitedFile(). Extraneous content (such as HTML markup around text) and inconsistencies in newline characters (\r\n for windows and \n on other systems) may lead to some issues in processing the content. Additional command capabilities may be implemented to help handle these issues. The following dialog is used to edit the command and illustrates the syntax for the command. This example reads stream gage data from the State of Colorado’s website. WebGet WebGet() Command Editor 499 WebGet() Command TSTool Documentation Command Reference – WebGet() -2 The command syntax is as follows: WebGet(Parameter=Value,…) Command Parameters Parameter Description Default URI The Uniform Resource Identifier (URI) for the content to be retrieved. This is often also referred to as the Uniform Resource Locator (URL). Global properties can be used with the ${Property} syntax. None – must be specified. LocalFile The local file in which to save the content. Global properties can be used with the ${Property} syntax. None – must be specified. 500 Command Reference: WeightTraces() Create a time series by weighting data from time series ensemble traces Version 10.00.00, 2011-03-28 The WeightTraces() command creates a new time series as a weighted sum of time series ensemble traces, for example as produced by a CreateEnsemble() command. If any trace contains missing data for a point, the resulting time series value will also be missing. Note that this approach may not be appropriate for some analyses – the user should evaluate the implications of whether the weighted result appropriately reflects the (in)dependence of input data. The following dialog is used to edit the command and illustrates the syntax of the command. WeightTraces WeightTraces() Command Editor 501 Command Reference – WeightTraces() -1 WeightTraces() Command TSTool Documentation The command syntax is as follows: WeightTraces(Parameter=Value,…) The following older command syntax is updated to the above syntax when a command file is read: TS Alias = WeightTraces(Parameter=Value,…) Command Parameters Parameter Description Default EnsembleID The ensemble identifier indicating time series to be processed (e.g., from a CreateEnemble() command). Time series matching the years specified by the Weights parameter will be processed. None – must be specified. NewTSID The time series identifier for the new time series that is created. This typically uses the same information as the original time series, with an added scenario. None – must be specified. Alias The alias to assign to the time series, as a literal string or using the special formatting characters listed by the command editor. The alias is a short identifier used by other commands to locate time series for processing, as an alternative to the time series identifier (TSID). None – must be specified. SpecifyWeightsHow Weights are currently only applied as AbsoluteWeights (in the future an option may be added to normalized weights to 1.0 accounting for missing data in the traces). Must be AbsoluteWeights. Weights Specify pairs of trace year and weights (0-1.0), used to create the new time series. Trace years must be manually entered because at the time that the command is edited, time series have not yet been queried. The weights do not need to add to 1. Example data are: 1995,.5,1998,.3,2005,.2 None – must be specified. Command Reference – WeightTraces() -2 502 TSTool Documentation WeightTraces() Command A sample commands file is as follows (longer commands that wrap are shown indented): # Create annual traces from a time series shifted to the current year # The original time series is read from HydroBase # # (1995-1998) ALAMOSA RIVER ABOVE JASPER, CO USGS Streamflow Day 08235350.USGS.Streamflow.Day~HydroBase CreateEnsemble(TSID="08235 350.USGS.Streamflow.Day",TraceLength=1Year, EnsembleID="Ensemble_Jasper", EnsembleName="ALAMOSA RIVER ABOVE JASPER, CO", ReferenceDate="2008-01-01",ShiftDataHow=ShiftToReference) WeightTraces(Alias=” WeightedTS”,EnsembleID="Ensemble_Jasper", SpecifyWeightsHow="AbsoluteWeights", Weights="1997,.5,1998,.4,1999,.1", NewTSID="08235350.USGS.Streamflow.Day.weighted") WriteDateValue(OutputFile="Results/W eightTraces_out.dv") UserManualExamples/TestCases/CommandReference/WeightTraces/WeightTraces.TSTool The results from the commands are shown in the following graph: WeighTraces_Graph Results of the WeightTraces() Command 503 Command Reference – WeightTraces() -3 WeightTraces() Command TSTool Documentation This page is intentionally blank. Command Reference – WeightTraces() -4 504 Command Reference: WriteCheckFile() Write a check file containing a summary of data/processing problems Version 09.03.04, 2009-04-23 The WriteCheckFile() command summarizes the results of command processing warning/failure messages in a “check file”. This file is useful for reviewing results and for quality control. The check file is essentially a persistent record of any problems that occurred during processing, whereas a full log file contains a sequential list of processing. The following dialog is used to edit the command and illustrates the syntax for the command. WriteCheckFile WriteCheckFile() Command Editor The command syntax is as follows: WriteCheckFile(Parameter=Value,…) Command Parameters Parameter Description Default OutputFile The name of the check file to create, enclosed in double quotes if the file contains spaces or other special characters. A path relative to the command file containing this command can be specified. Specify a filename with .html extension to generate an HTML file or .csv to generate a comma-separated value file suitable for use with Excel. The HTML file will contain more information and include navigation links. None – must be specified. 505 Command Reference – WriteCheckFile() -1 WriteCheckFile() Command TSTool Documentation This page is intentionally blank. Command Reference – WriteCheckFile() -2 506 Command Reference – WriteDateValue() -1 Command Reference: WriteDateValue() Write time series to a DateValue format file Version 10.06.00, 2012-04-05 The WriteDateValue() command writes time series to the specified DateValue format file. See the DateValue Input Type Appendix for more information about the file format. The time series being written must have the same data interval – use the TSList parameter to select appropriate time series. The following dialog is used to edit the command and illustrates the syntax of the command. WriteDateValue WriteDateValue() Command Editor The command syntax is as follows: WriteDateValue(Parameter=Value,…) Command Parameters Parameter Description Default TSList Indicates the list of time series to be processed, one of: • AllMatchingTSID – all time series that match the TSID (single TSID or TSID with wildcards) will be processed. • AllTS – all time series before the command. • EnsembleID – all time series in the ensemble will be processed. • FirstMatchingTSID – the first time series that matches the TSID (single TSID or TSID AllTS 507 WriteDateValue() Command TSTool Documentation Command Reference – WriteDateValue() -2 Parameter Description Default with wildcards) will be processed. • LastMatchingTSID – the last time series that matches the TSID (single TSID or TSID with wildcards) will be processed. • SelectedTS – the time series are those selected with the SelectTimeSeries() command. TSID The time series identifier or alias for the time series to be processed, using the * wildcard character to match multiple time series. Required if TSList=*TSID. EnsembleID The ensemble to be processed, if processing an ensemble. Required if TSList= EnsembleID. OutputFile The DateValue output file. The path to the file can be absolute or relative to the working directory (command file location). Global properties can be used to specify the filename, using the ${Property} syntax. None – must be specified. Delimiter The delimiter character to use between data values. Comma is the only other allowed value other than the default space and is recommended for irregular time series, which are output as blanks when date/times don’t align with other time series. Space. Precision The number of digits after the decimal for numerical output. 4 (in the future may default based on data type) MissingValue The value to write to the file to indicate a missing value in the time series. As initialized when reading the time series or creating a new time series, typically -999, NaN, or another value that is not expected in data. OutputStart The date/time for the start of the output. Use the global output period. OutputEnd The date/time for the end of the output. Use the global output period. Irregular Interval The interval (e.g., Day) used when writing irregular time series, to indicate the precision of date/times. This may be necessary when it is not possible to automatically determine the date/time precision. The date/time precision to format output is assumed to be Minute if unknown; however, specifying the irregular interval will inform the data processing. Determined from the period start date/time of each time series, defaulting to Minute where the date/time precision is set to “irregular” (unknown). A sample command file to process data from the State of Colorado’s HydroBase database is as follows: # 0100503 -RIVERSIDE CANAL 0100503.DWR.DivTotal.Month~HydroBase WriteDateValue(OutputFile="Diversions.dv") 508 Command Reference: WriteHecDss() Write time series to a HEC-DSS File Version 09.03.00, 2009-04-10 The WriteHecDss() command writes time series to a HEC-DSS file. See the HEC-DSS Input Type Appendix for information about how time series properties are output to HEC-DSS files. Current limitations of the command are: • Irregular time series are not supported – the focus of initial development has been regular interval time series. • 24-hour time series in TSTool cannot be written to HEC-DSS because HEC-DSS only supports 1DAY interval. Therefore, the