TSTool-Vol2-CommandReference

• Title: TSTool - CommandReference - 10.21.00
• Description: Command Reference for Version 10.21.00
• Decision Support - Doc Type: Software Documentation
• Date: 7/14/2013
• DSS Category: DMI Utilities
• Prepared By: Riverside Technology, Inc.

to be processed, one of: • AllMatchingTSID – all time series that match the TSID (single TSID or TSID with wildcards) will be processed. • AllTS – all time series before the command
will be processed. • EnsembleID – all time series in the ensemble will be processed. • FirstMatchingTSID – the first time series that matches the TSID will be processed. • LastMatchingTSID
– the last time series that matches the TSID (single TSID or TSID with wildcards) will be processed. • SelectedTS – the time series selected with the SelectTimeSeries() command will
be processed. AllTS TSID The time series identifier or alias for the time series to be modified, using the * wildcard character to match multiple time series. Required when TSList=*TSID.
EnsembleID The ensemble to be modified, if processing an ensemble. Required when TSList= Command Reference – ResequenceTimeSeriesData () -2 392
TSTool Documentation ResequenceTimeSeriesData() Command Parameter Description Default EnsembleID. TableID The identifier for the sequence table to use, which indicates the dates to use
when resequencing data (e.g., list of years for data sequence). For example, see the ReadTableFromDelimitedFile() command. The years should be consistent with the OutputYearType. None
– must be specified. TableColumn The column name containing the sequence information. Note that the input table must have column names in a header record. None – must be specified. TableRowStart
The first data row number (1+) containing the first year in the new sequence. Use all rows. TableRowEnd The last data row number (1+) containing the first year in the new sequence. Use
all rows. OutputYearType The output year type, indicating the year extent for the resequencing, one of: • Calendar – January to December • NovToDec – November of previous calendar year
to October of current year. • Water – October of previous calendar year to September of current year. Calendar OutputStart The output start as a four-digit year that is consistent with
OutputYearType. For example, if processing water years, the OutputStart would be the first water year in the output (and start in October of the previous calendar year). The output end
is relative to the output start and includes the number of years in the sequence. Same as the original input data or use the global output start if specified. The output months will
be adjusted for the output year type. NewScenario The new scenario to assign to the created time series, resulting in a unique TSID. Not specified, but a new scenario and/or alias must
be specified. Alias Alias to assign to the output time series. See the LegendFormat property described in the TSView Time Series Viewing Tools appendix. For example, %L is full location,
%T is data type, %I is interval, and %Z is scenario. Not specified, but a new scenario and/or alias must be specified. 39 3 Command Reference – ResequenceTimeSeriesData() -3
ResequenceTimeSeriesData() Command TSTool Documentation The following example: 1. Reads a list of time series from a StateMod model file. 2. Reads a sequence of years from a delimited
file. 3. Resequences the StateMod time series data. 4. Writes the resequenced file to a new StateMod file. # Read all demand time series… ReadStateMod(InputFile=”..\StateMod\gunnC2005.xbm”)
# Read the sequence of years to use… Table 0001HK0101 = ReadTableFromDelimitedFile(InputFile=”0001HK0101.csv”) # Resequence the StateMod time series… ResequenceTimeSeriesData(TSList=AllTS,TableID=”00
01HK0101”, TableColumn=”Trace1”,NewScenario=”KNN0101”,Alias=”%L-KNN0101”) # Write the resequenced data for StateMod WriteStateMod(TSList=AllMatchingTSID,TSID=”*KNN*”, OutputFile=”..\StateMod0101\gunn
C2005.xbm”) The year sequence is specified in a file similar to the following. # Some comments “Trace1”,”Trace2”,… 1905,1967,… 1920,1943,… etc. Variations on the example can be implemented,
for example, to process output products after the run. Command Reference – ResequenceTimeSeriesData () -4 394
Command Reference – RunCommands() -1 Command Reference: RunCommands() Run a command file Version 10.20.00, 2013-04-20 The RunCommands() command runs a command file using a separate command
processor as a “child” of the main processor. This command can be used to manage workflow where multiple commands files are run, and is also used extensively for testing, where a test
suite consists of running separate test case command files. Command files that are run can themselves include RunCommands() commands. Each command file that is run has knowledge if its
initial working directory and relative paths referenced in the command file are relative to this directory. This allows a master command file to reside in a different location than the
individual command files that are being run. The current working directory is reset to that of the command file being run. Data stores from the parent command processor are by default
passed to the child command processor. Consequently, database connections can be opened once and shared between command files. Currently the properties from the parent command file are
NOT applied to the initial conditions when running the command file. Therefore, global properties like input and output period are reset to defaults before running the command file.
A future enhancement may implement a command parameter to indicate whether to share the properties with the parent processor. The output from the command is also not added to the parent
processor. Again, a future enhancement may be to append output so that one final set of output is generated. There is currently no special handling of log files; consequently, if the
main command file opens a log file and then a command file is run that opens a new log file, the main log file will be closed. This behavior is being evaluated. The following dialog
is used to edit the command and illustrates the syntax for the command. RunCommands RunCommands() Command Editor 395
RunCommands() Command TSTool Documentation Command Reference – RunCommands() -2 The command syntax is as follows: RunCommands(Parameter=Value,…) Command Parameters Parameter Description
Default InputFile The name of the command file to run, enclosed in double quotes if the file contains spaces or other special characters. A path relative to the master command file can
be specified. None – must be specified. ExpectedStatus Used for testing – indicates the expected status from the command, one of: • Unknown • Success • Warning • Failure Success ShareDataStores
Indicate whether data stores in the parent should be shared with the child command processor. Normally this should be done so that databases can be opened once. Note that opening data
stores in the child command file will not make the data stores available in the parent. Share The following example illustrates how the RunCommands() command can be used to test TSTool
software (or any implementation of commands). First, individual command files are implemented to test specific functionality, which will result in warnings if a test fails: StartLog(LogFile="Results/
Test_ReadStateMod_1.TSTool.log") NewPatternTimeSeries(NewTSID="MyLoc..MyData.Day",Alias=”TS”, Description="Test data",SetStart="1950-01-01", SetEnd="1951-03-12",Units="CFS",PatternValues="5,10,12,13,
75") # Uncomment the following command to regenerate the expected results file. # WriteStateMod(TSList=AllTS, # OutputFile="ExpectedResults\Test_ReadStateMod_1_out.stm") ReadStateMod(InputFile="Expec
tedResults\Test_ReadStateMod_1_out.stm") CompareTimeSeries(Precision=3,Tolerance=".001",DiffFlag="X", WarnIfDifferent=True) Next, use the RunCommands() command to run one or more tests:
StartRegressionTestResultsReport( OutputFile="RunRegressionTest_commands_general.TSTool.out.txt") … RunCommands(InputFile="..\..\..\commands\general\ReadStateMod\Test_ReadStateMod_1.TSTool")
… Each of the above command files should produce expected results, without warnings. If any command file unexpectedly produces a warning, a warning will also be visible in TSTool. The
issue can then be evaluated to determine whether a software or configuration change is necessary. See the StartRegressionTestResultsReport() command documentation for an explanation
of how to disable a command file with #@enabled False or indicate its expected status for testing (e.g., @#expected Status Warning). 396
Command Reference: RunDSSUTL() Run the DSSUTL and other utility programs from the US Army Corps of Engineers Version 09.03.00, 2009-04-10 The RunDSSUTL() command runs the Army Corps
of Engineers’ DSSUTL program and other utility programs, which are used with HEC-DSS files. See also the HEC-DSS Input Type appendix. This command formats the command line for the program,
runs the program, and checks the exit value. A non-zero exit value will result in a failure status for the command. TSTool internally maintains a working directory that is used to convert
relative paths to absolute paths in order to locate files. The working directory is by default the location of the last command file that was opened. The location of the program being
run (e.g., DSSUTL.EXE) is determined by the operating system using the PATH environment variable; therefore, use the ${WorkingDir} property in the command line if the program location
is not in PATH. Use \” in the command line or arguments to surround whitespace. It is not clear whether DSSUTL and other program have limits on path or filename length, but if this appears
to be the case, use shorter names. If a program is not provided with correct input, it may go into interactive mode, in which case TSTool may appear to stop when running the command.
Currently there is no way to kill the process and TSTool must be stopped and restarted. The following table summarizes how the command treats input for various utility programs. Required
arguments are for the RunDSSUTL() command but may be optional if the program is run on the command line. RunDSSUTL() Command Handling of HEC-DSS Utility Program Input Progam Description
DSSFILE= Argument INPUT= Argument OUTPUT= Argument DSSUTL Data Storage System Utility Program Required Required Optional DSPLAY Data Storage System Graphics Utility Required Required
Optional DSSMATH Utility Program for Mathematical Manipulation of HEC-DSS Data Not used – use OPEN() command. Required Optional REPGEN Report Generator – not fully supported due to different
command line argument conventions. DSSTS Regular Interval Time-Series Data Entry Program Required Required Optional DSSITS Irregular Interval Time-Series Data Entry Program Required
Required Optional DSSPD Paired Data Entry Program Required Required Optional DSSTXT Text Data Entry Program Required Required Optional DWINDO Interactive Data Entry and Editing This
interactive program is not supported by RunDSSUTL() command. WATDSS Watstore to DSS Data Entry Program – not fully supported due to different command line argument Required Required
Optional 397 Command Reference – RunDSSUTL() -1
RunDSSUTL() Command TSTool Documentation Progam Description DSSFILE= Argument INPUT= OUTPUT= Argument Argument conventions. NWSDSS National Weather Service to Data Storage System Conversion
Utility – not fully supported due to different command line argument conventions. Required Required Optional PREAD Functions, Macros, and Screens – not fully supported due to interactive
prompts. The following dialog is used to edit the command and illustrates the command syntax. Note that the DSSUTL.EXE location is in this case not included in the PATH environment variable
and is specified with the DssutlProgram parameter, using ${WorkingDir}. The HEC-DSS and input files are relative to the working directory. RunDSSUTL RunDSSUTL() Command Editor when Specifying
Command Line Command Reference – RunDSSUTL() -2 398
TSTool Documentation RunDSSUTL() Command The command syntax is as follows: RunDSSUTL(Parameter=Value…) Command Parameters Parameter Description Default DssFile The HEC-DSS filename as
an absolute path or relative to the working directory. The file must exist because TSTool does not interface with the program interactive mode prompts. The parameter is passed to the
program using the DSSFILE= command line argument. None – must be specified for most programs. InputFile The DSS utility program command file to run. The file must exist because TSTool
does not interface with the utility program interactive mode prompts. The input file name is passed to the program using the INPUT= command line argument. None – must be specified. OutputFile
The DSS utility program output file, which contains logging information. This is passed to the program using the OUTPUT= command line argument. Specifying the argument will cause output
to be printed to the file and not the screen. Note that some utility program commands write to other output files (controlled by the command file or other command line arguments), which
should not be confused with the output file for this argument. Not required – output will be to screen if command shell window is shown. DssutlProgram The DSS utility program to run.
The PATH environment variable is used to locate the executable if a full path is not specified. Specify the specific DSS utility program to run if the default value is not appropriate.
If not specified, DSSUTL.EXE will be used and must be located in a directory listed in the PATH environment variable. 399 Command Reference – RunDSSUTL() -3
RunDSSUTL() Command TSTool Documentation This page is intentionally blank. Command Reference – RunDSSUTL() -4 400
Command Reference – RunningAverage() -1 Command Reference: RunningAverage() Convert time series data to running average values Version 10.13.00, 2012-10-25 The RunningAverage() command
converts a time series’ raw data values to a running average, resulting in data that are smoothed. New time series are NOT created (note that the newer RunningStatisticTimeSeries() command
has more flexibility and the RunningAverage() command may be phased out in the future). There are several approaches to computing the running average (as specified by the AverageMethod
command parameter): • The centered running average requires that the number intervals on each site of a point be specified (e.g., specifying 1 will average 3 values at each point). •
The previous/future running average requires that the number of intervals prior to or after the current point be specified. • The N-year running average is computed by averaging the
current year and N -1 values from previous years, for a specific date. An average value is produced only if N non-missing values are available. Currently N-year running average values
for Feb 29 for daily or finer data will always be missing because a sufficient number of values will not be found – an option may be added in the future to allow Feb 29 values to be
computed based on fewer than N values. • A special case of the N-year running average (NAllYear) is to use all previous years’ and the current value. The following dialog is used to
edit the command and illustrates the centered running average command syntax. RunningAverage_centered RunningAverage() Command Editor for Centered Running Average 401
RunningAverage() Command TSTool Documentation Command Reference – RunningAverage() -2 The following dialog illustrates the N-year running average command syntax. RunningAverage_nyear
RunningAverage() Command Editor for N-Year Running Average 402
TSTool Documentation RunningAverage() Command Command Reference – RunningAverage() -3 The command syntax is as follows: RunningAverage(Parameter=Value,…) Command Parameters Parameter
Description Default TSList Indicates the list of time series to be processed, one of: • AllMatchingTSID – all time series that match the TSID (single TSID or TSID with wildcards) will
be modified. • AllTS – all time series before the command. • EnsembleID – all time series in the ensemble will be modified. • LastMatchingTSID – the last time series that matches the
TSID (single TSID or TSID with wildcards) will be modified. • SelectedTS – the time series are those selected with the SelectTimeSeries() command. AllTS TSID The time series identifier
or alias for the time series to be modified, using the * wildcard character to match multiple time series. Required if TSList=*TSID. EnsembleID The ensemble to be modified, if processing
an ensemble. Required if TSList= EnsembleID. AverageMethod The method used to create the running average, one of: • Centered – values on each side of a date/time are averaged. • Future
– average the next N (bracket) values but do not include the current value. • FutureInclusive – average the next N (bracket) values and also include the current value. • NYear – values
for the current year and (N – 1) preceding years, for the same date/time, are averaged. • NAllYear – values for the current year and all preceding years, for the same date/time, are
averaged (missing values are allowed) • Previous – average the previous N (bracket) values but do not include the current value. • PreviousInclusive – average the previous N (bracket)
values and also include the current value. None – must be specified. Bracket For centered running average, the bracket is the number of points on each side of the current point (therefore
a value of 1 will average 3 data values). For N-year running average, the bracket is the total number of years to average, including the current year. None – must be specified. 403
RunningAverage() Command TSTool Documentation Command Reference – RunningAverage() -4 A sample command file to convert State of Colorado HydroBase diversion time series to running averages
is as follows: # 0100501 -EMPIRE DITCH ReadTimeSeries(Alias="Center","0100501.DWR.DivTotal.Month~HydroBase") RunningAverage(TSList=AllMatchingTSID,TSID="Center", AverageMethod=Centered,Bracket=3)
ReadTimeSeries(Alias="NYear","0100501.DWR.DivTotal.Month~HydroBase") RunningAverage(TSList=AllMatchingTSID,TSID="NYear", AverageMethod=NYear,Bracket=5) 0100501.DWR.DivTotal.Month~HydroBase
The resulting graph is as follows: RunningAverage_graph Results from RunningAverage() Commands 404
Command Reference – RunningStatisticTimeSeries() -1 Command Reference: RunningStatisticTimeSeries() Create a new time series containing running statistics computed from input Version
10.12.00, 2012-07-18 The RunningStatisticTimeSeries() command uses a sample of values from a time series to compute a running statistic, resulting in new time series. The two main purposes
of the command are: 1. Compute a running statistic around a moving point, in order to smooth the time series, for example to focus on underlying short-term forcings rather than variability
or noise 2. Compute a statistic by using values from the historical period, for example to illustrate how a daily value compares to historical values for the same day The sample is computed
relative to a date/time in the time series and consequently the resulting statistic may vary at each date/time in the time series. The resulting time series will have a time series identifier
(TSID) that is the same as the original, with “-Running-” and the statistic appended to the data type (an alias can be assigned to customize the identifier that is used for processing).
There are several approaches to determining the sample for the running statistic (as specified by the SampleMethod command parameter): • The centered running statistic requires that
the number intervals on each site of a point be specified (e.g., specifying 1 will use 3 values at each point). • The previous/future running statistic requires that the number of intervals
prior to or after the current point be specified. • The N-year running statistic is computed by processing the current year and N -1 values from previous years, for a specific date.
A resulting value is produced only if N non-missing values are available. Currently N-year running statistic values for Feb 29 for daily or finer data will always be missing because
a sufficient number of values will not be found – an option may be added in the future to allow Feb 29 values to be computed based on fewer than N values. • A special case of the N-year
running statistic (NAllYear) is to use all previous years’ and the current value. Statistics may be calculated directly from the sample or may be derived from an additional calculation.
For example, the Mean statistic is computed by computing the mean of the values in the sample, and is assigned as the output time series value for the date/time that defines the sample.
However, the PercentOfMean statistic is computed first by computing the Mean statistic and then dividing the original time series value by the mean, for each date/time in the time series.
Derived statistics could be computed for many statistics but are provided only for cases that have common use. 405
RunningStatisticTimeSeries() Command TSTool Documentation Command Reference – RunningStatisticTimeSeries() -2 The following dialog is used to edit the command and illustrates the centered
running average command syntax. RunningStatisticTimeSeries RunningStatisticTimeSeries() Command Editor for Centered Running Average The command syntax is as follows: RunningStatisticTimeSeries(Parame
ter=Value,…) Command Parameters Parameter Description Default TSList Indicates the list of time series to be processed, one of: 1. AllMatchingTSID – all time series that match the TSID
(single TSID or TSID with wildcards) 2. AllTS – all time series generated before the command 3. EnsembleID – all time series in the ensemble 4. FirstMatchingTSID – the first time series
that matches the TSID (single TSID or TSID with wildcards) 5. LastMatchingTSID – the last time series that matches the TSID (single TSID or TSID with wildcards) 6. SelectedTS – the time
series selected with the SelectTimeSeries() command AllTS TSID The time series identifier or alias for the time series to be processed, using the * wildcard character to match Required
if TSList=*TSID. 406
TSTool Documentation RunningStatisticTimeSeries() Command Command Reference – RunningStatisticTimeSeries() -3 Parameter Description Default multiple time series. EnsembleID The ensemble
to be processed, if processing an ensemble. Required if TSList= EnsembleID. Statistic The statistic to compute for each point in the created time series, one of: • ExceedanceProbability
– the probability that the value will be exceeded, best-suited for the N* sample methods (see discussion below about how statistics are computed) • GeometricMean – geometric mean value
• Lag-1AutoCorrelation – the autocorrelation between values and the those that follow in the next time step, given by: rk = Σi=1N-k(Yi -Ymean)(Yi + k -Ymean) Σi=1N(Yi -Ymean)2 • Max
– maximum value • Mean – arithmetic mean value • Median – median value • Min – minimum value • NonexceedanceProbability – the probability that the value will not be exceeded, 1-ExceedanceProbability,
best-suited for the N* sample methods (see discussion below about how statistics are computed) • PercentOfMax – percent of the Max statistic output • PercentOfMean – percent of the Mean
statistic output • PercentOfMedian – percent of the Median statistic output • PercentOfMin – percent of the Min statistic output • Skew – skew coefficient, as follows: Cs = N Σi=1N(Yi
-Ymean)3 (n – 1)(n – 2)s3 where s = standard deviation • StdDev – standard deviation • Total – sum of values • Variance – variance None – must be specified. SampleMethod The method used
to determine the data sample for each statistic calculation, one of: • Centered – N (bracket) values on each side of a date/time and the center value • Future – average the next N (bracket)
values but do not include the current value None – must be specified. 407
RunningStatisticTimeSeries() Command TSTool Documentation Command Reference – RunningStatisticTimeSeries() -4 Parameter Description Default • FutureInclusive – average the next N (bracket)
values and also include the current value • NYear – values for the current year and (N – 1) preceding years, for the same date/time in each year • NAllYear – values for the current year
and all preceding years, for the same date/time in each year (missing values are allowed) • Previous – the previous N (bracket) values but do not include the current value • PreviousInclusive
– the previous N (bracket) values and also include the current value If a sample method such as NAllYear is desired, but including previous, current, and future values, then the NewStatisticTimeSerie
s() command can be used. Bracket For centered SampleMethod, the bracket is the number of points on each side of the current point (therefore a value of 1 will average 3 data values).
For future and previous SampleMethod, the bracket is the number of previous or future values. For N-year SampleMethod, the bracket is the total number of years to process, including
the current year. None – must be specified. AllowMissing Count The number of values allowed to be missing in the sample and still compute the statistic. Care should be taken to specify
a value that is relatively small for the sample size. 0 – no missing values are allowed in the sample Alias The alias to assign to the time series, as a literal string or using the special
formatting characters listed by the command editor. The alias is a short identifier used by other commands to locate time series for processing, as an alternative to the time series
identifier (TSID). None – must be specified. ProbabilityUnits Units to use for calculated probability statistics. Fraction (0 – 1). The following table provides additional information
about how some statistics are computed. Statistic Computation Details Statistic Computation Details Exceedance Probability 1. Rank the values in the sample from highest to lowest. Duplicate
values are retained in the sample 2. Search the list of ranked values, starting from the largest: a. If the value exactly matches a value in the sample: i. The matched value has a position
i (where the largest value is in position i=1). ii. The exceedance probability is calculated as i/(n + 1), where n is the sample size. b. If the value is outside any values in the sample
(e.g., for Future and 408
TSTool Documentation RunningStatisticTimeSeries() Command Command Reference – RunningStatisticTimeSeries() -5 Statistic Computation Details Previous sample methods), then the exceedance
value is not calculated and warnings are generated. In this case a different sample method should be used. c. If the value does not exactly match a value in the sample (e.g., for Future
and Previous sample methods): i. Find the ranked values that bound the value. ii. The exceedance probability for each bounding value is calculated as i/(n + 1), where i is the list position
(1 for the largest value) and n is the sample size. iii. The exceedance probability for the specific value is interpolated from the bounding values. Note that the exceedance probability
is not recomputed by adding the value to the sample. If this is desired, use the FutureInclusive or PreviousInclusive sample methods. Duplicate values are handled by using the first
value found in the sequence of duplicates. A sample command file to convert State of Colorado HydroBase diversion time series to running averages is as follows: # SetInputPeriod(InputStart="1993-01",
InputEnd="2000-12") # 0100501 -EMPIRE DITCH 0100501.DWR.DivTotal.Month~HydroBase RunningStatisticTimeSeries(TSList=AllMatchingTSID, TSID="0100501.DWR.DivTotal.Month",Statistic=Mean,SampleMethod=Cente
red, Bracket=3,Alias="Centered") RunningStatisticTimeSeries(TSList=AllMatchingTSID, TSID="0100501.DWR.DivTotal.Month",Statistic=Mean,SampleMethod=NYear, Bracket=5,Alias="NYear") ProcessTSProduct(TSPr
oductFile="Test_RunningStatisticTimeSeries_Example.tsp") 409
RunningStatisticTimeSeries() Command TSTool Documentation Command Reference – RunningStatisticTimeSeries() -6 The resulting graph is as follows: RunningStatisticTimeSeries_graph Results
from RunningStatisticTimeSeries() Commands 410
Command Reference: RunProgram() Run an external program Version 09.03.00, 2009-04-08 The RunProgram() command runs an external program, given the full command line or individual command
line parts, and waits until the program is finished before processing additional commands. The TSTool command will indicate a failure if the exit status from the program being run is
non-zero. It is therefore possible to call an external program that reads and/or writes recognized time series formats to perform processing that TSTool cannot. One use of this command
is to create a calibration environment where a model is run and then the results are read and displayed using TSTool. It is also useful to use TSTool’s testing features to implement