TSTool-Vol2-CommandReference - Laserfiche WebLink

SimulatedTSList=AllMatchingTSID,SimulatedTSID="ts2",ErrorMeasure=PercentError) # Uncomment the following command to regenerate the expected results file. # WriteDateValue(OutputFile="ExpectedResults\ Test_ComputeErrorTimeSeries_1_out.dv") WriteDateValue(OutputFile="Results\Test_ComputeErrorTimeSeries_1_out.dv") CompareFiles(InputFile1="Results\Test_ComputeErrorTimeSeries_1_out.dv", InputFile2="ExpectedResults\Test_ComputeErrorTimeSeries_1_out.dv",WarnIfDifferent=True) Command Reference – ComputeErrorTimeSeries() -2 116 Command Reference: ConvertDataUnits() Convert time series data units Version 09.10.01, 2010-11-18 The ConvertDataUnits() command converts the data units for a time series (e.g., before output to a file). Some read and write commands also may allow units to be converted. The following dialog is used to edit the command and illustrates the syntax of the command. ConvertDataUnits ConvertDataUnits() Command Editor The Dimension choice should be selected to narrow the list of available units to the appropriate dimension. Next, select the New Data Units for the time series. The list of available data units is taken from the information described in the TSTool DATAUNIT file (see the TSTool Installation and Configuration Appendix for more information). If desired units are not available, contact the TSTool developers to suggest adding units to the DATAUNIT file or edit the command manually after initial creation. See also the TSTool View… Data Units menu to view the current data units. The dialog cannot display the current units for the time series because the units are not available until time series are actually processed – commands are edited before processing. 117 Command Reference – ConvertDataUnits() -1 ConvertDataUnits() Command TSTool Documentation The command syntax is as follows: ConvertDataUnits(Parameter=Value,…) Command Parameters Parameter Description Default TSList Indicates the list of time series to be processed, one of: • AllMatchingTSID – all time series that match the TSID (single TSID or TSID with wildcards) will be modified. • AllTS – all time series before the command. • EnsembleID – all time series in the ensemble will be modified. • LastMatchingTSID – the last time series that matches the TSID (single TSID or TSID with wildcards) will be modified. • SelectedTS – the time series are those selected with the SelectTimeSeries() command. AllTS TSID The time series identifier or alias for the time series to be modified, using the * wildcard character to match multiple time series. TSID or EnsembleID must be specified if identifiers are being matched. EnsembleID The ensemble to be modified, if processing an ensemble. TSID or EnsembleID must be specified if identifiers are being matched. NewUnits The new data units. None – must be specified. A sample commands file to convert the units of a time series from the State of Colorado’s HydroBase is as follows: # 08236000 -ALAMOSA RIVER ABOVE TERRACE RESERVOIR 08236000.DWR.Streamflow.Month~HydroBase ConvertDataUnits(TSList=AllMatchingTSI D, TSID="08236000.DWR.Streamflow.Month",NewUnits="CFSD") Command Reference – ConvertDataUnits() -2 118 Command Reference: Copy() Create a new time series as a copy of an existing time series Version 10.00.00, 2011-03-27 The Copy() command creates a copy of an existing time series, assigning an alias to the result. The copy is an exact copy except that the alias is different (the TSID must also specified and should be defined to be unique). The alias can then be used for further time series manipulation. A copy of a time series is useful when data filling or other manipulation will occur and time series that is unique from the original is needed. For example, if adding two time series, a copy of one time series can be made, and the second time series added to the copy – this ensures that there is not confusion with the original time series. Parameters are available to control how much of the original data are copied. The following dialog is used to edit the command and illustrates the syntax for the command. Copy Copy() Command Editor 119 Command Reference – Copy() -1 Copy() Command TSTool Documentation The command syntax is as follows: TS Alias = Copy(Parameter=Value,…) The following older command syntax is updated to the above syntax when a command file is read: TS Alias = Copy(Parameter=Value,…) Command Parameters Parameter Description Default TSID The time series identifier or alias of the time series to copy. The time series will be found by searching backwards from the copy command. None – must be specified. NewTSID A new time series identifier to assign to the copy. This is useful to avoid confusion with the original time series. Use the Edit button to edit the time series identifier parts. The data interval must match that of the original time series. Copy the original time series TSID. If NewTSID is specified but does not have a valid interval, copy the interval from TSID. The default cannot be determined if an alias is used for the input time series. Alias The alias to assign to the time series, as a literal string or using the special formatting characters listed by the command editor. The alias is a short identifier used by other commands to locate time series for processing, as an alternative to the time series identifier (TSID). None – must be specified. CopyDataFlags Indicates whether data flags are copied. Specify as False or True. True CopyHistory Indicates whether the time series manipulation history is copied. Specify as False or True. True A sample command file to read a time series and make a copy is as follows: # 08223000 -RIO GRANDE RIVER AT ALAMOSA 08223000.DWR.Streamflow.Month~HydroBase Copy(Alias=”Filled”,TSID="08223000.DWR.Streamflow.Month", NewTSID="08223000.DWR.Streamflow.Month.Filled" ) Command Reference – Copy() -2 120 Command Reference: CopyEnsemble() Create a new ensemble as a copy of an ensemble Version 08.15.00, 2008-05-04 The CopyEnsemble() command creates a copy of an ensemble, copying all time series in the ensemble and assigning a new identifier to the result. The copy is an exact copy except that the ensemble identifier is different (the TSIDs for each ensemble time series should also specified to be unique). The following dialog is used to edit the command and illustrates the syntax for the command. CopyEnsemble CopyEnsemble() Command Editor 121 Command Reference – CopyEnsemble() -1 CopyEnsemble() Command TSTool Documentation The command syntax is as follows: CopyEnsemble(Parameter=Value,…) Command Parameters Parameter Description Default EnsembleID The ensemble to copy. None – must be specified. NewEnsembleID The ensemble identifier for the new ensemble None – must be specified. NewEnsembleName The name for the new ensemble. Blank. NewTSID A new time series identifier to assign to time series in the new ensemble. This is useful to avoid confusion with the original time series. Use the Edit button to edit the time series identifier parts. The data interval and sequence number will be determined from the original time series. Copy the original time series TSID. A sample commands file to read a time series from the State of Colorado’s HydroBase, create an ensemble from the time series, and make a copy is as follows: # 09019500 -COLORADO RIVER NEAR GRANBY 09019500.USGS.Streamflow.Day~HydroBase CreateEnsemble(TSID="09019500.USGS.Streamflow.Day", TraceLength=1Year,EnsembleID="Ensemble_1",EnsembleName="Test Ensemble",ReferenceDate="2008-01-01",ShiftDataHow=ShiftToReference) CopyEnsemble(NewEn sembleID="Ensemble_2", NewEnsembleName="Test ensemble 2", NewTSID="09019500.USGS.Streamflow..copy",EnsembleID="Ensemble_1") Command Reference –CopyEnsemble() -2 122 Command Reference – CopyTable () -1 Command Reference: CopyTable() Create a table as a (partial) copy of a table Version 10.21.00, 2013-05-17 The CopyTable() command copies all or a subset of the columns from one table to create a new table. For example, this is useful to create one-column lists that can be used to expand template files with the ExpandTemplateFile() command, or to create a subset of columns to output to a file or write to a database. The following dialog is used to edit the command and illustrates the syntax of the command (in this case illustrating how values in a column named LocationID are copied to a new table). CopyTable CopyTable() Command Editor 123 CopyTable() Command TSTool Documentation Command Reference – CopyTable() -2 The command syntax is as follows: CopyTable(Parameter=Value,…) Command Parameters Parameter Description Default TableID The identifier for the original table. None – must be specified. NewTableID The identifier for the new table. None – must be specified. IncludeColumns The names of columns to copy, separated by commas. Copy all of the columns. DistinctColumns The names of columns to copy, separated by commas. Only distinct values from the specified column will be copied. For example, if column A contains strings X, Y, Z, Y, C, the resulting distinct value column will have rows with X, Y, Z, C. Currently only one column name can be specified but in the future more than one column may be allowed. This parameter overrides IncludeColumns. Don’t do a distinct comparison. ColumnMap The new names for the output columns, using syntax: OriginalColumn1:NewC olumn1, OriginalColumn2:NewColumn2 The column names in the copy will be the same as in the original table. ColumnFilters Filters that limit the number of rows being copied, using the syntax: FilterColumn1:FilterPattern1, FilterColumn2:FilterPattern2 Patterns can use * to indicate wildcards for matches. Only string values can be checked (other data types are converted to strings for comparison). Comparisons are case-independent. All patterns must be matched in order to copy the row. In the future a command may be added to perform queries on tables, similar to SQL for databases. No filtering. 124 Command Reference – CreateEnsembleFromOneTimeSeries() -1 Command Reference: CreateEnsembleFromOneTimeSeries() Create a new ensemble from a single time series Version 10.14.00, 2012-12-12 The CreateEnsembleFromOneTimeSeries() command creates an ensemble by splitting up a single time series into traces. For example, a historical time series can be split into 1-year overlapping traces that are shifted to start in the current year. The sequence number part of the time series identifier for each trace is set to the input starting year and will be shown as [Year] at the end of the time series identifier. The following dialog is used to edit the command and illustrates the syntax for the command. CreateEnsembleFromOneTimeSeries CreateEnsembleFromOneTimeSeries( ) Command Editor The command syntax is as follows: CreateEnsembleFromOneTimeSeries(Parameter=Value,…) 125 CreateEnsembleFromOneTimeSeries() Command TSTool Documentation Command Reference – CreateEnsembleFromOneTimeSeries() -2 Command Parameters Parameter Description Default TSID The time series identifier or alias for the time series used to create the ensemble. None – must be specified. InputStart The date/time to start transferring data from the time series. Use all data. InputEnd The date/time to end transferring data from the time series. Use all data. EnsembleID The new ensemble identifier. None – must be specified. Ensemble Name The name for the new ensemble. Blank. Alias The alias to assign to the time series, as a literal string or using the special formatting characters listed by the command editor. The alias is a short identifier used by other commands to locate time series for processing, as an alternative to the time series identifier (TSID). %L_%z (location_ sequence Number) TraceLength An interval for the trace length (e.g., 1Year, #Month or, #Day). 1Year ReferenceDate The reference date indicates the starting date for each trace. Each trace optionally can be shifted (see ShiftDataHow), in which case the year in the ReferenceDate is used for the common starting date. The reference date can be one of: • Blank, indicating that January 1 of the current year will be used. • A date/time string (use the format 01/01/YYYY or YYYYMM-DD). • CurrentToYear, CurrentToMonth, CurrentToDay, CurrentToHour, CurrentToMinute, indicating the current date/time to the specified precision. • A Current* value +-an interval, for example: CurrentToMinute – 7Day January 1 of the first year in the source time series. OutputYearType The output year type for the ensemble traces. The only impact from this parameter is that sequence number for the time series will be set to the start of the output year. This is useful because legends on graphs that use the sequence number (%z format specifier) will use the appropriate year type. The ReferenceDate should normally be specified as the first day of the output year (e.g., ReferenceDate=2012-10-01 for OutputYearType=Water). Calendar ShiftDataHow Indicates whether the traces should be shifted. Possible values are: • ShiftToReference – each trace will be shifted to the reference date, resulting in overlapping time series. • NoShift – plotting the traces will result in a total line that matches the original time series, except that each trace can be manipulated individually. NoShift 126 TSTool Documentation CreateEnsembleFromOneTimeSeries() Command Command Reference – CreateEnsembleFromOneTimeSeries() -3 A sample command file to read a time series from the State of Colorado’s HydroBase and create an ensemble from the time series is as follows: # 09019500 -COLORADO RIVER NEAR GRANBY 09019500.USGS.Streamflow.Day~HydroBase CreateEnsembleFromOneTimeSeries(TSID="090 19500.USGS.Streamflow.Day", TraceLength=1Year,EnsembleID="Ensemble_1",EnsembleName="Test Ensemble",ReferenceDate="2008-01-01",ShiftDataHow=ShiftToReference) The following figure illustrates a graph of the resulting ensemble: CreateEnsembleFromOneTimeSeries_Graph CreateEnsembleFromOneTimeSeries() Example Graph 127 CreateEnsembleFromOneTimeSeries() Command TSTool Documentation Command Reference – CreateEnsembleFromOneTimeSeries() -4 This page is intentionally blank. 128 Command Reference – CreateFromlist() -1 Command Reference: CreateFromList() Create one or more time series from a file containing a list of identifiers Version 10.21.00, 2013-05-17 See also the ReadTimeSeriesFromTable() command, which may replace this command in the future. The CreateFromList() command creates one or more time series using identifiers from a list file, an example of which is shown below: # Example list file. Comments start with the # character. # Column headings can be specified in the first non-comment row using quotes. “Structure ID”,”Structure Name” 500501,Ditch 501 500502,Ditch 502 # Invalid ID (see IfNotFound parameter) 509999,Ditch 9999 The command is typically used when reading time series from a database or binary file and can streamline processing in the following situations: • A list of identifiers may have been generated from a database query and saved to a file. • A list of identifiers may have been extracted from a model data set. TSTool reads the list file and internally creates a list of time series identifiers. The time series are of the standard form: Location.DataSource.DataT ype.Interval[.Scenario]~InputType[~InputName] where the brackets indicate optional information. TSTool then queries each time series, which can be processed further. Although it is possible to specify an input type that reads from files by also using the InputName, this is not generally recommended because the CreateFromList() command can only specify one input file name and the file will be reopened for each read. Instead, read commands for specific file formats should be used because these commands are typically optimized to read multiple time series from the files. In summary, the CreateFromList() command is useful with databases but performance may suffer when used with file input types. 129 CreateFromList() Command TSTool Documentation Command Reference – CreateFromList() -2 The following dialog is used to edit the command and illustrates the syntax of the command. CreateFromList CreateFromList() Command Editor The command syntax is as follows: CreateFromList(Parameter=Value, …) Command Parameters Parameter Description Default ListFile The name of the list file to read, surrounded by double quotes. None – must be specified. IDCol The column (1+) in the list file containing the location identifiers to use in time series identifiers. 1 Delim The delimiter characters that separate columns in the list file. If a space is used as the delimiter, surround with another delimiter characters or a character that is unlikely to be found so that the space is not discarded as white space (e.g., “~ ~”). Comma ID Indicate a pattern to filter the identifiers in the list file. For example, use A* to only Process all identifiers. 130 TSTool Documentation CreateFromList() Command Command Reference – CreateFromList() -3 Parameter Description Default process identifiers in the list file that start with A. DataSource The data source in the time series identifier, appropriate for InputType. For example, if using the State of Colorado’s HydroBase, USGS indicates that data are from the United States Geological Survey. See the input type appendices for more information on available data types. May or may not be required, depending on the input type. Refer to the input type appendices. DataType The data type in the time series identifier, as appropriate for InputType. For example, if using the State of Colorado’s HydroBase, DivTotal is used for diversion totals. See the input type appendices for more information on available data types. Usually required for an input type. Refer to the input type appendices. Interval Data interval in the time series identifier, using standard values such as 15Minute, 6Hour, Day, Month, Year. None – must be specified. Scenario Scenario in the time series identifier. Usually not required. InputType The input type in the time series identifier. For example, use HydroBase for the State of Colorado’s HydroBase database. Refer to the input type appendices or the TSTool main GUI for options. None – must be specified. InputName The input name in the time series identifier. Typically only required if the input type requires a file name. IfNotFound Indicates how to handle missing time series, one of: • Warn – generate fatal warnings and do not include in output. • Ignore – generate non-fatal warnings and do not include in output. • Default – generate non-fatal warnings and create empty time series for those that could not be found. This requires that a SetOutputPeriod() command be used before the command to define the period for default time series. Warn DefaultUnits Default units when IfNotFound=Default. Blank – no units. 131 CreateFromList() Command TSTool Documentation Command Reference – CreateFromList() -4 A sample command file to process monthly diversion data from the State of Colorado’s HydroBase database is as follows: # Read monthly diversion total from HydroBase for the structures in the list # file. The data source is set to DWR because data source is saved in # HydroBase. CreateFromList(ListFile= "Data\Diversions.txt",IDCol=1,DataSource=DWR, DataType=DivTotal,Interval=Month,InputType=HydroBase,IfNotFound=Default) 132 Command Reference – CreateRegressionTestCommandFile() -1 Command Reference: CreateRegressionTestCommandFile() Create a command file to run software regression tests Version 10.20.00, 2013-04-20 The CreateRegressionTestCommandFile() command is used for software testing and certification of processes used in operations. The command creates a command file that includes a StartRegressionTestResultsReport() and multiple RunCommands() commands. A starting search folder is provided and all files that match the given pattern (by convention Test_*.TSTool) are assumed to be command files that can be run to test the software. The resulting command file is a test suite comprised of all the individual tests and can be used to verify software before release. The goal is to have all tests pass before software release. The following table lists tags (annotations) that can be placed in # comments in command files to provide information for testing, for example: #@expectedStatus Failure Command # Comment Tags Comment Tag Description @enabled False The RunCommands() command will by default run the command file that is provided. However, if the @enabled False tag is specified in a comment in the command file, RunCommands() will skip the command file. This is useful to disable a test that needs additional work. @expectedStatus Failure @expectedStatus Warning The RunCommands() command ExpectedStatus parameter is by default Success. However, a different status can be specified if it is expected that a command file will result in Warning or Failure and still be a successful test. For example, if a command is obsolete and should generate a failure, the expected status can be specified as Failure and the test will pass. Another example is to test that the software properly treats a missing file as a failure. @os Windows @os UNIX The test is designed to work only on the specified platform and will be included in the test suite only if the IncludeOS parameter includes the corresponding operating system (OS) type. This is primarily used to test specific features of the OS and similar but separate test cases should be implemented for both OS types. If the OS type is not specified as a tag in a command file, the test is always included (see also the handling of included test suites). @readOnly Indicates that the command file should not be edited. TSTool will update old command syntax to current syntax when a command file is loaded. However, this tag will cause the software to warn the user when saving the command file, so that they can cancel. @testSuite ABC Indicate that the command file should be considered part of the specified test suite, as specified with the IncludeTestSuite 133 CreateRegressionTestCommandFile() Command TSTool Documentation Command Reference – CreateRegressionTestCommandFile() -2 Comment Tag Description parameter. The test is included in all test collections if the tag is not specified; therefore, for general tests, do not specify a test suite. This tag is useful if a group of tests require special setup, for example connecting to a database. The suite names should be decided upon by the test developer. The following dialog is used to edit the command and illustrates the syntax for the command. CreateRegressionTestCommandFi le CreateRegressionTestCommandFile() Command Editor The command syntax is as follows: CreateRegressionTestCommandFile(Parameter=Value,…) Command Parameters Parameter Description Default SearchFolder The folder to search for regression test command files. All subfolders will also be searched. None – must be specified. OutputFile The name of the command file to create, enclosed in double quotes if the file contains spaces or other special characters. A path relative to the command file containing this command can be specified. None – must be specified. SetupCommandFile The name of a TSTool command file that supplies setup commands, and which will be prepended to output. Use such a file to open database connections and set other global settings that apply to the entire test run. Do not include setup commands. FilenamePattern Pattern for TSTool command files, using wildcards. Test_*.TStool Append Indicate whether to append to the output file (True) or overwrite (False). This allows multiple directory trees to be searched for tests, where the first command True 134 TSTool Documentation CreateRegressionTestCommandFile() Command Command Reference – CreateRegressionTestCommandFile() -3 Parameter Description Default typically specifies False and additional commands specify True. IncludeTestSuite If *, all tests that match FilenamePattern and IncludeOS are included. If a test suite is specified, only include tests that have @testSuite tag values that match a value in IncludeTestSuite. One or more tags can be specified, separated by commas. * – include all test cases. IncludeOS If *, all tests that match FilenamePattern and IncludeTestSuite are included. If an OS is specified, only include tests that have @os tag values that match a value in IncludeTestSuite. This tag is typically specified once or not at all. * – include all test cases. See the Quality Control chapter of the TSTool documentation for how to set up a regression test. The following command file illustrates how to create a regression test suite. CreateRegressionTestCommandFile(SearchFolder="..\..\..\commands\general", OutputFile="..\run\RunRegressionTest_commands_general.TSTool",Append=False) An example of the output file from running the tests is: # File generated by... # program: TSTool 10.20.00 (2013-04-10) # user: sam # date: Sat Apr 20 13:36:05 MDT 2013 # host: AMAZON # directory: C:\Develop\TSTool_SourceBuild\TSTool\test\regression\TestSuites\commands_general\run # command line: TSTool # -home test/operational/CDSS # # Command file regression test report from StartRegressionTestResultsReport() and RunCommands() # # Explanation of columns: # # Num: count of the tests # Enabled: blank if test enabled or FALSE if "#@enabled false" in command file # Run Time: run time in milliseconds # Test Pass/Fail: # The test status below may be PASS or FAIL (or blank if disabled). # A test will pass if the command file actual status matches the expected status. # Disabled tests are not run and do not count as PASS or FAIL. # Search for *FAIL* to find failed tests. # Commands Expected Status: # Default is assumed to be SUCCESS. # "#@expectedStatus Warning|Failure" comment in command file overrides default. # Commands Actual Status: # The most severe status (Success|Warning|Failure) for each command file. # # | | |Test |Commands |Commands | # | |Run |Pass/|Expected |Actual | # Num|Enabled|Time |Fail |Status |Status |Command File #----+-------+-------+------+----------+-----------+-- -------------------------------------------------------------------------------------1| | 141| PASS |SUCCESS |SUCCESS |C:\Develop\TSTool_SourceBuild\TSTool\test\regression\commands\general\ARMA\Test_ ARMA_Day.TSTool 2| | 31| PASS |SUCCESS |SUCCESS |C:\Develop\TSTool_SourceBuild\TSTool\test\regression\commands\general\ARMA\Test_ARMA_Legacy.TSTool 3| | 31| PASS |SUCCESS |SUCCESS |C:\Develop\TSTool_ SourceBuild\TSTool\test\regression\commands\general\ARMA\Test_ARMA_Legacy_Ast.TSTool 4| | 15| PASS |SUCCESS |SUCCESS |C:\Develop\TSTool_SourceBuild\TSTool\test\regression\commands\general\ARMA\Test_A RMA_Legacy… … … … 17|FALSE | 0| |SUCCESS |UNKNOWN |C:\Develop\TSTool_SourceBuild\TSTool\test\regression\commands\general\WriteReclamationHDB\Test_WriteReclamationHDB_... #----+-------+-------+------+ ----------+-----------+---------------------------------------------------------------------------------------FAIL count = 0, 0.000% PASS count = 17, 100.000% Disabled count = 1 #-------------------- ------------Total = 18 135 CreateRegressionTestCommandFile() Command TSTool Documentation Command Reference – CreateRegressionTestCommandFile() -4 This page is intentionally blank. 136 Command Reference – Cumulate() -1 Command Reference: Cumulate() Convert time series data values to cumulative values Version 10.12.00, 2012-07-25 The Cumulate()command converts a time series into cumulative values, which is useful for: • comparing the cumulative trends of related time series (e.g., nearby gages or precipitation gages) and can serve as a substitute for the double-mass graph, which has difficulty handling missing data • checking mass balance when routing time series (the cumulative values before and after routine will track closely) • computing year-to-date totals such as cumulative precipitation The following dialog is used to edit the command and illustrates the syntax of the command. Cumulate Cumulate() Command Editor The command syntax is as follows: Cumulate(Parameter=Value,…) 137 Cumulate() Command TSTool Documentation Command Reference – Cumulate() -2 Command Parameters Parameter Description Default TSList Indicates the list of time series to be processed, one of: • AllMatchingTSID – all time series that match the TSID (single TSID or TSID with wildcards) will be modified. • AllTS – all time series before the command. • EnsembleID – all time series in the ensemble will be modified. • LastMatchingTSID – the last time series that matches the TSID (single TSID or TSID with wildcards) will be modified. • SelectedTS – the time