TSTool-Vol2-CommandReference - Laserfiche WebLink

6Hour INST time series from a Day MEAN time series: NewPatternTimeSeries(Alias=”DayMEAN”,NewTSID="ts1..SQME.Day",Description="Test data", SetStart="2006-12-01",SetEnd="2007-01-31", Units="CMSD",Patte rnValues="20,30,55,40,30,40,50,45,45,80,80,80,80") ChangeInterval(Alias=”6HourINST”,TSID="DayMEAN",NewInterval=6Hour,OldTimeScale=MEAN, NewTimeScale=INST,NewDataType=CMS) The following commands create a Day MEAN time series from a 6Hour INST time series: NewPatternTimeSeries(Alias=”6HourInst”,NewTSID="ts2..Flow.6Hour",IrregularInterval=6Hour, Description="Test data",SetStart="2006- 12-15 12",SetEnd="2007-01-29 00", Units="CFS",PatternValues="20,23,56,62,35,42") ChangeInterval(Alias=”DayMean2”,TSID="6HourInst",NewInterval=Day,OldTimeScale=INST, NewTimeScale=MEAN,HandleEndpointsH ow=IncludeFirstOnly) The following commands create a 3Hour INST time series from an Irregular (1Hour) INST time series: NewPatternTimeSeries(Alias=”IrregularINST”,NewTSID="ts1..Temp.Irregular",Irregu larInterval=1Hour, Description="Test data",SetStart="2006-12-15 00",SetEnd="2007-01-31 23",Units="DEGF", PatternValues="20,23,-999,45,-999,-999,56,62,0,-3") ChangeInterval(Alias=”3HourINST”,TSID="Irr egularINST",NewInterval=3Hour,OldTimeScale=INST, NewTimeScale=INST) 95 Command Reference – ChangeInterval() -15 ChangeInterval() Command TSTool Documentation This page is intentionally blank. Command Reference – ChangeInterval() -16 96 Command Reference: ChangePeriod() Change period of record for time series Version 09.10.01, 2010-11-18 The ChangePeriod() command changes the period for the given time series, for example to extend the time series. A longer period will be filled with missing values. The following dialog is used to edit the command and illustrates the syntax of the command. ChangePeriod ChangePeriod() Command Editor 97 Command Reference – ChangePeriod() -1 ChangePeriod() Command TSTool Documentation The command syntax is as follows: ChangePeriod(Parameter=Value,…) Command Parameters Parameter Description Default TSList Indicates the list of time series to be processed, one of: • AllMatchingTSID – all time series that match the TSID (single TSID or TSID with wildcards) will be modified. • AllTS – all time series before the command. • EnsembleID – all time series in the ensemble will be modified. • FirstMatchingTSID – the first time series that matches the TSID (single TSID or TSID with wildcards) will be modified. • LastMatchingTSID – the last time series that matches the TSID (single TSID or TSID with wildcards) will be modified. • SelectedTS – the time series are those selected with the SelectTimeSeries() command. AllTS TSID The time series identifier or alias for the time series to be modified, using the * wildcard character to match multiple time series. Required if TSList=*TSID. EnsembleID The ensemble to be modified, if processing an ensemble. Required if TSList=EnsembleID. NewStart The new period start, specified to precision that matches the time series data interval. Start will remain the same. NewEnd The new period end, specified to precision that matches the time series data interval. End will remain the same. A sample command file to change the period of a time series from the State of Colorado’s HydroBase is as follows: # 08236000 -ALAMOSA RIVER ABOVE TERRACE RESERVOIR 08236000.DWR.Streamflow.Month~Hydro Base ChangePeriod(TSList=AllTS,NewStart="1900-01") Command Reference – ChangePeriod() -2 98 Command Reference – CheckTimeSeries () -1 Command Reference: CheckTimeSeries() Check time series data values against criteria and optionally take action Version 10.03.00, 2011-12-19 The CheckTimeSeries() command checks time series data values against criteria, for example to identify missing, erroneous, or extreme data values. A warning is generated for each match and time series values optionally can be flagged, which allows annotation on graphs and reports. Values that meet the check criteria also can be removed (if irregular interval), or set to missing. The WriteCheckFile() command can be used to write a summary of the warnings. The CheckTimeSeriesStatistic() command checks a statistic for the entire time series (e.g., missing value count). See also the Delta() command, which creates new time series as the change between each value – this command may be necessary in cases where data periodically reset to a starting value, prior to using a performing a Change> check, for example. The following dialog is used to edit the command and illustrates the command syntax. CheckTimeSeries CheckTimeSeries() Command Editor 99 CheckTimeSeries() Command TSTool Documentation Command Reference – CheckTimeSeries () -2 The command syntax is as follows: CheckTimeSeries(Parameter=Value,…) Command Parameters Parameter Description Default TSList Indicates the list of time series to be processed, one of: • AllMatchingTSID – all time series that match the TSID (single TSID or TSID with wildcards) will be processed. • AllTS – all time series before the command will be processed. • EnsembleID – all time series in the ensemble will be processed. • FirstMatchingTSID – the first time series that matches the TSID (single TSID or TSID with wildcards) will be processed. • LastMatchingTSID – the last time series that matches the TSID (single TSID or TSID with wildcards) will be processed. • SelectedTS – the time series selected with the SelectTimeSeries() command will be processed. AllTS TSID The time series identifier or alias for the time series to be processed, using the * wildcard character to match multiple time series. Required if TSList=*TSID. EnsembleID The ensemble to be modified, if processing an ensemble. Required if TSList= EnsembleID. CheckCriteria The criteria that is checked, one of the following. Missing values are skipped except for cases where the statistic is specific to missing values. • AbsChange> – check for absolute change from one value to the next value > Value1 • AbsChangePercent> – check for absolute change in percent from one value to the next value > Value1. • Change> – check for change > Value1. • Change< – check for change < Value1. • InRange – check for value >= Value1 and <= Value2. • OutOfRange – check for value < Value1 or > Value2. • Missing – check for missing values. • Repeat – check for Value1 repeating values (i.e., if Value1=2, then the check will detect 2 adjacent values that are the same). If the flag or action are None – must be specified. 100 TSTool Documentation CheckTimeSeries() Command Command Reference – CheckTimeSeries() -3 Parameter Description Default specified, values Value1+ in the sequence are modified (i.e., if Value1=2, the 2nd and subsequent repeating values will be modified by the action). • < – check for values < Value1. • <= – check for values <= Value1. • > – check for values > Value1. • >= – check for values >= Value1. • == – check for values equal to Value1. Value1 A parameter that is used for specific CheckCriteria values. Value2 A parameter that is used for specific CheckCriteria values. AnalysisStart The date/time to start analyzing data. Analyze full period. AnalysisEnd The date/time to end analyzing data. Analyze full period. ProblemType The problem type that will be shown in warning messages. CheckCriteria MaxWarnings The maximum number of warnings to list for each time series, useful if analysis results in many warnings. List all warnings. Flag A string to use for a flag on values that are detected during the check, which will be shown in the HTML summary report. No flag. FlagDesc Description for the flag. No description. Action Action to take for matched values, in addition to generating warnings: • Remove – remove the values. For irregular interval time series the values will be removed. For regular interval time series the values will be set to missing. • SetMissing – set the values to missing. No action is taken. 101 CheckTimeSeries() Command TSTool Documentation Command Reference – CheckTimeSeries () -4 This page is intentionally blank. 102 Command Reference: CheckTimeSeriesStatistic() Check time series statistic against criteria Version 10.00.01, 2011-04-26 The CheckTimeSeriesStatistic() command checks a time series statistic against criteria, for example to perform quality control using full-period statistics. This command is essentially a combination of the CalculateTimeSeriesStatistic() command with features similar to the CheckTimeSeries() command; however, the latter checks individual data values and this command checks a statistic computed from the entire time series. The WriteCheckFile() command can be used to write a summary of the warnings. The following dialog is used to edit the command and illustrates the command syntax. 10 3 Command Reference – CheckTimeSeriesStatistic () -1 CheckTimeSeriesStatistic() Command TSTool Documentation CheckTimeSeriesStatistic CheckTimeSeriesStatistic() Command Editor The command syntax is as follows: CheckTimeSeriesStatistic(Parameter=Value,… ) Command Reference – CheckTimeSeriesStatistic () -2 104 TSTool Documentation CheckTimeSeriesStatistic() Command Command Parameters Parameter Description Default TSList Indicates the list of time series to be processed, one of: • AllMatchingTSID – all time series that match the TSID (single TSID or TSID with wildcards) will be processed. • AllTS – all time series before the command will be processed. • EnsembleID – all time series in the ensemble will be processed. • FirstMatchingTSID – the first time series that matches the TSID (single TSID or TSID with wildcards) will be processed. • LastMatchingTSID – the last time series that matches the TSID (single TSID or TSID with wildcards) will be processed. • SelectedTS – the time series selected with the SelectTimeSeries() command will be processed. AllTS TSID The time series identifier or alias for the time series to be processed, using the * wildcard character to match multiple time series. Required if TSList=*TSID. EnsembleID The ensemble to be modified, if processing an ensemble. Required if TSList= EnsembleID. Statistic Statistic to compute. Refer to the CalculateTimeSeriesStatistic() command documentation. None – must be specified. StatisticValue1 Input data required by the statistic. Currently the dialog does not check the value for correctness – it is checked when the statistic is computed. See the Calculate TimeSeries Statistic() command documentation. StatisticValue2 Input data required by the statistic. Currently the dialog does not check the value for correctness – it is checked when the statistic is computed. to the Calculate TimeSeries Statistic() command documentation. StatisticValue3 Input data required by the statistic. Currently the dialog does not check the value for correctness – it is checked when the statistic is computed. to the Calculate TimeSeries Statistic() command documentation. AnalysisStart The date/time to start analyzing data. Full period is analyzed. AnalysisEnd The date/time to end analyzing data. Full period is analyzed. TableID Identifier for table that receives the statistic. Optional – table output is not 10 5 Command Reference – CheckTimeSeriesStatistic() -3 CheckTimeSeriesStatistic() Command TSTool Documentation Parameter Description Default required. TableTSIDColumn Table column name that is used to look up the time series. If a matching TSID is not found, a row will be added to the table. If a TSID is found, the statistic cell value for the time series is modified. Optional – table output is not required. TableTSIDFormat The specification to format the time series identifier to insert into the TSID column. Use the format choices and other characters to define a unique identifier. Time series alias if available, or the time series identifier. TableStatistic Column Table column name to receive the statistic value. If not found in the table, a new column is added automatically. Optional – table output is not required. CheckCriteria The criteria that is checked, one of: • InRange – check for value >= Value1 and <= Value2. • OutOfRange – check for value < Value1 or > Value2. • < – check for values < CheckValue1. • <= – check for values <= CheckValue1. • > – check for values > CheckValue1. • >= – check for values >= CheckValue1. • == – check for values equal to CheckValue1. None – must be specified. CheckValue1 A parameter that is used for specific CheckCriteria values. CheckValue2 A parameter that is used for specific CheckCriteria values, currently only needed for InRange and OutOfRange criteria. ProblemType The problem type that will be shown in warning messages. Statistic-CheckCriteria IfCriteriaMet Indicate whether to set the command status if the statistic meets the criteria, one of: • Ignore – do not set the command status • Warn – set the command status to Warning • Fail – set the command status to Failure The command status will not be changed. PropertyName If the statistic meets the criteria, set the property identified by PropertyName to PropertyValue. No property is set. PropertyValue If the statistic meets the criteria, set the property identified by PropertyName to PropertyValue. No property is set. Command Reference – CheckTimeSeriesStatistic () -4 106 Command Reference – CompareFiles() -1 Command Reference: CompareFiles() Compare text files to determine whether they are different Version 10.01.00, 2011-11-15 The CompareFiles() command compares text files to determine differences. For example, the command can be used to compare old and new files produced by a software process. This command is suitable for comparing files that are similar, but is not suitable for comparing files that are very different, although it may be enhanced in the future to provide more sophisticated comparison features. Each line in the file is compared. By default, lines beginning with # are treated as comment lines and are ignored (see CommentLineChar to specify the comment indicator). Therefore, only non-comment lines are compared. Comment lines in the middle of the file are simply discarded. Differences and simple statistics are printed to the log file. A warning can be generated if a difference is detected or if no differences are detected (see also the CompareTimeSeries()and CompareTables() commands). The following dialog is used to edit the command and illustrates the syntax for the command. CompareFiles CompareFiles() Command Editor 107 CompareFiles() Command TSTool Documentation Command Reference – CompareFiles() -2 The command syntax is as follows: CompareFiles(Parameter=Value,…) Command Parameters Parameter Description Default InputFile1 The name of the first file to read. Enclose the name in double quotes to protect whitespace and special characters. Global properties can be used with the ${Property} syntax. None – the file name is required. InputFile2 The name of the second file to read. Enclose the name in double quotes to protect whitespace and special characters. Global properties can be used with the ${Property} syntax. None – the file name is required. CommentLineChar The character(s) that if found at the start of a line indicate comment lines. Comment lines are ignored in the comparison because they typically may include information such as date/time that changes even if the remainder of the file contents is the same. # IgnoreWhitespace If True, then each line is trimmed to remove leading and trailing whitespace characters (spaces and tabs) before doing the comparison. If False, then whitespace is retained for the comparison. False AllowedDiff The number of lines allowed to be different, when checking for differences. This is useful, for example, when a non-comment line contains the date/time when the file was generated. 0 IfDifferent Indicates the action to be taken if the files are different: • Ignore – do not generate warning • Warn – generate a warning message • Fail – generate a failure message Do not generate a warning if the files are different. Differences are printed to the log file. IfSame Indicates the action to be taken if the files are the same: • Ignore – do not generate warning • Warn – generate a warning message • Fail – generate a failure message Do not generate a warning if the files are the same. The following example illustrates how two files can be compared. For example, use similar commands to compare results from two model runs, two database queries, or when testing software: CompareFiles(InputFile1="Data/A1.txt",InputFile2=" Data/B1.txt", WarnIfDifferent=True) 108 Command Reference – CompareTables () -1 Command Reference: CompareTables() Compare tables Version 10.03.00, 2012-01-07 The CompareTables() command compares columns from two tables, saving the results in a new table. Comparisons are made using the data values formatted as strings based on the precision shown in tables. If the table was read with ReadFromDelimitedFile(), the precision for floating point numbers is set based on the largest number of digits after the decimal encountered in the input. Optionally, a precision and tolerance can be specified to control the comparison of floating point values. Values that are the same are shown in the new table without modification. Values that are different result in both table values being shown (as strings) to allow comparison. The command also allows the comparison table to be output to an HTML file, in which case different values are shown as red. The following dialog is used to edit the command and illustrates the syntax of the command. CompareTables CompareTables() Command Editor The command syntax is as follows: CompareTables(Parameter=Value,…) 109 CompareTables() Command TSTool Documentation Command Reference – CompareTables() -2 Command Parameters Parameter Description Default TableID1 The identifier for the first table to be compared. None – must be specified. TableID2 The identifier for the second table to be compared. None – must be specified. CompareColumns1 The names of columns to be compared from the first table, separated by columns. All columns will be compared. CompareColumns2 The names of columns to be compared from the second table, separated by columns. All columns will be compared. Precision The number of digits after the decimal to consider when comparing floating point values. If values are different to the specified (or default) precision, both values are shown in the comparison table. Format floating point numbers as strings for comparison according to the table column precision. Tolerance A value indicating the allowed difference between floating point values. The tolerance should be consistent with the precision (i.e., don’t specify a coarse precision and fine tolerance). If the difference is less than the tolerance, the values will not be marked as different. Floating point values must exactly match, according to the precision. AllowedDiff The allowed number of differences before triggering a Warn/Fail message (see IfDifferent). A value >= 0 indicates that the number of differences must be the same as the specified value. A negative value indicates that the number of differences can be less than or equal to the specified value. This parameter is useful for constructing tests where a specified number of differences is expected. 0 NewTableID The identifier for the new comparison table. Table1IDTable2IDcomparison OutputFile If specified, an HTML table will be created for the comparison table, in which different values are highlighted in red. No HTML output file will be created. IfDifferent Indicates the action to be taken if the tables are different: • Ignore – do not generate warning • Warn – generate a warning message • Fail – generate a failure message Do not generate a warning if the tables are different. IfSame Indicates the action to be taken if the tables are the same: • Ignore – do not generate warning • Warn – generate a warning message • Fail – generate a failure message Do not generate a warning if the tables are the same. 110 Command Reference: CompareTimeSeries() Compare time series to find data value differences Version 08.15.00, 2008-05-04 The CompareTimeSeries() command compares time series to determine data differences. Currently time series header information is NOT compared – only data values are compared. It is designed to process many time series in bulk fashion. For example, read commands can be used to read time series from two different versions of a database, or from two files. Time series to compare are determined by trying to match each available time series with another time series in the list (ignoring itself); consequently, the list of time series should contain only pairs of time series. Time series that are matched by TSID location and/or data type are compared value by value, with the differences computed as the value from the second time series minus the value from the first time series. The values can be rounded based on a specified precision. It may be important to read each set of time series from files to ensure that final round off is consistent. The checks occur by comparing the difference to one or more specified tolerances. Differences and simple statistics are printed to the log file. Values that are different can optionally be tagged with a character flag, for use with the graphing package. Time series of the differences can optionally be created. A warning can be generated if a difference is detected, or if no differences are detected (see also the CompareFiles() and CompareTables() commands). The following dialog is used to edit the command and illustrates the syntax for the command. CompareTimeSeries CompareTimeSeries() Command Editor 111 Command Reference – CompareTimeSeries() -1 CompareTimeSeries() Command TSTool Documentation The command syntax is as follows: CompareTimeSeries(Parameter=Value,…) Command Parameters Parameter Description Default MatchLocation Match the location part of time series identifiers when matching time series to compare. True MatchDataType Match the data type part of time series identifiers when matching time series to compare. False Precision When comparing data values, round the values to the given precision. For example, a precision of 2 will round to the hundredths place. This can be used to do comparisons on the lowest precision of the available time series. Compare the available values without rounding. Tolerance Specify a comma-separated list of values. The difference in the time series values will be compared to the tolerances and messages printed to the log file. A tolerance of zero will be used to detect differences. AnalysisStart The starting date/time to analyze for differences. Specify a date/time of appropriate precision for the time series or OutputStart to use the output start. Analyze all available data. AnalysisEnd The ending date/time to analyze for differences. Specify a date/time of appropriate precision for the time series or OutputEnd to use the output end. Analyze all available data. DiffFlag Specify as a single character to append a flag to the data flags for the time series. Each value that is different is flagged in both time series that are compared. The flag can be displayed by the graphing package. This is useful for verification processes. New time series will be created with the original identifier preceded by Diff_. Do not flag data. CreateDiffTS Indicate whether a time series should be created containing the differences between time series. This is useful to visually evaluate the differences and process the results with other commands. False WarnIfDifferent If True and at least one difference is detected, a warning will be generated by the command, which will result in software like TSTool displaying a warning. If False, only status messages are written to the log file. The warning is useful if it is critical to detect any change in the time series. Do not generate a warning if time series are different. Differences are printed to the log file. WarnIfSame If True and no differences are detected, a warning will be generated by the command, which will result in software like TSTool displaying a warning. If False, only status messages are written to the log file. The warning is useful if it is critical to detect that time series are the same. Do not generate a warning if time series are the same. Command Reference – CompareTimeSeries() -2 112 TSTool Documentation CompareTimeSeries() Command The following example illustrates how time series from two files can be compared. For example, use similar commands to compare results from two model runs or two database queries: # Example to compare files. Since they are different, a warning will be generated. ReadDateValue(InputFile="RawData1.dv") ReadDateValue(InputFile="RawData 1Scaled.dv") CompareTimeSeries(Precision=2,WarnIfDifferent=True) The following example compares matching time series for the full available period, doing checks for several tolerances: CompareTimeSeries(Precision=2,Tolerance="0,.1,.5,1",DiffFlag="x") The following example compares data only within the output period, as specified by the SetOutputPeriod() command: CompareTimeSeries(P recision=2,Tolerance="0,.1,.5,1", AnalysisStart="OutputStart",AnalysisEnd="OutputEnd",DiffFlag="x") 113 Command Reference – CompareTimeSeries() -3 CompareTimeSeries() Command TSTool Documentation This page is intentionally blank. Command Reference – CompareTimeSeries() -4 114 Command Reference: ComputeErrorTimeSeries() Compute the error between time series and create new time series for the results Version 10.00.01, 2011-05-12 The ComputeErrorTimeSeries() command computes the error between two time series as absolute value or percent, creating a new time series for each pair of time series that is compared. This is useful for comparing observed and simulated time series. The time series that are created have the simulated time series’ metadata but an alias can be assigned. The command can be used to process multiple pairs of time series, each determined using the appropriate *TSList parameter. The following dialog is used to edit the command and illustrates the command syntax. ComputeErrorTimeSeries ComputeErrorTimeSeries() Command Editor The command syntax is as follows: ComputeErrorTimeSeries(Parameter=Value,…) Command Parameters Parameter Description Default Observed TSList Indicates the list of observed time series to be processed, one of: • AllMatchingTSID – all time series that match the TSID (single TSID or TSID with wildcards). • AllTS – all time series before the command. • EnsembleID – all time series in the ensemble. • FirstMatchingTSID – the first time series that matches the TSID (single TSID or TSID with wildcards). AllTS 11 5 Command Reference – ComputeErrorTimeSeries() -1 ComputeErrorTimeSeries() Command TSTool Documentation Parameter Description Default • LastMatchingTSID – the last time series that matches the TSID (single TSID or TSID with wildcards). • SelectedTS – the time series are those selected with the SelectTimeSeries() command. Observed TSID The time series identifier or alias for the observed time series, using the * wildcard character to match multiple time series. Use when ObservedTSList= *MatchingTSID. Observed EnsembleID The observed ensemble to be compared, if processing an ensemble. Use when ObservedTSList= EnsembleID. Simulated TSList Indicates how to determine the list of simulated time series (see the explanation of ObservedTSList). AllTS Simulated TSID The time series identifier or alias for the simulated time series (see the explanation of ObservedTSID). Use when SimulatedTSList= *MatchingTSID. Simulated EnsembleID The ensemble identifier for the simulated time series (see the explanation of SimulatedEnsembleID). Use when SimulateddTSList= EnsembleID ErrorMeasure The error measure to compute, one of: • PercentError – Simulated minus observed, divided by observed. • AbsoluteError – not yet implemented. Alias The alias to assign to the time series, as a literal string or using the special formatting characters listed by the command editor. The alias is a short identifier used by other commands to locate time series for processing, as an alternative to the time series identifier (TSID). Alias will not be assigned. A sample command file is as follows (in this case using contrived data): RemoveFile(InputFile="Results\Test_ComputeErrorTimeSeries_1_out.dv",WarnIfMissing=False) NewPatternTimeSeries(Alias= ”ts1”,NewTSID="ts1..test.Day",Description="Test data", SetStart="1950-01-01",SetEnd="1951-03-12",Units="CFS",PatternValues="5,10,12,13,75") NewPatternTimeSeries(Alias=”ts2”,NewTSID="ts2..test.Day",De scription="Test data", SetStart="1950-01-01",SetEnd="1951-03-12",Units="CFS",PatternValues="6,12,14,11.5,80") ComputeErrorTimeSeries(ObservedTSList=AllMatchingTSID,ObservedTSID="ts1",