TSTool-Vol2-CommandReference - Laserfiche WebLink

– PrintTextFile() -1 PrintTextFile() Command TSTool Documentation The command syntax is as follows: PrintTextFile(Parameter=Value,…) Command Parameters Parameter Description Default InputFile The name of the text file to print. None – must be specified. PrinterName The name of the printer to use (e.g., \\MyComputer\MyPrinter or Adobe PDF) The default printer will be used. PaperSize The paper size to print. Because there are a number of standards for paper, the size is specified as standardsizename (e.g., na-letter for North American Lettter). For information on paper sizes, see: http://en.wikipedia.org/wiki/Paper_size. The default for the printer will be used. PaperSource The tray for the paper – currently not enabled. Use ShowDialog=True to select. The PaperSize is used. Orientation The paper orientation. The default for the printer and paper size will be used. MarginLeft The left margin for the orientation, inches. See above. MarginRight The right margin for the orientation, inches. See above. MarginTop The top margin for the orientation, inches. See above. MarginBottom The bottom margin for the orientation, inches. See above. LinesPerPage The number of lines per page to print. The font size is chosen accordingly. An even number is chosen with font size between 7 and 12 points. Header String included in the top left of every page. No header is shown. Footer String included in the bottom left of every page. No footer is shown. ShowLineCount Indicate whether the line count should be shown to the left of each line. False ShowPageCount Indicate whether the page count should be shown in the center of the footer. True DoubleSided Indicate whether printing should be double-sided, currently not enabled. Use ShowDialog=True to select. False OutputFile Specify an output file, in cases where the printer name corresponds to a file formatter (such as Adobe PDF). Content is sent to printer device, not a file. ShowDialog Indicate whether to show the printer dialog, which allows review and editing of printer parameters. This is useful for testing and for selecting advanced printer features not handled in batch mode by this command. False IfNotFound Indicate action if the file is not found, one of: • Ignore – ignore the missing file (do not warn). • Warn – generate a warning (use this if the file truly is expected and a missing file is a cause for concern). • Fail – generate a failure (use this if the file truly is expected and a missing file is a cause for concern). Ignore Command Reference – PrintTextFile() -2 282 TSTool Documentation PrintTextFile() Command This command can be used to test print features. For example, use a printer that outputs to a PDF, XPS, or other format file rather than a physical printer. If ShowDialog=True and printing to a file is indicated by specifying the OutputFile parameter (such as with Adobe PDF), the General tab on the print dialog will be similar to the following: PrintTextFile_DialogGeneral Print Job Dialog General Properties Note that Print to File is checked; however, the name of the file may not be displayed. Instead, the output file is specified by pressing Print, which displays the following dialog, with the initial choice matching the value of the OutputFile parameter: 283 Command Reference – PrintTextFile() -3 PrintTextFile() Command TSTool Documentation PrintTextFile_PrintToFile Print To File Dialog If running a command to write a PDF file, the following may be displayed: PrintTextFile_DistillerFontError Adobe PDF Error Unfortunately, there does not seem to be any way to change the printer setting from the print dialog and consequently PDF cannot be used. An alternate approach such as iText may be implemented for PDF. If a Microsoft XPS file is printed, the following software may need to be installed, in particular for Windows XP: http://www.microsoft.com/download/en/details.aspx? displaylang=en&id=11816 Command Reference – PrintTextFile() -4 284 Command Reference – ProcessTSProduct() -1 Command Reference: ProcessTSProduct() Process a time series product file to produce output Version 10.16.00, 2013-01-28 The ProcessTSProduct() command automates creation of time series data products. Products are described in time series product description (*.tsp) files, which are typically created by using the Save…Time Series Product choice in graph windows (a future enhancements may allow creation of text products from summary or table views). See the TSView Time Series Viewing Tools appendix for more information about time series products. For example, the following sequence of actions can be used to define and use time series product description files: 1. Use TSTool and interactively select time series using the main window. The time series identifiers and/or aliases will be referenced in the time series product. 2. Interactively view a graph (e.g., Results…Graph – Line) and edit its properties by right clicking on the graph and selecting the Properties choice (e.g., set titles and legend properties). 3. Save the graph as a time series product from the graph window using the Save…Time Series Product choice. Typically the product is saved in a location close to the command file. An example time series product file is as follows: [Product] ProductType = "Graph" [SubProduct 1] GraphType = "Line" MainTitleString = "Streamflow (Monthly Total)" [Data 1.1] TSID = "08223000.DWR.Streamflow.Month~HydroBase" [Data 1.2] TSID = "08220500.DWR.Stre amflow.Month~HydroBase" 4. Add a ProcessTSProduct() command to the original commands to allow the product to be created automatically. Select the time series product file created in the previous step. 5. Save the commands in a file (e.g., named stations.TSTool) so that they can be run again. The command file and time series product definition files must be used consistently (e.g., the time series identifiers and directory paths must be consistent). If the product does not appear as intended, especially for complicated products, it may be necessary to edit the file and make the following corrections: • Specify Color or other properties so that they are explicitly set and not defaulted. • Verify that file paths in TSID properties are valid for the machine (may need to convert absolute paths to relative paths). 285 ProcessTSProduct() Command TSTool Documentation Command Reference – ProcessTSProduct() -2 Time series identifiers in the product file are used as follows: • If the time series are in TSTool’s Results area, the time series will be used without rereading. • Otherwise, the TSID is used to read the time series and must therefore contain enough information to locate and read the time series, such as the ~InputType~InputName information on at the end of the TSID. If the TSAlias property is found in the product file, then the time series corresponding to the alias must be processed by a command file and be available in TSTool’s Results area. It is also possible to create a template time series product file and use the ExpandTemplateFile() command to automate creation of large numbers of graphs, for example to create images for a website. The following dialog is used to edit the ProcessTSProduct() command and illustrates the command syntax. The path to the file can be absolute or relative to the working directory. The Browse button can be used to select the time series product description file (if a relative path is desired, delete the leading path after the select or use the Remove Working Directory from TSP button). ProcessTSProduct ProcessTSProduct() Command Editor 286 TSTool Documentation ProcessTSProduct() Command Command Reference – ProcessTSProduct() -3 The command syntax is as follows: ProcessTSProduct(Parameter=Value,…) Command Parameters Parameter Description Default TSProductFile The time series product file to process. The path to the file can be absolute or relative to the working directory. The Browse button can be used to select the file to write (if a relative path is desired, delete the leading path after the select). None – must be specified. RunMode Indicate the run mode to process the product, one of: • BatchOnly – indicates that the product should only be processed in batch mode. • GUIOnly – indicates that the product should only be processed when the TSTool GUI is used (useful when Preview is set to Preview). • GUIAndBatch – indicates that the product should be processed in batch and GUI mode. None – must be specified. View Indicates whether the output should be previewed interactively, one of: • True – display the graph. • False – do not display the graph (specify the output file instead to automate image creation). None – must be specified. OutputFile The absolute or relative path to an output file. Use this parameter with View=False to automate image processing. If the filename ends in “jpg”, a JPEG image file will be produced. If the filename ends in “png”, a PNG file will be produced (recommended). Graph file will not be created. DefaultSaveFile Used with experimental feature to enabling editing in the time series table that corresponds to a graph view. Specify the default DateValue filename to save edits. Editing is disabled. VisibleStart The starting date/time to zoom for the initial (and image file) graph. Full period is visible. VisibleEnd The ending date/time to zoom for the initial (and image file) graph. Full period is visible. 287 ProcessTSProduct() Command TSTool Documentation Command Reference – ProcessTSProduct() -4 A sample command file to process a data product using State of Colorado HydroBase data is as follows: # 08235350 -ALAMOSA RIVER ABOVE JASPER 08235350.USGS.Streamflow.Day~HydroBase # 08236000 -ALAMOSA RIVER ABOVE TERRACE RESERVOIR 08236000.DWR.Streamflow.Day~HydroBase # 7337 -SAGUACHE 7337.NOAA.Precip.Month~HydroBase ProcessTSProduct(TSProductFile="Example_ProcessTSProduct.tsp") After using the above dialog to edit the command, the time series product can be processed from TSTool as follows: 1. Interactively load and run the command file: a. Open the command file, in this case containing the above commands file. b. Process the commands using Run All Commands. The graph will be displayed for review. 2. Load and run the command file in one step: Use the Run…Process TSProduct File menus to select and process the product file. The time series must be in the Results area or must be specified with enough information in the product file to read the time series. 3. Run TSTool in batch mode by specifying an output file (and optionally changing the RunMode parameter to BatchOnly) using: tstool –commands commands.TSTool The working directory will be set to the directory for the commands file and output will be relative to that directory. 288 Command Reference – ProfileCommands () -1 Command Reference: ProfileCommands() Profile the commands that have executed, to evaluate performance Version 10.17.00, 2013-02-18 The ProfileCommands() command summarizes run times and memory use for each command in the command list, and outputs the information to detail (row for each command) and summary (one row for each command name) tables. This command is useful for evaluating which commands are slow or use more memory in a command workflow, so that software and command file logic improvements can occur. The command is usually placed at the end of a command file. The following apply to command profiling: • Because the command is processed at the time it is encountered in the command list, the command itself and any subsequent commands are not included in the analysis. This generally is not an issue because the command will be used near the end of a workflow or at a strategic location where previous commands need to be examined. • Currently the memory statistics are rough because the heap size is determined at the start and end of each command’s execution and the Java runtime environment may allocate heap memory in blocks. In the future profiling data may be expanded to the estimated memory footprint of each command. • There is a slight performance and memory hit to collect profiling information. In the future processor property commands may be implemented to control how much profiling data are collected (specifically if memory for each command object is estimated). • If a command file is causing out of memory exceptions, then placing a ProfileCommands() command at the end of the command file likely will not be helpful. Instead, use a subset of the full command list so the ProfileCommands() command will be executed. Then evaluate the performance of the commands and determine if the command list logic can be optimized. If performance issues appear to be in the software itself, contact the developers to evaluate the software code. Also consider using the Free() and FreeTable() commands to free resources, especially if the results do not need to be available to users via the user interface. • The runtime percent for each command is calculated as a percentage of the total runtime (ignoring the ProfileCommands() command and subsequent commands). • The heap memory percentage delta for each command is calculated using the heap memory at the end of the command execution (not the heap memory at the end of the full run). Consequently, the delta reflects the memory use up to that point in time. • Command profiling currently only applies to run mode. Commands are executed in discovery mode when a command file is loaded. For example, a subset of time series data is retrieved so that time series identifiers can be created and passed to following commands, which allows choices to be populated in command editors. Profiling discover mode is not support but should use a fraction of full runtime resources. For large command files (e.g., those generated by templates), it may be appropriate or necessary to load the commands without running discovery (see the –nodiscovery command line parameter and the File…Open…Command File (no discovery) menu item. • Commands that generate many warning and failure messages will use more memory. Refer to the NumLogRecords column in the detail table to determine if this could be causing memory issues. • The command currently does not allow sorting output tables by a column. This feature may be added in the future. Use the interactive table view to sort by column (this is how the tables were sorted for the figures below). 289 ProfileCommands() Command TSTool Documentation Command Reference – ProfileCommands() -2 If loading or running commands are slow, the following actions might help: • Use the Free() and FreeTable() commands to free resources. The command will still take up some resources because it has a place in the command list, but data resources used by the command will be freed. • Review the profiling results to determine if certain commands are major resource users. Evaluate whether changes in the command logic can be implemented. Comment out blocks of commands (# commands will take fewer resources than /* */blocks) and try to isolate problems. It may be necessary to run smaller subsets of commands, for example by splitting up lists of input time series. • On Windows, use the Task Manager (run taskmgr) to review memory use by the javaw.exe program. If the memory use approaches the maximum, then the Java Runtime Environment likely will be spending time dealing with short memory and runtimes will increase until memory runs out. If necessary, change the –Xmx parameter in the TSTool.l4j.ini file located in the system folder under the software install. This parameter indicates the maximum heap memory that can be used by the software. For a typical 32-bit Windows computer with at least 4GB of memory, the –Xmx parameter may be set to as high as 1700mb; however, a number that is too high may not be possible due to memory being used by other applications on the computer. The following dialog is used to edit the command and illustrates the syntax of the command. ProfilkeCommands ProfileCommands() Command Editor 290 TSTool Documentation ProfileCommands () Command Command Reference – ProfileCommands () -3 The following figure illustrates the output summary table. Because command execution may be very fast, times are shown in milliseconds (1-1000th of a second). The table can be output to a file with other commands. ProfilkeCommands_Summary ProfileCommands() Command Summary Output Table The following figure illustrates the output detail table. Note that the heap memory is increased in blocks by the Java Runtime Environment so only large memory footprint commands trigger immediate heap memory increases. ProfilkeCommands_Detail ProfileCommands() Command Detail Output Table 291 ProfileCommands() Command TSTool Documentation Command Reference – ProfileCommands() -4 The command syntax is as follows: ProfileCommand(Parameter=Value,…) Command Parameters Parameter Description Default SummaryTableID The identifier for the summary table. Summary table will not be created. DetailTableID The identifier for the detail table. Detail table will not be created. 292 Command Reference: ReadDateValue() Read all time series from a DateValue File Version 10.00.01, 2011-05-15 The ReadDateValue() command reads all the time series in a DateValue file into memory. See the DateValue Input Type Appendix for information about the file format. The following dialog is used to edit the command and illustrates the command syntax. The path to the file can be absolute or relative to the working directory. DateValue files allow each time series to have an alias in addition to the time series identifier (TSID); however, the Alias parameter can be used to assign a new alias as the file is read. ReadDateValue ReadDateValue() Command Editor 293 Command Reference – ReadDateValue() -1 ReadDateValue() Command TSTool Documentation The command syntax is as follows: ReadDateValue(Parameter=Value,…) The following older command syntax is updated to the above syntax when a command file is read: TS Alias = ReadDateValue(Parameter=Value,…) Command Parameters Parameter Description Default InputFile The name of the DateValue input file to read, surrounded by double quotes to protect whitespace and special characters. Global property values can be used with the syntax ${PropertyName} (see also the SetProperty() command). None – must be specified. Alias The alias to assign to the time series, as a literal string or using the special formatting characters listed by the command editor. The alias is a short identifier used by other commands to locate time series for processing, as an alternative to the time series identifier (TSID). The alias in the file will be used if present. NewUnits Units to convert data to (must be in the system/DATAUNIT configuration file under the TSTool installation folder). Use the data units from the file. InputStart Starting date/time to read data, in precision consistent with data. Read all data. InputEnd Ending date/time to read data, in precision consistent with data. Read all data. A sample command file is as follows: ReadDateValue(InputFil e="Data\08251500.DWR.Streamflow.IRREGULAR.dv") Command Reference – ReadDateValue() -2 294 Command Reference – ReadDelimitedFile() -1 Command Reference: ReadDelimitedFile() Read time series from a delimited file Version 10.08.00, 2012-04-23 The ReadDelimitedFile() command reads one or more time series from a column-oriented delimited file, where columns contain date/time and values. This command is useful for processing comma-separated-value (CSV) files exported from spreadsheets and mining data from the web (see also the WebGet() and FTPGet() commands). The command processes files that include the following types of information: 1. Comments in the header (before data) and embedded in data records (e.g., because bad data values were commented out). 2. Column headers as non-commented line at the top of the file. 3. Data records, in column format, containing date/time strings, data values, and other information. 4. Metadata, such as station identifiers, data types, units, and interval may be read from the file or specified with command parameters. The mapping of data in the file to data in the time series occurs first by assigning column names, using one of the following methods: 1. Read column names from a line in the file, suitable when the column headings are simple strings and agree closely with the contents of the data columns. 2. Assign column names with command parameters. The file being read may include metadata within column headings and data records; however, the information can be difficult to extract because of formatting. For example, column headings may include the data type as “Precipitation\n(in)” (where \n indicates a newline). Consequently, the command supports assigning column names via command parameters in order to ensure robust data handling. In any case, rather than trying to automatically determine other metadata like data type and units from the column heading, the values can be assigned with the DataType and Units parameters. Additional functionality may be added in the future automate metadata discovery. Examples of use for the two cases are shown in the examples below. The command syntax is as follows: ReadDelimitedFile(Parameter=Value,…) Command Parameters Parameter Description Default InputFile The name of the delimited input file to read, surrounded by double quotes to protect whitespace and special characters. Global property values can be used with the syntax ${PropertyName} (see also the SetProperty() command). None – must be specified. Delimiter The delimiter character(s) that separate columns. None – must be specified. Treat Consecutive Delimiters Indicate whether consecutive delimiter characters should be treated as a single delimiter, for example, when multiple spaces are used to line up columns. False (columns are separated by a single delimiter 295 ReadDelimitedFile () Command TSTool Documentation Command Reference – ReadDelimitedFile () -2 AsOne character) Comment Character(s) that if found at the start of lines in the file, indicate that the line is a comment. The characters are interpreted individually (e.g., #$ indicates that lines starting with # or $ will be treated as comments). # SkipRows Indicate absolute rows (1+) in the file to skip, using single numbers and ranges a-b, separated by commas. Rows are skipped prior to other processing. No rows will be skipped. SkipRowsAfter Comments Indicate the number of rows to skip after header comments. Use this parameter to skip column headers prior to the data lines. This parameter is typically not used if column names are read from the file. No rows will be skipped. ColumnNames The user-specified names for columns in the file, used to ensure that column headings in files are properly interpreted. These names are used in other parameters to specify columns in the file. Separate column names with commas. Column names can be specified as literal strings or as FC[start:stop] to read columns from the file header (assumed to be the first row after leading comments), where start is 1+ and stop is blank to read all columns or a negative number to indicate the offset from the end column. None – must be specified. DateTime Column The column matching a value in ColumnNames, which indicates the date/time column in the file. Date and time are in one column with no separating delimiter characters. Required if DateColumn is not specified. DateTime Format The format for date/time strings in the date/time column. If blank, common formats such as ISO YYYY-MM-DD hh:mm and MM/DD/YYYY will automatically be detected. However, it may be necessary to specify the format to ensure proper parsing. This format will be used to parse date/times from the DateTimeColumn or the merged string from the DateColumn and TimeColumn (if specified). The format string will depend on the formatter type. Currently, only the “C” formatter is available, which uses C programming language specifiers. The resulting format includes the formatter and specifiers (e.g., C:%m%d%y). Will automatically be determined by examining date/time strings. DateColumn The column matching a string in ColumnNames, which indicates the date column in the file. Required if DateTimeColumn is not specified. TimeColumn The column matching a string in ColumnNames, which indicates the time column in the file. Specify this parameter when DateColumn is specified and time is specified in a separate column. The DateColumn and TimeColumn contents are merged with a joining colon character and are then treated as if DateTimeColumn had been specified. A time column is required only when DateColumn is specified and the interval requires time. ValueColumn The column(s) matching a string in ColumnNames, which indicate the data value columns. Separate column names with commas. The FC[start:stop] notation discussed None – must be specified. 296 TSTool Documentation ReadDelimitedFile () Command Command Reference – ReadDelimitedFile () -3 for ColumnNames can also be used. FlagColumn The column(s) matching a string in ColumnNames, which indicate the data flag columns. Separate column names with commas. The FC[start:stop] notation discussed for ColumnNames can also be used. If specified, the number of columns must match the ValueColumn parameter, although blanks are allowed. Double-quotes around flags are not considered part of the flag. Flags are not read. LocationID The location identifier(s) to assign to time series for each of the value columns (or specify one value to apply to all columns). The FC[start:stop] notation discussed for ColumnNames can also be used. None – must be specified. Provider The data provider identifier to assign to time series for each of the value columns (or specify one value to apply to all columns). No provider will be assigned. DataType The data type to assign to time series for each of the value columns (or specify one value to apply to all columns). Use the value column names for the data types. Interval The interval for the time series. Only one interval is recognized for all the time series in the file. Interval choices are provided when editing the command. If it is possible that the date/times are not evenly spaced, then use the IRREGULAR interval. None – must be specified. Scenario The scenario to assign to time series for each of the value columns (or specify one value to apply to all columns). No scenario will be assigned. Units The data units to assign to time series for each of the value columns (or specify one value to apply to all columns). No units will be assigned. Missing Strings that indicate missing data in the file (e.g., “m”). Interpret empty column values as missing data. Alias The alias to assign to time series, as a literal string or using the special formatting characters listed by the command editor. The alias is a short identifier used by other commands to locate time series for processing. No alias will be assigned. InputStart The date/time to start reading data. All data or global input start. InputEnd The date/time to end reading data. All data or global input end. Example of Column Names Assigned with Command Parameter The following example for two time series (gate height and discharge) illustrates a format where column headings are complex enough to require assignment of column names using a command parameter: #... #Data is returned in TAB delimited format. Data miners may find help on automating #queries and formatting parameters at http://www.dwr.state.co.us/help # #Gaging Station: ALVA B. ADAMS TUNNEL AT EAST PORTAL NEAR ESTES PARK (ADATUNCO) #Retrieved: 3/30/2010 03:04 297 ReadDelimitedFile () Command TSTool Documentation Command Reference – ReadDelimitedFile () -4 #-----------------------------------------------------------------------------Station Date/Time GAGE_HT (ft) DISCHRG (cfs) ADATUNCO 2006-10-01 00:00 2.34 225 ADATUNCO 2006-10-01 00:15 2.34 225 …etc… The following dialog is used to edit the command and illustrates the syntax for the command. The column headings are skipped because they are assigned with a command parameter. Because the delimiter is a tab, the space between date and time columns is NOT used as a delimiter and the date/time information is treated as one column. ReadDelimitedFile ReadDelimitedFile() Command Editor when Literally Specifying Column Names The following example command file retrieves real-time time series data from the State of Colorado’s website and reads the data: WebGet(URI="http://www.dwr.state.co.us/SurfaceWater/data/export_tabular.aspx? IDADATUNCO&MTYPEGAGE_HT,DISCHRG&INTERVAL1&START10/1/06&END10/6/06", LocalFile="Data\CO-DWR-ADATUNCO-tab.txt") ReadDelimitedFile(InputFile="Data\CO-DWR-ADATUNCO-tab.txt", 298 TSTool Documentation ReadDelimitedFile () Command Command Reference – ReadDelimitedFile () -5 Delimiter="\t",ColumnNames="ID,DateTime,GAGE_HT,DISCHRG", DateTimeColumn="DateTime",ValueColumn="GAGE_HT,