Latest program version checker and auto-downloader
vcheck [options]
vcheck is a tool for checking for latest versions of programs at \s-1HTTP\s0 and \s-1FTP\s0 locations given a list of URLs and (Perl-style) regular expressions to match, and to optionally download them automatically.
For a complete list of command line options, run
$ vcheck --help
vcheck's behavior can be influenced by both command line options and a configuration file, which at the same time serves as its data file, holding records of programs to check for. This config file is, by default (see \*(L"\s-1FILES\s0\*(R"), ~/.vcheck. It is structured according to a syntax which is printed in detail when run as
$ vcheck --grammar
Details about both the grammar in general and the meaning of involved keyword can be found in \*(L"\s-1GRAMMAR\s0\*(R". An example of what a config file looks like in principle can be found in \*(L"\s-1EXAMPLES\s0\*(R".
The basic purpose of vcheck is to check for new versions of programs listed in its config file. The script is able to cope with all kinds of common version numbers, including words like \*(L"pre\*(R" or \*(L"alpha\*(R", etc. When a new version was found, the config file is updated accordingly.
Furthermore, vcheck can be used to download files automatically if a new version is/was found, and even delete obsolete versions found locally automatically. A special field in each program's record in the config file tells it which version has last been downloaded. Where necessary or desired, the download can be disabled for specific programs, or disabled in general and allowed in special cases. Besides, you can specify preferences (both in general and on a per-program basis) as to what kind of files to download if new versions are available, say, in different formats.
vcheck also has features to limit the scope of programs to check or download to a subset, such as defined by:
a regular expression names have to match
a minimal urgency (which can be defined on a per-program basis, as levels of high, medium, and low)
those programs which haven't yet been downloaded since a new latest version was found
those programs which previous queries failed for (optionally, a certain minimum number of times)
a conjunctive combination of several of these conditions
In case you're behind a firewall, an \s-1HTTP\s0(-based) proxy can be defined in a number of ways (precendence in this order):
a specific \s-1HTTP\s0 or \s-1FTP\s0 proxy, respectively, defined in the config file
a common \s-1HTTP+FTP\s0 proxy defined in the config file
a specific \s-1HTTP\s0 or \s-1FTP\s0 proxy, defined via the environment variables $http_proxy/$HTTP_PROXY and $ftp_proxy/$FTP_PROXY, respectively (each in this order of precendence)
vcheck uses \s-1ANSI\s0 escape sequences to visually enhance its output. Success messages are usually printed in green, error messages are yellow or red (signalling severity). This feature can be disabled temporarily by using the corresponding command line switch, or permanently, via a setting in the configuration file (see \*(L"\s-1CONFIGURATION\s0 \s-1SECION\s0\*(R").
When run as
vcheck --grammar
vcheck will print its config file's grammar, i.e., the formal structure of the entries therein. The individual fields' names are printed along with short descriptions; details on their meaning and usage can be found below in this section.
Per default (i.e., if the script's name has not been changed (see \*(L"\s-1FILES\s0\*(R") and if not overridden via \*(C`--file\*(C'), vcheck reads its configuration from ~/.vcheck. This file will also be rewritten regularly whenever version information etc. about a program is updated. In the course of such rewrites, entries will be sorted in a definable fashion, and a hard-coded order of keywords and indentation scheme will be applied.
Basically, the config file may contain two types of records: a configuration section and any number of program sections. A record (or section\*(--these terms are used synonymously in this documentation) consists of a keyword marking its beginning and a name (this only goes for program sections), followed by an equal sign (`=') and a pair of curly braces (\*(L"{}\*(R"), between which the section's data is put.
Section data is a sequence of settings, or fields, of a number of types, some of which are obligatory while others are optional, separated by white space (typically, line feeds, to keep things readable). There are the following types of fields:
Keywords of this type set a property based on their mere presence. An example of this is the config section field dldefaultno: config = { dldefaultno }
String fields consist of a keyword followed by an equal sign (`=') and a string representing the field's value. If the string value contains white space or (double) quotation marks, it needs to be surrounded by (double) quotation marks (`"'). In this case, both quotation marks inside the string and backslashes need to be escaped by backslashes (`\'). Note that string values may not span several lines but have to be contained on a single one, and there may be validation rules as to what the value may be like. Besides, string fields are typically required to be of non-zero length. An example of this type of field is the prog section field comment: prog foo = { [...] comment = Hello! comment = "Comment with white space and \"quotes\"!" [...] }
String enumerations are basically string fields with but a limited set of allowed values. An example of this is the prog section field dl, whose value must be either \*(L"yes\*(R" or \*(L"no\*(R", if present: prog foo = { [...] dl = yes [...] }
The configuration section is optional and, if present, contains settings globally affecting vcheck's default behavior. The configuration section is unique per file (although multiple occurrences with non-conflicting settings are allowed, but these will be joined into a single section once the file is rewritten).
The keyword introducing a configuration section is config. Thus, a config section's principal layout looks like this:
config = { [...] }
The keywords allowed inside (\*(L"[...]\*(R") the config section are explained in detail below (listed in alphabetical order):
Specifies the checking urgency level to assume, unless specified otherwise in a program's record via prog.urgency. Urgencies allow for a crude selection of programs to check for via the \*(C`--urgency\*(C' command line parameter. In absence of this option, the default urgency is medium.
If included in the config section, causes the script to automatically look for and delete versions of a program obsoleted by a new download. May be overridden by prog.deleteold. See the latter for details. Special note: Use at your own risk!
By default, don't download. This causes the script to download only those programs whose dl option is explicitly set to yes when run with the \*(C`--download\*(C' parameter.
This option specifies an absolute path (i.e., relative to the root directory) of a directory where to put downloaded files. If the download directory isn't set via this or even more explicitly via a prog.dldir option, downloads will end up in that directory in which the script is executing.
Specifies a command to be executed after any successful download (unless overridden for a particular program via prog.dlexec). A successful download in this context is one whose file type has been recognized and whose integrity could be confirmed. In unizoid environments, the command is executed under whatever shell the environment variable $SHELL defines. The command string is subject to expansion of the following placeholders (see \*(L"\s-1PLACEHOLDERS\s0\*(R" for their meaning): \*(C`_\|_DLURL_\|_\*(C', \*(C`_\|_FILE_\|_\*(C', \*(C`_\|_NEWVER_\|_\*(C', \*(C`_\|_PROG_\|_\*(C', \*(C`_\|_RAWVER_\|_\*(C', \*(C`_\|_URL_\|_\*(C'. Additionally, `~/' will be replaced by the user's home directory. config.dlexec may prove useful to, e.g., automatically convert, say, gzipped to bzipped files using a helper script, or to log downloads (see \*(L"\s-1HINTS\s0\*(R").
A semicolon- (`;'-) separated list of Perl-style regular expressions defining download preferences. Each of the regular expressions is supposed to match a particular file type that's possible or likely to be encountered. The order in which the expressions occur defines their precedence (the first matching expression will determine which of a set of available file types of a given program version will be selected for download). This value is the default in effect unless specific preferences are defined on a per-program basis using prog.dlprefs. If neither config.dlprefs nor prog.dlprefs is set, the file to be downloaded is chosen pseudo-randomly, if multiple pattern matches occur. For these download preferences to make any sense, file- and version-matching expressions need to be sufficiently non-restrictive to match several possible extensions. For example, "foo-(\*(C`_\|_VER_\|_\*(C')\\.t\*(L" will match both \*(R".tar.gz\*(L" and \*(R".tar.bz2" files, and setting dlprefs to \*(L"\\.tar\\.bz2$;\\.tar\\.gz$\*(R" will cause the script to preferrably download \*(L".tar.bz2\*(R" files.
The number of times to retry downloading after a failed download. If this option isn't specified, the number of retries defaults to 0. A retry is considered to have failed if either the connection failed, the retrieved document was empty, or the file type has been recognized and its integrity verified.
If this option is set, all string parameters of configuration file options will be surrounded by double quotes. The default is to use quotes only where necessary (e.g., for string parameters containing white space).
If this option is set, commands executed thanks to newverexec or dlexec options will be echoed prior to execution. This option specifies a proxy to use for retrieving documents from \s-1FTP\s0 locations. It specifies either the complete \s-1URL\s0 or the server and port (as \*(L"server:port\*(R") of the proxy, and the proxy has to be a HTTP-based \s-1FTP\s0 proxy. This option takes precedence over config.proxy, if specified. If neither config.ftpproxy nor config.proxy is set, the script uses the value the environment variables $ftp_proxy or $FTP_PROXY (in this order of precedence) are set to, or no \s-1FTP\s0 proxy at all. This option specifies a proxy to use for retrieving documents from \s-1HTTP\s0 locations. It specifies either the complete \s-1URL\s0 or the server and port (as \*(L"server:port\*(R") of the proxy. This option takes precedence over config.proxy, if specified. If neither config.httpproxy nor config.proxy is set, the script uses the value the environment variables $http_proxy or $HTTP_PROXY (in this order of precedence) are set to, or no \s-1HTTP\s0 proxy at all. The date and time the script was last run updating the configuration file. This value is generated and updated automatically.
A command to be executed whenever a new version of a program is found, unless overridden on a per-program basis via prog.newverexec. The command is executed under whatever shell the environment variable $SHELL defines. The command string is subject to expansion of the following placeholders (see \*(L"\s-1PLACEHOLDERS\s0\*(R" for their meaning): \*(C`_\|_NEWVER_\|_\*(C', \*(C`_\|_PROG_\|_\*(C', \*(C`_\|_RAWVER_\|_\*(C', \*(C`_\|_URL_\|_\*(C'. Additionally, `~/' will be replaced by the user's home directory.
This option conserves some memory by not caching retrieved documents (those fetched from prog.url locations). By default, the script caches retrieved document so that program records referring to the same web page won't result in (unnecessary) multiple retrievals during the same session.
This option causes the script to generate plain (as opposed to ANSI-enhanced) output by default. The option may be overridden by specifying \*(C`--noplain\*(C' on the command line. This option specifies a proxy to use for retrieving documents from both \s-1HTTP\s0 and \s-1FTP\s0 locations. It specifies either the complete \s-1URL\s0 or the server and port (as \*(L"server:port\*(R") of the proxy. The proxy set via this option may be overridden via config.ftpproxy and/or config.httpproxy.
This option specifies whether to sort prog entries by program name (prog section identifier) or \s-1URL\s0 when rewriting the configuration file. The default is to sort by name.
Corresponds to the command line option \*(C`--xfersum\*(C'. If set, the script will print a total of the amount of data that has been received at exit. Can be overridden via the command line switch \*(C`--noxfersum\*(C'.
The time (in seconds) after which attempted remote retrievals should be aborted. The default is 90 seconds.
If this option is set, the script will also print version numbers that haven't been obsoleted. The default is to print only new versions (and error messages). This setting can be overridden via the command line switch \*(C`--noverbose\*(C'.
Program sections each define for a single program (package, ...) an \s-1HTTP\s0 or \s-1FTP\s0 \s-1URL\s0 based on which the latest version of that program available can be determined by vcheck using an additionally-defined regular expression. There can (hypothetically) be any number of program sections in a config file.
The keyword introducing a program section is prog. Each prog section is identified by a unique identifier (there may not be multiple prog sections with the same identifier). Thus, a config section's principal layout looks like this:
prog Foo = { [...] }
The keywords allowed inside (\*(L"[...]\*(R") a prog section are explained in detail below (listing in alphabetical order). All fields are optional and allowed but once per prog section, unless explicitly stated otherwise.
An arbitrary comment string. If multiple such entries exist for a single program record, their relative order will be maintained when rewriting the configuration file.
This option defines whether the script should look for and delete any obsolete versions of a program located in its download directory after each 'Usuccessful' download of a new version of that program. A successful download in this context is any download of a file of a known type whose integrity could be verified. Overrides config.deleteold; the deletion of obsolete versions is disabled by default and only activated by config.deleteold or prog.deleteold. Any occurrence of prog.dlexplicit or prog.dlintermediate in a program's record inhibits application of deleteold for that program. Special note: Use at your own risk!
This option causes the program record in question to be ignored (except when the command line switch \*(C`--force\*(C' is used).
This option specifies whether to download the program in question when the script is run with the \*(C`--download\*(C' option. By default, a program will be downloaded when a new version is found and the script is run with said parameter, unless config.dldefaultno is set. prog.dl overrides the latter option.
This option specifies a download directory on a per-program basis. If the directory is absolute (i.e., relative to the root directory, as indicated by a leading slash), it will be treated as an absolute path, otherwise it will be considered relative to either config.dldir, if specified, or the directory the script is executing in. The dldir string is subject to expansion of the following placeholders (see \*(L"\s-1PLACEHOLDERS\s0\*(R" for their meaning): \*(C`_\|_NEWVER_\|_\*(C'.
This option specifies the download directory of the last downloaded version of a program. It does not contains \*(L"\s-1PLACEHOLDERS\s0\*(R" unlike \*(C`prog.dldir\*(C'. If the directory is absolute (i.e., relative to the root directory, as indicated by a leading slash), it will be treated as an absolute path, otherwise it will be considered relative to either config.dldir, if specified, or the directory the script is executing in. This option is only used to store \*(L"\s-1PLACEHOLDER\s0\*(R" free dldir, it is overwriten at each new download.
Specifies a command to be executed after any successful download of the program, overriding config.dlexec (if set). A successful download in this context is one whose file type has been recognized and whose integrity could be confirmed. The command is executed under whatever shell the environment variable $SHELL is set to. The command string is subject to expansion of the following placeholders (see \*(L"\s-1PLACEHOLDERS\s0\*(R" for their meaning): \*(C`_\|_DLURL_\|_\*(C', \*(C`_\|_FILE_\|_\*(C', \*(C`_\|_NEWVER_\|_\*(C', \*(C`_\|_PROG_\|_\*(C', \*(C`_\|_RAWVER_\|_\*(C', \*(C`_\|_URL_\|_\*(C'. Additionally, `~/' will be replaced by the user's home directory.
Specifies an explicit download \s-1URL\s0. Whenever a new version of the program in question is found, the \s-1URL\s0 specified via this option will be downloaded (if requested) instead of the one deduced from prog.url and prog.regex. The command string is subject to expansion of the following placeholders (see \*(L"\s-1PLACEHOLDERS\s0\*(R" for their meaning): \*(C`_\|_NEWVER_\|_\*(C', \*(C`_\|_RAWVER_\|_\*(C'. This option can also be used to, e.g., download multiple packages on detection of a new version, provided that their names can be specified. For an example of this, see \*(L"\s-1EXAMPLES\s0\*(R".
If this option is set, intermediate versions (i.e., version referenced at url newer than dlversion but older than the most recent version available) will be downloaded as well if any are encountered when a new version of the program is found. This option is useful for downloading patches and suchlike, which depend on each other consecutively. The default is to ignore intermediate versions.
A semicolon- (`;'-) separated list of Perl-style regular expressions defining download preferences. Each of the regular expressions is supposed to match a particular file type that's possible or likely to be encountered. The order in which the expressions occur defines their precedence (the first matching expression will determine which of a set of available file types of a given program version will be selected for download). This value overrides default preferences possibly defined via config.dlprefs. If neither config.dlprefs nor prog.dlprefs is set, the file to be downloaded is chosen pseudo-randomly, if multiple pattern matches occur. For these download preferences to make any sense, file- and version-matching expressions need to be sufficiently non-restrictive to match several possible extensions. For example, "foo-(\*(C`_\|_VER_\|_\*(C')\\.t\*(L" will match both \*(R".tar.gz\*(L" and \*(R".tar.bz2\*(L" files, and setting `dlprefs' to \*(R"\\.tar\\.bz2$;\\.tar\\.gz$\*(L" will cause the script to preferrably download \*(R".tar.bz2" files.
Specifies an \s-1HTTP\s0 referrer to use when downloading a program package. By default, the version-determining document (i.e., the last url value, with placeholders expanded) is used.
This parameter stores the last downloaded version of the program in question and is updated whenever a new version is found (except when running in read-only mode). If prog.transform is set, the stored version will have been transformed from the one matched by prog.regex.
This field stores the number of errors during version checks and is reset once a check succeeds. A high value of this field is indicative of an outdated \s-1URL\s0 or file name matching regular expression and will be remarked upon by the script. Additionally, it is possible to limit the scope of an operation to erroneous records via the \*(C`--errors\*(C' command line parameter. This field stores the date and time the program in question was last checked (no matter whether successfully or unsuccessfully).
A command to be executed whenever a new version of a program is found, overriding a possible definition via config.newverexec. The command is executed under whatever shell the environment variable $SHELL defines. The command string is subject to expansion of the following placeholders (see \*(L"\s-1PLACEHOLDERS\s0\*(R" for their meaning): \*(C`_\|_NEWVER_\|_\*(C', \*(C`_\|_PROG_\|_\*(C', \*(C`_\|_RAWVER_\|_\*(C', \*(C`_\|_URL_\|_\*(C'. Additionally, `~/' will be replaced by the user's home directory.
This required field is supposed to contain a Perl-style regular expression matching desired versions of the program in question given the document at prog.url as input. Note that the regexp needn't match the complete file name\*(--when considering a download, the script will auto-expand the match as seen fit. Regular expressions for matching programs' version numbers have to be written in such a way that the $1 part (see the \*(L"perlre\*(R" man page), if the entire expression matches, is exactly the version number. The option is subject to placeholder expansion: \*(C`_\|_VER_\|_\*(C' will be replaced by a pre-manufactured (non-greedy) regular expression matching version numbers compliant with any of a number of common schemes. Note that in order to yield a $1 match as required, \*(C`_\|_VER_\|_\*(C' still needs to be put in parentheses. For examples of prog.regex values, see \*(L"\s-1EXAMPLES\s0\*(R". In order to cope with particularly complex remote scenarios (such as version-dependent directory hierarchies), multiple url, regex, and transform fields may be specified per program. In this case, the script will match urls and regexes starting with the first and continuously proceeding to the next field of each type (in sync, as long as both of them are available, or using the last one available otherwise) and match the regexp against the corresponding document. In order for this to be of any use, the second (and each potential later) url will have to contain a \*(C`_\|_NEWVER_\|_\*(C' or \*(C`_\|_RAWVER_\|_\*(C' placeholder (see \*(L"\s-1PLACEHOLDERS\s0\*(R") which will be replaced by the previously matched latest [transformed] version (the same substitution is done for regex). The version that will finally be considered the latest for the program in question will be the one determined by matching the last regex against the last urls document. For an example of how this can be used in practice, see \*(L"\s-1EXAMPLES\s0\*(R". Possible multiple transform fields will be processed in sync with the respective url and regex fields as long as additional transform fields are specified. If there are more url and/or regex fields than transform fields, the last-specified transform expression will be used for further iterations. If, on the other hand, there are more transform than url/regex fields, further retrievals/matches will be done based on the last url/regex. The author has, however, no idea how this could be of any use. When the config file is rewritten, multiple url, regex, and/or transform fields will be interleaved to facilitate comprehension and retain their relative order.
A Perl expression transforming a version number in $_ (obtained by a prog.regex match) in some way the user deems adequate. For examples of how this might come in handy, see \*(L"\s-1EXAMPLES\s0\*(R". The return value of the code fragment, i.e., the value of its last expression, is used as the transformed version and will henceforth be the basis for version comparison for the program in question.
Defines the urgency with which to check for the specified program. Urgencies allow for a crude selection of programs to check for via the \*(C`--urgency\*(C' command line parameter. If there is no urgency defined, it defaults to either config.defaulturgency (if set) or medium.
This required field defines the \s-1HTTP\s0 or \s-1FTP\s0 \s-1URL\s0 to retrieve as the document to scan for in order to detect the availability of new program versions by matching against prog.regex. Note that if the \s-1URL\s0 is a directory (especially, an \s-1FTP\s0 directory which is supposed to be listed), the \s-1URL\s0 needs to end in a slash (`/'). If the target document is an \s-1HTML\s0 page, its source code will be matched against prog.regex, aiming at links embedded in the document. An alternate download \s-1URL\s0 can be specified via prog.dlexplicit. In order to cope with particularly complex remote scenarios (such as version-dependent directory hierarchies), multiple url, regex, and transform fields may be specified per program. In this case, the script will match urls and regexes starting with the first and continuously proceeding to the next field of each type (in sync, as long as both of them are available, or using the last one available otherwise) and match the regexp against the corresponding document. In order for this to be of any use, the second (and each potential later) url will have to contain a \*(C`_\|_NEWVER_\|_\*(C' or \*(C`_\|_RAWVER_\|_\*(C' placeholder (see \*(L"\s-1PLACEHOLDERS\s0\*(R") which will be replaced by the previously matched latest [transformed] version (the same substitution is done for regex). The version that will finally be considered the latest for the program in question will be the one determined by matching the last regex against the last urls document. For an example of how this can be used in practice, see \*(L"\s-1EXAMPLES\s0\*(R". Possible multiple transform fields will be processed in sync with the respective url and regex fields as long as additional transform fields are specified. If there are more url and/or regex fields than transform fields, the last-specified transform expression will be used for further iterations. If, on the other hand, there are more transform than url/regex fields, further retrievals/matches will be done based on the last url/regex. The author has, however, no idea how this could be of any use. When the config file is rewritten, multiple url, regex, and/or transform fields will be interleaved and retain their relative order.
Stores the latest known version of the program. In contrast to prog.dlversion, this is the latest version detected, not the latest version downloaded. If a prog.transform option is set, the stored version will have been transformed from the one matched by prog.regex.
In a number of string fields, certain placeholders are subject to substitution by run-time values. These placeholders are (in alphabetical order): The (file) \s-1URL\s0 from which the latest version of the respective program was downloaded. The local path to the respective latest-version download. \*(C`_\|_NEWVER_\|_\*(C' is replaced by the latest transformed (or untransformed, if no transform expression is in effect) version available as determined by the script. When using multiple url/regex/transform fields in order to cope with more complex remote site hierarchies, \*(C`_\|_NEWVER1_\|_\*(C', \*(C`_\|_NEWVER2_\|_\*(C', ... give access to intermediately-determined versions. In this case, \*(C`_\|_NEWVER1_\|_\*(C' is replaced by the version matched by the first url/regex/transform tuple, \*(C`_\|_NEWVER2_\|_\*(C' matches the version matched by the second url/regex/transform tuple, and so on. The name (identifier) of the respective prog section. \*(C`_\|_RAWVER_\|_\*(C' is replaced by the latest version available as determined by the script. When using multiple url/regex fields in order to cope with more complex remote site hierarchies, \*(C`_\|_RAWVER1_\|_\*(C', \*(C`_\|_RAWVER2_\|_\*(C', ... give access to intermediately-determined versions. In this case, \*(C`_\|_RAWVER1_\|_\*(C' is replaced by the version matched by the first url/regex pair, \*(C`_\|_RAWVER2_\|_\*(C' matches the version matched by the second url/regex pair, and so on. The (last and expanded) \s-1URL\s0 used in order to determine the latest program version. A pre-manufactured (non-greedy) regular expression matching version numbers compliant with any of a number of common schemes.
If you use Vim (version 5 or higher) as your editor, you can tell vcheck to create a Vim syntax file providing syntax highlighting within the editor by running the script as vcheck --create-vim-syntax-file If you wish to have Vim apply the syntax rules automatically when editing \*(L"~/.vcheck\*(R", add this line: au BufEnter */.vcheck so $VIM/syntax/vcheck.vim or, alternatively, one with an explicit path: au BufEnter */.vcheck so /path/to/syntax/vcheck.vim to your \*(L"~/.vimrc\*(R" and substitute an appropriate path. Of course you need to as well be sure to copy the file into the designated directory.
It's no problem to just check for new versions by default and run vcheck again afterwards to download updated packages. Running the script as vcheck -dc or vcheck --download --catch-up respectively, will try to download only those files whose latest downloaded version has been knowingly obsoleted, without checking again for new versions of all other programs.
To check only those program locations that failed during the latest attempt(s), run vcheck -e or vcheck --errors respectively.
If you add a line dlretry = NUMBER to your config file's config section, vcheck will retry to download a file up to \s-1NUMBER\s0 times if it detects that it was received incompletely. This will be the case if:
the file has zero size
the downloaded file's extension was recognized, and a check by the respective decompressor etc. resulted in errors
vcheck caches data retrieved from URLs (unless nocache is set in the config file), so if you specify exactly the same \s-1URL\s0 for different programs, this won't result in multiple retrievals, thus improving efficiency.
If you're curious to know how many program records have actually been accumulated in your config file over time, run vcheck as vcheck --syntax This will check the config file's syntax and, as a side-effect, print the number of programs registered.
Even if you know from some other source that there is a new version of a program vcheck is configured for, you can still use that to download the package. Just use its matching capabilities, e.g.: vcheck -dm foo
If one of your records points to patches of some program, and you want to make sure you won't miss an intermediate one when downloading (and suppose you don't run vcheck in download mode too frequently), you can add the boolean field dlintermediate to the respective program's section in the config file, and vcheck will try to download all versions newer than dlversion. Note that in those circumstances, dlversion is set to the latest (intermediate) version the download attempt succeeded for (which means that, if, say, versions 1 through 3 are to be downloaded and all downloads except that of version 1 succeed, dlversion will nevertheless be set to 3). A useful example for this: prog Linux/patches = { dlintermediate dlprefs = \.bz2$;\.gz$ dlversion = 2.3.6 regex = patch-(_\|_VER_\|_)\.[bg]z url = ftp://ftp.kernel.org/pub/linux/kernel/v2.3/ version = 2.3.9 } Supposing that 2.3.9 still is the latest version, running this in download mode will retrieve Linux kernel patches 2.3.7 through 2.3.9, *.bz2 preferred to *.gz (but accepting the latter if the former is missing, rather than skipping the download entirely).
There may be complex remote site structures, involving version-dependent directory hierarchies, such as the layout used by the server for the \s-1AC\s0 series of Linux kernel patches. The principal layout of that site looks (or used to look, anyway) like this: ... .../linux-2.4/2.4.8/patch-2.4.8-ac1.gz .../linux-2.4/2.4.8/patch-2.4.8-ac2.gz ... .../linux-2.4/2.4.9/patch-2.4.9-ac1.gz .../linux-2.4/2.4.9/patch-2.4.9-ac2.gz ... The problem here is that the bottom-level directory's name varies depending on the regular Linux version an \s-1AC\s0 patch is based on. The way to deal with this most conveniently in vcheck looks like this: prog Linux/patch/AC = { dlintermediate url = http://www.kernel.org/.../linux-2.4/ regex = (\d+\.\d+\.\d+) url = http://www.kernel.org/.../linux-2.4/_\|_NEWVER_\|_/ regex = patch-(_\|_VER_\|_-ac\d+)\.gz } (Note that the URLs have been abbreviated for the sake of readability.) This kind of configuration will cause vcheck to start by retrieving the first url field's document and match the first regex against it. It will then proceed with the second url field's document, matching it against the second regex, replacing its _\|_NEWVER_\|_ placeholder by the latest version previously matched. The version finally determined as the current version for the program record is the one determined by the last match. On a side note, version numbers determined during matches further back than the previous one can be accessed via delimiters of the format _\|_NEWVER#_\|_, where `#' is a number indicating the number (1..) of the url/regex pair's version match it should be replaced by. For more details on the mechanism, see the descriptions of url and regex in \*(L"\s-1PROGRAM\s0 \s-1SECTIONS\s0\*(R", and \*(L"\s-1PLACEHOLDERS\s0\*(R". Regarding the example, it is left to the user to figure out how to extend the record to even automatically cope with changes to the Linux kernel's minor version. \*(C`:-)\*(C' Here's another example of a three-level hierarchy, which used to fit the \s-1GIMP\s0's site layout at one point in time: prog GIMP/devel/patch = { comment = "Will download complete package if no patch available." dlprefs = patch-.*?\.bz2$;patch-.*?\.gz$;gimp-.*?\.bz2$;gimp-.*?\.gz$ url = ftp://ftp.gimp.org/pub/gimp/ regex = (?<!\w)v(_\|_VER_\|_) url = ftp://ftp.gimp.org/pub/gimp/v_\|_NEWVER_\|_/ regex = (?<!\w)v(_\|_VER_\|_) url = ftp://ftp.gimp.org/pub/gimp/v_\|_NEWVER1_\|_/v_\|_NEWVER_\|_/ regex = (?:patch|gimp)-(_\|_VER_\|_)\.[bgt] }
If you want to retrieve some program whose version is, say, a date in format \*(L"dd-mm-yyyy\*(R", this will be misinterpreted by the version comparator because the most significant sub-\*(L"version\*(R" isn't the initial one. You can work around this by specifying some Perl expression transforming the original version in the respective program's section, such as: transform = "s/(\d+)-(\d+)-(\d+)/$3-$2-$1/; $_" This piece of code is given the respective version in $_, and after its evaluation, vcheck replaces the original value by what the eval() returns. Alternatively, this would achieve the same: transform = "join '-', reverse split /-/, $_"
Some sites use redirection scripts for download URLs. Consider a situation where a downloads page lists available packages of a program, with links pointing to some server-side script referring your browser to some \s-1URL\s0 which is in turn redirected by \s-1HTTP\s0 means to a final file \s-1URL\s0 (the \s-1PHP\s0 site, for example, used to make use of this obscure scheme at one time). The way to cope with this in vcheck consists in using a dlexplicit field like this: prog PHP = { dlexplicit = http://www.php.net/distributions/php-_\|_VER_\|_.tar.gz regex = php-(_\|_VER_\|_)\.t url = http://www.php.net/downloads.php } Effectively, this will use the actual url field only to determine the current version and then paste it into a pattern of the corresponding download \s-1URL\s0, thus bypassing the redirections. The obvious disadvantage of this feature consists in its increased dependency on server-side access structures.
Suppose you're interested in some program distributed via more than one package (such as Vim, which is split into a source and a run-time package). The means vcheck provides to cope with this once again is the dlexplicit option: prog Vim = { dlexplicit = ftp://ftp.vim.org/pub/editors/vim/unix/vim-_\|_VER_\|_-src.tar.gz dlexplicit = ftp://ftp.vim.org/pub/editors/vim/unix/vim-_\|_VER_\|_-rt.tar.gz regex = vim-(_\|_VER_\|_)(-src)?\.tar url = ftp://ftp.vim.org/pub/editors/vim/unix/ }
In order to have vcheck keep track of what has been downloaded (and when), you might add something like this to your config file: config = { dlexec = "echo `date +%Y-%m-%d` '_\|_PROG_\|_' '_\|_NEWVER_\|_' >>~/.vchecklog" } Note however that program-specific dlexec will take precedence over this setting.
With a little creativity, vcheck can be used to check not only for latest versions of programs or packages, but also web site updates and the like. Also, the newverexec (see \*(L"\s-1GRAMMAR\s0\*(R") field can be used to pass a link to an external download tool if for some reason vcheck's abilities prove insufficient for a particular scenario.
Please make sure to read what's printed by vcheck when run as
vcheck --help --grammar
as well as \*(L"\s-1GRAMMAR\s0\*(R" before reading this section, to learn about command line parameters and the configuration file's grammar. Done so? Then read on...
Suppose there's a config file ~/.vcheck with the following contents:
config = { dlprefs = \.tar\.bz2$;\.(tar\.|t)gz$;\.zip$ lastcheck = "1999-06-21 08:15" }
prog Foo = { dl = no errors = 2 regex = foo-(_\|_VER_\|_)\.tar urgency = high url = http://www.foo.org/pub/foo/ } prog Bar = { dlversion = 0.01beta regex = (?i:bar-(_\|_VER_\|_)\.tar) url = http://www.bar.org/bar/index.html version = 0.01 } prog Baz = { regex = baz-(\d+)\.tar urgency = low url = ftp://ftp.baz.net/pub/source/ version = 123 }
First of all, you can deduce from this what date and time vcheck was last run at with this config file. Trying to check for Foo resulted in errors of some kind during the last 2 attempts, and since there's no version field, it has presumably never been queried successfully. Foo is never to be downloaded. Bar's latest version as determined during one of the last checks was 0.01, but it wasn't downloaded (0.01beta is the version of the last download). Finally, Baz has never been downloaded (according to the config file, anyway). As for downloads in general, *.tar.bz2 is preferred to *.tar.gz and *.tgz, which in turn are more desirable than *.zip files. If no target matching any of these extensions case-insensitively is found, nothing will be downloaded.
Assume furthermore that the following references are currently mentioned at the respective URLs of each program:
for Foo: http://www.foo.org/pub/foo/foo-3.14.tar.gz http://www.foo.org/pub/foo/foo-3.14.tar.bz2 http://www.foo.org/pub/foo/foo-3.14a.tar.gz http://www.foo.org/pub/foo/foo-3.14alpha.tar.gz http://www.foo.org/pub/foo/Foo-4.0.tar.gz
for Bar: bar-0.01.zip BAR-0.01.tar.bz2
for Baz: http://www.baz.net/pub/download/baz-124.rpm
Now let's discuss what some specific calls to vcheck, each based on the above configuration, will result in. Again, for a complete list of command line options (all short options have an equivalent long one), see \*(C``vcheck --help`\*(C'. This will check for all programs without updating the config file. It'll report Foo 3.14 as new version (not 4.0, as regex doesn't match this), as well as Baz 124. This will check for all programs, report as above and try to download the following file: http://www.bar.org/bar/BAR-0.01.tar.bz2 Note that Baz 124 isn't among, because there wasn't a link conforming to dlprefs, and downloads of Foo have been disabled explicitly. The errors field of Foo is removed since the check succeeded. This will set dlversion = version for Bar and Baz, without checking for the availability of new versions. Effectively, this will prevent future calls to vcheck with parameter "\*(C`-d\*(C'" from downloading these files. This will step through all programs that downloads haven't been disabled for in principle and whose dlversion is lower than version (i.e., Bar and Baz in our example). For these, vcheck will requery the respective sites to determine a download \s-1URL\s0, and try to download http://www.bar.org/bar/BAR-0.01.tar.bz2 as in the above example. will check for new versions of Bar and Baz. Note that you may have to quote the leading exclamation mark as well as some characters used in regular expressions specified on the command line, in order to prevent your shell from interpreting them. will check only for Bar, as it is the only program whose urgency is at least medium and whose name contains a `b'. will check only for Foo, since checking for that failed previously.
First of all, vcheck isn't perfect, and it won't do in all kinds of situations. Yet I think it is able to cope with most of them, and if there's indeed some site which vcheck isn't able to determine download URLs from, or some version numbering scheme its heuristics choke on, you'll just have to deal with that manually. But for the majority of cases, vcheck should facilitate keeping your setup up-to-date.
Don't run multiple instances in non-read-only mode with the same configuration file, or else one will cause the changes made by the others to get lost. vcheck will prevent this situation from arising by employing a lock file, provided that your Perl setup supports it. Don't edit the config file while vcheck is running, either, or your changes will be overwritten when the script rewrites the file.
vcheck, the script itself
~/.vcheck, its configuration file In fact, vcheck doesn't look for a config file ~/.vcheck, but for one of the same name as the script (with a possible extension stripped off). So if you rename the script to, say, foo.pl and run it, it'll try to open ~/.foo.
~/.vcheck.lock, lock file created when not running in read-only mode Actually, the file's name is that of the config file with an extension of .lock added.
$http_proxy/$HTTP_PROXY and $ftp_proxy/$FTP_PROXY, each in this order of precedence, specify the \s-1HTTP\s0 and/or (HTTP-based) \s-1FTP\s0 proxy to use, unless overridden. The format is either \*(L"server:port\*(R" or a complete \s-1URL\s0.
$HOME, the current user's home directory
$SHELL, used by Perl in unizoid environments when executing helper applications
add option to config section allowing for the dlexec entry to be \*(L"inherited\*(R" from the config section rather than be overridden by per-program dlexecs (also define order of execution!)
check behavior if an \s-1HTTP\s0 url's download link references a different target base directory
make \*(L"--list\*(R" not rewrite the config file, thus allowing for it to be run in parallel to another instance
add an option to re-download the latest version (if local file doesn't exist)?
code clean-up: array used for download specifications -> hash
determine and describe way of reliably matching directories consisting of but a version number on an \s-1FTP\s0 server independently of whether the page in question is received by proxy or without one
scenario: link description contains version, download link entirely differently named
separate \*(C`--force\*(C' options for overriding disabled, dl?
max download size command line parameter?
extend Vim syntax file generation to highlight placeholders in string variables' values?
follow \s-1HTTP\s0 redirections
evaluate \s-1HTTP\s0 headers after retrievals
resume downloads?
XMLize the config file format???
make it multi-threaded???
All output is currently printed on \s-1STDOUT\s0.
Placeholders are used but no way of escaping literal occurrences of those strings is provided.
dlprefs uses semicolons as delimiters, but there's no way of escaping them if they are meant as a part of one of the regular expressions.
There's presumably little to do in order to get vcheck to run in Microsoft Windows. One issue worth noting is that directories (such as dldir values) are expecedted to use unizoid delimiters (i.e., slashes (`/'))--this should perhaps be revised to be portable.
Mail bug reports to the author.
vcheck is copyright (c) 1999-2001 by Marco Go\*:tze, <[email protected]>. It is distributed under the terms of the Artistic License, a copy of which is included with the script's distribution. Use at your own risk.