wcurl1

Usage

Overview

wcurl is a simple curl wrapper which lets you use curl to download files without having to remember any parameters.

Call wcurl with a list of URLs you want to download and wcurl picks sane defaults.

If you need anything more complex, you can provide any of curl's supported parameters via the --curl-options option. Beware that you likely should be using curl directly if your use case is not covered.

By default, wcurl does:

* Percent-encode whitespace in URLs;

* Download multiple URLs in parallel

if the installed curl's version is \>= 7.66.0 (--parallel);

* Use a total number of 5 parallel connections to the same protocol + hostname + port number target

if the installed curl's version is \>= 8.16.0 (--parallel-max-host);

* Follow redirects;

* Automatically choose a filename as output;

* Avoid overwriting files

if the installed curl's version is \>= 7.83.0 (--no-clobber);

* Perform retries;

* Set the downloaded file timestamp

to the value provided by the server, if available;

* Default to https

if the URL does not contain any scheme;

* Disable curl's URL globbing parser

so {} and [] characters in URLs are not treated specially;

* Percent-decode the resulting filename;

* Use 'index.html' as the default filename

if there is none in the URL, or if it is '-'.

Options

--curl-options, --curl-options=<CURL_OPTIONS>...

Specify extra options to be passed when invoking curl. May be specified more than once.

-o, -O, --output, --output=<PATH>

Use the provided output path instead of getting it from the URL. If multiple URLs are provided, resulting files share the same name with a number appended to the end (curl >= 7.83.0). If this option is provided multiple times, only the last value is considered.

--no-decode-filename

Do not percent-decode the output filename, even if the percent-encoding in the URL was done by wcurl, e.g.: The URL contained whitespace.

--dry-run

Do not actually execute curl, print what would be invoked.

-V, --version

Print version information.

-h, --help

Print help message.

Curl_options

Any option supported by curl can be set here. This is not used by wcurl; it is instead forwarded to the curl invocation.

The value is split on whitespace (spaces, tabs and newlines) and each resulting word is passed to curl as a separate argument. Quotes and backslashes are not interpreted, so an argument containing whitespace cannot be passed this way. For example, --curl-options='-H "Cookie: a=1; b=2"' passes the four arguments -H, "Cookie:, a=1; and b=2" to curl, which treats the extra words as URLs to download.

To pass arguments that contain whitespace, write them to a curl config file and pass it with --curl-options="--config <FILE>", or use curl directly.

Everything in <CURL_OPTIONS> becomes part of the curl command line, so never build it from untrusted input.

URL

URL to be downloaded. Anything that is not a parameter is considered an URL. Whitespace is percent-encoded and the URL is passed to curl, which then performs the parsing. May be specified more than once.

Examples

Download a single file:

wcurl example.com/filename.txt

Download two files in parallel:

wcurl example.com/filename1.txt example.com/filename2.txt

Download a file passing the --progress-bar and --http2 flags to curl:

wcurl --curl-options="--progress-bar --http2" example.com/filename.txt

Resume from an interrupted download. The options necessary to resume the download (--clobber --continue-at -) must be the last options specified in --curl-options. Note that the only way to resume interrupted downloads is to allow wcurl to overwrite the destination file:

wcurl --curl-options="--clobber --continue-at -" example.com/filename.txt

Download multiple files without a limit of concurrent connections per host (the default limit is 5):

wcurl --curl-options="--parallel-max-host 0" example.com/filename1.txt example.com/filename2.txt