curl is also capable of using client certificates to get/post files from sites that require valid certificates. The only drawback is that the certificate needs to be in PEM-format. PEM is a standard and open format to store certificates with, but it is not used by the most commonly used browsers. If you want curl to use the certificates you use with your favorite browser, you may need to download/compile a converter that can convert your browser's formatted certificates to PEM formatted ones.
As is mentioned above, you can download multiple files with one command line by simply adding more URLs. If you want those to get saved to a local file instead of just printed to stdout, you need to add one save option for each URL you specify. Note that this also goes for the -O option (but not --remote-name-all).
curl will connect to a server with IPv6 when a host lookup returns an IPv6 address and fall back to IPv4 if the connection fails. The --ipv4 and --ipv6 options can specify which address to use when both are available. IPv6 addresses can also be specified directly in URLs using the syntax:
AWS now publishes its current IP address ranges in JSON format. To view the current ranges, download the .json file using the following link. If you access this file programmatically, ensure that the application downloads the file only after successfully verifying the TLS certificate that is returned by the AWS server.
I have managed to push my application logs to AWS Cloudwatch by using the AWS CloudWatch log agent. But the CloudWatch web console does not seem to provide a button to allow you to download/export the log data from it.
aria2 is a utility for downloading files. The supported protocols areHTTP(S), FTP, SFTP, BitTorrent, and Metalink. aria2 can download afile from multiple sources/protocols and tries to utilize your maximumdownload bandwidth. It supports downloading a file from HTTP(S)/FTP/SFTP and BitTorrent at the same time, while the data downloaded fromHTTP(S)/FTP/SFTP is uploaded to the BitTorrent swarm. Using Metalinkchunk checksums, aria2 automatically validates chunks of data whiledownloading a file.
--max-concurrent-downloads limits the number of itemswhich are downloaded concurrently. --split and--min-split-size affect the number of connectionsinside each item. Imagine that you have an input file (see--input-file option) like this:
Here is 2 download items. aria2 can download these itemsconcurrently if the value more than or equal 2 is given to--max-concurrent-downloads. In each download item, youcan configure the number of connections using --split and/or --min-split-size, etc.
Check file integrity by validating piece hashes or a hash of entirefile. This option has effect only in BitTorrent, Metalink downloadswith checksums or HTTP(S)/FTP downloads with--checksum option. Ifpiece hashes are provided, this option can detect damaged portionsof a file and re-download them. If a hash of entire file isprovided, hash check is only done when file has been alreadydownload. This is determined by file length. If hash check fails,file is re-downloaded from scratch. If both piece hashes and a hashof entire file are provided, only piece hashes are used. Default:false
Continue downloading a partially downloaded file.Use this option to resume a download started by a web browser or anotherprogram which downloads files sequentially from the beginning.Currently this option is only applicable to HTTP(S)/FTP downloads.
Use a proxy server for all protocols. To override a previouslydefined proxy, use \"\". You also can override this setting and specify aproxy server for a particular protocol using --http-proxy,--https-proxy and --ftp-proxy options. This affects alldownloads.The format of PROXY is [http://][USER:PASSWORD@]HOST[:PORT].See also ENVIRONMENT section.
Set checksum. TYPE is hash type. The supported hash type is listedin Hash Algorithms in aria2c -v. DIGEST is hex digest. Forexample, setting sha-1 digest looks like this:sha-1=0192ba11326fe2298c8cb4de616f4d4140213838 This option appliesonly to HTTP(S)/FTP downloads.
If true is given, aria2 just checks whether the remote file isavailable and doesn't download data. This option has effect onHTTP/FTP download. BitTorrent downloads are canceled if true isspecified. Default: false
Close connection if download speed is lower than or equal to thisvalue(bytes per sec).0 means aria2 does not have a lowest speed limit.You can append K or M (1K = 1024, 1M = 1024K).This option does not affect BitTorrent downloads.Default: 0
If aria2 receives \"file not found\" status from the remote HTTP/FTPservers NUM times without getting a single byte, then force thedownload to fail. Specify 0 to disable this option. This optionsis effective only when using HTTP/FTP servers. The number of retryattempt is counted toward --max-tries, so it should beconfigured too.
aria2 does not split less than 2*SIZE byte range. For example,let's consider downloading 20MiB file. If SIZE is 10M, aria2 cansplit file into 2 range [0-10MiB) and [10MiB-20MiB) and download itusing 2 sources(if --split >= 2, of course). If SIZE is 15M,since 2*15M > 20MiB, aria2 does not split file and download it using1 source. You can append K or M (1K = 1024, 1M = 1024K).Possible Values: 1M -1024M Default: 20M
You cannot specify a file name for Metalink or BitTorrent downloads.The file name specified here is only used when the URIs fed to aria2are given on the command line directly, but not when using--input-file, --force-sequential option.
Specify piece selection algorithm used in HTTP/FTP download. Piecemeans fixed length segment which is downloaded in parallel insegmented download. If default is given, aria2 selects piece sothat it reduces the number of establishing connection. This isreasonable default behavior because establishing connection is anexpensive operation. If inorder is given, aria2 selects piecewhich has minimum index. Index=0 means first of the file. This willbe useful to view movie while downloading it.--enable-http-pipelining option maybe useful to reduce re-connection overhead. Please note that aria2honors--min-split-size option,so it will be necessary to specify a reasonable value to--min-split-size option.If random is given, aria2 selects piece randomly. Likeinorder, --min-split-size option is honored.If geom is given, at the beginning aria2 selects piece which hasminimum index like inorder, but it exponentially increasinglykeeps space from previously selected piece. This will reduce thenumber of establishing connection and at the same time it willdownload the beginning part of the file first. This will be usefulto view movie while downloading it.Default: default
Specify URI selection algorithm. The possible values are inorder,feedback and adaptive. If inorder is given, URI is tried inthe order appeared in the URI list. If feedback is given, aria2uses download speed observed in the previous downloads and choosefastest server in the URI list. This also effectively skips deadmirrors. The observed download speed is a part of performanceprofile of servers mentioned in --server-stat-of and--server-stat-if options. If adaptive is given, selects one ofthe best mirrors for the first and reserved connections. Forsupplementary ones, it returns mirrors which has not been testedyet, and if each of them has already been tested, returns mirrorswhich has to be tested again. Otherwise, it doesn't select anymoremirrors. Like feedback, it uses a performance profile of servers.Default: feedback
Use a proxy server for HTTPS. To override a previously defined proxy,use \"\".See also the --all-proxy option. This affects all https download.The format of PROXY is [http://][USER:PASSWORD@]HOST[:PORT]
Set an http referrer (Referer). This affects all http/https downloads.If * is given, the download URI is also used as the referrer.This may be useful when used together with the--parameterized-uri option.
Set file to download by specifying its index.You can find the file index using the --show-files option.Multiple indexes can be specified by using ,, for example: 3,6.You can also use - to specify a range: 1-5., and - can be used together: 1-5,8,9.When used with the -M option, index may vary depending on the query(see --metalink-* options).
In multi file torrent, the adjacent files specified by this option mayalso be downloaded. This is by design, not a bug.A single piece may include several files or part of files, and aria2writes the piece to the appropriate files.
Exclude seed only downloads when counting concurrent activedownloads (See -j option). This means that if -j3 isgiven and this option is turned on and 3 downloads are active andone of those enters seed mode, then it is excluded from activedownload count (thus it becomes 2), and the next download waiting inqueue gets started. But be aware that seeding item is stillrecognized as active download in RPC method. Default: false
Allow hook command invocation after hash check (see -Voption) in BitTorrent download. By default, when hash checksucceeds, the command given by --on-bt-download-completeis executed. To disable this action, give false to this option.Default: true
Specify the external IP address to use in BitTorrent download and DHT.It may be sent to BitTorrent tracker. For DHT, this option should beset to report that local node is downloading a particular torrent.This is critical to use DHT in a private network. Although thisfunction is named external, it can accept any kind of IPaddresses.
If true is given, after hash check using --check-integrity option andfile is complete, continue to seed file. If you want to check fileand download it only when it is damaged or incomplete, set thisoption to false. This option has effect only on BitTorrent download.Default: true
Before getting torrent metadata from DHT when downloading withmagnet link, first try to read file saved by--bt-save-metadata option. If it is successful, then skipdownloading metadata from DHT.Default: false
Try to download first and last pieces of each file first. This isuseful for previewing files. The argument can contain 2 keywords:head and tail. To include both keywords, they must be separatedby comma. These keywords can take one parameter, SIZE. For example,if head= is specified, pieces in the range of first SIZE bytesof each file get higher priority. tail= means the range oflast SIZE bytes of each file. SIZE can include K or M (1K = 1024,1M = 1024K). If SIZE is omitted, SIZE=1M is used. 59ce067264