tx258/lammps-gran-kokkos
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add a timeout option to the geturl command and set a default of 5 mins

Browse Source 
the library default is to wait forever. Also the limit for connecting
was reduced to 30 seconds from 300.
This commit is contained in:
Axel Kohlmeyer
2025-07-07 11:24:14 -04:00
parent 25556a1bc1
commit 1dc8c2d9a2
2 changed files with 27 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
doc/src/geturl.rst
View File

@ -12,13 +12,14 @@ Syntax
* url = URL of the file to download
* zero or more keyword argument pairs may be provided
* keyword = *output* or *verify* or *overwrite* or *verbose*
* keyword = *output* or *overwrite* or *timeout* or *verify* or *verbose*
.. parsed-literal::
*output* filename = write to *filename* instead of inferring the name from the URL
*verify* yes/no = verify SSL certificate and hostname if *yes*, do not if *no*
*overwrite* yes/no = if *yes* overwrite the output file in case it exists, do not if *no*
*timeout* time = stop download if not completed within given time in seconds
*verify* yes/no = verify SSL certificate and hostname if *yes*, do not if *no*
*verbose* yes/no = if *yes* write verbose debug output from libcurl to screen, do not if *no*
Examples
@ -42,15 +43,20 @@ large variety of protocols including "http", "https", "ftp", "scp",
The *output* keyword can be used to set the filename. By default, the last part
of the URL is used.
The *overwrite* keyword determines whether a file should be overwritten if it
already exists. If the argument is *no*, then the download will be skipped
if the file exists.
The *timeout* keyword can be used to modify the timeout for downloads in
seconds. After the timeout, the download will stop, even if incomplete.
The default time value is 300, i.e. 5 minutes. Setting the timeout to 0
means to wait forever.
The *verify* keyword determines whether ``libcurl`` will validate the
SSL certificate and hostname for encrypted connections. Turning this
off may be required when using a proxy or connecting to a server with a
self-signed SSL certificate.
The *overwrite* keyword determines whether a file should be overwritten if it
already exists. If the argument is *no*, then the download will be skipped
if the file exists.
The *verbose* keyword determines whether a detailed protocol of the steps
performed by libcurl is written to the screen. Using the argument *yes*
can be used to debug connection issues when the *geturl* command does not
@ -105,4 +111,4 @@ Related commands
Default
"""""""
*verify* = yes, *overwrite* = yes
*overwrite* = yes, *timeout* = 300, *verify* = yes, *verbose* = no

19
src/EXTRA-COMMAND/geturl.cpp
View File

@ -37,6 +37,7 @@ void GetURL::command(int narg, char **arg)
int verify = 1;
int overwrite = 1;
int verbose = 0;
int timeout = 300;
// process arguments
@ -45,10 +46,10 @@ void GetURL::command(int narg, char **arg)
// sanity check
if ((url.find(':') == std::string::npos) || (url.find('/') == std::string::npos))
error->all(FLERR, "URL '{}' is not a supported URL", url);
error->all(FLERR, Error::ARGZERO, "URL '{}' is not a supported URL", url);
std::string output = url.substr(url.find_last_of('/') + 1);
if (output.empty()) error->all(FLERR, "URL '{}' must end in a file string", url);
if (output.empty()) error->all(FLERR, Error::ARGZERO, "URL '{}' must end in a file string", url);
int iarg = 1;
while (iarg < narg) {
@ -60,6 +61,11 @@ void GetURL::command(int narg, char **arg)
if (iarg + 2 > narg) utils::missing_cmd_args(FLERR, "geturl overwrite", error);
overwrite = utils::logical(FLERR, arg[iarg + 1], false, lmp);
++iarg;
} else if (strcmp(arg[iarg], "timeout") == 0) {
if (iarg + 2 > narg) utils::missing_cmd_args(FLERR, "geturl timeout", error);
timeout = utils::inumeric(FLERR, arg[iarg + 1], false, lmp);
if (timeout < 0) error->all(FLERR, iarg + 1, "Invalid timeout {} for geturl", timeout);
++iarg;
} else if (strcmp(arg[iarg], "verify") == 0) {
if (iarg + 2 > narg) utils::missing_cmd_args(FLERR, "geturl verify", error);
verify = utils::logical(FLERR, arg[iarg + 1], false, lmp);
@ -69,7 +75,7 @@ void GetURL::command(int narg, char **arg)
verbose = utils::logical(FLERR, arg[iarg + 1], false, lmp);
++iarg;
} else {
error->all(FLERR, "Unknown geturl keyword: {}", arg[iarg]);
error->all(FLERR, iarg, "Unknown geturl keyword: {}", arg[iarg]);
}
++iarg;
}
@ -96,6 +102,9 @@ void GetURL::command(int narg, char **arg)
(void) curl_easy_setopt(curl, CURLOPT_WRITEDATA, (void *) out);
(void) curl_easy_setopt(curl, CURLOPT_FILETIME, 1L);
(void) curl_easy_setopt(curl, CURLOPT_FAILONERROR, 1L);
(void) curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 30L);
(void) curl_easy_setopt(curl, CURLOPT_TIMEOUT, timeout);
if (verbose && screen) {
(void) curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
(void) curl_easy_setopt(curl, CURLOPT_STDERR, (void *) screen);
@ -108,8 +117,8 @@ void GetURL::command(int narg, char **arg)
if (res != CURLE_OK) {
long response = 0L;
curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &response);
error->one(FLERR, "Download of {} failed with: {} {}", output, curl_easy_strerror(res),
response);
error->one(FLERR, Error::NOLASTLINE, "Download of {} failed with: {} {}", output,
curl_easy_strerror(res), response);
}
curl_easy_cleanup(curl);
}