bash - Curl: don't wait for response -
this question has answer here:
i have shell script relies on curl command this:
curl --request post -u name:pass -h "content-type: application/json" --data "{data}" https://url.com --cacert ./my_crt
i don't need response of command, , command in big loop, waiting responses take lot of time.
so, there way in bash same thing, without waiting response?
if have large number of requests want issue quickly, , don't care output, there 2 things should do:
- do more requests same connection.
for small requests, it's faster 10 requests each on 1 connection, 1 request each on 10 connections. henry's http post test server, difference 2.5x:
$ time in {1..10}; curl -f foo=bar https://posttestserver.com/post.php ; done dumped 1 post variables. view @ http://www.posttestserver.com/data/2016/06/09/11.44.48536583865 post body 0 chars long. (...) real 0m2.429s
vs
$ time { array=(); in {1..10}; array+=(--next -f foo=bar https://posttestserver.com/post.php ) ; done; curl "${array[@]}"; } dumped 1 post variables. view @ http://www.posttestserver.com/data/2016/06/09/11.45.461371907842 (...) real 0m1.079s
- process @ n connections in parallel, avoid dos'ing host or machine
here sem
gnu parallel limiting number of parallel connections 4. better version of backgrounding , waiting, since ensure full capacity.
for in {1..20} sem -j 4 curl -f foo=bar https://posttestserver.com/post.php done sem --wait
the number of parallel requests want depends on how beefy host is. realistic number 32+
combine 2 strategies, , should see hefty speedup without dos'ing yourself.
Comments
Post a Comment