[关闭]
@yangfch3 2016-05-20T08:05:58.000000Z 字数 11761 阅读 3246

Curl

shell


curl 是什么?

curl 的存在有两种形式:一种是编程用的函数库,一种是命令行工具,作用是发出网络请求,支持多种协议,然后得到和提取数据,显示在"标准输出"(stdout)上面。


curl -h

  1. Usage: curl [options...] <url>
  2. Options: (H) means HTTP/HTTPS only, (F) means FTP only
  3. --anyauth Pick "any" authentication method (H)
  4. -a, --append Append to target file when uploading (F/SFTP)
  5. --basic Use HTTP Basic Authentication (H)
  6. --cacert FILE CA certificate to verify peer against (SSL)
  7. --capath DIR CA directory to verify peer against (SSL)
  8. -E, --cert CERT[:PASSWD] Client certificate file and password (SSL)
  9. --cert-status Verify the status of the server certificate (SSL)
  10. --cert-type TYPE Certificate file type (DER/PEM/ENG) (SSL)
  11. --ciphers LIST SSL ciphers to use (SSL)
  12. --compressed Request compressed response (using deflate or gzip)
  13. -K, --config FILE Read config from FILE
  14. --connect-timeout SECONDS Maximum time allowed for connection
  15. -C, --continue-at OFFSET Resumed transfer OFFSET
  16. -b, --cookie STRING/FILE Read cookies from STRING/FILE (H)
  17. -c, --cookie-jar FILE Write cookies to FILE after operation (H)
  18. --create-dirs Create necessary local directory hierarchy
  19. --crlf Convert LF to CRLF in upload
  20. --crlfile FILE Get a CRL list in PEM format from the given file
  21. -d, --data DATA HTTP POST data (H)
  22. --data-raw DATA HTTP POST data, '@' allowed (H)
  23. --data-ascii DATA HTTP POST ASCII data (H)
  24. --data-binary DATA HTTP POST binary data (H)
  25. --data-urlencode DATA HTTP POST data url encoded (H)
  26. --delegation STRING GSS-API delegation permission
  27. --digest Use HTTP Digest Authentication (H)
  28. --disable-eprt Inhibit using EPRT or LPRT (F)
  29. --disable-epsv Inhibit using EPSV (F)
  30. --dns-servers DNS server addrs to use: 1.1.1.1;2.2.2.2
  31. --dns-interface Interface to use for DNS requests
  32. --dns-ipv4-addr IPv4 address to use for DNS requests, dot notation
  33. --dns-ipv6-addr IPv6 address to use for DNS requests, dot notation
  34. -D, --dump-header FILE Write the headers to FILE
  35. --egd-file FILE EGD socket path for random data (SSL)
  36. --engine ENGINE Crypto engine (use "--engine list" for list) (SSL)
  37. -f, --fail Fail silently (no output at all) on HTTP errors (H)
  38. --false-start Enable TLS False Start.
  39. -F, --form CONTENT Specify HTTP multipart POST data (H)
  40. --form-string STRING Specify HTTP multipart POST data (H)
  41. --ftp-account DATA Account data string (F)
  42. --ftp-alternative-to-user COMMAND String to replace "USER [name]" (F)
  43. --ftp-create-dirs Create the remote dirs if not present (F)
  44. --ftp-method [MULTICWD/NOCWD/SINGLECWD] Control CWD usage (F)
  45. --ftp-pasv Use PASV/EPSV instead of PORT (F)
  46. -P, --ftp-port ADR Use PORT with given address instead of PASV (F)
  47. --ftp-skip-pasv-ip Skip the IP address for PASV (F)
  48. --ftp-pret Send PRET before PASV (for drftpd) (F)
  49. --ftp-ssl-ccc Send CCC after authenticating (F)
  50. --ftp-ssl-ccc-mode ACTIVE/PASSIVE Set CCC mode (F)
  51. --ftp-ssl-control Require SSL/TLS for FTP login, clear for transfer (F)
  52. -G, --get Send the -d data with a HTTP GET (H)
  53. -g, --globoff Disable URL sequences and ranges using {} and []
  54. -H, --header LINE Pass custom header LINE to server (H)
  55. -I, --head Show document info only
  56. -h, --help This help text
  57. --hostpubmd5 MD5 Hex-encoded MD5 string of the host public key. (SSH)
  58. -0, --http1.0 Use HTTP 1.0 (H)
  59. --http1.1 Use HTTP 1.1 (H)
  60. --http2 Use HTTP 2 (H)
  61. --ignore-content-length Ignore the HTTP Content-Length header
  62. -i, --include Include protocol headers in the output (H/F)
  63. -k, --insecure Allow connections to SSL sites without certs (H)
  64. --interface INTERFACE Use network INTERFACE (or address)
  65. -4, --ipv4 Resolve name to IPv4 address
  66. -6, --ipv6 Resolve name to IPv6 address
  67. -j, --junk-session-cookies Ignore session cookies read from file (H)
  68. --keepalive-time SECONDS Wait SECONDS between keepalive probes
  69. --key KEY Private key file name (SSL/SSH)
  70. --key-type TYPE Private key file type (DER/PEM/ENG) (SSL)
  71. --krb LEVEL Enable Kerberos with security LEVEL (F)
  72. --libcurl FILE Dump libcurl equivalent code of this command line
  73. --limit-rate RATE Limit transfer speed to RATE
  74. -l, --list-only List only mode (F/POP3)
  75. --local-port RANGE Force use of RANGE for local port numbers
  76. -L, --location Follow redirects (H)
  77. --location-trusted Like '--location', and send auth to other hosts (H)
  78. --login-options OPTIONS Server login options (IMAP, POP3, SMTP)
  79. -M, --manual Display the full manual
  80. --mail-from FROM Mail from this address (SMTP)
  81. --mail-rcpt TO Mail to this/these addresses (SMTP)
  82. --mail-auth AUTH Originator address of the original email (SMTP)
  83. --max-filesize BYTES Maximum file size to download (H/F)
  84. --max-redirs NUM Maximum number of redirects allowed (H)
  85. -m, --max-time SECONDS Maximum time allowed for the transfer
  86. --metalink Process given URLs as metalink XML file
  87. --negotiate Use HTTP Negotiate (SPNEGO) authentication (H)
  88. -n, --netrc Must read .netrc for user name and password
  89. --netrc-optional Use either .netrc or URL; overrides -n
  90. --netrc-file FILE Specify FILE for netrc
  91. -:, --next Allows the following URL to use a separate set of options
  92. --no-alpn Disable the ALPN TLS extension (H)
  93. -N, --no-buffer Disable buffering of the output stream
  94. --no-keepalive Disable keepalive use on the connection
  95. --no-npn Disable the NPN TLS extension (H)
  96. --no-sessionid Disable SSL session-ID reusing (SSL)
  97. --noproxy List of hosts which do not use proxy
  98. --ntlm Use HTTP NTLM authentication (H)
  99. --oauth2-bearer TOKEN OAuth 2 Bearer Token (IMAP, POP3, SMTP)
  100. -o, --output FILE Write to FILE instead of stdout
  101. --pass PASS Pass phrase for the private key (SSL/SSH)
  102. --path-as-is Do not squash .. sequences in URL path
  103. --pinnedpubkey FILE/HASHES Public key to verify peer against (SSL)
  104. --post301 Do not switch to GET after following a 301 redirect (H)
  105. --post302 Do not switch to GET after following a 302 redirect (H)
  106. --post303 Do not switch to GET after following a 303 redirect (H)
  107. -#, --progress-bar Display transfer progress as a progress bar
  108. --proto PROTOCOLS Enable/disable PROTOCOLS
  109. --proto-default PROTOCOL Use PROTOCOL for any URL missing a scheme
  110. --proto-redir PROTOCOLS Enable/disable PROTOCOLS on redirect
  111. -x, --proxy [PROTOCOL://]HOST[:PORT] Use proxy on given port
  112. --proxy-anyauth Pick "any" proxy authentication method (H)
  113. --proxy-basic Use Basic authentication on the proxy (H)
  114. --proxy-digest Use Digest authentication on the proxy (H)
  115. --proxy-negotiate Use HTTP Negotiate (SPNEGO) authentication on the proxy (H)
  116. --proxy-ntlm Use NTLM authentication on the proxy (H)
  117. --proxy-service-name NAME SPNEGO proxy service name
  118. --service-name NAME SPNEGO service name
  119. -U, --proxy-user USER[:PASSWORD] Proxy user and password
  120. --proxy1.0 HOST[:PORT] Use HTTP/1.0 proxy on given port
  121. -p, --proxytunnel Operate through a HTTP proxy tunnel (using CONNECT)
  122. --pubkey KEY Public key file name (SSH)
  123. -Q, --quote CMD Send command(s) to server before transfer (F/SFTP)
  124. --random-file FILE File for reading random data from (SSL)
  125. -r, --range RANGE Retrieve only the bytes within RANGE
  126. --raw Do HTTP "raw"; no transfer decoding (H)
  127. -e, --referer Referer URL (H)
  128. -J, --remote-header-name Use the header-provided filename (H)
  129. -O, --remote-name Write output to a file named as the remote file
  130. --remote-name-all Use the remote file name for all URLs
  131. -R, --remote-time Set the remote file's time on the local output
  132. -X, --request COMMAND Specify request command to use
  133. --resolve HOST:PORT:ADDRESS Force resolve of HOST:PORT to ADDRESS
  134. --retry NUM Retry request NUM times if transient problems occur
  135. --retry-delay SECONDS Wait SECONDS between retries
  136. --retry-max-time SECONDS Retry only within this period
  137. --sasl-ir Enable initial response in SASL authentication
  138. -S, --show-error Show error. With -s, make curl show errors when they occur
  139. -s, --silent Silent mode (don't output anything)
  140. --socks4 HOST[:PORT] SOCKS4 proxy on given host + port
  141. --socks4a HOST[:PORT] SOCKS4a proxy on given host + port
  142. --socks5 HOST[:PORT] SOCKS5 proxy on given host + port
  143. --socks5-hostname HOST[:PORT] SOCKS5 proxy, pass host name to proxy
  144. --socks5-gssapi-service NAME SOCKS5 proxy service name for GSS-API
  145. --socks5-gssapi-nec Compatibility with NEC SOCKS5 server
  146. -Y, --speed-limit RATE Stop transfers below RATE for 'speed-time' secs
  147. -y, --speed-time SECONDS Trigger 'speed-limit' abort after SECONDS (default: 30)
  148. --ssl Try SSL/TLS (FTP, IMAP, POP3, SMTP)
  149. --ssl-reqd Require SSL/TLS (FTP, IMAP, POP3, SMTP)
  150. -2, --sslv2 Use SSLv2 (SSL)
  151. -3, --sslv3 Use SSLv3 (SSL)
  152. --ssl-allow-beast Allow security flaw to improve interop (SSL)
  153. --ssl-no-revoke Disable cert revocation checks (WinSSL)
  154. --stderr FILE Where to redirect stderr (use "-" for stdout)
  155. --tcp-nodelay Use the TCP_NODELAY option
  156. -t, --telnet-option OPT=VAL Set telnet option
  157. --tftp-blksize VALUE Set TFTP BLKSIZE option (must be >512)
  158. -z, --time-cond TIME Transfer based on a time condition
  159. -1, --tlsv1 Use >= TLSv1 (SSL)
  160. --tlsv1.0 Use TLSv1.0 (SSL)
  161. --tlsv1.1 Use TLSv1.1 (SSL)
  162. --tlsv1.2 Use TLSv1.2 (SSL)
  163. --trace FILE Write a debug trace to FILE
  164. --trace-ascii FILE Like --trace, but without hex output
  165. --trace-time Add time stamps to trace/verbose output
  166. --tr-encoding Request compressed transfer encoding (H)
  167. -T, --upload-file FILE Transfer FILE to destination
  168. --url URL URL to work with
  169. -B, --use-ascii Use ASCII/text transfer
  170. -u, --user USER[:PASSWORD] Server user and password
  171. --tlsuser USER TLS username
  172. --tlspassword STRING TLS password
  173. --tlsauthtype STRING TLS authentication type (default: SRP)
  174. --unix-socket FILE Connect through this Unix domain socket
  175. -A, --user-agent STRING Send User-Agent STRING to server (H)
  176. -v, --verbose Make the operation more talkative
  177. -V, --version Show version number and quit
  178. -w, --write-out FORMAT Use output FORMAT after completion
  179. --xattr Store metadata in extended file attributes
  180. -q Disable .curlrc (must be first parameter)

curl url

直接在 curl 命令后加上网址,就可以看到网页源码。以网址 www.sina.com 为例:

  1. $ curl www.sina.com
  1. <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
  2. <html>
  3. <head>
  4. <title>301 Moved Permanently</title>
  5. </head>
  6. <body>
  7. <h1>Moved Permanently</h1>
  8. <p>The document has moved
  9. <a href="http://www.sina.com.cn/">here</a>
  10. </p>
  11. </body>
  12. </html>

curl -o

使用 -o 参数需要,将返回信息转存至文本文件(新建或现有文件)。

  1. $ curl -o index.html www.sina.com

curl -L

当访问的网址有重定向等跳转机制时,使用 -L 参数可以实现自动跳转。

  1. $ curl -L www.sina.com

会自动跳转到 www.sina.com.cn。


curl -i(I)

-i 参数可以显示 http response 的头信息,连同网页代码一起。

  1. $ curl -i www.sina.com
  1. HTTP/1.0 301 Moved Permanently
  2. Date: Sat, 03 Sep 2011 23:44:10 GMT
  3. Server: Apache/2.0.54 (Unix)
  4. Location: http://www.sina.com.cn/
  5. Cache-Control: max-age=3600
  6. Expires: Sun, 04 Sep 2011 00:44:10 GMT
  7. Vary: Accept-Encoding
  8. Content-Length: 231
  9. Content-Type: text/html; charset=iso-8859-1
  10. Age: 3239
  11. X-Cache: HIT from sh201-9.sina.com.cn
  12. Connection: close
  13. <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
  14. <html><head>
  15. <title>301 Moved Permanently</title>
  16. </head><body>
  17. <h1>Moved Permanently</h1>
  18. <p>The document has moved <a href="http://www.sina.com.cn/">here</a>
  19. </p>
  20. </body></html>

-I 参数则是只显示 http response 的头信息,而不显示网页源码。


curl -v

-v 参数可以显示一次 http 通信的整个过程,包括端口连接和 http request 头信息。

  1. $ curl -v www.sina.com
  1.   * About to connect() to www.sina.com port 80 (#0)
  2.   * Trying 61.172.201.195... connected
  3.   * Connected to www.sina.com (61.172.201.195) port 80 (#0)
  4.   > GET / HTTP/1.1
  5.   > User-Agent: curl/7.21.3 (i686-pc-linux-gnu) libcurl/7.21.3 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.18
  6.   > Host: www.sina.com
  7.   > Accept: */*
  8.   >
  9.   * HTTP 1.0, assume close after body
  10.   < HTTP/1.0 301 Moved Permanently
  11.   < Date: Sun, 04 Sep 2011 00:42:39 GMT
  12.   < Server: Apache/2.0.54 (Unix)
  13.   < Location: http://www.sina.com.cn/
  14.   < Cache-Control: max-age=3600
  15.   < Expires: Sun, 04 Sep 2011 01:42:39 GMT
  16.   < Vary: Accept-Encoding
  17.   < Content-Length: 231
  18.   < Content-Type: text/html; charset=iso-8859-1
  19.   < X-Cache: MISS from sh201-19.sina.com.cn
  20.   < Connection: close
  21.   <
  22.   <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
  23.   <html><head>
  24.   <title>301 Moved Permanently</title>
  25.   </head><body>
  26.   <h1>Moved Permanently</h1>
  27.   <p>The document has moved <a href="http://www.sina.com.cn/">here</a>.</p>
  28.   </body></html>
  29.   * Closing connection #0

curl --trace(--trace-ascii)

如果你觉得上面 curl -v 的信息还不够,那么下面的命令可以查看更详细的通信过程。

  1.   $ curl --trace output.txt www.sina.com

或者

  1.   $ curl --trace-ascii output.txt www.sina.com

运行后,请打开 output.txt 文件查看。


发送表单信息

发送表单信息有 GETPOST 两种方法。GET 方法相对简单,只要把数据附在网址后面就行。

  1. $ curl example.com/form.cgi?data=xxx

POST 方法必须把数据和网址分开,curl 就要用到 -d--data 参数。

  1. $ curl -X POST --data "data=xxx" example.com/form.cgi

如果你的数据没有经过表单编码,还可以让 curl 为你编码,参数是 --data-urlencode

  1. $ curl -X POST--data-urlencode "date=April 1" example.com/form.cgi

HTTP 动词

curl 默认的 HTTP 动词是 GET,使用 -X 参数可以支持其他动词。

  1. $ curl -X POST www.example.com
  1. $ curl -X DELETE www.example.com

文件上传

假定文件上传的表单是下面这样:

  1. <form method="POST" enctype='multipart/form-data' action="upload.cgi">
  2. <input type=file name=upload>
  3. <input type=submit name=press value="OK">
  4. </form>

你可以用 curl 这样上传文件:

  1. $ curl --form upload=@localfilename --form press=OK upload.cgi

Referer 字段

有时你需要在 http request 头信息中,提供一个 referer 字段,表示你是从哪里跳转过来的。

  1. $ curl --referer http://www.example1.com http://www.example2.com

User Agent字段

这个字段是用来表示客户端的设备信息。服务器有时会根据这个字段,针对不同设备,返回不同格式的网页,比如手机版和桌面版。

iPhone4User Agent

  1. Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_0 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8A293 Safari/6531.22.7

curl 可以这样模拟:

  1. $ curl --user-agent "[User Agent]" [URL]

使用 --cookie 参数,可以让 curl 发送 cookie

  1. $ curl --cookie "name=xxx" www.example.com

至于具体的 cookie 的值,可以从 http response 头信息的 Set-Cookie 字段中得到。

-c cookie-file 可以保存服务器返回的 cookie 到文件;
-b cookie-file 可以使用这个文件作为 cookie 信息,进行后续的请求。

  1. $ curl -c cookies http://example.com
  2. $ curl -b cookies http://example.com

增加头信息

有时需要在 http request 之中,自行增加一个头信息。--header 参数就可以起到这个作用。

  1. $ curl --header "Content-Type:application/json" http://example.com

HTTP 认证

有些网域需要 HTTP 认证,这时 curl 需要用到--user参数。

  1. $ curl --user name:password example.com
添加新批注
在作者公开此批注前,只有你和作者可见。
回复批注