今天,系统日志里显示 Too many open files,于是使用lsof命令查看文件句柄数。发现是网络连接太多(CLOSE_WAIT).
linux平台下:使用下面命令
lsof -nl|grep CLOSE_WAIT|awk '{print $9,$10}'|wc -l
发现 CLOSE_WAIT|太多
展示片段如下:
192.168.10.32:52727->192.168.10.31:webcache (CLOSE_WAIT)
192.168.10.32:52747->192.168.10.31:webcache (CLOSE_WAIT)
我这是处理逻辑问题,curl_easy_init 和curl_easy_cleanup没有成对调用,
因触发异常导致的。解决办法是封装一个类(实际上,这个非常类似于java中的try{}finally{}用法,但比JAVA的看起来舒服
class AutoDestory{
public:
//构造函数中加入curl_easy_init
//析构函数加入curl_easy_cleanup
}
但考虑到程序使用CURL进行HTTP连接,ftp连接等是一个常见问题,于是搜索CURL和CLOSE_WAIT相关资料,归纳下。
【1】官方bug讨论区:
http://curl.haxx.se/mail/lib-2013-11/0006.html
curl-library Mailing List Archives
Many CLOSE_WAIT when handling lots of URLs
[PATCH] Cleanup dead connections in CLOSE_WAIT state sooner
这个是个bug,但是这个说的是"multi"的接口,与”easy“接口无关,且这个出现在 7.29.0.版本,且现在已经修复
curl-library Mailing List Archives
Re: [ curl-Bugs-1358860 ] libcurl not closing sockets
这个是一个2005年的帖子。。。。
1) libcurl only supervises sockets between a request and its response.
libcurl reads, writes and "supervises" the sockets during the whole operation:
the request and the subsequent response.
If the server keeps the connection open, so will libcurl to be able to re-use
the connection if the application decides to do so.
When libcurl has completed a transfer and leaves the connection open, there is
no "supervision" of any sockets as libcurl has no magic threads or any another
means of doing that.
When you then later on call curl_easy_perform() again with a URL that is fit
for connection re-use it checks if the connection previously left alive is
still alive and if so it re-uses it.
> 2) if the server closes a socket because it has been idle for X amount of
> time, libcurl will not be supervising the socket, hence not detect the
> close.
Correct. It won't detect that until you try to re-use that connection again.
> 3) to make libcurl supervise the socket during this time (i.e. after
> receiving the request) would be a fundamental change to the way libcurl is
> currently designed. Perhaps even introducing a performance hit.
Yes. It would require a separate thread or similar that would detect when the
peer closes the connection.
这篇说明,CURL没有去侦测CLOSE_WAIT,除非单独需要一个独立线程
【2】sourceforge说明
http://sourceforge.net/p/curl/bugs/558/
#558 CLOSE_WAIT problem
How many threads? How many sockets are in CLOSE_WAIT?
libcurl keep sockets "open" by default after a
curl_easy_perform(), to be prepared to re-use the connection
on a subsequent call to it again. But if the call doesn't
come within a limited time, the remote server might decide
to close down the connection anyway and the socket ends up
in CLOSE_WAIT.
You can avoid this by forcing libcurl to close the sockets
after use, which then of course disables persistent
connections on subsequent calls.
Can you reproduce this problem with a small stand-alone app
that you can provide the source for?
这里说了CLOSE_WAIT产生的原因,curl_easy_perform()默认会暂时延长关闭连接,这是为了下次请求可以重用该连接。但如果client端在限定时间内并没有数据传输,远程服务器会主动关闭该连接,这时连接的SOCKET就变成CLOSE_WAIT状态了。
下面是一个较好的解释:(注,这个也是他/她转载别人的)
CLOSE_WAIT问题解决
http://blog.csdn.net/hwz119/article/details/1611229
http://blog.csdn.net/hwz119/article/details/1611229
解决办法:
【方法1】
【官方】
http://curl.haxx.se/libcurl/c/CURLOPT_FORBID_REUSE.html
但里面有提醒,这个可能会对性能有很大影响,另外,这个与curl_easy_cleanup函数没有关系。
【方法2】
每次CURL都重新curl_easy_initcurl用完都调用curl_easy_cleanup
相关推荐
官方离线安装包,测试可用。使用rpm -ivh [rpm完整包名] 进行安装
window8 64位环境下安装magento报错PHP extension curl must be loaded,替换php_curl.dll
curl-7.19.7-26.el6_2.4.i686.rpm
curl-curl-7_47_1.zip
在Centos6.5系统上,执行curl时,会报错,需要更新openssl,附件就是需要的更新包openssl-1.0.1e-16.el6_5.14.x86_64.rpm,错误信息为:curl: (35) error:100AE081:elliptic curve routines:EC_GROUP_new_by_curve_...
php_curl.dll
curl-7_24_0.tar.gz A command line tool and library for transferring data with URL syntax, supporting HTTP, HTTPS, FTP, FTPS, GOPHER, TFTP, SCP, SFTP, SMB, TELNET, DICT, LDAP, LDAPS, FILE, IMAP, SMTP, ...
eAccelerator.dll ...php_curl.dll php_dba.dll php_dbase.dll php_exif.dll php_fdf.dll php_gd2.dll php_gettext.dll php_gmp.dll ...
windows下php7.3官方自带的php_curl.dll好像有问题,增加扩展curl经常失败
可联网机器(包含内网yum),可以解压后直接执行命令 yum update libcurl-8.4.0-1.el7.1.x86_64.rpm curl-8.4.0-1.el7.1.x86_64.rpm
在win8、win8.1之中,wamp启动curl出现无法启动的情况,应该就是php_curl.dll出现了问题,这里准备了4个不同版本的php_curl.dll,请对应版本下载,亲测,有效,所以上传上来,方便大家使用。
离线安装包,亲测可用
curl_cookie_get.php
WWW-Curl-3_per_www_curl_源码.rar
在windows上开发,下的是php_curl-5.4.3-VC9-x64版的wamp,在使用curl模块时出现错误,明明已经加载了,但是还是报curl没有加载的错误,后来在网上找了下,发现该版本的php_curl.dll有问题,于是又下载了份,覆盖掉...
WWW-Curl-3_per_www_curl.zip
curl必备链接库之一,开发游戏具备的lib
freeswitch xml_curl模块使用 freeswitch 数据库生成xml文件,使用php生成代码,替换掉directory,ivr,dialplan,等各种本地配置
libcurl 完整的封装 支持对任意的curl读写