- 論壇徽章:
- 0
|
相信是由於大量的信息經(jīng)由UDP發(fā)送造成的
Under heavy write loads, the Linux NFS client can generate many large UDP datagrams. This can quickly exhaust output socket buffer space on the client. If this occurs many times in a short time, the client sends the server a large number of fragments, but almost never gets a whole datagram's worth of fragments to the server. This fills the server's IP reassembly queue, causing it to become unreachable via UDP until it expels the useless fragments from the queue.
Note that the same thing can occur on servers that are under a heavy read load. If the server's output socket buffers are too small, large reads will cause them to overflow during IP fragmentation. The client's IP reassembly queue then fills with worthless fragments, and little UDP traffic can get to the client.
Here are some symptoms of this problem:
You use NFS over UDP with a large wsize (relative to the network's MTU), and your application workload is write-intensive, or with a large rsize with a read-intensive application.
You may see many fragmentation errors on your server or clients (netstat -s will tell the story).
Your server may periodically become very slow or unreachable.
Increasing the number of threads on your server has no effect on performance.
One or a small number of clients seem to make the server unusable.
The network path between your client and server may have a router or switch with small port buffers, or the path may contain links that run at different speeds (100Mb/s and GbE).
The fix is to make the Linux's IP fragmentation logic continue fragmenting a datagram even when output socket buffer space is over its limit. This fix appears in kernels newer than 2.4.20. You can work around this problem in one of several ways:
Use NFS over TCP. TCP does not use fragmentation, so it does not suffer from this problem. Using TCP may not be possible with older Linux NFS clients and servers that only support NFS over UDP.
If you can't use NFS over TCP, upgrade your clients to 2.4.20 or later.
If you can't upgrade your clients, increase the default size of your client's socket buffers (see below). 2.4.20 and later kernels do this automatically for the NFS client's socket buffers. See Section 5.3ff of the NFS How-To for more information.
If your rsize or wsize is very large, reduce it. This will reduce the load on your client's and server's output socket buffers.
Reduce network congestion by ensuring your GbE links use full flow control, that your switch and router ports use adequate buffer sizes, and that all links are negotiating their fastest settings. |
|
免費注冊
發(fā)表于 2006-03-31 09:52
