[lkml]   [2006]   [Aug]   [12]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: [RFC][PATCH 0/9] Network receive deadlock prevention for NBD
    On Sat, Aug 12, 2006 at 12:18:07PM +0200, Peter Zijlstra ( wrote:
    > > Does it? I though it is possible to only have 64k of working sockets per
    > > device in TCP.
    > 65535 sockets * 128 packets * 16384 bytes/packet =
    > 1^16 * 1^7 * 1^14 = 1^(16+7+14) = 1^37 = 128G of memory per IP
    > And systems with a lot of IP numbers are not unthinkable.
    > I wonder what kind of system you have to feel that that is not a
    > problem. (I'm not sure on the 128 packets per socket, and the 16k per
    > packet is considering jumbo frames without scather gather receive)

    :) have you ever tried to get those numbers in practice?

    There are couple of problems with that:
    1. You only have 1gbit/sec link (someone has 10gbit/sec).
    2. Depending on packet size those limit can be lower (and it _is_ lower
    for the real-world traffic where you need to send ACKs).
    3. If data transferred is supposed to be processed somehow, you need to
    add those cost too.
    4. When we are talking about OOM conditions, they whole system is slowed
    down hard, so you again can not achieve those limit.

    And there are a lot of other factors which can slow down you network
    (like netfilter, ipsec and others).

    > > If system is limited enough to provide enough memory for network tree
    > > allocator, it is possible to create it's own drop condition inside NTA,
    > > but it must be saparated from the weakest chain element in that
    > > conditions - SLAB OOM.
    > Hence the alternative allocator to use on tight memory conditions.

    You seems to not understand the situation.
    You want to make a progress in SLAB OOM condition which is bound to
    network and since progress requires allocation, it can deadlock.
    That is the problem - SLAB allocator got OOM condition.
    The only solution for that (expect more memory addition) is to make
    underlying system robust (in discussed case it is network). Robust means
    not depend on condition which leads to SLAB OOM.
    When network uses the same allocator, it depends on it, and thus it is
    possible to have (cut by you) a situation when reserve (which depends on
    SLAB and it's OOM too) is not filled or even does not exist.
    Underlying system can have it's own problems, millions of problems, but
    it does not matter when we are talking about higher-layer conditions -
    forward progress in this scenario does not depend on the weakest chain
    element in described conditions - it does not depend on SLAB OOM.

    If transferred to your implementation, then just steal some pages from
    SLAB when new network device is added and use them when OOM happens.
    It is much simpler and can help in the most of situations.

    Evgeniy Polyakov
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to
    More majordomo info at
    Please read the FAQ at

     \ /
      Last update: 2006-08-12 12:45    [W:0.024 / U:301.916 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site