lkml.org 
[lkml]   [2002]   [Oct]   [10]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    Date
    SubjectRe: [PATCH] O_STREAMING - flag for optimal streaming I/O
    On Wed, 9 Oct 2002, Mark Mielke wrote:

    > With this in mind, dynamic detection becomes a lot easier and less
    > error prone. The simplest way of detecting a file that would benefit
    > from having pages dropped is to keep a flag that indicates whether the
    > file was *ever* read non-sequentially. If a file was never read
    > non-sequentially, pages that are not at the beginning of the file, and
    > pages that are earlier than all seek points for the file, or that are
    > many pages later than all earlier seek points for the file may be
    > safely dropped. In fact, I am surprised this is not implemented
    > already. :-)

    there is a serious problem with this heristic, if you have a large video
    file that you one time skip into the center of, or rewind slightly that
    fill will then be flagged as not being suitable for O_STREAMING.

    it doesn't matter how other programs have used this file in the past, what
    matters is how this file is being used now.

    in your case of two webserver processes trying to access the same file
    both with O_STRAMING, let them, the first process starts reading things
    and flags the old blocks as 'discard immediatly' (would it possibly be
    worth adding another FIFO list to the VM to contain 'these pages can be
    thrown out of memory instantly with no additional work needed' pages??, or
    since I think there is one for freeable cache make a second for 'low
    priority' freeable pages such that you throw away all 'low priority' pages
    before you touch the normal freeable pages), the second process would then
    start reading the same file and the readahead for it would attempt to read
    the file in, and the system should be able to notice that these pages are
    already in memory (in the low priority freeable list) and rescue them up
    to the normal list.

    no need to have special code to try and decide if the various pointers are
    'close enough' togeather, the readahead size does that for you (and since
    readahead can be tuned as memory preasure grows this means that the
    definition of 'close enough' will change dynamicly)

    David Lang
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2005-03-22 13:29    [W:4.091 / U:0.208 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site