[lkml]   [1997]   [Nov]   [9]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    Subjectre: vfile_t (last hint)

    For the off_t msb flag hack, in the header:

    #if ((~0UL) == 0xffffffff) /* true on 32-bit long */
    /* not ANSI, signed might be big enough*/
    typedef unsigned long long voff_t;
    #define OPENFL 0x7fffffff /* VSEG.offset msb mask */
    typedef unsigned long voff_t; /* ditto on unsigned/signed question */
    #define OPENFL 0x7fffffffffffffff;

    struct filesegment {
    char * vfseg; /* filename, prefix + 'digit', use snprintf() */
    off_t offset;

    typedef struct filesegment VSEG;

    /* in vfinit(vfile_t vfile, char * vfprefix, flags, mode_t mode),
    initialize VSEG[index].vfseg to filenames, if VSEG[0].vfseg
    exists and has stat.st_size > 0, get VSEG[index].offset by
    calling fstat() or stat(). If VSEG[index].vfseg does not exist or
    has size 0, set
    VSEG[index].offset = OPENFL;

    keeping metadata in a section at the front of the first filesegment
    is a design issue anyone doing this may wish to consider; perhaps
    something like Clog might help with heterogenous endianness problems
    in file formats.

    when doing a vfread() or vfwrite(),
    if ((VSEG[index].offset & OPENFL) != 0), file is not open.
    if ((VSEG[index].offset == OPENFL), file has no data.

    You don't have very many filesegments, so leave any that you open()
    opened, and fsync() and close() them all at once with vfclose(). Tune
    this to the app and how many files it keeps open vs NR_OPEN. You only
    need one vfile_t struct per app instance probably, so you can add
    a vfnr_open member to a vfile_t struct if you find yourself overrunning
    limits on this. etc etc.

    Now if I really have and need 512gb or more of storage, why am I using a
    32-bit pc and need this hack instead of using an as400 or something
    comparably tuned for big filesystem database access? Realtime data
    acquisition, maybe, but this hack might have too much latency for that.

    Hence, I'm not going to write all these virtual big file interface
    functions to get around 32-bit type sizes. This code is public domain,
    if you need it, use it anyway you want.

    Regards, Clayton Weaver (Seattle)

     \ /
      Last update: 2005-03-22 13:40    [W:0.019 / U:94.320 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site