lkml.org 
[lkml]   [2013]   [Sep]   [30]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [RFC] extending splice for copy offloading
    On 09/30/2013 08:02 PM, Myklebust, Trond wrote:
    > On Mon, 2013-09-30 at 19:48 +0200, Bernd Schubert wrote:
    >> On 09/30/2013 07:44 PM, Myklebust, Trond wrote:
    >>> On Mon, 2013-09-30 at 19:17 +0200, Bernd Schubert wrote:
    >>>> It would be nice if there would be way if the file system would get a
    >>>> hint that the target file is supposed to be copy of another file. That
    >>>> way distributed file systems could also create the target-file with the
    >>>> correct meta-information (same storage targets as in-file has).
    >>>> Well, if we cannot agree on that, file system with a custom protocol at
    >>>> least can detect from 0 to SSIZE_MAX and then reset metadata. I'm not
    >>>> sure if this would work for pNFS, though.
    >>>
    >>> splice() does not create new files. What you appear to be asking for
    >>> lies way outside the scope of that system call interface.
    >>>
    >>
    >> Sorry I know, definitely outside the scope of splice, but in the context
    >> of offloaded file copies. So the question is, what is the best way to
    >> address/discuss that?
    >
    > Why does it need to be addressed in the first place?

    An offloaded copy is still not efficient if different storage
    servers/targets used by from-file and to-file.

    >
    > What is preventing an application from retrieving and setting this
    > information using standard libc functions such as fstat()+open(), and
    > supplemented with libattr attr_setf/getf(), and libacl acl_get_fd/set_fd
    > where appropriate?
    >

    At a minimum this requires network and metadata overhead. And while I'm
    working on FhGFS now, I still wonder what other file system need to do -
    for example Lustre pre-allocates storage-target files on creating a
    file, so file layout changes mean even more overhead there.
    Anyway, if we could agree on to use libattr or libacl to teach the file
    system about the upcoming splice call I would be fine. Metadata overhead
    is probably negligible for large files.




    Thanks,
    Bernd



    \
     
     \ /
      Last update: 2013-09-30 21:01    [W:7.624 / U:0.268 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site