lkml.org 
[lkml]   [2013]   [Sep]   [30]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [RFC] extending splice for copy offloading
On 09/30/2013 08:02 PM, Myklebust, Trond wrote:
> On Mon, 2013-09-30 at 19:48 +0200, Bernd Schubert wrote:
>> On 09/30/2013 07:44 PM, Myklebust, Trond wrote:
>>> On Mon, 2013-09-30 at 19:17 +0200, Bernd Schubert wrote:
>>>> It would be nice if there would be way if the file system would get a
>>>> hint that the target file is supposed to be copy of another file. That
>>>> way distributed file systems could also create the target-file with the
>>>> correct meta-information (same storage targets as in-file has).
>>>> Well, if we cannot agree on that, file system with a custom protocol at
>>>> least can detect from 0 to SSIZE_MAX and then reset metadata. I'm not
>>>> sure if this would work for pNFS, though.
>>>
>>> splice() does not create new files. What you appear to be asking for
>>> lies way outside the scope of that system call interface.
>>>
>>
>> Sorry I know, definitely outside the scope of splice, but in the context
>> of offloaded file copies. So the question is, what is the best way to
>> address/discuss that?
>
> Why does it need to be addressed in the first place?

An offloaded copy is still not efficient if different storage
servers/targets used by from-file and to-file.

>
> What is preventing an application from retrieving and setting this
> information using standard libc functions such as fstat()+open(), and
> supplemented with libattr attr_setf/getf(), and libacl acl_get_fd/set_fd
> where appropriate?
>

At a minimum this requires network and metadata overhead. And while I'm
working on FhGFS now, I still wonder what other file system need to do -
for example Lustre pre-allocates storage-target files on creating a
file, so file layout changes mean even more overhead there.
Anyway, if we could agree on to use libattr or libacl to teach the file
system about the upcoming splice call I would be fine. Metadata overhead
is probably negligible for large files.




Thanks,
Bernd



\
 
 \ /
  Last update: 2013-09-30 21:01    [W:0.322 / U:0.080 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site