Messages in this thread | | | Date | Mon, 30 Sep 2013 20:49:43 +0200 | From | Bernd Schubert <> | Subject | Re: [RFC] extending splice for copy offloading |
| |
On 09/30/2013 08:02 PM, Myklebust, Trond wrote: > On Mon, 2013-09-30 at 19:48 +0200, Bernd Schubert wrote: >> On 09/30/2013 07:44 PM, Myklebust, Trond wrote: >>> On Mon, 2013-09-30 at 19:17 +0200, Bernd Schubert wrote: >>>> It would be nice if there would be way if the file system would get a >>>> hint that the target file is supposed to be copy of another file. That >>>> way distributed file systems could also create the target-file with the >>>> correct meta-information (same storage targets as in-file has). >>>> Well, if we cannot agree on that, file system with a custom protocol at >>>> least can detect from 0 to SSIZE_MAX and then reset metadata. I'm not >>>> sure if this would work for pNFS, though. >>> >>> splice() does not create new files. What you appear to be asking for >>> lies way outside the scope of that system call interface. >>> >> >> Sorry I know, definitely outside the scope of splice, but in the context >> of offloaded file copies. So the question is, what is the best way to >> address/discuss that? > > Why does it need to be addressed in the first place?
An offloaded copy is still not efficient if different storage servers/targets used by from-file and to-file.
> > What is preventing an application from retrieving and setting this > information using standard libc functions such as fstat()+open(), and > supplemented with libattr attr_setf/getf(), and libacl acl_get_fd/set_fd > where appropriate? >
At a minimum this requires network and metadata overhead. And while I'm working on FhGFS now, I still wonder what other file system need to do - for example Lustre pre-allocates storage-target files on creating a file, so file layout changes mean even more overhead there. Anyway, if we could agree on to use libattr or libacl to teach the file system about the upcoming splice call I would be fine. Metadata overhead is probably negligible for large files.
Thanks, Bernd
| |