lkml.org 
[lkml]   [2020]   [May]   [31]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: [ANNOUNCE] Reiser5: Data Tiering. Burst Buffers. Speedup synchronous modifications
From
Date
On 05/30/2020 12:13 PM, Pavel Machek wrote:
> Hi!
>
>
>> For example, you can use proxy device to store hot data only. With
>> such strategy new logical blocks (which are always "cold") will always
>> go to the main storage (in contrast with Burst Buffers, where new
>> logical blocks first get written to the proxy disk). Once in a while
>> you need to scan your volume in order to push colder data out, and
>> pull hotter data in the proxy disk. Reiser5 contains a common
>> interface for this. It is possible to maintain per-file, or even per-
>> blocks-extent "temperature" of data (e.g. as a generation counter),
>
> Would it be possible to offer userland interface for this? I can
> probably say that mp3/video files should be cold, while some source
> files should be hot, etc...
>
> Best regards,
> Pavel
>

Hi Pavel,

Yes, it is possible. One just needs to add an ioctl handler for regular
files managed by a plugin with STRIPED_FILE_PLUGIN_ID. That handler is
to set user-defined "temperature" to a file.

Also we'll need an additional on-disk file attribute (32 (or 64?)-bit
field in the private part of inode) to store the "temperature" in. It
can be added by standard way via implementation of respective stat-data
extension in the file reiser4/plugin/item/static_stat.c

Finally, we'll need to handle temperature in the common migration
procedure balance_volume_asym(), which is responsible for clearing up
the proxy device. It should look like this:

...

if (!IS_ERR(inode) && inode_file_plugin(inode)->balance &&
file_is_cold_enough(inode)) {
reiser4_iget_complete(inode);
/*
* migrate data blocks of this file
*/
...

Currently it works as if all files are "cold" (i.e. migrates
everything).

Once I find the current stuff more-or-less stable I'll add temperature
support and send the patch.

Thanks,
Edward.

\
 
 \ /
  Last update: 2020-05-31 20:35    [W:0.043 / U:26.984 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site