Messages in this thread | | | Subject | Re: [ANNOUNCE] Reiser5: Data Tiering. Burst Buffers. Speedup synchronous modifications | From | Edward Shishkin <> | Date | Sun, 31 May 2020 20:34:25 +0200 |
| |
On 05/30/2020 12:13 PM, Pavel Machek wrote: > Hi! > > >> For example, you can use proxy device to store hot data only. With >> such strategy new logical blocks (which are always "cold") will always >> go to the main storage (in contrast with Burst Buffers, where new >> logical blocks first get written to the proxy disk). Once in a while >> you need to scan your volume in order to push colder data out, and >> pull hotter data in the proxy disk. Reiser5 contains a common >> interface for this. It is possible to maintain per-file, or even per- >> blocks-extent "temperature" of data (e.g. as a generation counter), > > Would it be possible to offer userland interface for this? I can > probably say that mp3/video files should be cold, while some source > files should be hot, etc... > > Best regards, > Pavel >
Hi Pavel,
Yes, it is possible. One just needs to add an ioctl handler for regular files managed by a plugin with STRIPED_FILE_PLUGIN_ID. That handler is to set user-defined "temperature" to a file.
Also we'll need an additional on-disk file attribute (32 (or 64?)-bit field in the private part of inode) to store the "temperature" in. It can be added by standard way via implementation of respective stat-data extension in the file reiser4/plugin/item/static_stat.c
Finally, we'll need to handle temperature in the common migration procedure balance_volume_asym(), which is responsible for clearing up the proxy device. It should look like this:
...
if (!IS_ERR(inode) && inode_file_plugin(inode)->balance && file_is_cold_enough(inode)) { reiser4_iget_complete(inode); /* * migrate data blocks of this file */ ...
Currently it works as if all files are "cold" (i.e. migrates everything).
Once I find the current stuff more-or-less stable I'll add temperature support and send the patch.
Thanks, Edward.
| |