lkml.org 
[lkml]   [2020]   [May]   [31]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: [ANNOUNCE] Reiser5: Data Tiering. Burst Buffers. Speedup synchronous modifications
From
Date
On 05/30/2020 02:32 PM, jose.r.r@metztli.com wrote:
> On Mon, May 25, 2020 at 6:08 PM Edward Shishkin <edward.shishkin@gmail.com> wrote:
>>
>> Reiser5: Data Tiering. Burst Buffers
>> Speedup synchronous modifications
>>
>>
>> Dumping peaks of IO load to a proxy device
>>
>>
>> Now you can add a small high-performance block device to your large
>> logical volume composed of relatively slow commodity disks and get
>> an impression that the whole your volume has throughput which is as
>> high, as the one of that "proxy" device!
>>
>> This is based on a simple observation that in real life IO load is
>> going by peaks, and the idea is to dump those peaks to a high-
>> performance "proxy" device. Usually you have enough time between peaks
>> to flush the proxy device, that is, to migrate the "hot data" from the
>> proxy device to slow media in background mode, so that your proxy
>> device is always ready to accept a new portion of "peaks".
>>
>> Such technique, which is also known as "Burst Buffers", initially
>> appeared in the area of HPC. Despite this fact, it is also important
>> for usual applications. In particular, it allows to speedup the ones,
>> which perform so-called "atomic updates".
>>
>>
>> Speedup "atomic updates" in user-space
>>
>>
>> There is a whole class of applications with high requirements to data
>> integrity. Such applications (typically data bases) want to be sure
>> that any data modifications either complete, or they don't. And they
>> don't appear as partially occurred. Some applications has weaker
>> requirements: with some restrictions they accept also partially
>> occurred modifications.
>>
>> Atomic updates in user space are performed via a sequence of 3 steps.
>> Suppose you need to modify data of some file "foo" in an atomic way.
>> For this you need to:
>>
>> 1. write a new temporary file "foo.tmp" with modified data
>> 2. issue fsync(2) against "foo.tmp"
>> 3. rename "foo.tmp" to "foo".
>>
>> At step 1 the file system populates page cache with new data
>> At step 2 the file system allocates disk addresses for all logical
>> blocks of the file foo.tmp and writes that file to disk. At step 3 all
>> blocks containing old data get released.
>>
>> Note that steps 2 and 3 become a reason of essential performance drop
>> on slow media. The situation gets improved, when all dirty data are
>> written to a dedicated high-performance proxy-disk, which exactly
>> happens in a file system with Burst Buffers support.
>>
>>
>> Speedup all synchronous modifications (TODO)
>> Burst Buffers and transaction manager
>>
>>
>> Not only dirty data pages, but also dirty meta-data pages can be
>> dumped to the proxy-device, so that step (3) above also won't
>> contribute to the performance drop.
>>
>> Moreover, not only new logical data blocks can be dumped to the proxy
>> disk. All dirty data pages, including ones, which already have
>> location on the main (slow) storage can also be relocated to the proxy
>> disk, thus, speeding up synchronous modification of files in _all_
>> cases (not only in atomic updates via write-fsync-rename sequence
>> described above).
>>
>> Indeed, let's remind that any modified page is always written to disk
>> in a context of committing some transaction. Depending on the commit
>> strategy (there are 2 ones "relocate" and "overwrite"), for each such
>> modified dirty page there are only 2 possibility:
>>
>> a) to be written right away to a new location,
>> b) to be written first to a temporary location (journal), then to be
>> written back to permanent location.
>>
>> With Burst buffers support in the case (a) the file system writes
>> dirty page right away to the proxy device. Then user should take care
>> to migrate it back to the permanent storage (see section "Flushing
>> proxy devise" below). In the case (b) the modified copy will be
>> written to the proxy device (wandering logs), then at checkpoint time
>> (playing a transaction) reiser4 transaction manager will write it to
>> the permanent location (on commodity disks). In this case user doesn't
>> need to worry on flushing proxy device, however, the procedure of
>> commit takes more time, as user should also wait for "checkpoint
>> completion".
>>
>> So from the standpoint of performance "write-anywhere" transaction
>> model (reiser4 mount option "txmod=wa") is more preferable then
>> journalling model (txmod=journal), or even hybrid model (txmod=hybrid)
>>
>>
>> Predictable and non-predictable migration
>> Meta-data migration
>>
>>
>> As we already mentioned, not only dirty data pages, but also dirty
>> meta-data pages can be dumped to the proxy-device. Note, however, that
>> not predictable meta-data migration is not possible because of
>> chicken-eggish problem. Indeed, non-predictable migration means that
>> nobody knows, on what device of your logical volume a stripe of data
>> will be relocated in the future. Such migration requires to record
>> location of data stripes. Now note, that such records is always a part
>> of meta-data. Hence, you are now able to migrate meta-data in
>> non-predictable way.
>>
>> However, it is perfectly possible to distribute/migrate meta-data in a
>> predictable way (it will be supported in so-called "symmetric" logical
>> volumes - currently not implemented). Classic example of predictable
>> migration is RAID arrays (once you add, or remove a device to/from the
>> array, all data blocks migrate in predictable way during rebalancing).
>> If relocation is predictable, then it is not need to record locations
>> of data stripes - it can always be calculated.
>>
>> Thus, non-predictable migration is applicable to data only.
>>
>>
>> Definition of data tiering.
>> Using proxy device to store hot data (TODO)
>>
>>
>> Now we can precisely define tiering as (meta-)data relocation in
>> accordance with some strategy (automatic, or user-defined), so that
>> every relocated unit always gets location on another device-component
>> of the logical volume.
>>
>> During such relocation block number B1 on device D1 gets released,
>> first address component is changed to D2, second component is changed
>> to 0 (which indicates not allocated block number), then the file
>> system allocates block number B2 on device D2:
>>
>> (D1, B1) -> (D2, 0) -> (D2, B2)
>>
>> Note that tiering is not defined for simple volumes (i.e. volumes,
>> consisting only of one device). Blocks relocation within one device
>> is always in a competence of a file system (to be precisely, of block
>> allocator.
>>
>> Burst buffers is just one of strategies, in accordance with which all
>> new logical blocks (optionally, all dirty pages) always get location
>> on a dedicated proxy device. As we have figured out, Burst Buffers is
>> useful for HPC applications, as well as for usual applications
>> executing fsync(2) frequently.
>>
>> There are other data tiering strategies, which can be useful for other
>> class of applications. All of them can be easily implemented in
>> Reiser5.
>>
>> For example, you can use proxy device to store hot data only. With
>> such strategy new logical blocks (which are always "cold") will always
>> go to the main storage (in contrast with Burst Buffers, where new
>> logical blocks first get written to the proxy disk). Once in a while
>> you need to scan your volume in order to push colder data out, and
>> pull hotter data in the proxy disk. Reiser5 contains a common
>> interface for this. It is possible to maintain per-file, or even per-
>> blocks-extent "temperature" of data (e.g. as a generation counter),
>> but we still don't have more or less satisfactory algorithms to
>> determine "critical temperature" for pushing data in/out proxy disk.
>>
>>
>> Getting started with proxy disk over logical volume
>>
>>
>> Just follow the administration guide:
>> https://reiser4.wiki.kernel.org/index.php/Proxy_Device_Administration
>>
> Re:
>> WARNING: THE STUFF IS NOT STABLE! Don't store important data on
>> Reiser5 logical volumes till beta-stability announcement.
> Will you be releasing reiser4 Software Format Release Number 4.0.2 for Linux kernel 5.6 patch?

I've uploaded it.
Please, find here:

https://sourceforge.net/projects/reiser4/files/reiser4-for-linux-5.x/

Thanks,
Edward

>
> From personal experience SFRN 4.0.2 is stable and all my data, local and cloud virtual machines instances, as well as computing for the last six(6)+ years is in that format/environment. Although I have not tried an Debian based installation with this second iteration of SFRN 5 I have no use for the kernel/reiser4progs until they play well with Debian installer, python, etc..
>
> Best Professional Regards.
>
>

\
 
 \ /
  Last update: 2020-05-31 20:24    [W:0.041 / U:2.400 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site