Messages in this thread | | | Subject | Re: [f2fs-dev] [PATCH 2/2] f2fs: tune discard speed with storage usage rate | From | Chao Yu <> | Date | Thu, 16 Aug 2018 19:25:26 +0800 |
| |
On 2018/8/16 9:23, Jaegeuk Kim wrote: > On 08/15, Ju Hyung Park wrote: >> btrfs also distinguishes high/low end SSDs with mount option "ssd" and >> "ssd_spread". >> I believe there is no implementation inside btrfs to do it automatically. >> >> Providing users an option with a well made documentation is acceptable imo. > > Yeah, I was thinking about this as well. How about adding "-o disk=%s" where > %s can be ufs/emmc/ssd/low_ssd/nvme/...?
Agreed,
> And, we also can decide the device type automatically w/ kmsg by default.
You mean by scanning kmsg to see the device type?
Thanks,
> >> >> And I personally think, if the default goes to a dynamically deciding >> method, it should tell it to kmsg. >> >> Thanks, >> >> On Wed, Aug 15, 2018, 12:31 PM Chao Yu <yuchao0@huawei.com> wrote: >> >>> On 2018/8/15 11:20, Jaegeuk Kim wrote: >>>> On 08/15, Chao Yu wrote: >>>>> On 2018/8/15 10:56, Jaegeuk Kim wrote: >>>>>> On 08/15, Chao Yu wrote: >>>>>>> On 2018/8/15 10:33, Jaegeuk Kim wrote: >>>>>>>> On 08/15, Chao Yu wrote: >>>>>>>>> On 2018/8/15 1:23, Jaegeuk Kim wrote: >>>>>>>>>> On 08/14, Chao Yu wrote: >>>>>>>>>>> On 2018/8/14 12:19, Jaegeuk Kim wrote: >>>>>>>>>>>> On 08/10, Chao Yu wrote: >>>>>>>>>>>>> Previously, discard speed was fixed mostly, and in high usage >>> rate >>>>>>>>>>>>> device, we will speed up issuing discard, but it doesn't make >>> sense >>>>>>>>>>>>> that in a non-full filesystem, we still issue discard with slow >>> speed. >>>>>>>>>>>> >>>>>>>>>>>> Could you please elaborate the problem in more detail? The speed >>> depends >>>>>>>>>>>> on how many candidates? >>>>>>>>>>> >>>>>>>>>>> undiscard blocks are all 4k granularity. >>>>>>>>>>> a) utility: filesystem: 20% + undiscard blocks: 20% = flash >>> storage: 40% >>>>>>>>>>> b) utility: filesystem: 40% + undiscard blocks: 25% = flash >>> storage: 65% >>>>>>>>>>> c) utility: filesystem: 60% + undiscard blocks: 30% = flash >>> storage: 100% >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 1. for case c), we need to speed up issuing discard based on >>> utilization of >>>>>>>>>>> "filesystem + undiscard" instead of just utilization of >>> filesystem. >>>>>>>>>>> >>>>>>>>>>> - if (utilization(sbi) > DEF_DISCARD_URGENT_UTIL) { >>>>>>>>>>> - dpolicy->granularity = 1; >>>>>>>>>>> - dpolicy->max_interval = >>> DEF_MIN_DISCARD_ISSUE_TIME; >>>>>>>>>>> - } >>>>>>>>>>> >>>>>>>>>>> 2. If free space in storage touches therein threshold, >>> performance will be very >>>>>>>>>>> sensitive. In low-end storage, with high usage in space, even >>> free space is >>>>>>>>>>> reduced by 1%, performance will decrease a lot. >>>>>>>>>> >>>>>>>>>> So, we may need to distinguish low-end vs. high-end storage. In >>> high-end case, >>>>>>>>>> it'd be better to avoid IO contention, while low-end device wants >>> to get more >>>>>>>>>> discard commands as much as possible. So, how about adding an >>> option for this >>>>>>>>>> as a tunable point? >>>>>>>>> >>>>>>>>> Agreed, how about adding a sysfs entry discard_tunning: >>>>>>>>> 1: enabled, use 4k granularity, self-adapted speed based on real >>> device free space. >>>>>>>>> 0: disabled, use dcc->discard_granularity, fixed speed. >>>>>>>>> >>>>>>>>> By default: enabled >>>>>>>>> >>>>>>>>> How do you think? >>>>>>>> >>>>>>>> I don't think this is proper with a sysfs entry, since we already >>> know the >>>>>>> >>>>>>> You mean by storage capacity? <= 32GB means low-end? >>>>>> >>>>>> Yes, that's current condition to judge it. If there is any other >>> method, it'd be >>>>> >>>>> That would be hard code... >>>>> >>>>> Still I have not got any other method to do the judgment except >>> capacity. >>>> >>>> Maybe ufs or emmc? >>> >>> Yeah, that's may be a good way. >>> >>> I remember very initial version UFS has very poor discard performance, for >>> that >>> kind of storage, it will be not accurate? >>> >>> Let me check whether there is a flag to distinguish ufs/emmc. >>> >>> Thanks, >>> >>>> >>>>> >>>>> Thanks, >>>>> >>>>>> better to change it. >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>>> device type when mounting the partition. We won't require to change >>> the policy >>>>>>>> on the fly. And, I still don't get to change the default. >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> IMO, in above cases, we'd better to issue discard with high speed >>> for c), middle >>>>>>>>>>> speed for b), and low speed for a). >>>>>>>>>>> >>>>>>>>>>> How do you think? >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Anyway, it comes out undiscarded block makes FTL GC be lower >>> efficient >>>>>>>>>>>>> and causing high lifetime overhead. >>>>>>>>>>>>> >>>>>>>>>>>>> Let's tune discard speed as below: >>>>>>>>>>>>> >>>>>>>>>>>>> a. adjust default issue interval: >>>>>>>>>>>>> original after >>>>>>>>>>>>> min_interval: 50ms 100ms >>>>>>>>>>>>> mid_interval: 500ms 1000ms >>>>>>>>>>>>> max_interval: 60000ms 10000ms >>>>>>>>>>>>> >>>>>>>>>>>>> b. if last time we stop issuing discard due to IO interruption >>> of user, >>>>>>>>>>>>> let's reset all {min,mid,max}_interval to default one. >>>>>>>>>>>>> >>>>>>>>>>>>> c. tune {min,mid,max}_interval with below calculation method: >>>>>>>>>>>>> >>>>>>>>>>>>> base_interval = default_interval / 10; >>>>>>>>>>>>> total_interval = default_interval - base_interval; >>>>>>>>>>>>> interval = base_interval + total_interval * (100 - dev_util) / >>> 100; >>>>>>>>>>>>> >>>>>>>>>>>>> For example: >>>>>>>>>>>>> min_interval (:100ms) >>>>>>>>>>>>> dev_util (%) interval (ms) >>>>>>>>>>>>> 0 100 >>>>>>>>>>>>> 10 91 >>>>>>>>>>>>> 20 82 >>>>>>>>>>>>> 30 73 >>>>>>>>>>>>> ... >>>>>>>>>>>>> 80 28 >>>>>>>>>>>>> 90 19 >>>>>>>>>>>>> 100 10 >>>>>>>>>>>>> >>>>>>>>>>>>> Signed-off-by: Chao Yu <yuchao0@huawei.com> >>>>>>>>>>>>> --- >>>>>>>>>>>>> fs/f2fs/f2fs.h | 11 ++++---- >>>>>>>>>>>>> fs/f2fs/segment.c | 64 >>> +++++++++++++++++++++++++++++++++++++---------- >>>>>>>>>>>>> fs/f2fs/segment.h | 9 +++++++ >>>>>>>>>>>>> fs/f2fs/super.c | 2 +- >>>>>>>>>>>>> 4 files changed, 67 insertions(+), 19 deletions(-) >>>>>>>>>>>>> >>>>>>>>>>>>> diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h >>>>>>>>>>>>> index 273ffdaf4891..a1dd2e1c3cb9 100644 >>>>>>>>>>>>> --- a/fs/f2fs/f2fs.h >>>>>>>>>>>>> +++ b/fs/f2fs/f2fs.h >>>>>>>>>>>>> @@ -185,10 +185,9 @@ enum { >>>>>>>>>>>>> >>>>>>>>>>>>> #define MAX_DISCARD_BLOCKS(sbi) BLKS_PER_SEC(sbi) >>>>>>>>>>>>> #define DEF_MAX_DISCARD_REQUEST 8 /* issue 8 >>> discards per round */ >>>>>>>>>>>>> -#define DEF_MIN_DISCARD_ISSUE_TIME 50 /* 50 ms, if >>> exists */ >>>>>>>>>>>>> -#define DEF_MID_DISCARD_ISSUE_TIME 500 /* 500 ms, if >>> device busy */ >>>>>>>>>>>>> -#define DEF_MAX_DISCARD_ISSUE_TIME 60000 /* 60 s, if no >>> candidates */ >>>>>>>>>>>>> -#define DEF_DISCARD_URGENT_UTIL 80 /* do more >>> discard over 80% */ >>>>>>>>>>>>> +#define DEF_MIN_DISCARD_ISSUE_TIME 100 /* 100 ms, if >>> exists */ >>>>>>>>>>>>> +#define DEF_MID_DISCARD_ISSUE_TIME 1000 /* 1000 ms, if >>> device busy */ >>>>>>>>>>>>> +#define DEF_MAX_DISCARD_ISSUE_TIME 10000 /* 10000 ms, if no >>> candidates */ >>>>>>>>>>>>> #define DEF_CP_INTERVAL 60 /* 60 secs >>> */ >>>>>>>>>>>>> #define DEF_IDLE_INTERVAL 5 /* 5 secs */ >>>>>>>>>>>>> >>>>>>>>>>>>> @@ -248,7 +247,8 @@ struct discard_entry { >>>>>>>>>>>>> }; >>>>>>>>>>>>> >>>>>>>>>>>>> /* default discard granularity of inner discard thread, unit: >>> block count */ >>>>>>>>>>>>> -#define DEFAULT_DISCARD_GRANULARITY 1 >>>>>>>>>>>>> +#define MID_DISCARD_GRANULARITY 16 >>>>>>>>>>>>> +#define MIN_DISCARD_GRANULARITY 1 >>>>>>>>>>>>> >>>>>>>>>>>>> /* max discard pend list number */ >>>>>>>>>>>>> #define MAX_PLIST_NUM 512 >>>>>>>>>>>>> @@ -330,6 +330,7 @@ struct discard_cmd_control { >>>>>>>>>>>>> atomic_t discard_cmd_cnt; /* # of cached cmd >>> count */ >>>>>>>>>>>>> struct rb_root root; /* root of discard >>> rb-tree */ >>>>>>>>>>>>> bool rbtree_check; /* config for >>> consistence check */ >>>>>>>>>>>>> + bool io_interrupted; /* last state of >>> io interrupted */ >>>>>>>>>>>>> }; >>>>>>>>>>>>> >>>>>>>>>>>>> /* for the list of fsync inodes, used only during recovery */ >>>>>>>>>>>>> diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c >>>>>>>>>>>>> index 8b52e8dfb12f..9564aaf1f27b 100644 >>>>>>>>>>>>> --- a/fs/f2fs/segment.c >>>>>>>>>>>>> +++ b/fs/f2fs/segment.c >>>>>>>>>>>>> @@ -968,6 +968,44 @@ static void __check_sit_bitmap(struct >>> f2fs_sb_info *sbi, >>>>>>>>>>>>> #endif >>>>>>>>>>>>> } >>>>>>>>>>>>> >>>>>>>>>>>>> +static void __adjust_discard_speed(unsigned int *interval, >>>>>>>>>>>>> + unsigned int def_interval, int >>> dev_util) >>>>>>>>>>>>> +{ >>>>>>>>>>>>> + unsigned int base_interval, total_interval; >>>>>>>>>>>>> + >>>>>>>>>>>>> + base_interval = def_interval / 10; >>>>>>>>>>>>> + total_interval = def_interval - base_interval; >>>>>>>>>>>>> + >>>>>>>>>>>>> + /* >>>>>>>>>>>>> + * if def_interval = 100, adjusted interval should be in >>> range of >>>>>>>>>>>>> + * [10, 100]. >>>>>>>>>>>>> + */ >>>>>>>>>>>>> + *interval = base_interval + total_interval * (100 - >>> dev_util) / 100; >>>>>>>>>>>>> +} >>>>>>>>>>>>> + >>>>>>>>>>>>> +static void __tune_discard_policy(struct f2fs_sb_info *sbi, >>>>>>>>>>>>> + struct discard_policy >>> *dpolicy) >>>>>>>>>>>>> +{ >>>>>>>>>>>>> + struct discard_cmd_control *dcc = SM_I(sbi)->dcc_info; >>>>>>>>>>>>> + int dev_util; >>>>>>>>>>>>> + >>>>>>>>>>>>> + if (dcc->io_interrupted) { >>>>>>>>>>>>> + dpolicy->min_interval = DEF_MIN_DISCARD_ISSUE_TIME; >>>>>>>>>>>>> + dpolicy->mid_interval = DEF_MID_DISCARD_ISSUE_TIME; >>>>>>>>>>>>> + dpolicy->max_interval = DEF_MAX_DISCARD_ISSUE_TIME; >>>>>>>>>>>>> + return; >>>>>>>>>>>>> + } >>>>>>>>>>>>> + >>>>>>>>>>>>> + dev_util = dev_utilization(sbi); >>>>>>>>>>>>> + >>>>>>>>>>>>> + __adjust_discard_speed(&dpolicy->min_interval, >>>>>>>>>>>>> + DEF_MIN_DISCARD_ISSUE_TIME, >>> dev_util); >>>>>>>>>>>>> + __adjust_discard_speed(&dpolicy->mid_interval, >>>>>>>>>>>>> + DEF_MID_DISCARD_ISSUE_TIME, >>> dev_util); >>>>>>>>>>>>> + __adjust_discard_speed(&dpolicy->max_interval, >>>>>>>>>>>>> + DEF_MAX_DISCARD_ISSUE_TIME, >>> dev_util); >>>>>>>>>>>>> +} >>>>>>>>>>>>> + >>>>>>>>>>>>> static void __init_discard_policy(struct f2fs_sb_info *sbi, >>>>>>>>>>>>> struct discard_policy *dpolicy, >>>>>>>>>>>>> int discard_type, unsigned int >>> granularity) >>>>>>>>>>>>> @@ -982,20 +1020,11 @@ static void __init_discard_policy(struct >>> f2fs_sb_info *sbi, >>>>>>>>>>>>> dpolicy->io_aware_gran = MAX_PLIST_NUM; >>>>>>>>>>>>> >>>>>>>>>>>>> if (discard_type == DPOLICY_BG) { >>>>>>>>>>>>> - dpolicy->min_interval = DEF_MIN_DISCARD_ISSUE_TIME; >>>>>>>>>>>>> - dpolicy->mid_interval = DEF_MID_DISCARD_ISSUE_TIME; >>>>>>>>>>>>> - dpolicy->max_interval = DEF_MAX_DISCARD_ISSUE_TIME; >>>>>>>>>>>>> dpolicy->io_aware = true; >>>>>>>>>>>>> dpolicy->sync = false; >>>>>>>>>>>>> dpolicy->ordered = true; >>>>>>>>>>>>> - if (utilization(sbi) > DEF_DISCARD_URGENT_UTIL) { >>>>>>>>>>>>> - dpolicy->granularity = 1; >>>>>>>>>>>>> - dpolicy->max_interval = >>> DEF_MIN_DISCARD_ISSUE_TIME; >>>>>>>>>>>>> - } >>>>>>>>>>>>> + __tune_discard_policy(sbi, dpolicy); >>>>>>>>>>>>> } else if (discard_type == DPOLICY_FORCE) { >>>>>>>>>>>>> - dpolicy->min_interval = DEF_MIN_DISCARD_ISSUE_TIME; >>>>>>>>>>>>> - dpolicy->mid_interval = DEF_MID_DISCARD_ISSUE_TIME; >>>>>>>>>>>>> - dpolicy->max_interval = DEF_MAX_DISCARD_ISSUE_TIME; >>>>>>>>>>>>> dpolicy->io_aware = false; >>>>>>>>>>>>> } else if (discard_type == DPOLICY_FSTRIM) { >>>>>>>>>>>>> dpolicy->io_aware = false; >>>>>>>>>>>>> @@ -1353,6 +1382,8 @@ static unsigned int >>> __issue_discard_cmd_orderly(struct f2fs_sb_info *sbi, >>>>>>>>>>>>> if (!issued && io_interrupted) >>>>>>>>>>>>> issued = -1; >>>>>>>>>>>>> >>>>>>>>>>>>> + dcc->io_interrupted = io_interrupted; >>>>>>>>>>>>> + >>>>>>>>>>>>> return issued; >>>>>>>>>>>>> } >>>>>>>>>>>>> >>>>>>>>>>>>> @@ -1370,7 +1401,7 @@ static int __issue_discard_cmd(struct >>> f2fs_sb_info *sbi, >>>>>>>>>>>>> if (i + 1 < dpolicy->granularity) >>>>>>>>>>>>> break; >>>>>>>>>>>>> >>>>>>>>>>>>> - if (i < DEFAULT_DISCARD_GRANULARITY && >>> dpolicy->ordered) >>>>>>>>>>>>> + if (i < MID_DISCARD_GRANULARITY && >>> dpolicy->ordered) >>>>>>>>>>>>> return __issue_discard_cmd_orderly(sbi, >>> dpolicy); >>>>>>>>>>>>> >>>>>>>>>>>>> pend_list = &dcc->pend_list[i]; >>>>>>>>>>>>> @@ -1407,6 +1438,8 @@ static int __issue_discard_cmd(struct >>> f2fs_sb_info *sbi, >>>>>>>>>>>>> if (!issued && io_interrupted) >>>>>>>>>>>>> issued = -1; >>>>>>>>>>>>> >>>>>>>>>>>>> + dcc->io_interrupted = io_interrupted; >>>>>>>>>>>>> + >>>>>>>>>>>>> return issued; >>>>>>>>>>>>> } >>>>>>>>>>>>> >>>>>>>>>>>>> @@ -1576,7 +1609,11 @@ static int issue_discard_thread(void >>> *data) >>>>>>>>>>>>> struct f2fs_sb_info *sbi = data; >>>>>>>>>>>>> struct discard_cmd_control *dcc = SM_I(sbi)->dcc_info; >>>>>>>>>>>>> wait_queue_head_t *q = &dcc->discard_wait_queue; >>>>>>>>>>>>> - struct discard_policy dpolicy; >>>>>>>>>>>>> + struct discard_policy dpolicy = { >>>>>>>>>>>>> + .min_interval = DEF_MIN_DISCARD_ISSUE_TIME, >>>>>>>>>>>>> + .mid_interval = DEF_MID_DISCARD_ISSUE_TIME, >>>>>>>>>>>>> + .max_interval = DEF_MAX_DISCARD_ISSUE_TIME, >>>>>>>>>>>>> + }; >>>>>>>>>>>>> unsigned int wait_ms = DEF_MIN_DISCARD_ISSUE_TIME; >>>>>>>>>>>>> int issued; >>>>>>>>>>>>> >>>>>>>>>>>>> @@ -1929,7 +1966,7 @@ static int >>> create_discard_cmd_control(struct f2fs_sb_info *sbi) >>>>>>>>>>>>> if (!dcc) >>>>>>>>>>>>> return -ENOMEM; >>>>>>>>>>>>> >>>>>>>>>>>>> - dcc->discard_granularity = DEFAULT_DISCARD_GRANULARITY; >>>>>>>>>>>>> + dcc->discard_granularity = MIN_DISCARD_GRANULARITY; >>>>>>>>>>>>> INIT_LIST_HEAD(&dcc->entry_list); >>>>>>>>>>>>> for (i = 0; i < MAX_PLIST_NUM; i++) >>>>>>>>>>>>> INIT_LIST_HEAD(&dcc->pend_list[i]); >>>>>>>>>>>>> @@ -1945,6 +1982,7 @@ static int >>> create_discard_cmd_control(struct f2fs_sb_info *sbi) >>>>>>>>>>>>> dcc->next_pos = 0; >>>>>>>>>>>>> dcc->root = RB_ROOT; >>>>>>>>>>>>> dcc->rbtree_check = false; >>>>>>>>>>>>> + dcc->io_interrupted = false; >>>>>>>>>>>>> >>>>>>>>>>>>> init_waitqueue_head(&dcc->discard_wait_queue); >>>>>>>>>>>>> SM_I(sbi)->dcc_info = dcc; >>>>>>>>>>>>> diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h >>>>>>>>>>>>> index 422b0ceb1eaa..63b4da72cd34 100644 >>>>>>>>>>>>> --- a/fs/f2fs/segment.h >>>>>>>>>>>>> +++ b/fs/f2fs/segment.h >>>>>>>>>>>>> @@ -616,6 +616,15 @@ static inline int utilization(struct >>> f2fs_sb_info *sbi) >>>>>>>>>>>>> sbi->user_block_count); >>>>>>>>>>>>> } >>>>>>>>>>>>> >>>>>>>>>>>>> +static inline int dev_utilization(struct f2fs_sb_info *sbi) >>>>>>>>>>>>> +{ >>>>>>>>>>>>> + unsigned int dev_blks; >>>>>>>>>>>>> + >>>>>>>>>>>>> + dev_blks = valid_user_blocks(sbi) + >>> SM_I(sbi)->dcc_info->undiscard_blks; >>>>>>>>>>>>> + return div_u64((u64)dev_blks * 100, >>>>>>>>>>>>> + MAIN_SEGS(sbi) << sbi->log_blocks_per_seg); >>>>>>>>>>>>> +} >>>>>>>>>>>>> + >>>>>>>>>>>>> /* >>>>>>>>>>>>> * Sometimes f2fs may be better to drop out-of-place update >>> policy. >>>>>>>>>>>>> * And, users can control the policy through sysfs entries. >>>>>>>>>>>>> diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c >>>>>>>>>>>>> index b055f2ea77c5..55ed76daad23 100644 >>>>>>>>>>>>> --- a/fs/f2fs/super.c >>>>>>>>>>>>> +++ b/fs/f2fs/super.c >>>>>>>>>>>>> @@ -2862,7 +2862,7 @@ static void f2fs_tuning_parameters(struct >>> f2fs_sb_info *sbi) >>>>>>>>>>>>> /* adjust parameters according to the volume size */ >>>>>>>>>>>>> if (sm_i->main_segments <= SMALL_VOLUME_SEGMENTS) { >>>>>>>>>>>>> F2FS_OPTION(sbi).alloc_mode = ALLOC_MODE_REUSE; >>>>>>>>>>>>> - sm_i->dcc_info->discard_granularity = 1; >>>>>>>>>>>>> + sm_i->dcc_info->discard_granularity = >>> MIN_DISCARD_GRANULARITY; >>>>>>>>>>>>> sm_i->ipu_policy = 1 << F2FS_IPU_FORCE; >>>>>>>>>>>>> } >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> 2.18.0.rc1 >>>>>>>>>>>> >>>>>>>>>>>> . >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> . >>>>>>>>>> >>>>>>>> >>>>>>>> . >>>>>>>> >>>>>> >>>>>> . >>>>>> >>>> >>>> . >>>> >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> Check out the vibrant tech community on one of the world's most >>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >>> _______________________________________________ >>> Linux-f2fs-devel mailing list >>> Linux-f2fs-devel@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel >>> > > . >
| |