lkml.org 
[lkml]   [2019]   [Aug]   [19]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: [PATCH] erofs: move erofs out of staging
From
Date
[...]
>>> I have made a simple fuzzer to inject messy in inode metadata,
>>> dir data, compressed indexes and super block,
>>> https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/commit/?h=experimental-fuzzer
>>>
>>> I am testing with some given dirs and the following script.
>>> Does it look reasonable?
>>>
>>> # !/bin/bash
>>>
>>> mkdir -p mntdir
>>>
>>> for ((i=0; i<1000; ++i)); do
>>> mkfs/mkfs.erofs -F$i testdir_fsl.fuzz.img testdir_fsl > /dev/null 2>&1
>>
>> mkfs fuzzes the image? Er....
>
> Thanks for your reply.
>
> First, This is just the first step of erofs fuzzer I wrote yesterday night...
>
>>
>> Over in XFS land we have an xfs debugging tool (xfs_db) that knows how
>> to dump (and write!) most every field of every metadata type. This
>> makes it fairly easy to write systematic level 0 fuzzing tests that
>> check how well the filesystem reacts to garbage data (zeroing,
>> randomizing, oneing, adding and subtracting small integers) in a field.
>> (It also knows how to trash entire blocks.)

The same tool exists for btrfs, although lacks the write ability, but
that dump is more comprehensive and a great tool to learn the on-disk
format.


And for the fuzzing defending part, just a few kernel releases ago,
there is none for btrfs, and now we have a full static verification
layer to cover (almost) all on-disk data at read and write time.
(Along with enhanced runtime check)

We have covered from vague values inside tree blocks and invalid/missing
cross-ref find at runtime.

Currently the two layered check works pretty fine (well, sometimes too
good to detect older, improper behaved kernel).
- Tree blocks with vague data just get rejected by verification layer
So that all members should fit on-disk format, from alignment to
generation to inode mode.

The error will trigger a good enough (TM) error message for developer
to read, and if we have other copies, we retry other copies just as
we hit a bad copy.

- At runtime, we have much less to check
Only cross-ref related things can be wrong now. since everything
inside a single tree block has already be checked.

In fact, from my respect of view, such read time check should be there
from the very beginning.
It acts kinda of a on-disk format spec. (In fact, by implementing the
verification layer itself, it already exposes a lot of btrfs design
trade-offs)

Even for a fs as complex (buggy) as btrfs, we only take 1K lines to
implement the verification layer.
So I'd like to see every new mainlined fs to have such ability.

>
> Actually, compared with XFS, EROFS has rather simple on-disk format.
> What we inject one time is quite deterministic.
>
> The first step just purposely writes some random fuzzed data to
> the base inode metadata, compressed indexes, or dir data field
> (one round one field) to make it validity and coverability.
>
>>
>> You might want to write such a debugging tool for erofs so that you can
>> take apart crashed images to get a better idea of what went wrong, and
>> to write easy fuzzing tests.
>
> Yes, we will do such a debugging tool of course. Actually Li Guifu is now
> developping a erofs-fuse to support old linux versions or other OSes for
> archiveing only use, we will base on that code to develop a better fuzzer
> tool as well.

Personally speaking, debugging tool is way more important than a running
kernel module/fuse.
It's human trying to write the code, most of time is spent educating
code readers, thus debugging tool is way more important than dead cold code.

Thanks,
Qu
>
> Thanks,
> Gao Xiang
>
>>
>> --D
>>
>>> umount mntdir
>>> mount -t erofs -o loop testdir_fsl.fuzz.img mntdir
>>> for j in `find mntdir -type f`; do
>>> md5sum $j > /dev/null
>>> done
>>> done
>>>
>>> Thanks,
>>> Gao Xiang
>>>
>>>>
>>>> Thanks,
>>>> Gao Xiang
>>>>

[unhandled content-type:application/pgp-signature]
\
 
 \ /
  Last update: 2019-08-20 02:58    [W:0.560 / U:0.056 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site