lkml.org 
[lkml]   [2005]   [Jul]   [13]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Merging relayfs?
Tomasz Kłoczko wrote:

> On Tue, 12 Jul 2005, Tom Zanussi wrote:
> [..]
>
>> > DTrace real examples shows something completly diffret.
>> > MANY things (if not ~almost all) can be kept only in aggregated form
>> > during experiments.
>>
>> But you can also do the aggregation in user space if you have a cheap
>> way of getting it there, as we've shown with some of the examples.
>
>
> Sorry but real life examples shows that store chunk of data in
> agregator is less expensive than context switch neccessary for store
> data or time neccasy for send and handle signal from buffer like "I'm
> full! let me out of here ..".
>
> [..]
>
>> > store raw data. What you need ? only one counter (few bytes)
>> instead of huge
>> > amount of memeory for buffer and store logs. Try measure something
>> like
>> > scheduler with possible small system distruption.
>>
>> Most of the time the data is just being buffered and only when the
>> buffer is full is it written to disk, as one write. If that's too
>> disruptive, then maybe you do need to do some aggregation in the kernel,
>> but it sounds like a special case.
>
>
> OK .. "so you can say better is stop flushing buffers on measure which
> wil take day or more" ? :_)
> Some DTrace probes/technik are specialy prepared for long or evel very
> long time experiment wich will only prodyce few lines results on end
> of experiment.
> Look at DTrace documentation for speculative tracing:
> http://docs.sun.com/app/docs/doc/817-6223/6mlkidli7?a=view
>
> Some experiments do not have deterinistic time and must be finished
> after i. e. "occasional failing". What if it will take so long so you
> will fill all avalaible storage in relayfs way ?
> OK, never mind .. you have discontinued storage. Using kind
> speculative tracing way I'll have result *just after* "occasional
> failing" and you will start parse data stored using relayfs.
>
> kloczek


O.K, Tomasz your point is we can do aggregation in the kernel and cut
down the amount of data that needs to be sent out from the kernel hence
we don't need an efficient, low overhead mechanism like relayfs to get
the data out of the kernel. Having relayfs doesn't prevent someone in
aggregating the data in the kernel, so it is not an argument for not
including relayfs in the kernel when it fills the need for those who
needs raw data.

I am part of a team working on systemtap where we are are developing a
tool similar to Dtrace that does some aggregation where appropriate but
nothing like fancy statistics etc. We use relayfs in our systemtap
project and based on my reading of Dtrace paper they use exactly
similar to relayfs buffering mechanism as well.

There are tools like itrace and Intel has one (i forgot the name) they
would like to get the raw data into user space and do all kinds of
fancy statistical analysis, visualization etc. Their value add is the
analysis of the data. I am sure you are not suggesting pushing
capabilities of those tools to the kernel, right.

As Steven Rostedt mentioned in his initial reply in this thread, many of
us have written adhoc buffering scheme similar to what relayfs provides
to debug kernel problems that happen after a long running test, if such
facility already exists in the kernel everyone doesn't have to develop one.

I would like to see relayfs merged.




-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-07-13 06:33    [W:1.113 / U:0.044 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site