[lkml]   [2017]   [Dec]   [29]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: ANNOUNCE: bpfd - a remote proxy daemon for executing bpf code (with corres. bcc changes)
Correcting Brenden's email address.

On Fri, Dec 29, 2017 at 12:58 AM, Joel Fernandes <> wrote:
> Hi Guys,
> I've been working on an idea I discussed with other kernel developers
> (Alexei, Josef etc) last LPC about how to make it easier to run bcc
> tools on remote systems.
> Use case
> ========
> Run bcc tools on a remotely connected system without having to load
> the entire LLVM infrastructure onto the remote target and have to sync
> the kernel sources with it.
> On architecture such as ARM64 especially, its a bit more work if you
> were to run the tools directly on the target itself (local to the
> target) because LLVM and Python have to be cross-compiled for it
> (along with syncing of kernel sources which takes up space and needs
> to be kept sync'ed for correct operation). I believe Facebook also has
> some usecases where they want to run bcc tools on remote instances.
> Lastly this is also the way arm64 development normally happens, you
> cross build for it and typically the ARM64 embedded systems may not
> have much space for kernel sources and clang so its better some times
> if the tools are remote. All our kernel development for android is
> cross developed with the cross-toolchain running remotely as well.
> I am looking forward to collaborating with interested developers on
> this idea and getting more feedback about the design etc. I am also
> planning to talk about it next year during SCALE and OSPM.
> Implementation
> ==============
> To facilitate this, I started working on a daemon called bpfd which is
> executed on the remote target and listening for commands:
> The daemon does more than proxy the bpf syscall, there's several
> things like registering a kprobe with perf, and perf callbacks that
> need to be replicated. All this infrastructure is pretty much code
> complete in bpfd.
> Sample commands sent to bpfd are as follows:
> ------------------------
> ; Program opensnoop
> BPF_CREATE_MAP 1 8 40 10240 0
> BPF_CREATE_MAP 4 4 4 2 0
> ------------------------
> Binary streams are communicated using base64 making it possible to
> keep interaction with binary simple.
> Several patches is written on the bcc side to be able to send these
> commands using a "remotes infrastructure", available in the branch at:
> My idea was to keep the remote infrastructure as generic/plug-and-play
> as possible - so in the future its easy to add other remotes like
> networking. Currently I've adb (android bridge) remote and a shell
> remote:
> The shell remote is more of a "test" remote that simply forks bpfd and
> communicates with it over stdio. This makes the development quite
> easy.
> Status
> ======
> What's working:
> - executing several bcc tools across process boundary using "shell"
> remote (bcc tools and bpfd both running on local x86 machine).
> - communication with remote arm64 android target using the "adb
> remote". But there are several issues to do with arm64 and bcc tools
> that I'm ironing out. Since my arm64 bcc hackery is a bit recent, I
> created a separate WIP branch here:
> I don't suspect
> these to be a major issue since I noticed some folks have been using
> bcc tools on arm64 already.
> Issues:
> - Since bcc is building with clang on x86 - the eBPF backend code is
> generated for x86. Although it loads fine on arm64, there seem several
> issues such as kprobe handler doesn't see arguments or return code
> correctly in opensnoop. This is (probably)easy to fix by just user
> telling bcc we're build for a certain architecture - but that would
> mean we carry code for each arch when building the bcc libraries and
> dynamically select the code path to run - than building for the C++
> compiler's target architecture.
> - Some operations are quite slow, such as stackcount when the number
> of stack traces are a lot. Each stack trace is a key and and every key
> iterated is at a cost, which adds up. Maybe we can batch these up so
> that they're faster instead of making each key iteration a separate
> remote command/response?
> - Some tools read the ps table on the local host. This needs to be
> remotely proxied.
> - Provide mechanism to make bcc/clang build eBPF for arm64 (using a
> command line switch) ?
> - Design a generic parser mechanism to be added to all bcc tools to be
> able to pass which remote method to use, what the remote architecture
> is and what the path to the kernel sources are (for kprobes to work)
> Thanks a lot to Alexei for discussing ideas in conference and for all
> the great advice and help.
> Regards,
> - Joel
> PS: We also have some usecases where our Android networking daemon has
> hardcoded eBPF asm and our teams want to write them C and load the
> binary stream. It seems bpfd can be a good fit here as well.

 \ /
  Last update: 2017-12-29 10:01    [W:0.062 / U:5.880 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site