lkml.org 
[lkml]   [2018]   [Dec]   [16]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [RFC] LKML Archive in Maildir Format
    On Sun, Dec 16, 2018 at 02:46:49PM -0500, Konstantin Ryabitsev wrote:
    > On Sun, Dec 16, 2018 at 09:06:39AM -1000, Joey Pabalinas wrote:
    > > I spent a lot of time trying to find an LKML archive in Maildir format
    > > that I could use for local searches with nutmuch or something, but all
    > > the links I was able to find were all dead.
    > >
    > > I ended up just compiling one myself and I currently host it at:
    > >
    > > https://alyptik.org/lkml.tar.xz
    >
    > You seem to have duplicated a lot of effort that has already been done
    > to compile the archive on lore.kernel.org.

    Absolutely correct, haha.

    >
    > > It's possible I'm the only weirdo who finds this kind of thing useful, but
    > > I figured I should share it just in case I'm not.
    >
    > The maildir format is kind of terrible for LKML, because having millions
    > of messages in a single directory is very hard on the underlying FS. If
    > you break it up into multiple folders, then it becomes difficult to
    > search. This is the main reason why we have chosen to go with the
    > public-inbox format, which solves both of these problems and allows for
    > a very efficient archive updating and replication using git.
    >
    > > It's about 1.1 million files, I was wondering if anyone had an idea of a
    > > better way to host this? I've tried Github and GitLab, but they don't
    > > appreciate repos with that many files, hah.
    >
    > Like I said, you seem to be going down the road we've already tried and
    > rejected. :)

    Yes, I had a strong suspicion I might be the only crazy person who prefers this
    kind of format :)

    My only comment on the public-mailbox choice is that the documentation
    is very sparse and erratic. Myself and a couple other people just
    couldn't figure out how to convert that format to Maildir or some other
    format you could feed into a reader like neomutt.

    Do you have any advice on how to convert those public-inbox files
    correctly?

    --
    Cheers,
    Joey Pabalinas
    [unhandled content-type:application/pgp-signature]
    \
     
     \ /
      Last update: 2018-12-16 20:54    [W:6.791 / U:0.016 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site