[lkml]   [2006]   [Jan]   [23]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectNeed insight on designing a package manager
    Hash: SHA1

    As a learning project, I'm designing my own package manager. Naturally,
    I dislike wasting time; the package manager I'm designing will fill some
    gaps in current package management design, and is in reality aimed at
    being a "final" implementation of this many-reinvented wheel. To this
    end, I need help; I've never made a functional package manager, and so I
    don't likely have ALL of the insight I'll need to do it right the first
    time. So those of you who have studied, maintained, or created package
    managers, please help me?

    The working project name is "Project Coon Fox," and will be later
    discarded for a real name. It is designed to replace dpkg/apt, rpm/yum,
    and autopackage in functionality; it is not designed to be a source
    build system, as Gentoo portage is.

    The original idea for Project Coon Fox was to create a package manager
    which did not use preinstall/postinstall scripts freely executed as root
    in a fully open security context. The following (long and boring)
    discussion will explain where I'm going with this, and what other design
    contingencies I've taken into account so far. If you don't care, you
    can stop reading here. If you don't care *much*, go ahead and skim. :)

    The problem with both dpkg (deb) and rpm (rpm) is that they execute bash
    scripts in the system administration context (root, full caps,
    sysadmin_r, etc); portage attempts to use a sandbox library, but this
    can be evaded by bringing your own syscall() code, as the sandbox only
    wrappers glibc functions. As you may guess, this can allow any
    malicious package to destroy the system; in fact, upon attempt to
    install Sun Java on Debian, I found that clicking "no" on the license
    permenantly alters the system in ways dpkg can't track and roll back,
    assuring you can never try to install the package again.

    The only real way to control and audit the installation process is to
    have the package manager interpret the installation scripts. Project
    Coon Fox will do exactly that. I've settled on writing a description in
    XML and using libxml2 to parse it; this is crude, but effective, and
    extensible via plug-ins. I allow for querying the user and making
    policy decisions in this script (i.e. file permissions, control of SUID,
    associations, mime types... any block of script can basically be
    controlled by user input). Policy integration will be implimented as
    well; policy plug-ins for SELinux are planned, and "meta-native"
    policies will allow a policy file to have chunks modified based on user

    Project Coon Fox will be able to audit the changes fully and partially.
    A "basic audit" mode will compare a set of rules to the changes and
    raise specific concerns to the user, allowing the installation to be
    aborted or the package to be modified, i.e. by removing SUID/SGID bits.
    This goes for specific policy as well; the final SELinux policy, for
    example, could be audited for "grants permissions beyond r_default in
    this given policy." Reactions could be to remove excess permissions.
    And of course, as all of this is tracked and logged, any of these
    changes can be easily reviewed later and undone.

    Install script handling will be modular. Plug-ins can add new install
    script descriptions, such as mime type handling, file associations,
    entries into scrollkeeper, registering of plug-ins or extensions for
    certain programs, etc. Such plug-ins are extremely dangerous; they need
    the control to make low-level system modifications. Random third
    parties should have no need to add a plug-in here; and if they do, there
    should be no need to make any part of its execution enter a closed
    source code block.

    Plug-ins will also allow for policy extensions, to understand SELinux
    and GrSecurity policy files and how to activate them in the system.
    These policy plug-ins will also translate policies written in a built-in
    meta-policy language to the native format of the target backend.

    I quickly made the decision to use SQLite for the back-end data store.
    I have already written some of the related code here; I refuse to write
    more until I have a clear view of what I am doing. Writing and
    rewriting code over and over because of minor changes in the way I'm
    going to handle things is not a good idea. SQLite was, of course,
    chosen for performance. Running a full RDBMS like MySQL or PostGreSQL
    is out of the question; embedded MySQL is out because it's not object
    oriented (SQLite lets me sqlite3_open() a database and get a handle to
    use; I can work on 100 db's if I want, all at once). Evidently the
    MySQL folk don't understand that C can be used for object oriented
    programming; it just doesn't do it in the language, as in C++ or Obj-C.

    This is one dilema point; I don't know all the information to store in
    the database. I'm working on this; could use some help.

    I decided to separate the concept of "Package" and "Install." The
    package portion of the program will consist of a library that creates a
    temporary directory tree representing the package, including the install
    scripts and system installation image. The install end will take a
    directory and act on it, installing its contents to the system. The
    install end will be capable of triggering callbacks to handle queries
    and auditing concerns.

    In effect, the core program will call the package end to unpack a
    package file; and then call the install end to execute the installation.
    The install portion in its final stages gives a complete audit of the
    changes, as well as a little extra data to identify the packages and
    some other control information; this all can then be stored.
    Effectively, as long as the APIs are the same and various other
    considerations are taken, mindful programmers can randomly replace any
    of the package, install, or database components with completely new code.

    Changes to how packages are handled can be made by modifying the package
    component of this model. In this component, I'm implimenting a
    contingency for LiveCDs. Packages can contain either files; or control
    information with a hash of individual files, their ownership,
    permissions, size, and path in the native package format. By giving the
    package component a base path to use, it will be possible to take a
    mounted image file i.e. from a LiveCD and a set of packages made by
    comparing full packages to this image file, and reconstruct the
    original, unpacked contents from this union. This would allow for
    packages on a LiveCD to count 20-50KiB on average, letting an install
    base of 1000 packages require about 50MiB. This will fit on a good
    LiveCD (i.e. the Ubuntu LiveCD can take it), allowing for a system to be
    "working" before it is even installed.

    My current largest dilema is dependency checking. I want a
    file-interface dependency model, handled by the install module. This
    means looking for either a program /bin/foo or /usr/bin/foo or "InPath
    foo" (a la autopackage IIRC) and comparing its command line interface;
    or finding a library in the same way and comparing its API.

    As we're not taking the massively insecure habit of allowing
    not-yet-installed programs to tell us to execute random bits of code
    that already exist on our system, this will probably require interface
    description control information to be packaged with everything about
    everything, including library interfaces, plug-in system versions and
    interfaces, shell command switches, shell command style (BSD vs GNU),
    etc. Anything pre-existing that would work properly here would be
    great; but I feel this may be one of those things where in order to
    really do it right (as in, aim for correctness instead of ease of
    implimentation), we're just going to have to bite the bullet and tell
    people that they simply have to maintain this information.

    OK so anyway if you read this far into all that BS, you must somewhat
    care. Any insight is appreciated, go ahead and mail me. There's no
    site yet; I document this stuff deeply in a mediawiki run inside qemu on
    my computer. If there's a good handfull of interested individuals who
    really want to have a go at it, have some really qualifying experience,
    and most importantly actually have the available time AND the will to
    commit to this project (this does not mean you're a CS student looking
    for a way to get experience; this means you're like a Gnome hacker or
    something and have extra time to spare on other projects, and like this
    one), I'll consider opening a project page on Sourceforge and
    transferring all current design documentation proper.

    One last thing, I'm a businessman, I think like a businessman, I make
    decisions like a businessman. I will be placing strict considerations
    that may not make sense to you into this project; for example some
    people just don't get that Live/Install DVDs are not "The Answer"
    because some of us don't want to download 1.2 gigs of data for something
    that could have been done in half that. Some of the considerations are
    designed around advanced security systems with strict requirements for
    program behavior, i.e. no runtime generated code. I would not run my
    business on compromises "because it's easier this way for the short run"
    or "we probably can get away with not doing that," and I won't run a
    project based on these shoddy and hackish practices either. Things need
    to be done right, they need to be done durable, and 30 years from now we
    need to look back on the designs we put down today and say, "You know,
    we hit that ALMOST dead on; but it didn't take very long or very much
    work to get it perfect from there." If you want to see a system built
    on hack after hack after hack of what "looks good now" changing over the
    years, install Windows ME.

    - --
    All content of all messages exchanged herein are left in the
    Public Domain, unless otherwise explicitly stated.

    Creative brains are a valuable, limited resource. They shouldn't be
    wasted on re-inventing the wheel when there are so many fascinating
    new problems waiting out there.
    -- Eric Steven Raymond
    Version: GnuPG v1.4.2 (GNU/Linux)
    Comment: Using GnuPG with Thunderbird -

    -----END PGP SIGNATURE-----
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to
    More majordomo info at
    Please read the FAQ at

     \ /
      Last update: 2006-01-23 11:47    [W:0.035 / U:6.140 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site