Today I discovered another linux kernel patchset/fork that attempts to bring many features not in mainline together, called pf-kernel.

Quoting from its website:

“pf-kernel is another Linux kernel fork, that provides you with new useful features, that are not merged into mainline. It’s not based on any existing Linux fork or patchset, but some parts of Zen kernel may be merged if there’s no official release of needed patch. The name of this fork is not connected with BSD Packet Filter. «pf» means «post-factum» in the short form.”

At this point the latest patchset includes the following:
* mainline update: 2.6.35.4
* -ck patchset (BFS included)
* BFQ
* TuxOnIce
* LinuxIMQ

Its main advantage over other forks seems to be that it’s regularly updated

I’m using it on my main system without any issues, so I added it to Gentoo’s tree:

* sys-kernel/pf-sources
Available versions:
(2.6.31_p9) (~)2.6.31_p9!b!s
(2.6.32_p16) (~)2.6.32_p16!b!s
(2.6.33_p4) (~)2.6.33_p4!b!s
(2.6.34_p7) (~)2.6.34_p7!b!s
(2.6.35_p7) (~)2.6.35_p7!b!s
{build deblob symlink}
Homepage: http://pf-kernel.org.ua/
Description: Linux kernel fork with new useful features not merged into mainline

Many thanks to Oleksandr for taking the time to create and maintain it

What ties in reckless glibc unmasking GTK+ 2.20 issues Ruby 1.9 porting and --as-needed failures all together? Okay the title is a dead giveaway for the answer: undefined symbols.

Before deepening within the topic I first have to tell you about symbols I guess; and to do so, and to continue further, I’ll be using C as the base language for everyone of my notes. When considering C, then, a symbol is any function or data (constant or variable) that is declared extern; that is anything that is neither static or defined in the same translation unit (that is, source file, most of the time).

Now, what nm shows as undefined (U code) is not really what we’re concerned about; for object files (.o, just intermediate) will report undefined symbols for any function or data element used that is not in the same translation unit; most of those get resolved at the time all the object files get linked in to form a final shared object or executable — actually, it’s a lot more complex than this, but since I don’t care about describing here symbolic resolution, please accept it like it was true.

The remaining symbols will be keeping the U code in the shared object or executable, but most of them won’t concern us: they will be loaded from the linked libraries, when the dynamic loader actually resolve them. So for instance, the executable built from the following source code, will have the printf symbol “undefined” (for nm), but it’ll be resolved by the dynamic linker just fine:

int main() {
  printf("Hello, world!");
}

I have explicitly avoided using the fprintf function, mostly because that would require a further undefined symbol, so…

Why do I say that undefined symbols are our worst enemy? Well, the problem is actually with undefined, unresolved symbols after the loader had its way. These are either symbols for functions and data that is not really defined, or is defined in libraries that are not linked in. The former case is what you get with most of the new-version compatibility problems (glibc, gtk, ruby); the latter is what you get with --as-needed.

Now, if you have a bit of practice with development and writing simple commands, you’d be now wondering why is this a kind of problem; if you were to mistype the function above into priltf – a symbol that does not exist, at least in the basic C library – the compiler will refuse to create an executable at all, even if the implicit declaration was only treated as a warning, because the symbol is, well, not defined. But this rule only applies, by default, to final executables, not to shared objects (shared libraries, dynamic libraries, .so, .dll or .dylib files).

For shared objects, you have to explicitly ask to refuse linking them with undefined reference, otherwise they are linked just fine, with no warning, no error, no bothering at all. The way you can tell the linker to refuse that kind of linkage is passing the -Wl,--no-undefined flag; this way if there is even a single symbol that is not defined in the current library or any of its dependencies the linker will refuse to complete the link. Unfortunately, using this by default is not going to work that well.

There are indeed some more or less good reasons to allow shared objects to have undefined symbols, and here come a few:

Multiple ABI-compatible libraries: okay this is a very far-fetched one, simply for the difficulty to have ABI-compatible libraries (it’s difficult enough to have them API-compatible!), but it happens; for instance on FreeBSD you – at least used to – have a few different implementations of the threading libraries, and have more or less the same situation for multiple OpenGL and mathematical libraries; the idea behind this is actually quite simply; if you have libA1 and libA2 providing the symbols, then libB linking to libA1, and libC linking to libA2, an executable foo linking to libB and libC would get both libraries linked together, and creating nasty symbol collisions.

Nowadays, FreeBSD handles this through a libmap.conf file that allows to link always the same library, but then switch at load-time with a different one; a similar approach is taken by things like libgssglue that allows to switch the GSSAPI implementation (which might be either of Kerberos or SPKM) with a configuration file. On Linux, beside this custom implementation, or hacks such as that used by Gentoo (eselect opengl) to handle the switch between different OpenGL implementations, there seem to be no interest in tackling the problem at the root. Indeed, I complained about that when --as-needed was softened to allow this situation although I guess it at least removed one common complain about adopting the option by default.

Plug-ins hosted by a standard executable: plug-ins are, generally speaking, shared objects; and with the exception of the most trivial plugins, whose output is only defined in terms of their input, they use functions that are provided by the software they plug. When they are hosted (loaded and used from) by a library, such as libxine, they are linked back to the library itself, and that makes sure that the symbols are known at the time of creating the plugin object. On the other hand, when the plug-ins are hosted by some software that is not a shared object (which is the case of, say, zsh), then you have no way to link them back, and the linker has no way to discern between undefined symbols that will be lifted from the host program, and those that are bad, and simply undefined.

Plug-ins providing symbols for other plug-ins : here you have a perfect example in the Ruby-GTK2 bindings; when I first introduced --no-undefined in the Gentoo packaging of Ruby (1.9 initially, nowadays all the three C-based implementations have the same flag passed on), we got reports of non-Portage users of Ruby-GTK2 having build failures. The reason? Since all the GObject-derived interfaces had to share the same tables and lists, the solution they chose was to export an interface, unrelated to the Ruby-extension interface (which is actually composed of a single function, bless them!), that the other extensions use; since you cannot reliably link modules one with the other, they don’t link to them and you get the usual problem of not distinguish between expected and unexpected undefined symbols.

Note: this particular case is not tremendously common; when loading plug-ins with dlopen() the default is to use the RTLD_LOCAL option, which means that the symbols are only available to the branch of libraries loaded together with that library or with explicit calls to dlsym(); this is a good thing because it reduces the chances of symbol collisions, and unexpected linking consequences. On the other hand, Ruby itself seems to go all the way against the common idea of safety: they require RTLD_GLOBAL (register all symbols in the global procedure linking table, so that they are available to be loaded at any point in the whole tree), and also require RTLD_LAZY, which makes it more troublesome if there are missing symbols — I’ll get later to what lazy bindings are.

Finally, the last case I can think of where there is at least some sense into all of this trouble, is reciprocating libraries, such as those in PulseAudio. In this situation, you have two libraries, each using symbols from one another. Since you need the other to fully link the one, but you need the one to link the other, you cannot exit the deadlock with --no-undefined turned on. This, and the executable-plugins-host, are the only two reasons that I find valid for not using --no-undefined by default — but unfortunately are not the only two used.

So, what about that lazy stuff? Well, the dynamic loader has to perform a “binding” of the undefined symbols to their definition; binding can happen in two modes, mainly: immediate (“now”) or lazy, the latter being the default. With lazy bindings, the loader will not try to find the definition to bind to the symbol until it’s actually needed (so until the function is called, or the data is fetched or written to); with immediate bindings, the loader will iterate over all the undefined symbols of an object when it is loaded (eventually loading up the dependencies). As you might guess, if there are undefined, unresolved symbol, the two binding types have very different behaviours. An immediately-loaded executable will fail to start, and a loaded library would fail dlopen(); a lazily-loaded executable will start up fine, and abort as soon as a symbol is hit that cannot be resolved; and a library would simply make its host program abort at the same way. Guess what’s safer?

With all these catches and issues, you can see why undefined symbols are a particularly nasty situation to deal with. To the best of my knowledge, there isn’t a real way to post-mortem an object to make sure that all its symbols are defined. I started writing support for that in Ruby-Elf but the results weren’t really… good. Lacking that, I’m not sure how we can proceed.

It would be possible to simply change the default to be --no-undefined, and work around with --undefined the few that require the undefined symbols to be there (we decided to proceed that way with Ruby); but given the kind of support I’ve received before in my drastic decisions, I don’t expect enough people to help me tackle that anytime soon — and I don’t have the material time to work on that, as you might guess.

Observant viewers may notice that I’ve begun uploading some of the stage tarballs. The first of the stages built is the stage 1 tarball for MIPS-I little-endian. Stage 2 is building as I type this, hopefully by this weekend it’ll be uploaded and stage 3 will be on its way.

I’ll let you know when all the MIPS-I little-endian builds are done, that way you won’t wind up downloading a partially uploaded file. That said, should you jump the gun, you should be able to resume the download to fetch the remainder. I’ve started with MIPS-I since that’s the lowest-common-denominator … MIPS-III will be next, followed by MIPS-IV.

I plan to get onto the big-endian port shortly. SGI owners have not been forgotten, just I’ve been busy.

Greetings! Today we have an excellent interview of two fine gentlemen, Mr. David Gallo and Mr. Cassius Adams from SevenL Networks Inc., conducted by our very own Joshua Saddler.

For those of you that have been around awhile, you will recognize the company represented in this interview. SevenL Networks Inc. has been an infrastructure sponsor for over six years now. David and Cassius took some time to fill us in on their use of Gentoo throughout their extensive operations. We hope you enjoy reading the complete interview which you will find here.

Discuss this!

The Gentoo Prefix project is still alive and still kicking. There has not been any major noteworthy highlights so you may not heard from us in some time. The number of users increases and the number of active contributors seems to stay the same or increase much more slowly. Gentoo Prefix was the reason I become a Gentoo Linux developer, so get involved..it is an easy gateway to being a Gentoo Developer if you are interested.

Some interesting things to note that I have been working on:

• My ~x86-linux binpkg repo for Gentoo Linux hosts is still running every night. I use this to easily find simple build errors in packages before they hit too many users.
• I started a ~amd64-linux binpkg repo to add coverage to my nightly automated testing. So, I’ve updated the instructions from the above post. This means that you can install your very own Gentoo Prefix installation on a Gentoo Linux host in 5 minutes for 32bit or 64 bit now.
• While those are updating everynight. I am now bootstrapping everynight too. I set up a small script that debootstraps a Debian Lenny chroot and then sets up an Gentoo Prefix inside the chroot. This will help finding bootstrapping bugs that brand new users may hit. Often times bootstrapping is more sensitive/fragile to tree changes than just updating.
• We are still migrating packages from the Gentoo Prefix tree to the Gentoo Linux tree. This is going slower than planned but there are not too many people working on it. Current: Over ~2000 packages migrated, still over 700 to go in our overlay. (Gentoo Prefix tree has over 7000 packages in it, but not all are tested/keyworded.)

For a stabilisation request of VLC 1.1.4, ffmpeg 0.6 is needed. So far I have checked all build dependencies and sorted out everything, but still all runtime dependencies need a check. So if you know a program that calls ffmpeg during operation and the current stable fails with ffmpeg 0.6, please tell me. Either by email or in the comments. Thank you very much for your support.

The lives of distributions packagers are full of words that make them cringe – backport, regression, hotfix, custom patch, … – but there are two that can make your day truly shine: merged upstream.

When you’re maintaining a package for any kind of software, you have to mediate between the original upstream intentions and requests, and the distribution policy; in Gentoo, that policy includes respecting the user-chosen flags, especially since some of those are mandated by us and are needed to make sure that the software installed on the users’ systems is actually working as intended, but that’s just one of a long list of things you have to care about.

Most of the time, because of this, you have to end up patching the software itself: modifying the code so that it behaves like the distribution wants, even if that diverges from upstream behaviour, or in the best of cases, making it abide to both restrictions. Sometimes, you have to take a fix that has been already applied to the upstream repository, for instance in a development branch, and apply to the currently packaged version (backport). The least boring case is when users report a problem to you, that might or might not apply to other distributions, and you have to fix it anew.

When that’s the case, Gentoo’s guideline is to write patches so that they can be sent upstream (I also wrote about that — and I’m tempted to just re-publish my articles on my website rather than keep them there, especially given that the page is more broken each time I visit it). Unfortunately, some upstream are more difficult to work than others, plus sometimes the patches are really too Gentoo-specific that they make no sense to be sent upstream (for instance the S/Key patch for sudo, as it’s to support the Gentoo-specific port of OpenBSD’s S/Key support).

Now, as part of the Ruby team I’ve already written over and over about our need to patch a huge number of gems and other Ruby libraries, a lot of times simply to fix their Rakefile, less often to fix their tests… and in very rare occasions to hack or unhack the packages. Thankfully, these patches tend to be merged in quite quickly — I’m still not sure whether that’s due to the use of GitHub and of merge requests, or because releasing a Ruby gem is so quick and easy (which is a definite pro of RubyGems even for us packagers).

With other kind of software, the merge-and-release approach is not that common, but luckily there are exceptions; Robin Gareus has been terrific at merging my patches for liboauth and release a new version, which means that while I added it to the tree with three patches to be applied, you can get it now with no patch at all: vanilla 0.8.9!

Less quick to release, but that’s understandable considering the criticality of it, has been Linux-PAM; the newly released version, 1.1.2, which I added to the tree today, comes back to two patches, against the previous six of 1.1.1-r2. The remaining two are a bit tricky; one is something I’m keeping around since the 0.99 series, so a very long time ago, and is just a simple way for us to not build a bunch of test programs that we wouldn’t be using anyway; the other is a fix for the Berkeley DB detection in configure to work with the libraries are installed in Gentoo (with the prefix on the library name, but no prefix on the ELF symbols; we use versioning instead to avoid collisions between them). I’m now trying to find a compromise with Thorsten Kukuk (my upstream Linux-PAM contact) so that the patches can be applied, and we can stop patching it altogether.

Being able to ship packages that are not patched at all is important for many reasons, and at least one applies even to people who don’t use that package at all. Obviously, staying closer to upstream’s code is a positive thing because it means that you don’t risk that upstream counts your users out of support range (well, they can still do that if you are using non-default build options, but in general, it’s much easier for them to help you out if you don’t touch their code), but sending your own fixes to the original developers has another important result: the same fix is available to all the users of the program, not just those using your particular distribution. And finally, even people not using it will have a positive result, as long as they use Gentoo: less patch files in the tree means less files, and less overhead — and trust me, there is lots of overhead in the tree as it is right now, not sure if worse or better than what we had at the time I originally wrote that rundown, but it certainly needs some kind of proper solutions to be devised.

So I’m happy to thank for their merges Robin (liboauth), Thorsten (Linux-PAM), Karel (util-linux-ng), Cole (virt-inst), Thibault (fcron), Ludovic (ccid), … — the list goes a lot further, they are just the most recent upstreams I’ve exchanged email with!

On a side note; yes I picked up co-maintainership of bti with Greg; the reason is that I’ve been using it to dent the tinderbox results. I’ve also branched it upstream to cleanup a few things in the build system, and to implement one feature I’d very much like to use (--background); those fixes will come straight on 028 release, though, as they are not critical.

Well, after a long hiatus and a few battles more recently, I now have the Qube2 building O32 stages for the little-endian port of Gentoo.

N32 will probably come soon with Matt Turner throwing some Broadcom BCM91250 heavyweights at the task. I intend to keep O32 going as there’s a lot of hardware out there that cannot use N32 that people use Gentoo on, and there are also a lot of systems which still run Gentoo O32. To provide a continued operating system for these systems, I’ll continue to maintain an O32 port.

At this point… I have produced a seed stage 3 based on modern sources, and have the Qube2 busily working through the final stage 1 for MIPS1 now (at package 55 of 60). I have the occasional hiccup such as baselayout trying to install a .keep file in a mounted /proc and dying… I’ve also had the man ebuild try to call groupadd before shadow was installed.

Other than that, things are moving forward, and I hope to have new stages to you shortly.

Currently, the releases 3.0 , 3.2, 3.4 and 3.5 of samba are available in Gentoo, but upstream has discontinued releases 3.0 and 3.2 for a while, and currently they are actively maintaining 3.4 and 3.5 .
For the above reason and in order to avoid security issues, I'm planning to drop releases 3.0 and 3.2 from the tree soon, but I'm interested in to know why you, fellow users, are using those old releases. Haven't you updated your system? Perhaps something is not working right for you with currently stable samba 3.4?
Let me know your thoughts

One of the nicest features of OpenRC/Baselayout 2 (that sooner or later will hit stable I’m sure) is that you can replace bash (which is slow, as its own documentation admits) with a faster, slimmer pure POSIX shell. Okay, so maybe Fedora is now moving away from shells altogether and I guess I can see why from some point of views; but as Nirbheek said it’s unlikely to be suitable for all use cases; in particular, our init system tends to work perfectly fine as is for servers and embedded systems, so … I don’t see the reason we should be switching there. Adding support for dash or any other POSIX sh-compatible fast shell is going to be a win-win situation there — do note that you still need bash to run ebuilds, though!

Now, you can use dash already for the basic init scripts provided by OpenRC and Baselayout, but all the packages need to install proper Posix SH-compatible init scripts if you want to use it with them. Thankfully a number of users seem to care about that, such as Davide last year and Kai right now.

But POSIX compatibility is not the only thing we should actually look out for in our init scripts:

• Eray pointed out that some scripts don’t re-create their /var/run subdirectories which is indeed a problem that should go fixed at some point; I had similar bad issues with running my own router based on Gentoo;
• one too often misused parameter of start-stop-daemon is --quiet which can be… way too quiet; if it’s passed, you’re not going to receive any output at all if the daemon you’ve tried to start fails; and that is a problem;
• there are problems with the way the system-services PAM chain is passed through so that limits are not respected (and if that’s the case, caps wouldn’t be respected coming from PAM either);
• the way LXC works, init scripts looking just at the process’s name could cause a guest’s daemon to stop if the host’s is closed… this is mostly exercised by killall but also start-stop-daemon when not given a pidfile (and rather just an executable) will have the same problem; and the same goes for pkill, goes without saying.

These are a number of polishing tasks that are by all count minors and not excessively important but… if you’ve free time and care about Gentoo on LXC, embedded or with fast startup, you might want to look into those. Just saying!

• keep on ignoring requests coming from a QA team member;
• keep on de-CCing QA on your bugs when said QA team member state that the fix is the wrong one, just stating that “it’s the wrong place open a new bug” when your solution was decided there;
• keep on complaining if less than 1% of bugs filed, out of literally thousands lack a log file;
• keep on asking me to not use the f-word because it makes it bad for you to be associated with Planet Gentoo (but on the other hand, feel no harm in being associated with people who repeatedly made Gentoo unusable for its users);
• keep on spitting on me for pointing out that your unmask ideas are between reckless and totally stupid;
• keep ignoring the bugs that are reported for your package;
• keep bumping packages you don’t maintain, without looking into the further QA-related issues, and without declaring yourself the maintainer;
• keep repeating the same mistakes and when asked to revise your attitude use the “but he did it as well” card.

Keep it up this way, then look back to see if there is QA at all.

The latest dev channel release, www-client/chromium-7.0.503.1, builds with use_system_icu=1, which means one less bundled library. It also seems that the binary size of /usr/lib/chromium-browser/chrome is now smaller.

We have still quite a lot of bundled libraries though, but we're really close to building with system sqlite. I'm working on upstream patches (both Chromium and SQLite) to make this possible.

Please note that as www-client/chromium uses more and more system libraries, it will also require more testing on the Gentoo side, because upstream QA only tests the bundled configuration.

I’ve been pushing the tinderbox one notch stricter from time to time; a few weeks ago I set up the tinderbox so that any network access beside for the basic protocols (HTTP, HTTPS, FTP and RSYNC) was denied; the idea is that if the ebuilds try to access network by themselves, something is wrong: once the files are fetched, that should be enough. Incidentally, this is why live ebuilds should not be in the tree.

Now, since I’ve received a request regarding the actual network traffic issued by the tinderbox, I decided to go one step further still, and make sure that beside for the tasks that do require network access the tinderbox does not connect to anything outside of the local network. To do so, I set up a local RSync mirror, then added a squid passthrough proxy, that does not cache anything; at that point, rather than allowing some protocols on the router for the tinderbox, I simply reject anything originating from the tinderbox to access Internet; all the outgoing connections originating from the tinderbox are done through Yamato, so I have something like this in my make.conf:

FETCHCOMMAND="/usr/bin/curl --location --proxy yamato.local:3128 --output \${DISTDIR}/\${FILE} \${URI}" 
RESUMECOMMAND="/usr/bin/curl --location --proxy yamato.local:3128 --continue-at - --output \${DISTDIR}/\${FILE} \${URI}"

Note: googling on how to set up those two variables in Gentoo to use curl I did find some descriptions on the Gentoo Forums that provide most of them; unfortunately all I found ignore the --location option, which makes it fail to fetch stuff from the SourceForge mirrors and any other mirroring system that uses 302 Moved responses.

I also modified the bti-calling script so that the identi.ca dents are sent properly through the proxy. I didn’t set the http_proxy variable, because that would have made moot the sealing. Instead, by setting it up this way, explicitly for the fetch and dent, if any testsuite tries to fetch something, even via HTTP, will be denied.

But… why should it be a problem if testsuites were to access services on the network? Well, the answer is actually easy once you understand two rules of Gentoo: what is not in package.mask is supposed to work, and any bug found needs to be fixable, and testsuites results need to be reproducible, to make sure that the package works. When you rely on external infrastructure like GIT repositories, you have no way to make sure that if there is a problem it can be fixed; and when your testsuite relies on remote network services, it might fail because of connection problems, and it will fail if the remote service is closed entirely.

I’ve also been tempted to remove IPv4 connectivity from the tinderbox at all; IPv6 should well be enough given that it only needs to connect to Yamato, and it would be under NAT anyway..

A lot of folks are raving about the next generation in init systems (aka systemd), and how it's (almost certainly) going to be the default init system for Fedora 14 (paid article, subscribe to LWN to read! [or wait a week]). It also seems that OpenSuse will be moving to systemd sometime in the near future (don't take my word for this though), and Debian has at least considered it. It is also well-known that Ubuntu will not be using systemd for the foreseeable future.

So where is Gentoo in all this? Our current init system is baselayout-1 in the stable tree, and openrc in the ~arch tree. The maintainer-wanted bug for systemd has been quite active with users posting preliminary ebuilds for it. The bug itself currently has >30 folks CCed (including me), and 86 votes. So users are definitely very interested in seeing systemd in Gentoo. However, it will take a lot of work before systemd can enter portage even as a masked ebuild.

Even after systemd enters portage, it is extremely unlikely that it will become the default init system for reasons that are listed below. Some developers are strongly in favour of moving from baselayout-1 to systemd, while some think it's a pile of crap that Gentoo should stay far away from. Neither of these opinions is shared by the majority of Gentoo devs. (that includes me :-)

In all likelihood, the end result will be that openrc will finally go stable (replacing baselayout-1), and if any developers are willing to spend the massive amount of time and effort required to make systemd usable in Gentoo, systemd will become an optional init system, strongly recommended for desktops/laptops.

Now why can't we throw out baselayout-1 as well as openrc and just use systemd? I was going to make a full list of the reasons, but as I was making it, I realised that I don't know enough details about systemd's requirements, what all it provides, what parts of Baselayout would need to be rewritten, how much porting of the tree (and systemd) would be needed, etc. So instead of hand-waving, I'll just list "needs several volunteer developers" as the blocker for now :)

I'm tempted to list myself as a future volunteer, but I won't do such a thing yet. Rest assured that if I do end up working on this, I'll be sure to blog about it. Although it is probably just a matter of "time" ;)

After my ranting about QA – which brought back our lead from hiding, at least for a while, let’s see how he fares this time – a number of people asked me what they could do to help QA improve. Given that we want both to test and prevent, here comes a list of desirable features and requests that could obviously be helpful:

• as usual, providing for another tinderbox would be a good thing as more situations could be tested; right now I’d very much like to have a server powerful enough to run a parallel tinderbox set up to test against stable-tree amd64 keyword, rather than the current unstable x86;
• repoman right now does not warn about network errors in HOMEPAGE or SRC_URI; see this bug as for why; it would be a good thing to have;
• again repoman, lacking network checks, lack a way to make sure that the herd specified in metadata.xml exists at all; it’s pretty important to do that since it happened before that herds weren’t updated at all;
• there is also no way to ensure that metadata.xml lists maintainers that have accounts in Bugzilla; with proxy-maintainership sometime it happens that either the email address is not properly updated (proxier fail) or the maintainer does not have an account on Bugzilla at all (which is against the rules!); unfortunately this requires login-access to bugzilla, and is thus a no-no right now;
• the files/ subdirectories are still a problem for the tree, especially since they are synced down to all users and should, thus, be kept slim, if not for the critical packages; as it turns out, between Samba and PostgreSQL they use over half a megabyte for files that a lot of people wouldn’t be using at all; we also had trouble with people not understanding that the 20KB size limit on files does not mean you can split it in two 10KB files, nor compress it to keep it in tree (beside the fact we’re using CVS, there is a separate patch repository for that; Robin is working on providing a reliable patch tarball hosting as well); see bug #274853 and bug #274868
• moving on to bugzilla, there is currently no way to display inline on the browser logs that are compressed with gzip, that the new Portage can generate and make the tinderbox work much easier; this makes bug reports a bit more complex to manage; if somebody knows how to get Bugzilla to provide proper headers so that the browsers can read them, it’d be pure magic;
• right now we lack a decent way to test for automagic dependencies; you could make use of something similar to my oneliner but it won’t consider transitive dependencies on virtual packages;
• with split-unsplit-split-unsplit cycles, it happens that a number of packages are in tree and are now obsoleted – this is why I had to remove a number of geda ebuilds the other day – removing these packages not only reduces the pressure on the tree itself, but also allows the tinderbox to run much more smoothly without me having to mask those packages manually: when they are visible, the tinderbox might end up trying to merge them by removing the (newer) un-split ebuild, or vice-versa;
• one more Portage (non-repoman!) feature that would be nice: per-package features or at least per-package “maintainer mode”: while Portage has a number of features to make sure that maintainers don’t commit crap (testsuite support; die-on-some-warnings; fix-or-die) – such as the stricter feature – but using them system-wide is unfeasible (I’m okay with fixing my own stuff, but I don’t want an unusable system because some other developer didn’t fix his).

During last two days I managed to install my HTPC (soon I will add printers and samba to it). There is a lot of useful information around the internet which helped me on the way. I really like to create Gentoo from scratch logs, so I would like to share it with you together with used patches (in my iElectric gentoo repository).

DISCLAIMER: This is far from perfect (even the ebuilds), but it works;)

DISCLAIMER2: I'm aware this is not the noobs guide, it's more of a reference manual

# some stolen from http://en.gentoo-wiki.com/wiki/Acer_Aspire_Revo_R3600_/_R3610#VDPAU

* ping google.com
* fdisk /dev/sda
# boot partition
n
p
1
<enter>
+30M
# swap partition
n
p
2
<enter>
+1G
# root partition
n
p
3
<enter>
<enter>
w

# create filesystems
mkfs.ext2 /dev/sda1
mkswap /dev/sda2
mkfs.ext4 /dev/sda3

# activate swap
swapon /dev/sda2

# prepare/mount chroot
mount /dev/sda3 /mnt/gentoo
mkdir /mnt/gentoo/boot
mount /dev/sda1 /mnt/gentoo/boot
cd /mnt/gentoo

# stage3/portage download (use d shortcut for download)
links http://www.gentoo.org/main/en/mirrors.xml
server->releases->x86->current stage3 i686 + DIGEST + CONTENTS
server->snapshosts->portage-latests + digest

# validate size
md5sum -c stage*.DIGEST
md5sum -c portage*.md5sum

# install portage/stage3
tar xvjpf stage3-*.tar.bz2
tar xvjf portage-latests.tar.bz2 -C /mnt/gentoo/usr

# select mirrors (in terminal 2)
mirrorselect -i -o >> /mnt/gentoo/etc/make.conf
mirrorselect -i -r -o >> /mnt/gentoo/etc/make.conf
# add slovenian mirror

# copy dns settings
cp -L /etc/resolv.conf /mnt/gentoo/etc/

# create local portage overlay
mkdir /mnt/gentoo/usr/local/portage

# edit make.conf (back in terminal 1)
vim /mnt/gentoo/etc/make.conf

# chroot
mount -t proc none /mnt/gentoo/proc
mount -o bind /dev /mnt/gentoo/dev
chroot /mnt/gentoo /bin/bash
env-update
. /etc/profile

# refresh tree
emerge --sync

# select profile
eselect profile list
eselect profile set #n

# emerge must-have stuff under install
emerge -v vim eix pciutils gentoo-sources

# setup slovene locale (terminal 2)
vim /etc/locale.gen
vim /etc/env.d/02locale
locale-gen

# setup slovene keymap
vim /etc/conf.d/keymaps

# setup zone
cp /usr/share/zoneinfo/Europe/Ljubljana /etc/localtime

# setup clock
vim /etc/conf.d/clock

# setup hostname
vim /etc/conf.d/hostname (server.domain.si)

# add basic startup scripts

rc-update add net.eth0 default
rc-update add sshd default

# install distcc
# http://www.gentoo.org/doc/en/distcc.xml
# http://www.gentoo.org/doc/en/cross-compiling-distcc.xml
emerge -av distcc
vim /etc/distcc/hosts
# do the magic in /usr/lib/distcc/bin/

# compile kernel
lspci (terminal 2)
cd /usr/src/linux
make menuconfig

- kernel settings
Procesesor type and features -->
  (Intel Atom) Processor family
Power management and ACPI options -->
  CPU frequency scaling -->
    everything
Device drivers -->
  Serial Ata and Parallel ATA drivers -->
    NVIDIA SATA support
  Networking drivers ->
    Ethernet (10 or 1000mbit) -->
      [M] nForce Ethernet support
    Wireless LAN -->
      Atheros Wireless Cards -->
        [M] everything
  Sound card support -->
    ALSA -->
      PCI Sound devices -->
        Intel HD Audio -->
          [*] everything
  [ ] Multiple devices driver support -->
    
File systems ->
  [*] Ext2
  [*] Ext4
  [*] FUSE

# compile kernel and place it
make && make modules_install
cp arch/i386/bzImage /boot/kernel-

# add modules to startup
vim /etc/modules.autoload.d/kernel2.6

# set root password
emerge -av pwgen dhcpcd grub
pwgen
passwd

# create user
# gost==guest
useradd -m -G users,wheel,video,cdrom -U -s /bin/bash ielectric
useradd -m -G users,video,cdrom -U -s /bin/bash gost

# setup grub conf
vim /boot/grub/grub.conf

# setup fstab
vim /etc/fstab

# setup MBR
grub --no-floppy
root (hd0,0)
setup (hd0)
quit

# cleanup and reboot
rm /portage*
rm /stage*

reboot

### after basic OS setup

# on local machine setup hosts
vim /etc/hosts
# back to server

# setup rc.conf
vim /etc/rc.conf

# local: copy ssh key over
cat ~/.ssh/id_rsa.pub | ssh server.domain.si "cat - >> ~/.ssh/authorized_keys"
# back to remote
chmod 744 /home/ielectric/.ssh/autorhized_keys

# setup sshd
vim /etc/ssh/sshd_config

# system update
eix-sync
emerge -DuNav world

# install autounmask flagedit
emerge -av autounmask gentoolkit flagedit

# eix and screen install
emerge -av eix screen
echo "*" > /etc/eix-sync.conf

# layman
flagedit app-portage/layman +subversion +git +mercurial
emerge -av layman
layman -a iElectric
layman -a sunrise

# ntp
emerge -av ntp
/etc/init.d/ntpd start
/etc/init.d/ntp-client start
rc-update add ntpd default
rc-update add ntp-client default

# update startup script
rc-update add acpid default
rc-update add nscd default
rc-update add dbus default
rc-update add hald default
rc-update add consolekit default

# bacula client
flagedit app-backup/bacula +bacula-nodir +bacula-nosd +bacula-clientonly
emerge -av bacula
# update directory name in bacula-df.conf
# update bacula-dir.conf on central server
/etc/init.d/bacula-fd start
rc-update add bacula-fd default

# logrotate, syslog-ng, vixie-cron, slocate
emerge -av slocate logrotate syslog-ng vixie-cron

rc-update add syslog-ng default
rc-update add vixie-cron default

# munin
echo "net-analyzer/munin postgres ssl" >> /etc/portage/package.use
echo "x11-libs/cairo svg" >> /etc/portage/package.use
emerge -av munin
sudo -u munin munin-node-configure --shell | bash
# update allowed host in /etc/munin/munin-node.conf
# update munin.conf on central server
/etc/init.d/munin-node start
rc-update add munin-node default

# update cronjob
crontab -e
slocate, eix-sync

# random stuff
emerge -av links mailx ntfs3g

# xbmc
# ebuild should be taken from iElectric overlay
autounmask -a media-tv/xbmc-9.11-r5
emerge -av xdm gdm gnome-session xbmc
rc-update add xdm default

# set default session to xbmc
cd /etc/X11/Sessions/
ln -s /usr/bin/xbmc-standalone
vim /etc/rc.conf

# switch to nvidia opengl implementation
eselect opengl set 1

# give guest user permissions for reboot/shutdown..
polkit-auth --user gost --grant org.freedesktop.hal.power-management.reboot
polkit-auth --user gost --grant org.freedesktop.hal.power-management.shutdown
polkit-auth --user gost --grant org.freedesktop.hal.power-management.suspend

# install transperancy skin
cd /home/gost/.xbmc/skin/
svn checkout http://transparency-xbmc.googlecode.com/svn/trunk/ Transparency

# install svn repo installer
http://marshalleq.wordpress.com/2009/08/22/how-to-install-svn-installer-on-xbmc-live-to-enable-automatic-skin-and-plugin-upgradesinstalls/

# install OpenSubtitles OSD
http://code.google.com/p/opensubtitles-osd/wiki/Instalation
# go to scripts and setup the plugin
# go to skin settings and enable subtitle OSD plugin -> default.py

# install remote support
# ebuild should be taken from iElectric overlay
# BIG BIG THANKS to
# http://forum.sabayon.org/viewtopic.php?f=56&t=20113&start=10#p114286
autounmask -a app-misc/lirc-0.8.6-r4
echo 'LIRC_DEVICES="wpc8769l"' >> /etc/make.conf
emerge -av lirc
rc-update add lirc default
echo "lirc_wb677" >> /etc/modules.autoload.d/kernel-2.6

# check if everything boots correctly
reboot

# TODO: list of all conf files
# TODO esata?

UPDATE: fixed typo and added dummy module for lirc

Testing an SSL service or TLS enabled service isn't all that easy using common techniques: using telnet or netcat you can only see that something answers on a given port. However, you'll get no useful information regarding the certificates in involved that way. OpenSSL and GNUTLS provide tools to help with that.

Testing SSL servers (in this example, https)

OpenSSL has the function s_client which can be used to get the information we want:

$ openssl s_client -connect www.ccc.de:443
CONNECTED(00000003)
depth=0 C = DE, ST = Hamburg, L = Hamburg, O = Chaos Computer Club...
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C = DE, ST = Hamburg, L = Hamburg, O = Chaos Computer Club...
verify error:num=27:certificate not trusted
verify return:1
depth=0 C = DE, ST = Hamburg, L = Hamburg, O = Chaos Computer Club...
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:/C=DE/ST=Hamburg/L=Hamburg/O=Chaos Computer Club e.V./CN=...
   i:/O=CAcert Inc./OU=http://www.CAcert.org/CN=CAcert Class 3 Root
---
Server certificate
-----BEGIN CERTIFICATE-----
< Certificate here >
-----END CERTIFICATE-----
subject=/C=DE/ST=Hamburg/L=Hamburg/O=Chaos Computer Club ...
issuer=/O=CAcert Inc./OU=http://www.CAcert.org/CN=CAcert Class 3 Root
---
No client certificate CA names sent
---
SSL handshake has read 1839 bytes and written 409 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 1024 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : DHE-RSA-AES256-SHA
    Session-ID: AF312E52F8D5F2B6441436F0D666588FD4C5...
    Session-ID-ctx:
    Master-Key: FF103B4A274E8B90E905AA07C6DB45AB3F32...
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    TLS session ticket:
        < hexdump here >
    Start Time: 1282561578
    Timeout   : 300 (sec)
    Verify return code: 21 (unable to verify the first certificate)
---

Now you're on a secured connection and can use HTTP as you're used to (i.e. GET, POST etc). Not that the output above also allows you to find out what certificates are in use, who issued them and when they will expire.

Testing TLS enabled services (here: SMTP+TLS)

While TLS can operate in the same way that SSL does (encryption starting with the very first bit transmitted), it also has a mode in which you first establish the connection and then secure it later on. In the case of SMTP, this happens after issuing EHLO bys sending "STARTTLS" (if the server advertises it in response to EHLO).

$ gnutls-cli -s mx.example.com -p 25
Resolving 'mx.example.com'...
Connecting to '192.168.0.1:25'...

- Simple Client Mode:

220 mx.example.com ESMTP Exim 4.69 Mon, 04 Aug 2008 11:59:34 +0200
EHLO client.example.com
250-mx.example.com
250-SIZE 52428800
250-PIPELINING
250-AUTH CRAM-MD5
250-STARTTLS
250 HELP
STARTTLS
220 TLS go ahead

At this point, the server expects the client (us) to initiate the TLS handshake. Since the GNUTLS client itself doesn't know this (and can't, because every protocol is different), we have to tell it to go into TLS mode. This is accomplished by sending it the ALARM signal (SIGALRM, usually #14). This can be done easily from another terminal:

$ pkill -ALRM gnutls-cli

As a result, the client does all the crypto stuff required and tells us which certificates it has encountered along the way:

*** Starting TLS handshake
- Ephemeral Diffie-Hellman parameters
 - Using prime: 776 bits
 - Secret key: 760 bits
 - Peer's public key: 768 bits
- Certificate type: X.509
 - Got a certificate list of 1 certificates.

 - Certificate[0] info:
 # The hostname in the certificate matches 'mx.example.com'.
 # valid since: Fri Jan 18 09:22:21 CET 2008
 # expires at: Wed Jul 16 10:22:21 CEST 2008
 # fingerprint: A6:8E:2F:1B:03:36:A8:BA:CF:9F:37:0E:53:7E:D0:4A
 # Subject's DN: CN=mx.example.com
 # Issuer's DN: O=Root CA,OU=http://www.cacert.org,CN=CA ...

- Peer's certificate issuer is unknown
- Peer's certificate is NOT trusted
- Version: TLS1.0
- Key Exchange: DHE-RSA
- Cipher: AES-128-CBC
- MAC: SHA1
- Compression: NULL

As with SSL, you now have a secured connection that you can use like any other SMTP connection (or exit):

ETRN
250 OK
QUIT
221 'mx.example.com' closing connection
- Peer has closed the GNUTLS connection
$

Security note

While both clients check the certificate for internal validity and expiry date, they can't do anything in the way of certificate chain validation unless you provide appropriate root certificates). This means that while they tell you the contents of the certificate, they usually can't tell you whether the certificate's signature is trusted in any sense. Hence, they tell you that they're not trusted. SSL does this with the line verify error:num=27:certificate not trusted, TLS does so with Peer's certificate is NOT trusted.

GSoC 2010 is history, but more and more is going on.

Let me rephrase a bit what was my project over this summer, what does it provide and what will the future bring.

gpypi2 has become quite more than it's primary goal, installation of Python packages has never been easier:

• Create/echo Gentoo Linux ebuilds from Python Package Index or from Python source (with help of distutils)
• Install that ebuild with invoking emerge
• Create overlay from all working packages on Python Package Index

Install

Very easy.

$ layman -f -o http://gpo.zugaina.org/lst/gpo-repositories.xml

### unmask gpypi2, easiest with autounmask
$ autounmask -a dev-python/gpypi2-9999

$ emerge -av gpypi2

gpypi2 -h for list of commands. Please report bugs to BitBucket issue tracker

Not over yet

My work at Gentoo. I will be back.

Special thanks to my mentor Jesus "neurogeek" Rivero, without him it would be impossible.

In general, I’m not an expert sysadmin; my work usually involves development rather than administration, but as many other distribution developers, I had to learn system administration to make sure that the packages do work on the users’ systems. This gets even messier when you deal with Gentoo and its almost infinite amount of combinations.

At any rate, I end up administering not only my local systems, but also two servers (thanks to IOS Solutions who provides xine with its own server for the site and bugzilla). I started using lighttpd, but for a long series of circumstances I ended up moving to Apache (mostly content negotiation things). I had to learn the hard way about a number of issues — luckily security was never involved.

My setup moved from a single lighttp instance to one Apache that kept running two static websites, one Bugzilla and one Typo instances, to two Apache on two servers, one running a static website and a Bugzilla instance, the other running a few static websites and a Typo instance via passenger. The latter is more or less what I have now.

From one side, Midas is keeping up the xine website (static, generated via XSLT after commit and push); from the other, Vanguard – the one I pay for – keeps this blog my website and a few more running. I used to have a gitweb instance (and Gitarella before that), but I stopped providing the git repositories myself, much easier to push them to Gitorious or GitHub as needed.

The static websites use my own generator for which I still have to find a proper license. Most of these sites are mine or simply of friends of mine, but with things changing a bit for me, I’m going to start offering that as a service package for my paying customers (you have no idea how many customers would just be interested in having a simple, static page updated once in a few months… as long as it looked cool).

But since I have, from time to time, to stop Apache to make changes to my blog – or in the past Passenger went crazy and Apache stopped from answering to requests at all – I’m not very convinced about running the two alongside for much longer. I’ve then decided it was a good idea to start figuring out an alternative approach; the solution I’m thinking of requires the use of two Apache instances on the same machine; since I cannot use different ports for them (well, I could run my blog over 443/SSL but I don’t think that would be that good of an idea for the read-only situation), I’ve now requested a second IP address (the current vserver solution I’m renting should support up to 4), and I’ll run two instances with that.. over the two different IP addresses.

Now, one of the nice things of splitting the two instances this way is that I don’t even need ModSecurity on the instance that only serves the static sites; while they are not really as static as a stone (I make use of content negotiation to support multiple languages on the same site, and mod_rewrite to set the forcing), there is no way that I can think of that any security issue is triggered while serving them. I could even use something different from Apache to serve them, but the few advanced features I make use of don’t make it easy to switch (content negotiation is one, another is rewritemaps to recover moved/broken URLs). And obviously, I wouldn’t need Passenger either.

But all the other modules? Well, those I’d need; and since by default they are actually shared modules (I have ranted about that last November), loading two copies of them means duplicating the .data.rel and the other Copy on Write sections. Not nice. So I finally bit the bullet and, knowing that Apache upstream allows using them as builtin, I set out to find if the Gentoo packaging allows for that situation. Indeed it does, but by mishandling the static USE flag which made it quite harder to find out. After enabling that one, disabling the mem_cache, file_cache and cache modules (that are not loaded by default but are still built, and would be built-in when using the static USE flag), and restarting Apache, the process map looked much better, as now the apache2 processes have quite less files open (and thus a much neater memory map).

One thing that is interesting to note: right now, I’ve not been using mod_perl for Bugzilla because of the configuration trouble; one day I might actually try that. Possibly with a second Apache instance on Midas, open only on SSL, with a CACert certificate.

Now it might very well be possible that you were to need a particular module only in one case, such as mod_ssl to run a separate process for an SSL-enabled Apache 2 instance… in that case, one possible solution, even though not extremely nice, is to use the EXTRA_ECONF trick that I already described.. in this case, you could create a /etc/portage/env/www-servers/apache file with this content:

export EXTRA_ECONF="${EXTRA_ECONF} --enable-ssl=shared"

On a separate note, I think one of the reasons why our developers let the default be dynamic modules is more related to the psychology of calling it “shared”. It makes it sound like it’s wasting memory when you have multiple processes using a “non-shared” module.. when in reality you’re creating much more private memory mappings with the shared version. Oh well.

Unfortunately, as it happens, the init system we have in place does not allow for more than one Apache system to be running; it really requires different configuration files and probably a new init script, so I’ll have to come back to this stuff in the next days, for the remaining parts.

There are though three almost completely unrelated notes that I want to sneak in:

• I’m considering a USE=minimal (or an inverse, default-enabled, USE=noisy) for pambase; it would basically disable modules such as pam_mail (tells you if you have unread mail in your queue — only useful if you have a local MDA), pam_motd (gives you the Message of the Day of the system) and pam_tally/pam_lastlog (keep track of login/su requests). The reason is that these modules are kept loaded in memory by, among others, sshd sessions, and I can’t find any usefulness in them for most desktop systems, or single-user managed servers (I definitely don’t leave a motd to myself).
• While I know that Nathan complained to me about that, I think I start to understand why the majority of websites seem to stick with www or some other third-level domain: almost no DNS service seem to actually allow for CNAME to be used on the origin record (that is, the second-level domain); this means that you end up with the two-levels domain to point directly to an IP, and changing a lot of those is not a fun task, if you’re switching the hosting from one server to another.
• CACert and Google Chrome/Chromium don’t seem to get along at all. Not only I’ve been unable to tell it to accept the CACert root certificate, but while trying to generate a new client certificate with it, the page is frozen solid. And if I try to install it after generating it with Firefox, well… it errors out entirely.

I'm not sure why it hasn't been posted before, or maybe it has: there's a tracker bug for portage migration from cvs to git. Things start to get more real.

By the way, if you were reading my posts on gentoo-dev, that's the kind of project visibility I like to see.

So a week or so ago I masked Webmin, because it dinged a lot on my tinderbox and I decided to take a look at it. The ChangeLog shows a very bleak story: webmin hasn’t had a dedicated maintainer since March 2008; that’s two and a half years ago; the last five versions were bumped by Patrick (who, if you should be reminded, rarely picks up the pieces of what he might break); last October, Victor “fixed” a bunch of sandbox violations in the 1.490 version.

Given the hits on the tinderbox and the above-noted history; I decided to mask it until somebody actually stepped up to maintain it; this was also positively acknowledged by the security team: having a package with the security issue of Webmin lacking a dedicated maintainer is calling for trouble.

Since then I received a long list of email messages of users using Webmin, either in production or for local environments, wondering why I masked it and insisting that it is in its design to access files owned by other packages anyway. I guess I’ll have to rephrase the masking reason given that, that wasn’t what I meant.

It is a common, enforced policy in Gentoo that all the packages are self-contained, and mess the least possible with the running system as they can, so that if you just emerge MySQL, it won’t start up right away, or even at the next reboot; this is one thing that put Gentoo drastically in contrast with most other distributions. And what a lot of people, me included, love the most.

If there are commands to execute to properly setup the package before making use of it, we have the special pkg_config() function that can be called through emerge --config — there aren’t many packages doing that though.

Instead, webmin-1.510.ebuild goes the totally wrong way: it tries to second-guess the user’s setup; it runs the setup step of Webmin directly within src_install() which should be protected by the sandbox; but Victor’s “fixes” actually simply allowed the ebuild to access what it shouldn’t be, not at that state at least: real devices, kernel modules, and the cron configuration.

How did I notice that? Very simple, actually: the tinderbox, just like my own system, uses fcron as cron daemon. The fcron configuration file is not predicted within the ebuild, so it still triggered sandbox violation, and caused my tinderbox to start screaming at me like my mother when I come back at 6am (don’t get me started).

Now, if that was the only problem it wouldn’t be much; but the ebuild also fail to actually provide a decent/stable PAM configuration, and a lot of the shell code in the ebuild file is just… icky.

What can be done to save webmin? Well, more than certainly it needs a new maintainer, one who does a lot more than copying a file and running cvs add on it. The new maintainer will have to rewrite most of the ebuild, implement the good parts of it as pkg_config and make sure that it respects the user’s settings.

For most other packages, like I did for Ruby packages, I would have said that I’m happy to be hired to take care of the problematic package, with a reasonable fee to keep maintaining it from then on. But given it’s Webmin we’re talking about.. I’m not sure if I’m ready to pick it up. If somebody else actually uses it, and feels like maintaining it, it’d be best. I’m still open to be the proxying maintainer if somebody feels like pick up the pieces of it all, but wants to have somebody else reviewing the ebuild before it get pushed to users.

Orphaned ( aka maintainer-needed ) packages, are those that nobody, neither a maintainer nor a herd, is looking after them. Such packages oftet get removed from portage because either they don’t work anymore or there are just too many open bugs and nobody really cares about them.

Recently the TreeCleaner Project, as a decent way to inform everybody ( both devs and users ) about packages that are candidates for removal, introduced a new package to track them.

http://www.gentoo.org/proj/en/qa/treecleaners/maintainer-needed.xml

If you want to proxy maintain one of them, just send an e-mail to gentoo-dev mailing list and who knows, maybe a dev will pop up and work with you :-).  If you haven’t heard about proxy maintainers before just take a look  here and here.

I just released gentoo-sources 2.6.32-r14, 2.6.34-r6 and 2.6.35-r2. These all include the patch for the local privilege escalation flaw bug that was recently announced. So, I do recommended all gentoo-sources users upgrade to these latest versions.

There is also a bug fix included for people that were experiencing the freeze/oops in 2.6.35.2 as referenced in this bug.

Please note that 2.6.33 is no longer being supported by Gentoo-Sources.

Alright, so in my quest to move forward with my life, I'm going to be discarding certain projects that I really haven't been making a priority.  Planet Larry is one of those.  I've always thought that the idea of a planet feed for a Linux distribution's user base is a good one, but it's just never become a priority for me and so it's suffered where it could have really taken off and done well.  I'm hoping the next owner will be a better steward.

So, if you are sincerely interested in running it, shoot me an email at beandog at gentoo dot org and I'll let you know everything that's involved (hint: not much).

Edit: I should add, this doesn't require being a Gentoo developer.  The Planet was never an officially sanctioned Gentoo project, and it was never intended to be.  Anyone with the ganas can run it.

Edit: Thanks to all the people who have offered to help.  I'll say the same thing I said to all of them: I'm going to wait a few more days to see who else steps up and has ideas for the site.  Also, no, I haven't decided what to do with Znurt for now.  I don't have any plans of giving up ownership yet.

Coding for Google Summer of code has ended on Monday so here is a quick overview of what application can do. It’s pretty basic web application, but should do the work in the beginning I’ll continue working on it so some shiny features I (and not only I) wanted in the application shall get here in the future.

Users

Each user in application has a role. Role can be guest, recruit, mentor or recruiter. There are also two groups with additional permissions: project leads and administrators. Every role has different permissions. User who is not logged in is guest, who can only view some questions and read question descriptions. After creating account user will be recruit or mentor. To be mentor after registration user should use OpenID https://dev.gentoo.org/~nick to register. Recruiters can promote recruits to mentors and demote mentors to recruits. Administrators can demote recruiters to recruits or mentors and promote recruiters to administrators. User must be recruiter to be administrator.

Mentors and recruiters must be Gentoo developers for at least 6 months so application requires them to provide their nicks. Application can check if mentors provided nick of Gentoo developer and if developer with this nick is with Gentoo long enough.

Application can also fetch list of (sub)*project leads from gentoo.org and mark users who are project leads as project leads.

Questions

Any signed up user may create a question. Questions created by administrators are public: all users can view them and answer them. Questions created by non-administrators are private (or suggested) questions: only creator and recruiters can view those questions. Administrator can approve suggested question and then it becomes a public question.

Questions belong to categories and groups. It probably could use a better wording to avoid confusion (as I explain below they are quite different things but names may be confusing).

Recruiters tell users questions in which categories they should answer. They do so by adding relationships between users and categories.

Question in group are alternative questions – user should answer only one question from every group. Application choses randomly question user should answer from every question group. Guests can’t see grouped questions – but they can see description of group (so they can prepare to answer question). Recruits can see only those grouped questions they should answer. Mentors can see grouped questions their recruits should answer. Recruiters can see all questions.

Question can have content of one of three types:

• Text questions. recruits write their answers. Recruiters and mentors then check answer. Mentor of recruit who gave the answer can make comments to help recruit improve answer. Recruiters can give reference answers for questions (only recruiters can see reference answers).
• Multiple choice questions. When recruit answers all multiple choice questions [s]he should [s]he will see a message on the home page indicating if all answers were correct or if there was at least one wrong. Application checks if answer is correct by comparing it to reference answers. All multiple choice questions should have reference – if there is no reference answer for question application will assume recruits answer was correct
• Email questions. To answer email question recruit sends email to application. The email should match specified conditions (not visible to recruits).

Email notifications

Application sends email in few cases: When recruit gives a new answer or changes existing answer application sends notification to mentor of the recruit. When mentor comments answer application sends notification to user who gave the answer. When new public question appears application sends notifications to recruits who should answer it.

OpenID

It’s possible to sign up and login with OpenID.

Feedback

When answering questions recruits can express their opinion about documentation.

The coding part of Google Summer of Code™ 2010 is over. I'd like to sum it up and describe future plans related to idea I've chosen to work on, by the way learning AsciiDoc (Yes, I'm writing it in AsciiDoc and will convert to HTML later.) My reports during GSoC were a bit messy and here you have better written all-in-one. ;-)

Done

Dracut performs well on Gentoo now. Code of all (except of few dedicated to S/390) modules were reviewed and adjusted if necessary. The modules were tested with test suites delivered with Dracut sources. Module redhat-i18n was replaced by generic i18n to support internationalization for every GNU/Linux distribution. There were made some enhancements too. crypt module has been extended with support for keys on removable media and written new module – gensplash – based on Genkernel's code, supporting framebuffer splash. Moreover there were done several fixes and enhancements not necessarily related to my subjects. Harald Hoyer rewarded my contribution by adding my surname to AUTHORS file and manpage.

For two latest Dracut releases – 006 and 007 (the James Bond release ;-)) – ebuilds was created. They are based on dracut-004.ebuild maintained by my mentor Lance Albertson (Ramereth) and significantly rewritten (using a bit of Bash 4). Ebuilds have been already put into Portage tree by Ramereth.

Next step was integrating Dracut with Genkernel. Genkernel options were preserved, but initramfs builder was replaced with Dracut. Replacing it allowed to remove above 12 thousands lines of code from Genkernel. Some of minor functionality was removed with those lines, but with Dracut there was introduced much more. New code was written in modern Bash. By the way Genkernel was enhanced with tiny features not exactly related to my subject. Manpage was rewritten from troff to DocBook which then made updating it much easier.

Genkernel ebuild (from Portage tree) thanks to removal of old code could become simpler and cleaner and so it has been rewritten. For now it works only with my fork of Genkernel on GitHub.

Catalyst needed almost no changes since Genkernel's options haven't changed much. And so it was with ebuild. Unfortunately Catalyst couldn't be tested with new Genkernel because of my ancient machine and extremely low bandwidth. (Fortunately I'm gonna have 2 Mbit/s ADSL soon.)

Moreover aidecoe's overlay was created. It includes ebuilds for: Dracut, Genkernel, Catalyst and Plymouth. About first two I've written above. Catalyst ebuild is just copied from Portage tree. In Plymouth ebuild from bug #274065 a few minor fixes have been done. Plymouth is new boot splash which is supported by Dracut.

Learnt and thanks

What's probably most important I've found out how things works in developer's side. I've written much of code in many languages before and even some team projects I was working on. During the GSoC I made good usage from mailing lists, IRC and Bugzilla. I wasn't very sociable before and thanks to circumstances and my mentors – Ramereth and Kyron – I got into community and made new awesome acquaintances.

I have extremely improved my Git skills. I was a noob in the beginning of GSoC, because I had been using it only for my small school projects. For team projects I'm using Mercurial or SVN, because I usually work with Windows-folks. Git is undoubtedly the best. Here goes special thanks for Ramereth who helped me much with my issues related to Git.

All coding during GSoC was in Bash. It was really good practice. I've learnt many new tricks and become more fluent in this language. Here special thanks goes to Victor Lowther who often commented my code on initramfs mailing list and made an impression on me of Bash guru :-); Harald Hoyer who always reviewed my Dracut's patches and suggested how to do some things better; my backup mentor Eric Thibodeau (Kyron) giving useful hints and #bash at FreeNode. Learnt much thanks to them. And of course Dracut's code itself contributed to my knowledge.

Ebuilds. Never written any before. Thanks to Ramereth's hints and his Dracut ebuild I've written new ones for Dracut and Genkernel which looks quite cute (only if I can judge my work ;-)).

The future

First of all Ramereth and I are going to start process of me becoming Gentoo developer.

Genkernel is cleaned from old builder code. It may be made even simpler and more flexible. I'd like to take the project under my wings (yes, I have wings; and horns too) as soon as I become a Gentoo dev. I would start from rewriting it in modern bash and then suggesting with reports on Bugzilla resolving them slowly one by one. Configs for arches would use some kind of inheritance which might ease the maintenance. Moreover we could all join our forces and have recent kernel configs for every architecture.

Catalyst scared me a bit and I'd like to have not much to do with this tool, but with good help (I've recently discovered that Kyron knows that tool quite well) things may go in other way.

Being a developer of Dracut I'd like to continue the contribution and so I'm doing at the moment. You can spy on me on initramfs mailing list. Recently I've committed a few patches (yes, I know GSoC is over, I won't include those patches in my package).

Interview with longtime Gentoo user Mona, third place finisher in the recent 2010 Gentoo screenshot contest. If you would prefer to read the interview, Mona provided a transcript below.

LINKS:
Gentoo Screenshots
http://www.gentoo.org/main/en/shots.xml

Interview Transcript
http://linuxcrazy.com/podcasts/mona.txt

Download

One unfortunately still too common practice I find in my fellow developers is to rely too much on overlays to get users involved; my personal preference on this matter is getting people to proxy-maintain packages, and the reason is that this way I can make sure the fixes propagate to all the other users in a timely matter, as well as being able to intercept mistakes before I commit them to the tree.

But there are other reasons why I dislike overlays; for instance, they often clash enough with each other, or mix should-be-working packages with don’t-even-try … which is the case of the current Gnome overlay. I used to use the Gnome overlay so I could test and help reporting bugs before the next release hit ~arch; unfortunately since a while ago now, the overlay contains Gnome3/Gtk+3 packages that really shouldn’t be mixed in on a system that is actually used.

This became obnoxious to me the moment I went to actually try Rygel (so that I could actually get rid of MediaTomb, if it worked and I could add it to the tree — that code is noxious!). The problem is not in the high reliance of Rygel on Vala, that would be good enough, given that we have it in tree; the problem is that the Rygel UI (and after trying it out I can safely say that you don’t want to try without the UI) requires either Gtk+3 (no way!) or Gtk+ 2.21… which is the “devel” branch and is present only on the gnome overlay. Not even masked in tree.

It wouldn’t have been too bad, if it wasn’t that upstream (finally!) split gdk-pixbuf from gtk+ itself, so you should finally be able to use librsvg without X11 on the system (which is why my charts are available only as SVG and cannot be seen by some browsers who have trouble displaying embedded SVG). Unfortunately, this also means that they changed the path gdk-pixbuf uses to load the loaders (no pun intended); and the current ~arch librsvg won’t pick that up. Again, the librsvg in the overlay has automagic-deps trouble, and require both Gtk+2 and Gtk+3 to be present to work. D’oh!

This is nothing new, what is the problem? Well, ostensibly beside the fact that Arun blogs about something we can’t have ;) — not your fault, I know, not picking on you, don’t worry Arun!

The problem comes when I’ve asked before why the Gnome stuff is not pushed in main tree under p.mask, like most other teams work, especially given that I can make use of the tinderbox to check reverse dependencies before it’s unleashed, rather than have to report them afterwards. Indeed, Gtk+ and other libraries’ updates tend to be quite boring because there is way too much software that define GTK_NO_DEPRECATED and similar, which should only be used during development, and thus fail when stuff they us do get deprecated. Of course even if they didn’t define that, the code would be failing at the following update when they get removed, but that’s beside the point now.

Interestingly enough, though, the effect of (at least some) more recent deprecation seem to be causing the same kind of issue (if, by the mere fact we’re talking about Gtk+, to a lesser severity) of the recent glibc-2.12 release in the form of undefined symbols where GTK_* macros are used.

As you might suspect the tinderbox already stumbled across a few of these packages; while the quick-fix is generally quick (drop the NO_DEPRECATED definition), the complete fix (use the correct non-deprecated API) takes a while, and I can’t blame the maintainers for waiting to hear from upstream on that matter, especially given the way gtk+ is always dropped like a bombshell. Just to be on the safe side, I’ve now added some further tests to ensure that neither the “symbols” requirements caused by gtk+-2.20 nor those caused by glibc-2.12 will be left standing without further notice. If the tinderbox will ever build such a broken package, it’ll be reporting it to me so that I can file the proper bug.

Now, the gtk+-2.21 situation seem to start just as well; gtkhtml fails to build and it’s even part of Gnome itself. I will be begging the Gnome team again, starting from here, for them to add the ebuilds as soon as they are usable in main tree, under p.mask, so that the tinderbox can start churning at them.

But since people seem to think I write too much negatively, I have to say that at least a few developers seem to actually keep in mind there is the tinderbox available; Samuli, Alexis and Jeroen asked for feedback before unmasking (XFCE, Ocaml and libevent-2 respectively), and the problems found have been taken care of much more quickly then even I expected, for the most part. So if you’re a package maintainer and want the reverse dependencies of your package tested before unmasking a version into ~arch, just drop me a mail and I’ll set up a special run. It can take anything between half a day to a week or two depending on the size of the revdep tree and the queued up runs (right now it’s completing a full-system-set rebuild to see if there were more issues with glibc-2.12; turns out I only hit another problem and that was related to GNU make 3.82 instead (another “good” scary bump).

After this post, you can guess the following run is going to target gtk+-2.20. If there are no further runs, after that it’ll resume the daily-build of the tree.

Please keep in mind though: package.mask-ed packages are fine, but the tinderbox will not test any overlay. Get your fixes in the tree proper!

I’ve recently ranted about the fact that GLIBC 2.12 was added to portage in a timeframe that, in both my personal and professional opinion, was too short. I’ve not dug too much into what the problems with that particular version, and why I think it needed to stay in package.mask for a little longer.

The main reason why we’re seeing failures is because the developers have been, for a few versions already, cleaning up the headers so that including only one of them won’t cause a number of other, near-unrelated headers to be included. This is the same problem as I described with -O0 and libintl.h almost two years ago.

Now, on FreeBSD, and Mac OS X by reflection, a number of these cleanups have been done a long time ago, or the problem was never introduced: they both try to stick to the minimal subset of interfaces you need to bring in. This is why half the time what you have to do to port something to FreeBSD is just adding a bunch of header inclusions. This should mean the whole situation is easily handled, and should already be fixed in most situations, but that’s far from the case. Not only, for no good reason at all, a number of projects protect the inclusion of extra headers with #ifdef calls for FreeBSD and Mac OS X, but one of the particular headers cleaned up causes a much worse problem than the usual “foo not defined” or constants not found.

The problem is that the stat.h inclusion has been dropped by a number of headers, which means it has to be added back to the code itself. Unfortunately, since C allows for implicit declarations (Portage does warn of them!), this means that a call of S_ISDIR(mymode) and similar will appear to the compiler like an implicit function declaration, and will then emit a requirement for the (undefined) symbol S_ISDIR… which of course is not a function but a macro defined in the header file. Again, this is not as troublesome as it looks: the linker will catch the symbol as undefined and halt the build, in the best of cases. And just to make sure, Portage logs these problem further, stating that they can create problems at runtime — of course that would be more useful if developers actually paid attention and fixed them, but let’s not go there for now.

The real problem come when this kind of mistake is present in shared objects; by default ld will allow shared objects to have undefined reference, especially if they are plugins (since then the symbols may be resolved by the host program, if not by a library that they can be linked to). Of course, most sane libraries will be using --no-undefined for the final link… but not all of them do so. A common problem situation is Ruby and its extensions – at least outside of Gentoo, given that all our current Ruby ebuilds force --no-undefined at extension link time. The only warning you have there is the implicit-declaration of the fake-function, but that’s far from being difficult to oversee.

And before you suggest that, no -Werror-implicit-declaration is not going to be a good idea: most of the autoconf tests fail if that is passed, which result in a non-buildable system; that’s why it’s one of the few flags I play with on all my autoconf-based projects!

Also, as Ruby itself proved – ask me again why I always rant when I write about Ruby – there are ways around the warning that don’t constitute a fix even for the most optimistic of the developers.

But then, yuo have the million euro question: how severe is this bug? The only way to judge that is to understand what symptoms it causes. Given that’s not a security feature, you’re not going to have security bugs here, but you have:

• build failures: obnoxious but manageable; things just failing to build are not going to worry me most of the time; when the fix is as easy as adding a missing include line, I have no reserve against going ~arch;
• runtime load/startup failure: already a bit less acceptable; failing to load a plugin/extension with a “undefined symbol” error is rarely something users will like to see; it’s bad but not too bad;
• runtime abortions: now this is what I’m upset about; having missing symbols at runtime mean that you cannot either trust the software you built, nor that which starts up cleaning; aborting at runtime means that your software might be in the middle of some transactions when it reaches the error situation, and could even corrupt your data; this is made more likely given the fact that it’s part of the stat(2) handling code!

There is a good thing though: -Wl,-z,now which is part of the default settings for hardened profiles, or that can be set at runtime by setting LD_BIND_NOW=1 in the environment, will ensure that all the symbols are bound when starting up the process, rather than lazily while the code executes; it can reduce the risk of the missing symbol to be hit in the middle of the transaction. Unfortunately it does not work the same way with extensions for languages like Ruby and Python, but at least alleviate a bit the problem.

Combine what I just wrote with the fact that even a package part of the system set (m4, used by autoconf, not something you can live without) failed to build with this glibc version by the time it went unmasked, showing a lack of system rebuild on the system of the developer choosing to unmask it, and you might guess why I ended up using such strong words in my previous post.

It’s been a while since kargig and I worked together to bring sys-apps/halevt to Gentoo portage tree. To be honest, I didn’t quite use it, since all of my PCs where configured to use ivman, so I was too lazy to do any changes. However, due to the upcoming removal of ivman, I decided to migrate all of my PCs to halevt. The migration was quite smooth except a minor ( or major, depends on your point of view ) issue; Removal devices such us USB disks or external HDDs, were mounted just fine, but regular users didn’t have write access to them. Digging around, I found this post on Gentoo forums. What I had to do was to replace every

exec=”halevt-mount -u $hal.udi$ -o sync -m 002″

to

exec=”halevt-mount -s”

It seems to me that the default conf file that gets installed with halevt is rather useless for normal users because they are unable to write on external devices. If you feel like I should ship another configuration file, optimized for your needs, I am willing to hear your suggestions and come up with a rather convenient file :)

Previously thought impossible, thanks to syslinux 4 this feat can now be achieved in a few easy steps.

Background

• Why GPT?
  GPT, short form of GUID Partition Table, is the standard for EFI systems and is intended to replace MBR style partition tables. Advantages of GPT include support for up to 128 partitions (no distinction between primary and logical), no limitation to 2³² sector disks (2 TiB for 512 byte sectors), and increased redundancy for easier recovery.
• Why btrfs?
  No real reason, other than it being the new hotness (in spite of some design flaws).
  
• Why syslinux?
  EXTLINUX from the syslinux family of boot loaders can boot both from GPT and btrfs. An alternative would be GRUB 2, which however uses not the standardized method described in T13 EDD-4 Annex 1 and needs special patching for btrfs still.

Prerequisites

• A USB flash drive of >256 MiB capacity
• sys-apps/gdisk
• >=sys-boot/syslinux-4.00
• sys-fs/btrfs-progs

Partitioning

# gdisk /dev/sdc
GPT fdisk (gdisk) version 0.6.9

Partition table scan:
  MBR: not present
  BSD: not present
  APM: not present
  GPT: not present

Creating new GPT entries.

Command (? for help): n
(answer the questions)
Command (? for help): x
Expert command (? for help): a
Using 1
Attribute value is 0000000000000000. Set fields are:
Known attributes are:
0 - system partition
1 - hide from EFI
2 - legacy BIOS bootable
60 - read-only
62 - hidden
63 - do not automount
Toggle which attribute field (0-63, 64 to exit): 61

Installing the MBR/GPT boot code and EXTLINUX boot loader

# dd if=/usr/share/syslinux/gptmbr.bin of=/dev/sdc

# mkfs.btrfs /dev/sdc1

# mount /dev/sdc1 /mnt/usb
# extlinux -i /mnt/usb
/mnt/usb is device /dev/sdc1
# cp /usr/share/syslinux/menu.c32 /mnt/usb 

Configuring the boot loader

# mkdir /mnt/usb/debian-amd64
# cd /mnt/usb/debian-amd64
# wget http://ftp.debian.org/debian/dists/squeeze/main/installer-amd64/current/images/netboot/debian-installer/amd64/linux
 http://ftp.debian.org/debian/dists/squeeze/main/installer-amd64/current/images/netboot/debian-installer/amd64/initrd.gz

DEFAULT menu.c32
PROMPT 0
MENU TITLE Select Installer

LABEL debian-amd64
   MENU LABEL Debian Squeeze amd64
   KERNEL /debian-amd64/linux
   APPEND initrd=/debian-amd64/initrd.gz

Other Thoughts

What? Free and Open Source Software Conference (FrOSCon) - 5th year

When? 21st, August 2010 - 22nd, August 2010

Where? Hochschule Bonn-Rhein-Sieg | Grantham-Allee 20 53757 | Sankt Augustin, Germany

FrOSCon 5 is almost here, and Gentoo will be there!

Free Software and Open Source – these are the topics of FrOSCon. Many exciting programs with talks, workshops and an exhibition, supported by LUUSA and FrOSCon e.V. The Saturday night social event gives you the opportunity to exchange opinions with other visitors, speakers or volunteers.

Whether you're a developer, user, or simply curious, be sure and stop by our Gentoo booth. We will have Gentoo and Larry t-shirts, plus much more! See you there!

Hi people!

I'm done with my work on the Google Summer of Code for this year and I would like to thank to all the people that helped me during the last 3 months.

Special thanks goes to my mentor (Denis Dupeyron) by all the friendship, support and the daily talks. I know that it was not easy for him. ;)

I would also like to thank:

• Sebastien Fabbro and all the people from #gentoo-science by the cool chat.
• The Gentoo GSoC admin, Donnie Berkholz, by the support.
• Brian Harring and Robin H. Johnson by the help with the infrastructure.
• All the people from the Google Open Source Office, mainly the GSoC admin, Carol Smith, by the support and the GSoC itself.
• The GNU Octave developers by the great piece of software.
• All the Gentoo community.

Thank you all for make my Winter(?) coolest. ;)

I'm sorry if I forgot someone.

That's all for now...

Oooops... Wait! I'm not leaving the Gentoo development as can looks like, of course! My work is only starting. :)

Thanks again!

Genkernel

• removed old initramfs builder code – above 12.000 lines
• converted man page from troff to DocBook (by hand and that's why so pretty ;-)) and updated to recent changes in Genkernel
• some minor fixes

Catalyst

• patch just removes a few unnecessary lines and add --generic (Genkernel produces --hostonly initramfs by default)

aidecoe's overlay

1. sys-boot/plymouth
   Added new package. Ebuild taken from bug #274065. I've done some minor fixes and reposted as plymouth-0.8.3-r2.ebuild and plymouth-0.8.3-r3.ebuild
2. dev-util/catalyst
   Nothing new, just copied from Portage tree and updated Git URI.
3. sys-kernel/genkernel
   Completely rewritten. Shortly:
   • use my fork of Genkernel
   • depend on >=dracut-007
   • install bash-completion script only if bash-completion USE flag is enabled
   • removed download of additional packages since Dracut takes required things from the system
   • using ebuild functions instead of cp, sed and others
   • DocBook related dependencies
4. sys-kernel/dracut
   Changes:
   • fixed QA warning
   • warning about QEMU
   • 9999 version has plymouth dependency now

What still needs to be done

• test Catalyst with new Genkernel
• netboot substitute

References

• my repos on GitHub:
  • http://github.com/aidecoe/aidecoe-overlay/
  • http://github.com/aidecoe/dracut/
  • http://github.com/aidecoe/catalyst/
  • http://github.com/aidecoe/genkernel/
• Bugzilla:
  • Dracut-006: http://bugs.gentoo.org/show_bug.cgi?id=329003
  • Dracut-007: http://bugs.gentoo.org/show_bug.cgi?id=331903
  • Plymouth: http://bugs.gentoo.org/show_bug.cgi?id=274065

~

• I haven't posted latest Genkernel and Catalyst patches on Bugzilla. Post them?
• I'll post full summary soon.

Again, dated material (June 2010) but some people may find it interesting…

Gentoo developer amne made it into the Das war der Linuxtag 2010 video.
Seek to 17:18 if you’re in a hurry.

There seems to be a semi-meme going around on some of the planets I read.

My concern is long directory paths and I wanted a dymanic solution for it. Surely, I cannot be the first person to think of it, but I haven’t seen it in use anywhere else.

jolexa @ helios :: ~ %%

and
jolexa @ helios :: ~ %% cd projects/prefix-tinderbox/really_long_dirname_so_that_binpkgs_can_be_shortened/a/b/
jolexa @ helios :: .../a/b %%

(the colors and wrapping are representitive only)

and the code that produces that:

_chomp_path() {
    local path=${1/${HOME}/\~}
    local last=${path} sedout= count=0 count2=0
    sedout=$(echo ${path} | sed -e 's:/: :g')
    for i in ${sedout}; do
        (( count++ ))
    done
    if ((count > 2)); then
        last="..."
        for i in ${sedout}; do
        (( count2++ ))
        if (( count2 >=  count - 1 )); then
            last+="/$i"
        fi
        done
    fi
    echo ${last}
}

PS1='\[\033[1;34m\]\u \[\033[1;32m\]@\[\033[1;34m\] \h \[\033[1;30m\]::\[\033[1;37m\] $(_chomp_path $(pwd)) \[\033[1;30m\]%%\[\033[0m\] '

If anyone wants to improve that function, let me know. It “fails” on directories with spaces in it.

I’ve added ebuilds in Gentoo’s portage for a new webkit browser called luakit.

[I] www-client/luakit
Available versions: (~)2010.08.07 (~)2010.08.13 **9999
Homepage: http://www.luakit.org
Description: a webkit-gtk based, micro-browser framework in Lua

It is a webkit based, low footprint browser written in C and Lua, modeled after awesome wm, my favorite window manager

Vim-like bindings, low memory usage, lua configuration file (if you use awesome you will feel at home) and rapid development are some of luakit‘s key points.

To try it out:

emerge -av luakit

(~testing only, so you have to keyword it if running stable)

For more information, visit the luakit website, or join #luakit (OFTC IRC network)

The Gentoo KDE Team decided not to include KDE 4.5.0 in the main portage tree, due to the lack of the KDEPIM suite, which would cause a lot of trouble to users trying to upgrade. Thus, it will remain in the kde overlay hardmasked. It can be installed with KDEPIM 4.4.5 though, if the user adjusts the keyword files accordingly. The first KDE 4.5.X version that will include the KDEPIM suite will be added to portage as well.

Furthermore, we decided to keep a bit more the live ebuilds of the 4.4 branch, and not to provide any snapshots yet. The Gentoo KDE Guide has been updated to adjust those decisions.

EDIT: Also take a look at comment 10, which has a better explanation

=-=-=-=-=
Powered by Blogilo

Following my last post about Gentoo QA and Diego’s one , I will try to explain why this QA issue is that important.  As you already know, the Gentoo Council decided to add -Wl,–as-needed to default LDFLAGS. If you don’t know what does this mean, you may want to read this article from the QA documentation. However looking at this traker, I see more than 90 bugs not respecting LDFLAGS at all. Can you see any *doh* here? Yeah thats right. While the move towards -Wl,–as-needed  addition to LDFLAGS is a big step forward, the tree is not ready for this just yet. Thanks to Diego’s tinderbox for filling all those bugs, we are now able to fix these packages once and for all.

Recently, a discussion took place on gentoo-dev ML about this issue and as it turned out, few developers didn’t actually know how to track packages that don’t respect LDFLAGS. One way is to read the build system or just observe the output during build time. Another decent way is to add –Wl,–hash-style=gnu to default LDFLAGS.  This is how it works: The hash-style ( in our case GNU ) ends up in the final binary. If portage can’t find it there, it simply means that the ldflags where just ignored ( Thanks to Samuli for the explanation ). More info about –hash-style can be found on ld man page.

For our fellow Arch Testers, amd64/x86 pages[1][2] have just been updated to include this piece of information, so unless you want to use my QA powerzz against you, simply adjust your LDFLAGS before you perform any kind of testing.

Update (24/8/2010):

-Wl,–hash-style=gnu is now default on linux/amd64/10.0/developer profile. Lets see if this help us spot more broken packages

[1]http://www.gentoo.org/proj/en/base/x86/arch-testers-faq.xml

[2]http://www.gentoo.org/proj/en/base/amd64/at/index.xml

So, this is the last progress report of this years GSoC. That doesn’t mean that this project is done, but GSoC has come to an end.

To wrap things up, the version, which will hopefully be added to the tree soon is the one which sets the caps directly in the livefs. Why? Because I still haven’t figured out a good fallback-mechanism, if the livefs doesn’t support caps and the caps can’t be reapplied after the copy.
Don’t worry, I’m working on that, but it won’t be finished by Aug 16th, which is the Pencils Down Date of GSoC. So flameeyes and me decided, that it would be good, to see this version as a final result.

Thanks you all for trying my stuff and commenting. This was the best summer of my life so far, I learned a lot and had a great time. Thanks to my mentor Diego E. “Flameeyes” Pettenò for giving me feedback and encouraging me, when I wasn’t so sure I’m on the right path.

As for my involvement with Gentoo, there is already a recruitment bug for me in bugzilla ;).

This is in response to a rescue operation I did yesterday evening, after being notified of Bug #332257. In essence, I've failed to include two USE flags (force-cgi-redirect and discard-path) which should have been there and castrated a third one (cgi).

I snuck those missing two back in and fixed USE="cgi" to also enable fastcgi, like 5.2.13 did. So if your FastCGI setup stopped working after an upgrade to 5.2.14, please resync and remerge php-5.2.14, the regression has been fixed.

A friendly user also asked me about some other changes, I only then realized I failed to document: the sybase, java-external and fastbuild USE flags are gone from 5.2.14. You'll notice that they also don't appear on php-5.3.3. This has a reason, which is probably only appearent if you follow php-bugs@gentoo.org on our Bugzilla, so here goes:

sybase

... was dropped when Bug #281316 came up. I learned that sybase-ct as provided by dev-db/freetds is a near complete stand-in for sybase. If the "near complete" doesn't work for you and you need sybase back, please comment below. But note PHP-5.3 has dropped support for that extension, so you'll need to find another way, anyway.

java-external

... has never done much to the PHP build - it only pulled in dev-php5/php-java-bridge and enabled USE="session". Now, the java-bridge package has not been available in our tree since 2007 and the official status of the "java" extension according to the PHP manual is unmaintained and dead. So I decided to drop this. If you need Java support, please head over to Zend, but I've no experience with their Java stuff, so I can't comment further.

fastbuild

... was an experimental patch to the PHP build system I dropped in a bid to provide a more "vanilla" PHP. To be honest, it wasn't in hoffies eblits for PHP-5.3, as the patch would have probably needed modifications to it in several places. And I was not in favor of adding another eblit to accomodate PHP-5.2 here. To be even more honest, I'm scared by PHP's build system so much I want to destroy and rebuild it from the ashes

There is a new beta version of Google Chrome, and now portage also has www-client/chromium-6.0.472.33 in ~arch. There are some new exciting features in the 6.x series, like improved UI, autofill, better sync, and more.

Our security team got around to filing Bug #332039, which covers all the security fixes in PHP-5.2.14 and PHP-5.3.3.

It's Gentoo policy to have a normal testing period of approximately 30 days. The period can be shortened or omitted if there are security problems in a current stable version. This is what's happening in the transition from php-5.2.13 to 5.2.14 right now. I intentionally migrated 5.2.14 from the overlay a few days after 5.3.3, so we could test-drive the new eblits backed system with 5.2.14 that makes maintaining PHP so much easier. While all eblits originally were designed for use in 5.3.2, I only needed to update one eblit to accommodate 5.2.14.

But there was another change with 5.2.14 that will enable me to fix bug #310383 once php-5.2.14 is stable on all architectures: 5.2.14 finally makes use of EAPI2. It heavily uses USE dependencies, and also, by popular request, includes USE defaults. This feature auto-selects USE flags if you don't disable them via /etc/portage/package.use. By default, you're now getting the same PHP features you'd get by downloading the source and just running ./configure.

The change to USE defaults also obsoletes settings certain USE flags in the default make.conf, which is what bug #310383 is about.

There's also a downside to the heavy use of EAPI2 features: we now force certain USE flags on, if you enable a specific USE flag. This is a replacement for the manual die message you've probably seen once in a while. An example of this is USE="truetype", which needs the gd (or gd-external) USE flag on. This currently results in two kinds of error messages, a nice one and a very confusing one. Observe (sorry about the wrapping here):

terra mabi # USE="readline libedit" emerge -1 php
These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild N ] dev-libs/libedit-20090923.3.0 456 kB
[ebuild R ] dev-lang/php-5.3.3-r1 USE="libedit* readline" 0 kB

Total: 2 packages (1 new, 1 reinstall), Size of downloads: 456 kB

!!! Multiple package instances within a single package slot have been pulled
!!! into the dependency graph, resulting in a slot conflict:

dev-lang/php:5

('installed', '/', 'dev-lang/php-5.3.3-r1', 'nomerge') pulled in by
=dev-lang/php-5.3.3-r1[-libedit] required by ('ebuild', '/', 'dev-lang/php-5.3.3-r1', 'merge')
(and 1 more)

('ebuild', '/', 'dev-lang/php-5.3.3-r1', 'merge') pulled in by
php

Explanation:

New USE for 'dev-lang/php:5' are incorrectly set. In order to solve
this, adjust USE to satisfy '=dev-lang/php-5.3.3-r1[-libedit]'.

While I find the fact that portage is complaining about a "slot" problem misleading, it gives a nice and clear explanation about what to do to recover. Compare to this:

terra mabi # USE="sharedext sharedmem threads" emerge =dev-lang/php-5.2.14

These are the packages that would be merged, in order:

Calculating dependencies... done!

emerge: there are no ebuilds built with USE flags to satisfy "=dev-lang/php-5.2.14[-threads]".
!!! One of the following packages is required to complete your request:
- dev-lang/php-5.2.14 (Change USE: -threads)
(dependency required by "dev-lang/php-5.2.14" [ebuild])
(dependency required by "=dev-lang/php-5.2.14" [argument])

Duh! I've no clue about the root issue by looking at the error message. Turns out USE="sharedmem" clashes with USE="threads", but you can only discover this by looking at the ebuild. I'm currently working out how to get you a more informative message.

That's the news for today. Stay tuned while we work out the details about slotting PHP to minor versions (certainly the topic of one of the next posts), which we hope to have out before PHP gets a 5.4 version.

Long live Gentoo! As the quantity and quality of this year's entries will attest, Gentoo is alive, well, and taking no prisoners!

Thanks to everyone who entered the contest. There were 103 total entries using 7 different window managers and desktop environments.

The Winners

1. Nickname: Chopinzee. Desktop: E17 built from the official EFL overlay.

2. Nickname: whtwtr. Name: Shawn H. Desktop: Compiz window manager with the Xfce panel.

3. Nickname: GentooApologetin. Desktop: Custom Xfce.

4. Nickname: shpaq. Name: Michał Laszuk. Desktop: Gnome with the Gant icon theme.

5. Nickname: nm. Desktop: dwm (simplicity is elegance).

For all the specifications and cool details please visit the main Gentoo screenshots page. Make sure to check out all the entries!

discuss this!

This month's Xfce desktop:

icons: awoken
gtk+: axiomd
xfwm4: axiomd
background: The Crown of the Sun
cursor: Obsidian xcursors

The uncluttered version that shows off the wallpaper and conky configuration:

I built my environment around the wallpaper, an image of a solar eclipse, bringing out the haunting beauty of the sun's corona. I cropped this photo from APOD to fit my screen dimensions.

With such a beautiful cosmic backdrop, I had to search for matching theme elements. I used the same window manager and gtk+ theme, axiomd. It's nice and dark, with moon dust highlights.

It's been a long, long time since I last installed conky. I decided to give it another go, now that it's capable of doing beautiful things with Cairo and Lua. I was especially impressed by this configuration I found on the Arch Linux forums.

I made a few modifications to the ring meter scripts for conky. The end result is pretty decent, considering I haven't done much heavy tweaking yet. You'll need to emerge conky with the lua-cairo and lua-imlib USE flags set, or else the scripts won't function.

The rings frame the corona, with just a touch of transparency to blend it into the deeper space backdrop. From left to right, the rings measure: CPU core 2 load, memory usage, /usr/portage, /, and CPU core 1 load. Adding, removing, shrinking, or expanding rings is pretty easy. The ring scripts are well-commented. The biggest obstacle I've run into so far is adapting the configs to my screen size, ensuring that items are placed just right. I could tweak the ring's curvature to precisely match the eclipse, but it's close enough as it is.

I picked up the icon set because it's very attractive for both dark and light environments. It's very flexible, with numerous alternative icon versions, extra standalone icons, many distribution logos, and a number of helpful scripts inside the tarball. I used one of the included Gentoo logos as my Xfce menu icon.

The mouse cursor theme is glossy and dark, yet it has a few blue animations to add a splash of color. To get it, run emerge obsidian-xcursors.

Applications

In the foreground, Decibel Audio Player is running in the "mini" mode, playing a beautiful track by Planet Boelex.

Thunar is the filemanager open in the background. An Xfce terminal displays an eix-sync operation.

Running in the panel are an assortment of application launchers, including customized dropdown menus for frequently used programs.

After the Xfce menu, launchers, and taskbar, the notification area holds the tray icon for Decibel Audio Player. Then a genmon applet that runs my lastsync.sh Portage script. After genmon, there are plugins for volume control, the Orage clock, and local weather.

Now that I'm using conky, I can probably find a way to integrate the weather, clock, and Portage sync script with the existing ring meters, or even run it in another instance off to the side. Anything to reduce my crowded top panel.

Hi all…

I’ll put this in the handbook when I get around to updating that… but in short, those who need to be able to netboot SGI systems from a Windows host, might wish to give tftpd32 a try.  I’ve been tinkering with this TFTP server in order to load some ARM-based devices (Ka-Ro TX27s using RedBoot).  I find this server daemon is inconsistent with its loading speed when transferring files to a TX27 (sometimes speed is good; around 1MB/sec, other times it crawls at 3KB/sec), however experimenting with it in a VirtualBox VM running Windows 2000, it works flawlessly for netbooting the SGI O2 I have here.

Below is a screenshot of the settings used.  A Linux or Unix-based TFTP server would be better if that can be organised, but for those who have no option, this may be a viable alternative.

TFTPD32 Configuration, tested with an SGI O2

First of all new version of Dracut was released — 007 — the James Bond release. ;-) The new version includes all changes I've made in Dracut. In past week I've made a few fixes and docs updates, sent gensplash (framebuffer splash) into upstream and Harald updated man page with my surname as one of developers what makes me very happy. :-) The ebuild for 007 is attached to bug #331903 [0].

I've made following changes in previous patch for Genkernel:

• gen_dracut.sh - refactorized
• updated to dracut-007
• more verbose
• final info for user about kernel args
• params for gensplash module may be passed by genkernel args (previously only
  by /etc/conf.d/splash was possible)
• minor fixes

New patch is attached to the same bug #330127 [1] as previous.

For Catalyst I made just a tiny patch:

• for Catalyst we'd like to have generic image
• warning for old generator specific options
• for netboot target use old genkernel generator

To include new features we just have to provide Genkernel arguments to gk_mainargs. There's no much to do with Catalyst. If something would be wrong with initramfs in Catalyst it should be fixed in Genkernel. For bad news I couldn't test Catalyst. It's really hard using it, especially with 8 years old computer and bandwidth about 16 KiB/s with 1 GiB download limit… Spend almost week trying to play with it but with no much success. Although changes  are tiny, so there should be no problem. In case sth goes wrong I'm always reachable via e-mail, XMPP and IRC.

My Questions

• What about using DocBook for Genkernel man pages? XML is much nicer than troff. Dracut just moved to DocBook and would be nice if Genkernel too. There's tool for convertion man page → DocBook — doclifter [3].
• What about Genkernel's old internal initramfs generator? I'd like to remove it. It would make Genkernel's code, ebuild, help and man page much simpler. But on the other hand there two things not yet supported in Dracut: busybox and netboot in exact way. Netboot is some kind of rescue image. With Dracut we can have rescue too — with debug and nfs modules we have quite much handy tools. Then just go into shell with rdshell kernel argument. For me would be sufficient, but not sure what is really expected from netboot.

Plans for this week

• update Genkernel man pages (either with really ugly troff or less ugly XML or maybe you might suggest something even better?)
• clean old generator code if it would be OK
• maybe handle netboot stuff after gathered remarks

References:

• [0] Dracut ebuild: http://bugs.gentoo.org/show_bug.cgi?id=331903
• [1] Genkernel patch: http://bugs.gentoo.org/show_bug.cgi?id=330127
• [2] Catalyst patch: http://bugs.gentoo.org/show_bug.cgi?id=331923
• [3] doclifter: http://www.catb.org/~esr/doclifter/
• my forks of Dracut, Genkernel, Catalyst and also my overlay: http://github.com/aidecoe/

Greetings, following is the weekly update for gpypi2 project. It's main purpose is to generate ebuilds from Python Package Index. Quick links to project info:

• Repository, issue tracker
• Hudson CI instance
• Sphinx documentation generated by hudson on each commit

GPyPi knows about metadata.xml, ChangeLog and Manifest!

Previous week (2nd - 9th August)

Task: metadata, echangelog and manifest generation

In the last week of implementing features I will add support for Gentoo developers workflow.

Done. Implementation is in gpypi2/workflow.py. One issue though: echangelog has no idea how to add files to SCM, will have to find another abstraction tool for that.

Task: Atomic actions

Currently if anything fails in the process, data will lay around the filesytem.
I'll implement most basic cleanup (rollback) on failures.

I haven't touched this one, mostly because I don't have a good idea how to handle tracking of files.

Task: Exceptions

Code currently uses a bit of exceptions and a bit of log.error.
I'll make that a bit structured and also document what exceptions are raised in functions

Done. Exceptions are documented and properly catched in cli.py code.

Task: integration tests and documentation

Update documentation and write some integration tests.

Partly done. Tests still need some love, but that will not be part of the GSOC (I still have to properly implement SrcUriNamer.)

GSoC is over, I have met my deliverables. It was a lot of fun, can't wait for Gentoo community to start using my tool. Project is far far from finished, I will continue to work on it and polish it. At the end, Gentoo is my primary choice for platform (even on laptop).

I can't wait for next year to apply for the project and continue to work on OSS. Cheers guys!

Hi folks,

here is my weekly report #11.

I'm writing this a bit early because I'll be traveling back to the town where I live and study (Leopoldina/MG, Brazil) tomorrow. Yes, my vacation from college ended. :(