You probably remember my previous notes about Wordpress, FTP and the problem with security. At the end after a (boring) set up session I was able to get vsftpd provide FTPS service, which should be usable both by Wordpress and by Dreamweaver, so that my friend the webmaster can upload through it directly.

This is important because as it happens I have another prospective customer who’s going to run Wordpress, and FTPS now start to look more interesting than SSH, as it doesn’t require me to give shell access to the server either.

Unfortunately I’m a bit worried (maybe more than I should be) for the use of standard passwords rather than certificates or keypairs for authentication. Which meant I went tried to think of other alternatives.. of which there are mostly two: Google Authenticator and YubiKey .

The latter I knew by name already because I proxy-maintain the required software for Brant, and I know it’s outdated already and would require a new maintainer who can deal with those packages – I already posted about hardware-related maintenance for what it’s worth – so it was my first choice: while it meant I had to spend some money, it would have solved my problem and improved Gentoo, even if just for a tiny bit. The price for YubiKey devices is also low enough that, if I felt like providing more FTPS access to customers, I could simply bill it to them without many complaints.

So I went on the manufacturer’s (Yubico’s) website and tried to buy two of them (one for me to test and set up, and one to give my friend to access the server); despite publishing the prices in dollars, they sell through Sweden and UK, which means they are part of EU’s VAT area, and me being a registered business within EU, I should receive a reverse-charge invoice by stating my own VAT ID… never had much of a problem with it, as many of my suppliers are sparse through Europe, I registered for the “foreign-enabled” registry right when I opened business — don’t ask me why Italian (and Spanish as far as I can tell) business owners are not enabled by default to have intra-union suppliers.

Now trouble starts: since, as I just noted, not all VAT IDs are valid to use for intra-union trade, there has to be a way to ensure you’re dealing with an acceptable party. This is implemented through VIES the VAT Information Exchange System which, for what concerns Italian businesses, only tells you a boolean result of valid/invalid (and not the full registration data that most other states seem to provide). I knew VIES from a previous business agreement, but I never cared much. Turns out though that most e-Shops I encountered validate the VAT ID after order completed ­— or in the case of Amazon it seems like they check their internal database as well as VIES.

Yubico instead validates the request through VIES at the time of registration:

VAT Number could not be validated with VIES at this time. This typically happens when the service is under maintenance. Please retry after some time. For urgent orders, please contact order@yubico.com

Considering that the VIES website has a long disclaimer (which I can’t quote here for reasons that will be clear in a moment) stating that they do not guarantee the availability of the service at any time, and only seem to guarantee the validity of the data to the extent that the law ask them to (which probably means “as long as the states’ own databases are correct”), relying on such a service for registration is .. bad.

The VIES website is indeed down since at least 11am today (over four hours ago as I write this); for a moment they also gave me an interesting page (which I forgot to save), telling me that there were too many requests’ failures from “my IP address” … listing an IP address in the 212/8 range — my actual IP address is in the 94/8 range.

What’s the end result here? I’ll probably waste some more time trying to get Google Authenticator; Yubico basically lost a customer and a (possible) contributor by trying and failing to be smarter and won’t have a dedicated maintainer in Gentoo in the near future. It’s sad, because it seems to be easily the most cost- and time-effective solution out there (Google Authenticator is free, but it requires a greater investment of time, and time is money as we all should know).

Seems the patch I committed for the fix was corrupted.  So, I am rebuilding and releasing kernels for 3.2 , 3.1 and 3.0.

Thanks for wired for pointing this out.  I will be removing the ones from yesterday.

The following kernels now contain the fix:

gentoo-sources-3.2.1-r2

gentoo-sources-3.1.10-r1

gentoo-sources-3.0.17-r2

In my previous installment I ranted about. among other things, the way Rails suggests you to keep a world-writeable log file for the production environment. As I said at the end, I planned on looking at the syslogger gem and that was actually quite helpful.

The idea goes like this: by using syslogger you can tell Rails that the logs have to go through the syslog; in my case that means it goes to metalog, which then filters on the webapp names and pushes it to /var/log/rails, taking care of rotating the log as needed (either due to size or time — the former is quite useful to avoid that rogue bots cause a DoS, which happened to me when I was inexperienced with these technologies!). Of course, this only works on Unix, but that’s what I care about anyway.

Beside the placement of the logs, using metalog for me also means I can filter important messages and show them in the important messages’ log rather than being just limited to a hidden log file within the app’s own tree, and also means that I can mix in the messages of all the running applications, rather than having each report to a different file. If I were to use syslog-ng instead, I could easily make it send the logs via network to another box and aggregate all of them there… but I really don’t see the point (yet) for that, and the features that metalog comes with tramp easily the network support.

So how do you achieve this? It’s actually pretty easy. Obviously it starts with installing dev-ruby/syslogger (in Gentoo, through Portage, everywhere else, via gem); then you can configure this very easily on both Rails 2.3 and 3.x series (I have one server running Rails 2.3, the other 3.1… I have yet to set up Typo 6.x, but I’ll probably do that at some point in the near future, although unlikely before FOSDEM).

The trick is all in config/environments/production.rb, where you have to tell Rails to use a custom Logger; there is already an example, commented-out like that refers to the other gem, SyslogLogger, but you should change it to something like this

  config.logger = Syslogger.new("yourappname")

This way you can distinguish each application’s messages in the log. Then in the metalog.conf file you can have:

Rails apps : 
  program_regex = "^(typo|radiant|yourappname)"
  logdir = "/var/log/rails"
  maxfiles = 5
  break = 1

so that everything is then readable as /var/log/rails/current.

I’m not sure how much it impacts performance; I’d be surprised if it decreased them, as metalog also buffers the disk writes, but you never know until you check for sure; in general I still prefer if the (multiple) Rails processes send everything to metalog for my own convenience.

Interestingly, if you have a webapp that does not deal with on-disk files directly, but just with a database, by using syslogger you’re basically limiting the writing to the cache directories only, which is probably a positive note.

I just released gentoo-sources-3.2.1-r1 for Linux Local Privilege Escalation via SUID /proc/pid/mem .

I plan on creating releases for additional kernels with this patch through the day.

See the link for more info on the privilege escalation.

The following kernel versions contain the patch:

gentoo-sources-3.2.1-r1

gentoo-sources-3.1.10

gentoo-sources-3.0.17-r1

Usually, whenever a new KDE release is published, Gentoo users can update already the same day, as suddenly a complete and polished set of ebuilds appears in the portage tree. (Stay tuned on upcoming wednesday for KDE 4.8.0, it's shaping up very nicely!) How is this possible? Well... let me explain.

If you're a stable version user, you may have never heard of so-called live ebuilds. This is a special variant, usually denoted by a version number ending in 9999, that does not rely on a source tarball. Instead, it contains a URL of a revsion control system (say on anongit.kde.org). When you emerge such a version of a package, the sources of the specified branch are checked out or updated to the newest upstream state, and that is used for building the installation package. Obviously this is not for everyone; depending how well upstream structures commits, things may not build for a while, contain fresh bugs, ... Also, reporting bugs from live versions on Gentoo bugzilla is discouraged as most of the times we can't do anything about it (do it only if you are sure it's a problem with the ebuilds, not with the source). If you're running live, you should be willing to hack yourself and work with upstream.

However, many of the Gentoo KDE team members run these live ebuilds, partly the current bugfix branch (i.e. KDE/4.8), partly even git master. They continuously keep the live ebuilds in the Gentoo KDE overlay updated to the newest state of the source. When a release is made, the corresponding live ebuilds of this branch are copied to the version ebuilds. For example, the KDE/4.8 branch live ebuilds have the version number 4.8.49.9999 (i.e.
kde-base/kdelibs-4.8.49.9999), so when the pre-release tarballs for KDE 4.8.0 were released to the packagers a few days ago, we only had to copy all 4.8.49.9999 ebuilds to 4.8.0 and immediately had a working set for testing. Most problems at that point are only caused by changes in tarball packaging. As distribution packagers get the pre-release tarballs (that still may change due to last-minute bugfixes) a week before the official release date, these can easily be fixed in time.

This also means that KDE maintenance in Gentoo is really a team effort. Whoever moves a released version to the main portage tree and/or commits bugfixes there builds on all the work that the team has done in the overlay in the meantime. Cheers!

You might or might not remember my fighting with mod_perl and my finding a bug in the handling of logs if Apache’s error log is set to use the syslog interface (which in my case would be metalog). For those wondering the upstream bug is still untouched goes without saying. This should have told me that there aren’t many people using Apache’s syslog support, but sometimes I’m stubborn.

Anyway, yesterday I finally put into so-called “production” the webapp I described last week for handling customers’ computers. I got it working in no time after mongoid started to behave (tests are still restricted, because a couple fail and I’m not sure why — I’ll have to work on that with the next release that require quite fewer hacks to test cleanly). I did encounter a nasty bug in "best_in_place"http://rubygems.org/gems/best_in_place which I ended up fixing in Gentoo even though upstream hasn’t merged my branch yet.

To get it in “production” I simply mean configuring it to run on the twin server of this blog’s, which I’ve been using for another customer as well — and got ready for a third. Since Rails 3.1 was already installed on that box, it was quite easy to move my new app there. All it took was installing the few new gems I needed and…

Well here’s the interesting thing: I didn’t want for my application to run as my user, while obviously I wanted to check out the sources with my user so that I could get it to update with git … how do you do that? Well, Passenger is able to run the application under whatever user owns the config/environment.rb file, so you’d expect it to be able to run under an arbitrary user as well — which is the case, but only if you’re using version 3 (which is not stable in Gentoo as of yet).

So anyway I set up the new passenger to change the user, make public/assets/ and another directory I write to group-writable (the app user and my user are in the same group), and then I’m basically done, I think. I start up and I’m done with it, I think… but the hostnames tell me that “something went wrong”, without any clue as to what.

Okay so the default for Passenger is to not have any log at all, not a problem, I’ll just increase the level to 1 and see the error… or not? I still get no output in Apache’s error log .. which is still set to syslog… don’t tell me… I set Passenger to log to file, and lo and behold it works fine. I wonder if it’s time for me to learn Apache’s API and get to fix both, since it looks like I’m one of the very few people who would like to use syslog as Apache’s error log.

After getting Passenger to finally tell me what’s wrong, I find out both the reason why Rails wasn’t starting (I forgot to enable two USE flags in dev-ruby/barby which I use for generating the QR code on the label), but I also see this:

Rails Error: Unable to access log file. Please ensure that /var/www/${vhost}/log/production.log exists and is chmod 0666. The log level has been raised to WARN and the output directed to STDERR until the problem is fixed.
Please note that logging negatively impacts client-side performance. You should set your logging level no lower than :info in production.

What? Rails is really telling its users to create a world writeable log file, when it fails to write to it? Are they freaking kidding me? Is this really a suggestion coming from the developers of a framework for Web Applications which should be security-sensitive? … Okay so one can be smarter than them and do the right thing (in my case make sure that the log file is actually group-writeable) but if this is the kind of suggestions they find proper to tell you, it’s no wonder what happened with Diaspora. So it’s one more reason why Rails shouldn’t be for the faint hearted and that you should pay a very good sysadmin if you want to run a Rails application.

Oh and by the way the cherry on top of this is that instead of just sending the log to stderr, leaving it to Passenger to wrangle – which would have worked out nicely if Passenger had a way to distinguish which app the errors are coming from – Rails also moves the log level to warning, just to spite you. And then tells you that it impacts performances! Ain’t that lovely?

Plan for the day? If I find some extra free time I’d like to give a try and package (not necessarily in this order) syslogger so that the whole production.log thing can go away fast.

Today, olemarkus finally removed the dev-lang/php-5.2.17 ebuild from the gentoo-x86 tree.

So it's been nearly 5 years since php-5.2.1-r3 got introduced to the main tree. And it comes almost exactly a year after the last release of php-5.2 was announced on php.net. Half a decade lifetime is pretty decent for release cycles, ain't it?

But with all the new and shiny features in 5.3 (and 5.4!), there's really nothing bad about letting php-5.2 die.

So by all means: PHP is dead, long live PHP!

P.S.: If you find any zombies, ie packages you want to merge but that require php-5.2 to function, please notify the Gentoo PHP team via our bug tracker or leave a comment here. Thanks a lot!

Since the general trend on many linux distros is towards requiring /usr to be mounted at boot time, I figured I’d see what it would take to get it working using dracut.

I’ve been messing with dracut for a while, and for some reason it stubbornly refuses to detect my raid devices. The kernel autodetection works fine, but this is disabled when booting from an initramfs. Dracut would timeout and drop me to a dash shell, and if I just typed mdadm -As followed by exit it would boot just fine.

Dracut is using udev to set up raid devices, and obviously that is not working.

Beyond this, I’d like to get my /usr mounted pre-boot, and there is a module called usrmount that purports to do just this. However, it isn’t working in my case because /usr is a bind mount to a subdir on an lvm volume, and it just isn’t figuring that out (it doesn’t even run lvm in the first place despite having the module installed, let alone figuring out what to mount in what order – I suspect the lvm module only works if root is on lvm).

My solution to both problems is to build my own simple dracut module. If you want to try it out:

1. cd /usr/lib/dracut/modules.d/
2. mkdir 91local
3. cat > 91local/module-setup.sh
   #!/bin/bash
   # -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*-
   # ex: ts=8 sw=4 sts=4 et filetype=sh

   check() {
   return 0
   }

   depends() {
   return 0
   }

   install() {
inst_hook pre-trigger 91 "$moddir/mount-local.sh"
}


4. cat > 91local/mount-local.sh
   #!/bin/sh
   # -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*-
   # ex: ts=8 sw=4 sts=4 et filetype=sh

   mount_local()
   {
   mdadm -As
   lvm pvscan
   lvm vgscan
   lvm lvscan
   lvm vgchange -ay
   }

   mount_local

Then run dracut to build your initramfs, and it should let mdadm and lvm auto-detect everything before it gets to mounting stuff. You can then use the fstab-sys to mount whatever you need to mount user. However, in your fstab.sys if you’re configuring a bindmount be sure to prepend /sysroot/ before the source directory.
Example fstab.sys:
/dev/vg1/data /data ext4 noatime,user_xattr,barrier=1 0 0
/sysroot/data/usr /usr none bind 0 0
/sysroot/data/var /var none bind 0 0

Hopefully this helps somebody out – the dracut documentation is pretty sparse. In fact, if somebody connected to dracut stumbles upon this I’d be open to a better way of hooking my script – pre-trigger just doesn’t seem right – I’d rather let udev try to do everything first. However, I couldn’t find any way to hook after udev runs but before it bombs out not finding my root device. Suggestions welcome.

1) Roll call

alexxy, jmbsvicetto, dilfridge, johu, mschiff, tampakrap, Thev00d00

2) Electing a new team leader

Since one year is not over yet, it will be skipped for the next meeting.

3) What shall we do with kdepim-4.4

KDEPIM 4.4 is not supported any more by upstream, but on the other hand KDEPIM2 is still too buggy. We had a discussion if we should remove it completely or if we should continue maintain it, despite the compatibility bugs that started to emerge with newer KDE versions. Final decision is that we will continue support it as long it works with newer KDE SC releases. We’ll keep the kdepim-l10n split package to provide the translations for it.

4) kdeenablefinal revisited

Since upstream doesn’t seem to care about it much, plus it doesn’t make much sense now that there are many split tarballs, we decided to remove it the next day after the meeting.

5) phonon-xine removal

KDE upstream acknowledged that this is not maintained anymore. It’s already masked since 2011/12/01. Will be last rited and removed 15 days afterwards.

6) Qt 4.8

We expect no big issues with it. Kdenlive is the only known application that does not build at the moment and will be patched. kde-base/kstyles-4.7.* needs to be rebuilt after the upgrade, which we’ll solve with a combination of revbump/dependencies (otherwise KDE apps using oxygen style crash).

7) Dropping RPATH from installed binaries

Postponed for next meeting, need more info from reavertm and/or hardened herd.

To eselect Boost or not to eselect boost

No final decision was taken, discussion will be moved to -dev mailing list.

9) Bugs

* dev-util/cmake picks always the latest boost. Fix in overlay since 13. Dec. Move to tree? https://bugs.gentoo.org/show_bug.cgi?id=335108

see 8.

* cmake-utils.eclass PREFIX is not defined, any progress? https://bugs.gentoo.org/show_bug.cgi?id=358059

Postponed for next meeting

* Remove hard dep on media-libs/phonon from kde-base/kdelibs https://bugs.gentoo.org/show_bug.cgi?id=356681 https://bugs.gentoo.org/show_bug.cgi?id=388041

Although it is possible to build kdelibs against qt-phonon, it is not recommended by upstream. Decision postponed for next meeting.

* Eclass problem with handbook without LINGUAS. https://bugs.gentoo.org/show_bug.cgi?id=372457

Needs more analysis. Postponed.

* MacOSX request for cmake-utils.eclass: Remove force of  CMAKE_BUILD_WITH_INSTALL_RPATH=TRUE https://bugs.gentoo.org/show_bug.cgi?id=398437

That was a request by the Gentoo Prefix team, and got accepted

* Revise the change “semantic-desktop? -> semantic-desktop=”. Why was the change needed. https://bugs.gentoo.org/show_bug.cgi?id=396491

We had split opinions on this. Skipped for next meeting, as we need reavertm’s input on this.

10) Open floor

• Tampakrap will make a KDE SC 4.8 release party in Prague, more info coming soon.
• Qt meeting on Thursday 26th Jan.
• See you at fosdem

Milan is from Foresight Media s.r.o, who produce interactive Touch Tables, that run Gentoo Linux. One of the products are a low cost alternative to Microsoft's Surface. Be sure to check out an overview of their products. Milan, thank you very much for your time. My first question is this:

1. Who is Milan and how did you get started with Gentoo?
   • I guess I'm a product designer, developer, entrepreneur and part time artist living in Central Europe usually in Vienna, Bratislava and Prague. To be able to create inventions, new gadgets you either need a ton of money or you learn how to do many things by yourself in a garage or in my case in my atelier. For me it would be quite depressing to ‘just design' something :) And so I create prototypes, which I push into serial production like my touch table designs. When I was 13 years old I accidentally saw a magazine with a penguin. I thought it was a cool logo of something. Then I saw it said that there's a free CD of an operating system that I haven't heard of. I could not hack my pre-installed commercial software enough and so I gave it a try and I guess it's the usual story of many Open Source and Linux geeks from there on :) After some time using various Linux distributions I saw that the complexity and the number of regressions in many of them has become so high over the years that I needed a system that would let me stay in control and a system that would value it's own design. Gentoo was a natural choice.

Be sure to check out the Full Interview!

Discuss this!

SCALE 10x is almost here, and Gentoo will be there!

Southern California's premier open-source software event is just around the corner, running from Friday, January 20 through Sunday, January 22. Several Gentoo developers will be there; it will be even bigger than previous years.

We'll be showing off some nifty devices running Gentoo, and we'll be giving out installation media. Whether you're a developer, user, or simply curious, be sure to stop by booth #70. See you there!

Discuss this!

To my European readers: if you care about the impact of social technologies like Git (and GitHub) & how they’re transforming software development, or the impact of social technology on communities, and you enjoy good beer, you need to be at Monki Gras. I just posted over at my RedMonk blog about how the previous conference in the series, Monktoberfest, was the best conference of my life. And I’ve been to many.

Monki Gras is Feb. 1–2 in London. The timing’s perfect to stop by just before FOSDEM (and that’s exactly what I’m doing). Registration is dirt-cheap, speakers are universally top-notch, and you’ll also get some world-class beers in the package.

Yesterday (2012/01/16 20:00 UTC) we had the first Gentoo KDE team meeting this year. The meeting happened in #gentoo-meetings on freenode.

• Participants: alexxy, dilfridge, jmbsvicetto, johu, mschiff, tampakrap, Thev00d00
• Agenda
• Log
• Lead election is delayed, because 12 months not over
• We keep kdepim-4.4 in tree as long as it works and provide kdepim-l10n package
• Kdeenablefinal build feature will be removed today
• Phonon xine backend will be removed in 15 days
• We expect no big issues with Qt 4.8,  only kdenlive is not building at the moment
• eselect boost vs. latest boost is not a Gentoo KDE scope only issue, we will move the discussion to the gentoo-dev mailing list
• … read log or wait for the full summary by tampakrap
• My netbook had a kernel panic while the meeting  :-/

I’ve spent the first week of the year on vacation with some friends. The second week of the year has been mixed between going on with the jobs I should have gotten working already, fighting a bad case of cold, and getting insulted by a customer of mine for actually having gotten real vacation time for once in two years. More to the point: said customer doesn’t actually pay me overtime, or actually at all for the support.

Tonight I wanted to relax and think about my own needs. Not personal needs, alas, but at least needs for my work to become easier. Since I haven’t made any progress at all regarding RT I decided to look into a different need of mine: cataloguing customers’ computers.

I originally simply kept a file listing the computers I set up for customers — then I started getting more customers, and sometimes getting a computer back after many months since last time. And I started forgetting which computer was which. Nowadays I have 79 computers on my “database” (which is just a git repository with a bunch of HTML files as well as lshw dumps), without counting those that have been dismissed.

To recognise the computers, I started printing labels with a QR Code on them, which contains the URL of the computer’s HTML file on my website (password-protected). My original method required me to feed a multi-label A4 sheet into my laser printer and print one, two or three labels out on that… but it turned out to be a waste of time and of money in sheets, given that most of the time I ended up wasting half of it, as the printer refused to print aligned more than half the time. I’ve since bought a Dymo label printer, which is why you’ll find their drivers in Portage maintained by yours truly — the nice thing about Dymo’s label printers is that their drivers are fully GPL-2, while as far as I can tell both Zebra and Brother have binary blobs, that make them unsuitable for use on amd64-based systems.

As you can tell, there are a few things that I did in Gentoo that relate to this little “database” of mine: the lshw fixes to try getting it back into SysRescueCD (it’s still not there — and I lost the password for my account on their forums), the Dymo drivers noted above, and dev-ruby/barby which is a quite interesting library that allows you to generate almost any kind of barcode. And now it’s time of MongoDB Ruby libraries as I’m trying to write an actual web application to manage the “database” and make it a real database.

Today’s achievement is big: I finally got Rails (3.1) to play nice with MongoDB. Not using MongoMapper, the author of which, as I already talked about I would prefer not having much to discuss with. But thanks to Mauro I got pointed at Mongoid which is a much more well developed alternative.

Okay sure there are quite a few things to kink out in the packaging of Mongoid – for instance the fact that the gem packages a Rakefile that relies on a (missing) Gemfile, or the fact that two out of three rspec targets in said Rakefile fail, one of which by crashing the interpeter – but at least their unit-tests work, and the code works as intended when loaded it up. Which is more I can say about MongoMapper.

Oh and it doesn’t seem to require extra code to be added just to work correctly with Passenger.

The only problem I have now is fixing up one side issue: how do I print the labels once I load this into my webserver? I could download the PDF I use to print the label and then print that.. but it’s a bit of a time-waster. Of course both the server and Yamato (where the label printer is connected) are IPv6-enabled and .. well, the IPP protocol used by CUPS is fine to be used over the internet, as it can use SSL encryption. Which yes, means that I’ll be setting up a web application … that calls home to print a label, how crazy is that?

My only issue with this is that I’d rather not install cups on the webserver (especially since there is currently no way to just build the client side of it, which would be the only part of it I would need on the server — yeah I know, it’s funky), so I can’t just call lpr mylabel.pdf… and as far as I can tell, the only way to access IPP from Ruby is one of the many CUPS library bindings available as gems, which are all 0.0 versions, and do not inspire me the least. Since IPP is based off HTTP, I would have expected more implementations of it, to be honest.

Possibly, it should be possible to extend some HTTP Ruby library to send IPP requests as well; for what I’m concerned, I’d just need the “Print-Job” method to be implemented, which would allow me to send the PDF file to be printed with the default options. I guess I’ll resolve that bit once I’m done with the rest of my application, though.

I’m no fan of initramfs. All my systems boot up just fine without it, so I often see it as an additional layer of obfuscation. But there are definitely cases where initramfs is needed, and from the looks of it, we might be needing to push out some documentation and support for initramfs. Since my primary focus is to look at a hardened system, I started playing with initramfs together with Gentoo Hardened, grSecurity and SELinux. And what a challenge it was…

But first, a quick introduction to initramfs. The Linux kernel supports initrd images for quite some time. These images are best seen as loopback-mountable images containing a whole file system that the Linux kernel boots as the root device. On this initrd image, a set of tools and scripts then prepare the system and finally switch towards the real root device. The initrd feature was often used when the root device is a network-mounted location or on a file system that requires additional activities (like an encrypted file system or even on LVM. But it also had some difficulties with it.

Using a loopback-mountable image means that this is seen as a full device (with file system on it), so the Linux kernel also tries caching the files on it, which leads to some unwanted memory consumption. It is a static environment, so it is hard to grow or shrink it. Every time an administrator creates an initrd, he needs to carefully design (capacity-wise) the environment not to request too much or too little memory.

Enter initramfs. The concept is similar: an environment that the Linux kernel boots as a root device which is used to prepare for booting further from the real root file systems. But it uses a different approach. First of all, it is no longer a loopback-mountable image, but a cpio archive that is used on a tmpfs file system. Unlike initrd, tmpfs can grow or shrink as necessary, so the administrator doesn’t need to plan the capacity of the image. And because it is a tmpfs file system, the Linux kernel doesn’t try to cache the files in memory (as it knows they already are in memory).

There are undoubtedly more advantages to initramfs, but let’s stick to the primary objective of this post: talk about its implementation on a hardened system.

I started playing with dracut, a tool to create initramfs archives which is seen as a widely popular implementation (and suggested on the gentoo development mailinglist). It uses a simple, modular approach to building initramfs archives. It has a base, which includes a small init script and some device handling (based on udev), and modules that you can add depending on your situation (such as adding support for RAID devices, LVM, NFS mounted file systems etc.)

On a SELinux system (using a strict policy, enforcing mode) running dracut in the sysadm_t domain doesn’t work, so I had to create a dracut_t domain (which has been pushed to the Portage tree yesterday). But other than that, it is for me sufficient to call dracut to create an initramfs:

# dracut -f "" 3.1.6-hardened

My grub then has an additional set of lines like so:

title Gentoo Linux Hardened (initramfs)
root (hd0,0)
kernel /boot/vmlinuz-3.1.6-hardened root=/dev/vda1 console=ttyS0 console=tty0
initrd /boot/initramfs-3.1.6-hardened.img

Sadly, the bugger didn’t boot. The first problem I hit was that the Linux kernel I boot has chroot restrictions in it (grSecurity). These restrictions further tighten chroot environments so that it is much more difficult to “escape” a chroot. But dracut, and probably all others, use chroot to further prepare the bootup and eventually switch to the chrooted environment to boot up further. Having the chroot restrictions enabled effectively means that I cannot use initramfs environments. To work around, I enabled sysctl support for all the chroot restrictions and made sure that their default behavior is to be disabled. Then, when the system boots up, it enables the restrictions later in the boot process (through the sysctl.conf settings) and then locks these settings (thanks to grSecurity’s grsec_lock feature) so that they cannot be disabled anymore later.

But no, I did get further, up to the point that either the openrc init is called (which tries to load in the SELinux policy and then breaks) or that the initramfs tries to load the SELinux policy – and then breaks. The problem here is that there is too much happening before the SELinux policy is loaded. Files are created (such as device files) or manipulated, chroots are prepared, udev is (temporarily) ran, mounts are created, … all before a SELinux policy is loaded. As a result, the files on the system have incorrect contexts and the moment the SELinux policy is loaded, the processes get denied all access and other privileges they want against these (wrongly) labeled files. And since after loading the SELinux policy, the process runs in kernel_t domain, it doesn’t have the privileges to relabel the entire system, let alone call commands.

This is currently where I’m stuck. I can get the thing boot up, if you temporarily work in permissive mode. When the openrc init is eventually called, things proceed as usual and the moment udev is started (again, now from the openrc init) it is possible to switch to enforcing mode. All processes are running by then in the correct domain and there do not seem to be any files left with wrong contexts (since the initramfs is not reachable anymore and the device files in /dev are now set again by udev which is SELinux aware.

But if you want to boot up in enforcing straight away, there are still things to investigate. I think I’ll need to put the policy in the initramfs as well (which has the huge downside that every update on the policy requires a rebuild of the initramfs as well). In that case I can load the policy early up the chain and have the initramfs work further running in an enforced situation. Or I completely regard the initramfs as an “always trusted” environment and wait for openrc’s init to load the SELinux policy. In that case, I need to find a way to relabel the (temporarily created) /dev entries (like console, kmsg, …) before the policy is loaded.

Definitely to be continued…

Maybe some of you have noticed that CUPS 1.5.0 is still hard-masked. Well, the reason for that is simple- at home, I'm nearly not printing at all, and at work, I have to rely on printing too much to tinker with it on the side a bit. So... if you would like to help, please unmask net-print/cups-1.5.0-r2 and give it a try. You will for sure find some problems, as the only thing I tested looong time ago was building it, never actually running it. Report them on bugs.gentoo.org, and we'll have a look... with a bit of luck, the package mask can then go away at some point. Any feedback (also positive) is appreciated. Cheers!

As known as #335108. This is (was) a long term bug in Gentoo KDE scope. The problem is that if you have two or more different boost versions installed, the latest version will be used at build time, regardless which version is (e)selected. Real world example we have boost  1.46.1 and 1.47.0 installed selected the 1.46 slot, the 1.47 slot would be used at build:

$ eselect boost list
Available boost versions:
 [1]   boost-1.46/default *
 [2]   boost-1.47/default

Last night i patched dev-util/cmake-2.8.6 successfully and made the revision bump today in the kde-overlay. So please test =dev-util/cmake-2.8.6-r5, in the case your maintained package is cmake based and needs dev-util/boost at build time. You should test at least with two different boost versions and of course switch between those to check that the selected version is used.

I bumped dev-util/cmake-2.8.7 in the overlay too. The patch is also included in this version.

Start your engines…

While testing kernel 3.1.6 for bug #396469 I got the common kernel panic "not syncing: VFS: Unable to mount root fs".

It's easy to fix, here's the grub config before:


title Gentoo Linux
root (hd0,0)
kernel /boot/vmlinuz-3.1.6-gentoo

And fixed one:


title Gentoo Linux
root (hd0,0)
kernel /boot/vmlinuz-3.1.6-gentoo root=/dev/sda1

I had to pass an explicit root= parameter. How to figure it out? mount -l or cat /proc/mounts are not so helpful:


rootfs on / type rootfs (rw)
/dev/root on / type ext3 (rw,noatime,errors=continue,barrier=1,data=writeback)

This is just a short post to let my followers know that I’ll be at FOSDEM next month. I’ve booked the flight back in September and I booked the hotel yesterday, so it’s all set. I just hope not to get lost through Bruxelles.

The only reason why I’m posting this is, actually, that I need some suggestion from somebody who knows Belgium: both my phone operators lack dedicated roaming up there, so I’ll probably end up with an hefty bill waiting for me back home. Given in Italy you really can’t get a local pre-paid SIM to user your phone if you’re a tourist, I’m not sure if the same holds true in Belgium. And most importantly, whether I could re-use such a SIM over the years (as I plan on coming to FOSDEM with regularity, if I survive the trip alone this time).

At any rate, if you want to discuss anything in person, I’ll be the guy with the strange hat and the purse satchel (geek points for getting the reference), hanging around with the Gentoo or libav folks.

I thought it would be easier to do a round of stable kernel releases in the middle of the larger kernel merge window, to prevent the next round from being so big (given that there are a lot of patches usually applying during the -rc1 merge window cycle).

So, I've now done:

• Linux 2.6.32.54-rc1 release
• Linux 3.0.17-rc1 release
• Linux 3.1.9-rc1 release
• Linux 3.2.1-rc1 release

Please go test and let me know if there are any problems with any of these kernels. If I've missed any patches that you feel should be in them, also please let me know.

Note, this is most likely going to be the LAST 3.1.y kernel release, so please move off to the 3.2 kernel at this point in time. Maintaining so many different kernel branches all at once is not trivial, and I want to minimize it if at all possible.

Some time ago when we were filing the first stable request for KDE 4.7 I decided I'd have to give KDEPIM-4.7 also a try. I used to be a pine (and later alpine) user for ages, some time ago I switched to kmail1 (maybe at version 4.2 or so) and have been using it ever since... About the setup, incoming mail is stored and filtered server-side (Novell Groupwise), and accessed as disconnected IMAP.

1) Office desktop.
This one's usually running the latest and greatest KDE RC or beta, with a static IP and something like a 100MBps-FD link to the mail server. No local folders, so this was the natural candidate to test first. I did not even try the data migration, but wiped my entire local configuration of the KDEPIM programs as thoroughly as possible before updating. On the whole this went rather well. From 4.7.3 to 4.7.95 I've seen my share of kmail2 / kontact / akonadi crashes, but none of them really led to bigger problems. Impressively, I could also see each of these crash bugs I hit get fixed on KDE bugzilla in the meantime! Since upgrading to 4.7.97 I haven't had a crash anymore.
The only thing that is really bugging me a the moment is that I absolutely can't drag an e-mail message into another folder (I always have to right-click on the entry in the message list, select "Move to folder", ...). However, that is likely a problem in one of the underlying libraries. :| I would really like to help debugging this problem; if you can give me any clue where to look, please message...

2) Laptop.
Going home for christmas meant spending some time on a train. The days before I had upgraded this box to QT-4.8.0 and KDE+KDEPIM 4.7.95. Again, clean start with a new configuration. On the train I spent some time ironing out bugs there. The GPRS connection was happily going down and up again with every tunnel or less-populated countryside. That's when the akonadi backend started really acting up. When I arrived at my family's place, my e-mail had become fully non-functional (no fetching e-mails, no sending e-mails, the backend making kmail hang, regular crashes, and all fumbling with akonadiconsole did not help). After a while of trying, I gave up, wiped my entire KDEPIM configuration and data and downgraded to 4.4 again. I was impressed how responsive my e-mail program suddenly became.
Talking to other people, it seems that some have problems with bad internet connections, some don't. Maybe I should have used networkmanager, at least Alexxy reports that it's working fine with it.

3) Home desktop.
Well... Here I store my e-mail archive since 1996- that's maildir folders with roughly 50000 messages. Maybe I'll consider upgrading KDEPIM sometime around KDE 4.9.

I'm sorry if this blog entry is demotivating for the program developers. Some time ago I followed part of an animated discussion on irc between a kdepim user and a kdepim developer. Quoting a small part,

<user> first: fix bugs. second:dont change data formats. third: change gui only if absolutly nessary.
<developer> no
<developer> first: have fun; second: make it work for others

As 3.2 is now out, here's a note as to the current status of the different stable/longterm kernel trees.

First off, please everyone remember to mark any patch that you want to have applied to the stable kernel trees with a simple:

Cc: stable <stable@vger.kernel.org>

marking in the Signed-off-by: area. Once the patch hits Linus's tree, I will automatically be notified of it and it will be applied if possible. If it does not applied, you will be notified of that.

Note that the address is stable@vger.kernel.org, not the older address that used to be used before October of 2011.

At this time, all stable and longterm kernel trees are being maintained in one big git tree, located at:

git.kernel.org:/pub/scm/linux/kernel/git/stable/linux-stable.git

There are different branches for every different major kernel version.

Here's the different active kernel versions that I am maintaining at the moment:

• 3.2.y - this will be maintained until 3.3 comes out
• 3.1.y - there will be only one, maybe two, more releases of this tree
• 3.0.y - this is the new "longterm" kernel release, it will be maintained for 2 years at the minimum by me.
• 2.6.32.y - this is the previous "longterm" kernel release. It is approaching it's end-of-life, and I think I only have another month or so doing releases of this. After I am finished with it, it might be picked up by someone else, but I'm not going to promise anything.

All other longterm kernels are being maintained in various forms (usually quite sporadically, if at all), by other people, and I can not speak for their lifetime at all, that is up to those individuals.

If anyone has any questions about any of this, please let me know.

I've just finished stabilizing KDE 4.7.4 on x86, and it was about 300 packages, so it would be very hard to do without good tools. I have some pretty nice scripts, so it took me maybe a few supervision and everything has been done automatically (even then it can take 3-4 hours, so trying to do that manually would be a huge waste of time and take even longer).

Here's how I did that. First I created a ~/kde-stable.txt file with the following contents:


# bug #396359
=app-misc/strigi-0.7.7
=app-office/akonadi-server-1.6.2-r1
=dev-libs/shared-desktop-ontologies-0.8.1
=kde-base/activitymanager-4.7.4
=kde-base/akonadiconsole-4.7.4
=kde-base/akregator-4.7.4

NetworkManager 0.9 has changed a lot of stuff. Before NM 0.9, there are user connections and system connections. Ifnet plug-in is made to only take charge of system connections. But after NM 0.9, all connections become system connections and ifnet tries to manage all of them. Then some bad things might happen. The problem is reported by David Narvaez and he helped a lot on it. Bug #392409 is also caused by the problem. This is fixed in networkmanager-0.9.2.0-r2: if a user does not check “available to all users”, ifnet will not handle the connection any more and the default plug-in “keyfile” will do the job.

Another important change is that ifnet now supports openrc style. As bash arrays are not allowed by openrc, ifnet caused Bug #369711. After networkmanager-0.9.2.0-r2, ifnet can read both the old baselayout style (with bash arrays) and openrc style (without bash arrays), but it only writes to /etc/conf.d/net with openrc style.

Now that I think of it, I really regret I didn’t make systemd ebuild install it to /usr from the very beginning. But the harm has been done already, and I’d like to move it ASAP and that’s why I’d like to sum up problems with that and possible ways of proceeding with it.

The idea

The idea is simple as it is: move systemd install to /usr prefix completely. Right now, there are no technical benefits from keeping it in rootfs. It already depends on libdbus, which is installed in /usr, and I expect more dependencies over time. There’s no reason to move all those packages into rootfs.

Most importantly, the above information allows me to assume that such move won’t hurt our split-/usr users — because they already had to have /usr mounted for systemd to run.

The trouble

The main problem with the move is that unit files were installed into /lib location by a number of packages. The files can be moved into the new location cleanly only through rebuilding the packages which provide it. They need to stay in search path for systemd to work.

These unit files are symlinked from within /etc/systemd as well. Whenever we move a single unit, we need to update the symlink as well. I’d really like to avoid forcing users to manually fix that, and the eclass doesn’t export pkg_postinst() which could help doing it automatically.

The last problem is that people have the systemd location hardcoded into the kernel command-line. This one should be relatively easy to avoid as we can keep compatibility symlink for some time.

Solution 1: one big move

The simplest solution for the migration is to move all the relevant units in the systemd ebuild. This way, the unit integrity can be preserved, and symlinks could be updated at once as well. The risk of system breakage should be reduced to minimum.

However, there is an important disadvantage of that method. All those files would be moved out-of-scope of the Package Manager. Thus, after rebuild of every single package providing systemd units, all of our users will have to fight file collisions.

The same will likely apply to our new users, because they will have at least some units installed by random packages already. Users not ever intending to use systemd won’t be hurt because the move of unit files will be transparent to them as any other package file move.

Solution 2: temporary support for two locations

Right now, systemd already supports multiple locations for unit files. As a temporary hack, we could just add /lib/systemd/system to that list. This way, all unit files still installed in /lib will still work as expected when systemd is installed into /usr.

Sadly, this won’t handle updating /etc symlinks. I could, however, fix that easily by adding a simple .path unit or another solution updating symlinks as soon as files are removed from /lib.

Other solutions?

Well, I think the second one is the best we can do. Do you have any other ideas? I guess that udev could face a similar problem if we decide to actually move it into /usr. And there the thing is even worse because the rule install location is usually hardcoded into the ebuild or package build system itself; we will probably need to have even more degree of compatibility.

Figure 1.1: Heating your boxes since 1999

Gentoo Linux is proud to announce the availability of a new LiveDVD to celebrate the continued collaboration between Gentoo users and developers. The LiveDVD features a superb list of packages, some of which are listed below.

A special thanks to the Gentoo Infrastructure Team. Their hard work behind the scenes provide the resources, services and technology necessary to support the Gentoo Linux project.

• Packages included in this release: Linux Kernel 3.1.5, Xorg 1.10.4, KDE 4.7.4, Gnome 3.2.1 XFCE 4.8, Fluxbox 1.3.2 Firefox 9.0, LibreOffice 3.4.99.2, Gimp 2.6.11, Blender 2.60, Amarok 2.5 , VLC 1.1.13, Chromium 16.0 and much more ...
• If you want to see if your package is included we have generated both the x86 package list, and amd64 package list. You may also be interested in checking out the FAQ or the updated artwork plus dvd cases and covers for the 12.0 release.
• Special Features:
  • Writable file systems using AUFS so you can emerge new packages!
  • Persistence for $HOME is available; press F9 for more info!

The LiveDVD is available in two flavors: a hybrid x86/x86_64 version, and an x86_64 multi lib version. The livedvd-x86-amd64-32ul-12.0 version will work on 32-bit x86 or 64-bit x86_64. If your CPU architecture is x86, then boot with the default gentoo kernel. If your arch is amd64, boot with the gentoo64 kernel. This means you can boot a 64-bit kernel and install a customized 64-bit user land while using the provided 32-bit user land. The livedvd-amd64-multilib-12.0 version is for x86_64 only.

If you are ready to check it out, let our bouncer direct you to the closest x86 image or amd64 image file. If you would prefer to take it easy on the bandwidth and help the community, torrents are also provided.

If you need support or have any questions, please visit the discussion thread on our forum.

Thank you for your continued support,
Gentoo Linux Developers, the Gentoo Foundation, and the Gentoo-Ten Project.

Hi all,

Two weeks ago I got a Beaglebone board from the people at Beagleboard.org to create the documentation I always create with every device I get.

Like always i’d like to announce the guide for installing Gentoo in the Beaglebone. Have a look at: http://dev.gentoo.org/~armin76/arm/beaglebone/install.xml . Feel free to send any corrections my way.

The Beaglebone is a bit different from the devices i got lately, as it lacks video out(well, there’s video out with an LCD connector but i lack an LCD screen), and its pretty simple apart from that. One of the big points of the Beaglebone is the price(89$) and the ability of creating addon boards for it, which are already famous amongst the Beagleboard people.

The specs of the Beaglebone are:
# ARMv7-A 500MHz(USB power)/720MHz(PSU power) TI AM3358/9 ARM Cortex-A8 processor
# 256MB DDR2 RAM
# SMSC LAN8710 Ethernet card
#
# 1x microSDHC slot
# 1x USB 2.0 Type-A port
# 1x mini-USB 2.0 OTG port
# 1x RJ45
#
# Reset and user-defined button

Yes, the processor runs at 500MHz when being powered using the mini-USB port, and 720MHz when using a power supply. More info about the specs in Beaglebone’s webpage.

For those curious as me, here’s the bootlog and the cpuinfo.

All the hardware provided by the Beaglebone works fine, except the USB port. But thats due to a bug in the USB driver used on the Beaglebone. The bug appears when you disconnect the first USB device you connect. Once that happens, the USB port won’t recognize any new USB device.

The workaround is to do:
echo F > /proc/driver/musb_hdrc.1

After that, the USB port will work again. This idea comes from the Angstrom people. In the guide i’ve documented an udev rule to workaround this issue, its from Angstrom as well.

I’d like to thank the people at Beagleboard.org for providing me a Beaglebone to document this. Next step is getting everything upstream

Have fun!

Thanks to Luca Barbato (known as lu_zero in Gentooland), I’ll be able to put my hands for some days on this nice ARM board made by Genesi, the Efika MX.

As it happened with the BeagleBone, the idea is to work out the kernel, bootloader integration and create weekly automatic builds of Gentoo and Sabayon images (those you can just cat to the MMC card). It is more or less what happens with Gentoo stage3 autobuilds, but here we have kernel binaries and u-boot, so that the image can boot straight away.

So far, qemu-user (statically compiled) + OpenSUSE patches, besides minor bugs, are working quite well. Depending on the availability of these boards on the market and project financial capabilities, we’ll be able to couple qemu-user with native hardware on the build server making them work side by side without the need of a shared binhost repo (sshfs + chroot magic, I’ll talk about it in future).

One more step for these chroots will be attaching matter to it, as we already do for i686 and x86_64 ones. This way the compilation of new ebuilds (and 70% of chroot work) will be completely automated.

One week ago, the BeagleBone I ordered from Tigal.com eventually landed on my desk. As you may know, it’s an ARMv7a OMAP device with 256Mb RAM, USB 2.0 and FastEthernet 10/100. The board doesn’t come with HDMI output but daughter boards are expected to be shipped soon it seems.

It ships with Angstrom Linux, an embedded distribution that is no way close to Gentoo, at least in my opinion. I found it kinda broken and slow. opkg, the package manager, is a nightmare. I had no choice, that thing had to be dumped ASAP. And that’s what I did.

First thing I did was copycating the Angstrom kernel configuration by copying /proc/config.gz in a safe place and starting to merge the Beagleboard kernel tree into my Linux 3.1 branch. This of course means that the AM335x is not yet supported by the vanilla kernel. Last I heard is that there are plans to merge the patches in the 3.3 merge window… You can find two ebuilds in the “sabayon-distro” overlay: sys-kernel/beagleboard-sources and sys-kernel/linux-beagleboard, providing sources and binaries respectively.
Introducing ARM support in sabayon-kernel.eclass (the eclass that builds kernel binaries using genkernel) was quite straightforward, it now builds uImages directly!

The boot strategy works like this: u-boot.img searches /boot/uImage into the root filesystem (ext4 doesn’t seem to work with my image). In our case, /boot/uImage is a symlink pointing to a versioned file (the one installed by sys-kernel/linux-beagleboard). You can manage the symlink using eselect-uimage, from the “sabayon” overlay and shipped with the Sabayon images already. This means that you can change the boot kernel at runtime without even touching the boot partition!

The second thing was setting up a chroot, both Entropy build chroot (for pushing out binary packages to the armv7l repo) and “image” chroot (the one from where images are generated) using qemu-user to emulate armv7l. In order to be able to prepare disk images using loop devices, I also completely rewrote the famous “mkcard.txt” script, dropping bc dependency (hey, bash can do math already!!). You can find it here, as well as molecules that we use to build the ARMv7a images for Beagle{Bone,Board}.

If you are interested in knowing more about how I managed to get Sabayon on ARM, have a look at the “Hitchhikers Guide to the BeagleBone” on our wiki.

I uploaded the Sabayon ARMv7 images on our mirrors this morning, under the iso/daily directory, in different sizes (depending on your MMC card size): 4GB, 8GB, 16GB.

a75fd2a7d9cae17762034c5c049a08fc Sabayon_Linux_DAILY_armv7a_Base_16GB.img.xz

0e17081050fa19c7f769318e3235ebaa Sabayon_Linux_DAILY_armv7a_Base_4GB.img.xz

56606bc906715cdebce02611ae03285c Sabayon_Linux_DAILY_armv7a_Base_8GB.img.xz

Installing them onto your MMC card is as easy as running:

xzcat <image file>.xz > /dev/sdX

Where /dev/sdX is your memory card device (might be mmcblk0).
They come in different sizes, make sure to match the advertised image size with your MMC device one.

If you have 32GB or 64GB MMCs you have two choices: either use the 16GB version and create a separate partition later or take the bootfs and rootfs images (they come in different sizes but the content is the same) from the same dir:

93960b28cde8dde51f2d29cc0c76f6bb Sabayon_Linux_DAILY_armv7a_Base_16GB.img.bootfs.tar.xz

e85fe8d344dacf79eec94562a59c6750 Sabayon_Linux_DAILY_armv7a_Base_16GB.img.rootfs.tar.xz

93960b28cde8dde51f2d29cc0c76f6bb Sabayon_Linux_DAILY_armv7a_Base_4GB.img.bootfs.tar.xz

1a6bce6f585d52f2b50806bd2bd69578 Sabayon_Linux_DAILY_armv7a_Base_4GB.img.rootfs.tar.xz

93960b28cde8dde51f2d29cc0c76f6bb Sabayon_Linux_DAILY_armv7a_Base_8GB.img.bootfs.tar.xz

13b2a47a88c55c8692ce61fc2fd42022 Sabayon_Linux_DAILY_armv7a_Base_8GB.img.rootfs.tar.xz

This way you can create your own partition layout and then unpack the content into the respective partitions. So easy. No grub nor MBR nightmare!
If you are as lazy as me, here is the download link to the directory (using the GARR mirror). But you are encouraged to use our download page to find the mirror closest to you.

The root password is: root. The OS is set to automatically boot and start eth0 and sshd (so you can connect to it via ssh). During the first boot, there is a script that configures some stuff and reboots your device automatically (so on the first boot it will just reboot). If you want to change the locale, edit /etc/env.d/02locale and run locale-gen !. The System is already configured to allow serial login (at least this works on the BeagleBone out of the box).

That’s it. You have a great distro (Gentoo) with a great Package Manager, Entropy (Sabayon) in a credit-card sized device.
If you appreciate our efforts towards the ARM architecture, please consider to donate us either hardware or MONEY to buy it! (yeah, we can’t just handle money, we always need money!).
If you’re a developer and interested in ARM stuff, why don’t you join us? We can improve both Gentoo and Sabayon together!

Please note: I only have a BeagleBone for now thus I wasn’t able to test out the images on the BeagleBoard. Moreover, the boot partition contains Beagle* related boot binaries that won’t work on other OMAP devices out of the box (but still, we provide split boot and root filesystem images).

Please note 2: I consider this a tech preview because at the time of writing, we only have 400 binary packages available for install. You can browse them using http://packages.sabayon.org.

DONATE US !

Today I finally closed bug 203706, 4 years after it was initially opened. That means that you can now go and install ruby 1.9.3 without having to unmask the packages and associated use flag. It also means that we'll try to add the ruby19 flag, when possible, to new ebuilds as we add them. With ruby 1.8 no longer in maintenance starting June 2012 we can now work towards making ruby 1.9 the primary stable ruby implementation. At this point in time, though, we don't officially support running your whole system on ruby 1.9 yet.

Unless you take action yourself nothing will change for the moment. If you want to start installing your ruby packages also for ruby 1.9, then you must add ruby19 to your RUBY_TARGETS in /etc/make.conf.

  RUBY_TARGETS="ruby18 ruby19"

Note that this will only work in the testing tree for the moment. If you find bugs, please report them or drop by in #gentoo-ruby.

I have been playing with the Gentoo Wiki the last few days and am very impressed with the work that both the wiki teams as well as existing contributors have already done to the place. The look and feel is very slick and editing works just as expected. One of the changes I made was to move SELinux module information to the wiki. This documentation was originally intended to be published on the Gentoo SELinux Project page, but is easily accessible and maintainable on the wiki too.

So I went a step further and dug up my original GLEP 0051 – Gentoo Knowledge Base proposal and checked how far I could use the Gentoo WiKi for this purpose. From the looks of it, the WiKi can offer a great deal of leverage for this idea and although not everything is supported through the WiKi (like natural search language and such), that might have been overshooting a bit. So we received a Gentoo WiKi Knowledge Base namespace under which the Knowledgebase entries can reside.

Now what is the idea behind such a knowledge base? Well, first of all, the articles below this prefix should all follow the same structure (as explained in the main page) and be sufficiently specific so that the title of the entry should leave little room for misinterpretation. But other than that, there is no limit as to what the Knowledge Base could hold. To that respect, the knowledge base section then provides a (hopefully) thorough listing of common and less common issues with a good explanation why the problem occurred and how to resolve it.

For the time being, the location doesn’t hold that many entries yet, but I will add them as they come along. And of course, feedback is always appreciated ;-)

On a second note, I’d like to give my PoV on the wiki and its relation with the official Gentoo documentation. Unlike what might be circulating, I’m definitely not against the wiki for documentation, on the contrary. Wiki’s have proven to be a good resource for documentation, and because we can never have enough documentation writers, every method for getting more documentation is welcome. But because of its online nature, offline documentation development (which I frequently do) is not possible. Also, keeping translations in sync might be a bit more challenging compared to a file-based solution with version control (otoh, I have little experience with WiKi translations so I might be wrong here).

I strongly believe that the wiki will play a big role in Gentoo’s documentation assets. Many of the documents currently managed by the GDP or the subprojects might be suited to be hosted on the WiKi, especially when those documents are too specific (and as such would require a very specific developer profile to maintain the documents). In such cases, the maintainers of those documents should be able to pick the most efficient method. But for very generic documents, this might not be an easy option.

At least the Gentoo documents now support CC-BY-SA 3.0, so we can migrate documents from the wiki to the main site, and the 2.5 version currently used the most on the main site should be forward compatible with 3.0 (if I read the legalese text well) so we might be able to migrate documents from the main site to the wiki too.

Edit: a3li created the “Knowledge Base” namespace on the wiki, so I updated the links in my post. Thanks for the work on the wiki, a3li!

One of the features supported through OVAL (and Open-SCAP) is to generate fix scripts when a test has failed. The administrator can then verify this script (of course) and then execute it to correct wrong settings. So I decided to play around with this as well and enhanced the Gentoo Security Benchmark (XCCDF source) with some fixables (like for the sysctl settings). And lo and behold: the thing works ;-)

After evaluating the XCCDF (together with the OVAL document) against my system, I had Open-SCAP generate a fix script:

# oscap xccdf generate fix --result-id OSCAP-Test-Gentoo-Default xccdf-results.xml
#!/bin/bash
# OpenSCAP fix generator output for benchmark: Gentoo Security Benchmark

# XCCDF rule: rule-sysctl-ipv4-forward
echo 0 > /proc/sys/net/ipv4/ip_forward

# generated: 2011-12-23T14:53:03+01:00
# END OF SCRIPT

Now isn’t that nice. But generating a fix script is one thing, maintaining the XCCDF and OVAL documents is a completely other picture.

One of the downsides that I talked about earlier already is that OVAL has quite an extensible language (it’s a large XML document). Although this extensibility is very flexible and powerful, when you want to add generic tests (like validating sysctl values or matching regular expressions in files) having to write over 30 lines of XML code for a single test is time-consuming at the least. So I quickly scripted something to help me maintain these settings.

The Generating OVAL documents with genoval.sh document explains this script (which is retrievable from my git repository) whose primary purpose is to transform a single line into the entire OVAL structure. With this script, I can now just say gentoo variable USE must contain ssl and it generates both the rules in the XCCDF as the OVAL statements in the OVAL document.

Okay, it’s a script, not a feature-full application, but at least it helps me (and perhaps others as well).

I’ll write another post, expanding from a single tweet of mine:

If you release code and are not interested in helping distributions package it… don’t release code. Honestly.

It might sound harsh and mean, but I mean it, and I think it’s for the best.

You might have a number of different reasons to work on opensource projects, but you can basically find that the people who write open source projects and then don’t care about distributions and other kind of integrators, tend to be those developers that only work for their own ego. I don’t mean they write bad code, sometimes they might actually be the best programmer, but their dislike for distributors is poisonous for the whole opensource environment.

Why do I say this? Well, the problem is that if you release good opensource code, whether it’s packaged by distributions or not, it might very well happen that some other developer will find and use it, that’s what you want, when you release it opensource, isn’t it? Now when that project is no longer a library or backend tool, but rather an user facing tool that the developer wants to see used, there will be requests to package it. But once that happens, you might be stuck with some backend dependency that cannot be clearly packaged. In that situation, the whole frontend can’t be packaged, and that means that the distribution’s users will end up suffering.

This is getting even worse with gems, especially when the most insightful comment in my latest post comes from an user that seems to proclaim himself a troll (or at least the nick suggests strongly so). Indeed, trying to excuse Ruby gems developers for not caring about our needs (which usually boil down to having tests around, a clear repository, the ability to rebuild documentation and tests without bringing in a whole lot of gem-building tools – jewelwer, hoe, and so on – and most importantly code that works and tests green, which is by far not obvious when dealing with gems), is just saying that you care about open source for your own feel good, rather than for the whole environment.

This gets worrisome when it comes to Ruby stuff, especially because of the ties with Rails. I guess the majority of the gems are developed for usage with Rails, which means web-facing code; if they are developed without caring about the opensource environment and their possible users, it would be silly not to expect that at least a part of them are so insecure they are likely to cause vulnerabilities in code that is actually published for the general usage.

So I think I just decided to make two personal rules, that I wish were widespread:

• if you do not plan on helping distributions to get your software packaged, don’t release it opensource;
• if you plan on making your software opensource, do not use software that distributions can’t package.

It is definitely not a coincidence that whenever I have to dive into Gentoo Ruby packaging I end up writing a long series of articles for my blog that should have the tag “Rant” attached, until I end up deciding that it’s not worth it and I should rather do something else.

The problem is that, as I said many times before (and I guess the Debian Ruby team agrees as well), the whole design of RubyGems makes it very difficult to package them properly, and at the same time provides the developers with enough concepts to make the packaging even more tricky than it would by merely due tot he format.

As the title says, for one reason or another, RubyGems’s main accomplishment is simply to put extensions’ developers and distributions’ packages one against the other, with the former group insisting on doing things “fun”, and the latter doing things "right’. I guess most of the members of the former group also never tried managing a long term deployment of their application outside of things like Heroku (that are paid to take care of that).

And before somebody tells me I’m being mean by painting the developers puny with their concept of fun, it’s not my fault if in the space of an hour after tweeting a shorter version of the first paragraph of this post, two people told me that “development is fun”… I’m afraid for most people that’s what matters, it being fun, not reliable or solid…

At any rate… even though as we speak nobody expressed interest (via flattr) on packaging of the Ruby MongoDB driver that I posted about yesterday, I started looking into it (mostly because I’m doing another computer recovery for a customer and thus I had some free time in my hands while I waited for antivirus to complete, dd_rescue to copy data over, and so on so forth).

I was able to get some basic gems for bson and mongo working, which were part of the hydra repository I noted, but the problems started when I looked into plucky which is the “thin layer” used by the actual ORM. It is not surprising that this gem also is “neutered” to the point of being useless for Gentoo packaging requirements, but there are more issues. First of all it required one more totally new gem to be packaged – log_buddy which also required some fixes – that is not listed in the RubyGems website (which is proper, if you consider that the tests are not executable from the gem file), but most importantly, it relied on the matchy gem.

This is something I already had to deal with, as it was in another long list of dependencies last year or the one before (I honestly forgot). This gem is interesting: while the package is dev-ruby/matchy, it was only available as a person-specific gem in Gemcutter: jnunemaker-matchy and mcmire-matchy; the former is the original (0.4.0), while the latter is a fork that fixed a few issues, among which there was the main problem: jnunemaker-matchy is available neither as a tarball nor as a git tag.

For the package that originally required matchy for us (dev-ruby/crack), mcmire’s fork worked quite well, and indeed it was just a matter of telling it to use the other gem for it to work. That’s not the case for plucky, even thought jnunemaker didn’t release any version of matchy in two years, it only works with his version of matchy. Which meant packaging that one as well, for now.

Did I tell you that mcmire’s version works with Ruby 1.9, while jnunemaker’s doesn’t? No? Well, I’m telling you now. Just so you know, almost in 2012, this is a big deal.

And no, there is not a 0.4.0 yet. Two years after release. The code stagnated since then.

Oh and plucky’s tests will fail depending on how Ruby decides to sort an Hash’s keys array. Array comparison in Ruby is (obviously) ordered.

Then you look at the actual mongo_mapper gem that was the leaf of the whole tree.. and you find out that running the tests without bundler fixing the dependencies is actually impossible (due to the three versions of i18n that we have to allow side-installation of). And the Gemfile, while never declaring dependencies on the official Mongo driver (it gets it through plucky), looks for bson_ext (the compiled C extension, that in Gentoo was not going to exist, since it’s actually installed by the same bson package — I’ll have to create a fake gemspec for it just so it can be satisfied).

And this actually brings us to a different problem as well: even though plucky has been updated (to version 0.4.3) in November, it still requires series 1.3 of the Mongo driver. Version 1.4.0 was released in September, and we’re at version 1.5.2.

And I didn’t name SystemTimer gem, which is declared a requirement during development (but not by the gem of course, since you’re not supposed to run tests there) only for Ruby 1.8 (actually only for mri18, what about Ruby EE?) which lacks an indication of a homepage in the RubyGems website….

I love Ruby. I hate its development.

In bug 394393, a number of x11 packages are going stable, including x11-base/xorg-server-1.11.2-r2.
After you upgrade, don't forget to rebuild your x11-drivers packages. Use the command that is shown in the elog messages to find out which. In case you missed it:

# qlist -I -C x11-drivers/

# emerge -1 $(qlist -I -C x11-drivers/)

As I said on twitter I think I have a part of me that’s pretty much a masochist, as I’ve had the nice idea of trying out MongoDB for a personal experiment (a webapp to manage clients’ hardware configuration registration, stuff for my usual “day job”), so I started looking into packaging Ruby (and Rails) support for it.

Luckily Rails 3 started abstracting the ORM as well, allowing an almost drop-in replacement of ActiveRecord with a MongoDB adapter instead.. there is a nice guide that actually tells you almost all that is needed in the basic case (of course it could be more automated, but that’s beside the point here). The core of that guide is the mongo_mapper gem which then depends on a thin layer that is further built on top of the driver depending in turn on bson that provides a pure Ruby interface as well as a JRuby version… there is a separate gem for the C-based extension … if your head is spinning or are wondering if I became a linkspammer, I can’t blame you.

The end result is that at the very least I got to package

• bson
• mongo
• plucky
• mongo_mapper

Not too shabby, but I got in way worse situations before, such as the dependency web of Bones and its extensions. What I didn’t expect was the way these packages are developed rather than packaged.

The gems, as usual, are shallow, and contain no tests, Rakefile, or any other useful files that we need to package them properly as ebuilds. So I had to rely on the GitHub repositories, thankfully we do that mostly transparently in Ruby eclasses (as it’s way too common for us to have to rely on snapshots); unfortunately while for plucky and mongo_mapper the situation was clear (the homepage of the gems points to GitHub, and the two have separate repositories), the situation gets complex for the mongo driver and related gems.

First, let’s ignore the issue with bson and bson_ext (the C-based extension)… that is stuff we would generally can deal with in Gentoo itself (the C extension will be built for all the targets supporting it; JRuby will get its own extension, and all in the same package). The problem is that, after digging through the MongoDB website – finding the list of repositories is definitely not easy – the repository for both mongo and bson (and bson_ext!) is the same … and not in the way that Rails 2.3 was all in the same repositories (with each gem having its own subdirectory), but merging the content of the three gems in the same structure. Saying that it’s messy is not enough.

It is no doubt to me that I’ll handle that just fine at some point in the future, but it really makes me wonder how is it possible for them to consider this a good design and development practice. It’s a mystery. For now I don’t think I’ll spend much more time on this issue as I have other tasks for my job to take care of, which moves this to the back-burner…

And on that topic I’d like to try again an experiment to gauge the interest on packaging these gems; my blog is Flattr enabled – even though lately AdSense on this and Autotools Mythbuster is getting me more money than Flattr – and this post is as well. If you’ve got an account (or feel like opening one), and you’re interested in seeing ebuilds for the Mongo Ruby driver in Gentoo, give a flattr to this post. If I see that count increasing in the next two days I’ll use the Christmas weekend to work on it.

On december 14th, the Gentoo Hardened project had its monthly online meeting to discuss the current state of affairs of its projects and subprojects. Amongst them, the updates on the SELinux-front were presented as well.

Since last meeting, the follow topics passed the revue.

• sec-policy/selinux-base-policy, which is the “master” of our SELinux policies and contains those SELinux modules that are somewhat indivisible (hence the name, “base”), is now at revision 8. I tend to describe the changes on the gentoo-hardened mailinglist, and this is not different for rev 8. I haven’t stabilized the rev 6 one yet although I promised too, I’ll try to find some time to do that this evening.
• We had a regression with newrole for some time. Luckily, Jory “Anarchy” Pratt found the issue. Drop the setuid bit from the binary, and the application works again as it should. This will be included in the next policycoreutils bump.
• The last available sudo package now builds with native SELinux support as well, which allows users to add ROLE= and TYPE= information in the sudoers file. As such, users do not need to call newrole when they need to transition to a specific role for just a single command – sudo can now take care of that.
• The older selinux/v2refpolicy/* profiles have been deprecated. If you want to use a SELinux-enabled profile, you need to use a profile that ends with /selinux, such as default/linux/amd64/10.0/selinux or hardened/linux/amd64/selinux. Of course we prefer you to use a hardened profile ;-)
• Documentation-wise,
  • the Gentoo Hardened SELinux Handbook has been updated to reflect the profile changes
  • the SELinux bugreporting guide has been put online to inform users what kind of information is needed for us to fix issues or denials that they might see
  • the SELinux FAQ has been updated with the questions Applications do not transition on a nosuid partition and Why do I always need to re-authenticate when operating init scripts?.

That’s about it. Not a too busy month but progress anyhow.

Attention Gentoo users: If you're running kmail-4.4.11.1 and kde-4.7.3, and if you have been hit by bug 382411 (problems with local filters, cannot upload unread message on imap; see also here and here), please keyword kde-base/kdepimlibs-4.7.3-r1 and give it a try. (You may want to log out and log back in.) This upgrade seems to fix the problem... If I get some positive feedback, I'll fast-track it for stabilization. Cheers!

Next tides of users slowly notice that a number of unneeded files is installed on their systems. They enumerate systemd unit files, logrotate files, take their pitchforks and start their cruciates against Gentoo developers wasting their precious disk space.

Let me tell you a story. The story starts when Uncle Scarabeus wants to add bash-completion support into libreoffice ebuild. He considers this a minor addon, not worth the half a day necessary to rebuild libreoffice, so he doesn’t revbump it. He simply assumes the change will be propagated nicely when users upgrade to the next version.

Of course, some users will already come shouting here: that’s against the policy! Yes, indeed it is. But is it worth the hassle? Should all libreoffice users be forced to rebuild that huge package just to get a single tiny file installed? He could wait and add that along with the next version. Well, if he wouldn’t forget about it.

But that’s not really important part here. Because, to his surprise, many users have actually noticed the change. That’s because the use of bash-completion.eclass has caused the ebuild to have IUSE=bash-completion; and many of the --newuse Portage option users have rebuilt the package. A few others, like me, just stopped using that option.

That’s when the discussion started. We — the few devs actually caring about discussing — decided that it is quite pointless to control installing tiny files through USEflags. Of course, the libreoffice is an elephant-case here but so-called regular packages aren’t much better here. Is there really a reason to rebuild even 10 C files when the only thing going to change is a single, tiny text file being installed or not?

Another solution is to split those files into separate ebuilds. But that’s usually inconvenient both for users and devs. Users have to notice that they need to emerge an additional package to get the particular file installed, and devs need to maintain that additional package. That starts to become really ridiculous with files like systemd units which are often generated during build-time and store installation paths.

So what to use? INSTALL_MASK, obviously. It’s an ancient Portage magic which allows you to control which files will be punted from installed files. You can use app-portage/install-mask to quickly set it for the files you don’t want. It’s as simple as:

# install-mask -a systemd logrotate

Well, after almost a year of university-/work induced slacking, I'm finally going to work some more on Gentoo PHP.

Speaking of which, I've just closed Bug #324665 (Merge dev-php5 and dev-php) as RESOLVED FIXED. After a year of procrastination, I've finally whipped up a script to migrate the complete dev-php5 category into dev-php. Thanks goes to olemarkus and darkside for checking my script and suggesting ways to go about this.

If there are any issues with the migration, please comment here or on the bug.

P.S.: Yes, I know the overlay is broken because of this. Neither Ole nor me have any use for it at the moment. Plus, merging user-contributed stuff (thanks jamie!) from the overlay to the gentoo-x86 cvs tree is just plain painful. So unless we can keep Gentoo PHP development completely in the overlay and make the push as simple as git pull overlay, you probably won't see much action in the overlay.

Agreeing with some advice on gentoo-dev, I’m going to post this as a blog entry instead of a Gentoo news item. The quick version of this update is expect to see 0.24.1 in portage in a few days. The long version follows…

As many have noticed, the Gentoo version of MythTV has been significantly lagging the upstream version. We plan to introduce 0.24.1, which is the current upstream stable version, in the next few days. Before going through the upgrade we wanted to make you aware of our future plans for MythTV in Gentoo and some of your alternatives.

The Gentoo MythTV package will generally try to stay closer to upstream than it has in the last year, but probably will only be updated a few times per year at most. It will benefit from the full Gentoo QA process, including introduction to ~arch followed by stabilization. Dependencies will also be controlled in line with Gentoo QA so things should suddenly not stop working without warning. Versions that are stabilized will stay in the tree longer after they are superseded for those who want to take their time with upgrades.

The versions of MythTV in Gentoo may also not support the full range of upstream plugins.

There is an alternative for those of you who want a more bleeding-edge experience. Upstream maintains a Gentoo overlay at: git://github.com/MythTV/packaging.git

The upstream repository will be the first to receive updates and will have many more intermediate versions including bugfixes. The overlay also supports all the upstream plugins. However, it is not subject to Gentoo QA policy – dependencies could stop working or disappear without warning, and older versions may not be kept around. Generally speaking, we have not heard complaints from those who use it. Any issues with the upstream overlay should be reported to the MythTV project and not to Gentoo bugzilla.

If you’re already using the upstream overlay you shouldn’t be impacted by anything done in Portage – your versions will generally stay higher than our own.

If we drop your favorite plugin and you want it back and are willing to help maintain it, send us a note at mythtv@gentoo.org. If you’re willing to test it and ensure it works for everybody else, we can keep it in the portage tree.

This is my first step in blogging universe. I did that never before and hope my english isn’t to bad. The reason, why i decided to blog from now on, is that i reached the Gentoo developer state. Which gives me the chance to get a central theme to write. My personal goal is at least one post per month, maybe a little bit more if there is something i want to share. For sure the content of this blog won’t be Gentoo only related.

Lets talk about the way i became a Gentoo developer. I get  in touch with Linux in the end  of 2001. The reason why i tried out linux was that i want to discover something new in the computer world. New in the meaning of gain experience. So the first distribution was SuSe linux. Followed up by Debian, Red Hat, Mandrake and some other that i forgot in the meantime. None of these really satisfyed me. In 2003 i heard about a Linux distribution with a very good package manager, a good written installation handbook and all packages build from source. The name of the distrubution was Gentoo, as you may have already guessed. I was really enthusiastic about it, started the installation and felt in love. In the first years of using it, i had tried out other distributions  on seperate partions from time to time. But whenever i have done this, the partion is faster deleted as you can count to three.

While my time as Gentoo user i tried out several window manager/desktop environments. First was Gnome which i had have no longer than 1 week. It was to slow (yeah my former pc was not that good) and i didn’t like the handling at all. So i decided to take a light weight window manager. I tested White-, Black-, Open- and Fluxbox, choosen the last mentioned finally. After several time and new hardware i wanted something new and found XFCE. I liked it very much and sticked with it a long time. A friend convinced me to have a look at KDE 3.5, but after short testing period i had the feeling that XFCE is much better and switched back. With one exception to the other window manager/desktop environments experiences: didn’t unmerge it, because i used a lot of KDE apps from this time on and wanted to give KDE a chance with newer versions. As KDE 4.0 was released, i know that this would be my next desktop environmet and finally switched over with KDE 4.1.

I used IRC all the time, but with KDE my old IRC client XChat felt not so integrated into the desktop, so i searched for a new one and found Quassel. After some time i started to contribute to this opensource project (bug triaging, wiki & code contributions) and got an core account, which gaves me the posibility to idle in several gentoo related channels and of course one them was #gentoo-kde. In July this year i made some contributions to the kde overlay. The Gentoo KDE project lead tampakrap gave me write access to the overlay and asked me very quickly, if I want to become a Gentoo developer and that he would mentor me. I was surprised and honored, so my fast answer was “why not?”. After some weeks i started the ebuild quiz and finished it in September and started with the end quiz in mid October. While the end quiz we decided to switch over to the Gentoo recruiters web application (the usability is not so good at the moment) and  he created the “New Developer” bug. In the start of November all questions were approved by him, so the waiting period for a recruiter started. On 2011-12-01 the Gentoo recruiter hwoarang picked me up. The recruiting process was finished on 2011-12-07, after the third & final review session.  The official mailing list announcement was done  one day later on 2011-12-08.

My special thank goes to my mentor Theo “tampakrap” Chatzimichos. He has done his job as mentor wonderful. I have no comparison to other mentors, but i can say that he is the best . If he make the offer to mentor you, i can truly say, just do it™. And of course i want to thank  all the members out of the Gentoo KDE project for the support and my recruiter hwoarang for pick me up & recruiting me finally.

Yes I’m writing again about Ruby after a very long hiatus… it feels like there’s a pattern behind this it seems.

After my recent changes, Hans approached me today with two issues: the first is that I should stop adding virtual/ruby-test-unit dependencies on ebuilds since (finally!) Ruby 1.9 gained a completely compatible test-unit implementation, which means you can rely on packages to behave correctly without adding extra dependencies, and old, stale, and darcs-mantained gems to keep around — hopefully, at least. The other issue is more troublesome, and relates to bug #380409 where users are being forced to get Ruby Enterprise installed in their system.

Why would this happen? Well, as you can see in that bug, once the version of ruby-rdoc that references Ruby Enterprise (this applies to the other virtuals as well, though) was marked stable, Portage wanted to update it, causing the new implementation to be requested by the dependency tree. You can easily guess that we didn’t have in mind this behaviour when we decided to go with the virtuals. So what’s going on?

When a package, such as redmine (which I fixed yesterday to do this) requires the virtual/ruby-ssl ebuild (which relates to a Ruby implementation that includes the ssl extension, an USE flag for the C-based implementations, but a separate package for JRuby), the expanded dependencies appear like this: ruby_targets_ruby18? ( virtual/ruby-ssl[ruby_targets_ruby18] ) … since each one of those virtuals lives in a different slot, and only supports one target, you get requested one ebuild for each target you requested … and the target definition on those is forced on.

Since the dependencies resolve to a straight slot, Portage wouldn’t try to “update” the virtual to get the highest-version one .. which would then bring in a different Ruby implementation altogether. So why is bug #380409 there? As Hans points out the problem has to be that the virtuals are added to the world set: being in the world set (unbound from a slot) means that Portage will try to get the highest-version of the package installed, no matter which slot it belongs to. Incidentally, this is also why you got Python 3 installed on every system even though you didn’t ask for it, unless you mask it explicitly.

Handling of world set’s content is fishy at its best; when you install a package with a straight emerge command, you’re “selecting” the package, and that adds it to the world set; if you want to avoid that, you have to do so with emerge -1 .. and most people forget about that. This is probably why the bug entered on users’ systems.

So how do you clean up the situation so that it behaves as expected instead? Well, it’s actually easier than you’d expect:

qlist -IC virtual/ruby | xargs emerge --deselect

This removes the packages from the world file, but keeps them installed, so that you won’t be asked to merge them again later. And it handles all the Ruby virtuals at once, so even if in your case the problem was with rubygems, this will fix it.

PostgreSQL 8.2 has reached the end of its life. Start making your transition now.

From the release announcement (postgresql.org) on December 5, 2011:

This is also the last update for PostgreSQL 8.2, which is now End-Of-Life (EOL). Users of version 8.2 should plan to upgrade their PostgreSQL installations to 8.3 or later within the next couple of months. For more information, see our Release Support Policy. [wiki.postgresql.org]

For whoever has never heard that expression, unit tests are a way to test the functionality of a program or library in an automated way, by for example checking the evaluation result of test data. This is a great thing and can help massively in identifying coding mistakes. CMake provides a mechanism, and in many KDE packages test routines are included.
So far, so good, this is the theory, now let's look at reality. In Gentoo, the main KDE distribution is split into 295 (!) packages. Many dont have any tests (oxygen-icons comes to my mind as a nice example). Of those that have, 37 now have all the test routines hard disabled, and I guess a few are missing that I still have to add to the list. Why? Well, first of all, you have to know that compilation usually runs in Gentoo as an unprivileged user detached from any graphical desktop session, and this user also runs the test routines. In addition, tests should also run fine "offline", i.e. without internet connection. One of our developers is running a great thing called a "tinderbox", where build and unit tests in various configurations are automated- in an isolated environment, and possibly even without network access. So, what can go wrong?

• The tests need X11. Well, actually we can get around this, with a nice hack called virtualx. A background framebuffer X server is started for the test phase only, and the tests are redirected to that display. Works even sometimes. :o)
• The tests need a dbus session bus. I think we could get around this too, but nobody bothered to do it so far and I'm not a dbus expert. 
• The tests need an entire KDE desktop to interact with. Yes, that happens, and unfortunately, we have to disable them in this case, see above. 
• The tests try to download a data set from, say, IMDB, CDDB, or similar. Bad, bad, bad, see above.
• Next, the tests pop up dialog boxes asking for user interaction. A window asking for a GnuPG passphrase is my personal favourite... 
• And last, what I particularly like is a unit test that fails... and after digging into the test log, I see a fat comment "This does not work yet and needs to be fixed!"

The PostgreSQL ebuilds and initscript are now capable of using either /run or /var/run for the default socket directory.

The ebuilds will determine which to use at build time. The logic simply being, “Does /run exist?” If it does, everything is adjusted accordingly. If not, everything is adjusted accordingly. The old initscript is incapable of using /run or any other socket directory. So, if you upgrade to the latest version of the ebuilds and fail to upgrade the initscript, you’re going to have issues if you have a /run directory.

Also, this may cause a hiccup when an application is expecting the default location to be in /var/run/postgresql rather than /run/postgresql. The proper solution is not to rebuild everything — you can if you want, but that’s time consuming — but to specify the directory to look in for the socket. Anywhere you’d specify the host you can actually specify the socket directory. Or, you should be able to if the application did it right.

Something that is more important for those who have /run or /var/run on tmpfs, the initscript now takes care of creating the directory if it’s missing. With proper permissions of course.

The next neat thing is that the initscript respects two settings it previously ignored. Well, it ignored one setting outright, and allowed you to set another in /etc/conf.d/postgresql-${SLOT} only. If you set port or socket_directory in the postgresql.conf, the initscript will now pick it up and use those settings over what’s set in /etc/conf.d/postgresql-${SLOT}.

The growing pains aren’t over just yet. Out of three machines, one machine didn’t like the new initscript. It couldn’t shutdown an existing, running server. The other two did it just fine. Same setups. Nothing fancy. Just inexplicable. It’s easy enough to shutdown the server manually, and cleanly. Issue the following substituting the path for wherever your server’s data directory actually resides.

# kill -SIGTERM $(head -n1 /var/lib/postgresql/9.1/data/postmaster.pid)

If that doesn’t do the trick, you can substitute -SIGTERM with -SIGINT as that will close client connections and rollback the transactions those clients may have been in, as that is the most common cause as to why -SIGTERM wouldn’t work.

Merry Christmas!

on monday, december 12, i’ll be performing at the downtown independent for a musical showcase of monome-wielding artists, along with hardware and software presentations. live visuals all night long by OICHO.

i’ll be doing just a short set (as ioflow); the rest of the lineup are the real heavyweights. the inventors of the monome will be there, along with a stellar group of super-talented world-touring artists. these folks rock. if there’s any chance you can make it, you should totally be there! come hear some tunes and learn about the open-source software and hardware we use. i’ll be running an all-linux software stack, powered by gentoo linux.

i guarantee there will be some seriously fun beats, so get your tickets!

Thanks to Diego E. Pettenò (flameeyes) we’ve got some updated initscripts that are a good deal cleaner.

I did experience one inexplicable hiccup where it wouldn’t stop the server. If you run into that, just do:

kill -SIGINT $(head -n1 ${DATA_DIR}/postmaster.pid)

Where ${DATA_DIR} is the data directory for PostgreSQL server. (Usually, /var/lib/postgresql/$SLOT/data/.)

There have been a couple of fixes regarding Perl, most notably bug 378865 (bugs.gentoo.org).

Find all the changes at the official post. (postgresql.org)

Have fun!

Another KDE version stabilized in Gentoo... x86 is done with KDE 4.7.3 since this morning, amd64 is following right now. For those of you who don't read Portage news items, here's again my message about KDEPIM:

The stable upgrade from KDEPIM 4.4.11.1 to KDEPIM 4.7.3 is a MAJOR upgrade with potential for major breakage. Therefore we will try to keep and support the old, so-far stable KDEPIM 4.4.11.1 as long as possible.
If you don't want to upgrade your KDEPIM yet but keep the old version, please download this mask file and add it into your /etc/portage/package.mask.
If you decide to upgrade, please have a look at the upgrade guide first.