You might remember that for a while I worked on getting quagga in shape in Gentoo. The reason why I was doing that is that I needed quagga for the ADSL PCI modem I was using at home to work. Since right now I’m on the other side of the world, and my router decided to die, I’m probably going to stop maintaining Quagga altogether.

There are other reasons to be, which is probably why for a while we had a Quagga ebuild with invalid copyright headers (it was a contribution of somebody working somewhere, but over time it has been rewritten to the point it didn’t really made sense not to use our standard copyright header). From one side it’s the bad state of the documentation, which makes it very difficult to understand how to set up even the most obvious of the situations, but the main issue is the problem with the way the Quagga project is branching around.

So let’s take a step back and see one thing about Quagga: when I picked it up, there were two or three external patches configured by USE flags; these are usually very old and they are not included in the main Quagga sources. It’s not minimal patches either but they introduce major new functionality, and they are very intrusive (which is why they are not simply always included). This is probably due to the fact that Quagga is designed to be the routing daemon for Linux, with a number of possible protocol frontends connecting to the same backend (zebra). Over time instead of self-contained, easily out-of-date patches to implement new protocols, we started having whole new repositories (or at least branches) with said functionalities, thanks to the move to GIT, which makes it too easy to fork even if that’s not always a bad thing.

So now you get all these repositories with extra implementations, not all of which are compatible with one another, and most of which are not supported by upstream. Is that enough trouble? Not really. As I said before, Paul Jakma who’s the main developer of the project is of the idea that he doesn’t need a “stable” release, so he only makes release when he cares, and maintained that it’s the vendors’ task to maintain backports. On that spirit, some people started the Release Engineering for Quagga, but ….

When you think about a “Release Engineering” branch, you think of something akin to Greg’s stable kernel releases, so you get the latest version, and then you patch over it to make sure that it works fine, backporting the new features and fixes that hit master. Instead what happens here is that Quagga-RE forked off version 0.99.17 (we’re now to 0.99.21 on main, although Gentoo is still on .20 since I really can’t be bothered), and they are applying patches over that.

Okay so that’s still something, you get the backports from master on a known good revision is a good idea, isn’t it? Yes it would be a good idea if it wasn’t that … it’s actually new features applied over the old version! If you check you see that they have implemented a number of features in the RE branch which are not in master… to the result that you have a master that is neither a super-set nor a sub-set of the RE branch!

Add to this that some of the contributors of new code seems to have not clear what a license is and they cause discussion on the mailing list on the interpretation of the code’s license, and you can probably see why I don’t care about keeping this running, given I’m not using it in production anywhere.

Beforehand I was still caring about this knowing that Alin was using it, and he was co-maintaining it … but now that Alin has been retired, I’d be the sole maintainer of a piece of software that rarely works correctly, and is schizophrenic in its development, so I really don’t have extra time to spend on this.

So to finish this post with a very clear message: if you use Gentoo and rely on Quagga working for production usage, please step up now or it might just break without notice, as nobody’s caring for it! And if a Quagga maintainer reads this, please, please start making sense on your releases, I beg you.

I'm only beginning my experiments with SELinux, and neverallow (which is basically an assertion that prevents inserting certain allow rules) seemed a bit weird to me.

While experimenting with some local policies though, after an update (selinux-base-policy and other sec-policy packages) my local policy triggered a neverallow rule about sys_module capability being unnecessarily granted.

In fact, re-compiling the local policy and loading the new version made the error disappear. Now this is indeed useful, because binary policy files are arguably harder to inspect, and if they get out of sync with the base policy, it's easy to introduce errors like in this case.

Another conclusion is that learning takes time: it was the update that triggered this situation. I wonder what else awaits me in the SELinux land. ;-)

Recently I started experimenting with SELinux and created a new VM for that. I've used the excellent Gentoo SELinux Handbook and mostly got things running. I also submitted bug #411005, bug #411365, and bug #411377 to make it even better.

SELinux may seem scary or complicated at the first glance, but my experience so far was pretty good. I did encounter some AVC denials, but that was fine. Here are my general notes about running a SELinux system:

1. UNIX permissions work normally. If UNIX permissions deny access, SELinux rules are not checked. Access is just denied, as expected. SELinux rules can result in more denials, but can never allow more than UNIX permissions.
   
2. Permissive mode is really useful. In that mode the denials are only logged, not enforced. The system should work as usual, and you have chance to note and fix eventual problems. You can actually switch between permissive and enforcing on the fly (setenforce 1, setenforce 0), so you can experiment whether things would work in enforcing mode and quickly switch back otherwise.
   
3. Targeted policy is recommended for desktop systems. It's part of general advice to "start small", but desktop systems have many more available applications anyway. Usually servers are minimal installations, so it's easier to write complete strict policy for them. Targeted SELinux policy uses unconfined_t context by default, which has no restrictions at all. Only selected applications have a specific policy just for them.
   
4. You don't need to worry about SELinux users with targeted policy. Another part that makes SELinux more complicated is that normal Linux users are mapped to SELinux users. This is important when using strict policy, but with the targeted one everyone is just mapped to unconfined_u.
   
5. Most access decisions are made based just on the type. Usual SELinux context, e.g. unconfined_u:object_r:user_home_t may seem quite complicated. However, corresponding allow rule would most likely only need to mention the type, i.e. user_home_t (to be precise, two types: types of the object and type of the subject).
   
6. Many confusions are just matter of vocabulary. For example, there is no practical difference between domain and type (source). My understanding is that domains are types used to confine processes, so every domain is a type, but not every type is a domain.
   
7. Allow rules are actually pretty simple. For example "allow example_t self:process execmem;". Ignore the syntax, the meaning of each rule is really straightforward, and I think it's no different with alternative MAC (Mandatory Access Control) implementations. And when writing a policy for specific domain (in this case example_t), all of the rules are going to start with "allow example_t ...".

As summary says in Gentoo you can finally decide which implementation of libav you want, good old ffmpeg named flavor or the new and shiny libav one.

14 months after I introduced it to main tree we finally got it stable almost everywhere (just ppc missing yet as usual :P). We did *censored* of work in order to support both implementations, introducing virtual package, proper (non-conditional) patches everywhere, etc. etc.

So why should you switch your package?

Basically it does not matter for you. We can say it can be done as personal prefference.
We can say that ffmpeg merges from libav tree and throws some experimental patches over it. Thats why my personal prefference is with libav folks as I don’t trust any idea of merging anything. Otoh the ffmpeg is the “orginal” (well most devs moved under libav) upstream thus they should get our trust for what they are doing.

Today from the distro PoV there are some distributions using libav (Debian, Sabayon) and also ffmpeg (openSUSE [external repo called packman]).

As we in Gentoo really are pro-choice just pick yourself and let us know how happy are you with such decision :-P

So how can I switch?

Just edit /etc/portage/package.use* to keep same useflag from media-video/ffmpeg to media-video/libav and run:

emerge -C ffmpeg && emerge -1v libav

After this move you need to recompile all the packages depending on ffmpeg. This can be done with little help of revdep_rebuild or by using portage-2.2 and its sets by running:

emerge @preserved-rebuild

packages that hard require media-video/ffmpeg

As reported in the first comment the mplayer1 does now strictly require ffmpeg. So I did a little check to see which packages hard-require the other implementation and here is the list:

games-arcade/performous-0.6.1 (media-video/ffmpeg)
media-libs/ffmpegsource-2.16.2.1_pre587 (>=media-video/ffmpeg-0.9)
media-libs/mediastreamer-2.3.0-r1 (video ? media-video/ffmpeg)
media-plugins/audacious-plugins-3.0.3 (ffmpeg ? >=media-video/ffmpeg-0.7.3)
media-plugins/audacious-plugins-3.1 (ffmpeg ? >=media-video/ffmpeg-0.7.3)
media-plugins/audacious-plugins-3.1.1 (ffmpeg ? >=media-video/ffmpeg-0.7.3)
media-plugins/audacious-plugins-3.2 (ffmpeg ? >=media-video/ffmpeg-0.7.3)
media-plugins/audacious-plugins-3.2.1 (ffmpeg ? >=media-video/ffmpeg-0.7.3)
media-plugins/audacious-plugins-3.2.2-r1 (ffmpeg ? >=media-video/ffmpeg-0.7.3)
media-plugins/mediastreamer-x264-1.1.7 (media-video/ffmpeg)
media-plugins/mediastreamer-x264-1.3.3 (media-video/ffmpeg)
media-video/dv2sub-0.3 (kino ? media-video/ffmpeg)
media-video/mplayer-1.0_rc4_p20120213 (>=media-video/ffmpeg-0.10)
media-video/mplayer-1.0_rc4_p20120405 (>=media-video/ffmpeg-0.10.2)
media-video/mplayer-9999 (>media-video/ffmpeg-0.10.2)
media-video/transcode-1.1.5-r2 (postproc ? media-video/ffmpeg)
net-libs/opal-2.2.11 (>=media-video/ffmpeg-0.4.7)
net-libs/opal-3.6.8 (ffmpeg ? >=media-video/ffmpeg-0.5[encode])
                    (x264 ? >=media-video/ffmpeg-0.4.7)
net-libs/opal-3.6.8-r1 (ffmpeg ? >=media-video/ffmpeg-0.5[encode])
                       (x264 ? >=media-video/ffmpeg-0.4.7)
net-libs/openh323-1.18.0 (>=media-video/ffmpeg-0.4.7)

I bet most of these could work with libav, so if you are user of such package and want to try libav just edit the dependency to be on virtual/ffmpeg instead of the media-video/ffmpeg and see if it works.

Parallel build, and parallel install, is not hard, in the sense that it usually doesn’t give you new, undocumented challenges; it actually seems to be repeating the same problems over and over again. Sometimes the exact same problem — as it seems Ruby upstream applied the stupid Funtoo patch instead of mine, and that made it fail again on a parallel install. Luckily I was able to fix it again for good, and now is sent to the ruby-core mailing list.

Another issue came up today, when I noticed a bug for OpenSC which turned out to be a parallel install failure. While Michelangelo’s quick fix is actually a smart way to deal with it quickly, I’ve preferred applying the correct fix, which I also sent to the opensc-devel mailing list.

So this is just a quick post to remember you all: if you see failures such as “file already exists”, remember you’re in front of a parallel install failure, and you can see my previous blog to understand how to fix it properly.

While I’m still trying to figure out how to get the logs analysed for the tinderbox, I’ve been spending some time to work on Gentoo’s Ruby packaging again, which is something that happens from time to time as you know.

In this case the spark is the fact that I want to make sure that my systems work with Ruby 1.9. Mostly, this is because the blog engine I’m using (Typo) is no longer supported on Ruby 1.8, and while I did spend some time to get it to work, I’m not interested in keeping it that way forever.

I started by converting my box database so that it would run on Ruby 1.9. This was also particularly important because Mongoid 3 is also not going to support Ruby 1.8. This was helped by the fact that finally bson-1.6 and mongo-1.6 are working correctly with Ruby 1.9 (the previous minor, 1.5, was failing tests). Next step of course will be to get them working on JRuby.

Unfortunately, while now my application is working fine with Ruby 1.9, Typo is still a no-go… reason? It still relies on Rails 3.0, which is not supported on 1.9 in Gentoo, mostly due to its dependencies. For instance it still wants i18n-0.5, which doesn’t work on 1.9, and it tries to get ruby-debug in (which is handled in different gems altogether for Ruby 1.9, don’t ask). The end result is that I’ve still not migrated my blog to the server running 1.9, and I’m not sure when and if that will happen, at this point.. but things seem to fall into place, at least a bit.

Hopefully, before end of the summer, Ruby 1.9 will be the default Ruby interpreter for Gentoo, and next year we’ll probably move off Ruby 1.8 altogether. At some later point, I’d also like to try using JRuby for Rails, since that seems to have its own advantages — my only main problem is that I have to use JDBC to reach PostgreSQL, as the pg gem does not work (and that’s upsetting as that is what my symbol collision analysis script is using).

So, these are my Ruby 1.9 pains for now, I hope to have better news in a while.

If you need to convert .mts files to .mov (so that e.g. iMovie can import them), I found ffmpeg to be the best tool for the task (I don't want to install and run "free format converters" that are usually Windows-only and come from untrusted sources). This post is inspired by iMovie and MTS blog post, but I think my final bash one-liner is better and simpler:

for x in *.MTS; do ffmpeg -i ${x} -vcodec mpeg1video -acodec mp2 -ac 2 -sameq ${x/.MTS/.mov}; done

Note the useful bash substitution of extension, ${x/.MTS/.mov}. Enjoy!

That’s right, it’s finally out! Thanks go out to all our contributors for the great work (there’s too many — see the shortlog!). The highlights of the release follow. Head over to the announcement or release notes for more details.

• Dynamic sample rate switching by Pierre-Louis Bossart: This makes PulseAudio even more power efficient.

• Jack detection by David Henningsson: Separate volumes for your laptop speakers and headphones, and more stuff coming soon.

• Major echo canceller improvements by me: Based on the WebRTC.org audio processing library, we now do better echo cancellation, remove the need to fiddle with the mic volume knob and have fixed AEC between laptop speakers and a USB webcam mic.

• A virtual surround module by Niels Ole Salscheider: Try it out for some virtual surround sound shininess!

• Support for Xen guests by Giorgos Boutsiouki: Should make audio virtualisation in guests more efficient.

Special thanks from me to Collabora for giving me some time for upstream work.

Packages are available on Gentoo, Arch, and probably soon on other distributions if they’re not already there.

What? FOSSCOMM 2012

When? 12th, May 2012 - 13th, May 2012

Where? Technological Educational Institute of Serres, Greece

FOSSCOMM 2012 is almost here, and Gentoo will be there!

Check out our Gentoo presentation and workshop on DistCC/CrossDev. We will have a booth with Gentoo promo stuff, flyers, stickers, badges, live CD's, DVD's and much more! Whether you're a developer, user, or simply curious, be sure and stop by. See you there!

Dimitris Papapoulios and Pavlos Ratis contributed the draft for this announcement.

As from today we have new app-officeext category that will be holding various office extensions. You all for sure know that when I say office I mean libreoffice/openoffice extensions. This means that we can now easily add new and shiny ones if you desire so. Please open bugs for your favorite extensions and we will ship them ASAP (also prefferably open stable requests for the current ones so I don’t have to remember to do it :P).

You might be able to find some fancy extensions there already that might help you improve your workflow with libreoffice.

Here is the list I will do this afternoon as they are part of the libreoffice core we never bothered to provide yet:
google-docs ; barcode ; diagram ; hunart ; numbertext ; oooblogger ; typo ; validator ; watch-window ; ct2n (requres two patches from lo tree -> repack).

Apache Office 3.4 released

From other news there is now openoffice 3.4 release made by folks from Apache Foundation. Gentoo will not package this release as it is basically equal to what we would call libreoffice-3.4.1 with some missing features and lacking all the build fixes that were put in lo-3.4.

But by the Gentoo here I just mean that I am epically lazy to solve again all the build issues 3.3 had. So if anyone is willing to maintain and package it we will gladly add it to the main tree (with the named person as maitnainer). You don’t even have to be a developer, there is a nice proxy-maintainer project so poke me mail if you are interested.

What we could also do is to add hardmasked binary version.That stuff contains security issues and is basically the same like old libreoffice/openoffice binaries which tried to contain everything from python to icu. Also for this someone interested is required. So mail me/comment here.

Dictionaries

There will be progress when I find some more time. New simpler eclass for myspell (actually hunspell) dictionaries is in place and I just need to check and verify all the releases. For this I got nice script from Caolan McNamara (LO guy working for Redhat) that contains upstreams and versioning schemes for most of the languages.

This will also be used with utilizing the euscan tool writen by Corentin Chary to watch for newer versions. Where I will also try to bump those with compatible licensing directly in dictionaries repo of libreoffice so you will be able to enjoy the spellchecking even on fancy platforms that none sane would use (riiiight? :-P).

cups dependency

Lots of you lads whined^Wcomplained about the fact cups is actual hard dependency now in libreoffice. With help of one brave lad (Dave Flogeras) we managed to bake configure switch for cups again. It is currently under review on libreoffice mailinglist (not really it already passed and I was just lazy to commit it in which I will do this afternoon). After it is included our live ebuild will again have switch to disable printing completely.

Internal lo extensions

Libreoffice itself has some internal extensions that are built as part of the build process. In history those were never built, or always built, depending on the complexity of their deps.

This now changes. There is new LIBREOFICE_EXTENSIONS use expand variable that can be used same way like VIDEO_CARDS in make.conf.

All of the extensions are already in except reportbuilder, because I was unable to find some java packages (I mailed java team for help here so those hopefully will be identified soon).

If you are interested in contributing to libreoffice consult various comments I put in the live ebuild as I am too sexy^Wlazy to open easy hacks on freedesktop bugzilla. Any help wrt unbundling stuff is welcome. Currently there is even problem with building javascript engine interface because bundled rhino uses too patched debug interfaces (one can either remove it or make work on the currently provided one) so if you like javascript and java you can have fun.

Public announcement: we've just changed the default settings in cmake-utils.eclass, and as a result now cmake-based builds will display a verbose build log instead of the terse "one line per file" version. In the same way as with autotools, the entire compiler / linker / whatever command line is printed, which should simplify bug hunting a lot...

Here's a funny game - let's try to use a LSI controller from the command line in Linux.

First difficulty: Since a short while ago all downloads are behind a registration wall. Ok, no biggie, let's register then (grumble, grumble, moan.)

The registration demands a password containing at least seven characters, including at least one non-alphanumeric. Yey!

Then there's the Captcha, which looks quite amusing, but ... erf. After three or four guesses you might even get it right.

Fast forward to the driver download page. The specific model (according to lspci) doesn't even exist. Oh, haha, u so funny! But no worry, all the MegaRAID models share the same driver.

Now, here it gets a bit esoteric:

$ file 8.02.21_MegaCLI.zip 
8.02.21_MegaCLI.zip: Zip archive data, at least v2.0 to extract
$ unzip 8.02.21_MegaCLI.zip 
Archive:  8.02.21_MegaCLI.zip
  inflating: 8.02.21_MegaCLI.txt     
  inflating: 8.02.21_Windows_MegaCLI/MegaCli.exe  
  inflating: 8.02.21_Windows_MegaCLI/MegaCli64.exe  
  inflating: 8.02.21_Windows_MegaCLI/Readme.txt  
  inflating: 8.02.21_VMware_MegaCLI/MegaCli  
 extracting: 8.02.21_VMware_MegaCLI/MegaCli.zip  
  inflating: 8.02.21_Solaris_MegaCLI/MegaCli  
  inflating: 8.02.21_Solaris_MegaCLI/MegaCli.pkg  
  inflating: 8.02.21_Solaris_MegaCLI/readme.txt  
 extracting: 8.02.21_Linux_MegaCLI/MegaCliLin.zip  
  inflating: 8.02.21_Linux_MegaCLI/readme.txt  
  inflating: 8.02.21_FreeBSD_MegaCLI/MegaCli  
  inflating: 8.02.21_FreeBSD_MegaCLI/MegaCli64  
  inflating: 8.02.21_DOS_MegaCLI/LICENSE_DOS32A.txt  
  inflating: 8.02.21_DOS_MegaCLI/MegaCli.exe

$ unzip 8.02.21_Linux_MegaCLI/MegaCliLin.zip
Archive:  8.02.21_Linux_MegaCLI/MegaCliLin.zip
  inflating: MegaCli-8.02.21-1.noarch.rpm  
  inflating: Lib_Utils-1.00-09.noarch.rpm 

$ rpm2tar MegaCli-8.02.21-1.noarch.rpm
$ tar --list -f MegaCli-8.02.21-1.noarch.tar 
./
./opt/
./opt/MegaRAID/
./opt/MegaRAID/MegaCli/
./opt/MegaRAID/MegaCli/MegaCli                                                                                                                                                                                                               
./opt/MegaRAID/MegaCli/MegaCli64
$ file ./opt/MegaRAID/MegaCli/MegaCli64 
./opt/MegaRAID/MegaCli/MegaCli64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.4.0, stripped

$ ./opt/MegaRAID/MegaCli/MegaCli64 
Fatal error - Command Tool invoked with wrong parameters
Exit Code: 0x01

Hi all,

After some months of people asking me to update the documentation of the pandaboard, i’ve finally done it. The previous documentation was a bit outdated since the Pandaboard ES came into scene.

The problem was on the bootloader part, which didn’t work because they stopped developing X-Loader and integrated it to U-Boot. With an updated U-Boot this problem is solved.

Also on the drivers part, ubuntu released ubuntu 12.04, precise pangolin, with updated packages for the graphics part and a new kernel version. I follow whats stable on ubuntu, so if there’s some problem one could reproduce it with ubuntu and then complain to ubuntu developers After all, TI developers don’t get paid to support Gentoo but to support Ubuntu…

Anyway, ubuntu 12.04 switched from softfp toolchain to hardfloat toolchain(which Debian and Ubuntu call armhf), this means that the graphics drivers which are closed-source must have their binary blobs+libraries built as hardfloat. This means that if you want to use the new versions, you must reinstall your system with a hardfloat stage3…

More on the graphics part…until now since some libraries provided by the package collided with the ones from mesa, i had to remove them. Thanks to lu_zero, eselect-opengl now supports dynamic switching of those libraries, however i still have to fix the ebuild to use this new function.

The new guide is here: http://dev.gentoo.org/~armin76/arm/pandaboard/install.xml … its the same link as the older one

I’d like to thank to the different people that reported me what they had to do to make it work with the Pandaboard ES, since i don’t have one. Thanks!

Just a very quick paragraph on a just-reported issue: if you upgrade your SELinux utilities to the latest version and you switch from /selinux to /sys/fs/selinux as the mountpoint for the SELinux file system, you might get into issues. Apparently, init (which is responsible for mounting the SELinux file system through a call to libselinux) is trying to mount it on – well yes – /sys/fs/selinux but at that time, /sys is not mounted yet.

I haven’t been able to reproduce just yet, because I just recently had to move all my systems to use an initramfs (thank you you-need-an-initramfs-when-you-have-a-separate-usr-partition) which premounts /sys. But the current workaround should be to keep /selinux for now. The utilities support it still, and that gives me some time to look and investigate the issue.

It’s been a while since I last wrote about parallel building. This has only to do with the fact that the tinderbox hasn’t been running for a long time (I’m almost set up with the new one!), and not with the many people who complained to me that spending time in getting parallel build systems to work is a waste of time.

This argument has been helped by the presence of a --jobs option to Portage, with them insisting that the future will have Portage building packages in parallel, so that the whole process will take less time, rather than shortening the single build time. I said before that I didn’t feel like it was going to help much, and now I definitely have some first hand experience to tell you that it doesn’t help at all.

The new tinderbox is a 32-way system; it has two 16-core CPUs, and enough RAM for each of them; you can easily build with 64 process at once, but I’m actually trying to push it further by using the unbound -j option (this is not proper, I know, but still). While this works nicely, we still have too many packages that force serial-building due to broken build systems; and a few that break in these conditions that would very rarely break on systems with just four or eight cores, such as lynx .

I then tried, during the first two rebuilds of world (one to set my choices in USE flags and packages, the other to build it hardened), running with five jobs in parallel… between the issue of the huge system set (yes that’s 4.24 years old article), and the fact that it’s much more likely to have many packages depending on one, rather than one depending on many, this still does not saturate the CPUs, if you’re still building serially.

Honestly seeing such a monstrous system take as much as my laptop, which is 1/4 in cores and 1/4 in RAM, to build the basic system was a bit… appalling.

The huge trouble seem to be for packages that don’t use make, but that could, under certain circumstances, be able to perform parallel building. The main problem with that is that we still don’t have a variable that tells us exactly how many build jobs we have to start, instead relying on the MAKEOPTS variable. Some ebuilds actually try to parse it to extract the number of jobs, but that would fail with configurations such as mine. I guess I should propose that addition for the next EAPI version… then we might actually be able to make use of it in the Ruby eclasses to run tests in parallel, which would make testing so much faster.

Speaking about parallel testing, the next automake major release (1.13 — 1.12 was released but it’s not in tree yet, as far as I can tell) will execute tests in parallel by default; this was optional starting 1.11 and now it’s going to be the default (you can still opt-out of course). That’s going to be very nice, but we’ll also have to change our src_test defaults, which still uses emake -j1 which forces serialisation.

Speaking about which, even if your package does not support parallel testing, you should use parallel make, at least with automake, to call make check; the reason is that the check target should also build the tests’ utilities and units, and the build can be sped up a lot by building them in parallel, especially for test frameworks that rely on a number of small units instead of one big executable.

Thankfully, for the day there are two more packages fixed to build in parallel: Lynx (which goes down from 110 to 46 seconds to build!) and Avahi (which I fixed so that it will install in parallel fine).

For those who wonder why Excelsior is now built for a week and it’s still not running in full capacity, the reason is relatively simple: I’m still trying to figure out how to handle network security so that I can give other developers access to it, and not risk that we have a security breach in other places.

The first problem is that the current setup I need to use and the final one will be very different: right now the system is connected to the internal network of my employer, while in production it will be DMZ’d, with a public IP available straight to it. The latter setup will let me set up an IPv6 tunnel, which means all the internal containers will be available by themselves, while the current one prevents me to set it up at all.

Previously this was much easier to deal with simply because I had control over an external firewall myself, which took care of most of the filtering, which combined with LXC networking options made it decently easy to deal with. Unfortunately this time I have to do without this.

One of the things that might not be obvious from the post above, nor from the documentation, is that the three setups I have drawn are the ones that require the least amount of configuration on the host-side: if you use the standard bridge, you just need to start net.br0 (or equivalent), and LXC will connect the host-side virtual Ethernet pair to the bridge by itself. If you’re using the MACVLAN modes instead, the containers will piggy-back the interface of the host, and then rely on the external router/firewall/switch to deal with the configuration — by the way I’m glad that man lxc.conf now actually says that you’re supposed to have a reflective switch to use those modes. You can probably guess it’s not that great an idea if you expect lots of internal communication.

What I’m going to do for now is setting up a system that is defended as much as possible by depth, with iptables carrying out enough filtering that I should be realistically safe. Unfortunately just iptables is not enough and what you need is iptables and ebtables (for Ethernet Bridging filtering), to make sure that the containers’ don’t dupe your IPs or something.

The idea is that one IPv6 is enabled, you can jump straight into the tinderboxes, which is required to control it, but until then, the host system acts as a jump host through a custom scponly setup, which only allows forwarding, as ProxyCommand, port 22 of others boxes within that same system.

I’d like to show more documentation of what I’m trying to do, and what I achieved already, but to do so, I’m afraid I’ll be needing some more … visual examples, as it’s very hard to explain it in words, while it should be much more clearer with a series of drawings. I guess I’ll start working on them soonish, maybe if Luca can package Synfig for me…

For now, this is enough, I guess.

Today I’ve stabilized the sec-policy/selinux-* packages that provide the 20120215 “series” of SELinux policies. Together with the stabilization, the more recent userspace tools (like the policycoreutils as well as libraries like libsemanage and libselinux) have been pushed out as well. I will be dropping the older policies and userspace tools soon (as they are now deprecated). The documentation has been updated to reflect this too.

Some of the enhancements include
• support for permissive domains (allowing users to mark one specific SELinux domain, such as mplayer_t, as permissive (even though the rest of the system is running in enforcing mode)
• support for file context translations, so we can now say “/usr/lib64 (and below) should have the same contexts as /usr/lib”
• support for role attributes, which means for policy developers, we now have similar freedom as with type attributes
• support for named file transitions, so a policy rule can say that domain A, if creating a file in a directory labeled B, then that specific file should have label C. Same for directories, btw.

Although some of these enhancements were available as features individually, the policies we had were not aligned with it – and now, that has changed ;-)

Okay so Excelsior is here, and it’s installed, and here starts the new list of troubles, which seems to start, as usual, with my favourite system ever: LXC which is the basis the Tinderbox work upon.

The first problem is not strictly tied to LXC, but one of the dependencies required: the lynx browser fails to build in parallel if there are enough cores available (which there certainly are here!). This bug should be relatively easy to fix but I haven’t had time to look into it just yet.

A second issue came up due to the way I was proceeding to do the install, outside of office hours, and is that the ebuild is using the kernel sources, I think to identify which capabilities are available on the system. This should be fixed as well, so that it checks the capabilities on the installed linux-headers instead of the sources, which might not be the same.

The third issue is funny: Excelsior is going to use an hardened kernel. The reason is relatively simple to understand: it’s going to run a lot of code of unknown origins, it’ll allow other people in, one wants to be as as possible… unfortunately it seems like this is not, by default, a good configuration to use with LXC.

In particular, grsecurity is designed by default to limit what you can do within a chroot, by applying a long list of restrictions. This is fine, if not for the fact that LXC also chroots to start its own set up process. I’m now fixing the ebuild to warn about those options that you have to (or might) want to disable in your GrSec setup to use LXC.

Interestingly, it’s not a good idea to disable all of them, since a few are actually very good if you want to use LXC, such as the mknod restriction, which is very good in particular if you want to make sure that only a subset of the devices are accessible (even when counting in the allowed/non-allowed access of the devices cgroup).

In particular, these have to be disabled:

• Deny mounts (CONFIG_GRKERNSEC_CHROOT_MOUNT)
• Deny pivot_root in chroot (CONFIG_GRKERNSEC_CHROOT_PIVOT)
• Capability restrictions (CONFIG_GRKERNSEC_CHROOT_CAPS)

while the double-chroot would be counter-synergistic as it would disallow services within the container to further chroot to allow a defense-in-depth approach.

Then there is another issue. Before starting to set up the actual tinderbox, I wanted to prepare another container, which is the one I’ll be using for my own purposes, including bumping of Ruby packages and stuff like that. Since the system is supposed to stay very locked down, this time I want to mount the block device straight into the container, which is a supported configuration…. but it turns out that the configuration parser, trying to workaround old issues (yes that’s a one and a half years’ old post) will ignore any mount request that doesn’t have the destination rootfs prefixed.

Unfortunately when you mount a block device, it means that you’ll end up with something along the lines of /dev/sdb1/usr/portage. This also collides with the documentation in man lxc.conf:

If the rootfs is an image file or a device block and the fstab is used to mount a point somewhere in this rootfs, the path of the rootfs mount point should be prefixed with the /usr/lib/lxc/rootfs default path or the value of lxc.rootfs.mount if specified.

Anyway this should be fixed in 0.8.0_rc2-r2 which is now in tree, I’ve not been able to test it thoroughly yet, so holler at me if something doesn’t work.

Glad to see that libbash is accepted by Google Summer of Code this year. Congratulations to André Aparício! Petteri and I will both mentor this project during the summer. Before GSoC begins, I’d like to explain the current status of this project.

I worked on this project in GSoC 2011. After the summer ended, I spent months seeking a job so I didn’t pay much attention to the project at that time. And later, I made several commits to the project and it’s looking better.

So far, we can generate correct metadata for 13849 ebuilds (5435 more than before). Nearly half of the ebuilds in the main tree can be handled by libbash now. So the goal of this year’s GSoC is to make libbash be able to handle the rest of the ebuilds.

What’s stopping us from handling more ebuilds? There are several eclasses that our grammar file can not parse. That means any ebuild that inherits from these eclasses can not be handled by libbash. On the other hand, the runtime part is not fully implemented, which is due to lack of grammar support or low priority. I have to say it’s hard to tell all the problems we are facing at once. I have been working in this pattern for quite some time:
1. Use our implementation of instruo to generate metadata for the ebuilds in the main tree.
2. Sort the error output to see which errors are the biggest enemies.
3. Solve the problems that are figured out from the error output
Generating ebuild metadata will take about 10 minutes so this process is not fast. I need to solve as many problems as I can at one pass. Typically new errors will occur after I fix the old ones. And the new errors are not introduced by the new commits. They were just hidden by the old ones. That’s why it’s hard to move forward and handle more ebuilds in a short time.

In future, the error output should be improved so it would be easy for us to figure out more problems. After constantly working on the error output for some time, I’m sure we will get fewer errors and handle more ebuilds. Looking forward to seeing a working libbash for all the ebuilds at the end of this summer.

Let’s start with the good news: most of Excelsior has arrived and it’s already set up. The only thing that is missing is … the CPUs, which are coming in from Philadelphia, and should arrive here tomorrow, standing to Amazon’s tracking. As I said, the server will be co-located by my current employer, so that’s one issue not to worry about.

Without Yamato, it turns out that my ability to bump the version of my own packages is vastly reduced, mostly because I don’t want to install packages such as MongoDB on this laptop just to test out Ruby Gems, and at the same time I don’t want to have too many extraneous packages installed. Luckily this means that starting tomorrow we should be all ready to start the install phase.

One of the things I’ve been keeping busy with was the split hardware IDs package — sys-apps/hwids, which I’m bumping weekly. This from one side makes it much less important to use the (now gone) network cron scripts to update the IDs files, and on the other allows people who don’t want their systems to access the network directly to be kept up-to-date with the files themselves. This is the first week I’m skipping over the bump, simply because … there is no new content!

I’ve added a new device today to the USB IDs database though so that should mean that next week we might have an update. And tomorrow I’ll probably update it with the possibly missing subsystem IDs for the devices on Excelsior, which will go to the PCI IDs database where I already sent my laptop’s and one of the local server’s subsystems.

Speaking about device identification I can understand why Kay thinks that it might be better to have a general database of everything, instead of multiple small databases… for instance it would be nice if I could just update one database with the IDs of my new external HDD (WD My Passport), and let smartctl know that it has to connect to it with the SAT method, instead of having to write it on a page and then remember about it myself. Speaking about which, WD still is my favourite HDD vendor.

Anyway, thanks once more to all the people who helped the new Excelsior to be built; tomorrow I’ll post a few more details about it, including some photos hopefully, as I’ve got my camera with me as well. There has actually been some trouble with the SSDs and the mounting bays, which I think would be a valuable lesson not only for me.

I recently bumped shadow-4.1.5-r1 and pambase-20120417. bug #412721 has been filed about people being locked out after the update.

The fix is to run dispatch-conf. The reports are unclear, but you might not get any message from emerge that it is necessary. If dispatch-conf doesn't update anything, make sure to manually re-emerge pambase.

What's going on? /etc/pam.d/{login,passwd,su} are being moved from shadow package to pambase, so they can be shared with hardened-shadow. They are not really shadow-specific, but Gentoo-specific, so it makes sense to make pambase own those files.

On request of Matthew Marchese, I now automatically build an ePub version of Linux Sea for those that like to read such resources on a digital reader. Thanks to the use of DocBook, this was simply a matter of using its xsl-stylesheets/epub/docbook.xsl stylesheet against the DocBook sources and zip the created directory structures (OEBPS and META-INF) to get to the ePub file.

I am really happy to announce that I’ve been invited to the Tizen Developer Conference, in San Francisco, CA next month (May 7-9). I’m going with a friend of mine, Michele Tameni, who seems to have won the Intel consolation prize (damn you!).
I really need to thank Giovanni Martinelli and Mauro Fantechi for sponsoring my trip there and across the whole San Francisco Bay Area (I’ve been told that it’s full of Gentooers, gonna catch you!).

This is a great opportunity for me to meet a good deal of hardcore FLOSS devs and have a beer together (just one?). There are also several exciting talks I couldn’t really miss.

Feel free to contact me if you want to meet up for a drink. A post to the gentoo-core ML will follow next week.

If you speak some fancy language other than english, then this post is exactly for you. While I was working on updating some of the lingua packages in the cvs based on euscan reports I checked their websites to ensure the euscan reports to be correct (in some cases it might miss a bump, or report some beta files). And I hit huge problem, all of those damn pages are writen in native language (eg hungarian spell is in hungarian, dutch in dutch…) which effectively means I can’t read s**t. It might be briliant to get more contributors, but most packagers for the software are developers which just try to make the users happy and do not speak your language at all (google translate is not much help in some cases).

To the point. If you can could you look up dictionaries for your language and check for new versions (ispell/aspell/myspell/…)? If there are some just drop me mail, open new bug and CC me on it, or whatever suits your plan.

Another problem with our app-dicts team is that it is not existing. Currently it is just pva (doing his best) and nobody else did any relevant work on dictionaries since 2k8. We have old support code for openoffice-1/2 and so on. Luckily it does not break anything much, but still it is quite cruft. So if you ever considered being Gentoo contributor this might be easy area how to get involved.

And for the end we need to rework all the myspell-* language packages as they still use old openoffice infrastructure. This was shot down when oracle given all the stuff to apache foundation so all the links are dead. Job here is to search up dict/thes/whatever and set up proper homepage and download locations, maybe asking on irc channels of both projects ( libreoffice and openoffice) to see where they moved it up.

So anyone up for any of the tasks (no, I don’t want to do it because I am lazy, but I have quite lot of stuff within the libreo stuff itself)?

Recently Gentoo bug #412031 (">=www-client/chromium-18.0 should not force the media-libs/libjpeg-turbo dependency") has been filed. Samuli did the right thing and closed it, but I'd like to use this as an opportunity to explain the issue in more detail.

The original bug is #393471, filed by Gentoo Chromium team member Mike Gilbert four months ago and fixed in February. An optimization done in WebKit JPEG decoding code (which speeds up JPEG decoding by up to 2x) assumed that the code will run against the same version of JPEG library it has been compiled with. In general such assumptions do not hold (with the note of SONAMEs, i.e. same library SONAME should mean it's binary-compatible; more on that later), but Google Chrome bundles all of those third-party libraries anyway, so for Google it's true.

I started a discussion on chromium-dev mailing list, and also had direct e-mail exchange with developers doing the WebKit and libjpeg-turbo changes. The end result is we'd either have to keep a local patch in Gentoo to revert the optimization, or just always depend on libjpeg-turbo.

Of course the most cool would be to do the optimization The Right Way, by doing runtime detection of JPEG extensions, as made possible by DRC, libjpeg-turbo developer and explained in bug #393471 comment 23. jcstest.c in libjpeg-turbo source tree indeed shows how to do that. The only reason I haven't done that yet in WebKit is lack of time (it's not just producing the patch; compiling WebKit and running its layout tests takes a lot of it; and then dealing with any possible breakages), but I plan to do that in the future (help is welcome).

Speaking about The Right Way, another broken thing is that libraries that have the same SONAME (in this case libjpeg and libjpeg-turbo) behave differently when using JPEG extensions. You may recognize the libpng "breakages", but it seems to me libpng upstream actively cleans up the interface of the library, and it obviously updates SONAME accordingly. That's a good thing.

Hopefully it's now more clear why the Gentoo package www-client/chromium depends on libjpeg-turbo and not virtual/jpeg.

In my previous post, a very valid question was raised by Alexander E. Patrakov: why still use chroot if you have SELinux?

Both chroot (especially with the additional restrictions that grSecurity enables on chroots that make it more difficult to break out of a chroot) and SELinux try to isolate an application so it only has access to those resources it needs. Chroot does this on file-level basis (and a bit more with grSecurity), SELinux on more general resources. However, things that make SELinux strong (flexible and detailed policy language, fine-grained authorizations) are also its weakness (consolidating files into groups having the same file label), and chroot does have an advantage on this.

Suppose that a flaw exists in BIND through which an attacker can read files on the host (through BIND). With SELinux, the domain in which BIND runs is prohibited from accessing and reading files whose label is not one of the labels that the policy thinks BIND should be able to read. More specifically, the BIND policy in the reference policy (which is what both Gentoo and RedHat base their policies on, and generally policies are only enlarged, never really shrinked):

• etc_runtime_t (read) means access to the files in /etc that are modified at runtime (like mtab, profile.env, gentoo’s /etc/env.d)
• named_var_run_t (read) is access to /var/run/bind and /var/run/named (and a few other related locations)
• named_checkconf_exec_t (read/execute) is access to read and execute /usr/sbin/named-checkconf
• named_conf_t (read) to read the BIND-related configuration files
• dnssec_t (read) to read the DNSSEC keyfiles
• locale_t (read) to access /etc/localtime, /usr/share/locale/*, /usr/share/zoneinfo/*
• etc_t (read) to read the general configuration files in /etc (including passwd, fstab, …)
• proc_t (read), proc_net_t (read) and sysfs_t (read) to access those pseudo filesystems
• udev_tbl_t (read) to access /dev/.udev and /var/run/udev (but I have no idea yet why this is in)
• named_log_t (read/write) for the log files of BIND
• net_conf_t (read) to access /etc/hosts (including deny/allow), resolv.conf, …
• named_exec_t (read/execute) the BIND executables
• named_zone_t (read) to access the zone files, also write access in case of slave system
• cert_t (read) to read certificate information
• named_cache_t (read/write) to access its cache
• named_tmp_t (read/write) to work with temporary files

Isolation provided by SELinux is as powerful as the width of its labeling. For instance, by giving the named daemon read access to /etc files like passwd, fstab, group, hosts, resolv.conf and more, a malicious user who can exploit this hypothetical vulnerability can obtain information that might help him in his further attempts. By chrooting BIND, the files placed in the chroot itself should not offer the information he might be looking for (for instance, the passwd file, if needed at all, is limited to just the named and root accounts, etc.)

Chrooting, but not enabling SELinux, could lead to escalation. A chroot cannot restrict what a process is allowed to do beyond the regular access privileges that are given on the user. If a user can upload an exploit through BIND and have BIND execute it, he can use this as an attack vector for further activities. SELinux here prohibits BIND to write stuff it can also execute (there is no write and execute privilege defined here). It also ensures that the BIND daemon never exists his security domain (transitioning towards another domain with perhaps other privileges) as there are no transition rules from named_t to any other domain.

Another MAC system that would be better suited to fit both is grSecurity’s RBAC model. Iirc, it uses path definitions to say which files are allowed to access and which not. The weakness SELinux here has (aggregation into sets of files with the same label) doesn’t exist for grSecurity. This debate on path-based versus label-based access controls have been going on for very long time now – just google it ;-)

So, Alexander, in short: chroot further limits the SELinux-allowed privileges to a more fine-grained set of file system resources (files/directories).

BIND, or Berkeley Internet Name Domain, is one of the Internet’s most popular domain name service software (DNS). It has seen its set of security flaws in the past, which is not that strange as it is such a frequently used service on the Internet. In this post, I’ll give a quick intro on how to use it in Gentoo Hardened (with PaX)… chrooted… for IPv6… with SELinux ;-)

Installing is of course, as usual, dead easy on Gentoo (Hardened/SELinux). Make sure you have USE=”ipv6″ set, and then emerge bind. Also install bind-tools as they contain some great tools to help with DNS troubleshooting. Then we’re editing /etc/conf.d/named to set the CHROOT variable. I also set CHROOT_NOMOUNT so that Gentoo doesn’t bind-mount the information in the chroot but instead uses the files in the chroot.

CHROOT="/var/named/chroot"
CHROOT_NOMOUNT="1"

Now we need to either temporarily add some privileges in SELinux, or run the portage_t domain in permissive mode. If you go for privileges, then add the following:

allow portage_t var_t:chr_file { create getattr setattr };

If you however want to temporarily run the portage_t domain in permissive mode, do that as follows:

~# semanage permissive -a portage_t

We are doing this because we are now going to ask the BIND ebuild to prepare the chroot for us. Doing so however requires portage to work on our live file system (and not in the regular “sandbox” mode). SELinux however forces portage in the portage_t domain and only gives it the privileges it needs for building and installing software.

~# emerge --config bind

When done, remove the previous SELinux allow rules again (or set the portage_t domain back in enforcing mode, through semanage permissive -d portage_t). Next we need to relabel the files in the chroot. By default, all files are labeled by SELinux as var_t in that location because it isn’t aware that it needs to see /var/named/chroot as a “root” location.

~# setfiles -r /var/named/chroot /etc/selinux/strict/contexts/files/file_contexts /var/named/chroot

So far so good. Now let’s create a simple named.conf file (in /var/named/chroot/etc/bind):

options {
  directory "/var/bind";
  pid-file "/var/run/named/named.pid";
  statistics-file "/var/run/named/named.stats";
  listen-on { 127.0.0.1; };
  listen-on-v6 { 2001:db8:81:21::ac:98ad:5fe1; };
  allow-query { any; };
  zone-statistics yes;
  allow-transfer { 2001:db8:81:22::ae:6b01:e3d8; };
  notify yes;
  recursion no;
  version "[nope]";
};

# Access to DNS for local addresses (i.e. genfic-owned)
view "local" {
  match-clients { 2001:db8:81::/48; };
  recursion yes;
  zone "genfic.com" { type master; file "pri/com.genfic"; };
  zone "1.8.0.0.8.b.d.0.1.0.0.2.ip6.arpa" { type master; file "pri/inv.com.genfic"; };
};

The zone files referenced in the configuration file are located in /var/named/chroot/var/bind (in a subdirectory called pri – which I use for “primary”). The regular one would look similar to this:

$TTL 1h ;
$ORIGIN genfic.com.
@       IN      SOA     ns.genfic.com. ns.genfic.com. (
                        2012041101
                        1d
                        2h
                        4w
                        1h )

        IN      NS      ns.genfic.com.
        IN      NS      ns2.genfic.com.
        IN      MX      10      mail.genfic.com.
        IN      MX      20      mail2.genfic.com.

genfic.com.     IN      AAAA    2001:db8:81:80::dd:13ed:c49e;
ns              IN      AAAA    2001:db8:81:21::ac:98ad:5fe1;
ns2             IN      AAAA    2001:db8:81:22::ae:6b01:e3d8;
www             IN      CNAME   genfic.com.;
mail            IN      AAAA    2001:db8:81:21::b0:0738:8ad5;
mail2           IN      AAAA    2001:db8:81:22::50:5e9f:e569;
; (...)

while the one for reverse lookups looks like so:

$TTL 1h ;
@       IN      SOA     1.8.0.0.8.b.d.0.1.0.0.2.ip6.arpa ns.genfic.com. (
                        2012041101
                        1d
                        2h
                        4w
                        1h )

        IN      NS      ns.genfic.com.
        IN      NS      ns2.genfic.com.

$ORIGIN 1.8.0.0.8.b.d.0.1.0.0.2.ip6.arpa.

1.e.f.5.d.a.8.9.c.a.0.0.0.0.0.0.1.2.0.0         IN      PTR     ns.genfic.com.
8.d.3.e.1.0.b.6.e.a.0.0.0.0.0.0.2.2.0.0         IN      PTR     ns2.genfic.com.
; (...)

We can now start the init script:

~# rc-service named start

On the slave, don’t set the allow-transfer directive and set its type to “slave”. In each zone, you will need to tell where the master is:

zone "genfic.com" {
  type slave;
  masters { 2001:db8:81:21::ac:98ad:5fe1; }
  file "sec/com.genfic";
};

By default, the SELinux policy for BIND does not allow BIND to write stuff in its directories. On the slave system, you will need to change this. A SELinux boolean here does the trick:

~# setsebool -P named_write_master_zones on;

There ya go ;-) Okay, all very condensely written, but it should give some feedback on how to proceed. I’m adding this information to the new online resource I’m writing – A Gentoo Linux Advanced Reference Architecture. Nothing really ready yet, just writing as I go forward with exploring these technologies…

Optional runtime dependencies (or «suggested dependencies») are one of the late problems we’re facing in Gentoo. There’s definitely a need for some standard solution, and it’d be great to put it in the next EAPI. Sadly, there’s no consensus how to solve it.

The optional dependencies problem

Gentoo has a very neat solution for handling optional dependencies and optional features, probably ever since the beginning. It is called «USE flags» and they work very well with the «traditional» optional dependencies. By that, I mean optional dependencies which are both build- and run-time.

Such a dependencies have to be pulled in before the build process starts, and usually require passing specific options in the configure phase. What’s important, both enabling and disabling features requires rebuilding the program in question because of code branches being switched. Thus, it’s perfectly fine if changing USE flags implies rebuilding the package.

Sadly, when it comes to optional runtime dependencies, USE flags are not a perfect solution. «Switching» such a dependency doesn’t require rebuilding the program anymore. It’s usually not even switching — the program can determine in runtime whether a particular dependency is available, and either enable or disable respective features. Simple like that.

If one decides to use USE flags for that, they become partially meaningless. Unless flags start stripping off the code (which is a bad idea), feature availability is dependency- rather than flag-based. So, USE=-ssl is irrelevant if, say, pyopenssl is installed. What’s even worse, flag imply needless rebuilding of such packages just to pull in an additional dependency.

The simple hack — pkg_postinst() messages

The simplest solution right now is just listing the suggested dependencies in pkg_postinst() messages. Combined with has_version helper, those messages can give a pretty nice output, pointing out already installed packages — just take a look at sys-apps/systemd ebuild.

Of course, it’s not a real solution, rather relying on user doing the hard work. The biggest disadvantage is that the dependencies are often going to end up in @world. And then, if user decides to unmerge our package, portage is unable to find and unmerge them as well.

The SDEPEND solution

A pretty common idea is to establish a new variable called SDEPEND (for «suggested»). Such a variable would simply list relevant dependencies, and let portage handle the UI part somehow. It is a minimalistic solution, quite consistent with other parts of PMS. Sadly, it has a few big shortcomings.

First, using our current dependency syntax, you can’t specify that a particular feature requires more than one package; in other words, that two or more suggested dependencies are supposed to be pulled in together. Of course, solving this one would be pretty easy — e.g. by allowing grouping them with parantheses.

A much more important issue is describing what particular dependencies do. Although sometimes this could be guessed by package descriptions pretty well, usually a more friendly text would be great. So, we end up having to implement that somehow.

And that’s usually when Ciaran comes in with ugly exherbism DEPENDENCIES. Sure, it solves most of the issues pointed out here but, hell, do we really want such a thing? Isn’t dependency syntax obscure enough already?

And it’s all rather dependency-oriented. In other words, package comes first, then goes the feature description. «Pass dev-python/pyopenssl or dev-python/python-gnutls to enable secure connections support». I don’t think that’s the most user friendly solution.

The USE flag solution

Another solution is brining a new category of USE flags. It’s not important whether they would be specified using a special variable, common USE_EXPAND or another magical features. In fact, that could be a thing totally separate from USE flags. The point is that some of the package flags would be runtime-switchable.

Unlike traditional USE flags, such flags wouldn’t be stored in vdb. They would be evaluated in place instead, using package.use or similar files, and the dependency tree would use current state of such flags. Of course, they would be allowed for RDEPEND (PDEPEND) use only.

Why reuse USE flags for that? Because it’s the most user-friendly solution. User doesn’t have to learn anything new. He/she enables a flag, does emerge -vDtN @world and notices that new dependencies are pulled in but the package doesn’t have to be rebuilt for that.

If we just add some additional magic for regular USE flags, enabling run-time dependant SSL support could be exactly the same as enabling build-time one — even using the same USE=ssl.

And we could basically even give «backwards» support for older EAPIs. Package managers not supporting the new feature would simply treat runtime-switchable USE flags as regular USE flags, requiring rebuilds of the package.

A quick help request from the community: if you know of any Gentoo documents that need updates in order for end users to know when and how to use initramfs, please file bugreports and have them block bug #407959. Currently, we have updated the Gentoo Handbook, Gentoo Quickinstall guides and added an Initial ramfs Guide.

The tracker bug is also used to check if and when the eventual roll-out of software can happen, and we want to make sure that we do not forget documentation (something we learned from the openrc migration). Not that the change is as large as was the case with openrc, but it is still nice to have updated documentation in time ;-)

Let's all congratulate the Calligra team for releasing Calligra 2.4.0, the first version of their integrated application suite for KDE. It has really gone a long way from the old KOffice codebase. Of course, Gentoo already has ebuilds as app-office/calligra. Give it a try! Cheers!

I don’t think it’s much more of a secret anymore that I’m planning on moving to the United States before end of the year if I can. This means not only that I have to bring down and move around the business happening in Italy, but also that I have to find a way to rebuild my own infrastructure here in the US.

This has become more important since yesterday my home router in Italy decided to have hiccups that it didn’t have in a long time, and I’m now unable to reach any of the computers back at home. Including the monitoring services, the containers I use to build the packages for the remote servers, and so on so forth. Given I was already planning retiring Yamato after almost four years of almost uninterrupted service, simply because the mainboards started acting up, and adding to that, the fact that shipping it between EU and US is going to be a pain, and more expensive than the system is worth, I decided it’s time to get a completely new box, here in the US.

At first I was looking for dedicated servers offerings, but the monthly prices are definitely too steep for my taste ­— yes of course I could get one in the EU but that’s not the same thing. Besides, I’d really much prefer having something that, in the worst case, still allow me direct access to the box as needed. After discussing it with KingTaco, I decided to check out what was available from NewEgg, and the result is that I found what would be the perfect box to host the next round of testings: a dual 16-core, 32GB RAM, 4TB HDD system, for just under four thousands dollars.

Of course that’s still quite a bit of money especially considering I’m trying to account for all factors of moving here, closing the business in Italy, and still giving enough to my mother to keep on going for a while…

And here’s where the pledgie is coming into view. While I’m going to use the new box for a few personal projects as well, it’s obvious that its main use will be again tinderboxing, and since this time I don’t intend for it to be hosted within my home, I want to be able to give access to other developers to the different tinderboxes as well. Add to that the enormous amount of time required to go through the logs and report bugs (most of which I’ll be unable to spend during my work hours), and you can probably see why I’m asking again for some help to go back at tinderboxing.

While I set up the pledgie to the actual, full price of the box, this does not mean I’m not going to pay my share: both PayPal and Pledgie take out their own fees, which is actually quite steep in by itself (PayPal’s more than Pledgie’s!), and there is the issue of finding a co-location, which I’ll also take care of paying for. For more details of what I had in mind, see the following, quoted text of the Pledgie page:

Those of you who have been following me for long enough know this already, but for those who haven’t, my Tinderbox is an effort to keep compiling and installing the packages available in Gentoo to make it possible to know whether they are working correctly or not.

While this is not considered an “official” effort, the fact that I’ve not been running this for almost an year starts to show with packages being bumped without their reverse dependencies being checked, and new automagic dependencies cropping up that weren’t noticed before.

Also considering that I’m now in the process of moving from Europe to the United States, and the fact that the motherboard of the system I’ve been using to run tinderboxing has been showing problems (which is the reason why I stopped working on it), I think it’s time to get a new system (from scratch at this point), and restart the effort.

This time the system will be clearly cut so that the tinderboxing is handled in a separate environment, so that other developers will also have access to it, but I’ll also keep running some of my personal projects in there, and some work-related software. From my side, I’m going to take care of assembly and maintenance of the system, and will pay for its connectivity (likely in a co-lo near where I’m going to stay in the US).

But since it’s still a non-trivial amount of money that is required for a system powerful enough not to be replaced in the next few years, I’m going to ask for the help of you all out there. The target is set to the exact price of the system I’m aiming for on newegg but unfortunately that also means that it includes sales taxes. Please note that while the following specs could look like overkill, this time I aim on running multiple, isolated tinderboxes, in particular I have three configurations I care about: amd64, amd64-hardened and x86.

• Dual Opteron 6272 16-core 2.1GHz CPU
• 32GB DDR3 ECC RAM
• 4×1TB Western Digital Caviar Black HDDs
• 32GB OCZ Onyx SSD (for operating system)

Please keep in mind that the hardware is only one fraction of the value behind the tinderbox, as the most important part is reporting the bugs, which is still not automated, so while I’m asking you to pledge money, I’m pledging a lot of my free time for this project again.

Please help, if you can. Thank you immensely.
Diego Elio Pettenò aka Flameeyes

For those who wonder how it’s possible to have a donation before the Pledgie started, I had some extra funds from previous donations which I’m pledging myself against this. Furthermore I’m going to pledge all the Flattr funds that I can get into my PayPal account in the future months. For those who don’t want to use PayPal, I’ll be tallying up donations coming in form of gift certificates for Amazon or iTunes (both US) manually, just make sure to tell if you want to have your name shown or not.

Imagine you are running... say... Kubuntu, OpenSuSE, or Fedora, or any other of these famous and fantastic binary Linux distributions. You find a bug, report it to the authors of the package, and they come up with a possible solution. "Hey, here's a small patch to the source, could you please try if applying that helps?" Well... Either now you compile manually, which may or may not give the same packages as your distribution. Or you start setting up the full build system of your distro, which will take a while...
With Gentoo as a source-based distribution, however, this is very easy. Let's assume you want to apply a patch for more debug output to some kde plasma applets. The patch file is called debug.patch, the package name for the plasma applets is kde-base/plasma-apps. All you have to do is

mkdir -p /etc/portage/patches/kde-base/plasma-apps
cp debug.patch /etc/portage/patches/kde-base/plasma-apps/
emerge -a1 kde-base/plasma-apps

 >>> Preparing source in /var/tmp/portage/kde-base/plasma-apps-4.8.2/work/plasma-apps-4.8.2 ...
 * Applying user patches from /etc/portage/patches//kde-base/plasma-apps ...
 *   debug.patch ...                      [ ok ]
 * Done with patching
>>> Source prepared.

When I posted about the hwids data I was not expecting to find what I just found today, and honestly, I’m wondering if I’m always looking for stuff under rocks.

My reason to split out the ID databases off their respective utilities (pciutils and usbutils) was simple enough: I didn’t need to deal with the two utilities, both of which are GPL-2, when the database themselves are available under the BSD 3-clauses license; it was just a matter of removing code, and avoiding auditing of projects that we don’t need to rely upon.

The fact that it was still a pet peeve of mine to not have an extra package taking care of it, rather than bundling them, was just an added bonus.

So after creating a silly placeholder which is fine for our needs here, with the blessing of Greg I created a new package, sys-apps/hwids (I wanted to call it hwdata, but we have both gentoo-hwdata and redhat-hwdata that install very different stuff), which has its own repository and with a live ebuild that simply fetches the files out of the respective website. I’m going to update the package weekly, if there are changes, so that we always have a more up-to-date version of it, and we won’t be needing the network cron scripts at all either.

I’ve also updated lshw to support the split package, so that it doesn’t install its own ids files anymore… of course that is only half the work done there, since the lshw package has two more datafiles: oui.txt and manuf.txt. The latter comes out of Wireshark, while the former is downloaded straight from IEEE’s Public OUI registry and that’s where the trouble starts.

The problem is that while you’re free to download the oui.txt file, you won’t find any kind of license on the file itself. I’ve sent a request to IEEE for some clarification on the matter and their answer is a clear “you cannot redistribute that file” (even though Ulrich, while not a lawyer, pointed out that it’s factual information which might not be copyrightable at all — see Threshold of originality for further details.

So why would I care about that single file given that lshw is a minor package by itself, and shouldn’t matter that much? Well, the answer is actually easy to give: bluez also contains a copy of it. And we’re redistributing that for sure, at least in source form. Sabayon is actually distributing binaries of it.

Interestingly enough, neither Debian’s lshw package nor their Bluez one do install the oui.txt file and I wouldn’t be surprised if their source archives have been neutered made Free by removing the distributed copy of the file.

What should we do about this? Unfortunately, that’s one question I don’t have an answer for myself yet, but I guess it should be clear enough that you can’t always assume that what upstream declares to be the case… actually is the case, especially for what concerns licensing. And this is the reason why, even though we don’t have any problem with releasing the source of all the GPL’d packages we have, we’d like to reduce as much as possible the amount of licenses I have to go through.

For quite some time I have been using Meld to detect relevant changes between two releases when I update a package in Gentoo. I run

# ebuild foobar-1.2.ebuild manifest prepare
# ebuild foobar-1.3.ebuild manifest prepare

and throw Meld at both outputs

# meld /var/tmp/portage/[..]/foobar-1.2/work/foobar-1.2/ \
       /var/tmp/portage/[..]/foobar-1.3/work/foobar-1.3/ &

Meld makes it easy to see what has changed and (especially) what has not changed. With sole diff -r that would be difficult.

I usually start by inspecting changes to configure.ac. If upstream did a good job that diff tells what dependencies to touch, already. Near the left and right margin you can see where else the file has been modified. No need to scroll-search down for more: you already know what you get.

The NEWS and ChangeLog files usually offer pointers of interest, too.

If my hint on Meld made a single Gentoo packager juggler’s life easier or more efficient, I have achieved what I was aiming for. Sorry for the noise to everyone else.

PS: The second preview image up there has been losslessly reduced by 40% in size just by running it through optipng -o7.

Even though it’s Sunday, I’m going back to write about my work on using Busybox as OpenRC shell, and not only.

One of the things we’ve decided for the firmware of the device is to reduce the amount of involved projects as much as possible. The reason falls into the realm of license auditing which is also the reason why I made sure that in Gentoo we now have a split hwids package which is available under a more “easygoing” license. The issue is not as much as not wanting to use GPL’d software, but the issue that the more GPL’d software you have in your firmware, the more you have to be careful to do everything by the letter, as it’s easy to get something wrong even with the best of intents.

Maybe even if I told myself that I disagree with Donnie’s post I’m actually agreeing with my actions.

To be very clear, once the device is available to the customers, we’re going to publish the whole sources of the open-source components, including those that we’re not required to because they are BSD, and even to people who wouldn’t receive the firmware itself, simply because there is no sense in keeping them private. Moreover, there is no change we made that is not trickling upstream already, either in Gentoo or to the original projects, which are mostly Aravis (which Luca is working on) and the Linux kernel (as I had to write some Super I/O drivers, which I intend to improve over the next few months).

Anyway, since we wanted to reduce the amount of projects, no matter what the license, that we depend on, we’ve also decided to replace sysvinit with BusyBox itself. This is not as much of a gain as one would expect, but it’s still significant. The main trick is that you can’t use the default /etc/inittab file Gentoo comes with, but you have to replace it with one compatible with BusyBox’s own init binary.

::sysinit:/sbin/rc sysinit
::wait:/sbin/rc boot
::wait:/sbin/rc

Okay possibly I could have dropped the boot runlevel altogether and just used sysinit and then default, but there is a trick to keep in mind: if you try to mimic the “classic” inittab file, you’ll be using rc default as the last entry: unfortunately if you do it that way, the softlevel= parameter to be passed at the kernel is not going to work at all. I’m not sure why it works on the standard setup, probably due to the implementation of the System V numbered runlevels, but for sure it doesn’t work with BusyBox, as the code in rc.c takes the default as the name of the runlevel to start — not giving a particular runlevel makes it decide which one to start depending on the parameter given, and defaults to, well, default, if none is set explicitly.

Beside this issue, and the small issue with sysctl init script not working correctly with Busybox’s own sysctl command, it seems like, for the very small set of init scripts we’re dealing with, BusyBox is a godsend.

But this is not all, of course! I’m also creating a few init scripts for Busybox applets themselves, for instance I have a bb.acpid script that is used to handle the acpid replacement within BusyBox, and the next up is going to be a (multiplexed) bb.udhcpd script for its own DHCP server. These will soon arrive in a Gentoo near you, and should be very nice if you’re not looking for a full-fledged DHCP server.

The main issues with this, as far as I can tell, is that there is little to no documentation on the format of the configuration file: for what concerns acpid I had to read through the source and guess what it needed, and that’s not so easy to do after a while as you have both common and non-common parser functions… at any rate, you’ll see more documentation about this appear in this blog and (if I can find more time to do this), probably on the Gentoo Wiki, as I think it would be a good idea to document it somewhere — you can make it more likely for me to write more documentation about the stuff I’m not doing for work by sending a thank you or you can otherwise wait for me to have more time.

If you are using stable profiles, you might want to verify if you are already running a kernel with devtmpfs support enabled. Why? Well, currently you might not need it, but the upcoming openrc/udev packages require it and they currently do not fail at install time if you have it enabled or not. As a result, upgrading these packages might give you a system that might fail to boot (if you have no initramfs but separate /usr partition) or gives many errors (if you have an initramfs).

To verify if it is enabled, check your kernel configuration:


# zgrep DEVTMPFS /proc/config.gz
# CONFIG_DEVTMPFS is not set


If you get the output as described above, best update your kernel configuration to include it. The second devtmpfs-related option (to automatically mount it on /dev) is not needed afaik.

And for those that have been with Gentoo for a while – devtmpfs is not devfs. Well, it is. But it isn’t. Somewhat. Oh well, there’s discussion on that which I’m not going to elaborate on. Safe to say that we’re getting older if we start feeling “Been there, done that, got the t-shirt” ;-)

Edit: as Robin mentioned in the comments, the udev ebuild does check at it. However, it doesn’t fail an installation so you could miss the message. Apologies for the lies, Robin ;-) Post updated.

While repoman does a good job of finding smells in ebuilds, a tool to evaluate an overlay with respect to the state of the Gentoo main tree has to my knowledge been missing so far.

overlint is a simple command line tool. From a technical view point it reports

• which version bumps from the overlay are missing from the main tree (e.g. the overlay has 7.1 but the main tree has 7.0, only),
• which revision bumps are missing from the main tree (e.g. the overlay has 3.0-r1 but the main tree has 3.0, only), and
• which exact same revisions exist in both trees with differing ebuilds.

On a higher level these findings often indicate that

• certain changes are still to be integrated with the Gentoo main tree to benefit a wider audience and/or that
• an ebuild appeared in an overlay first but can be removed now as the Gentoo main tree has grown an equivalent (or identical) copy.

As a consequence overlint has two main use cases, each with a different user audience:

• Overlay maintainers can use overlint to better keep their overlay in shape.
• Gentoo developers and proxy maintainers can use overlint to detect valuable patches missing from the main tree.

Here is example output of overlint 0.4.1 for the calculate overlay:

# overlint-cli /var/lib/layman/calculate/
===============================================================
Version bumps missing from Gentoo main tree
===============================================================
net-misc/
  italc :: 1.0.13, 2.0.0
net-print/
  foo2zjs :: 20081129, 20110512
net-wireless/
  madwifi-ng :: 0.9.4.4178.20120131
  madwifi-ng-tools :: 0.9.4.4178.20120131

===============================================================
Revision bumps missing from Gentoo main tree
===============================================================
app-arch/
  unzip :: 6.0-r9
app-text/
  wgetpaste :: 2.18-r1

===============================================================
Ebuils that differ at same revision
===============================================================
app-forensics/
  unhide :: 20110113
dev-util/
  bin_replace_string :: 0.2
  qt-creator :: 2.4.1
sys-auth/
  pam_keystore :: 0.1.3
sys-block/
  tw_cli :: 9.5.3
sys-boot/
  grub :: 1.99-r2
sys-libs/
  talloc :: 2.0.7
virtual/
  linux-sources :: 0

To get it run:

# sudo emerge -av app-portage/overlint

The source code is up on http://git.overlays.gentoo.org/gitweb/?p=proj/overlint.git;a=summary. For small patches just send them along, for bigger ones get in touch before the work, please. Thanks!

CUPS 1.5 has now been in the tree for quite some time, so we should think about getting it stable. As I'm actually not really a person who prints a lot or uses a lot of different printer drivers, it would however be great to get some more widespread testing... So please: If you are still running stable CUPS (i.e. net-print/cups-1.4.8-r1), please give the newest ~arch ebuild a try, and if you are encountering problems, please file bug reports on Gentoo bugzilla! Thanks a lot in advance!!!

Time has come to admit that 3.4 was not a great release as whole while the 3.5 series provies much much better experience wrt building and having runtime features working.

To accomodate the need for “Best office suite TM” I plan to stabilise 3.5.2.2 rather soon when I manage to polish few corner cases. If anyone wants to contribute with fixes now it would be good idea to look on the buglist.

Currently I am tracking 2 buildsystem bugs (test releated, hopefully cornercases) which I can’t reproduce (#393413, #401507). The best course of actions here is to identify the failing testcase and I will try to get me or other developers to fix the test to behave. (I really hope that soon the buildsystem stuff will be sorted out and we could execute ALL the tests in the test phase).

For the runtime the situation is way more shiny. There is just one questionable regression. Libreoffice quickstarter is yet again broken #408685. I am not exactly sure if we want to keep the feature in as it is quite often borked and I simply can’t use it at all in KDE. The only benefit it has nowdays is the lovely systemtray icon where you can launch apps. If someone would like that feature he should get in touch with us on lo mailinglist or upstream irc channel to rewrite it properly and provide just the launcher feature. The preload hacking is no longer really needed as the speedup with latest versions is really not visible.

ON Feature requests side you get almost all candies and there are not much opened bugs. There is one request asking for moving the sofficerc file to /etc/ which has valid points, but requires quite lot changes and I am too sexy^Wlazy for it.

Other feature that I plan to implement after the stabilisation is moving all the extensions to USE_EXPAND variable meaning you could select all the extensions with special useflags. If you are not exactly sure what the heck is USE_EXPAND see how xorg-drivers ebuild handles them and how you set ie. INPUT_DEVICES in make.conf. But I would like to hear first from you guys if you think you would like this change as I don’t mind providing it as normal useflags like now. The only thing it would do is make more visible what is option for the actual libreoffice and what controls just some optional extensions.

There are couple of points to note when upgrading to postfix-2.9:

* Postfix daemons now live under /usr/libexec/postfix.  Please adjust your main.cf accordingly by running etc-update/dispatch-conf or similar and accepting the new daemon_directory setting.  Otherwise, postfix will not be able to find the binaries it is looking for.
* There is a new berkdb USE flag.  It is on by default.  If you want to use cdb only or perhaps no db at all (who needs those bloated berkeley db libraries anyway huh? :) ), turn off the berkdb flag.  But please be careful as hash and btree lookups need berkdb tables.  In other words, if you use hash or btree tables and turn berkdb off, your setup will not work.
* Postfix init script is multi-instance aware now.  It is backward compatible so there is no rush but if you are running multiple instances of postfix, please read multi_instance_readme at postfix web site and start using it.  It is a rather flexible method of managing postfix instances.
* Finally, there is still no upstream support for vda USE flag for postfix-2.9 - vda flag is currently blocked for postfix-2.9.  vda is an external patch for handling email quotas in postfix.  I really don't like that patch as it does not play nicely at edge cases.  We will not be dropping support for vda as that would break existing setups.  However, there are better ways to handling quotas in your mail infrastructure:  Handle it via your IMAP server, which is the correct place to manage quotas by the way as the IMAP server knows best how much email you got (dovecot in particular comes to mind).  Or handle it via filesystem quotas.  Or have a cron job that periodically checks user quotas and updates a recipient access list.  Anyway, whatever choice you make, try and migrate away from vda USE flag if at all possible.

A news item before stabilization is also in the works.  Cheers.

Students, only a few days remain to apply for Gentoo's 7th year in the Google Summer of Code! GSoC pays college students $5000 to work full-time on an open-source project for a summer. Check out our GSoC 2012 homepage if you are interested in this year's GSoC for Gentoo. We particularly encourage applications from students who are new to Gentoo development—many of our students become Gentoo developers after a successful summer.

Interested students can browse Gentoo's project ideas. Student applications will be accepted until 1900 UTC April 6, so start now if you haven't already!

Developers, if you'd like to apply to be a mentor, you can do so on the webapp. Please read the mentoring guide before applying.

Students, this Friday at 1900 UTC is the deadline to apply for this year’s GSoC. It’s an awesome program that pays you to work on open-source projects for a summer (where you == a university/college student).

It’s by no means too late, but start your application today. You can find more information on Gentoo’s projects here (click on the Ideas page to get started; also see our application guidelines) and on the broader GSoC program here.

Good luck!

Figure 1.1: Heating your boxes since 1999

Gentoo Linux is proud to announce the availability of a new LiveDVD to celebrate the continued collaboration between Gentoo users and developers. The LiveDVD features a superb list of packages, some of which are listed below.

A special thanks to the Gentoo Infrastructure Team. Their hard work behind the scenes provide the resources, services and technology necessary to support the Gentoo Linux project.

• Packages included in this release: Linux Kernel 3.3.0, Xorg 1.12.0, KDE 4.8.1, Gnome 3.2.1, XFCE 4.8, Fluxbox 1.3.2, Firefox 11.0, LibreOffice 3.5.1.2, Gimp 2.6.12, Blender 2.60, Amarok 2.5 , VLC 2.0.1 and much more ...
• If you want to see if your package is included we have generated both the x86 package list, and amd64 package list. You may also be interested in checking out the FAQ. There is no new artwork for the 12.1 release, but you can still get the 12.0 artwork plus DVD cases and covers for the 12.0 release.
• Special Features:
  • ZFSOnLinux
  • Writable file systems using AUFS so you can emerge new packages!
  • Persistence for $HOME is available; press F9 for more info!

The LiveDVD is available in two flavors: a hybrid x86/x86_64 version, and an x86_64 multi lib version. The livedvd-x86-amd64-32ul-12.1 version will work on 32-bit x86 or 64-bit x86_64. If your CPU architecture is x86, then boot with the default gentoo kernel. If your arch is amd64, boot with the gentoo64 kernel. This means you can boot a 64-bit kernel and install a customized 64-bit user land while using the provided 32-bit user land. The livedvd-amd64-multilib-12.1 version is for x86_64 only.

If you are ready to check it out, let our bouncer direct you to the closest x86 image or amd64 image file. If you would prefer to take it easy on the bandwidth and help the community, torrents are also provided.

If you need support or have any questions, please visit the discussion thread on our forum.

Thank you for your continued support,
Gentoo Linux Developers, the Gentoo Foundation, and the Gentoo-Ten Project.

Thanks for your partipation in our April Fool's joke. This release is 100% real, it just has an extra feature when the date happens to be April 1st: The most intelligent Gentoo Install Wizard yet! If you want to check it out, just set your date before you boot the LiveDVD, and be sure to have a Linux-supported webcam plugged in.

I’m sure this is going to blow your mind completely.

During the last two months I’ve been busy re-thinking the way Package Management works in Linux distros in terms of User interaction and I ended up designing the new Sabayon Entropy GUI using a “less is more” approach. In layman terms, the idea is to carry out all the activities through two, and only two, main “interaction points”: a search bar and a notification bar (for sending feedback to User).

But why not going further and supporting Android Market Google Play Applications directly in Rigo?
This is exactly what I did, and it’s been even easier than implementing the whole Entropy Package Manager thanks to the fact that Android apps are self-contained and no dependency management is required.

So, yeah, Sabayon now supports Android Market Applications natively, both installation and runtime execution (through the Dalvik Java VM).

We still don’t have proper integration of CRAN and BIOC in Gentoo. I have proposed a GSoC project to remedy that. Feel free to contact me about it.

In the meantime if you need to install CRAN or BIOC packages you probably use install.packages() in R. This resolves R dependencies and installs all you need automatically. You are on your own about system dependencies though, but this is what CRAN integration in Portage is about. Upstream recommends you su root when installing R packages from within R. This is not always possible and I hope you will agree with me this is not the right thing to do. At best you will make your package manager cry due to collisions. At worst, well, it can be way worse. The recommended way should be for users to manage their R packages in their personal library. If you don’t have one, a personal library will be created for you the first time you use install.packages() or update.packages(). So there is nothing special to do and it mostly works.

One problem arises when a package you are installing depends on a version of a recommended package newer than the one you have. Recommended packages are some of those which are installed by Portage when you emerge R, thus they are owned by root. Non-root users can’t update them in the main library, which they don’t have access to, but they can do so in their personal library. However, there is a bug in R which will let you update recommended packages in your personal library, but only if you didn’t have one when you started the process. If you had one, as it’s most likely the case, the install will just fail. Frustrating.

About four weeks ago I silently added two patches to dev-lang/R-2.14.2 in order to fix this behavior. So far nobody complained. Yesterday I bumped R to 2.15.0 and the patches still apply cleanly. Please test and report back.

I have filed a bug upstream about these patches. Let’s see what they make out of them. The real solution to this problem though, at least as far as Gentoo is concerned, is to have Portage manage your R packages. And I’m sure it will come someday.

It's my pleasure to announce that version v0.3 of Trojitá, a fast and lightweight IMAP e-mail client based on the Qt library, is now available. For more details of what Trojitá is and who should use it, please see below.

(Permanent link to this announcement.)

Changes since version v0.2.9.4

This release is a major one, bringing new features, plenty of bugfixes and two new contributors to the table. A quick summary of what has happened:

• Drastic reduction in memory usage
• Tagging e-mails (contributed by Shanti Bouchez)
• Faster fetching of data
• Much more efficient support for threading
• More robust IMAP support
• A new Debug menu
• Autocompletion of e-mail addresses (contributed by Thomas Gahr)
• Reporting server/client configuration through ID
• Support of multipart/related for full rfc2387 compliance
• Showing Trojita's homepage on start
• GUI fixes (now finally works in dark themes)
• SMTP works over SSL (contributed by Shanti Bouchez)
• Expanded unit test coverage
• Plenty of bugfixes and further improvements

What is Trojitá

Trojitá is a very fast and lightweight IMAP e-mail client written using the Qt library.

What could make Trojitá interesting for you

• It's a pure Qt4 application with no additional dependencies; it builds in two-and-half minutes on a five-year-old laptop
• Robust IMAP core implemented using Qt's Model-View framework
• Standards compliance is a design goal
• On-demand message list and body part loading ("lazy loading")
• Offline IMAP support
• Support for bandwidth-saving mode aimed at mobile users with expensive connection
• IMAP over SSH -- instead of going over an SSL socket, the server could be accessed via SSH
• Safe and robust dealing with HTML mail

Please note that the "message sending" and "message composing" features of Trojitá are a bit lagging at this point and therefore it is not recommended to use Trojitá for these tasks apart from testing.

Certain features of Trojitá depend on the IMAP server's functionality. Trojitá is written from the bottom-up as an IMAP client and is designed around its feature set. For example, threaded message viewing (ie. "conversations") is supported only if the server implements an appropriate extension for now.

Trojitá is under heavy development, features are added on almost daily basis and the codebase is rapidly maturing. Certain useful features are still missing, there is no support for searching, for example. We have tickets opened for these, so please Cc yourself at the task tracker if you would like to follow the progress here.

Finally, as with any software, Trojitá has some bugs which are already known and reported in the issue tracker and some which are still waiting for discovery. That said, it is safe to use to for *reading* mail. I've been doing that for several years on a production account, and I have never lost a mail with Trojitá. Please do not send e-mails with current version, though, as it is known to produce non-standard messages in certain circumstances.

Where I can get it

Our web has all the required information, but if you are impatient and just want to grab the tarball for v0.3, download from Sourceforge:

Trojitá is known to work on Linux, Mac OS X and Windows. It should also run on all platforms supported by Qt. A MeeGo version is still pending, though.

Reporting bugs and wishes

Community

Trojita could always use more people in the community. Areas in which people are needed most, as well as general guidelines about how we prefer to work are documented at the wiki.

The IRC channel is #trojita on irc.freenode.net, the mailing list is trojita@lists.flaska.net (moderation required for non-subscribers, archived at various places etc).

Acknowledgment

A huge thank you goes to two new contributors who have submitted patches to make Trojitá better. It's my please to introduce Shanti Bouchez who is responsible for the new feature of tagging messages (and fixed STARTTLS for SMTP subscription in the process, among other things). The second contributor is Thomas Gahr who added e-mail auto-completion and fixed bugs.

Since its inception in 2006, many other people have contributed to Trojitá as well. I'd like to mention patches from Benson Tsai, John Rogelstad, Andrew Brouwers, Gil Moskowitz, Jiří Helebrant, Jun Yang, Justin J, and Tomáš Kouba, who have all sent patches in. Finally, another huge thank you goes to anyone who has reported bugs or helped make Trojitá better in any way.

Cheers,
Jan

Date: 25 March 2012
Time: 1300 UTC
Place: #gentoo-meetings @freenode

The full meeting log can be found here. Meeting summary is written by kensington.

1) Roll call
hwoarang, johu, pesa, tampakrap, wired, yngwin

2) Qt 5
* pesa is working on it locally, and started a new eclass from scratch. He’ll
push to the overlay once upstream settles down, probably after a packaged alpha
is released (note that binary compatibility is not guaranteed with the final
release).

* Upstream is planning split tarballs for later releases, and we’re going to
follow that except for qtbase, which will probably be split the same as in Qt 4
- qt-{core,gui,sql,dbus}.

* Need to handle slotting of qmake. Any ebuild calling qmake directly should be
fixed to use eqmake{4,5}. Autotools etc. is sometimes used to detect qmake which
shouldn’t be broken. qmake{4,5} will probably suffice, but we should watch
upstreams / other distros to see what they do, and revisit the issue later.

3) qt4.eclass deprecation
* wired maintains a list of ebuilds still using the class:
http://dev.gentoo.org/~wired/checks/qt4.eclass.html and there are about 60 left
in the tree

* We should try to clear out the ebuilds in testing first. Since there is a
repoman warning, we can start filing bugs / stable requests to get them fixed.

4) Unmaintained/obsolete packages
* qgtkstyle – this is duplicated in qt-gui[gtkstyle]. hwoarang will lastrite it
so it can be treecleaned.

* qvfb – it is stuck at 4.6.3 and nothing in the tree uses it, so we don’t care
about it. Since is has no bugs there’s no need to remove it, so hwoarang will
set it to maintainer-needed.

5) Get eclass translation handling into official tree
* The code in the overlay covers enough cases to make it worth merging it to the
tree. pesa will submit the code to the dev mailing list for review before it it
is merged into qt4-r2.

* With regards to eclass development in the future, we want to change the
workflow to match how the KDE herd does it. That is, replace qt4-edge with
qt4-r2 and use the overlay as a staging area.

6) Open bugs
* Bug #398885 – x11-libs/qt-assistant-4.7.4 qdoc3 loops forever on arm & ppc
This is blocking 4.7.4 stabilising on arm and is therefore a high priority. ppc
already stabilised, so the bug might only affect certain machines. We should
retry the out-of-portage build (see comment #1) and investigate further.

* Bug #367583 – x11-libs/qt-dbus-4.7.2 -
../../include/QtCore/../../src/corelib/arch/qatomic_generic.h:197: error:
invalid conversion from ‘const void*’ to ‘void*’
We have two candidate patches to choose from – one eclass, one code. pesa will
try to get an opinion from upstream about the code patch.

* Bug #401557 – x11-libs/qt-core-4.8 -
hint: qt-4.8 : add a ewarn to recompile cairo *after* qt-4.8
wired will add a warning to either qt-core or qt-gui.

* Bug #398497 – /usr/include/qt4/Gentoo/gentoo-qconfig.h should be under package
manager control
This bug is probably the result of an oversight by the original eclass authors,
and we agree that the file should indeed be under package manager control.

* Bug #372721 – [qt overlay] x11-libs/libmeegotouch-9999 doesn’t compile
Nobody cares about this package, so pesa will remove it from the overlay and
mark it WONTFIX.

* Bug #388551 – x11-libs/qt-gui should depend on gnome-base/libgnomeui-2 when
USE=”gtkstyle” is enabled
During the last meeting we decided to add an elog, but nobody actually did it.

* Bug #285743 – “webkit” USE flag standardization
We decided to do this 2 years ago, but never got around to it. The flag mostly
has two meanings: (1) add support for HTML rendering and (2) build bindings for
WebKit. wired will propose (1) to the dev mailing list: “Enable support for the
WebKit html rendering engine”, and (2) can be a local flag description.

* Bug #404283 – media-gfx/imagemagick – convert: unable to close module `SVG’:
/usr/lib64/qt4/libQtGui.so.4: undefined symbol:
_ZN11QMetaObject11removeGuardEPP7QObject @
warning/module.c/UnregisterModule/1605.
wired will request more information from the reporter.

Date: 22 March 2012
Time: 2000 UTC
Place: #gentoo-meetings @freenode

The full meeting log can be found here. Meeting summary is written by dastergon.

1) Roll call

alexxy, dilfridge, johu, mschiff, scarabeus, tampakrap

2) Electing a new team leader

nominees:

Accepted: dilfridge, johu
Refused: tampakrap, scarabeus

results:
johu -> dilfridge
dilfridge -> johu
tampakrap -> dilfridge
alexxy -> dilfridge
mschiff -> dilfridge
scarabeus -> dilfridge
———————-
dilfridge:5 – johu:1
———————-
dilfridge is the new KDE team leader with the majority of votes.
Congratulations!!!

3) Dropping RPATH from installed binaries

Add a RPATH entry for every library dir that is not in the system library
directories in ld.conf are not automatically considered as system library
dirs, but only some static list of dirs.
Everyone agreed on RPATH removal. We will introduce a patch with KDE SC 4.8.2
to remove the RPATH and move it to the main tree.

4) Bugs

* Remove hard dep on media-libs/phonon from kde-base/kdelibs
https://bugs.gentoo.org/show_bug.cgi?id=356681
https://bugs.gentoo.org/show_bug.cgi?id=388041
Upstream says “technically you can replace phonon with qt-phonon, but that’s
just stupid because you lose functionality”. Another argument against it, is
that qt-phonon will be removed with qt5. We wont fix the bug and keep phonon as
hard dep. johu will take care of the bug after meeting summary is available.

* Eclass problem with handbook without LINGUAS.
https://bugs.gentoo.org/show_bug.cgi?id=372457
The handbook eclass code is pretty confusing for all in meeting present
members. It was written by reavertm. We decided that we will mail reveartm to
fix handbook eclass code, because he has the most knowledge in it. dilfridge
will take care of contacting him, it’s his first lead task.

* Revise the change “semantic-desktop? -> semantic-desktop=”.
https://bugs.gentoo.org/show_bug.cgi?id=396491
Some packages rely on semantic-desktop capabilities in other ones. tampakrap
is volunteers to take care of the bug.

5) Open floor

* KDE 4.8 stabilization
KWin does not build anymore without opengl support. kde-base/kmail-4.8.1
crashes, but upstream patch can be backported. KDE SC 4.8.1 has tons of
bugfixes, compared to to 4.8.0. The majority votes for KDE 4.8.1 as stable
candidate. johu will prepare stabilization.

* Test failures
creffett brought up dbus-related test failures. johu recommended to get
virtual-dbus eclass running. dilfridge suggested if one or two test fails
or restrict. The problems with virtual-dbus is that you can’t run bash
commands in it’s environment. creffett will be responsible for this.

* New members
Welcome to three new members of the project: creffett, kensington and
dastergon. They are in the process to become gentoo developer and mentored by
tampakrap. creffett and kensington have already open ‘new developer’ bug.
dastergon doesn’t have an open bug yet.

* Comeback
scarabeus re-joined KDE herd. Welcome back!!!

* Reduced work time
mschiff mentioned that he will not have much time for the KDE herd due to
real life priorities, he will be back in may. tampakrap will spent most of his
time for gentoo infra team.

I've promised that I'll dig into some old file formats and check how well they can be accessed on today's systems with free software.

Today, I'll start with audio formats. To begin, in general there are two kinds of audio formats. Streamed audio formats start with a more or less raw audio stream, apply some encoding and sometimes (lossless or lossy) compression. There are also tracker audio formats. They have internal information on tone pitches and instruments. Most really old computer audio files are tracker formats (like the popular C64 SID format). This blog post will be about streamed audio formats and I'll save the tracked ones for a later one.

The file formats I've chosen are more or less random, the main criteria being that I once stepped over them and still remember that. There's a hughe collection of all kinds of media file samples on the mplayer server.

The single most important project regarding exotic audio or video formats is ffmpeg, a library that does despite its name much more than decoding mpeg. All major free software media players use ffmpeg.

The file formats I've investigated:

• Some of the very first files distributing music through the Internet I remember were real audio files (extension .ra or .rm) from the german punk band WIZO. Real audio has a whole bunch of variants, scanning through some of my old backups, most of them used either AC-3 or Real Audio 2.0 as their codec. Thanks to Waybach Machine, you can still find the WIZO downloads (Raum der Zeit - Techno is AC-3, the others are RealAudio 2.0).
• vqf (or TwinVQ) was once announced having better quality than MP3 and was discussed as its successor. However, it seems it is almost completely distinct today, I didn't find anything at all (except in the above mentioned sample collection) in vqf format for download.
• Monkey's audio, extension .ape, is a lossless audio codec, which is itself licensed under some kind of noncommercial-use-only license that doesn't qualify as free software. It's not really old, as it's still being developed, but I added it as another example of an uncommon format.
• Shorten (extension shn) is an old lossless audio format, which was often used by the etree project that collects recordings of concerts. Today, it is mostly deprecated by flac, but the old recordings are still available.
• voc: The popular dos floppy copying program vgacopy had sound before I had a soundcard - it used the pc speaker to play .voc files it had shipped. It's a format used by some Creative software for their SoundBlaster. It's a more-or-less raw audio format like wav.

For everybody out there complaining about liferea's decreased performance in the 1.8.x releases, upstream has released 1.8.3 which uses sqlite 3 WAL journaling.

This basically improves the overall liferea's performance, so if you are hating 1.8.x, please try the latest 1.8.3