tl;dr: app-portage/tatt now also does keywording bugs and includes bugbrowser code stolen from arch-tools.

It had been a while since I last worked on my arch testing tool tatt.  In the meantime Pawel’s arch-tools have gained momentum. Since I like to do archtesting from time to time, but felt too busy to develop tatt any further, last fall I thought of switching.  I have now abandoned that plan. Pawel’s and my workflow just seem very different and I think it is good to have two different approaches to automated arch testing. In the end this will mean more issues identified, because different tests are carried out. Tatt’s design is very centered around the individual bug (though this may change in the future). I think of it as “Give it a bugnumber and it will create all the scripts for automated testing and committing”.  Here’s how I currently use it:

# tatt -b ${bugnumber}

This will create a couple of scripts for USE-flag testing, rdep-testing, commit, clean-up, and also unmask the packages found in the title of the bug. When run with the -b option, the bug title is searched for package atoms. If there is a list of packages to be stabilized, I dump this list into a file and call tatt with

# tatt -b ${bugnumber} -f ${file-with-packages}

In this case it will use all the packages found in the file and write a single scripts that does testing for all of them.  When the scripts have been created, I run them

# sh ${something}-useflags.sh

# sh ${something}-rdeps.sh

The something is a jobname, automatically determined from the filename with -f, or the packagename with just -b, or passed on to tatt with the -j option. These scripts will run various emerge commands and protocol the output in  ${something}.report.  If everthing went well, I then do all the manual testing which may include running the program for a couple of hours. This part is very important especially for packages in the games-* category! If something did not work, then comes the manual investigation, so let’s assume that everything is fine for now. In this case I commit the stabilization to CVS by running the commit-script:

# sh ${something}-commit.sh

(Actually, since the test machine is virtual and I try to keep it tidy, I mirror the script to my dev-machine and commit from there, but you get the point). Once the commit is done, I clean-up the bug using

# tatt -r ${bugnumber} -m “Funny message”

This will comment on the bug with your funny message and remove your arch alias from CC. You can also pass the -c option which will close the bug as fixed. For all this bugzilla-automagic your pybugz should be logged into bugzilla or at least not ask for the password all the time. You have to take care of that yourself, for instance by making “bugz” an alias.

All this is long implemented.  What’s new: tatt now automatically determines whether the current job is a stable or keywording bug and writes the correct commit scripts. I also shamelessly merged some of Pawels bugbrowser code into tatt, so you can try this experimental feature using the -o option. Instead of passing a bugnumber, you would select the bug from the ncurses-browser, but again. My workflow is somehow different.

So much for now, let me know if you tried tatt and I would be delighted to receive pull requests on github.

Sitrep

I’ve been quiet lately due to the stream of changes around me. So shortly I am no longer only user of my desktop nor the major user of it. Thanks to that and issues that Gentoo showed while used by others I simply was unable to afford doing 24/7 tech support for issues with it main disk was sweeped with windows 7 and gentoo was moved to old 250 GB hdd that was long due after its survival date. This disk sadly died, so my remaining Gentoo install is HTPC which has no power to actually build libreoffice in any timely manner (what can one expect from Zacate platform).

Given to that I retired from office team in Gentoo and expected others to enjoy the building the beast. Which should not be that hard as I still work on it as upstream, I just don’t have the Gentoo part. As you can see the mail I was quite wrong as there is nobody actually willing to pick this thing up.

So I need new HDD and given to nowday prices for this components I really don’t want to buy it myself, so anyone wants to donate me some small ssd (30GB or larger as gentoo install + debug symbols take around 20 here [ggdb is huge])? On the other hand as council members behave really warmly these days (I would not care if normal dev said something like this but Markos is one of 7 elected ones picked by everyone) I think I should just ignore Gentoo and go ahead look around for some nicer distro…

Now back to interesting stuff.

libreoffice 3.5 release

3.5 will contain the lot of shiny stuff. Current 3.5.0.3 has fixed nsplugin and various other tiny issues like lpsolver and so on.
It still contains lot of tiny bugs everyone can see here and there, but they are getting fixed as we go so 3.5.1 or 3.5.2 will be ready for corporate environments again.

libreoffice trunk

There is steady stream of patches going into master branch at git.fdo and one of most notable thing is that we no longer use set_soenv perl script and most of the things are converted to gbuild (less paralel make issues, more compilance to MAKEOPTS). Other than that everything is same shiny like in 3.5 so no big difference on usability yet.

VLC translations

In mid-january it was brought to my attention that I am still listed as the one responsible for czech translation at VLC despite translating few strings in 2009. So I started complete overhaul using the KDE translation memory to improve it and remove all the wrong translations (you would not believe how much wrongly translated strings i found so far [I am at 50% of reviewing the current state]). So hopefully I will get this mostly done for 2.0 release so even Czech people can enjoy nice and translated media player.

The current status:

scarabeus@bugaboo: ~/l10n $ LC_ALL=C msgfmt --statistics vlc_new_cs.po
1766 translated messages, 4120 untranslated messages.
scarabeus@bugaboo: ~/l10n $ LC_ALL=C msgfmt --statistics vlc_cs.po
2140 translated messages, 1942 fuzzy translations, 1804 untranslated messages.

Gaming and other stuff

Anyone wants to play Battlefiled 3 or Counterstrike (actually anything you seen on the screenshot above :P)?

KDE release party in Prague

There will be release party for the above stuff in Prague on 24.2. at SUSE Linux offices and I will be giving some talk about KDE and LO (at least Tampakrap convinced me I should) so if you want to meet me (and other more cool KDE hackers) you should show up.

Update

As few people contacted me already I think i would like to buy one of these (one, two) or their smaller variants. If you want to realculate the price for Euros/USD divide by 25/17 respectively.

So if you want to donate feel free to sent money on paypal or mail me if you would rather want IBAN.
I will post results as another blog at point when I have enough money or in 30 days whatever will be sooner.

Those knowing me from my blog for a long time will probably be .. amazed to know that I’m typing this while on an American Airlines flight over the USA … I left from Brussels with Luca directed to Los Angeles, but that’s a story for another day.

I’m happy I’ve been able to be at FOSDEM this year — and I hope I’ll be at the one next year sa well! I almost didn’t make it (or to be precise I almost decided to fly back to Venice ASAP) once my flight was diverted through Luxemburg, instead of landing to the Brussels National airport, due to weather conditions, but after a three hours bus ride, and a very welcome (although not really comforting) taxi ride with Luca and Josh, I was able to be at the beer event as well.

Funny tidbit: when I departed I took my hat with me, as I knew it was cold.. I got it just before the new year’s eve in “a new shop” that opened at the mall near me. Most of my friends and acquaintances found it funny and strange, so I was intending to use it as a way to be easily spotted … I did not consider that Celio (the “new shop”) is based in France, and seems to be well established in Belgium as well, even if it’s really just arrived in Italy… I have seen that most of the hats in Brussels, especially among the locals outside of FOSDEM, were not much different from it..

Honestly I haven’t been able to assist to many talks, although there was one I couldn’t miss, from the CentOS guys, about their efforts at applying continuous integration on the distribution; with my interest in the tinderbox it was natural for me to be interested in their method as well. While they didn’t solve the API/ABI compatibility checks in a more complete way than us, like I was hoping, they did give me an idea for the chance to implement a pkg_test() function, which would run post-installation tests, designed to be used only on tinderbox-like builds, and not user (nor developer) installs.

I was happy I finally met Fabio, Donnie, Sejo and the many other Gentoo developers (and ex-developers as well); I was also able to get back in touch with Jo (directhex) from the Mono team, and to greet famous developers like Charles from JRuby and Michael Meeks (now from LibreOffice), who was the one introducing me to the magnificient and scary work of ELF symbols’ collisions, and thus the main motivator (unbeknown to him!) for me to write Ruby-Elf together with the symbol collision script. And of course, I finally met the VLC developers, and I promised I’ll do my best to be in Paris at the next DevDays.

Unfortunately Hans couldn’t be there (and I hope to hear from him soon), so me and Petteri took over his talk — if you look up the video, please do not laugh too hard; I’m not used to speak in public and I think it was something like my third public talk in my whole life, and the first in English. Maybe I’ll prepare something more complete for next year, it might be interesting. In that case I hope Charles will be able to assist as it’ll certainly talk about JRuby!

For those who wondered why I wasn’t at the keysigning event; beside the weather, the printable list of keys has arrived on thursday when I was just tidying up a few customers’ tasks and I ended up not having the time to actually print it out.. this was made worse by changing my plan the very last day to get to LA instead of going straight back to Venice.

If you followed my twitter stream during the event you know already I’m very opinionanted about one of the talks I assisted… but that’s yet another story for yet another day, I’d rather not waste time writing about it here.

Anyway, just wanted to say I’m very happy to have been there, very happy to have met developers and users – and I’m sorry I’m not naming everyone here, it would be a very long list! I hope to be around more often for sure.

P.S.: if anybody who’s reading this has seen a clumsy guy with a black man-purse falling down the stairs within the AW building on Saturday morning.. that was me. Ouch! I hurt myself, but luckily nothing extremely serious.

During the FOSDEM weekend, I’ve been also able to write down some wiki notes about getting Sabayon (and Gentoo) working on the BeagleBoard xM and PandaBoard. You may be interested in reading the Hitchhikers guides for these two boards (but don’t get excited too much, our chroots are currently hardfp, this means no 3D until Imagination/TI/whatever don’t release the hardfp version of their OpenGL libraries — or unless I make a softfp chroot). Considering that even Ubuntu is switching to hardfp, I really feel optimistic about it.

• Hitchhikers Guide to the BeagleBoard xM
• Hitchhikers Guide to the PandaBoard

Currently, you can find the BeagleBoard xM image on our ISO mirrors (under the “daily” directory), while the PandaBoard one will appear next Sunday. Both ship with the graphic stack, using omapfb (sigh), LXDE and Midori as Web Browser. NetworkManager is there as well. As you can imagine, there is still a lot of work to do (mainly making code compile, did I say Chromium?). Now I’m going to focus on the Efika MX a bit, it really looks to be orders of magnitude faster than the PandaBoard (woot!).

Enjoy arm and say thanks to Tigal for having sent me the boards.

So, it’s beeen a quite exciting weekend at ULB in Bruxelles.

I would really like to say thanks to all the people I’ve been talking with during these days. Hope you all had a good time there. It’s always nice to meet other devs IRL and share opinions on stuff.

Besides this, many exciting things are around the corner during the next 10-15 days. I’m almost done writing about the BeagleBoard xM and PandaBoard on the Sabayon wiki (feel free to copycat the stuff to Gentoo wiki, no problems here). This means that I am eventually going to start messing with the Efika MX nettop, can’t wait can’t wait. People from Genesi USA are awesome, so is their hardware, no kidding.

And, last but not least, Sabayon 8: I just need to find time to write the full release notes (tomorrow nite hopefully).

We’re happy to announce a KDE 4.8 Release Party in Prague,CZ!

The party will take place on Friday, 24th of February, 17:00, at the SUSE Linux building (Map, KDE Community Wiki). There will be KDE and openSUSE swag available, KDE SC 4.8 live CDs, plus some short KDE related talks. We’re also gonna have some drinks, a KDE Cake, and lots of fun!

PS In case you are a KDE contributor and would like to give a short talk about it, feel free to send a mail to me or Michal (for czech mails, michal [at] hrusecky [dot] net)

PS 2 Czech announcement and poster in Michal’s blog post

Recently with 1.7.9 release git gets support for git commit signing. This was one of the last show stoppers to migrate gentoo-x86 tree from cvs to git. In recent portage versions like 2.1.10.45 and 2.2.0_alpha85 support for git gpg signed commits was also added. So in kde overlay we decided to test with feature with thin manifests.

New repoman can use git gpg signed commits if you do the following thing

1. Enable special tag in metadata/layout.conf
   sign-commits = true
2. Install git >=1.7.9
3. Configure your gpg sign key for git via
   git config --global user.signingkey $KEYID

After first commit you can chek that gpg signature was added to special field in commit via

git log --show-signature

It will look like this

commit 9b3cafc7efb2c17b0f2baffae530196014967921
gpg: enabled debug flags: memstat
gpg: Signature made Sun Feb 5 21:06:06 2012 MSK using RSA key ID F82F92E6
gpg: Good signature from “Alexey Shvetsov ”
gpg: aka “Alexey ‘alexxy’ Shvetsov ”
gpg: aka “Alexey ‘alexxy’ Shvetsov ”
gpg: aka “Alexey Shvetsov ”
random usage: poolsize=600 mixed=0 polls=0/0 added=0/0
outmix=0 getlvl1=0/0 getlvl2=0/0
secmem usage: 0/32768 bytes in 0 blocks
Author: Alexey Shvetsov
Date: Sun Feb 5 21:05:48 2012 +0400

[metadata] Enable git signed commits by default

Signed-off-by: Alexey Shvetsov

Gentoo Prefix is still alive and going strong. In my opinion, Gentoo Prefix remains a strong point of Gentoo Linux and really establishes that Gentoo Linux is a metadistribution. In this post I want to focus on the numbers. The number of packages in the Gentoo Prefix tree, specifically. But first, a history lesson. It wasn’t until EAPI3 in Gentoo that “allowed” Gentoo Prefix variables into the main Gentoo Linux tree. That was in late 2011, but Gentoo Prefix existed much before then, all the way back to 2006 (at least). Before EAPI3, the prefix team made slight modifications to ebuilds and placed them in a repo and called it the tree of packages for Gentoo Prefix. This worked fine, but we had growing pains. The major issue was that we were getting too successful to manage the increased contributions from users. In other words, as the number of “forked” packages grew, the amount of maintenance time increased greatly – this is due to the fact that it is a chore to keep our forks synced. At least, a large chore for a small team. This is why we looked for help and adoption from the other pool of 200 Gentoo Developers, hence EAPI3 and beyond. Since supporting Gentoo Prefix is not a big use of overall developer time, this has gone over quite well in my opinion – yes, there are some pain points at times I do realize. Enough history, here are the numbers:

• Number of packages in Gentoo Linux: 15554 packages in 154 categories.
• Number of total* packages in Gentoo Prefix: 9483 packages in 154 categories.
• Number of KEYWORDED packages in Gentoo Prefix: About 3000 for the most popular arch
• Number of packages still NOT in the main Gentoo Linux tree: 369 packages

* The total packages in the tree also contains non-keyworded packages because that just makes life simple. Once packages started migrating to the main tree, I helped think of this “whitelist” concept. The short version of the whitelist is that if a package is listed in that text file, it gets included in the Gentoo Prefix tree as a direct copy of the version in the Gentoo Linux tree. The presense of the package in the old repo means that it is used instead. Eventually, this concept will go away and we will overlay the Gentoo Linux tree directly.

So why is it taking so long to migrate ALL packages to the Gentoo Linux tree? Well, that is where the rubber meets the road and we get into roadblocks. A roadblock for us could be a number of things, such as a disagreement with the Gentoo Linux maintainer, some patches existing that we don’t feel are a good fit for Gentoo Linux, or even us being lazy and not submitting stuff to upstream. We also don’t want to push invasive changes to Gentoo Linux for critical packages, like the toolchain for example.

It has long since been our agenda to not add anymore packages to the old repo and going forward only adding new stuff to Gentoo Linux directly. I hope we can make a dent in those remaining 369 in 2012!

If you do, maybe you want to consider proxy-maintaining it as it now on its way out. Upstream has a much newer version available, and we in the Proxy Maintainers team will be glad to steer you in the right direction when you need the help.

Just send us an email.

I keep getting e-mails asking me why app-text/xpdf is masked for removal from the portage tree. It's getting too much to reply individually, so let me sum up the situation here in a blog post.

# Andreas K. Hüttel <dilfridge@gentoo.org> (27 Jan 2012)
# Has developed into an unmaintainable mess, and everyone who
# knows about it is either retired or missing in action.
# Several minor bugs and one ugly security issues (#386271).
# Masked for removal because of lack of maintainer.
# Please try app-text/epdfview as light-weight replacement.
app-text/xpdf

• app-text/zathura (based on poppler & gtk+)
• app-text/apvlv (based on poppler & gtk+)
• app-text/epdfview (based on poppler & gtk+, ~unmaintained)
• app-text/mupdf
• app-text/gv (based on ghostscript)
• app-text/evince (based on poppler, Gnome application)
• kde-base/okular (based on poppler, KDE application)
• app-text/acroread (yes I know...)

sed -i 's/gregkh@suse.de/gregkh@linuxfoundation.org/g' .addressbook

As discussed in the last Gentoo Qt meeting, we moved our overlay from gitorious to git.overlays.gentoo.org. This is going to be the final move, I promise

Along with that, we decided to change the overlay from qting-edge to just qt. Layman list is alreay updated, so if you still have the old one, you should remove it and add the new one:

# layman -f
# layman -d qting-edge
# layman -a qt

Keep in mind that this overlay contains mostly live ebuilds of Qt (branches 4.7 and master), so make sure that you really need it before blindly adding it (the same applies for the kde overlay). Enjoy!

1. Roll call

johu, hwoarang, pesa, tampakrap, wired

2. Qt 4.8

* cairo fails to build, patched ebuild available in qting-edge, #380013

Cairo build issue is fixed in qting-edge overlay, will be moved together with Qt 4.8.0 to tree.

* qt now defaults to the raster graphicssystem, we should remove raster USE flag, #398283

Wired created a eselect module to choose the Qt graphicsystem. Raster is default, other selectable are opengl, openvg and native. Raster use flag is not needed anymore, qt-gui depends on the new eselect module.

* do we really want to keep qpa USE flag?

qpa and c++0x will be masked in tree.

* are we going to fix #363939 for 4.8?

Wired fixed this bug in qt 4.8.0. Qt 4.8 will be moved to tree on next weekend. Dilfridge prepares kde-base/kstyles-4.7.4 to be rebuild together with Qt 4.8.0 to prevent crashes in KDE apps with Oxygen style.

3. Minor arches and Qt >= 4.7

Upstream supports official amd64, arm and x86, but other arches also considered in configure script. Keep stable keywords for minor arches in Qt 4.6. Wait for minor arches arm, ppc, ppc64 in current stabilization in Qt 4.7.4. Drop sparc keywords in Qt 4.8.0.

4. Overlay migration to git.overlays.gentoo.org

Tampakrap will set up overlay on git.overlays.gentoo.org on next weekend. The new overlay will be renamed to qt instead of qting-edge.

5. Open bugs

* #398885 qdoc3 broken on arm

We will ask the reporter if it works when he builds manually by providing him a configure command to make sure he tries the proper build.

* #394533 Libreoffice crashes in qt on exit

Can’t be reproduced with Libreoffice 3.5.0.1, seems to be resolved by upstream.

* #392433 desktop file name issues

Will be fixed in Qt 4.8.0, so that qt-gui and qt-assistant no longer pass absolute paths to make_desktop_entry().

* #388551 qt-gui[gtkstyle] should depend on gnome-base/libgnomeui-2

We will add a elog message in qt-gui[gtkstyle] saying that for things to work you either need libgnomeui or that variable set properly in your env.

* #382559 qt_mkspecs_dir() returns bad spec directory

The bug will be marked as RESOLVED WORKSFORME, because we can’t reproduce it. Additionally we change the eclass not to use LIBDIR in favor of get_libdir() after Qt 4.8 hits the portage tree.

* #359391 qt4-build.eclass should check for —buildpkgonly before downgrade sanity check

Resolution will be RESOLVED WONTFIX. Sanity check is there for a reason. It’s not a matter of source or binary downgrade.

Since one hour ago, Qt-4.8.0 is in Gentoo portage tree. New major release so lots of new (or broken) stuff. The most important feature in this release is the integration of a new eselect module. This module will allow you to set your default graphics engine without the need to recompile Qt (x11-libs/qt-gui to be precise) from scratch. So, provided you have qt-gui-4.8.0 installed, you should be able to use the eselect module as follows:

hwoarang@mystical ~$ eselect qtgraphicssystem list

Available Qt Graphics Systems:
 [1] native
 [2] opengl
 [3] raster *

(note: if you have x11-libs/qt-openvg installed, one more option should be available)

Simply select your graphics system of preference, and then logout and login again.

hwoarang@mystical ~$ eselect qtgraphicssystem set 2
Setting opengl as your active Qt Graphics System... done
Please logout for changes to take effect.

Thanks to Alex(wired) for the eselect module implementation.
Enjoy ;-)

A small notification to tell you that the SELinux policies that were pushed to the main tree 30 days (or more) ago have now been stabilized (none of them introduced problems, although some of them have other bugs still open which are either fixed in ~arch or will be fixed in the hardened-dev overlay soon). I’ll be working on pushing an additional set of changes to hardened-dev overlay today as it includes fixes for openrc that are quite important, and might even push this to the tree faster than usual.

The reference policy is also working on a new release, so the moment it is released we will be picking that up as well (give or take a month, since my availability will be a bit less the next month).

In this podcast, create a best off cd with soundconverter and gnomebaker. The new Gentoo LiveDVD with persistance. The Northeast Linux Fest Saturday March 17, 2012, Worcester MA. Samsung ML3312 and Linux plus an Interview with Milan Kazarka.

Links

Northeast Linux Fest | Saturday March 17, 2012.
http://www.northeastlinuxfest.org/

Gnome Shell Extensions
https://extensions.gnome.org/

Gentoo 12.0 LiveDVD
http://www.gentoo.org/news/20120102-livedvd.xml

Samsung ML-3312ND
http://www.samsung.com/us/support/downloads/ML-3312ND/XAA
http://gpo.zugaina.org/net-print/samsung-unified-linux-driver

Interview with Milan Kazarka
http://www.gentoo.org/news/20120119-milan-interview-announcement.xml

Download

You probably remember my previous notes about Wordpress, FTP and the problem with security. At the end after a (boring) set up session I was able to get vsftpd provide FTPS service, which should be usable both by Wordpress and by Dreamweaver, so that my friend the webmaster can upload through it directly.

This is important because as it happens I have another prospective customer who’s going to run Wordpress, and FTPS now start to look more interesting than SSH, as it doesn’t require me to give shell access to the server either.

Unfortunately I’m a bit worried (maybe more than I should be) for the use of standard passwords rather than certificates or keypairs for authentication. Which meant I went tried to think of other alternatives.. of which there are mostly two: Google Authenticator and YubiKey .

The latter I knew by name already because I proxy-maintain the required software for Brant, and I know it’s outdated already and would require a new maintainer who can deal with those packages – I already posted about hardware-related maintenance for what it’s worth – so it was my first choice: while it meant I had to spend some money, it would have solved my problem and improved Gentoo, even if just for a tiny bit. The price for YubiKey devices is also low enough that, if I felt like providing more FTPS access to customers, I could simply bill it to them without many complaints.

So I went on the manufacturer’s (Yubico’s) website and tried to buy two of them (one for me to test and set up, and one to give my friend to access the server); despite publishing the prices in dollars, they sell through Sweden and UK, which means they are part of EU’s VAT area, and me being a registered business within EU, I should receive a reverse-charge invoice by stating my own VAT ID… never had much of a problem with it, as many of my suppliers are sparse through Europe, I registered for the “foreign-enabled” registry right when I opened business — don’t ask me why Italian (and Spanish as far as I can tell) business owners are not enabled by default to have intra-union suppliers.

Now trouble starts: since, as I just noted, not all VAT IDs are valid to use for intra-union trade, there has to be a way to ensure you’re dealing with an acceptable party. This is implemented through VIES the VAT Information Exchange System which, for what concerns Italian businesses, only tells you a boolean result of valid/invalid (and not the full registration data that most other states seem to provide). I knew VIES from a previous business agreement, but I never cared much. Turns out though that most e-Shops I encountered validate the VAT ID after order completed ­— or in the case of Amazon it seems like they check their internal database as well as VIES.

Yubico instead validates the request through VIES at the time of registration:

VAT Number could not be validated with VIES at this time. This typically happens when the service is under maintenance. Please retry after some time. For urgent orders, please contact order@yubico.com

Considering that the VIES website has a long disclaimer (which I can’t quote here for reasons that will be clear in a moment) stating that they do not guarantee the availability of the service at any time, and only seem to guarantee the validity of the data to the extent that the law ask them to (which probably means “as long as the states’ own databases are correct”), relying on such a service for registration is .. bad.

The VIES website is indeed down since at least 11am today (over four hours ago as I write this); for a moment they also gave me an interesting page (which I forgot to save), telling me that there were too many requests’ failures from “my IP address” … listing an IP address in the 212/8 range — my actual IP address is in the 94/8 range.

What’s the end result here? I’ll probably waste some more time trying to get Google Authenticator; Yubico basically lost a customer and a (possible) contributor by trying and failing to be smarter and won’t have a dedicated maintainer in Gentoo in the near future. It’s sad, because it seems to be easily the most cost- and time-effective solution out there (Google Authenticator is free, but it requires a greater investment of time, and time is money as we all should know).

Seems the patch I committed for the fix was corrupted.  So, I am rebuilding and releasing kernels for 3.2 , 3.1 and 3.0.

Thanks for wired for pointing this out.  I will be removing the ones from yesterday.

The following kernels now contain the fix:

gentoo-sources-3.2.1-r2

gentoo-sources-3.1.10-r1

gentoo-sources-3.0.17-r2

In my previous installment I ranted about. among other things, the way Rails suggests you to keep a world-writeable log file for the production environment. As I said at the end, I planned on looking at the syslogger gem and that was actually quite helpful.

The idea goes like this: by using syslogger you can tell Rails that the logs have to go through the syslog; in my case that means it goes to metalog, which then filters on the webapp names and pushes it to /var/log/rails, taking care of rotating the log as needed (either due to size or time — the former is quite useful to avoid that rogue bots cause a DoS, which happened to me when I was inexperienced with these technologies!). Of course, this only works on Unix, but that’s what I care about anyway.

Beside the placement of the logs, using metalog for me also means I can filter important messages and show them in the important messages’ log rather than being just limited to a hidden log file within the app’s own tree, and also means that I can mix in the messages of all the running applications, rather than having each report to a different file. If I were to use syslog-ng instead, I could easily make it send the logs via network to another box and aggregate all of them there… but I really don’t see the point (yet) for that, and the features that metalog comes with tramp easily the network support.

So how do you achieve this? It’s actually pretty easy. Obviously it starts with installing dev-ruby/syslogger (in Gentoo, through Portage, everywhere else, via gem); then you can configure this very easily on both Rails 2.3 and 3.x series (I have one server running Rails 2.3, the other 3.1… I have yet to set up Typo 6.x, but I’ll probably do that at some point in the near future, although unlikely before FOSDEM).

The trick is all in config/environments/production.rb, where you have to tell Rails to use a custom Logger; there is already an example, commented-out like that refers to the other gem, SyslogLogger, but you should change it to something like this

  config.logger = Syslogger.new("yourappname")

This way you can distinguish each application’s messages in the log. Then in the metalog.conf file you can have:

Rails apps : 
  program_regex = "^(typo|radiant|yourappname)"
  logdir = "/var/log/rails"
  maxfiles = 5
  break = 1

so that everything is then readable as /var/log/rails/current.

I’m not sure how much it impacts performance; I’d be surprised if it decreased them, as metalog also buffers the disk writes, but you never know until you check for sure; in general I still prefer if the (multiple) Rails processes send everything to metalog for my own convenience.

Interestingly, if you have a webapp that does not deal with on-disk files directly, but just with a database, by using syslogger you’re basically limiting the writing to the cache directories only, which is probably a positive note.

I just released gentoo-sources-3.2.1-r1 for Linux Local Privilege Escalation via SUID /proc/pid/mem .

I plan on creating releases for additional kernels with this patch through the day.

See the link for more info on the privilege escalation.

The following kernel versions contain the patch:

gentoo-sources-3.2.1-r1

gentoo-sources-3.1.10

gentoo-sources-3.0.17-r1

Usually, whenever a new KDE release is published, Gentoo users can update already the same day, as suddenly a complete and polished set of ebuilds appears in the portage tree. (Stay tuned on upcoming wednesday for KDE 4.8.0, it's shaping up very nicely!) How is this possible? Well... let me explain.

If you're a stable version user, you may have never heard of so-called live ebuilds. This is a special variant, usually denoted by a version number ending in 9999, that does not rely on a source tarball. Instead, it contains a URL of a revsion control system (say on anongit.kde.org). When you emerge such a version of a package, the sources of the specified branch are checked out or updated to the newest upstream state, and that is used for building the installation package. Obviously this is not for everyone; depending how well upstream structures commits, things may not build for a while, contain fresh bugs, ... Also, reporting bugs from live versions on Gentoo bugzilla is discouraged as most of the times we can't do anything about it (do it only if you are sure it's a problem with the ebuilds, not with the source). If you're running live, you should be willing to hack yourself and work with upstream.

However, many of the Gentoo KDE team members run these live ebuilds, partly the current bugfix branch (i.e. KDE/4.8), partly even git master. They continuously keep the live ebuilds in the Gentoo KDE overlay updated to the newest state of the source. When a release is made, the corresponding live ebuilds of this branch are copied to the version ebuilds. For example, the KDE/4.8 branch live ebuilds have the version number 4.8.49.9999 (i.e.
kde-base/kdelibs-4.8.49.9999), so when the pre-release tarballs for KDE 4.8.0 were released to the packagers a few days ago, we only had to copy all 4.8.49.9999 ebuilds to 4.8.0 and immediately had a working set for testing. Most problems at that point are only caused by changes in tarball packaging. As distribution packagers get the pre-release tarballs (that still may change due to last-minute bugfixes) a week before the official release date, these can easily be fixed in time.

This also means that KDE maintenance in Gentoo is really a team effort. Whoever moves a released version to the main portage tree and/or commits bugfixes there builds on all the work that the team has done in the overlay in the meantime. Cheers!

You might or might not remember my fighting with mod_perl and my finding a bug in the handling of logs if Apache’s error log is set to use the syslog interface (which in my case would be metalog). For those wondering the upstream bug is still untouched goes without saying. This should have told me that there aren’t many people using Apache’s syslog support, but sometimes I’m stubborn.

Anyway, yesterday I finally put into so-called “production” the webapp I described last week for handling customers’ computers. I got it working in no time after mongoid started to behave (tests are still restricted, because a couple fail and I’m not sure why — I’ll have to work on that with the next release that require quite fewer hacks to test cleanly). I did encounter a nasty bug in "best_in_place"http://rubygems.org/gems/best_in_place which I ended up fixing in Gentoo even though upstream hasn’t merged my branch yet.

To get it in “production” I simply mean configuring it to run on the twin server of this blog’s, which I’ve been using for another customer as well — and got ready for a third. Since Rails 3.1 was already installed on that box, it was quite easy to move my new app there. All it took was installing the few new gems I needed and…

Well here’s the interesting thing: I didn’t want for my application to run as my user, while obviously I wanted to check out the sources with my user so that I could get it to update with git … how do you do that? Well, Passenger is able to run the application under whatever user owns the config/environment.rb file, so you’d expect it to be able to run under an arbitrary user as well — which is the case, but only if you’re using version 3 (which is not stable in Gentoo as of yet).

So anyway I set up the new passenger to change the user, make public/assets/ and another directory I write to group-writable (the app user and my user are in the same group), and then I’m basically done, I think. I start up and I’m done with it, I think… but the hostnames tell me that “something went wrong”, without any clue as to what.

Okay so the default for Passenger is to not have any log at all, not a problem, I’ll just increase the level to 1 and see the error… or not? I still get no output in Apache’s error log .. which is still set to syslog… don’t tell me… I set Passenger to log to file, and lo and behold it works fine. I wonder if it’s time for me to learn Apache’s API and get to fix both, since it looks like I’m one of the very few people who would like to use syslog as Apache’s error log.

After getting Passenger to finally tell me what’s wrong, I find out both the reason why Rails wasn’t starting (I forgot to enable two USE flags in dev-ruby/barby which I use for generating the QR code on the label), but I also see this:

Rails Error: Unable to access log file. Please ensure that /var/www/${vhost}/log/production.log exists and is chmod 0666. The log level has been raised to WARN and the output directed to STDERR until the problem is fixed.
Please note that logging negatively impacts client-side performance. You should set your logging level no lower than :info in production.

What? Rails is really telling its users to create a world writeable log file, when it fails to write to it? Are they freaking kidding me? Is this really a suggestion coming from the developers of a framework for Web Applications which should be security-sensitive? … Okay so one can be smarter than them and do the right thing (in my case make sure that the log file is actually group-writeable) but if this is the kind of suggestions they find proper to tell you, it’s no wonder what happened with Diaspora. So it’s one more reason why Rails shouldn’t be for the faint hearted and that you should pay a very good sysadmin if you want to run a Rails application.

Oh and by the way the cherry on top of this is that instead of just sending the log to stderr, leaving it to Passenger to wrangle – which would have worked out nicely if Passenger had a way to distinguish which app the errors are coming from – Rails also moves the log level to warning, just to spite you. And then tells you that it impacts performances! Ain’t that lovely?

Plan for the day? If I find some extra free time I’d like to give a try and package (not necessarily in this order) syslogger so that the whole production.log thing can go away fast.

Today, olemarkus finally removed the dev-lang/php-5.2.17 ebuild from the gentoo-x86 tree.

So it's been nearly 5 years since php-5.2.1-r3 got introduced to the main tree. And it comes almost exactly a year after the last release of php-5.2 was announced on php.net. Half a decade lifetime is pretty decent for release cycles, ain't it?

But with all the new and shiny features in 5.3 (and 5.4!), there's really nothing bad about letting php-5.2 die.

So by all means: PHP is dead, long live PHP!

P.S.: If you find any zombies, ie packages you want to merge but that require php-5.2 to function, please notify the Gentoo PHP team via our bug tracker or leave a comment here. Thanks a lot!

Since the general trend on many linux distros is towards requiring /usr to be mounted at boot time, I figured I’d see what it would take to get it working using dracut.

I’ve been messing with dracut for a while, and for some reason it stubbornly refuses to detect my raid devices. The kernel autodetection works fine, but this is disabled when booting from an initramfs. Dracut would timeout and drop me to a dash shell, and if I just typed mdadm -As followed by exit it would boot just fine.

Dracut is using udev to set up raid devices, and obviously that is not working.

Beyond this, I’d like to get my /usr mounted pre-boot, and there is a module called usrmount that purports to do just this. However, it isn’t working in my case because /usr is a bind mount to a subdir on an lvm volume, and it just isn’t figuring that out (it doesn’t even run lvm in the first place despite having the module installed, let alone figuring out what to mount in what order – I suspect the lvm module only works if root is on lvm).

My solution to both problems is to build my own simple dracut module. If you want to try it out:

1. cd /usr/lib/dracut/modules.d/
2. mkdir 91local
3. cat > 91local/module-setup.sh
   #!/bin/bash
   # -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*-
   # ex: ts=8 sw=4 sts=4 et filetype=sh

   check() {
   return 0
   }

   depends() {
   return 0
   }

   install() {
inst_hook pre-trigger 91 "$moddir/mount-local.sh"
}


4. cat > 91local/mount-local.sh
   #!/bin/sh
   # -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*-
   # ex: ts=8 sw=4 sts=4 et filetype=sh

   mount_local()
   {
   mdadm -As
   lvm pvscan
   lvm vgscan
   lvm lvscan
   lvm vgchange -ay
   }

   mount_local

Then run dracut to build your initramfs, and it should let mdadm and lvm auto-detect everything before it gets to mounting stuff. You can then use the fstab-sys to mount whatever you need to mount user. However, in your fstab.sys if you’re configuring a bindmount be sure to prepend /sysroot/ before the source directory.
Example fstab.sys:
/dev/vg1/data /data ext4 noatime,user_xattr,barrier=1 0 0
/sysroot/data/usr /usr none bind 0 0
/sysroot/data/var /var none bind 0 0

Hopefully this helps somebody out – the dracut documentation is pretty sparse. In fact, if somebody connected to dracut stumbles upon this I’d be open to a better way of hooking my script – pre-trigger just doesn’t seem right – I’d rather let udev try to do everything first. However, I couldn’t find any way to hook after udev runs but before it bombs out not finding my root device. Suggestions welcome.

1) Roll call

alexxy, jmbsvicetto, dilfridge, johu, mschiff, tampakrap, Thev00d00

2) Electing a new team leader

Since one year is not over yet, it will be skipped for the next meeting.

3) What shall we do with kdepim-4.4

KDEPIM 4.4 is not supported any more by upstream, but on the other hand KDEPIM2 is still too buggy. We had a discussion if we should remove it completely or if we should continue maintain it, despite the compatibility bugs that started to emerge with newer KDE versions. Final decision is that we will continue support it as long it works with newer KDE SC releases. We’ll keep the kdepim-l10n split package to provide the translations for it.

4) kdeenablefinal revisited

Since upstream doesn’t seem to care about it much, plus it doesn’t make much sense now that there are many split tarballs, we decided to remove it the next day after the meeting.

5) phonon-xine removal

KDE upstream acknowledged that this is not maintained anymore. It’s already masked since 2011/12/01. Will be last rited and removed 15 days afterwards.

6) Qt 4.8

We expect no big issues with it. Kdenlive is the only known application that does not build at the moment and will be patched. kde-base/kstyles-4.7.* needs to be rebuilt after the upgrade, which we’ll solve with a combination of revbump/dependencies (otherwise KDE apps using oxygen style crash).

7) Dropping RPATH from installed binaries

Postponed for next meeting, need more info from reavertm and/or hardened herd.

To eselect Boost or not to eselect boost

No final decision was taken, discussion will be moved to -dev mailing list.

9) Bugs

* dev-util/cmake picks always the latest boost. Fix in overlay since 13. Dec. Move to tree? https://bugs.gentoo.org/show_bug.cgi?id=335108

see 8.

* cmake-utils.eclass PREFIX is not defined, any progress? https://bugs.gentoo.org/show_bug.cgi?id=358059

Postponed for next meeting

* Remove hard dep on media-libs/phonon from kde-base/kdelibs https://bugs.gentoo.org/show_bug.cgi?id=356681 https://bugs.gentoo.org/show_bug.cgi?id=388041

Although it is possible to build kdelibs against qt-phonon, it is not recommended by upstream. Decision postponed for next meeting.

* Eclass problem with handbook without LINGUAS. https://bugs.gentoo.org/show_bug.cgi?id=372457

Needs more analysis. Postponed.

* MacOSX request for cmake-utils.eclass: Remove force of  CMAKE_BUILD_WITH_INSTALL_RPATH=TRUE https://bugs.gentoo.org/show_bug.cgi?id=398437

That was a request by the Gentoo Prefix team, and got accepted

* Revise the change “semantic-desktop? -> semantic-desktop=”. Why was the change needed. https://bugs.gentoo.org/show_bug.cgi?id=396491

We had split opinions on this. Skipped for next meeting, as we need reavertm’s input on this.

10) Open floor

• Tampakrap will make a KDE SC 4.8 release party in Prague, more info coming soon.
• Qt meeting on Thursday 26th Jan.
• See you at fosdem

Milan is from Foresight Media s.r.o, who produce interactive Touch Tables, that run Gentoo Linux. One of the products are a low cost alternative to Microsoft's Surface. Be sure to check out an overview of their products. Milan, thank you very much for your time. My first question is this:

1. Who is Milan and how did you get started with Gentoo?
   • I guess I'm a product designer, developer, entrepreneur and part time artist living in Central Europe usually in Vienna, Bratislava and Prague. To be able to create inventions, new gadgets you either need a ton of money or you learn how to do many things by yourself in a garage or in my case in my atelier. For me it would be quite depressing to ‘just design' something :) And so I create prototypes, which I push into serial production like my touch table designs. When I was 13 years old I accidentally saw a magazine with a penguin. I thought it was a cool logo of something. Then I saw it said that there's a free CD of an operating system that I haven't heard of. I could not hack my pre-installed commercial software enough and so I gave it a try and I guess it's the usual story of many Open Source and Linux geeks from there on :) After some time using various Linux distributions I saw that the complexity and the number of regressions in many of them has become so high over the years that I needed a system that would let me stay in control and a system that would value it's own design. Gentoo was a natural choice.

Be sure to check out the Full Interview!

Discuss this!

SCALE 10x is almost here, and Gentoo will be there!

Southern California's premier open-source software event is just around the corner, running from Friday, January 20 through Sunday, January 22. Several Gentoo developers will be there; it will be even bigger than previous years.

We'll be showing off some nifty devices running Gentoo, and we'll be giving out installation media. Whether you're a developer, user, or simply curious, be sure to stop by booth #70. See you there!

Discuss this!

To my European readers: if you care about the impact of social technologies like Git (and GitHub) & how they’re transforming software development, or the impact of social technology on communities, and you enjoy good beer, you need to be at Monki Gras. I just posted over at my RedMonk blog about how the previous conference in the series, Monktoberfest, was the best conference of my life. And I’ve been to many.

Monki Gras is Feb. 1–2 in London. The timing’s perfect to stop by just before FOSDEM (and that’s exactly what I’m doing). Registration is dirt-cheap, speakers are universally top-notch, and you’ll also get some world-class beers in the package.

Yesterday (2012/01/16 20:00 UTC) we had the first Gentoo KDE team meeting this year. The meeting happened in #gentoo-meetings on freenode.

• Participants: alexxy, dilfridge, jmbsvicetto, johu, mschiff, tampakrap, Thev00d00
• Agenda
• Log
• Lead election is delayed, because 12 months not over
• We keep kdepim-4.4 in tree as long as it works and provide kdepim-l10n package
• Kdeenablefinal build feature will be removed today
• Phonon xine backend will be removed in 15 days
• We expect no big issues with Qt 4.8,  only kdenlive is not building at the moment
• eselect boost vs. latest boost is not a Gentoo KDE scope only issue, we will move the discussion to the gentoo-dev mailing list
• … read log or wait for the full summary by tampakrap
• My netbook had a kernel panic while the meeting  :-/

I’ve spent the first week of the year on vacation with some friends. The second week of the year has been mixed between going on with the jobs I should have gotten working already, fighting a bad case of cold, and getting insulted by a customer of mine for actually having gotten real vacation time for once in two years. More to the point: said customer doesn’t actually pay me overtime, or actually at all for the support.

Tonight I wanted to relax and think about my own needs. Not personal needs, alas, but at least needs for my work to become easier. Since I haven’t made any progress at all regarding RT I decided to look into a different need of mine: cataloguing customers’ computers.

I originally simply kept a file listing the computers I set up for customers — then I started getting more customers, and sometimes getting a computer back after many months since last time. And I started forgetting which computer was which. Nowadays I have 79 computers on my “database” (which is just a git repository with a bunch of HTML files as well as lshw dumps), without counting those that have been dismissed.

To recognise the computers, I started printing labels with a QR Code on them, which contains the URL of the computer’s HTML file on my website (password-protected). My original method required me to feed a multi-label A4 sheet into my laser printer and print one, two or three labels out on that… but it turned out to be a waste of time and of money in sheets, given that most of the time I ended up wasting half of it, as the printer refused to print aligned more than half the time. I’ve since bought a Dymo label printer, which is why you’ll find their drivers in Portage maintained by yours truly — the nice thing about Dymo’s label printers is that their drivers are fully GPL-2, while as far as I can tell both Zebra and Brother have binary blobs, that make them unsuitable for use on amd64-based systems.

As you can tell, there are a few things that I did in Gentoo that relate to this little “database” of mine: the lshw fixes to try getting it back into SysRescueCD (it’s still not there — and I lost the password for my account on their forums), the Dymo drivers noted above, and dev-ruby/barby which is a quite interesting library that allows you to generate almost any kind of barcode. And now it’s time of MongoDB Ruby libraries as I’m trying to write an actual web application to manage the “database” and make it a real database.

Today’s achievement is big: I finally got Rails (3.1) to play nice with MongoDB. Not using MongoMapper, the author of which, as I already talked about I would prefer not having much to discuss with. But thanks to Mauro I got pointed at Mongoid which is a much more well developed alternative.

Okay sure there are quite a few things to kink out in the packaging of Mongoid – for instance the fact that the gem packages a Rakefile that relies on a (missing) Gemfile, or the fact that two out of three rspec targets in said Rakefile fail, one of which by crashing the interpeter – but at least their unit-tests work, and the code works as intended when loaded it up. Which is more I can say about MongoMapper.

Oh and it doesn’t seem to require extra code to be added just to work correctly with Passenger.

The only problem I have now is fixing up one side issue: how do I print the labels once I load this into my webserver? I could download the PDF I use to print the label and then print that.. but it’s a bit of a time-waster. Of course both the server and Yamato (where the label printer is connected) are IPv6-enabled and .. well, the IPP protocol used by CUPS is fine to be used over the internet, as it can use SSL encryption. Which yes, means that I’ll be setting up a web application … that calls home to print a label, how crazy is that?

My only issue with this is that I’d rather not install cups on the webserver (especially since there is currently no way to just build the client side of it, which would be the only part of it I would need on the server — yeah I know, it’s funky), so I can’t just call lpr mylabel.pdf… and as far as I can tell, the only way to access IPP from Ruby is one of the many CUPS library bindings available as gems, which are all 0.0 versions, and do not inspire me the least. Since IPP is based off HTTP, I would have expected more implementations of it, to be honest.

Possibly, it should be possible to extend some HTTP Ruby library to send IPP requests as well; for what I’m concerned, I’d just need the “Print-Job” method to be implemented, which would allow me to send the PDF file to be printed with the default options. I guess I’ll resolve that bit once I’m done with the rest of my application, though.

I’m no fan of initramfs. All my systems boot up just fine without it, so I often see it as an additional layer of obfuscation. But there are definitely cases where initramfs is needed, and from the looks of it, we might be needing to push out some documentation and support for initramfs. Since my primary focus is to look at a hardened system, I started playing with initramfs together with Gentoo Hardened, grSecurity and SELinux. And what a challenge it was…

But first, a quick introduction to initramfs. The Linux kernel supports initrd images for quite some time. These images are best seen as loopback-mountable images containing a whole file system that the Linux kernel boots as the root device. On this initrd image, a set of tools and scripts then prepare the system and finally switch towards the real root device. The initrd feature was often used when the root device is a network-mounted location or on a file system that requires additional activities (like an encrypted file system or even on LVM. But it also had some difficulties with it.

Using a loopback-mountable image means that this is seen as a full device (with file system on it), so the Linux kernel also tries caching the files on it, which leads to some unwanted memory consumption. It is a static environment, so it is hard to grow or shrink it. Every time an administrator creates an initrd, he needs to carefully design (capacity-wise) the environment not to request too much or too little memory.

Enter initramfs. The concept is similar: an environment that the Linux kernel boots as a root device which is used to prepare for booting further from the real root file systems. But it uses a different approach. First of all, it is no longer a loopback-mountable image, but a cpio archive that is used on a tmpfs file system. Unlike initrd, tmpfs can grow or shrink as necessary, so the administrator doesn’t need to plan the capacity of the image. And because it is a tmpfs file system, the Linux kernel doesn’t try to cache the files in memory (as it knows they already are in memory).

There are undoubtedly more advantages to initramfs, but let’s stick to the primary objective of this post: talk about its implementation on a hardened system.

I started playing with dracut, a tool to create initramfs archives which is seen as a widely popular implementation (and suggested on the gentoo development mailinglist). It uses a simple, modular approach to building initramfs archives. It has a base, which includes a small init script and some device handling (based on udev), and modules that you can add depending on your situation (such as adding support for RAID devices, LVM, NFS mounted file systems etc.)

On a SELinux system (using a strict policy, enforcing mode) running dracut in the sysadm_t domain doesn’t work, so I had to create a dracut_t domain (which has been pushed to the Portage tree yesterday). But other than that, it is for me sufficient to call dracut to create an initramfs:

# dracut -f "" 3.1.6-hardened

My grub then has an additional set of lines like so:

title Gentoo Linux Hardened (initramfs)
root (hd0,0)
kernel /boot/vmlinuz-3.1.6-hardened root=/dev/vda1 console=ttyS0 console=tty0
initrd /boot/initramfs-3.1.6-hardened.img

Sadly, the bugger didn’t boot. The first problem I hit was that the Linux kernel I boot has chroot restrictions in it (grSecurity). These restrictions further tighten chroot environments so that it is much more difficult to “escape” a chroot. But dracut, and probably all others, use chroot to further prepare the bootup and eventually switch to the chrooted environment to boot up further. Having the chroot restrictions enabled effectively means that I cannot use initramfs environments. To work around, I enabled sysctl support for all the chroot restrictions and made sure that their default behavior is to be disabled. Then, when the system boots up, it enables the restrictions later in the boot process (through the sysctl.conf settings) and then locks these settings (thanks to grSecurity’s grsec_lock feature) so that they cannot be disabled anymore later.

But no, I did get further, up to the point that either the openrc init is called (which tries to load in the SELinux policy and then breaks) or that the initramfs tries to load the SELinux policy – and then breaks. The problem here is that there is too much happening before the SELinux policy is loaded. Files are created (such as device files) or manipulated, chroots are prepared, udev is (temporarily) ran, mounts are created, … all before a SELinux policy is loaded. As a result, the files on the system have incorrect contexts and the moment the SELinux policy is loaded, the processes get denied all access and other privileges they want against these (wrongly) labeled files. And since after loading the SELinux policy, the process runs in kernel_t domain, it doesn’t have the privileges to relabel the entire system, let alone call commands.

This is currently where I’m stuck. I can get the thing boot up, if you temporarily work in permissive mode. When the openrc init is eventually called, things proceed as usual and the moment udev is started (again, now from the openrc init) it is possible to switch to enforcing mode. All processes are running by then in the correct domain and there do not seem to be any files left with wrong contexts (since the initramfs is not reachable anymore and the device files in /dev are now set again by udev which is SELinux aware.

But if you want to boot up in enforcing straight away, there are still things to investigate. I think I’ll need to put the policy in the initramfs as well (which has the huge downside that every update on the policy requires a rebuild of the initramfs as well). In that case I can load the policy early up the chain and have the initramfs work further running in an enforced situation. Or I completely regard the initramfs as an “always trusted” environment and wait for openrc’s init to load the SELinux policy. In that case, I need to find a way to relabel the (temporarily created) /dev entries (like console, kmsg, …) before the policy is loaded.

Definitely to be continued…

Maybe some of you have noticed that CUPS 1.5.0 is still hard-masked. Well, the reason for that is simple- at home, I'm nearly not printing at all, and at work, I have to rely on printing too much to tinker with it on the side a bit. So... if you would like to help, please unmask net-print/cups-1.5.0-r2 and give it a try. You will for sure find some problems, as the only thing I tested looong time ago was building it, never actually running it. Report them on bugs.gentoo.org, and we'll have a look... with a bit of luck, the package mask can then go away at some point. Any feedback (also positive) is appreciated. Cheers!

As known as #335108. This is (was) a long term bug in Gentoo KDE scope. The problem is that if you have two or more different boost versions installed, the latest version will be used at build time, regardless which version is (e)selected. Real world example we have boost  1.46.1 and 1.47.0 installed selected the 1.46 slot, the 1.47 slot would be used at build:

$ eselect boost list
Available boost versions:
 [1]   boost-1.46/default *
 [2]   boost-1.47/default

Last night i patched dev-util/cmake-2.8.6 successfully and made the revision bump today in the kde-overlay. So please test =dev-util/cmake-2.8.6-r5, in the case your maintained package is cmake based and needs dev-util/boost at build time. You should test at least with two different boost versions and of course switch between those to check that the selected version is used.

I bumped dev-util/cmake-2.8.7 in the overlay too. The patch is also included in this version.

Start your engines…

While testing kernel 3.1.6 for bug #396469 I got the common kernel panic "not syncing: VFS: Unable to mount root fs".

It's easy to fix, here's the grub config before:


title Gentoo Linux
root (hd0,0)
kernel /boot/vmlinuz-3.1.6-gentoo

And fixed one:


title Gentoo Linux
root (hd0,0)
kernel /boot/vmlinuz-3.1.6-gentoo root=/dev/sda1

I had to pass an explicit root= parameter. How to figure it out? mount -l or cat /proc/mounts are not so helpful:


rootfs on / type rootfs (rw)
/dev/root on / type ext3 (rw,noatime,errors=continue,barrier=1,data=writeback)

This is just a short post to let my followers know that I’ll be at FOSDEM next month. I’ve booked the flight back in September and I booked the hotel yesterday, so it’s all set. I just hope not to get lost through Bruxelles.

The only reason why I’m posting this is, actually, that I need some suggestion from somebody who knows Belgium: both my phone operators lack dedicated roaming up there, so I’ll probably end up with an hefty bill waiting for me back home. Given in Italy you really can’t get a local pre-paid SIM to user your phone if you’re a tourist, I’m not sure if the same holds true in Belgium. And most importantly, whether I could re-use such a SIM over the years (as I plan on coming to FOSDEM with regularity, if I survive the trip alone this time).

At any rate, if you want to discuss anything in person, I’ll be the guy with the strange hat and the purse satchel (geek points for getting the reference), hanging around with the Gentoo or libav folks.

I thought it would be easier to do a round of stable kernel releases in the middle of the larger kernel merge window, to prevent the next round from being so big (given that there are a lot of patches usually applying during the -rc1 merge window cycle).

So, I've now done:

• Linux 2.6.32.54-rc1 release
• Linux 3.0.17-rc1 release
• Linux 3.1.9-rc1 release
• Linux 3.2.1-rc1 release

Please go test and let me know if there are any problems with any of these kernels. If I've missed any patches that you feel should be in them, also please let me know.

Note, this is most likely going to be the LAST 3.1.y kernel release, so please move off to the 3.2 kernel at this point in time. Maintaining so many different kernel branches all at once is not trivial, and I want to minimize it if at all possible.

Some time ago when we were filing the first stable request for KDE 4.7 I decided I'd have to give KDEPIM-4.7 also a try. I used to be a pine (and later alpine) user for ages, some time ago I switched to kmail1 (maybe at version 4.2 or so) and have been using it ever since... About the setup, incoming mail is stored and filtered server-side (Novell Groupwise), and accessed as disconnected IMAP.

1) Office desktop.
This one's usually running the latest and greatest KDE RC or beta, with a static IP and something like a 100MBps-FD link to the mail server. No local folders, so this was the natural candidate to test first. I did not even try the data migration, but wiped my entire local configuration of the KDEPIM programs as thoroughly as possible before updating. On the whole this went rather well. From 4.7.3 to 4.7.95 I've seen my share of kmail2 / kontact / akonadi crashes, but none of them really led to bigger problems. Impressively, I could also see each of these crash bugs I hit get fixed on KDE bugzilla in the meantime! Since upgrading to 4.7.97 I haven't had a crash anymore.
The only thing that is really bugging me a the moment is that I absolutely can't drag an e-mail message into another folder (I always have to right-click on the entry in the message list, select "Move to folder", ...). However, that is likely a problem in one of the underlying libraries. :| I would really like to help debugging this problem; if you can give me any clue where to look, please message...

2) Laptop.
Going home for christmas meant spending some time on a train. The days before I had upgraded this box to QT-4.8.0 and KDE+KDEPIM 4.7.95. Again, clean start with a new configuration. On the train I spent some time ironing out bugs there. The GPRS connection was happily going down and up again with every tunnel or less-populated countryside. That's when the akonadi backend started really acting up. When I arrived at my family's place, my e-mail had become fully non-functional (no fetching e-mails, no sending e-mails, the backend making kmail hang, regular crashes, and all fumbling with akonadiconsole did not help). After a while of trying, I gave up, wiped my entire KDEPIM configuration and data and downgraded to 4.4 again. I was impressed how responsive my e-mail program suddenly became.
Talking to other people, it seems that some have problems with bad internet connections, some don't. Maybe I should have used networkmanager, at least Alexxy reports that it's working fine with it.

3) Home desktop.
Well... Here I store my e-mail archive since 1996- that's maildir folders with roughly 50000 messages. Maybe I'll consider upgrading KDEPIM sometime around KDE 4.9.

I'm sorry if this blog entry is demotivating for the program developers. Some time ago I followed part of an animated discussion on irc between a kdepim user and a kdepim developer. Quoting a small part,

<user> first: fix bugs. second:dont change data formats. third: change gui only if absolutly nessary.
<developer> no
<developer> first: have fun; second: make it work for others

As 3.2 is now out, here's a note as to the current status of the different stable/longterm kernel trees.

First off, please everyone remember to mark any patch that you want to have applied to the stable kernel trees with a simple:

Cc: stable <stable@vger.kernel.org>

marking in the Signed-off-by: area. Once the patch hits Linus's tree, I will automatically be notified of it and it will be applied if possible. If it does not applied, you will be notified of that.

Note that the address is stable@vger.kernel.org, not the older address that used to be used before October of 2011.

At this time, all stable and longterm kernel trees are being maintained in one big git tree, located at:

git.kernel.org:/pub/scm/linux/kernel/git/stable/linux-stable.git

There are different branches for every different major kernel version.

Here's the different active kernel versions that I am maintaining at the moment:

• 3.2.y - this will be maintained until 3.3 comes out
• 3.1.y - there will be only one, maybe two, more releases of this tree
• 3.0.y - this is the new "longterm" kernel release, it will be maintained for 2 years at the minimum by me.
• 2.6.32.y - this is the previous "longterm" kernel release. It is approaching it's end-of-life, and I think I only have another month or so doing releases of this. After I am finished with it, it might be picked up by someone else, but I'm not going to promise anything.

All other longterm kernels are being maintained in various forms (usually quite sporadically, if at all), by other people, and I can not speak for their lifetime at all, that is up to those individuals.

If anyone has any questions about any of this, please let me know.

I've just finished stabilizing KDE 4.7.4 on x86, and it was about 300 packages, so it would be very hard to do without good tools. I have some pretty nice scripts, so it took me maybe a few supervision and everything has been done automatically (even then it can take 3-4 hours, so trying to do that manually would be a huge waste of time and take even longer).

Here's how I did that. First I created a ~/kde-stable.txt file with the following contents:


# bug #396359
=app-misc/strigi-0.7.7
=app-office/akonadi-server-1.6.2-r1
=dev-libs/shared-desktop-ontologies-0.8.1
=kde-base/activitymanager-4.7.4
=kde-base/akonadiconsole-4.7.4
=kde-base/akregator-4.7.4

NetworkManager 0.9 has changed a lot of stuff. Before NM 0.9, there are user connections and system connections. Ifnet plug-in is made to only take charge of system connections. But after NM 0.9, all connections become system connections and ifnet tries to manage all of them. Then some bad things might happen. The problem is reported by David Narvaez and he helped a lot on it. Bug #392409 is also caused by the problem. This is fixed in networkmanager-0.9.2.0-r2: if a user does not check “available to all users”, ifnet will not handle the connection any more and the default plug-in “keyfile” will do the job.

Another important change is that ifnet now supports openrc style. As bash arrays are not allowed by openrc, ifnet caused Bug #369711. After networkmanager-0.9.2.0-r2, ifnet can read both the old baselayout style (with bash arrays) and openrc style (without bash arrays), but it only writes to /etc/conf.d/net with openrc style.

Now that I think of it, I really regret I didn’t make systemd ebuild install it to /usr from the very beginning. But the harm has been done already, and I’d like to move it ASAP and that’s why I’d like to sum up problems with that and possible ways of proceeding with it.

The idea

The idea is simple as it is: move systemd install to /usr prefix completely. Right now, there are no technical benefits from keeping it in rootfs. It already depends on libdbus, which is installed in /usr, and I expect more dependencies over time. There’s no reason to move all those packages into rootfs.

Most importantly, the above information allows me to assume that such move won’t hurt our split-/usr users — because they already had to have /usr mounted for systemd to run.

The trouble

The main problem with the move is that unit files were installed into /lib location by a number of packages. The files can be moved into the new location cleanly only through rebuilding the packages which provide it. They need to stay in search path for systemd to work.

These unit files are symlinked from within /etc/systemd as well. Whenever we move a single unit, we need to update the symlink as well. I’d really like to avoid forcing users to manually fix that, and the eclass doesn’t export pkg_postinst() which could help doing it automatically.

The last problem is that people have the systemd location hardcoded into the kernel command-line. This one should be relatively easy to avoid as we can keep compatibility symlink for some time.

Solution 1: one big move

The simplest solution for the migration is to move all the relevant units in the systemd ebuild. This way, the unit integrity can be preserved, and symlinks could be updated at once as well. The risk of system breakage should be reduced to minimum.

However, there is an important disadvantage of that method. All those files would be moved out-of-scope of the Package Manager. Thus, after rebuild of every single package providing systemd units, all of our users will have to fight file collisions.

The same will likely apply to our new users, because they will have at least some units installed by random packages already. Users not ever intending to use systemd won’t be hurt because the move of unit files will be transparent to them as any other package file move.

Solution 2: temporary support for two locations

Right now, systemd already supports multiple locations for unit files. As a temporary hack, we could just add /lib/systemd/system to that list. This way, all unit files still installed in /lib will still work as expected when systemd is installed into /usr.

Sadly, this won’t handle updating /etc symlinks. I could, however, fix that easily by adding a simple .path unit or another solution updating symlinks as soon as files are removed from /lib.

Other solutions?

Well, I think the second one is the best we can do. Do you have any other ideas? I guess that udev could face a similar problem if we decide to actually move it into /usr. And there the thing is even worse because the rule install location is usually hardcoded into the ebuild or package build system itself; we will probably need to have even more degree of compatibility.

Figure 1.1: Heating your boxes since 1999

Gentoo Linux is proud to announce the availability of a new LiveDVD to celebrate the continued collaboration between Gentoo users and developers. The LiveDVD features a superb list of packages, some of which are listed below.

A special thanks to the Gentoo Infrastructure Team. Their hard work behind the scenes provide the resources, services and technology necessary to support the Gentoo Linux project.

• Packages included in this release: Linux Kernel 3.1.5, Xorg 1.10.4, KDE 4.7.4, Gnome 3.2.1 XFCE 4.8, Fluxbox 1.3.2 Firefox 9.0, LibreOffice 3.4.99.2, Gimp 2.6.11, Blender 2.60, Amarok 2.5 , VLC 1.1.13, Chromium 16.0 and much more ...
• If you want to see if your package is included we have generated both the x86 package list, and amd64 package list. You may also be interested in checking out the FAQ or the updated artwork plus dvd cases and covers for the 12.0 release.
• Special Features:
  • Writable file systems using AUFS so you can emerge new packages!
  • Persistence for $HOME is available; press F9 for more info!

The LiveDVD is available in two flavors: a hybrid x86/x86_64 version, and an x86_64 multi lib version. The livedvd-x86-amd64-32ul-12.0 version will work on 32-bit x86 or 64-bit x86_64. If your CPU architecture is x86, then boot with the default gentoo kernel. If your arch is amd64, boot with the gentoo64 kernel. This means you can boot a 64-bit kernel and install a customized 64-bit user land while using the provided 32-bit user land. The livedvd-amd64-multilib-12.0 version is for x86_64 only.

If you are ready to check it out, let our bouncer direct you to the closest x86 image or amd64 image file. If you would prefer to take it easy on the bandwidth and help the community, torrents are also provided.

If you need support or have any questions, please visit the discussion thread on our forum.

Thank you for your continued support,
Gentoo Linux Developers, the Gentoo Foundation, and the Gentoo-Ten Project.

Hi all,

Two weeks ago I got a Beaglebone board from the people at Beagleboard.org to create the documentation I always create with every device I get.

Like always i’d like to announce the guide for installing Gentoo in the Beaglebone. Have a look at: http://dev.gentoo.org/~armin76/arm/beaglebone/install.xml . Feel free to send any corrections my way.

The Beaglebone is a bit different from the devices i got lately, as it lacks video out(well, there’s video out with an LCD connector but i lack an LCD screen), and its pretty simple apart from that. One of the big points of the Beaglebone is the price(89$) and the ability of creating addon boards for it, which are already famous amongst the Beagleboard people.

The specs of the Beaglebone are:
# ARMv7-A 500MHz(USB power)/720MHz(PSU power) TI AM3358/9 ARM Cortex-A8 processor
# 256MB DDR2 RAM
# SMSC LAN8710 Ethernet card
#
# 1x microSDHC slot
# 1x USB 2.0 Type-A port
# 1x mini-USB 2.0 OTG port
# 1x RJ45
#
# Reset and user-defined button

Yes, the processor runs at 500MHz when being powered using the mini-USB port, and 720MHz when using a power supply. More info about the specs in Beaglebone’s webpage.

For those curious as me, here’s the bootlog and the cpuinfo.

All the hardware provided by the Beaglebone works fine, except the USB port. But thats due to a bug in the USB driver used on the Beaglebone. The bug appears when you disconnect the first USB device you connect. Once that happens, the USB port won’t recognize any new USB device.

The workaround is to do:
echo F > /proc/driver/musb_hdrc.1

After that, the USB port will work again. This idea comes from the Angstrom people. In the guide i’ve documented an udev rule to workaround this issue, its from Angstrom as well.

I’d like to thank the people at Beagleboard.org for providing me a Beaglebone to document this. Next step is getting everything upstream

Have fun!

Thanks to Luca Barbato (known as lu_zero in Gentooland), I’ll be able to put my hands for some days on this nice ARM board made by Genesi, the Efika MX.

As it happened with the BeagleBone, the idea is to work out the kernel, bootloader integration and create weekly automatic builds of Gentoo and Sabayon images (those you can just cat to the MMC card). It is more or less what happens with Gentoo stage3 autobuilds, but here we have kernel binaries and u-boot, so that the image can boot straight away.

So far, qemu-user (statically compiled) + OpenSUSE patches, besides minor bugs, are working quite well. Depending on the availability of these boards on the market and project financial capabilities, we’ll be able to couple qemu-user with native hardware on the build server making them work side by side without the need of a shared binhost repo (sshfs + chroot magic, I’ll talk about it in future).

One more step for these chroots will be attaching matter to it, as we already do for i686 and x86_64 ones. This way the compilation of new ebuilds (and 70% of chroot work) will be completely automated.

One week ago, the BeagleBone I ordered from Tigal.com eventually landed on my desk. As you may know, it’s an ARMv7a OMAP device with 256Mb RAM, USB 2.0 and FastEthernet 10/100. The board doesn’t come with HDMI output but daughter boards are expected to be shipped soon it seems.

It ships with Angstrom Linux, an embedded distribution that is no way close to Gentoo, at least in my opinion. I found it kinda broken and slow. opkg, the package manager, is a nightmare. I had no choice, that thing had to be dumped ASAP. And that’s what I did.

First thing I did was copycating the Angstrom kernel configuration by copying /proc/config.gz in a safe place and starting to merge the Beagleboard kernel tree into my Linux 3.1 branch. This of course means that the AM335x is not yet supported by the vanilla kernel. Last I heard is that there are plans to merge the patches in the 3.3 merge window… You can find two ebuilds in the “sabayon-distro” overlay: sys-kernel/beagleboard-sources and sys-kernel/linux-beagleboard, providing sources and binaries respectively.
Introducing ARM support in sabayon-kernel.eclass (the eclass that builds kernel binaries using genkernel) was quite straightforward, it now builds uImages directly!

The boot strategy works like this: u-boot.img searches /boot/uImage into the root filesystem (ext4 doesn’t seem to work with my image). In our case, /boot/uImage is a symlink pointing to a versioned file (the one installed by sys-kernel/linux-beagleboard). You can manage the symlink using eselect-uimage, from the “sabayon” overlay and shipped with the Sabayon images already. This means that you can change the boot kernel at runtime without even touching the boot partition!

The second thing was setting up a chroot, both Entropy build chroot (for pushing out binary packages to the armv7l repo) and “image” chroot (the one from where images are generated) using qemu-user to emulate armv7l. In order to be able to prepare disk images using loop devices, I also completely rewrote the famous “mkcard.txt” script, dropping bc dependency (hey, bash can do math already!!). You can find it here, as well as molecules that we use to build the ARMv7a images for Beagle{Bone,Board}.

If you are interested in knowing more about how I managed to get Sabayon on ARM, have a look at the “Hitchhikers Guide to the BeagleBone” on our wiki.

I uploaded the Sabayon ARMv7 images on our mirrors this morning, under the iso/daily directory, in different sizes (depending on your MMC card size): 4GB, 8GB, 16GB.

a75fd2a7d9cae17762034c5c049a08fc Sabayon_Linux_DAILY_armv7a_Base_16GB.img.xz

0e17081050fa19c7f769318e3235ebaa Sabayon_Linux_DAILY_armv7a_Base_4GB.img.xz

56606bc906715cdebce02611ae03285c Sabayon_Linux_DAILY_armv7a_Base_8GB.img.xz

Installing them onto your MMC card is as easy as running:

xzcat <image file>.xz > /dev/sdX

Where /dev/sdX is your memory card device (might be mmcblk0).
They come in different sizes, make sure to match the advertised image size with your MMC device one.

If you have 32GB or 64GB MMCs you have two choices: either use the 16GB version and create a separate partition later or take the bootfs and rootfs images (they come in different sizes but the content is the same) from the same dir:

93960b28cde8dde51f2d29cc0c76f6bb Sabayon_Linux_DAILY_armv7a_Base_16GB.img.bootfs.tar.xz

e85fe8d344dacf79eec94562a59c6750 Sabayon_Linux_DAILY_armv7a_Base_16GB.img.rootfs.tar.xz

93960b28cde8dde51f2d29cc0c76f6bb Sabayon_Linux_DAILY_armv7a_Base_4GB.img.bootfs.tar.xz

1a6bce6f585d52f2b50806bd2bd69578 Sabayon_Linux_DAILY_armv7a_Base_4GB.img.rootfs.tar.xz

93960b28cde8dde51f2d29cc0c76f6bb Sabayon_Linux_DAILY_armv7a_Base_8GB.img.bootfs.tar.xz

13b2a47a88c55c8692ce61fc2fd42022 Sabayon_Linux_DAILY_armv7a_Base_8GB.img.rootfs.tar.xz

This way you can create your own partition layout and then unpack the content into the respective partitions. So easy. No grub nor MBR nightmare!
If you are as lazy as me, here is the download link to the directory (using the GARR mirror). But you are encouraged to use our download page to find the mirror closest to you.

The root password is: root. The OS is set to automatically boot and start eth0 and sshd (so you can connect to it via ssh). During the first boot, there is a script that configures some stuff and reboots your device automatically (so on the first boot it will just reboot). If you want to change the locale, edit /etc/env.d/02locale and run locale-gen !. The System is already configured to allow serial login (at least this works on the BeagleBone out of the box).

That’s it. You have a great distro (Gentoo) with a great Package Manager, Entropy (Sabayon) in a credit-card sized device.
If you appreciate our efforts towards the ARM architecture, please consider to donate us either hardware or MONEY to buy it! (yeah, we can’t just handle money, we always need money!).
If you’re a developer and interested in ARM stuff, why don’t you join us? We can improve both Gentoo and Sabayon together!

Please note: I only have a BeagleBone for now thus I wasn’t able to test out the images on the BeagleBoard. Moreover, the boot partition contains Beagle* related boot binaries that won’t work on other OMAP devices out of the box (but still, we provide split boot and root filesystem images).

Please note 2: I consider this a tech preview because at the time of writing, we only have 400 binary packages available for install. You can browse them using http://packages.sabayon.org.

DONATE US !

Today I finally closed bug 203706, 4 years after it was initially opened. That means that you can now go and install ruby 1.9.3 without having to unmask the packages and associated use flag. It also means that we'll try to add the ruby19 flag, when possible, to new ebuilds as we add them. With ruby 1.8 no longer in maintenance starting June 2012 we can now work towards making ruby 1.9 the primary stable ruby implementation. At this point in time, though, we don't officially support running your whole system on ruby 1.9 yet.

Unless you take action yourself nothing will change for the moment. If you want to start installing your ruby packages also for ruby 1.9, then you must add ruby19 to your RUBY_TARGETS in /etc/make.conf.

  RUBY_TARGETS="ruby18 ruby19"

Note that this will only work in the testing tree for the moment. If you find bugs, please report them or drop by in #gentoo-ruby.

I have been playing with the Gentoo Wiki the last few days and am very impressed with the work that both the wiki teams as well as existing contributors have already done to the place. The look and feel is very slick and editing works just as expected. One of the changes I made was to move SELinux module information to the wiki. This documentation was originally intended to be published on the Gentoo SELinux Project page, but is easily accessible and maintainable on the wiki too.

So I went a step further and dug up my original GLEP 0051 – Gentoo Knowledge Base proposal and checked how far I could use the Gentoo WiKi for this purpose. From the looks of it, the WiKi can offer a great deal of leverage for this idea and although not everything is supported through the WiKi (like natural search language and such), that might have been overshooting a bit. So we received a Gentoo WiKi Knowledge Base namespace under which the Knowledgebase entries can reside.

Now what is the idea behind such a knowledge base? Well, first of all, the articles below this prefix should all follow the same structure (as explained in the main page) and be sufficiently specific so that the title of the entry should leave little room for misinterpretation. But other than that, there is no limit as to what the Knowledge Base could hold. To that respect, the knowledge base section then provides a (hopefully) thorough listing of common and less common issues with a good explanation why the problem occurred and how to resolve it.

For the time being, the location doesn’t hold that many entries yet, but I will add them as they come along. And of course, feedback is always appreciated ;-)

On a second note, I’d like to give my PoV on the wiki and its relation with the official Gentoo documentation. Unlike what might be circulating, I’m definitely not against the wiki for documentation, on the contrary. Wiki’s have proven to be a good resource for documentation, and because we can never have enough documentation writers, every method for getting more documentation is welcome. But because of its online nature, offline documentation development (which I frequently do) is not possible. Also, keeping translations in sync might be a bit more challenging compared to a file-based solution with version control (otoh, I have little experience with WiKi translations so I might be wrong here).

I strongly believe that the wiki will play a big role in Gentoo’s documentation assets. Many of the documents currently managed by the GDP or the subprojects might be suited to be hosted on the WiKi, especially when those documents are too specific (and as such would require a very specific developer profile to maintain the documents). In such cases, the maintainers of those documents should be able to pick the most efficient method. But for very generic documents, this might not be an easy option.

At least the Gentoo documents now support CC-BY-SA 3.0, so we can migrate documents from the wiki to the main site, and the 2.5 version currently used the most on the main site should be forward compatible with 3.0 (if I read the legalese text well) so we might be able to migrate documents from the main site to the wiki too.

Edit: a3li created the “Knowledge Base” namespace on the wiki, so I updated the links in my post. Thanks for the work on the wiki, a3li!

One of the features supported through OVAL (and Open-SCAP) is to generate fix scripts when a test has failed. The administrator can then verify this script (of course) and then execute it to correct wrong settings. So I decided to play around with this as well and enhanced the Gentoo Security Benchmark (XCCDF source) with some fixables (like for the sysctl settings). And lo and behold: the thing works ;-)

After evaluating the XCCDF (together with the OVAL document) against my system, I had Open-SCAP generate a fix script:

# oscap xccdf generate fix --result-id OSCAP-Test-Gentoo-Default xccdf-results.xml
#!/bin/bash
# OpenSCAP fix generator output for benchmark: Gentoo Security Benchmark

# XCCDF rule: rule-sysctl-ipv4-forward
echo 0 > /proc/sys/net/ipv4/ip_forward

# generated: 2011-12-23T14:53:03+01:00
# END OF SCRIPT

Now isn’t that nice. But generating a fix script is one thing, maintaining the XCCDF and OVAL documents is a completely other picture.

One of the downsides that I talked about earlier already is that OVAL has quite an extensible language (it’s a large XML document). Although this extensibility is very flexible and powerful, when you want to add generic tests (like validating sysctl values or matching regular expressions in files) having to write over 30 lines of XML code for a single test is time-consuming at the least. So I quickly scripted something to help me maintain these settings.

The Generating OVAL documents with genoval.sh document explains this script (which is retrievable from my git repository) whose primary purpose is to transform a single line into the entire OVAL structure. With this script, I can now just say gentoo variable USE must contain ssl and it generates both the rules in the XCCDF as the OVAL statements in the OVAL document.

Okay, it’s a script, not a feature-full application, but at least it helps me (and perhaps others as well).

I’ll write another post, expanding from a single tweet of mine:

If you release code and are not interested in helping distributions package it… don’t release code. Honestly.

It might sound harsh and mean, but I mean it, and I think it’s for the best.

You might have a number of different reasons to work on opensource projects, but you can basically find that the people who write open source projects and then don’t care about distributions and other kind of integrators, tend to be those developers that only work for their own ego. I don’t mean they write bad code, sometimes they might actually be the best programmer, but their dislike for distributors is poisonous for the whole opensource environment.

Why do I say this? Well, the problem is that if you release good opensource code, whether it’s packaged by distributions or not, it might very well happen that some other developer will find and use it, that’s what you want, when you release it opensource, isn’t it? Now when that project is no longer a library or backend tool, but rather an user facing tool that the developer wants to see used, there will be requests to package it. But once that happens, you might be stuck with some backend dependency that cannot be clearly packaged. In that situation, the whole frontend can’t be packaged, and that means that the distribution’s users will end up suffering.

This is getting even worse with gems, especially when the most insightful comment in my latest post comes from an user that seems to proclaim himself a troll (or at least the nick suggests strongly so). Indeed, trying to excuse Ruby gems developers for not caring about our needs (which usually boil down to having tests around, a clear repository, the ability to rebuild documentation and tests without bringing in a whole lot of gem-building tools – jewelwer, hoe, and so on – and most importantly code that works and tests green, which is by far not obvious when dealing with gems), is just saying that you care about open source for your own feel good, rather than for the whole environment.

This gets worrisome when it comes to Ruby stuff, especially because of the ties with Rails. I guess the majority of the gems are developed for usage with Rails, which means web-facing code; if they are developed without caring about the opensource environment and their possible users, it would be silly not to expect that at least a part of them are so insecure they are likely to cause vulnerabilities in code that is actually published for the general usage.

So I think I just decided to make two personal rules, that I wish were widespread:

• if you do not plan on helping distributions to get your software packaged, don’t release it opensource;
• if you plan on making your software opensource, do not use software that distributions can’t package.