Gentoo Logo
Gentoo Logo Side
Gentoo Spaceship

Contributors:
. Alec Warner
. Alex Alexander
. Alex Legler
. Alexis Ballier
. Andreas Proschofsky
. Andrew Gaffney
. Arun Raghavan
. Ben de Groot
. Bernard Cafarelli
. Bjarke Istrup Pedersen
. Brent Baude
. Caleb Tennis
. Christian Faulhammer
. Christian Zoffoli
. Damien Krotkine
. Daniel Drake
. Daniel Gryniewicz
. Daniel Ostrow
. David Abbott
. David Shakaryan
. Davide Italiano
. Denis Dupeyron
. Diego E. Pettenò
. Donnie Berkholz
. Doug Goldstein
. Gentoo Haskell Herd
. Gentoo News
. Gilles Dartiguelongue
. Greg KH
. Gunnar Wrobel
. Gustavo Felisberto
. Hanno Böck
. Hans de Graaff
. Ioannis Aslanidis
. Jan Kundrát
. Jeffrey Gardner
. Jeremy Olexa
. Joe Peterson
. Jonathan Smith
. Jorge Manuel B. S. Vicetto
. Joseph Jezak
. Josh Saddler
. José Alberto Suárez López
. Kenneth Prugh
. Krzysiek Pawlik
. Lance Albertson
. Luca Barbato
. Luis Francisco Araujo
. Marcus Hanwell
. Mark Kowarsky
. Mark Loeser
. Markos Chandras
. Markus Ullmann
. Mart Raudsepp
. Matthias Geerdsen
. Michael Marineau
. Michal Januszewski
. Mike Doty
. Mike Pagano
. Nathan Zachary
. Ned Ludd
. Nirbheek Chauhan
. Olivier Crête
. Patrick Kursawe
. Patrick Lauer
. Patrick McLean
. Paul de Vrieze
. Peter Weller
. Petteri Räty
. Piotr Jaroszyński
. Remi Cardona
. Renat Lumpau
. Rob Cakebread
. Robert Buchholz
. Robin Johnson
. Romain Perier
. Ryan Hill
. Sebastian Pipping
. Serkan Kaba
. Shyam Mani
. Steev Klimaszewski
. Stefan Schweizer
. Steve Dibb
. Stuart Longland
. Sune Kloppenborg Jeppesen
. Sven Wegener
. Thilo Bangert
. Thomas Anderson
. Timothy Redaelli
. Tiziano Müller
. Tobias Klausmann
. Tobias Scherbaum
. Yuval Yaari
. Zack Medico
. Zhang Le

Last updated:
October 01, 2009, 23:03 UTC

Disclaimer:
Views expressed in the content published here do not necessarily represent the views of Gentoo Linux or the Gentoo Foundation.


Bugs? Comments? Suggestions? Contact us: planet@gentoo.org

Powered by:
Planet Venus

Welcome to Planet Gentoo, an aggregation of Gentoo-related weblog articles written by Gentoo developers. For a broader range of topics, you might be interested in Gentoo Universe.

October 01, 2009
Remi Cardona a.k.a. remi (homepage, stats, bugs)

Turned out that xorg-server 1.6 is pretty much ready for stabilization, as only a handful of bugs were reported over the testing period since last week, and they only concerned the stabilization list.

Without further ado, I've asked our faithful Arch Teams (pretty much all of 'em) to stabilize xorg-server 1.6 and friends. amd64 was the first one to the finish line with a stabilization done in under a day!

Gentoo is again back in business for X. Woo!

Now to all stable users, don't forget to read the upgrade guides we wrote :

Don't forget, please file bugs in bugzilla.

flagedit 0.0.8 (October 01, 2009, 11:57 UTC)

As promised, I released a new version, flagedit-0.0.8.

Nothing really exciting in here, flagedit now supports /etc/portage/*.keyword as folders (bug #241668 )

I'll move on and try to implement features suggested in the previous blog entry comments, namely :

  • metadata.xml description support from Donnie Berkholz : not sure what the idea is ?
  • set/unset/(reset) hardmask on a package from pavel : that's a great idea.
  • clean obsolete package.keywords entries : I can do that.

Back to Gentoo development (October 01, 2009, 11:56 UTC)

It's been quite a while since I've done anything for Gentoo. It's a new year, and it's a good time to start contributing again.

You know how it is : real life versus OSS contributions... Well the real life got the best of me for too long, but although things was very interesting - albeit time consuming - I miss the good old satisfaction of updating the ebuild of my small software that nobody uses :)

I'm even more motivated by the fact that I realized recently that I own some piece of software that are actually used by quite a lot of people (more than 2 is a lot for me).

Namely : Flagedit the Ultimate Console Based Use Flags Editor. Albeit a bit old, it's still used b quit a lot of people, and it really needs some love from me.

So I'm going to start committing stuff again. My motto : start small so you don't stop. So I started by updating the homepage and gave flagedit a new home .

Next on list : bug 241668 to support folder structure of portage

But hey, you fellow reader : what would you like to see improved in flagedit ?

Note : if you want to beta test upcoming versions of flagedit, drop me a mail, or comment.

booh 0.9.1 ebuild (October 01, 2009, 10:00 UTC)

It's been a long time since my last commit in the gentoo tree ! I had lost motivation and interest. But now I'm back with one simple goal for now : just maintain my ebuilds, don't get involved in any discussion, and take it easy :)

So I'm happy to celebrate my comeback with the much awaited ebuild for the new version of booh : 0.9.1.

This new version (well actually 0.9.0) brings a lot of new features and programs : album2booh booh-classifier booh-fix-whitebalance booh-gamma-correction webalbum2booh

Enjoy!

server reinstallation (October 01, 2009, 09:59 UTC)

The server I'm paying for (dedibox) crashed for unknown reason, I had to reinstall it. Luckilly enough I could boot in rescue mode and archive my important data, it helped to get everything up and running again, or at least the important bits (gentoo, apache, this blog, postgresql)

The newest addition on this server :

  • squid to act as proxy
  • backup-manager as a backup solution

backup-manager is not very powerful, but it's small, light and simple to install and configure. All you need is Perl and gettext :) It'll do until I migrate to bacula.

ssh, keychain (October 01, 2009, 09:57 UTC)

On our way back from FOSDEM, I had a quick discussion about ssh with Chris, and it motivated me to clean up all my ssh keys, passphrases, agents.

So now I use different keys for work and home, and ssh keychain on both.

Next move is to add my work identity to my home session to be able to connect directly to servers at work without having to go through my workstation at work. Without putting my home private id on my machine at work, nor copying my home public id on all servers at work. It should be possible I've heard :)

Anyway, here is briefly how I did it : ssh-keygen (dsa as main key). Then install keychain (see http://www.gentoo.org/proj/en/keychain/ and con figure it a bit. I added the following script in /etc/profile.d/keychain.sh (gentoo host), and I use the built in keychain on my mac.

#!/bin/bash
# start keychain, with the private keys to be cached
/usr/bin/keychain ~/.ssh/id_dsa
# then load the generated files
for i in ~/.keychain/*-sh*; do
echo "sourcing $i"
source $i
done

I know, I know, everybody is supposed to know everything about ssh, but I'm happy to admit that I learnt 2 or 3 things while setting up everything properly. Besides, how many of you have no passphrase on your ssh key ? :)

FastCgi and Movable Type (October 01, 2009, 09:40 UTC)

This blog now runs on apache + mod_fastcgi. I can feel the difference, especially given the fact that the hardware is not that powerful. I was curious aboud mod_fcgid, which claims to be better but compatible, but I'm not sure why exactly (I think something to do with better timing in spawning / killing cgi persistents processes).

I was willing to move to lighttpd as well, but it looks like it's more work than just installing and configuring mod_fastcgi for apache. By the way, you have to make sure that FCGI (the Perl module) is installed otherwise Movable Type will refuse to work.

September 30, 2009
Marcus Hanwell a.k.a. cryos (homepage, stats, bugs)
The Big Move and New Position at Kitware (September 30, 2009, 23:40 UTC)

On Monday 21 September we packed the majority of our belongings into the back of a Penske truck and made the 500 mile drive (in convoy - Louise, William, Dax and myself) from Pittsburgh, PA to Clifton Park, NY. Since then we have been unloading the truck, unpacking our things into our new home and doing all those things you have to do when you move house, and several things necessary when moving between states and jobs.

Me in the Penske truck before returning it

This is certainly the most rural house I have lived in since I was very young. We found a nice duplex on the outskirts of Clifton Park, it uses well water and I am the proud owner of the contents of two full propane tanks (no natural gas lines run out to the house). We also have a really nice wood fire in the living room, and I snagged the family room and am using it as a large home office! Thankfully they were able to hook up a cable Internet connection on Tuesday last week, and so I was not offline for too long.

Tomorrow is my first day with Kitware, I will be attending a training course being run by Kitware for the remainder of the week and so won't have my first day in the office until next Monday. I will be working in the scientific visualization group on projects such as ParaView, and have had lots of ideas for future Avogadro development over the last few weeks. I am very much looking forward to working in some new areas, but also to enhancing the previous research and development I have done in the area of visualization in chemistry. I am also looking forward to working on CMake.

September 29, 2009

Global Financial Derivative trading company, OSTC Poland, uses Gentoo Linux in significant sectors of its IT infrastructure. We spoke with long time Gentoo user and head of OSTC Poland's IT department, Patryk Rządziński, to learn more about how and where Gentoo is used. We discovered, as you will read in the full interview, that Gentoo, and more generally open source software, serves well in the commercial world.

Diego E. Pettenò a.k.a. flameeyes (homepage, stats, bugs)
Another C++ piece hits the dust (September 29, 2009, 11:19 UTC)

You might remember my problem with C++ especially on servers; yes that’s a post of almost two years ago. Well today I was able to go one step further, and kill another piece of C++ from at least my main system (it’s going to take a little more for it to apply to the critical systems). And that piece is nothing less than groff.

Indeed, last night we were talking in #gentoo-it about FreeBSD’s base system and the fact that, for them similarly to us, their only piece of C++ code in base system is groff; an user pointed out that they were considering switching to something else, so a Google run later I come up with the heirloom project website.

The heirloom project contains some tools ported from the OpenSolaris code base, but working fine on Linux and other OSes; indeed, they work quite well in Gentoo, after creating an ebuild for them, removed groff from profiles, and fixed the dependencies of man and zsh.

A few notes though:

  • the work is not complete yet so pleas don’t start already complaining if something doesn’t look right;
  • man is not configured out of the box; I’m currently wondering what’s the best way to do this;
  • after configuring (manually) man, you should be able to read most man pages without glitches;
  • for some reason, we currently install man pages in different encodings (for instance man’s own man page in Italian is written in latin-1); heirloom-doctools use UTF-8 by default, which is a good thing, I guess; groff does seem to have a lot of problems with UTF-8 (and man as well, since the localised Italian output often have broken encoding!);
  • groff (and man) both have special handling of Japanese for some reason, I don’t know whether the heirloom utilities are better or worse for Japanese, somebody should look into it.
  • September 28, 2009
    Diego E. Pettenò a.k.a. flameeyes (homepage, stats, bugs)
    Removing .la files, for dum^W uncertain people (September 28, 2009, 14:53 UTC)

    Since I have been still fighting with the damned .la files and I’m pretty sure that even though I have explained some use cases most of my colleagues haven’t really applied them, I decided to go with a different approach this time: graphical guides.

    Since the post about the tree size has gotten so much feedback, probably because the graphs impacted on people, this might actually prove useful.

    .la files removal flowchart

    Note: I first tried to draw the chart with Inkscape, but the connector available on its code only draws straight lines, which are unusable for stuff like this; I found no way to anchor lines to an arbitrary point of objects either, so I gave up; dia is tremendously bad to work with; kivio 2 is not in Portage nor available as binary package for either Windows or OSX; OpenOffice to the rescue, worked almost flawlessly, unfortunately I didn’t want to waste time to define customised colours so you get the bad and boring ones in the image.

    As you can see from this graph, my idea is that, at the end, every .la file is removed. Of course this is not immediate and depends on a series of factors; this graph shows at least the basic question you got to ask yourself when you have to deal with shared libraries. Please note that this does not apply the same to plugins and for that I’ll post another, different flow chart.

    • Does the package install internal libraries only? A lot of packages provide convenience libraries to share code between different executable programs (see this post for more information about it); this can be detected easily: there are no include files installed by default, the library is not in the ld path (such as /usr/lib/packagename). In this case, the .la files are not useful at all, and can be removed straight away.
    • Does the package only install shared objects? The .la files are only meaningful for static libraries that have no dependency information; if a package is not installing static libraries (.a files) it needs not the .la files.
    • Do the libraries in the package need other libraries? If the libraries are standalone, and only depend on the C library (libc.so), then there is no dependency information useful in the .la file, and can be dropped.
    • Is pkg-config the official way to link to the libraries? When using pkg-config, the dependency information is moved inside the .pc file, so the copy in the .la file is redundant, and thus unnecessary.
    • Is the package new? When adding a new package into Portage, there is no reason to keep the .la files around when the conditions shown above apply. For packages that are already in portage, the removal of .la files need to be considerate, or you’ll get the same kind of fire I got for trying to remove some (useless) .la files out of the blue. Not a situation that I like, but so is life.

    EntropyKey packaging in Gentoo (September 28, 2009, 11:30 UTC)

    As I have announced before I ordered two EntropyKey (one for Yamato so I can test it, and one for the router to put it into production). They arrived today, so I went on and packaged the ekeyd software (the daemon that takes care of feeding the entropy data to the kernel), which is now in portage as app-crypt/ekeyd.

    There are though a few notes on both the package and the packaging procedure that I’d like to write about, for posterity.

    Firstly, I really don’t see it as a really good move to make use of LUA in such a project; this is both because it looks like overkill to me, and because LUA itself isn’t really extremely standardised between distributions. In Gentoo, additionally, it’s using the wrong path for the compiled extensions, and is thus not really multilib-safe (although it’s debatable how useful multilib is getting, but that’s beside the point for now).

    Another problem is, the software not only uses the base LUA code, but also needs the luasocket extension. And not even a vanilla luasocket, because it needs Unix socket support, and that is not built by default by the source package; I had to patch the sources to force building and installing it.

    Beside these two problems, and the fact that the Makefile isn’t really extremely straightforward (and I needed to hack it a bit around to avoid -Werror and gzipping of man pages (Portage takes care of that), packaging wasn’t that much of a problem; the code seems clean and with the exception of some format warnings (reason why -Werror would have been a problem), no other problem was found (the package uses -fno-strict-aliasing though, which means that some optimisations will be discarded, too bad.

    For what concerns the use of the package, the current ebuild in Portage is good enough for use, I’m using it myself, but it has some things that are still incomplete. For instance, it currently does not check for the Linux kernel options for CDC (and contextually I should probably keep a table of kernels to warn about — Linux 2.6.31 will not work for instance, out of the box, because of a bug in the CDC driver), and the userland USB daemon lacks an init script (which I would probably make much easier to use than the actual daemon: the daemon wants to know the USB bus position of the key, since I don’t like to rely on that I’d rather make use of lsusb or some other method to get the position from the ID pair and the serial of the key itself).

    I am also pondering about moving the two daemons in /usr/libexec to avoid polluting root’s path with daemon commands (since they should only be started by the init script).

    So as you might guess, there’s going to be a r1 version probably even today, depends on how much time I have free (I have lots of stuff to do, sigh).

    Tobias Scherbaum a.k.a. dertobi123 (homepage, stats, bugs)
    Förderverein Gentoo e.V. strebt Auflösung an. (September 28, 2009, 10:20 UTC)

    (For readers on planet.g.o: This one is in German language only, but i do consider this important enough to be published on planet.g.o although.)

    Wichtig: Ich schreibe hier als Privatperson, nicht als Vorstandsmitglied des Förderverein Gentoo e.V.!

    Die am vergangenen Freitag stattgefundene Mitgliederversammlung hat sich mehrheitlich dafür ausgesprochen, den Förderverein Gentoo e.V. aufzulösen. Entgegen der Tagesordnung wurde mangels Kandidaten kein neuer Vorstand gewählt, so dass der bisherige Vorstand weiterhin geschäftsführend im Amt bleibt. Alle Mitglieder des Vereins werden in den kommenden Tagen angeschrieben und zu einer außerordentlichen Mitgliederversammlung eingeladen, die voraussichtlich am 07.11. in Bottrop stattfinden wird und in welcher über die Auflösung formal beschlossen werden kann.

    Vorweg: Auf die Gentoo-Distribution an sich hat dies keinerlei Auswirkungen, Gentoo ist nicht tot und wird weiter bestehen – auch unabhängig von einem Förderverein im deutschsprachigen Raum.

    Was ist der Förderverein Gentoo e.V.? Der Verein wurde im Herbst 2003 am Rande der Practical Linux in Gießen mit dem Ziel gegründet, die Verbreitung und Präsenz von Gentoo im deutschsprachigen Raum zu fördern und unterstützen.

    Warum will sich der Verein auflösen? Es konnte aus dem – kleinen – Kreis der bei der Mitgliederversammlung anwesenden Mitglieder kein neuer Vorstand gefunden werden. Die Mehrheit der anwesenden Mitglieder sieht keine Perspektive den Verein weiter sinnvoll zu betreiben. Es ist seit Gründung des Vereins im Jahr 2003 nicht gelungen, eine kritische Maße zu erzeugen, die sich und den Verein gegenseitig pusht und weiterbringt, aktiv am Vereinsleben teilnimmt. Realistisch betrachtet war ich in der vergangenen Zeit ein Einzelkämpfer – zu wenig für einen Verein.

    Welche Konsequenzen ergeben sich aus der Entscheidung? Die Präsenz von Gentoo auf Messen und Veranstaltungen wird darunter leiden. Auch wenn es um den Verein in der Vergangenheit ruhig war, konnte dieser dennoch über Mitgliedsbeiträge und Zuwendungen die Finanzierung von Flyern und Bannern, Vorfinanzierung von T-Shirts, etc. pp. sicherstellen. Ob sich Privatpersonen finden, die künftig nicht unerhebliche Beträge vorfinanzieren oder für Flyer etc. aufwenden wollen und können, wird sich zeigen müssen. Insgesamt gehe ich davon aus, dass sich die Präsenz auf Veranstaltungen zunächst nicht großartig verändern wird – langfristig mag ich keine Perspektiven abgeben. Was diese Entscheidung für mich persönlich bedeutet, wird sich auch noch zeigen müssen – meiner Motivation sind insbesondere diverse Begleitumstände der Mitgliederversammlung nicht sonderlich zuträglich. Auch fraglich ist zunächst, inwieweit es das Portal gentoo.de künftig noch geben wird, wenn kein Verein zur Finanzierung des Betrieb des Servers bereit steht. Und: Es gibt sicherlich weitere Implikationen einer Vereinsauflösung, die sich erst im Nachhinein zeigen werden.

    Schade.

    Diego E. Pettenò a.k.a. flameeyes (homepage, stats, bugs)
    The size of the Gentoo tree (September 28, 2009, 01:54 UTC)

    You might have noticed that I started working on cleaning up the tree (before I had a few problems with my system, but that’s for another day). Some people wondered whether that’s really going to make much difference, so I wanted to take a look at it myself. I was already quite sure that, while reducing the size of filesdir is important, especially to avoid more stuff to be added to the tree, getting rid of all the filesdir wouldn’t really make a terrible impact. Some extra time at hand, some find commands later, and Google Docs, lead to this:

    As you can see, the big part of the tree is ate up by the support files, more than twice the size of all the ebuilds; files/ directories are just little more than ebuilds, and there is a huge amount of filesystem allocation overhead, even if my tree is in a filesystem with 1 KiB blocks. Another interesting note is that the licenses use up more space than the whole set of profiles, scripts and eclasses!

    For the sake of finding something to work on, let’s break up the support files class into a different graph:

    So much for those who complained that adding information about packages in metadata.xml is wasting space for users… the real space waster are change logs instead! But they are useful to keep around, for a while at least. I guess what we really need is better ChangeLog integration in repoman, so that a) it updates the ChangeLog on commit (stopping developers from committing without updating them!) and b) it can delete older and obsolete entries (like, keep the most recent 40 changes or so).

    September 27, 2009
    Gentoo Ten LiveDVD Testing (September 27, 2009, 17:02 UTC)

    Attention Gentoo Community,

    In honor of Gentoo's 10th birthday, we are producing a new livedvd! We need YOU to test it on as many x86 and x86_64 machines as you can and post bugs. Please report your bugs. Feel free to entertain yourself and fellow Gentoo rock stars on our forum.

    The livedvd-x86-x86_64 will work on x86 or x86_64. If your arch is x86 boot with the default gentoo. If your arch is amd64 boot with gentoo64. The livedvd-amd64 is for well, amd64 only.

    UPDATE: Due to demand, we have moved the files onto the regular mirror infrastructure. Please select your archiecture to be redirected to a mirror: x86amd64

    So, give us a hand by testing like crazy AND posting bugs and we'll have the greatest livedvd ever.

    Thanks for your continued support,

    The Gentoo-Ten Project

    David Abbott contributed to the draft for this announcement.

    September 26, 2009
    Patrick Lauer a.k.a. bonsaikitten (homepage, stats, bugs)
    Gentoo usage in commercial environments (September 26, 2009, 12:01 UTC)

    Just sent this to the gentoo-dev@ mailinglist in the hope of getting some excellent propaganda material out of it:

    [snip]
    Hello everybody!

    As Gentoo approaches its 10th birthday I've been wondering how and where it is used. We used to have some great stories from companies in the weekly newsletter, but that one has become very dormant a while ago.

    I'd like to collect your success stories, endorsements and case studies so we can present to the rest of the world how using Gentoo makes your life easier and is totally awesome. If you don't want to have that information public I'll gladly anonymize it as long as I can be reasonably certain that you really exist. What is important is that you, if you actively use it in a commercial environment, write me whatever you think is important. Or you motivate someone you know to write it. Do your contribution to making things better :)

    Everything from "I use it and it's great!" to a story starting on a rainy day in November 1885 is good. Don't be afraid, I'll work with you on making it into something readable.
    And if you have specific criticism I'll take that too - maybe we can find an easy way to improve things. That is in your best interest too, so go ahead. Invest a few minutes of your time so we can save you more time!
    [snip]

    If you think you can contribute something just send me an email and I'll see what I can do.

    September 25, 2009
    Diego E. Pettenò a.k.a. flameeyes (homepage, stats, bugs)

    So I got a new video card, since the old one was quite fuzzy (compiz went haywire repeatedly, or froze, or emacs decided to ovrewrite the window instead of clearing it before scrolling, the Tyan logo at boot up had the wrong colours, …). Since finding r500 cards on the market is near-impossible, I got an HD4350; I got one that would support 1920×1440 resolution, which seems to be available on some pretty high-end 24" monitors, which hopefully one day I’ll get (the 20" feels small lately).

    Now, since this also has an r700-series CPU, I had to fiddle quite a bit to get it to work, and the results are, sincerely, not extremely nice.

    First stop: trying the experimental radeon driver with all the extra support enabled. Using GIT versions of the driver, mesa, libdrm and the kernel itself (long story anyway), I get to the point it actually starts up after creating the modeline manually (because otherwise the monitor gets out of range… on a DVI connection). It seems to work, KMS enabled and all the stuff.. it worked fine, for a few hours, then it kernel panicked (and the panic wasn’t sent to the serial console, just the vga console!).

    Second stop: non-KMS drivers, still with all the GIT software… it works, sort-of. When compiz is enabled (and I use compiz to be more productive: desktop wall and obscured non-focus windows are definitely useful), menus and textboxes corrupts… to the point of being unusable.

    At the end, I had to install ati-drivers, and I’m now running the proprietary version of the drivers; on the upside, it works quite decently for now. I’ll stick this way until radeon can deal with this decently, which means when KMS won’t freeze my system. Maybe 2.6.23?

    September 24, 2009
    Patrick Lauer a.k.a. bonsaikitten (homepage, stats, bugs)
    Gentoo Codeswarm (September 24, 2009, 01:09 UTC)

    When I saw people do codeswarms of amarok and banshee and other small projects I thought "Hey, I can do that too!"

    Now, after some work, and lots of CPU-cycles used, I have some results to show. And they are quite ... nice? awesome? For you to decide.

    It all begins with pulling the data from cvs. Since I was in a hurry I did a naughty thing - do a checkout from anoncvs.gentoo.org, run cvs log > logfile
    That is naughty because it causes a huge load on the server. Polite persons will create a local copy of the repository and run their own cvs server.

    After ca. 50 minutes I had a logfile of 456926138 bytes, or roughly 450MB. This gets transformed into a list of events for codeswarm. The transformation itself is quite fast, but takes lots of RAM. In this case just over a gigabyte. So there I was with a 220MB XML dump to feed into the renderer ...

    The first attempt died nicely because the default of using 1GB RAM cripples the render speed. Well, actually, my first first attempt died a horrible death because some [censored] in the Java world don't check what dynamic objects they load. And this means I had to run it all in a 32bit chroot. Grate phun. Grrr :D

    Processing that amount of data looked quite reasonable for the first hour or so. Then it slowed down quite a bit, then even more. Increasing the memory helped a lot - I just let the JVM have 3.5G (it's a crippled 32bit process after all) and it effectively used ~2.5G. I think this shows the limitations of the current approach quite well.

    Still it was too slow and too dense. The amount of files overwhelmed the screen so that it was looking very ... crowded

    And that's a bit bland. I did notice that you can colour things, so obviously I split by categories. The colours are mostly random, but seem to work well. And in testing things I realized that the drawing time is negligible compared to the physics / preprocessing, so obviously I increased the resolution a bit. Hey, we live in the modern times of Full HD ... why compromise with legacy formats!

    real    2876m48.775s
    user    2897m44.342s
    sys     10m28.831s
    
    That's what "time" had to say about my little experiment. Almost 2 days of CPU-time at 2.6Ghz ... and that's with a naughty tweak. I reduced the amount of items because I realized that it was going to take Too Long (tm). So instead of representing each file ... why not represent each directory as one "thing"?
                        filename = filename.replace('files/','')
                        filename = filename.replace('/var/cvsroot/gentoo-x86/','')
                        filename = filename.replace('Attic','')
                        filename = "/".join(filename.split('/')[:-1])
    
    That's all the mods I needed - strip the repo location at the front, fold files/ and Attic/ away and cut off the filename. Ends up with "category/package" instead, which for the purposes of Gentoo is a much more useful identifier. And the rendering time went down from one frame every 30 minutes worst case to about one frame every 3 minutes worst case. Y'know, the "unmodified" version is still processing after over 118 hours CPU time ... not that nice :)

    One positive effect is that it reduces the input from over 200MB to just around 100MB.
    Now after 48h CPU I have 13367 single PNGs with a total size of pretty exactly 5GB. Feeding those into mencoder generates 557.8 seconds of video with an average bitrate of 1737.2kB. The encoding takes 22 minutes walltime or 40 minutes CPU-time - in other words mencoder uses two threads properly. Good to know!

    So just to warn you, the result of my little experiment is a high-bitrate 1080p video. It is about 116MB in size ...
    Get it here
    (US mirror)

    Enjoy!

    September 23, 2009
    Remi Cardona a.k.a. remi (homepage, stats, bugs)

    Here we go again. This time, we're going for the double sweepstakes as we're trying to stabilize both libxcb 1.4 and xorg-server 1.6!

    For once, the server upgrade shouldn't be too hard and the server upgrade guide is remarkably slim. There's been no change in input, nor HAL, nor just about anything else for most users.

    Only Intel users might have a few surprises with DRI2 and UXA. But at this point, they should be good surprises :)

    However, the libxcb upgrade is going to cause more troubles for some users. If you are/were using x11-libs/libX11 with USE="xcb", then you might have to rebuild lots of packages. This is why we've taken such a long time to unmask and stabilize libxcb 1.4. But now, we've worked hard to write a proper libxcb upgrade guide, which users are definitely going to want to read.

    I would say the libxcb guide is more important than the xorg-server upgrade guide.

    Anyhow, right now, I calling out for help among our stable users. If you've always wanted to contribute to Gentoo but didn't know how, here's your chance.

    1. Grab the stabilization list from bug #282290
    2. Append it to your /etc/portage/package.keywords
    3. run : emerge -DuNa world like always to update

    Again, don't forget to read both upgrade guides before running the update, just so you don't start panicking when portage is unable to continue the upgrade.

    As always, you're more than welcome to CC yourself in the stabilization tracker, but please DO NOT COMMENT IN THE TRACKER if you have issues, you'll just annoy everyone there. Just file a new bug and we'll look at it.

    Don't forget that most X maintainers lurk in #gentoo-desktop on FreeNode if you have any questions.

    That's all for today

    Steve Dibb a.k.a. beandog (homepage, stats, bugs)
    new pear setup (September 23, 2009, 02:59 UTC)

    I've just commited PEAR 1.8.1 to the tree (and will do 1.9.0 shortly to get us up to speed), but I wanted to let users know about a change in the way packages are installed.  Actually, it only affects the base packages.  Up until now, the PEAR-PEAR package in portage included all the necessary deps in one ebuild.  With this new version, I've split each package up into its own ebuild.

    There's a couple of reasons for this, but the most important is that it will give us flexibility to deal with changes from upstream.  For example, with 1.8.1 and above, PEAR changed it's base XML dependency to XML_Util.  That one was already in the tree, and so the new pear base system relies on that.  If the other base ones change between versions, we can focus on that.

    Another nice little change is that the base system ebuild now is just dev-php/pear.  So, "emerge pear" and you're done.

    The new versions are all currently marked as unstable across the arches.  I would appreciate any and all feedback on the change.  I'm still a bit skeptical that this is the best approach, and a bit nervous at any fallout that may occur, so please file bug reports and let me know if you have any issues.  Thanks, all. :)

    September 22, 2009
    Diego E. Pettenò a.k.a. flameeyes (homepage, stats, bugs)
    My health and my week off (September 22, 2009, 12:41 UTC)

    I took a week off from everything this week, mostly. The reason for that is to be found in my recent health problems although it might not be easily categorised as such.

    The problem is: my eyesight got a lot worse. Due to diabetes, I’m due to have eyesight problems for quite a while, and right now I’m at an historic low. I really cannot look at the monitor for more than twenty minutes, nor I can read anything at all; I cannot play with the PSP, I cannot do crosswords, I cannot read the SMS I receive (and I can only send them thanks to the qwerty keyboard in the E75).

    Luckily, it’s not impairing my general movement (yet) so I’m still going around and doing the tasks that I have to, away from the keyboard. But for all that involves using a monitor, well, I’m just off for now. This all started when I stated to take meds, so hopefully it’ll get stabilised soon. I ordered a new pair of glasses, a temporary one though, that will arrive on Saturday. With that I’ll literally be back in business. I just hope I won’t start ordering €60 glasses every other month to cover for the eyesight change…

    Anyway, thanks to Rcomian for the OpenMP book, it arrived today, I’ll be reading it and starting to apply it as soon as I’ll be able to see!

    September 21, 2009
    Robin Johnson a.k.a. robbat2 (homepage, stats, bugs)
    Visualizing Gentoo profiles (September 21, 2009, 10:31 UTC)

    To add a new USE flag, that's globally enabled for all Linux profiles, what's the minimum set of profiles that need to change? Deprecated profiles must be handled as well, for users that need to migrate still.

    I ran into this today, while working on the USE=modules changes for linux-mod.eclass.

    As an attempt to solve this, I munged up some GraphViz work to show profile inheritance, pictures as the end. Both sets have the trailing profiles "/desktop", "/developer", "/server" turned off for the 2008.0 and 10.0 releases, to cut down on the noise.

    Graphs and script for download.

    My answers as to which profiles:

    • default-linux
    • default/linux
    • base
    • embedded

    Odd observations

    • Several Prefix profiles (linux/{amd64,ia64,x86} link to 2008.0 profiles explicitly instead of the generic architecture)
    • default/linux does not bring in base. Some profiles at a glance neglect this and might not have base brought in at all.
    • "embedded" is all alone in the tree.

    Thumbnail of one graph

    Question for any skilled GraphViz users:

    If all nodes in a given subgroup/cluster have an edge going to a single destination node, is there any way to get graphviz to replace them with a single fat edge from cluster to destination node?

    Kenneth Prugh a.k.a. ken69267 (homepage, stats, bugs)
    Java with Opera 10 on Gentoo AMD64 (September 21, 2009, 04:15 UTC)

    So I recently was messing around with Opera 64bit on Gentoo and trying to get Java to work. The process was actually a lot easier than I expected. First I made sure I had the latest sun-jdk installed, which was dev-java/sun-jdk-1.6.0.16 in this case. Now to make it work with Opera I did the following:

    cd /opt/sun-jdk-1.6.0.16/jre/lib/amd64/

    Opera doesn’t seem to work by default like this, probably because it ignores the plugin and uses java directly. Because of this, you need to symlink the libjvm.so so Opera can find it:

    ln -s server/libjvm.so .

    Set opera to use /opt/sun-jdk-1.6.0.16/jre/lib/amd64/ as the java path and restart opera. All should be working now, you can check by following this test page: http://www.opera.com/media/applets/clock/

    If anyone knows a more proper way of doing this, be sure to let me know

    September 19, 2009
    Patrick Lauer a.k.a. bonsaikitten (homepage, stats, bugs)
    How not to write an app (September 19, 2009, 22:02 UTC)

    So it seems that Ciaran is upset again. Which seems to be quite normal. But he has some good advice, and I thought I'd share some with you too.

    Let's start with an easy one. Descriptive error messages. This is good:

    $ bash -x foobar.sh
    bash: foobar.sh: No such file or directory
    
    This is not:
      ... In program paludis -ip portage:                                                                                                
      ... When performing install action from command line:                                                                              
      ... When adding install target 'portage':                                                                                          
      ... When parsing user package dep spec 'portage':                                                                                  
      ... When parsing generic package dep spec 'portage':                                                                               
      ... When disambiguating package name 'portage':                                                                                    
      ... When finding all versions in some arbitrary order from packages matching */portage with filter all matches filtered through supports action install:                                                                                                                
      ... When loading entries for virtuals repository:                                                                                  
      ... When finding all versions sorted from packages matching sys-kernel/gentoo-sources with filter all matches filtered through supports action install:                                                                                                                 
      ... When generating metadata for ID 'sys-kernel/gentoo-sources-2.6.28-r5::gentoo':                                                 
      ... When running an ebuild command on 'sys-kernel/gentoo-sources-2.6.28-r5::gentoo':                                               
      ... When running ebuild command to generate metadata for 'sys-kernel/gentoo-sources-2.6.28-r5::gentoo':                            
      ... When running command 'sandbox /usr/libexec/paludis/ebuild.bash '/usr/portage/sys-kernel/gentoo-sources/gentoo-sources-2.6.28-r5.ebuild' metadata':                                                                                                                  
      ... In ebuild pipe command handler for 'LOG0qaglobal scope tr':                                                                    
      ... global scope tr                                                                                                                
    paludis@1253390790: [QA e.child.message] (same context) global scope tr                                                              
    paludis@1253390790: [QA e.child.message] (same context) global scope tr                                                              
    paludis@1253390790: [QA e.child.message] (same context) global scope tr                                                              
    paludis@1253390790: [QA e.child.message] (same context) global scope tr                                                              
    paludis@1253390790: [QA e.child.message] (same context) global scope tr                                                              
    paludis@1253390790: [QA e.child.message] (same context) global scope tr                                                              
    paludis@1253390790: [QA e.child.message] (same context) global scope tr                                                              
    paludis@1253390790: [QA e.child.message] (same context) global scope tr                                                              
    paludis@1253390790: [QA e.child.message] (same context) global scope tr                                                              
    paludis@1253390790: [QA e.child.message] (same context) global scope tr                                                              
    paludis@1253390790: [QA e.child.message] (same context) global scope tr                                                              
    paludis@1253390790: [QA e.child.message] (same context) global scope tr                                                              
    paludis@1253390790: [QA e.child.message] In thread ID '19463':
    
    That's about 1/35th of the whole output ... which is just drowning out any reasonable information. Complete output is just under 750 lines, so good luck finding any error. But I guess there's a reason for that.

    Now something really neat:
      ... In program paludis -ip portage:                                                                                                
      ... When performing install action from command line:                                                                              
      ... When adding install target 'portage':                                                                                          
      ... When parsing user package dep spec 'portage':                                                                                  
      ... When parsing generic package dep spec 'portage':                                                                               
      ... When disambiguating package name 'portage':                                                                                    
      ... Package name 'portage' is ambiguous, assuming you meant 'sys-apps/portage' (candidates were 'sys-apps/portage', 'virtual/portage')  
    
    Let me say this as clearly as possible. Never assume. If you try to guess what I might have meant to say you may do something I explicitly do not want to happen or even something which breaks things. If your app is unable to unambiguously parse my input quit.
    Failure to do so will reduce in lots of tears and you falling on sharp objects. Multiple times.

    And of course ...
    Fetch error:
      * In program paludis -ip portage:
      * When performing install action from command line:
      * When executing install task:
      * When performing pretend actions:
      * When fetching 'dev-db/sqlite-3.6.18:3::gentoo':
      * Fetch error: Fetch of 'dev-db/sqlite-3.6.18:3::gentoo' failed
    
    I don't think that means what you think it does. A pretend action that doesn't only pretend, but claims to fetch things? That better be a small oopsie in the explanation that doesn't, in any way, describe what actually happens.

    Now, Ciaran. You can try to ridicule me through your blog. You can relabel comments as my name if you don't like them (it's your blog after all - even though that's exquisitely rude) and then deny you did it (we call that lying).
    But if you have to do so do it directly. Don't try to be funny (you saw what that did in the Funtoo-uses-Internet-Explorer Situation).
    Don't label any criticism as FUD (see "Ten Ways PMS Raped your Baby").

    And if you fork PMS then do so. You have your own hosting already. Now all you need is your own mailinglist and please leave those that work on the official version alone. And don't command us around, we're not your slaves.

    Think you can do that? That would be awesome. Thanks!

    Reality vs. PMS (September 19, 2009, 21:23 UTC)

    If you ever get bored enough (which, on an epic scale from fighting with the Spartans at the Thermopylae to reading the telephone book should be just around counting the grains of rice in a 50kg-bag) and start reading PMS, you might get a few of those "Wait a minute ..." thoughts that happen when things aren't quite right.

    One of those is around page 26, where PMS defines that bash 3.0 or higher is to be used. Which actually means "it has to be bash 3.0 compatible, but may be any higher version". Or to be even more precise: Ebuilds are not allowed to use any features not in bash 3.0, but the execution environment may be any higher version if it is compatible. How could we have a straightforward text that doesn't need interpretation! The lack of flamewars! The lack of circular reasoning!

    Now if you look around on your installed machines you may notice that:

    • The oldest available ebuild is app-shells/bash-3.1_p17
    • Portage depends on >=app-shells/bash-3.2
    • Current stable on amd64 and x86 is bash4
    Which means that you can't even use bash 3.0 anymore. And if you had an install that old you'd run into many upgrade issues - portage blocking bash, bash blocking portage, portage being unable to read EAPI1 ebuilds. A big mess with no clean upgrade path ... apart from injecting binpkgs and pretending that mess never happened.
    From bash Changelog:
    *bash-3.0-r12 (05 Jul 2005)
    *bash-3.1 (10 Dec 2005)
    *bash-3.2 (12 Oct 2006)
    *bash-4.0 (21 Feb 2009)
    
    So anyone not having bash 3.2 hasn't updated since May 2007. Seriously. That's when bash-3.2_p17 went stable. (October was just when the ebuild was added, the Changelogs are a bit tricky to parse)

    So for all intensive purposes (and of course all intents and purposes) we can assume bash 3.2 installed anyway. Plus there's no older ebuild anyway, apart from the 3.1_p17 that is a leftover from a horrible upgrade path.
    Even worse, for at least a year there has been a "contamination" of the tree in the form of bash 3.2 features like the "+=" assignment. Which means that some ebuilds won't work with 3.0 anyway. Extra bonus? Many eclasses use it, which means, in short, that you need 3.2 or higher. And it has been tolerated (maybe even encouraged) for such a long time that there's no way to undo that change now.

    The obvious thing to do (fix PMS) has been ignored for quite some time. So I thought I'd send an email to the gentoo-pms mailinglist to get it sorted out. Guess what?
    The amazing one-character patch was denied.

    That's the best way to make PMS irrelevant - refuse to let it document reality.

    David Abbott a.k.a. dabbott (homepage, stats, bugs)

    jorge
    Today I am pleased to introduce to all of you, Jorge Manuel Vicetto, Gentoo Developer, KDE Lead, and also a member of, Forums, Sparc, UserRel, DevRel, and in his spare time helps run the elections. Jorge is from Terceira island in the Azores.

    If you would prefer to read the interview;
    http://www.gentoo.org/proj/en/pr/20090918-jmbsvicetto-interview.xml

    LINKS:
    Gentoo KDE Project
    http://www.gentoo.org/proj/en/desktop/kde/index.xml
    Jorge's Boxes
    http://dev.gentoo.org/~jmbsvicetto/boxes/
    Get Involved
    http://dev.gentoo.org/~jmbsvicetto/recruiting/default-recruiters.txt
    Bugzilla
    https://bugs.gentoo.org/

    irc network freenode channel #linuxcrazy

    Download

    ogg

    mp3

    September 18, 2009
    Jeremy Olexa a.k.a. darkside (homepage, stats, bugs)

    Searched high and low to find this silly little info. Finally found it here.

    
    # define a pattern for the host url finding
    # %% => % sign
    # %0 => domain name + tld
    # %1 => tld
    # %2 => domain name without tld
    # %3 => subdomain 1 name
    # %4 => subdomain 2 name
    
    # Set default vhost server location here
    evhost.path-pattern = "/www/%0/htdocs"
    
    # If we don't have a %3, default to htdocs
    $HTTP["host"] =~ "^[^.]+\.[^.]+$" {
        evhost.path-pattern = "/www/%2.%1/htdocs/"
    }
    
    # If we don't have a %4, find the subdomain
    $HTTP["host"] =~ "^[^.]+\.[^.]+\.[^.]+$" {
        evhost.path-pattern = "/www/%2.%1/subdomains/%3/"
    }
    
    # If we have a %4, find the subdomain2.subdomain1. If we have have %4+, use %4
    # anyway. If you have 4+, write a explicit rule for doc-root.
    $HTTP["host"] =~ "^.+\.[^.]+\.[^.]+\.[^.]+$" {
        evhost.path-pattern = "/www/%2.%1/subdomains/%4.%3/"
    }

    Now my only wishlist is to do something similar for var.logdir, but %[1-4] is only set up with evhost. So, do you do anything smarter than this for your server?

    Daniel Drake a.k.a. dsd (homepage, stats, bugs)
    A new OLPC deployment in Nigeria (September 18, 2009, 18:48 UTC)

    Earlier this month, I found myself embarking on a last-minute journey to a One Laptop per Child workshop in Port Harcourt, Nigeria. They are preparing for a 6000 XO laptop deployment in that region of the country. I attended to support the 5-day event, which was coordinated by Michael Tempel and Claudia Urrea.

    The laptops were donated by Rusal/Alscom, and the deployment will be primarily run by Schlumberger’s Excellence in Educational Development division. SEED already uses its local staff to run some education projects in Nigeria (and many other countries).

    We covered a range of activities including Write, Record and Paint, then spent a decent amount of time on TurtleArt and Scratch. We also handed out 75 sensors and ran some activities around the measurement of humidity, temperature and light.
    The abilities of the attendees (the vast majority of which were first-time users) grew very nicely during the workshop.

    The group consisted of about 80 teachers and 20 children. Both the size of the group and the wide age range presented some challenges. Usually, such workshops would be run only with the core teams behind the deployments (a smaller group overall), but the way that things are developing in Nigeria meant that it made more sense for us to work with a larger group at this time.

    One of the biggest stumbling blocks of the workshop was working with the Scratch activity – while a very valuable activity, its integration with the Sugar desktop environment is damagingly minimal. For example, instead of using Sugar’s simplistic “Journal” storage system, it presents regular-looking Load and Save dialogs which proved very challenging for the new users. Seeing people struggle with interfaces that we see all over the desktop computing world really reminds you just how ground-breaking and significant the simplicity of Sugar is.

    We also ran into some problems with the Paint activity regularly freezing the systems, unreliable saving/loading with USB disks, and some confusion relating to Measure’s sensors interface. We were kept on our feet at all times, meaning that I didn’t really have time to sit down and diagnose the issues in detail, but they should not be hard to reproduce at a later date. The issues were not significant enough to cause any major disruption.

    I ran a few technical sessions with a few people who will become the basis of the technical team, plus some interested volunteers and teachers. As I’ve seen in other places, this was challenging because prior experience with Linux is low and internet access is scarce, so “look up this topic on the OLPC wiki” is not an answer that everyone is able to work with, and “run this command at the terminal” requires a decent amount of explanation. Nevertheless, the enthusiasm is overwhelming (as always) and I went away feeling confident that the core team is now comfortably in a position to get things moving and pass on knowledge to a wider group.

    Our hours were cut short by the high security around us, meaning that we didn’t have much non-workshop time to interact with the attendees (or really see the country at all), but overall it was a lovely group to work with and I’m confident that we’re going to see great things come out of their efforts. I’m now back in Nepal until November.

    Diego E. Pettenò a.k.a. flameeyes (homepage, stats, bugs)
    More router improvements (September 18, 2009, 15:57 UTC)

    My router project is the idea of running Gentoo Linux as main home gateway, on an embedded-like system (not really embedded since it’s a Celeron 2.80GHz), without Portage installed in the runtime system, and without some part of the so-called system (that I still think should be reduced).

    While there are still lots of minor details I haven’t even started looking into yet, there are a few things that already started being fixed up, for instance last week I cracked down on packages that don’t set RDEPEND/DEPEND properly (hostapd was in that list and I needed it).

    Today one more little fix entered the tree that was required by my router: glibc no longer rutnime-depend on binutils; or rather it does no longer need to. Previously the nscd (name service cache daemon) init script used the strings command to find out the pid file to use for each start and stop request. Since the file does not change on disk after the build, at least 2.10.1 now checks for the pidfile at install time and then replace it in the init script. Dropping the dependency.

    Now I got to say that the router is working mostly fine, so I don’t think I’ll be tinkering with it for a while, at least until I get the entropy key and I’ll start packaging the ekeyd daemon. This is also due to the fact that I have to reduce the time employed in that to return to work and other more important Gentoo things. This does not mean I’ll abandon the idea of fixing the system set so that it can be vastly reduced.

    Hopefully I’ll be able to entangle enough between my normal Gentoo work and the router-specific work in the future. In the mean time, I’m happy to accept gifts (especially useful stuff like the testing strips — the Italian health service only pass me 50 strips every three months, which is less than one test a day) and kudos to prod me on continuing pursuing all free software work I have scheduled.

    September 17, 2009
    Diego E. Pettenò a.k.a. flameeyes (homepage, stats, bugs)

    New chapter of my router project if you don’t care to follow it you probably don’t want to read this at all.

    Libero – or Infostrada, Wind, how the heck do you want to call it today – is my provider. Like other providers in Italy, who have probably noticed their users using OpenDNS instead of the standard DNS they provide, they started providing “captive redirects” on failed urls: when you mistype an URL or you try to access an hostname that does not exist, they redirect to their own servers, using their own “search engine” (nowadays just a Google frontend!).

    This breaks quite a few assumption, included the fact that the .local domains won’t resolve in the standard DNS servers, which in turn makes nss-mdns almost unusable.

    Up to a couple of months ago, Libero only provided this service in the primary nameserver, and if you switched around primary and secondary servers, you sidestepped the issue (that was the actual advertised procedure by the Libero staff, on the public page that was linked from within the search results). Unfortunately this had other side effects, for instance the time needed for the update of records more than doubled, which was quite boring with dynamic DNS and with newly-created domains.

    Luckily, pdnsd supports blocking particular IP returned by the results to avoid the fake records created for captive redirects, and the example configuration file itself provides an example for using that with OpenDNS to avoid falling into their redirected Google host (quite evil of them in my opinion). And in particular, at the time, there was only one host used for captive redirect, so the rule was quite simple.

    Fast forwards to today, the rule have changed; first of all it seems like Libero now uses redirects on both servers (or the secondary fails so often that it always responds from the primary), and most importantly they increased the number of IPs the redirects respond from. After counting four different IPs I decided to go with something more drastic, and ended up blacklisting the whole /24 network that they belong to (which is assigned, in RIPE, to Tiscali France… which is quite strange). I’m not sure if I ended up blacklisting more than I should have; for now it blacklists just enough for me to keep on browsing the net without adverse effects that I can see, and it also no longer stop me from enjoying .local domains… and Firefox auto-search with Google when the hostname does not exist.

    For those interested, the configuration section is this one:

    server {
    label= “libero”;
    ip = 193.70.152.15, 193.70.152.25;
    proxy_only=on;
    timeout=4;
    reject = 195.210.87.131/32, 62.210.183.0/24;
    }

    The first IP (a single host) is the one that was used earlier, I keep it on the blacklist just to be on the safe side.

    September 15, 2009
    Kenneth Prugh a.k.a. ken69267 (homepage, stats, bugs)
    X11 forwarding a chroot (September 15, 2009, 19:11 UTC)

    I figured I’d blog this so I wouldn’t stop forgetting how to do this and possibly learn a “better” way to do this. I’m going to assume that vanilla x11 forwarding works outside the chroot as a starting point.

    The process is relatively simple that I use. First I ssh into my server as a user using the -Y option like so:

    ssh -Y ken@fooserver

    Then I su to root:

    su

    Now comes the part that makes the forwarding work. We need to export the magic cookie that our current session is using so that the chroot can import it and use it. We do this like so:

    xauth list > /tmp/X

    This saves the magic cookie info into a file under /tmp so that we may access it later. Now its time to chroot in:

    chroot /mnt/devchroot/ /bin/zsh

    and I generally like to set my PS1 to remind me that it’s a chroot but is not necessary:

    PS1="dev $PS1"

    Now we need to import the magic cookie after chrooting in:

    xauth add $(cat /tmp/X)

    At this point x11 forwarding should now be working in the chroot. An easy way to test is to simply launch an xterm inside the chroot. xterm&

    If the xterm appears everything is in working condition. At this point you should probably delete the /tmp/X file for security.

    Diego E. Pettenò a.k.a. flameeyes (homepage, stats, bugs)
    First startup of the router (September 15, 2009, 01:36 UTC)

    Tonight I tried for the first time the router in its official capacity as my main home gateway. It wasn’t really a good start to be honest.

    The first problem has been the noise: my mother complained that the fans were too loud, so I wanted to go with my backup plan (replacing the main CPU fan with a fanless heatsink. Unfortunately it didn’t work out at all: there’s a capacitor in the way where the heatsink should go. Minus my intervention in form of a powerdrill over the (quite expensive) copper heatsink, it will never fit; my intervention is cheduled for tomorrow.

    Sidestepped that for a moment (“Sure mom, I’ll fix it, just give me tonight to test it out!”), the next problem waiting in line was with the startup: I made a mistake in the hurry of fixing up the init scripts to actually start, and I had to take the nullmodem cable again and fix up the boot with the serial console; unfortunately I wanted to do that with my newly fixed MacBook Pro running the newly updated Snow Leopard, but the nullmodem cable had the WCH314 serial converter rather than the PL2303 (the only one I have at home that works with OS X – note to self: order some more PL2303 converters), so I had to pick up the right one again.

    _Queued up to fix tomorrow I got: a very custom init script to convert the ethers file into a dhcpd-compatible list of known clients, and a fix to the pdnsd init script so that it will create the cache directory if it doesn’t exist (otherwise, the daemon will silently fail to work which is definitely not what you want!)._

    The final problem is with the DHCP protocol and the modem itself. The modem is actually a so-called modem/router, running Linux itself, as well. Unfortunately, it seems like the way it handles the DHCP requests is not fully compatible with either dhcpcd or dhclient; the former will try to validate the provided address and then times out (failing back to ipv4ll addresses, zeroconf), and the latter tries to renew the lease every 30 seconds, without actually setting up the routes for the Internet connections.

    On the other hand, hostapd seems to work fine and seems to handle multiple clients just fine; thanks to the fact that I finally can handle this stuff just like I want it to, I created a single, open, wireless network (I live in the middle of nowhere, whoever comes near my wifi enough to connect would be in my garden!), where the authorized clients will sit in one subnet, and are allowed to talk to each other, and the unauthorized clients are left in a different subnet, able to talk between them but not to the authorized ones, and can still connect to the internet (but only passively). The latter is quite helpful so that I don’t have to register all the laptops I get to fix, or all the PSPs that connect at my home.

    For all those who thought that the whole idea was moot and that using Gentoo in such a system is too difficult: the only ebuild I had to locally overlay was file, which is now fixed in Portage and even in the stable systems; the rest worked fine with some tweaks; of course there are a few more issues (for instance a lot of packages install Perl-based scripts that are absolutely not mandatory, and I’d like for those to have a perl USE flag in the future), but the whole idea is not bogus and it woks fine. Using simply emerge --configroot and some custom configuration files, the resulting system is 164MB big, and with the due fixes to device.map even setting up grub was quite painless.

    I guess the absolute final step would be to create a Rails application to manage the router, akin to the web interface of most commercial solutions. Yes I know that dd-wrt and other opensource firmware for “classic” routers have interfaces already, but if I have to write something, I’m most certainly going for implementing it in Ruby, as silly as that might sound. And to make stuff worse, if I do, I’ll be using sudo to launch the commands, getting the password via net… okay I’m definitely overthinking something I’m most likely never going to do.

    And for those of you who know me and my mania with Star Trek names (even my cellphones are called Danube and Delta Flyer ), this one got a quite famous name: Deep Space 9. After all, it is somewhat of a base station to another quadrant of the net!

    P.S.: Here I might as well ask some help to the lazyweb; I am planning on two things that I haven’t started even implementing yet: IPv6 support for my network and QoS for the VoIP connections (I got two in this network usually, my cellphone and the DECT phone). For the former, I did request registration with SIXXS but I missed the “no free mail providers” bit and registered with the GMail address, and was thus rejected, now waiting in the queue to see if the staff can rescue my request or not; in the mean time I have no idea how to set up IPv6 properly to avoid making myself open to the world; ideas?

    For what concern QoS does anybody have some easy link that explains how to set it up? All the stuff I found skimming through seem to be trying to explain how it work more than how to make it work; and I really don’t care how it works as much as making sure that all the VoIP traffic can trump the P2P and HTTP traffic (so that if I’m downloading a new ISO of FreeBSD I can still make calls properly). Ideas?

    September 14, 2009
    Patrick Lauer a.k.a. bonsaikitten (homepage, stats, bugs)
    Javaaaaaaargh. (September 14, 2009, 22:54 UTC)

    I presume my liking of Java is known well enough ;)

    So I tried a Java app today just to see how well it works. And suddenly ... this:

    *** glibc detected *** /usr/lib/jvm/sun-jdk-1.6/bin/java: free():
    invalid pointer: 0x00000000420eef40 ***                                        
        
    ======= Backtrace: ========= 
    /lib/libc.so.6[0x3002271e46]
    /opt/sun-jdk-1.6.0.16/jre/lib/amd64/libdcpr.so[0x7f8c503d2b52]             
    /opt/sun-jdk-1.6.0.16/jre/lib/amd64/libdcpr.so[0x7f8c503dbe05]             
    /opt/sun-jdk-1.6.0.16/jre/lib/amd64/libdcpr.so(Java_sun_dc_pr_PathFiller_reset+
    0x43)[0x7f8c503d51c3]                                                      
    [0x7f8cf52a4f50]                                                                
                                                                              
    ======= Memory map: ========                                                    
                                                                               
    40000000-40009000 r-xp 00000000 09:00 85671701 /opt/sun-jdk-1.6.0.16/bin/java  
    40108000-4010b000 rwxp 00008000 09:00 85671701 /opt/sun-jdk-1.6.0.16/bin/java   
    41716000-42288000 rwxp 00000000 00:00 0        [heap]
    
    So I think "well, sun-jdk is teh shizzle, I'll try ... err ... no, that one is fetch restricted. So is this one. Err, yeah, icedtea-bin!
    Guess what. Same result. Not good.
    Now I'm thinking "A crash _in the Java core_? No wai!".
    /opt/sun-jdk-1.6.0.16/jre/lib/amd64/libzip.so(
    Java_java_util_zip_Deflater_deflateBytes+0x305)[0x7fe671a33e95]
    [0x7fe66eb3d4a2]
    
    What, a crash in libzip? That smells bad. Random code execution bad. I find that very unlikely! How on earth does this happen?
    Now, let's go bleeding edge and try to build icedtea from source. That only takes ... sigh. About one hour on a quadcore. Grumble grumble yell moan grumble.

    And? Can you guess it? Implodes the same way. This is getting really interesting!

    Hmm. Looks like all the code this app uses is pure Java. Hmm, this makes no sense. But ... wait ... what's that? "core.jar" - guessing from imports this is the Processing library. That looks like a candidate. Let's have a look at the homepage ... yes, this seems to be C++ code with a Java interface. Ok, this could mess with things. But how?
    Just injecting the .jar with the newest version doesn't change a thing. Crummy. And I do notice that their LINUX tarball has files with the .dll ending. File says:
    PE32 executable for MS Windows (DLL) (GUI) Intel 80386 32-bit
    
    Uhm. Yah. No good.
    But I can try to build it from source, yes?
    Yes. These nice people have a svn repository I can pull from. And as a Gentoo user I'm used to playing with such things.

    If you have a weak stomach you should stop reading now!
    Now, svn checkout. unzip, make, c++, I have all of those (Gentoo sei Dank!). Let's have a look at the build syste... the build... what the ...
    Ok, there's a build/ directory with linux/ and windows/ and other directories. And in those directories are, uhm, scripts. And the eclipse editor preferences? What the ... but I digress. Scripts. There's one called "make.sh". Good! Let's have a look.

    #!/bin/sh
    
    Ok, so they aren't using any bash features I hope!
      ARCH=`uname -m`
      if [ $ARCH = "i686" ]
      then
        echo Extracting JRE...
        tar --extract --file=jre.tgz --ungzip --directory=work
      else
    #    echo This is not my beautiful house.
    #    if [ $ARCH = "x86_64" ]
    #    then
    #      echo You gots the 64.
    #    fi
    
    Wat. SRSLY. I lack words to describe this. So the whole 64bit part is commented out. Why do I think that that's not a good idea? And what is "Extracting JRE..." supposed to mean? Noone sane would ... but ... omg. MAH BRAINZ. There's a FULL JRE in the svn checkout.
        echo "
    The Java bundle that is included with Processing supports only i686 by default.
    To build the code, you will need to install the Java 1.5.0_15 JDK (not a JRE,
    and not any other version), and create a symlink to the directory where it is
    installed. Create the symlink in the \"work\" directory, and named it \"java\":
    ln -s /path/to/jdk1.5.0_15 `pwd`/work/java"
        exit
    
    Some things are not meant to be seen. Like this message, which throws up many questions (and the rest of my lunch). At this point my motivation approached zero (or rather, my motivation to do useful things. I had a strong motivation to hurt some people).

    And I try to run that crashy code in my i686 chroot. Guess what. Yes!

    Now I do wonder. Who is to blame here?
    The clueless upstream that doesn't even know how to cut a release without including their editor config?
    The Java runtime that loads any crap without doing sanity checks? (Seriously ... how can you load a 32bit .so or even a .dll into your adress space without immediately exploding?)
    Or me, who still thinks there are Java apps out there that don't make you go insane?

    So, after a few hours of tracking it down I'm again reminded why I usually don't touch Java at all. The amount of stupid I've found today is enough to last a small african country for a year ...
    Oh, one small final note. I can't really blame people for not knowing platform-specific things. But there's a fine line between being a bit ignorant and being purposefully incompetent. If you don't know how to package for a platform find someone who does. Ask on IRC, or if you don't like that on mailinglists. But please, don't release broken stuff like that. It's very rude and can be devastating for the mental state of people that might use your software!

    I just ran into “No space left on device” from Portage again. In such a situation previously I ran

    # df -H

    just to find that there was space left on all mounted devices! Confused I deleted a few distfiles through

    # rm -vf /usr/portage/distfiles/*

    and found Portage to work again magically. This time I remembered a chat with rbu: Inodes can run out, too. Checking the number of free inodes I was in trouble indeed:

    # df -i -H
    Filesystem            Inodes   IUsed   IFree IUse% Mounted on
    ..
    /dev/sda7               1.3M    1.3M       1  100% /var
    ..

    That’s all I wanted to share.

    (PS: With /var on a different partition now deleting distfiles no longer helps…)

    September 13, 2009
    Diego E. Pettenò a.k.a. flameeyes (homepage, stats, bugs)
    Fixing libtool (September 13, 2009, 20:46 UTC)

    You might have noticed that Gentoo has been lacking libtool 2.2 support for quite a while; this has been quite a problem because libtool 2.2 among other things shorten the time needed for the execution of ./configure as it doesn’t unconditionally check for C++ and Fortran, and at the same time, quite a bit of packages have been using the new libtool now.

    While a lot of packages using libtools were easily worked around by removing the libtool macro files and regenerating the autotools using whatever libtool the system cam with (so that they could be marked stable even with libtool 1.5), this only worked as long as upstream didn’t decide to update the calls to libtool as they should have been doing. And more importantly, the LTDL-based packages, like PulseAudio, almost always failed to support both libtool 1.5 and 2.2, so lots moved to simply supporting 2.2. PulseAudio, well, being one of them.

    So to have new versions of PulseAudio stable, we needed the new libtool stable. Piece of cake, no? Well, no, beside the packages failing with libtool-2.2 there were a few test failures reported in the stable request bug, that went lingering for a while; as much as I like autotools, I’m not the maintainer in Gentoo for any of them, since I’m not in the base system team. But since people started complaining (somewhat rightfully) about PulseAudio 0.9.9 being ancient, and asked for a stable of a newer version, on Sunday I went to look at the test failures.

    The results, have been much simpler than I expected; especially counting in that nobody else looked at them in the time before, with the exception of a few users playing with the vanilla USE flag:

    • one test was caused by the elibtoolize patches — we don’t want those applied to libtool itself!
    • one test failed when the system lacked German locale (it tests localised messages, and uses the German locale to do that); this was already worked around upstream by skipping it whenever the German locale is unavailable, backported to the Gentoo ebuild;
    • one more test fails because… the other failed; the test that ensures that libtool works even when the maximum size of parameters is very short simply re-executes the testsuite recursively; this gives the test two error-out cases: when libtool is broken with short max parameters, and when something else is broken, not nice!
    • finally, there was another test failure, that I couldn’t reproduce, but that seemed tied to --as-needed; after some headbanging around, I noticed the way the test failed (indirect link), and indeed, this seems to be a problem with the older, stricter --as-needed behaviour; I already wrote about the softer --as-needed back in February, if you don’t know what I’m talking about;

    As it turns out, getting the testsuite proper so that it can be trusted wasn’t especially cumbersome or painful; it only took half an afternoon of mine to fix most of the evident problems, and the last one is not a stable showstopper (as much as I like --as-needed, a failure in a testsuite because of it is not a stable showstopper, although I’d like to get the test fixed).

    What can be noticed here is that once again we’re lacking people who actually go check the problems and fix them up. We need more people who can look at this kind of stuff, we need more users to insightfully note the bugs; if it wasn’t for Pacho Ramos and Dustin Polke who, in the bug looked up possible points of failures (icecream and the vanilla USE flag — the latter made a bell ring on my head about elibtoolize), then libtool 2.2 would probably be still far, far away for Gentoo stable. Thanks guys!

    September 12, 2009
    Patrick Lauer a.k.a. bonsaikitten (homepage, stats, bugs)
    Building Stuff, the Gentoo way (September 12, 2009, 18:40 UTC)

    Today's blog post shall be focussed on how to efficiently build packages. So here's the boring part: Hardware specs.

    Processor: AMD Phenom(tm) 9950 Quad-Core Processor @ 2.6Ghz
    RAM: 8GB
    /var/tmp/portage: 8GB SSD (2x4GB Compact Flash / RAID0 )
    Storage: 4-disk SATA RAID-5 (mdadm)

    Now that's quite powerful, but still you easily end up waiting some time for things to be compiled. And of course you usually don't want to test in your livesystem environment. So chroots to the rescue!
    Currently I have 4 chroots I regularly use: i686 and amd64, stable and testing. That makes testing things quite a bit easier, especially when you run into some bugs that only happen on stable systems. The setup is quite simple, and because I'm lazy I've added some small scripts that mount everything I need.
    All chroots share /usr/portage and /usr/portage/distfiles. That's a bindmount - saves lots of space and keeps them all in sync. One "special" thing is /usr/portage/packages, that's a bindmount per chroot. So I have a packages dir with amd64-stable, amd64-unstable etc. subdirectories, which makes searching quite a bit easier if I ever need a binpkg.

    The make.conf is as default as possible, but if you build lots 'o stuff you end up with a few interesting mods:

    FEATURES="buildpkg"
    NOCOLOR="true"
    PORT_LOGDIR="/var/log/portage"
    PORTAGE_ELOG_SYSTEM="save"
    PORTAGE_ELOG_CLASSES="info warn error log qa"
    USE="X gcj objc"
    VIDEO_CARDS="vga"
    

    Why would I do that? Well, first of all buildpkg. That saves a binpkg of everything built, which saves quite a bit of time. The whole logging thing is very convenient if you build things with a script like this:
    for i in `pquery --max -a 'x11-drivers/*'`; do emerge $i; done
    
    Why would I do such a thing? Well, maybe someone asks me if all x11-drivers work with the new mesa or xorg-server. And like this I just kick it off and grep the logfiles later. Obvious, eh?
    Lastly the useflags are optimized to make the toolchain capable of building exotic stuff like gnustep or java packages out of the box. Otherwise you'd soon be rebuilding gcc, which takes time and effort ... so let's add it as early as possible. And VIDEO_CARDS to make xorg "thinner" - less drivers installed, and we don't actually use the drivers anyway!

    Now if I want to test a package I usually use:
    FEATURES="test" emerge -1avk somepackage
    
    which has the advantage that it uses binpkgs if available (k), shows me what will happen and (a)sks. And -1 / --oneshot so it doesn't end up in world file, so I can get rid of everything with a simple emerge --depclean.
    Since there are so many packages where tests fail I often short-circuit and use --onlydeps to get everything installed without having to run and fail tests and only run the tests of the package I actually want to test. As you can see that's quite streamlined to get as many things compiled with as little effort as possible. To work on things I usually group one console tab in the shared PORTDIR (which in my case is a cvs checkout) with one console tab in the chroot. That way I can easily toggle between ebuild editing and compiling. Usually I only have two such tab groups open because otherwise I tend to forget things. Whenever possible I try to avoid doing two things at once in one chroot because that can cause some headaches, it is easier for me to use a different chroot for it. And setting up a new one is quite easy :)

    A short while ago I deleted ~12000 binary packages because I was getting a few inconsistencies - the gcc 4.3 to 4.4 migration cost me quite a lot in terms of build time. But now I've mostly caught up, the common packages exist as binaries for fast install and everything else would need to be compiled anyway.
    So much for the "how to compile things" part. Next week on BBC: How to make an omelette without eggs!

    September 11, 2009
    Greg KH a.k.a. gregkh (homepage, stats, bugs)
    LinuxCon 2009 tutorial (September 11, 2009, 20:48 UTC)

    Somehow I got convinced to give a tutorial at LinuxCon this year, and it was originally scheduled to be my normal "Write a Real, Working, Linux Driver" tutorial I've been giving for the past 4 years or so (which happens to be online here, if you are bored and need something to fall asleep to.)

    But that's old-hat, as people on 4 major continents have seen it before. So, to try to break up the boredom, I'm please to announce a change:


    Write and Submit Your First Kernel Patch

    This tutorial will cover the steps necessary to properly compose, describe, and submit a Linux kernel patch. It will cover the basic usage of git, and how that works with the Linux kernel development cycle. As part of the tutorial, every attendee will compose and submit a patch to the Linux kernel that will be included in the main kernel tree.

    Every attendee should have a solid grasp of the C language, and know how to build and install, a Linux kernel from scratch (if not, reading the book, Linux Kernel in a Nutshell, free online, ahead of time would be a very good idea.) The latest source tree, from the git repository, of the Linux kernel should be installed on every attendees laptop before they arrive.


    Sign up on the tutorial web site if you are going to attend so I get a clue how many people to expect. Right now I have unique material for 100 people to write new patches for, but can come up with more if needed.

    See you all at LinuxCon, should be a fun time. I'm also giving a few other talks there as well, so come and heckle.

    In 2006 Jo Vermeulen compared Bazaar and Git performance-wise. Up to today Bazaar has a bad reputation regarding speed and from the results of Jo you see at least that Git is incredibly fast, Bazaar is usable but a bit slow on the uptake in some scenarios. Jo strictly did not use any remote operations which are hard to compare, but from some own tests I do know that Git is incredibly fast there, too, while Bazaar can be really slow on the initial clone operations. The latter fact may be history now, as the new 2a repository format has been introduced with Bazaar 1.17 and enabled by default in 2.0 (both are in Portage). It gives improved speed and flexibility while using up a bit more disk space than the old formats.

    Back to what we are really looking at: The scenario. Operations chosen by Jo were:

    • Repository initialisation
    • Adding of a Linux kernel tree
    • Diffing (not possible with current Git)
    • Commit of many files (whole kernel tree)
    • Diff on empty changeset
    • Output of status information
    • Small commit

    The results were clearly in favour of Git, especially the empty diff was quite awkard for Bazaar. The same operations were done by Jordan Mantha in 2008, also including Mercurial in a follow-up.

    A clever idea by Jordan was to provide time ratios, as test environments and settings were a bit different (other kernel version, different computer system). He concluded that Bazaar got faster, so did Git, but in the end Bazaar gained more ground. So now is the time to repeat the tests with current versions of both SCM systems, the following table only has the ratios of the other two guys.

    You will note that Git is mostly faster than Bazaar (apart from the add operation), but the absolute time of Bazaar is more than sufficient (got faster since 2006 and 2008) and outweighed by its better user interface in my eyes. Ranked by used space Git (471MB) wins, followed by Bazaar (482MB) and Mercurial (554MB); the numbers are for the whole tree including working tree.

    The used versions are given in the table, all times in seconds. For the sake of completeness, Mercurial is also listed in the table and compared to the ratios Jordan reached.

    Edit: I corrected the repository sizes because Git's gc command gives a good improvement in repository size. Additionally I should add that no custom options were used, just the straight commands.

    Task

    Git 1.6.4.2

    Bazaar 2.0 RC1

    Ratio

    Ratio Git 1.5.4.3/Bzr 1.3.1

    Ratio Git 0.99.9c/Bzr 0.7pre

    Mercurial 1.3.1

    Ratio

    Ratio Git 1.5.4.3/Bzr 1.3.1/Hg 0.9.5

    Initialization

    0.008

    0.783

    0.010

    0.257

    0.100

    0.265

    1 : 97.88 : 33.13

    1: 3.88 : 1.59

    Adding files

    14.612

    6.556

    2.229

    2.930

    1.320

    1.867

    7.83 : 3.51 : 1

    5.65 : 1.92 : 1

    Committing large changes

    5.270

    52.685

    0.100

    0.410

    0.440

    41.836

    1 : 10 : 7.94

    1 : 2.41 : 1.68

    Diffing no changes

    0.110

    0.182

    0.604

    0.007

    0.00025

    0.731

    1 : 1.66 : 6.65

    1 : 138 : 3.91

    Getting repo status

    0.313

    0.818

    0.383

    0.305

    0.022

    0.630

    1 : 2.61 : 2.01

    1.14 : 3.74 : 1

    Small commit

    0.338

    1.282

    0.264

    0.044

    0.058

    1.278

    1 : 3.79 : 3.78

    1 : 22.7 : 4.82

    gentoo-dev for Gentoo users (September 11, 2009, 00:56 UTC)

    Maybe I’m the last to stumble upon this… anyway: Nabble is monitoring gentoo-dev offering a search and browse interface and two feeds for it: “new message” and “new threads”. I was thinking the “new threads” one could be a great tool to Gentoo users that are not interested in every single mail (I currently am) but still would like to keep on track of the big picture of gentoo-dev.   I would have needed just that for the Google Summer of Code mailing list.

    Patrick Lauer a.k.a. bonsaikitten (homepage, stats, bugs)
    "/usr/portage is my overlay" (September 11, 2009, 00:33 UTC)

    ... and I wish more devs would focus on getting things in the tree. The proliferation of overlays is really nice because some have quite exotic stuff, but I find it frustrating to have to use 12 overlays to get all the apps I want/need. So please, if you can, consolidate. Push things to sunrise instead of your private overlay. And if it works (even it a bit ugly) push it to our main tree. Mask it if you don't like it. But please try to avoid handing users this nice puzzle game with 12 incompatible overlays that break random other stuff ...

    September 10, 2009
    Patrick Lauer a.k.a. bonsaikitten (homepage, stats, bugs)
    Small status update (September 10, 2009, 20:31 UTC)

    I've been trying to fix stuff as good as I can. Still there are tons of trivial bugs, so if something doesn't compile I usually just leave a note on the corresponding bug and continue with something else. It's kinda rude, but that way I get more fixes per timeunit done.

    Y'all might have noticed me bumping postgresql and samba. Those were lagging behind so much, and many people use them - so I spent some time getting us up to speed there. Still lots of minor issues, but at least we have something from this decade now. Amusing thing: Just as I started bumping postgresql to 8.4 upstream reported a few security issues and bumped it. So I spent quite some time compiling postgresql 8.0 8.1 8.2 8.3 and 8.4 - 7.4 needs some more attention, but then who uses something that old ;)

    I'm getting quite frustrated with test failures. In my chroots I use FEATURES="test buildpkg" so during the initial compile the tests are run, then reuse the binpkg to save time. With gcc 4.4.1 I had so many linking inconsistencies I decided to drop my binpkgs and start "from scratch". That has slowed me down a bit because I compile more, but it also makes me find lots of seriously messed up tests. Don't even think about suggesting that for future EAPIs as default. It's a retarded idea that does not work.
    The amd64 chroot has close to 1k packages already after the gcc 4.4.1 upgrade, so I seem to build a bit of everything now. Lots of bumps, especially in dev-python. The quadcore definitely pays off for that. If anyone wants to motivate me ... I could use some more harddisks. Never enough of those ;)

    I just bumped virtualbox to 3.0.6, a day after the announcement. I wouldn't be able to do that without helpers like Alessio who feed me things. They allow me to reduce my role to pure integration testing, so I can cover more ground, they get their favourite packages fixed and everyone profits. If you want to get involved feel free to bother me, that usually has a non-null chance of getting bugs fixed. We still have about 10k open bugs, so there's always something to do. And I can only fix what I'm aware of (same goes for everyone else. File bugs. Write patches. Make it easy for us to fix stuff.)

    While I'm spending most of my time at the tip of bleeding edge these days I am aware of the lag with getting things marked stable. That's mostly a manpower issue, so if you have some spare time and want to improve things that might have the largest payoff at the moment. And don't think you're not qualified - they let n00bs like me have access to the tree, so you're qualified for stable testing.

    So what's next? Things are getting into a better shape, but there's always room for improvement. Upstreams release a constant trickle of new stuff which we have to integrate. That takes time and we need to be aware of it, so if it hasn't been bumped after a week or so feel free to file us a bug. If you have ideas how to make things better don't hold back - maybe it's an awesome and helps a lot. Help us help you!

    A worthy goal?
    One bug a day. I challenge you to either open or fix one a day - doesn't take much time, but if we get 30 people doing it for a year that'd be the amount of open bugs we have. Or 300 people for one month ... Imagine what that'd do to your Gentoo! And now stop dreaming and go fix stuff.

    Mike Pagano a.k.a. mpagano (homepage, stats, bugs)
    gentoo-sources-2.6.31 released (September 10, 2009, 01:22 UTC)

    I just committed gentoo-sources-2.6.31 to the portage tree.

    Check out Kernel Newbies for the ChangeLog.

    September 09, 2009
    Greg KH a.k.a. gregkh (homepage, stats, bugs)
    Staging tree status for the .32 kernel merge (September 09, 2009, 21:48 UTC)

    This was originally sent to the linux-kernel and driver-devel mailing lists. Based on some of the feedback I got, I figured I should post it here as well.


    Here's a summary of the state of the drivers/staging/ tree, basically what will be coming in the 2.6.32 merge, and what the status of the different drivers are so far.

    First off, drivers/staging/ is NOT a dumping ground for dead code. If no one steps up to maintain and work to get the code merged into the main portion of the kernel, the drivers will be removed.

    As proof of that, the epl (Ethernet Power Link) driver will be removed in the .32 kernel, as no one is working on it, the upstream developers never respond to my emails, and no one seems to care about it.

    The pata_rdc driver is also going to be removed, as there is a "better" one being merged through the libata tree for this hardware.

    So, taking the drivers in chunks, here's some that have had active development on for the .32 release:

    • rt* wireless drivers. Bart has done amazing work merging all of these together into something much better than they originally were. And even better, they still work! Great job Bart!

    • rtl* wireless drivers. Again, Bart has been doing great work here.

    • wlan-ng driver: a bit of work here, but this seems to be dropping off, with the loss of a test platform for the driver. Hopefully someone has a device around and can help out here.

    • comedi drivers had only a bit of work done, lots more is needed here, let's not loose the fact that this is getting closer to a mergable shape.

    • android drivers have had a bit of work done, but upstream seems to not care at all about what is going on here, as they are working to forward port their code to the 2.6.29! kernel. {sigh}. If this keeps up, the drivers will be dropped in the 2.6.32 kernel release. Note, Pavel has been adding some of the Dream hardware drivers, which are separate from the core Android drivers. I have no objection to those, but they should work to get merged to their "correct" places in the tree in another release or so.

    • w35und driver. It's slowly being worked on.

    • echo driver. This one is now in good enough shape to merge into the main kernel tree. I'll send out review patches soon for this.

    • eth131x driver. Alan Cox is working on fixing up the issues in this driver. Hopefully it will get into mergable shape soon.

    New drivers that will show up in the .32 kernel release:

    • vt66* wireless drivers. These VIA drivers are being actively worked on to get into a much better shape. Nice job.

    • new rt3090, and rtl8192e wireless drivers have been added and worked on cleaning up issues involved in them.

    • hv (Microsoft Hyper-V) drivers. Over 200 patches make up the massive cleanup effort needed to just get this code into a semi-sane kernel coding style (someone owes me a bit bottle of rum for that work!) Unfortunately the Microsoft developers seem to have disappeared, and no one is answering my emails. If they do not show back up to claim this driver soon, it will be removed in the 2.6.33 release. So sad...

    • quatech_usb2 driver. I don't know if it quite works, but someone is developing it, so I'm not complaining :)

    • VME bus drivers. Yeah! They are progressing nicely.

    • SEP and RAR drivers. Alan Cox has been working on cleaning these up a lot.

    • IIO (Industrial I/O), these are new drivers that are being actively worked on.

    • pohmelfs and dst. It seems that DST is dead, so I think I will remove it in .33. pohmelfs is under active development outside of the tree, and hopefully patches start moving in here to help out with keeping it up to date.

    • cowloop. Yes, another COW driver! :) Seriously, this does things that DM can't do, so it might be useful. The upstream developer is interested in getting this merged properly, and is working on cleaning it up.

    Drivers not being actively worked on:

    • otus This is sitting here until a "real" wireless driver will be merged through the wireless tree. Hopefully that happens soon.

    • agnx wireless driver. No one seems to care about it. If no one steps up soon, it will be removed in .33.

    • altpciechdma Upstream developers seem to have disappeared. Again, if no one steps up, it will be removed in .33

    • asus_oled This only needs minor cleanups to get merged properly into the main tree. If someone wants an easy project, this would be it.

    • at76_usb wireless driver. Again, no one working on it, it will be dropped in .33.

    • b3dfg I really do not think anyone cares about this. again, will be dropped if this is true in .33.

    • cpc-usb After the initial flurry of development, everyone seems to have run away. Was it the fact that I hadn't showered in a few days? Again, will be removed if no one comes back. And I am wearing deodorant now...

    • frontier A nice driver, again, should not be hard to get merged into the main tree, if someone wants an easy project...

    • go7007 Ugh. Unless someone steps up now to take this over, it's going to be removed in .33. There is no hardware made with this anymore, and no specs around that I know of, and the code isn't the nicest in the world.

    • heci A wonderful example of a company throwing code over the wall, watching it get rejected, and then running away as fast as possible, all the while yelling over their shoulder, "it's required on all new systems, you will love it!" We don't, it sucks, either fix it up, or I am removing it.

    • line6 Another nice driver that should be simple to get merged. Please, if you are looking for something to do, this is it.

    • me4000 and meilhaus They work on the same hardware, and they duplicate the existing COMEDI drivers. Someone thinks that custom userspace interfaces are fun and required. Turns out that being special and unique is not what to do here, use the COMEDI drivers instead. These will be removed. Heck, I'll go remove them for .32, there is no reason these should still be around, except to watch the RT guys squirm as they try to figure out the byzantine locking and build logic here (which certainly does count for something, cheap entertainment is always good.)

    • mimio Another driver that should be simple to get merged. Someone just step up to do this please, there are users of this hardware out there.

    • p9auth While it seemed like a good idea, I don't think that anyone actually uses this. It will be removed in .33 unless someone comes forward.

    • panel Another one that should be simple to merge. Anyone?

    • phison What? I thought I asked for this to be merged a while ago, sorry about that, no reason it should still be in staging anymore, it's just so small it slipped through the cracks...

    • poch A long-suffering company is enduring the slowest developers in the world here. Hopefully the code will be replaced with a UIO driver, but testing the userspace side seems to be difficult and slow. I have to give Redrapids major credit for being patient here, they are being amazing.

    • rspiusb A weird, very expensive camera, from a company that does not want to release the specs, and wants custom userspace interfaces. The code hasn't built since the 2.6.20 days. I'll go delete it now from .32, it doesn't deserve to live as no one cares about it, least of all, the original authors of the code :(

    • slicoss and sxg These are being developed by a consulting company for the main producer of the chips. Yet they seem to have disappeared half-way through the job. Odd. Hopefully they come back soon.

    • stlc45xx Another wireless driver that no one seems to care about. So sad. I guess no one will miss it when it goes away in .33.

    • udlfb Video over USB, it doesn't get anymore whacked than that. This is still being developed but the developer doesn't like to do incremental updates for some odd reason. Hopefully he pops up again with an update. But for now, it is quite workable for a number of developers.

    • usbip USB over IP. I guess if you ran video over IP to your USB device, that would be more whacked than just video over USB. This did get one big update during the .32 development cycle, hopefully the developer can come back again when they get some free time to continue working on it. Rumor has it that some major distros are starting to rely on this code, so it would be nice to get their help to get it working better...

    That should cover all of the 600+ patches in the staging tree for the .32 kernel merge, and the existing drivers/staging/ tree. If I missed anything, please let me know.

    Bernard Cafarelli a.k.a. voyageur (homepage, stats, bugs)
    neatx and chromium in portage status updates (September 09, 2009, 12:05 UTC)

    Yesterday, I finally found the bug which prevented neatx from working on my system (thanks upstream for the debugging), so in your next portage sync, you'll find net-misc/neatx-0.3.1_p43 ready for your testing! If you don't need vnc/sound/printer tunneling or load-balancing, neatx is easier to set up than freenx and works great out-of-the-box. Thanks again to Mike Auty (ikelos) for his work on the ebuild.

    Another work-in-progress for me these days is a source ebuild for chromium (open-source version of Google Chrome). A binary version (chromium-bin) has been available in portage for some time now (with amd64 support added recently), but source version ebuild had some problems. Now my current version (available in my overlay for the curious) has fixed most of them, including use of system libraries, makefiles use instead of scons, --as-needed support, ... So why is it not yet in portage? Well, for now the tarballs from upstream are not yet available, so you won't go past the fetch phase ;) These should be available soon, once available you can expect chromium to quickly land in a portage tree near you.

    By the way, if chromium crashes at startup for you (either binary or source version), they finally found the cause: you are probably using nvidia-drivers and nvidia opengl (via eselect opengl). However the libGL.so from nvidia overrides dlsym/dlopen (dynamic linker functions) with broken replacements, breaking applications relying on these functions! Chromium devs implemented a workaround, available for -bin in versions >=4.0.208.0_p25708, but expect some breakage in time-related functions. All the gory details are here: http://code.google.com/p/chromium/issues/detail?id=16800

    And now to change a bit from technical talks, I wanted to say a big "thank you" to all of you Gentoo users who spend time filing bugreports, fixing, writing or rewriting ebuilds, debugging and finding the cause for all sorts of bugs (finding that some dynamic linkers break with specific video cards for example...), in short to all of you who work to make your distro a better one! And recently, a special thanks to Bernd Lommerzheim, who helps me a lot in proftpd maintenance, up to providing an entirely new ebuild for latest version, with lots of fixes and new features.

    texmfind, finally updated (September 09, 2009, 08:12 UTC)

    Originally texmfind has been written by a Gentoo user to find out which TeX file (LaTeX, too) is available in which package so you can emerge the correct one. Unfortunately it has not been updated for TeXLive 2008 and so looses some functionality, although some entries are still valid. Now I forked the project and updated it. The ebuild 2008.1 is already available in testing. Check it out, a stabilisation should happen soon.

    September 08, 2009
    Alex Alexander a.k.a. wired (homepage, stats, bugs)

    Nokia switched their Qt git master to version 4.7 recently, creating a new branch for 4.6.

    They also created two new branches, master-stable and 4.6-stable. Commits pushed to those branches are tested in Nokia’s testing farms, ensuring that they’ll always build.

    The Gentoo Qt team provides various live Qt ebuilds in our official overlay, qting-edge [1]. These ebuilds now include x11-libs/qt-*-4.6.9999, building code from the 4.6 Qt branch, as well as a new USE flag called stable-branch, available (and enabled by default) in 4.6.9999 and 4.9999. This USE flag enables/disables the use of the stable branches, allowing you to choose between last-minute code VS tested, known to compile code.

    Either way, its bleeding edge!

    If you need help, leave a comment or visit us @ Freenode IRC, #gentoo-kde

    [1] to install qting-edge, make sure you have layman installed and configured, then run layman -a qting-edge.

    share this post: Digg del.icio.us Google Bookmarks Identi.ca Slashdot Facebook Twitter

    September 07, 2009
    Knitting and Gentoo (September 07, 2009, 19:00 UTC)

    Ravelry is a knit and crochet community. In an interview with Tim Bray of ongoing,  their site engineer Casey Forbes says: “We have 7 servers running Gentoo Linux and virtualized into a total of 13 virtual servers with Xen.” On these servers they use nginx, HAProxy, Apache, MySQL, and of course Ruby on Rails (with Passenger). And how they use it!

    We’ve got 430,000 registered users, in a month we’ll see 200,000 of those, about 135,000 in a week and about 70,000 in a day.
    We peak at 3.6 million pageviews per day. That’s registered users only (doesn’t include the very few pages that are Google accessible) and does not include the usual API calls, RSS feeds, AJAX.
    Actual requests that hit Rails per day is 10 million.
    900 new users sign up per day.
    The forums are very active with about 50,000 new posts being written each day.

    Thanks for sharing the details, it’s what keeps us developers running. At least those as vain as me.

    September 02, 2009
    g-CTAN has made...nearly (September 02, 2009, 16:47 UTC)

    Now a live ebuild for g-CTAN has been added to the tree as I now want some real life testing: Please go and emerge app-portage/g-ctan. g-CTAN is similar to g-cpan which creates an ebuild for packages from the Comprehensive TeX Archive Network (CTAN). The usage is easy, just call the --help option to learn more after you emerged it. There are two posts by me, that explain some more details about it. There are still rough edges, but please don't hesitate to report bugs either over Gentoo Bugzilla, Launchpad or email. The one thing I still would like to solve is to filter out all packages from the listing that have not been updated since the release of TeXLive.

    Alex Alexander a.k.a. wired (homepage, stats, bugs)
    kde 4.3.1 released, in gentoo (September 02, 2009, 12:42 UTC)

    UPDATE: 4.3.1 is now unmasked for amd64/x86 and the masking issue has been fixed by jmbsvicetto, so we won’t have this issue again in the future! The post below is updated to reflect on it.

    The first bugfix release for KDE 4.3, 4.3.1, is now available.

    You can read about the improvements it brings here.

    Ebuilds for KDE 4.3.1 are already available in gentoo for architectures amd64 and x86.

    To upgrade from 4.3.0 you simply need to

    emerge --sync
    emerge -avDuN world

    This should be a straightforward update for most people.

    If you’re upgrading from an older KDE 4 version (or clean-installing KDE 4 on a stable system) you should keyword-unmask KDE 4.3 by following this post’s instructions. You’ll also find some troubleshooting hints there.

    The ebuilds are hard.masked for all other architectures due to bug 280312.

    As usual, if you have any issues, feel free to leave a comment or visit us @ IRC: freenode/#gentoo-kde :)

    share this post: Digg del.icio.us Google Bookmarks Identi.ca Slashdot Facebook Twitter

    September 01, 2009
    Steve Dibb a.k.a. beandog (homepage, stats, bugs)
    three ways to install alsa drivers (September 01, 2009, 16:18 UTC)

    One thing I'm noticing a bit of confusion on in general online is what the docs or me mean when it says to install the ALSA sound card drivers as modules.  So, lemme clarify real quick. :)

    There are two *places* to get the drivers from: either in the kernel, or from the alsa-driver package.  But, when using the kernel drivers, like many other drivers there, they can either be compiled in statically or loaded as modules as the computer is booting up.  So, there are actually two ways to install the drivers as modules, which could be a bit confusing.

    So, a quick list:

    1) In-kernel drivers (statically compiled)

    2) In-kernel drivers (modules)

    3) External drivers (alsa-driver package, modules)

    The first two are the officially supported methods by the ALSA team, so I'll quickly focus on those two.  Now the, recommended way to do things is #2 -- select them as modules in the kernel and build them that way when you are setting up ALSA for the very first time.  Why?  Well, the answer is really that it gives you a lot more options.

    Let's say, for instance, that you aren't sure which driver your card requires.  So, you flip on a few that look like it's the right one, and set them to be installed as modules.  Once they are there,  you can run alsaconf, which is a part of alsa-utils.  The alsaconf program will do the detective work for you by looking at the modules that are available on the system, and the cards that you have on your box, and then load the modules and update your module list so that they will load up the next time you boot your box as well.  Pretty simple, right?  It sure is a lot faster than compiling one driver in the kernel, rebooting, testing if that works, trying a separate one, rebooting, etc.

    Another reason is that there may be some options you need to pass to your module.  This is rare, but it does happen.  If you are loading them as modules already, then it's just a simple tweak to do change the settings, again, without having to reboot and re-test everything.

    So, that's the reason we recommend you load them as modules.  It's just gonna make life a bit easier the first time around, as you are trying to determine what you have.  Once you know what driver is required, you can always go back into the kernel and compile it in statically, and be done with it.  There's no reason to keep it as a module, unless you want to.

    Finally, a quick note about the alsa-driver package.  It's often said that it is unmaintained, and the reason for that is because I, personally, am the only one who is keeping it on life support.  That is, I'm the maintainer, not the ALSA herd.  It's only in the tree as a convenience to people who need to use it for whatever reason.  Some of the reasons could be that you needed to see if the latest release from upstream is fixing some issues of yours, so you'd use the live ebuild.  Or, you may want to use an older kernel but still keep the newer version of ALSA.  Or whatever.  The problem, though, is that I don't have the technical skills to troubleshoot your issues if something goes wrong.  My solution every time is  pretty much going to shrug and say "Sorry, that sucks.  Try the live ebuild, or something else."  It's not that I don't want to help, it's that in this case, I can't.

    Anyway, that's it ... I hope that clears up a few issues.  When I have time, I'll be revising the ALSA docs.  No idea when that'll be though.  Don't hold your breath.  In the meantime, if you have issues, my recommendation is to post on the Gentoo Forums in the Multimedia forum and ask for some help, or there's always bugzilla.  Chances are you'll get a response faster on the forums, though.  Good luck, and God speed. :)