Gentoo Logo
Gentoo Logo Side
Gentoo Spaceship

Contributors:
. Alec Warner
. Alexis Ballier
. Ali Polatel
. Anant Narayanan
. Andreas Proschofsky
. Andrew Gaffney
. Ben de Groot
. Benedikt Boehm
. Bernard Cafarelli
. Bjarke Istrup Pedersen
. Brent Baude
. Caleb Tennis
. Christian Faulhammer
. Christian Zoffoli
. Damien Krotkine
. Daniel Drake
. Daniel Gryniewicz
. Daniel Ostrow
. David Shakaryan
. Davide Italiano
. Dawid Węgliński
. Diego Pettenò
. Donnie Berkholz
. Doug Goldstein
. Fernando J. Pereda
. Gentoo News
. Grant Goodyear
. Greg KH
. Gunnar Wrobel
. Gustavo Felisberto
. Hanno Böck
. Hans de Graaff
. Ioannis Aslanidis
. Jan Kundrát
. Jeffrey Gardner
. Jeremy Olexa
. Joe Peterson
. Jonathan Smith
. Jorge Manuel B. S. Vicetto
. Joseph Jezak
. Josh Saddler
. Joshua Jackson
. Joshua Nichols
. José Alberto Suárez López
. Kenneth Prugh
. Krzysiek Pawlik
. Lance Albertson
. Luca Barbato
. Luis Francisco Araujo
. Marcus Hanwell
. Marius Mauch
. Mark Kowarsky
. Mark Loeser
. Markus Ullmann
. Mart Raudsepp
. Matthias Geerdsen
. Michael Marineau
. Michal Januszewski
. Mike Doty
. Mike Pagano
. Ned Ludd
. Olivier Crête
. Patrick Kursawe
. Patrick Lauer
. Patrick McLean
. Paul de Vrieze
. Peter Weller
. Petteri Räty
. Pieter Van den Abeele
. Piotr Jaroszyński
. Remi Cardona
. Renat Lumpau
. Rob Cakebread
. Robert Buchholz
. Robin Johnson
. Ryan Hill
. Serkan Kaba
. Shyam Mani
. Steev Klimaszewski
. Stefan Schweizer
. Steve Dibb
. Stuart Longland
. Sune Kloppenborg Jeppesen
. Sven Vermeulen
. Sven Wegener
. Thilo Bangert
. Thomas Anderson
. Timothy Redaelli
. Tiziano Müller
. Tobias Klausmann
. Tobias Scherbaum
. Yuval Yaari
. Zack Medico
. Zhang Le

Last updated:
December 31, 2008, 23:06 UTC

Disclaimer:
Views expressed in the content published here do not necessarily represent the views of Gentoo Linux or the Gentoo Foundation.


Bugs? Comments? Suggestions? Contact us: planet@gentoo.org

Powered by:
Planet

Welcome to Gentoo Universe, an aggregation of weblog articles on all topics written by Gentoo developers. For a more refined aggregation of Gentoo-related topics only, you might be interested in Planet Gentoo.

December 31, 2008
Steve Dibb a.k.a. beandog (homepage, stats, bugs)
another mythfrontend (December 31, 2008, 20:39 UTC)

Steve Dibb

I’ve been poking on and off at the idea of getting a second frontend for MythTV.  I already have my Mini-ITX all setup in my living room, but since I also have a TV in my bedroom as well, I’d like to get one in there as well.  The problem is, I’m not really sure if I’d use it, so I don’t know if I want to jump into spending the $600 to $800 to buy all the hardware.

I never thought I’d really use my other one either, and it actually wasn’t until a few months after I set it all up completely that I finally started playing with it.  That’s a bit of an odd thing about me — I enjoy the challenge and the possibility of setting something up more than using it.  Practicality really does take about the third or fourth position on the list of priorities when it comes to setting something up.  Filling a void is usually number one. :)

Since this idea costs a lot of money, and it’s not something I can throw together lightly, I want to experiment a bit first, and see if I’ll ever actually use the thing before committing.  My first frontend actually came together quite by accident — I had both the motherboard and the HTPC case, but for some reason I had it in my mind that they would never work together because of the cabling or something.  One day it dawned on me that maybe I should actually try, and when I did, it came together great, and I’ve been using it ever since.

Interestingly enough, though, once I got it working, I still wasn’t interested in playing with it … something was always missing.  But since I got my main objective complete, I was happy anyway.  It wasn’t until a few weeks later that I took a closer look at what was stopping me from using it, and I realized it was all the navigation issues that I had with mythvideo, which I patched.  Now I use it on a regular basis — it’s great to have a lot of media on demand and nicely categorized and accessible.  It’s still not perfect, but at least now it’s more of a software issue than hardware.

I actually like the setup so much, in fact, that I’m thinking about cancelling my cable TV altogether and just watching stuff from my library.  That kind of reinforces the idea of getting a second frontend, though.

So, the idea I have so far is to find a cheap frontend that I can play with to see if having a second frontend is even worth all the hassle.  That’s easy enough … it’ll cost me less money, and while it won’t be elegant, I’ll at least be able to see if it’s something I’d actually use or if I’m just looking for a project to play with.

I also have some spare parts that I can put things together, so that makes things helpful.  Unfortunately, I also have the kiss of death whenever it comes to hardware, and everything I touch inexplicably stops working sooner or later for some reason.  I don’t know why, but apparently it works doubly effective when it comes to sound cards.  The onboard sound on my Mini-ITX board died, and I have no idea why.  Well, no idea other than it’s a VIA chipset.  There’s room for one PCI slot, so I put an older SoundBlaster in there, which works great.  I’d much rather have that slot open to use something else …. say, a PCI nvidia VGA card that’d put out a much better picture than the cheap VIA one, but whatever.

I have an extra desktop, which is an old ASUS amd64 motherboard with an nvidia chipset that has worked really well for me.  In fact, I think it was the first 64-bit computer I put together way back in the day.  Somehow, the onboard audio died on this one too.  I have no idea how it happened here, either.  I found a computer store nearby my house that sells used computer parts, so I went there and found an old PCI sound card for only $10.  I bought that, figuring that the older it is, the better it would be supported in Linux.  My bad luck paid me a visit again though, and I managed to find one of the few cards that ALSA never supported.  Whoops.  So now I’m back to square one when it comes to audio.

I could try buying another PCI card, or I can try and find a USB sound device instead.  I’m a bit nervous about those, as I worry that something might happen where either ALSA has issues with it or there is just some small latency that would drive me insane.  Plus, I’m factoring in even more money to support an old piece of hardware, that could instead be invested towards a newer machine which I know would work flawlessly.

I went on Craigslist to see if I could find any cheap sound cards, and instead I found a cheap computer.  I picked up a Gateway Pentium3 for $20.  It has an Intel chipset (sound, VGA) and an onboard NIC (3com).

Ah, crap, I just realized something.  I thought I’d save all this money, after all … $20 for a whole new box instead of just a sound card, but I’m gonna have to get a new PCI video card with S-Video out, which is gonna cost me another $40 anyway.  Dangit.

Oh well, maybe I can use the box for a noisemaker for New Year’s.  The hard drive sounds like someone threw a cat in the washing machine.

Leave a comment

Jeremy Olexa a.k.a. darkside (homepage, stats, bugs)
Today’s rant: What is ‘bikeshedding’ ? (December 31, 2008, 18:08 UTC)


I knew the general concept of this word but it was thrown around today so I decided to research it.

Parkinson’s Law of Triviality (also known as the bicycle shed example, and by the expression colour of the bikeshed) is C. Northcote Parkinson’s 1957 argument that organisations give disproportionate weight to trivial issues. source

Made famous in software development by Poul-Henning Kamp (FreeBSD dev)

“The really, really short answer is that you should not. The somewhat longer answer is that just because you are capable of building a bikeshed does not mean you should stop others from building one just because you do not like the color they plan to paint it. This is a metaphor indicating that you need not argue about every little feature just because you know enough to do so. Some people have commented that the amount of noise generated by a change is inversely proportional to the complexity of the change.” source

Seems to be true to me. We shouldn’t needlessly argue about trivial details.

I leave you with this..Is there a software development group in existence that does not bikeshed?:

Futile investment of time and energy in marginal technical issues, often including annoying propaganda. (as defined by Wiktionary)

I doubt it, but is it always a bad thing? Comments?

      

Josh Saddler a.k.a. nightmorph (homepage, stats, bugs)
Graphics shuffle (December 31, 2008, 08:01 UTC)

Josh Saddler

On Christmas Eve, a special present arrived from UPS: the HIS Radeon X1950 Pro I purchased on eBay. For the week prior to Christmas I removed the discrete nVidia 7600GT and ran off the integrated nVidia Geforce 8200 chip in my motherboard. Utter pain!

Drawing the screen, whether compositing was on or off, was painfully slow. Running any kind of game was out of the question. UT2004 was impossible. I managed to gain a bit of 2D speed by adding GlyphCache and InitialPixmapPlacement options to xorg.conf, but the desktop was still slow as molasses. Made using the computer quite painful. I can personally verify all the reports that nVidia's drivers for the Geforce 8000 series suck balls are quite true. The only thing I gained was being able to run the framebuffer console at 1440x900, my monitor's native resolution. The Geforce 8200 supports this framebuffer mode; the 7600GT only supports up to 1024x768. Not that it matters once Xorg is launched. Anyway, that was a miserable failure, so I was really happy when the HIS Radeon card showed up.

To be honest, I spent a few more bucks on it than I'd like. With shipping, it was about $51. But I figured this could be a tech toy for the next several months. After this fall's debacle with that HIS RadeonHD 4670, I picked up an older R500 card for half the cost, and this one is at the top of the line for its generation. It should have been an upgrade on my nVidia 7600GT even with the FOSS drivers. With all the documentation ATI has released, the developers of the FOSS driver (xf86-video-ati in my case) were able to get working 2D and 3D acceleration some months ago. So, emboldened by all the articles and forums posts over at Phoronix on the exciting things happening to the FOSS Radeon/Xorg/Mesa stack, I gave it a whirl.

The Good

1. There is indeed 3D acceleration. It's partly usable.
2. The 2D acceleration is the fastest of any chip I tried, faster than even the 7600GT with the proprietary driver. Once I switched from XAA to EXA acceleration, it was even faster!
3. Running at my monitor's native resolution at the framebuffer console is possible.
4. It was nice to be able to remove all proprietary kernel modules.
5. The whole desktop stack loads a bit faster, with less modesetting flicker.
6. 3D performance is actually better with the FOSS drivers than it is with ATI's Catalyst (fglrx) driver.

All the stuttering and lockups I'd run into with the RadeonHD 4670 card a few months ago? Yeah, I now believe those weren't hardware issues at all, but shitty, shitty fglrx driver code. I ran into the exact same thing when trying to use fglrx with the X1950 Pro. UT2004 was a constant stutter-fest. Absolutely unusable. When it comes to the proprietary vs. FOSS drivers for usability, there's no contest. FOSS wins across the board.

The Bad

1. Keywording 70 packages or so in package.keywords is a tedious chore. I was after the latest X-server, Radeon, and Mesa updates, which necessitated moving to ~arch for most of the required X packages.
2. I can't switch virtual terminals. The monitor shuts off if it's running on anything but VT7 once X is loaded. Apparently I'm not the only one to experience this issue with this card.
3. Poor 3D performance. I had to turn down all settings to minimums in UT2004, though I kept the resolution maxed. And even with all the minimizing, framerates grew pretty choppy throughout the game. Though the R500 performance has come a long way in the Radeon driver, it's still nowhere near the level offered by my 7600GT and the proprietary nVidia driver. I dunno if the RadeonHD driver would offer any improvement; it shares a large part of its codebase with the Radeon driver.
4. The "gotchas" involved with switching from the proprietary nVidia driver to anything else. If you switch from one proprietary driver to an open-source driver, or a proprietary (nVidia) to proprietary (ATI), you'll have to manually delete a few libGL files, as the symlinks get shattered in a way that eselect doesn't know how to handle. Let's hope that bug gets fixed soon.

The Ugly

1. The fan. I think the video card's fan may have been damaged in transit. I took out the card after just a week because the noise from the fan was so damned annoying. Now, it's not that it's particularly loud; it's not all that much louder than the system fans (which are pretty quiet even when at max). No, what really grated on me was the hideous noise character of this fan. I've asked for some help from folks in the know, so we'll see where this goes. Too bad too; it uses the same IceQ cooler that the 4670 uses, and the 4670's cooler was amazing. I couldn't hardly hear it no matter the load on the card. It had a smooth, pleasant noise character, blending right in with my system fans at low RPMs.
2. Running into the ALSA and OpenAL updates at the same time I was trying to upgrade my hardware and its drivers. ALSA cannot compile. Bug still not fixed.

The new OpenAL version seems to be from a different upstream, one who has no idea what he's doing as far as documentation goes. I had a working config file for .0.0.8, and 1.5.304 broke it. There's nothing but an extremely sparse sample config to suggest what to do. No matter what I put in the new .ini-style config file, I couldn't make it pick up my microphone. When it finally did seem to be able to identify plughw:0,0, it then petulantly died with the message that the requested buffer size was too large. Based on IRC logs I found via Google, upstream suggests that's ALSA's fault, not OpenAL. Whatever, man. All I know is that the previous versions of OpenAL have always worked regardless of my ALSA version. The new one doesn't. So I added 1.5 to package.mask and downgraded. Presto, working microphone. Just the thing for the upcoming UT2004 tourney.
3. Spending $50 just a week before ATI releases the long-awaited R600/R700 programming documentation. Yeah, I'm kicking myself a bit. I'm wishing that I still had that HIS RadeonHD 4670, something that should have better performance than even an X1950 Pro, no matter which drivers are used. But as it is, the FOSS driver devs don't really expect to get a working driver with any kind of OpenGL acceleration for another few months. Approximate feature parity with the R500's current driver codebase is expected in another 8 months or so. So it'd be a long wait, but one that I'm starting to think is worth it.
4. Did I mention the fan? I can't stick that thing back in the case until I've found a cheap solution to silencing the beast. It's not worth pouring a lot of money into it. I mean, if money is being tossed about, I may as well pick up a silent low- to mid-range 4000 card off NewEgg (again).

* * *

The X1950 Pro is currently stored in the closet. I'll dig it out again once I find some solutions to various bugs. Or when more 3D performance improvements are merged. I'd like to use it for UT2004 as well as general desktop work, but I need better 3D performance. And I need to fix that fan! Maybe I can find a decently priced Arctic Cooling Accelero S1 rev 2 someplace.

I'm also really looking forward to the coming KMS and GEM support for R500 cards, hopefully that will all be merged into the 2.6.29 kernel. Just a few more months . . .

Leave a comment

December 30, 2008
Kenneth Prugh a.k.a. ken69267 (homepage, stats, bugs)
The 2.6.28 kernel (December 30, 2008, 21:10 UTC)

So I finally got around to updating my kernel on my thinkpad to 2.6.28, the latest release from kernel.org. I’ve seen nothing but good things so far.

  • My media keys work again! Sound and Brightness keys are now actually working properly with Gnome. I don’t know if it is an exact result from the new kernel, but either way I am happy.

  • I can still suspend. I usually screw up my kernels each release so I can’t suspend to ram or wake properly. WiFi seems the same, was hoping for some improvements but I can’t see any for now.

I still don’t see the added benefits of GEM yet. Perhaps some of my stuff is not upgraded to a new enough version to use GEM. I assume this because glxgears still says

Failed to initialize TTM buffer manager. Falling back to classic.

Meh, I guess I can wait awhile longer for my Intel graphics to stop sucking massively. I just wanna play starcraft but it craps out on “Blah Unsupported visual” even though an earlier X/mesa/driver combo worked.

I’m also pondering the venture into using EXT4 on my / partition. I don’t know how to proceed though. Does one run tune2fs from a livecd or can one do it on the running system? I assume livecd is required as one will need to fsck before mounting EXT4.

I plan to try EXT4 following this advice. Any comments or suggestions before I take to the leap to EXT4 would be appreciated.

Diego Pettenò a.k.a. flameeyes (homepage, stats, bugs)
Avoid LDPATH pollution (December 30, 2008, 13:35 UTC)

Diego Pettenò

There is one thing I noticed working on my linking collision script. While most of the software properly creates subdirectories to put plugins in, so that they don’t clash with others and it does not pollute the LDPATH space, there are quite a few packages that don’t do that at all and install their plugins straight into the libdir.

Not only that, some packages install static archive versions of their own plugins, with no good reason since they are never linked in statically, but always dlopen’d.

Please don’t pollute LDPATH, if you can, make sure the plugins are installed in “pkglibdir” (that is /usr/lib/packagename) and make sure that they only install the shared object file, and eventually the libtool archive if the software uses libltdl to load them. The static archive is almost always unneeded and just a waste of time.

Also please remember that if you install core libraries in a path outside of the standard libdir (which is very good if the libraries are not to be linked against!), you should probably make sure that there is a proper runpath in the executables. What runpath does is to tell the linker to look for libraries in a path that is otherwise not accessible through the standard configuration files (/etc/ld.so.conf). A common mistake here is to install an env file that makes LDPATH (or even worse LD_LIBRARY_PATH) to the directory where the core internal libraries are installed.

While this works, it does not make much difference than having it in the standard library path: both the runtime linker and the link editor will use the path from the configuration file anyway, so the library is not going to be hidden like you’d want it to for a private library. On the other hand, if just the executable provides their own runpath, then the two linkers will ignore the libraries altogether.

So please, be careful with what you push in the library path, okay?

December 29, 2008
Marcus Hanwell a.k.a. cryos (homepage, stats, bugs)
I am Going to Camp KDE! (December 29, 2008, 21:16 UTC)

Marcus Hanwell

I just wanted to let anyone who has not spotted it already know that I am going to Camp KDE in January! I am really looking forward to it and hope to be able to talk with lots of developers and users. I will be giving half of the talk about KDE and Distros. Specifically Gentoo, but also adding my viewpoint as a downstream consumer of KDE.

I am not officially talking about Avogadro, Kalzium and all the work I have been doing there. Unfortunately there was not enough time in the schedule to fit that in too. I would be very happy to talk to people about it, and will have slides, videos and screenshots with me. So, if you are interested and we can find a corner of a room let me know!

I am registered, have flights and a room booked. So much to do - I did all that weeks ago but only just found time to announce it.

Leave a comment

Git and Automatic ChangeLog Generation (December 29, 2008, 12:10 UTC)

Marcus Hanwell

After our move to use Git and GitHub to host our repository I got thinking about ChangeLogs. Having a version controlled file, where you manually add details about what the version control system should be recording seems like it should not be necessary. I searched and couldn't find a solution that generates ChangeLogs in the style we prefer, which is a variant of the GNU ChangeLog.

So I wrote a quick Python script to try and accomplish this task. It may not be the prettiest Python code as I have never written more than five or six lines of Python before. ChangeLogs always seemed to be the biggest source of merge conflicts whenever we would work in branches, or just all be working at the same time. This is why I think it is necessary to automatically generate something like this that can be generated with source tarballs.

I called it gitlog2changelog.py and it has all of the basics down already. It may not be the most general script but works pretty well for us. I need to add some extra parsing for file creation/deletion so that we can add the + or - in front of the file names. Is there a general need for this? Are there better scripts out there that I dd not spot?

2008-12-29  Tim Vandermeersch <email@protected>

  ∗ libavogadro/src/pluginmanager.cpp: replace getenv(...) with
  QProcess::systemEnvironment()

  ∗ libavogadro/src/elementtranslate.h: Replace "A_DECL_EXPORT extern ..." with
  "A_EXPORT extern ..."

  ∗ CMakeLists.txt: use /MD compiler flag for MSVC

2008-12-28  Marcus D. Hanwell <email@protected>

  ∗ libavogadro/src/molecule.cpp, libavogadro/src/molecule.h,
  libavogadro/src/python/molecule.cpp: Lots of documentation updates,
  reorganised the functions and grouped in Doxygen tags. Some minor changes
  too, more are needed for const correctness.

  ∗ testfiles/multicubes.cube.gz: Removed from our source as it is the same
  size as all the other files put together. May be we should provide a more
  extensive sample of files in a separate distribution.

  ∗ libavogadro/src/engines/bsdyengine.cpp: Ported to use the new bond position
  functions.

  ∗ libavogadro/src/bond.cpp, libavogadro/src/bond.h: Added functions to
  retrieve bond positions, still need to implement the mid-point function.

  ∗ libavogadro/src/bond.h: Documentation updates.

  ∗ libavogadro/src/python/bond.cpp: Added missing Atom include.

  ∗ libavogadro/src/atom.cpp, libavogadro/src/atom.h: Documentation updates,
  added member function groupings and a destructor.

  ∗ libavogadro/src/atom.cpp, libavogadro/src/bond.cpp, libavogadro/src/bond.h:
  Added some atom accessor functions to the Bond class. This should make using
  bonds easier. Fixed assignment order in Atom constructor.

Leave a comment

Zhang Le a.k.a. r0bertz (homepage, stats, bugs)

Zhang Le Actually I was intended to send this to LKML, but just before I was about to send I found there was already some discussions about this: http://marc.info/?l=linux-sparc&m=122956478603612&w=2. So instead I decided to just post it here.

To make long story short, please take a look at these two files first:

http://sourceware.org/cgi-bin/cvsweb.cgi/libc/string/endian.h?rev=1.10&content-type=text/x-cvsweb-markup&cvsroot=glibc
This file will become part of /usr/include/endian.h.

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=include/linux/byteorder.h;h=29f002d73d989c577f0e79a8100d6fb7e0abb188;hb=bc2aa80e18a1b43ea2b8066500006b729c4ba4a7
This file will be /usr/include/linux/byteorder.h

Obviously, this test in linux/byteorder.h will fail:

#if defined(__LITTLE_ENDIAN) && defined(__BIG_ENDIAN)
# error Fix asm/byteorder.h to define one endianness
#endif
Because linux/types.h will include sys/types.h, which will in turn include endian.h, and in endian.h both of them are defined, but as a value not a flag.

It seems that there is an inconsistency on handling __LITTLE_ENDIAN/__BIG_ENDIAN between Linux and glibc. Linux treats them like a flag, while glibc treats them like a value. glibc uses __BYTE_ORDER to determine the endianness.

This problem must be solved, or any userland program including kernel headers which include {asm,linux}/byteorder.h will fail to build, e.g. glibc.

It seems to me that the most straight forward way, but also very intrusive way is to make the handling of these two marco consistent in the two projects. That'll be a very large changeset. Also it will suddenly change a convention in a project that has already been followed for a long time. So I don't think people will accept this.

How this problem will be solved is still remained to be seen.

EDIT: someone reported that it worked well on ~amd64. So I modified the title and removed the last sentence. I will see if this problem still exists on my x86 notebook.

EDIT 2: I confirm that this problem does not exists on x86. I will find out what exactly is going on loongson, or rather mips. :)

Ryan Hill a.k.a. dirtyepic (homepage, stats, bugs)
obligatory xmas post (December 29, 2008, 01:17 UTC)

Ryan Hill Yay, exactly what everyone wants to read, another xmas post.

Too bad, i'm doing it anyways.

Xmas kind of sprung itself on me this year, or rather i wasn't paying attention as usual and December just went and ended on me. Shopping was done on the 23rd while driving home. Despite that i managed to get everything i was looking for. We had xmas at our house this year and it was pretty nice - about 20 people and way too much food. I ended up with some very nice DVD's and some much welcomed new clothes.

Altus dropped off a seriously expensive and hardcore winter parka and a nice bunnyhug ("hooded sweatshirt" for you non-Skatchewainians). I guess they're trying hard to keep me around. I've already agreed to go back to work for them this summer for an as-yet-undetermined signing bonus and a decent pay hike. I may still decide to stay with them for my co-op work term later in my fifth semester, but there are a ton of other companies that would pay much more, probably even cover my full tuition, so we'll see.

I decided to park the beater I've been driving and take the new car that I've been trying to sell the last six months back to MJ with me, and in the process of emptying it out I found my old O2 in the trunk. Just for the hell of it I decided to mess around and see if i could figure out what killed it a year ago. After it defrosted and i wiped everything down and pieced it back together, I plugged it in and to my surprise it booted! Well, kinda - for some reason it goes into init 6 as soon as it hits init 3, giving me a nice reboot loop - but the hardware itself seems fine, the disk is intact, i was able to start kde, etc. by interrupting the boot and bringing everything up manually. So if I can figure out what's screwed in the startup sequence I might be back in business.

But for now I'm going to play Persona 4, watch my new MST3k DVD's, and enjoy some time off.

Leave a comment

December 28, 2008
Gunnar Wrobel a.k.a. wrobel (homepage, stats, bugs)
layman-1.2.2 is out (December 28, 2008, 23:06 UTC)

Gunnar Wrobel The next layman version has been released and fixes a few minor bugs:

  • layman -L: better use of screen real estate for source URLs (#251032, submitted by Martin von Gagern)
  • Execute subprocesses in a shell. (#235165)
  • layman/overlays/git.py (GitOverlay.sync): app-portage/layman - 'layman -S --quiet' yields "git: 'pull-q' is not a git-command." (#247964)
Thanks to all the contributers!

Diego Pettenò a.k.a. flameeyes (homepage, stats, bugs)
Blockers abuse (December 28, 2008, 14:18 UTC)

Diego Pettenò

I’ve already written about some of the differences between my and Patrick’s tinderboxes, one of which is that my tinderbox does not only install one package, but tries to install all of them at once. This is a necessity for me to have enough material to work on with my library collisions script, and still have so many side effects that makes it funny to work with.

The first problem is that sometimes packages don’t get merged together because of file collisions, which most of the time are caused by packages that install commands with name too generic, and some other times because they regenerate files that should not be present in the final image (like iconv’s charset.alias that gets generated on Gentoo/FreeBSD systems with no good reason at all).

The second problem derives from the way the first problem is handled. When two packages install a file with the same name, rather than renaming one of them or both, it’s customary to actually “fix” the problem by adding blockers to each of the two packages so that they cannot get installed together. While it’s certainly better to have it expressed that way rather than having the merge to fail after the compilation and install phases, it’s not really a solution since it still disallows having the two packages on the same system. While this is acceptable for packages like the different GhostScript implementations that apply for the same task, this is not much of a solution when the packages are entirely independent one from the other and have very different tasks.

I also have found one particular package (pnet) which had a very funky solution to the collision between that and boehm-gc, considering that it was installing a private copy of that. Obviously this was not the proper fix by a mile’s look.

If you have two packages that block each other you have a few different ways to deal with that; if they provide the same function, you might as well install them with a prefix and then write an eselect module to choose between one or the other (which is something that ghostscript could very well be doing); if they only install executables with generic name, they might be changed to prefix the command name with the package name. But sometimes these commands are not to be used by the users at all, and are rather internal commands used by the scripts for processing; in those cases, it would be a nice idea to make those get into /usr/libexec/$PN/ so that they are taken out of the users’ path, and won’t collide one with the other.

While dealing with packages that install colliding files is not so easy, there is need for developers to deal with them in a less “works for me” way, and think more of the general picture, as it is, there are enough packages in the tree that blocks each other with no real good reason, and this is upsetting.

December 27, 2008
Diego Pettenò a.k.a. flameeyes (homepage, stats, bugs)
The mailing lists problem (December 27, 2008, 19:43 UTC)

Diego Pettenò

For a while I’ve been using GMane for almost all the mailing list I’m following. This made it much easier to deal with it because it mean that I didn’t have to download a local copy of all the messages and it also allowed me for a much cleaner access to archives. Unfortunately this has a downside, as it requires me to use an NNTP client, which is something that hasn’t been much cool to do lately.

In the last few few months I’ve used gnus as my client, but the problem with that is that it still needs a separate software from Evolution (which is my mail client0, and it had the nasty issue of not saving the read messages when emacs closed unexpectedly, say when X died, and because of a bug, emacs daemon died with it.

Now of course I could use mailing lists like most of the other people in the world do, by receiving them on my mail account, but I’d rather not fill my GMail “All Mail” folder with all the messages coming from mailing list, especially for when I have to access that data from an UMTS connection where I pay the traffic.

What other solutions are there for me? I’m considering the idea of doing what I did when there was a limit of 20MB on an email account from my provider, but no limit on the number of accounts, having one account per mailing list. Now the limit is 1GB but I haven’t been using those accounts in quite a long time. Even if they are available in IMAP when I’m on their connection they are only available through webmail from outside; not like that’s too much of an issue for me, since I need mailing list mostly when I’m not around hospitals or stuff like that, so I could just use that.

For this to work, though, I need a few tools working on IMAP, for instance I’d need a script that could expunge old archives, when the threads haven’t been updated in the last few months; I’ll also need a script that would automatically filter the mailing list as they arrive in Inbox (waiting with the IDLE command), since my provider does not allow me to filter the messages server-side. And such a script will have to run on Yamato since it has to be on their network.

Then there is the problem that Evolution saves its cache in my home directory, which is under RAID6; there is no need for the cache to be on my home, when XDG_CACHE_DIR is pointing to a private subdirectory in /var/cache. This includes the 200MB of SQLite database that is currently using. Does anybody know if there is a way to get Evolution to respect the XDG_CACHE_DIR variable?

To-Do lists, tasks, what to do (December 27, 2008, 17:45 UTC)

Diego Pettenò

These holidays really sucked, for a long series of reasons, and in general, I’m not feeling well neither emotionally nor physically. But they offered me the time to think about what I want to do. I’ve been working on my collision detection script lately and I’m now confident I can make it work as a proper tool to identify issues, and I’d love to work on fixing those issues with upstream, but the problem is I lack the time to do that.

Even if I do improve a project a day, it’s never going to be enough, because in a few months I’m going to need to do it again because code would have rot and stuff like that. I need help for this. One thing I’m going to do is working on a personal archive of autoconf macros I can use on different projects; the attributes.m4 file that comes with xine, lscube and Lennart’s projects is already a personal archive of macros under some aspects, the problem is that its history is shared among different repositories which is very nasty. I’ve avoided up to now to create a repository for that, splitting it up in different macro files (since it’s far from being just attributes checking any longer), but I think i should look for a solution for that problem rather than continuing to procrastinate that.

Today I stopped procrastinating on getting rid of JFS for my root filesystem: since kernel 2.6.28 was released, I’m now starting the long awaited conversion of my partitions to ext4, starting from the repositories (and here git wins against Mercurial big time: once the git repositories are repacked, copying them over is very very quick), while copying over openjdk, icedtea and xine repositories that use Mercurial take so much longer.

Talking about xine, I’m going to do some more work on that in the next days, mostly code cleanup if I can, but I’m also planning on setting up a Transifex instance on this server for xine (and my own projects); hopefully it’ll make it simpler to provide translated versions of xine-lib, xine-ui and gxine. As well as of my own tools that need to be translated one way or another.

There are so many things I’d have to do that I haven’t been able to in months, reading is one of those, but I’m going to preserve that for when I’m going to the hospital next month for check ups; I’m not going to bring my laptop with me this time, nor any handheld console. I’ll be around on the cellphone a bit maybe, but that’ll be it. For the rest of the time I’ll be reading and listening to music (I’m not going to leave the iPod at home, knowing hospitals it will come handy). Actually, since I just have to have a CT scan, a chest X-Ray and an MRCP (MRI), I don’t strictly need to stay in the hospital; but not having a driving license does not help; although I guess even if I had one, I’d better not be driving after they have tests on me.

I’m going to spend the new year’s eve alone at home, maybe with my mom and my sister with her husband and my nephew. On the whole, it’s not going to be holiday either, so like it happened on Christmas, I’m going to spend most of the day working on some analysis or similar. I’ve seen that some of the issues I’ve brought up lately have started being taken care of, which is very good.

I know this post sounds pretty incoherent, I guess I’m incoherent myself at the moment. Anyway if you wish to help out with anything at all, feel free to drop a line.

Stuart Longland a.k.a. redhatter (homepage, stats, bugs)
Gentoo/MIPS uClibc Stages (December 27, 2008, 03:38 UTC)

Stuart Longland

Hi all…

Well, after a long hiatus, I’m working on new stages for the Gentoo/MIPS port.  These new ones will be based on uClibc, and are a compliment to the existing glibc-based stages.  At the moment, the initial seed environment is still being cross-compiled from my i686 host, and for now I’m focussing on mipsel but will soon turn my attention to MIPS.

These stages initially will not operate with a page size of 16KB or more, so unfortunately aren’t much good for Lemote users (I’m working with the uClibc people on this) but Cobalt and SGI users will soon once again be able to use uClibc as their system libc.  The new stages will be based on uClibc 0.9.30… which is yet to be keyworded.

Leave a comment

December 26, 2008
Stuart Longland a.k.a. redhatter (homepage, stats, bugs)

Stuart Longland

Open letter to Minister Stephen Conroy regarding the proposed internet filter.

Over the last year, we’ve heard a number of people step up, complaining about this proposal, and what it will mean in terms of freedom-of-speech, and internet speeds.  I also heard a rumour that mentioned the blocking of peer-to-peer traffic.

Now, it is very noble of the government to be that concirned with the issues involving pornography and other objectionable material, that they are pushing forward with developing and introducing this filter to the masses.  Others have already pointed out many of the ethical and technical issues with the proposal, which I note, to date, are still not addressed.

The latest proposal however, has been to block peer-to-peer traffic.  I strongly urge those in the government to carefully consider the consequences before taking on such a drastic action.  Ignoring the fact that Bit-Torrent and similar protocols can, and are, used for legal purposes (such as distribution of open source software) as well as for piracy… a lot of other peer-to-peer protocols exist, that many people expect to be able to use freely.

Consider the following applications/protocols:

  • Skype
  • MSN Messenger
  • Ekiga
  • Yahoo! Messenger
  • TeamSpeak
  • EchoLink
  • IRLP
  • IRC DCC
  • VNC
  • RDP
  • SIP (most VoIP systems)
  • Hamachi
  • … etc

All of these, rely on peer-to-peer traffic to operate, and are used lawfully.  Blocking SIP could be disasterous to our telecommunications, since many people rely on this protocol as their primary home phone service! Such a move could proove highly unpopular with the voting public, and deadly to businesses that rely on VoIP services.  In a time of global economic crisis, is this really what you want?!

It is true that many of us absolutely hate it, when a politician breaks an election promise.  However, we are more than too happy to forgive politicians for breaking such promises, when such promises are mearly implementing bad policy.  I urge this government to consider the above, in addition to the comments made by others on this topic, before going ahead with such disasterous propositions.

Leave a comment

December 25, 2008
Diego Pettenò a.k.a. flameeyes (homepage, stats, bugs)
Reinventing the wheels for fun and.... (December 25, 2008, 23:41 UTC)

Diego Pettenò

So it’s Christmas day, and I’m skimming through the 32+ MB of logs generated by my elven script and I found some nice nuggets:

Symbol getopt_long@@ (32-bit UNIX System V ABI Intel 80386) present 35 times
  /usr/bin/graph
  /usr/bin/ebzipinfo
  /usr/bin/spline
  /usr/bin/double
  /usr/bin/uupick
  /usr/bin/autotrace
  /usr/bin/uustat
  /usr/sbin/ndtpd
  /usr/bin/ode
  /usr/sbin/ndtpcheck
  /usr/bin/ebrefile
  /usr/bin/uucp
  /usr/sbin/ndtpcontrol
  /usr/bin/uulog
  /usr/sbin/uuxqt
  /usr/bin/ebzip
  /usr/bin/faad
  /usr/bin/lha
  /usr/lib/libsox.so.1.0.0
  /usr/sbin/uucico
  /usr/bin/ebinfo
  /usr/sbin/uuconv
  /usr/bin/plot
  /usr/bin/tek2plot
  /usr/lib/libc.so.5
  /usr/bin/uuname
  /usr/bin/uux
  /usr/bin/ebstopcode
  /usr/bin/stklos
  /usr/bin/cu
  /usr/bin/plotfont
  /usr/bin/ebfont
  /usr/bin/ebunzip
  /usr/bin/pic2plot
  /usr/sbin/uuchk
Symbol strncasecmp@@ (32-bit UNIX System V ABI Intel 80386) present 5 times
  /usr/bin/xpilots
  /usr/bin/gargoyle-agility
  /usr/lib/libc.so.5
  /usr/bin/xedit
  /usr/bin/xpilot
Symbol strnlen@@ (32-bit UNIX System V ABI Intel 80386) present 5 times
  /usr/bin/tarsync
  /usr/lib/libCw.so.1.0.0
  /usr/lib/libmba.so.0.9.1
  /usr/lib/python2.5/site-packages/numarray/_chararray.so
  /usr/bin/linksys-tftp
Symbol stricmp@@ (32-bit UNIX System V ABI Intel 80386) present 5 times
  /usr/bin/qemacs
  /usr/lib/libraidutil.so.0.0.0
  /usr/lib/openbabel/2.2.0/inchiformat.so
  /usr/lib/libxerces-c-3.0.so
  /usr/lib/libIL.so.1.0.0

Now if you ignore the references to the old compatibility libc.so.5 you can still find that there are quite a few programs that reinvent the wheel, reimplementing some functions that the C library already provides. Now, this would be fine and dandy if the implementation was subtly different, or was done with some particular purpose in mind, like glib’s functions, but I really can’t find the reason for the situation above to happen.

These are usually smaller functions, but still there is no reason for them to be present since anyway it’s more than likely that most of their use is replaced away by the compiler itself; more interesting is their presence in shared objects, since that would interpose around other calls, although this most likely won’t happen on glibc based systems since they provide versioned symbols which wouldn’t then interpose by default. That’s still a problem for *BSD systems though.

This has a much lower priority in my list than identifying all libraries bundled by various packages (even when they cannot be fixed, because they are proprietary or something else), because these are unlikely to turn out being security issues. On the other hand, the idea of doing such pointless duplication of common functions might as well caus security issues to be hidden, for instance if a package was to reimplement mktemp, then it would most likely be a problem.

Anyway for those interested to find out what’s duplicating code in their system, the new hit parade, derived directly from the Bug shows SQLite3 trying to climb up the ladder, together with boehm-gc. Why people can’t understand that there are libraries in the system already?

Zhang Le a.k.a. r0bertz (homepage, stats, bugs)

Zhang Le I got a donation of a Lemote Loongson 2F box somewhere around July this year and have been working on it in my spare time since I got it.

The other day I made a summary about what I have done so far and posted it on Lemote's bbs. The links is http://www.lemote.com/bbs/viewthread.php?tid=20134
My work involves toolchain, kernel, xorg-server MIPS or Loongson support and userland library/application gcc 4.4 patch.

The most prominent achievement so far is an N32 ABI stage3 (well, actually just a tarball, not made using catalyst) optimized for Loongson 2F with MIPS PLT support. It is actually not that easy as you would've imagined, N32 has many problems as you can see from the above posted link. I posted it on Lemote's bbs along with some instructions of how to use it: http://www.lemote.com/bbs/viewthread.php?tid=20125

According to some testers, performance of some applications in this system has an up to 30% increase comparing with the performance of the same apps in system Using O32 ABI without optimization for Loongson 2F and MIPS PLT support.

I have just got laid off by my former employer (however no need to worry for me, new jobs will find me soon), that means currently I have a lot of time doing things I'd like to do, like Loongson. Now I am still working on the problems I have encountered so far, currently xulrunner for N32 ABI, which will lead ultimately to firefox on MIPS N32 ABI userland system.
https://developer.mozilla.org/en/xptcall_FAQ
http://mxr.mozilla.org/mozilla/source/xpcom/reflect/xptcall/porting.html
I wish I could finish it before 2009 comes, ;)

Thomas Anderson a.k.a. gentoofan23 (homepage, stats, bugs)
My Christmas present to the Gentoo community (December 25, 2008, 16:06 UTC)


After testing with Markus, I’ve stabilized linux 2.6.27 for amd64. Markus has stabilized it for x86. A few notes:

1) You’ll likely need to upgrade nvidia-drivers, because we needed to stabilize new versions of that

2) openafs 1.4.8 and qc-usb-messenger went stable with it as well, so upgrade that.

3) I fixed up lirc-0.8.3-r2 so it now has compatibility with 2.6.27.

So that’s my Christmas present to Gentoo users.Have fun with it!

      

Hanno Böck a.k.a. hanno (homepage, stats, bugs)

Hanno Böck The free software projects for media playing did a good job in the past on supporting a wide variety of formats. From the common to many very obscure formats, current versions of the free software mediaplayers were usually able to play them. Today it's even common to suggest vlc for Windows users if they can't play unusual media formats.

Though there were a few exceptions, the most notable probably the long-time missing support for many of the Real formats. While these are rarely used today, many archived videos in the Internet still rely on it. For example, many german television stations provide real video files on their webpages.

Recently and without much public notion, ffmpeg first got support for RV40, some weeks later also for RV30. This fills a long time gap in free software support for video formats. ffmpeg is used by all major free software video players (vlc, xine, mplayer), so you should get the support within some time in all of them. For now, it's quite easy to checkout mplayer from subversion and build it on your own.

Want something to try out? Here's a video from Desert Planet in real format.

The only gap I know of a format that really got usage in the wild and that is not yet supported by free software is WMA3.

Leave a comment

December 24, 2008
Jeremy Olexa a.k.a. darkside (homepage, stats, bugs)
Gentoo Prefix now supports FreeMiNT (December 24, 2008, 17:40 UTC)


So..what does this mean? In a nutshell..Gentoo Prefix works on the OS that runs on an Atari. Cool, eh?

We owe this to the hard work of AlanH, as seen here and here.  As always, we are at #gentoo-prefix and the gentoo-alt mailing list if you are interested.

      

Diego Pettenò a.k.a. flameeyes (homepage, stats, bugs)
Library on a collision course! On the screen! (December 24, 2008, 14:55 UTC)

Diego Pettenò

So after almost an year, I returned working on the library collision detection that comes with ruby-elf. I now have a much more powerful system to work on, but I also have a much bigger set of samples to scan, in my tinderbox.

Beside a few issues that I’ve had to taken care of, I’ve now got a database that contains a huge amount of data to process and useful information to derive. Just to give some statistics, the script harvested 24602 ELF files, counting 2713395 symbols, of which 326399 are duplicate between objects. These statistics have already counted out all the suppressed symbols known up to now, but obviously there are more yet unknown.

To stick with statistics for now, rather than actual results, it’s interesting to know that about 1365671 C++ ABI symbols, I sincerely wonder how many of these should be hidden instead.

On a more interesting note, Samba confirms itself sub-optimal by now having yet a convenience library for its shared functions, and copying over the symbols between the various pieces of the puzzle, included six different Python modules, whose total size would probably be cut in half I guess.

Sticking with the Python side, there is one damn issue that is really upsetting me: about thirty different packages link the Python interpreter statically rather than dynamically, resulting in around 30 different copies of Python itself in the full of Portage. Nasty. The problem is that the ebuild installs the shared and static libraries in two different paths, one of which being private “config” path for Python. The packages picking that up will explicitly request it at link time, causing Python to be linked in statically rather than dynamically:

Symbol PyBaseString_Type@@ (32-bit UNIX System V ABI Intel 80386) present 30 times
  /usr/games/bin/diameter
  /usr/bin/vim
  /usr/bin/gedit
  /usr/lib/root/libPyROOT.so.5.22
  /usr/lib/python2.5/site-packages/opencv/_highgui.so
  /usr/lib/libpython2.5.so.1.0
  /usr/bin/eog
  /usr/games/lib/mcl/plugins/python.so
  /usr/sbin/bextract
  /usr/bin/zapping
  /usr/lib/python2.5/site-packages/opencv/_cv.so
  /usr/bin/gvim
  /usr/bin/gwp
  /usr/bin/cooledit
  /usr/lib/planner/plugins/libpython-plugin.so
  /usr/sbin/bacula-fd
  /usr/sbin/bacula-sd
  /usr/lib/dia/libpython_plugin.so
  /usr/lib/xchat/plugins/python.so
  /usr/lib/xchat-gnome/plugins/python.so
  /usr/sbin/bacula-dir
  /usr/lib/gnumeric/1.8.3/plugins/python-loader/python_loader.so
  /usr/games/bin/adonthell
  /usr/games/bin/kiki
  /usr/lib/libgnt.so.0.0.0
  /usr/bin/epiphany
  /usr/lib/python2.5/site-packages/_lcms.so
  /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Inline/Python/Python.so
  /usr/lib/apache2/modules/mod_wsgi.so
  /usr/lib/apache2/modules/mod_python.so

As you can see this includes two Apache modules, and quit a few pieces of GNOME. This is quite nasty. My suggestion until this is sorted out is not to enable python USE flag unless you really really really need it. The nastiest bit is that since there has been a Python vulnerability if you didn’t rebuild these packages after the bump, you’ll have them using a vulnerable interpreter, still. Do I really have to spell out how bad that is for stuff like Apache modules?

Josh Saddler a.k.a. nightmorph (homepage, stats, bugs)
December public service announcement (December 24, 2008, 07:01 UTC)

Josh Saddler

For all the forumites, bloggers, wikifolks, mailing list members, bugzilla commenters, and general Gentoo netizens:

Please stop spreading bad advice such as ebuild foo digest. It does not exist. Digests have not been present in the Portage tree for a year, ever since the tree was converted to Manifest2 format.

If you've made a local modification to an ebuild, patch, or added anything else to your local overlay and need to make Portage aware of the changes so the package can be properly merged, do not use the "digest" command. Use "manifest" instead. For example:

ebuild foo-1.0.17.ebuild manifest

Digests have been dead for a long time. Please don't encourage bad practices.

This has been a December public service announcement. Thank you for flying Gentoo Air, and please enjoy the rest of your Christmas!

Leave a comment

Marcus Hanwell a.k.a. cryos (homepage, stats, bugs)
Avogadro Has Moved to GitHub and Git (December 24, 2008, 00:15 UTC)

Marcus Hanwell

After some discussion on IRC the Avogadro Project has moved its version control over to GitHub. Our new repository can be found here. Most of us are old Subversion users, and a few of us started out with CVS. I think we are still getting our head around the workflow but feel it is worth the effort for all the advantages the move will bring.

There are some great visualisations from GitHub. I have always loved the speed of Git when compared to other version control I have used. It is also great for adding in bigger changes and not having to halt development in other areas for fear of merge issues. We shall see over the coming months how positive the move was, but I am confident it will be. I have been using Git locally through git svn for over a year now I think.

For those who might wonder, the conversion was not totally automatic. The branches and tags git-svn leaves you with are not Git branches and tags. It basically required me to manually create the branches and tags. So for our repository I ran the following,

mkdir avogadro-git
cd avogadro-git
git svn init https://avogadro.svn.sourceforge.net/svnroot/avogadro --stdlayout
git config svn.authorsfile ../avogadro/authors.txt
git svn fetch

At that point I went and grabbed a coffee, took care of the dog...

git remote add origin git@github.com:cryos/avogadro.git
git push origin master

This got us most of the way there but lacked all of our tags and branches. I found that none of the push options worked as the tags and branches needed to be made into full Git entities first.

git branch -a (show all the branches)
git branch 0.8 0.8
git branch 0.6 0.6
git branch primitive primitive
git tag -f 0.6.0 tags/0.6.0
git tag -f 0.6.1 tags/0.6.1
git push --all origin
git push --tags origin

After all that we have our tags as Git tags and our branches as Git branches. You can browse them and clone the repository. A few of us have been experimenting and everything looks good. Adding current developers as collaborators enables them to push directly to the repository. There are also some web interfaces that allow for pulling from forks.

So if all goes to plan now, there will be no more commits to our old Subversion repository. We have preserved all of our history and I made sure the author metadata was improved. Hopefully this will make our development process more streamlined. We appreciate any and all tips, this looks like a good guide to keeping a fork in sync, pushing and pulling where necessary.

Now back to coding - we want to get a new release out!



Leave a comment

December 23, 2008
Diego Pettenò a.k.a. flameeyes (homepage, stats, bugs)

Diego Pettenò

Short preamble: I’m in a very depressed mood, like I haven’t been in month; this is very bad for my health but usually means I can focus on things much better, so you might actually find out I’m doing more than usual. Of course there is also to count in that I’m working during holidays so it’s not going to be all nice at all, even counting my depression off.

As I’ve written, I don’t trust closed-source software even the slightest even though it does not really mean that free software is much better, process-wise, dealing with bundled libraries (like the bundled libs bug shows), with free software, or at least open-source software, there is the chance to check the sources out to fix the eventual issues.

This means that I won’t be using closed source software where security is a major concern, but since sometimes I have to use closed-source software, like Skype, or Sun’s compiler, it’s obvious that I have to find a compromise so I can still use them and yet feel reasonably safe. This is what is usually called having a mitigation strategy.

One of the most complex and well known mitigation strategies is of course SElinux, which makes a Linux system more like an APC than a computer. But such a system is probably safe to consider overkill for most systems, especially power user desktop systems.

Since this is, as I said, overkill, I’m more prone to look at smaller strategies, one of which I already discussed about: pam_mktemp . This module allows to create per-user private directories that make it much harder to exploit insecure temporary files vulnerabilities. Which is very nice since this seems to be a very common class of vulnerabilities, and my data shows that there is way too much software that still uses insecure functions to create temporary files, closed and open source alike.

Unfortunately, as you can read in my earlier blog post, this is not automatically a way out of the problem. The start-stop-daemon command from OpenRC plays nice with this just in the last release, and even with that, there are problems. The first problem is that the way pam_mktemp works, there is a need for the software calling PAM to open the session to properly set up the environment with its changes (which is what s-s-d lacked in previous versions). This causes for instance the gnome-keyring daemon to start with the wrong temporary directory when started by the PAM session chain. Even though pam_mktemp is invoked before the daemon, by the time it’s started the TMPDIR variable is not set in the environment. The reason for this is that the variable should not be changed if the session chain aborts the login.

The second problem is that not all software supports TMPDIR properly; Emacs has been fixed recently and now the emacs daemon starts up properly, but other software ignores TMPDIR altogether. VirtualBox (of which I still have things to say beside this) does not respect it for instance, which means that the module wouldn’t have spared you from the recent vulnerability that involved the software.

The third problem is that sometimes software expects TMPDIR to be world-readable, which is a bad assumption; Samba does this, and since s-s-d is now fixed, it now fails to work on my system. I still haven’t found out whether the PAM session chain was called at that point, and it’s just duplicating the problem with s-s-d with a different symptom, or if it fails to call it entirely. In either case, it’s a thing that has to be fixed to make sure that mitigation strategies like this one get in the default spirit of users.

But again this is just one part of the problem, and one part of mitigation. Other problems relate to the way we run some of the services, a lot of which still run as root rather than under a unprivileged user; while the git-daemon issue is now solved and the default install does not run as root any longer, there are more daemons that have the same problems.

Just as an example, I noticed that the iSCSI daemon ietd still runs under root, and I’ve added that to the list of software I have to check to see if I can improve it. Similarly, the init script for mpd does not use s-s-d to switch user but leaves it to mpd itself, spawning it by default with unneeded root privileges, and additionally not allowing pam_mktemp to create a new temporary directory for the mpd user (I have to spend some time on that since I’d also like to provide an alternative init script with multiplexing, which would then allow to run multiple mpds for different users, and in my case to just have the single mpd running as my own user rather than a different user entirely).

At any rate, I’m going to continue my best to make sure that secure defaults are in place in Gentoo, and that further mitigation strategies can be made available so that the users forced to use proprietary closed-source software don’t need to just accept whatever comes their way. Please join my efforts, if you can, by checking which software ignores TMPDIR and asking nicely upstream to fix the issue.

It’s Alive! (December 23, 2008, 18:17 UTC)

Daniel Gryniewicz

32-bit userland in KVM

That is totem playing an ogg in a Gnome desktop in KVM on my 32bit-userland image.  It works!  Woo hoo!

Now all that’s left is to try it on real hardware, and figure out why the kernel fails to build.

Leave a comment

Marcus Hanwell a.k.a. cryos (homepage, stats, bugs)
Avogadro, Git, GitHub and New Toys (December 23, 2008, 10:55 UTC)

Marcus Hanwell

I have been using Git and Git SVN for quite some time now. It took me a little while to get into Git and see what all the fuss with DVCS was about. Now I find myself enjoying using Git more and more when it is the only version control in use in a project.

Git SVN is certainly a great compatibility layer, and it has allowed me to use Git as my version control system locally without requiring that the projects I contribute to switch. I had heard lots of good things about GitHub, but had not found the time to check it out until Geoff showed me Avogadro after he had pushed it to GitHub. There are some great stats and it looks like is has some great features. I was originally put off by the 100MB limit on storage as it seemed a little low.

When I got home yesterday evening I started playing with it and ultimately made three Avogadro repositories - the third one looks like it is the charm. I had not been worried about importing author metadata previously as I had just been using it locally, but after reading this short guide to migrating to Git I had all the tools I needed. A short trawl through our mailing lists, web pages and history to update author information and I had a shiny new Avogadro git repository.

I am still getting the hang of how all this works. I have added several other committers it matched up as collaborators, but did not spot what exactly that means yet. I was able to get CIA working in our IRC channel, pushed some changes and even synced up a commit from this morning. So I think all is well and I will likely keep this repo up to date with our development. It only has trunk in it, but shows photos of your friendly Avogadro developers and a really nice visualisation of development over time. I think it should open up with the latest stats first, but other that that am impressed. It complements some of the ohloh analysis quite nicely too.

Now can we all please move our source over to some kind of Git repository please!

Leave a comment

December 22, 2008
Tobias Klausmann a.k.a. klausman (homepage, stats, bugs)
Inside an Alpha, Part I (December 22, 2008, 14:13 UTC)

As part of a miniseries, I'll describe the insides of the more common Alphas. Nothing too technical, just nice pictures and a word or two regarding the pitfalls and unique features of the Alpha machines I have access to.

The images in this post are rather small. If you'd like to have a peek at larger versions of them, you can browse the gallery.

First in the series is my trusty Compaq AlphaStation XP1000. Here it is, with the front door closed:


Note that it's about the size of a midi tower, but way heavier. I guess it's close to 15kg in all. And there's not much hardware in it: aside from the necessities just a few extra PCI cards, one disk and two CD-Rs. It's also relatively short (in depth) for a midi tower. How that turns out to be of disadvantage, we'll see later on.

Here's the back side of the unit. As you see it has one Ethernet port, two USB connectors, Line/Headphone in/out, PS/2 mouse and keyboard, two serial ports and a parallel port. At the bottom right are the PCI cards: one 53c895, an ELSA GLoria II, a cheap PCI audio card and a cheap Ethernet card. Audio and Ethernet I added because the default ones work sort of ok, but those two work way better. The same goes for the 895 vs the on-board ISP logic SCSI HBA. The two blanked slots on the left are for purposes unknown to me. As we'll see later, there is little to support any cards there, so I suspect it's intended for additional ports that don't fit the PCI/ISA slot back-end. Also note at the top left the indentation and two small slot blanks. On a PWS 500au, which has a very similar case, those two are used for the audio and network connector boards. Finally, the power supply you see at the center is not anything compatible with an ATX or AT power supply, at least connector-wise. It's rated at 300 Watts when running on 230V.


This class of workstations opens to the right, that is, opposite of the common PC case. At the top there is the CPU board (more on that later), left middle are the two CD-R drives and floppy drive. At the middle right there's the power supply and below it a cage for two 3.5" hard drives. below all this are the PCI and ISA connectors and the on-board connectors for IDE, SCSI, Floppy etc.


As you can see, the whole case is rather cramped. The sheet metal barrier between the middle and lower parts has a rather small hole for cabling, so it's always fiddly to install more than just a hard-disk and a CDR. Also, the rear end of the CD-Rs is quite close to the power supply and hard drives, so it's very easy to cinch cabling. On top of that, access to the PCI and ISA cards is rather difficult, especially if you have cabling going to one of the cards (as I do).

From the top, we see the CPU fan (black rectangle at the front) and air duct (grey) and the CPU right behind it. Behind that are the memory banks. Obscured in this picture is the chipset, we'll see that later on.


The CPU board slides out horizontally as you can see here. Note that while the levers help pulling the board out, they do nothing for pushing it back in. You have to be extra careful to get it to seat properly. At the top center you can make out the NVRAM battery. It's easily accessible and of a readily available type. You can also see the two holes in the CPU heat sink. Those provide access to two hex nuts that hold the sink in place. The bolts are part of the CPU packaging.


The CPU board from the rear end. The four metal-plated chips on the right are the chip set (memory controller etc.). Those chips run very hot (by spec). I've measured them running at up to 65C under full load.


This concludes our tour of the XP1000. As you can see, the machines while nice to work with from the software/OS side are not ideal if you swap around hardware a lot. Especially the drive cages could have used more room around them. Still, I like the machines. They're built like tanks and the fans still are the first ones the machine came with and there is no rattling or grinding.

December 21, 2008
Diego Pettenò a.k.a. flameeyes (homepage, stats, bugs)
A couple of thoughts about package splitting (December 21, 2008, 22:17 UTC)

Diego Pettenò

In my post regarding remote debugging (which I promised to finish with a second part, I just didn’t have time to test a couple of things), I’ve suggested the idea I’d like to have some kind of package splitting in Portage, to create multiple binary packages out of a single source package and ebuild, similarly to what distributions based on RPM or deb do (let’s call them RedHat and Debian, for historical reasons).

Now, I want to make sure nobody misunderstand me: I don’t intend to propose this as a way of removing the fine-grained control USE flags give us; I sincerely love that; and I also love not having to worry about installing -dev and -devel packages on my machines to be able to build software, even outside of the package manager’s control. I really find these two are strengths of Gentoo, rather than weakness, so I have no intention to fiddle with them. On the other hand, I think there are enough uses that would allow for an even finer control on binpkg level.

I’ve already given a scenario in my post about remote server debugging, but let’s try to show something different, something I’ve actually been thinking about myself. Yes I know this is a very vested interest for me, but I also think this is what makes Free Software great most of the time; we’re not solutions looking for problem, but usually solutions to problem one had at least at one point in time. Just like my writing support for NFS export on the HFS+ filesystem in Linux.

So let me try to introduce the scenario I’ve been thinking about. As it happens, I tend to a series of boxes in many offices for friends and friends of friends in my spare time, on the side. It’s not too bad, it does not pay my bills, but it does pay for some side things, which is good. Now since these offices usually use Windows, even though I obviously install Firefox as the second step after doing the system updates, it’s not unlikely that every other time I go there I have to clean up the systems. I think there are computers I’ve wiped up and reinstalled a few times already. I’ve now been thinking about setting up some firewalls based on Snort or similar. Since I am who I am, these would end up being Gentoo-based (as a side note, I’m tempted to set it up here so I can finally stop having trouble with Vista-based laptops that mess up my network). Oh and please, I know it might sound very stupid considering there are solutions good for this already, but considering how much I’m paid and the amount of money they are ready to spend (read: near to none), I would find it nicer to be paid to work on some Gentoo-related stuff than be paid to just look up and learn how to use already made equipment. Of course if you have suggestion, they are welcome anyway.

So anyway, in this situation I’d have to set up boxes that would usually feel very embedded-like: a common basis, the minimum maintenance possible, upgrades when needed. Donnie’s idea of using remote package fetching and instant deletion is not that good for this because it still requires a huge pipe to shove the data around; not only I don’t have so much upload bandwidth to employ for binpkging a whole system with debug information, it would also be a hit that most of my users wouldn’t like to have, on their bandwidth (if they want to use BitTorrent or look up p0rn from the office is not my problem).

With this in mind, I’d sincerely find it much nicer to be able to split packages, Portage-side, into multiple binary packages that can be fetched, synced, or whatever else, independently, as needed. As I proposed, a binpkg for the debug information files, but also a binpkg for documentation (including man and info pages), one for development data (headers, pkg-config), and maybe one for the prepared sources, that I want to talk about in a moment. With an environment variable it shouldn’t be much of a problem to choose which ones of these split binary packages to install in the system without explicit request; with a default including all of them but the debug informations and the sources. This would also replace the INSTALL_MASK approach as well as noinfo, noman, nodoc FEATURES. It wouldn’t be like a logical split of a package in multiple entries in the system, but rather a way to choose which parts to install, complementary to USE flags.

As for packaging the sources as I said above, there are two interesting points to be made for that, or maybe three. The first problem is that when you have to distribute a system based on Gentoo, you cannot just provide the binaries; since many packages are released under the GNU GPL version 2, even if you didn’t change the sources at all you should be distributing them alongside the binaries; and we modify a lot of sources. For license compliance we should also provide the full set of sources from which the code is derived. This is especially tricky for embedded systems. By packaging up the sources used for the builds, embedded distributors would be able to just provide all the -src subpackages as the full sources for the system.

The second point is that you can use the source packages for debugging too. Since there is, as far as I know,no way to fully embed the source code of software in the debug section of the files generated from that, the only way for GDB to display the source code lines during debugging is having the source files used for build available during the debugging session. This can easily be done by packaging up the sources and installing them in, say, /usr/src/portage/ when they are needed, from a subpackage.

A final point would be that by packaging sources in sub-packages, and distributing them, we could be reducing the overhead for users to unpack (maybe with uncommon package formats) and prepare sources (maybe with lots of patches and autotools rebuilding). Let’s say that every 6 hours a server produces md5-based source subpackages for all the ebuilds of the tree, or a subset of them. Users would then use those sources primarily, but still having the ebuilds to provide all the data and workflow so that the original untouched source would be enough to compile the package. Of course this would then require us to express dependencies on a per-phase basis, since then autotools wouldn’t be required at buildtime at all.

Okay I guess I’m really dreaming lately, but I think that throwing around some ideas is still better than not doing so, they can always be picked up and worked on; sometimes it worked.

Daniel Drake a.k.a. dsd (homepage, stats, bugs)
XO laptop on The Gadget Show (December 21, 2008, 16:56 UTC)

The XO is hitting mainstream UK TV! The laptop will be reviewed on The Gadget Show, 8pm on Monday 22nd December, on Five.

Update: It got rescheduled at last minute. It’ll hopefully be aired after Christmas.

Leave a comment

Joshua Nichols a.k.a. nichoj (homepage, stats, bugs)
vimpocalypse (December 21, 2008, 00:02 UTC)

Joshua Nichols

Switching from TextMate to vim or emacs has been a trend on the rise. I'm riding the trend too, and it feels pretty good.

Coming from TextMate, there were a lot of things I missed. There were a lot of things I didn't even realize existed.

This post is meant to help folks who are trying to make the same switch. It isn't going to be the most extensive resource, but should point you in the right direction.

Oh, and just as a warning, one thing I've notice about reading vim resources is that they usually assume a certain level of familiarity with vim, and I don't think this will post will be any different. If something is over your head, well, that gives you a new area to learn about :)

Resources

A Byte of Vim: This ebook starts at the very beginning, and walks you through pretty much every day-to-day type editing you're likely to encounter. I've been using vim for several years, but I must have been using it pretty poorly, because I learned a lot from it.

Google: Seriously. Whenever there was something I didn't know how to do, googling for vim <insert thing here> worked nearly every time. Sometimes it took a few incantations.

Vim Wikia: In said google queries, this wiki showed up about 50% of the time. Lots of great tutorials, walkthroughs, and tips.

7 Habits For Effective Text Editing 2.0: Bram Moolenaar, one of the main authors of vim, gave this talk at Google. One of the takeaways I had from this is that it doesn't matter which editor you use, as long as you are using it effectively. If you do something that feels repetitive, stop, find a better way, and then start using it. In another way, constantly refactor how you use your editor to improve efficiency.

Friends, Coworkers, and Family: You probably know some people using vim. They make a good resource as well. I primarily tap them as a resource for how they are using a particular command/plugin, or how they are achieving a particular goal. Be sure to do some research first, or you might get hit with a letmegooglethatforyou.

Plugins

vim is amazingly extensible. There are also a lot of clever folks across the tubes. As a result of this, there are a lot of great plugins out there to assist your editing.

These are the plugins I'm using, and I'll include any tweaks or mappings I'm using. These would go in your ~/.vimrc

rails.vim: For me, it's all about being able to navigate around your project. For example, while editing a controller, you can easily switch to the functional test, or from a model to the unit test, and vice versa (:A). If you have your cursor on a render :partial, you can easily navigate to the partial (gf). This is only a sampling of what it can do though.

NERD Commenter: Lets you easily comment out code. I made command-/ comment out code, kinda like in textmate.

" bind command-/ to toggle comment
" requires NERD Commenter to be installed: http://www.vim.org/scripts/script.php?script_id=1218
nmap <D-/> ,c<space>
vmap <D-/> ,c<space>
imap <D-/> <C-O>,c<space>

NERD Tree: File navigation using a directory tree. I setup \d to toggle it being visible.

" bind \d to toggle file browser
" requires NERDTree
nmap <leader>d :NERDTreeToggle<CR>

autoclose.vim: This will automatically close things like quotes, brackets, parantheses, etc. I've found it to be a little buggy at times, so I'm considering dropping it, or finding something else. YMMV.

bufexplorer.vim: vim keeps track of all the files you've edited, and has ways of navigating between them (:bp and :bn for example). bufexplorer gives you a much more visual way of do this. It displays a window with all the files you've edited, you navigate to the file you want, and hit enter, and whoops, editing the file now. By default, it binds to \be.

endwise.vim: Kind of similar to autoclose, but more for structures you'd encounter while programming, such as if statements and class declarations.

fuzzyfinder_textmate: This is the texmate equivalent of 'Go to File'. Pull it up, and start typing a filename. I bind it to \t (command-t opens a tab in MacVim).

" binds \ t to textmate-style fuzzy finder
map <leader>t :FuzzyFinderTextMate<CR>
let g:fuzzy_matching_limit=60 " this seems to help performance
let g:fuzzy_ceiling=20000     " I have some projects with a lot of files...

allml: Gives you some ways to interact with the various HTML-like code that various programming languages / frameworks provide, like shortcuts to create <% %> in ERB

matchit: vim has builtin support for 'bouncing' between parentheses using the % key. matchit extends it further to work with things like if blocks, class declarations, etc.

gist.vim: Post stuff to Gist.

snippetsEmu: Emulates textmate snippets. I've found I don't actually use this as much as I would have thought.

surround.vim: Tools for 'surrounding' your text. For example, add quotations, switch it to paraentheses, etc.

taglist.vim: vim has support for browsing by 'tags', which is to say, by method name, class name, etc. taglist gives you a way of seeing tags in the current file. It brings up a new window, which you can browse to the tag you want, and then automatically navigate to it. I've tried to make it act like 'Go To Symbol' in textmate. \T toggles the taglist, it gets focus, and then goes away after you navigate to something.

" binds \ T to taglist (sorta like textmate apple-shift-t)
map <leader>T :TlistToggle<CR>
let Tlist_Show_Menu=1
let Tlist_GainFocus_On_ToggleOpen=1
let Tlist_Close_OnSelect=1
let Tlist_Compact_Format=1

ack: Unfortunately not a packaged plugin, but you can just drop the code snippet (the second, updated one) in ~/.vim/plugin/ack.vim. This lets you search for stuff in your current directory, and gives you a way to browse the results. Kinda like textmate's 'Find in Project'. I made \F start the command off for you.

" \F to startup an ack search
map <leader>F :Ack<space>

Sagelike advice

These are bits and pieces I've picked up along the way.

TextMate-like functionality

Not textmate specific, but you can map stuff to the command key in MacVim using M. For example, <M-/> would be command-/

If you have problems with fuzzfinder_textmate being able to find the Ruby classes, I found just placing fuzzy_file_finder.rb in ~/.vim/ruby fixed the problem.

Insert hashrockets (=>) while in insert-mode, just like in TextMate:

" bind control-l to hashrocket
imap <C-l> <Space>=><Space>

Indent and unident with command-[ and command-]:

" bind command-] to shift right
nmap <D-]> >>
vmap <D-]> >>
imap <D-]> <C-O>>>

" bind command-[ to shift left
nmap <D-[> <<
vmap <D-[> <<
imap <D-[> <C-O><<

Switch to different tabs using command-1 through command-9:

" open tabs with command-<tab number>
map <D-1> :tabn 1<CR>
map <D-2> :tabn 2<CR>
map <D-3> :tabn 3<CR>
map <D-4> :tabn 4<CR>
map <D-5> :tabn 5<CR>
map <D-6> :tabn 6<CR>
map <D-7> :tabn 7<CR>
map <D-8> :tabn 8<CR>
map <D-9> :tabn 9<CR>

Simplify window navigation

vim supports splitting a window vertically and horizontally, and then navigating between them. The default way kinda hurts my hands. Let's reduce it to \s to split horizontally, \v to split vertically, and \w to cycle to the next window.

" window splitting mappings
" split vertically with <leader> v
" split horizontally with <leader> s
nmap <leader>v :vsplit<CR> <C-w><C-w>
nmap <leader>s :split<CR> <C-w><C-w>

" Make it way easier to switch windows (<leader>w)
nmap <leader>w <C-w><C-w>_

Gleefully stolen from James Golick's dotfiles.

Other stuff, with less explanation

set nocompatible          " We're running Vim, not Vi!
syntax on                 " Enable syntax highlighting
filetype plugin indent on " Enable filetype-specific indenting and plugins

set incsearch             " Incremental searching
set hlsearch              " Highlight search results once found:
                          " http://vim.wikia.com/wiki/Highlight_all_search_pattern_matches
set number                " can has line numbers?
set cursorline            " highlight the current line the cursor is on
set showmatch             "sm:    flashes matching brackets or parentheses
set smarttab              "sta:   helps with backspacing because of expandtab

Fin

Whew. That was a lot longer than I had anticipated. Be sure to leave a comment if you have a question, or have better ways of doing any of this.

December 20, 2008
Diego Pettenò a.k.a. flameeyes (homepage, stats, bugs)
Why a tinderbox is not enough (not even two) (December 20, 2008, 21:40 UTC)

Diego Pettenò

I’ve already written something about automation for Gentoo bug search, but I think sometimes it’s not easy to understand that just using a huge tinderbox, even distributed, is not going to help much to make sure that software works. The problem is that sometimes, even if software builds fine, it’s just going to break at runtime, and even though tests help, when they are handled properly, they are far from complete solutions.

Problems like the one described in the post above for hfsplusutils are just impossible to gather at build time without running the tests on sample data (which reminds me of the uif2iso problem but this can get even more subtle: while it’s true that most --as-needed failure happen at buildtime, there are quite a few that will not hit until runtime; one such cases happened to me with libcompizconfig, compizconfig-python and fusion-icon: this last one software wouldn’t start because Python failed to load the second, because the first one wasn’t linking to libX11.

Now of course this could have been found if either libcompizconfig or compizconfig-python had a testsuite, but since I already said that running a tinderbox run with testsuite is probably not something that I would like to do on a daily basis.

Especially for old software, there are problems like endianness issues, 64-bit arches and PIC code that are almost impossible to figure out at buildtime, and that need to be checked during software use. But it’s not just that. For binary packages, especially those of proprietary software, there usually isn’t a testsuite, for this reason, their executables are rarely checked at runtime for consistency. This becomes a problem because sometimes you have software that links against old versions of libraries. This is the same reason why adding VirtualBox 2.1.0 binary package in the tree is going to take a while for Alessio: it uses the old ABI of libcap, which will require resurrecting, and maintaining, an old verison of libcap just for that. (I have a few more issues to talk about regarding the new VirtualBox release but I’ll get to that in another moment).

And yet, the reasons why neither my nor Patrick’s tinderbox can be a replacement for a more throughout approach to packages testing are not finished here. But before proceeding to more, I have to make a distinction between the different approaches me and Patrick took. Patrick’s tinderbox removes all the superfluous packages from the system when installing a new one, which is very good to test for missing dependencies; my method instead iterates over each of the packages in the tree and installs them one by one in the system, filling up the space, which can easily ignore missing dependencies but provides more interesting results regarding iteration of particular ebuilds.

So while my method glosses over broken runtime and buildtime dependencies, like pkg-config not depended upon and similar, Patrick’s method is not going to hit problems like dev-scheme/chicken breaking most of the Mono packages (that would pick up /usr/bin/csc as the C# compiler rather than mcsc), or collisions between unrelated packages.

This means that either one of the two tinderboxes is just not enough to find all the issues, and even the two of them together won’t be enough. Even adding AutoTua to that, it’s just not going to cut it. As Jeremy said on a blog post of mine, we need humans (developers and users) to report issues. I start to feel we also have a need for some real numbers of how many users use packages. Yes I know that’s going to be a popularity contest, and it’s likely that there will be people that would just go on to submit fake results, but even for tree cleaning, it’s important to know whether packages don’t have bugs failed against them because they are good, or just because nobody has used them in so much time.

Oh and so that you know, I currently have little less than 1500 bugs open that I reported (and over 3000 bugs that I reported since I started contributing to Gentoo), and all of them are reported by hand, there are still issues that force me not to use scripts like pybugz. I’ll see to write about them, maybe Zac can see to find a solution to those, like he has been doing quite a while lately for me. Thanks Zac!

Gentoo News

The time has come! Our release engineers have been refining their automated builds of the minimal CD and stage3 tarballs, and the first builds are uploaded to our mirrors. Select your favorite mirror and navigate to the /experimental/ directory. Under the x86, amd64, ia64, alpha, and sparc64 architectures, the autobuilds directory contains stages. So far, minimal CD images are only available for x86 and amd64.

We're still working to add automated builds for ppc, ppc64 and hppa as well as CD images for architectures lacking them. We built the stage3 tarballs from the latest stable packages. Fresh builds will show up every week, although sometimes we might skip a week if build problems crop up.

Because these builds are automated, they have not been rigorously tested as the old releases. Occasionally, you might run into problems. If that happens, just try a file from a different week.

Please try out these new builds and leave your feedback on the forums. As always, please report any bugs on Bugzilla.

Happy holidays!

Discuss this!

Ben de Groot contributed the draft for this announcement.

32bit-userland on Gentoo amd64 (December 20, 2008, 00:00 UTC)

Daniel Gryniewicz

For the past couple of weeks, I’ve been working on getting 32bit-userland working on Gentoo amd64.  It’s at least partly working now;  it boots, you can log in, and I’m in the process of emerging gnome.

However, it’s not a smooth road.  I had to use catalyst to make my own stages.  No amount of massaging or editing or copying or screaming got me from a 64-bit stage3 to a 32bit-userland.  In addition, there’s something wrong with my toolchain somehwere.  I can’t build sandbox, for example, and had to use a binpkg from my 64-bit system.  Also, the kernel fails to build it’s config programs, so I can’t configure a kernel.

Finally, there are a number of broken packages; that is, packages that detect in some way or another what bitness they think you’re running and then use ASM coded for that.  This, obviously, fails.  Fortunately, this is so far only 3 packages: glib, libgcrypt, and mesa.  We’ll see how many more I hit before I get a working system.

Just a note: I’m not posting any of my profiles yet.  It’s not yet working, and it’s not for faint of heart.  If you are really interested, give me a ping on IRC and I’ll see if I can help you.  Otherwise, just wait until I can lick it into shape.

Leave a comment

December 19, 2008
Diego Pettenò a.k.a. flameeyes (homepage, stats, bugs)
Debug code and debug information (December 19, 2008, 23:59 UTC)

Diego Pettenò

There are many different but common misconceptions about debugging that are spread among those users who, having never learnt a programming language, cannot understand properly the difference between debug code and debug information. Some of these misconception causes misunderstanding with Gentoo’s way of handling the two things as separate and distinct feature of a software.

First of all, let’s start with what the -g, -ggdb and -g3 options are supposed to do. These three flags are used to add debug information, in form of either stabs or DWARF data, depending on the architecture, to the compiled files, may they be object files, shared objects or final executables. This data is used by debuggers like gdb to provide a meaningful backtrace, and is added to some special sections of the file. The difference between a file that is built with these options and a file that is built without those can be removed by using the strip command, since they don’t go touching the actual executable code or data entries. The only software that is susceptible to break with -g3 is the dynamic loader, and even that I’m not sure why.

The various level of debugging information are used to provide various level of backtracing, starting from giving the name of the functions called, and arriving to have the line numbers, the source lines, and macro expansion (which is especially useful when debugging stuff like scanelf, that is composed of a huge amount of macro-based meta-functions. Even when the full debug information is enabled in files, it’s not hindering performance, if not during the first scan and read of the ELF files, since the loader does not load the debug information by default, they are not in sections that are allocated in memory at runtime at all. (This is something you can easily understand once you know the difference between allocated and non-allocated ELF segments).

Debug code, instead, means adding special instruction in the executable code for debugging purposes; the most simple example is the assert() macro used to make sure that unexpected code paths are not taken; although this is often misused as a way to enact limitations in functions, the original idea behind assertions was to make the program die in a way easy to debug when a condition supposed to be always true was instead false. These checks wouldn’t be needed during standard usage, or should be handled gracefully if they indeed happen, so the assertions would just need to be taken out of the built code at that point, which is exactly what -DNDEBUG does. (On an autotools-related note, the AC_HEADER_ASSERT autoconf macro not only checks for the correct header, but also provides an easy to use --disable-assert option for the configure script to disable assertions altogether). Unfortunately nowadays assertions are often used to check the behaviour of code at runtime, even though an error would then cause the abort of the software, which makes it more difficult to just disable them altogether for final users.

But debug code can be much more complex, and might slow down operation a lot; it might be logging data extensively, it might fill the terminal with pointless information, it might check every and each step during processing. This type of code must certainly not be enabled for users’ runtime or their work would be greatly hindered.

Now that the distinctions are made, you can see why splitdebug/strip FEATURES are distinct from the debug USE flag. If you want to just get a backtrace for a crash you got during execution, you need debug information, you don’t need debug code; if possible, debug code might actually stop the software from crashing; as could reducing the optimisation flags. For users, it’s more than likely than the debug USE flag wouldn’t be useful at all; for developers who know what to do, this fine-grained control is most likely the best option they have.

So please next time you think about mixing the debug USE flag and the splitdebug/strip FEATURES in the same idea, try to think of what exactly you want to achieve. And no, disabling -O2 is not always a good idea to have a meaningful backtrace, especially since as I said, -O0 might make stuff not build, so you shouldn’t be ready to just enable that unconditionally to get a backtrace for a bug report.

December 18, 2008
Diego Pettenò a.k.a. flameeyes (homepage, stats, bugs)

Diego Pettenò

If you follow my blog since I started writing, you might remember my post about imported libraries from last January and the follow up related to OpenOffice you might know I did start some major work toward identifying imported libraries using my collision detection script and that I postponed till I had enough horsepower to run the script again.

And this is another reason why I’m working on installing as many packages as possible on my testing chroot. Now, of course the primary reason was to test for --as-needed support, but I’ve also been busy checking build with glibc 2.8, and GCC 4.3, and recently glibc 2.9 . And in addition to this, the build is also providing me with some data about imported libraries.

With this simple piece of script, I’m doing a very rough cut analysis of the software that gets installed, to check for the most commonly imported libraries: zlib, expat, bz2lib, libpng, jpeg, and FFmpeg:

    rm -f "${T}"/flameeyes-scanelf-bundled.log
    for symbol in adler32 BZ2_decompress jpeg_mem_init XML_Parse avcodec_init png_get_libpng_ver; do
    scanelf -qRs +$symbol "${D}" >> "${T}"/flameeyes-scanelf-bundled.log
    done
    if [[ -s "${T}"/flameeyes-scanelf-bundled.log ]]; then
    ewarn "Flameeyes QA Warning! Possibly bundled libraries"
    cat "${T}"/flameeyes-scanelf-bundled.log
    fi

This checks for some symbols that are usually not present without the rest of the library, and although it gives a few false positives, it does produce interesting results. For instance while I knew FFmpeg is very often imported, and I expected zlib to be copied in every other software, it’s interesting to know that expat as much used as zlib, and every time it’s imported rather than used from the system. This goes for both Free and Open Source Software and for proprietary closed-source software. The difference is that while you can fix the F/OSS software, you cannot fix the proprietary software.

What is the problem with imported libraries? The basic one is that they waste space and memory since they duplicate code already present in the system, but there is also one other issue: they create situations where even old, known, and widely fixed issue remain around for months, even years after they were disclosed. What preserved proprietary software this well to this point is mostly related to the so-called “security through obscurity”http://en.wikipedia.org/wiki/Security_through_obscurity. You usually don’t know that the code is there and you don’t know in which codepath it’s used, which makes it much harder for novices to identify how to exploit those vulnerabilities. Unfortunately, this is far from being a true form of security.

Most people would now wonder, how can they mask the use of particular code? The first option is to build the library inside the software, which hides it to the eyes of the most naïve researchers; by not loading explicitly the library it’s not possible to identify its use through the loading of the library itself. But of course the references to those libraries remain in the code, and indeed most of the times you’ll find the libraries’ symbols as defined inside executables and libraries of proprietary software. Which is exactly what my rough script checks. I could use pfunct from the seven dwarves to get the data out of DWARF debugging information, but proprietary software is obviously built without debug information so it would just waste my time. If they used hidden visibility, finding out the bundled libraries would be much much harder.

Of course, finding which version of a library is bundled in an open source software package is trivial, since you just have to look for the headers to find the one defining the version—although expat often is stripped out of the expat.h header that contains that information. On proprietary software is quite more difficult.

For this reason I produced a set of three utilities that, given a shared object, find out the version of the bundled library. As it is it quite obviously doesn’t work on final executables, but it’s a start at least. Running these tools on a series of proprietary software packages that bundled the libraries caused me some kind of hysteria: lots and lots of software still uses very old zlib versions, as well as libpng versions. The current status is worrisome .

Now, can somebody really trust proprietary software at this point? The only way I can trust Free Software is by making sure I can fix it, but there are so many forks and copies and bundles and morphings that evaluating the security of the software is difficult even there; on proprietary software, where you cannot be really sure at all about the origin of the software, the embedded libraries, and stuff like that, there’s no way I can trust that.

I think I’ll try my best to improve the situation of Free Software even when it comes to security; as the IE bug demonstrated, free software solutions like Firefox can be considered working secure alternatives even by media, we should try to play that card much more often.

Bernard Cafarelli a.k.a. voyageur (homepage, stats, bugs)
X2go ebuilds status update (December 18, 2008, 21:34 UTC)

Bernard Cafarelli

In a previous post, I spoke about new ebuilds for X2go client and server, a GPL remote desktop solution that's based on NX technology, but in a different way compared to nxserver. With lots of help from Joachim Langenbach in bug #249600, I'm glad to say now that ebuilds in the NX overlay for both client and server work fine, and will probably moved to portage soon (ldap management ebuilds will probably wait a bit longer in the overlay, as I cannot test them for now).

For pros and cons, X2go does not need a special "nx" user on the server, you can use your ssh private key (as you log in directly as your real user), both client and server are GPL are not limited in number of connections, remote mounting of directories is easy (via sshfs/fuse), administration of the server can be done via kde control panel elements (including ldap accounts, if you use it). However x2go requires a running postgresql database on the server, does not support VNC/rdesktop proxying or shadow sessions, and some of the nice advanced features seem reserved to dedicated thin clients setups (like saving your session on a usb key, ...).

Anyway, if you're curious to try it, 'layman -a nx' and then emerge either x2goclient or x2goserver depending on which computer you are ;)

Leave a comment

Kenneth Prugh a.k.a. ken69267 (homepage, stats, bugs)
Python and XML DOM fun (December 18, 2008, 21:17 UTC)

So I have recently been playing with Java a great deal and recently learned how to parse XML with it. Java is a bit overkill for a multitude of small projects or scripts I want to develop however, so I have decided to scratch my python itch once again.

Googling around and looking at the python docs I didn’t really see an example that I could easily follow or adapt. So in the end I took a recent Java project and converted the XML part of it to python to bring life to this little example code.

GBug is a tiny little python app that uses DOM to parse XML from the Gentoo Bugzilla and provides some simple information about the requested bug number. I believe the example is pretty clear, although I wrote it in just a few minutes so it could be done better. I hope someone finds the example useful.

#!/usr/bin/env python

# Licensed under the GPLv3 by Kenneth Prugh [ken69267@gmail.com] (C) 2008

# DOM example adapted from one of my Java projects which displays info about
# the desired bug from the Gentoo Bugzilla

from xml.dom import minidom
import urllib2

class gbug():
    def getValueForTag(self, Element, tag):
        NodeList = Element.getElementsByTagName(tag)

        Element = NodeList.item(0)

        return Element.firstChild.nodeValue

    def getBug(self, bugID):
        self.bugID = bugID

        bugURL = urllib2.urlopen("http://bugs.gentoo.org/show_bug.cgi?ctype=xml&id="+ \
                bugID)

        doc = minidom.parse(bugURL)

        Element = doc.documentElement

        bugNodes = doc.getElementsByTagName("bug")
        bugEL = bugNodes.item(0)

        print "Description: " + self.getValueForTag(bugEL, "short_desc")
        print "Reporter: " + self.getValueForTag(bugEL, "reporter")
        print "Created: " + self.getValueForTag(bugEL, "creation_ts")
        print "Status: " +  self.getValueForTag(bugEL, "bug_status")
        print "Assigned to: " + self.getValueForTag(bugEL, "assigned_to")

main = gbug()
input = raw_input("bug #: ")
main.getBug(input)

Sample Output:

    ken@t61 python % python gbug.py
    bug #: 216776
    Description: dev-java/smack-3.0.4 version bump
    Reporter: ken69267@gentoo.org
    Created: 2008-04-07 21:34 0000
    Status: NEW
    Assigned to: java@gentoo.org

Download gbug.py

Sven Vermeulen a.k.a. swift (homepage, stats, bugs)
Extremely simple task manager (December 18, 2008, 20:46 UTC)

At work, I am often busy with quite a few projects. Yet, at times, I have no outstanding tasks because all of my tasks can only start when an event has occurred (like a server which is made available, or a budget that is approved) or another task has finished.

To keep track of my work, I write an extremely simple task manager: an XML file (for the data), XSL file (for the rendering) and HTML/CSS file (to render and use the browsers’ XSL capabilities). I call it taskviewer due to lack of more imagination ;-)

It is a simple manager with no user interface for managing it at all - so you’ll need to edit the XML file yourself. However, the HTML/CSS file, together with the XSL file, renders the content of the XML file in such a way that you have a nice overview of your tasks.

It’s “features” are simple:

  • keep track of events you are waiting for
  • keep track of a tasks’ dependencies (events or other tasks)
  • get an overview of tasks that can immediately start versus that are blocked, waiting for its dependencies to finish

There is an example available online with some hypothetical data.

If you know of a simple program (preferably java or one available for both Windows and Linux) that has similar features (especially tracking tasks depending on certain events), please do tell me. I’ve looked at tools like taskjuggler but couldn’t find one that remains simple yet has these features.

Leave a comment

Diego Pettenò a.k.a. flameeyes (homepage, stats, bugs)
Filesystems -- ext4dev fails (December 18, 2008, 18:45 UTC)

Diego Pettenò

flame@yamato ~ % touch /var/tmp/portage/test
touch: cannot touch `/var/tmp/portage/test': No space left on device
flame@yamato ~ % df -h | grep /var/tmp
                       32G  7.2G   23G  25% /var/tmp
flame@yamato ~ % df -i | grep /var/tmp
                     2097152  419433 1677719   21% /var/tmp

Now, a mount cycle later it worked fine, but it’s still not too nice since it caused all the emerge running to fail, just like XFS did, but without leaving trace on the kernel log, which makes it obnoxious since it’s hard to debug. I hope 2.6.28 is going to be better, certainly the tinderboxing is a nice way to stress-test filesystems.

I start to consider the idea of OpenSolaris, NFS, and InfiniBand…

Stuart Longland a.k.a. redhatter (homepage, stats, bugs)
Gentoo/MIPS Update (December 18, 2008, 10:34 UTC)

Stuart Longland

Hi all… figured I better update everyone on what’s happening on the MIPS front.

Kumba has been working on newer kernel ebuilds to support various platforms, specifically IP30, which has been troublesome of late. At present, the best kernel revision to use is still 2.6.23 on Cobalt, Loongson 2E and O2.

Another alternative, is to download your own kernels from the Linux/MIPS website (not kernel.org), and research the patches needed to get things running yourself… in fact… Lemote Loongson 2E, Cobalt Qube2 and SGI Indy/O2 users should find minimal to no patching is necessary as the vast majority of changes have already been accepted into the mainline kernel.

Stages… there have been no further stages built at the moment from the 2008.0 ones. Partly because some of us have been just too busy. I’ve started to make inroads into a uClibc-based set of stages however, having got a uClibc-0.9.30 toolchain built the other day targetted at mipsel-unknown-linux-uclibc. It is hoped a workable environment can be constructed in the coming months that may turn into a new set of stages available for this platform.

Desktop Applications… have been lagging quite a bit. I have some patches for Thunderbird 2.0 that gets things rolling again… and may just work for big-endian MIPS (which was broken at last check) but due to lack of time, I haven’t been able to investigate further. A similar set of patches got Firefox 3.0 partially working… more work needs to be done.

My own status… I’m not planning on leaving anytime soon… just been extremely busy. As it is now, I’m passed my 8:00PM bed time (need to be up at 4:00AM tomorrow, currently it is 8:35PM AEST), so I’ll leave it at that.

Leave a comment

Ben de Groot a.k.a. yngwin (homepage, stats, bugs)
Supporting Songbird (December 18, 2008, 01:47 UTC)

One of the Songbird developers wrote a good blogpost about the problems that distros meet when trying to support Songbird, an open source iTunes competitor. The post concentrates on Ubuntu, as one of the most popular distros, but the concerns raised there in essence are the same for all distros. We have our own bug open for this new package, and it’s not really going anywhere. For us a similar compromise as for other distros exists: we do have a songbird-bin ebuild in Sunrise overlay.

But nobody wants to maintain an extra copy of xulrunner, or backport Songbird’s patches to the system xulrunner (which is used by Firefox among other packages). The whole idea seems very Windows-like to me: take existing packages, patch them to bits, mash it together with your own application into one big binary package. This is certainly not the way things are normally done on the Linux side of things.

I would urge the Songbird developers to work with upstream (Mozilla in the case of Xulrunner) to get their patches accepted, or some other kind of solution to make Songbird and vanilla Xulrunner to work together nicely. That would make it a lot easier for us, and any other Linux distro, as well as the BSDs, to import this package into our official repositories and give it proper support.

In the comments to that post you can read that OpenSuse and Fedora have the exact same problems. With such a unanimous voice from the distributions, I would say the ball is now in Songbird’s court, if they care about Linux support.

December 17, 2008
Diego Pettenò a.k.a. flameeyes (homepage, stats, bugs)
Remote debugging with GDB; part 1: Gentoo (December 17, 2008, 18:52 UTC)

Diego Pettenò

In the last couple of days I had to focus my attention on just one issue, a work issue which I’m not going to discuss here since it wouldn’t make much sense either way. But this brought me to think about one particular point: remote debugging, and how to make it easier with Gentoo.

Remote debugging is a method commonly used when you have to debug issues on small, embedded systems, where running a full blown debugger is out of question. With GDB, this consists of running on the target system a tiny gdbserver proces, which can be controlled by a master gdb instance via either serial port or TCP connection. The advantages don’t stop at not needing to run the full blown debugger: the target system may also be equipped with stripped programs with no debug information, and keep the debug information entirely on the system where the master gdb instance is being executed.

This is all fine and dandy for embedded systems but it also does not stop to this, you can easily make use of this technique to debug issues on remote servers, where you might not want to upload all the debug symbols for the software causing the trouble, it then becomes a nice Swiss Army knife for system administrators.

There are a few issues you got to work with when you use gdbserver though, some of which I think should be improved in Gentoo itself. First of all, to debug a remote system that is running a different architecture than yours, you need a cross-debugger; luckily crossdev can build that for you, then you need the actual cross-built gdbserver. Unfortunately, even though the server is small and self-sufficient, it is currently only being built by sys-devel/gdb which is not so nice for embedded systems; we’d need a minimal or server-only USE flag for that package or even better a dev-util/gdbserver standalone package so that it could be cross-built and installed without building the full blown debugger which is not useful at all.

Then there is the problem of debug information. In Gentoo we already provide some useful support for that through the splitdebug feature, which takes all the debug information from an ELF file, executable or library, and splits it out in a .debug file that only contains the information useful during debugging. This split does not really help much on a desktop system, since the debug information wouldn’t be loaded anyway, my reasoning to have it separated is to make sure I can drop them all at once if I’m very short on space, without breaking my system. On the other hand, it is very useful to have it around for embedded systems for instance, although it currently is a bit clumsy to use.

Right now one common way to achieve proper archiving of debug information and stripping them in production is using the buildpkg feature together with splitdebug, and set up an INSTALL_MASK variable for /usr/lib/debug when doing the build of the root. The alternative is to simply remove that directory before preparing the tarball of the rootfs or stuff like that. This works decently, since the binary packages will have the full debug information, and you’d just need to reinstall the package you need debug information for without the INSTALL_MASK. Unfortunately this will end up replacing the files from the rest of the package, which is not so nice because it might change the timestamps on the filesystem, as well as wasting time, and eventually flash too.

This also does not play entirely nice with remote server administration. The server where this blog is hosted is a Gentoo vserver guest, it was installed starting from a standard amd64 stage, then I built a local chroot starting from the same stage, setting it up exactly as I wanted it to be; finally, I synced over the Portage configuration files, the tree and the binary packages built of all it, and installed them. The remote copy of the packages archive is bigger than the actual used packages, since it contains also the packages that are just build dependencies, but the overhead of this I can ignore without too much problems. On the other hand, if I were to package in all the debug information, and just avoid installing it with INSTALL_MASK, the overhead wouldn’t be this ignorable.

My ideal solution for this would involve making Portage more aware of the splitdebug feature, and actually split it out on package level too, similarly to what RPM does with the -dbg package. By creating a -debug or -dbg binpkg to the side of each package that would otherwise have /usr/lib/debug in it, and giving the user an option on whether to install the sub-package, it would be possible to know whether to merge on the root filesystem the debug information or ot, without using INSTALL_MASK. Additionally, having a common suffix for these packages would allow me to just ignore them when syncing them to the remote server, removing the overhead.

Dreaming a bit more, it could be possible to design multiple sub-package automatic generation, to resemble a bit what binary distributions like Debian and RedHat have been doing all these years, to split documentation in its -doc package, the headers and static libraries in -dev and so on. Then it would just require to give the user ability to choose which subpackages to install by default, and a per-package override. A normal desktop Gentoo system would probably want to have everything installed by default, but if you’re deploying Gentoo-based systems, you probably would just have a chroot on a build machine that does the work, and then the system would just get the subset needed (with or without documentation). Maybe it’s not going to be easy to implement, and I’m sure it’s going to be controversial, but I think it might be worth looking into it. Implementing it in a non-disruptive way (with respect to the average user and developer workflow) is probably going to make it feasible.

Tomorrow, hopefully, I’ll be writing some more distribution-agnostic instructions on how to remotely debug applications using GDB.

Petteri Räty a.k.a. betelgeuse (homepage, stats, bugs)
Binary package for icedtea6 (December 17, 2008, 17:01 UTC)

Petteri Räty

Quite a few people have reported problems building icedtea6 or needed dependencies on our IRC channel and as the build is quite resource intensive, Caster has now made binary builds for icedtea6. The package is available via layman using:

layman -a java-overlay
emerge icedtea6-bin

The binary package should also make it easier to bootstrap the from source build. The binaries are built in stable chroots so they should run for our stable users too. Please report any problems to https://bugs.gentoo.org with [java-overlay] in the subject. For amd64 users this should be the easiest way to get a 64 bit browser plugin.

Leave a comment

Hanno Böck a.k.a. hanno (homepage, stats, bugs)
Interview on FSFE webpage (December 17, 2008, 16:28 UTC)

Hanno Böck As an FSFE fellow, I got interviewed for their webpage.

You can read it here.

Leave a comment

Anant Narayanan a.k.a. anant (homepage, stats, bugs)
My first meal! (December 17, 2008, 02:28 UTC)

Anant Narayanan

Today, after roaming the earth for 21 years depending on someone else to cook my food for me, I made a giant leap: I cooked my own meal. Right from buying groceries to cleaning up the dishes afterward :)

Here’s a picture of the modest beginning:

It looks a lot more delicious than it really was: just boiled vegetables with garlic bread - dressed with salt, pepper and a dollop of butter - rather bland for an Indian tongue. But, oh well, it’s a start ;)

Leave a comment

December 16, 2008
Robin Johnson a.k.a. robbat2 (homepage, stats, bugs)

Robin Johnson

Now for the second set of statistics. These aren't directly useful to mirrors in estimating their traffic, but instead gives a good overview of how our mirroring setup works internally, and now much traffic is involved in the fan-out stage. Distfiles are the main content moved around by this system, but it is also used for the other directories for releases, experimental and snapshots.

A very quick overview of the existing setup:

  1. Developer uploads new distfile directly to dev.gentoo.org.
  2. The master-distfiles box pulls from dev.gentoo.org hourly.
  3. The master-distfiles box checks every ebuild, and downloads missing distfiles from their primary URI if they do not exist. The daily distfile report is also created at this point.
  4. Every hour, the cluster master of ftp.osuosl.org pulls the latest content from master-distfiles. (Averages 240MB/day of traffic).
  5. The OSL FTP cluster master (in Oregon) pushes to it's slave locations in Atlanta and Chicago.
  6. All distfiles mirrors pick up their content from one of the FTP nodes - Internet2-connected hosts are directed via DNS to an Internet2-connected slave for performance.

Each of the distfiles mirrors has about 140-160MB of upstream traffic every day (including both the new files and the rsync overhead for scanning). If there are no files changed, the rsync traffic for a directory scan is 1-2MB. While this isn't a lot of traffic, it's very spiky, as mirrors tend to be on fast links.

The new weekly builds from the Release Engineering team will probably be adding another 1.3GB per week, staggered as one arch per day.

I got a small subset of the logs from the OSU FTP cluster for processing some of these statistics. They cover the 24 hour period of 2008/08/07 UTC. It does not have data of which traffic went via Internet2, and I've grouped the sources by country code (using IP::Country::Fast from CPAN).

CC OutBytesCount, [Notes]
South America
AR 1498379141
BR 1498405221
==2996784362
Europe
AT 3202290562
BA 1498404221
BE 1464739661
BG 2199886072
CH 1496743121
CZ 8062803705
DE 149092997310
DK 2295154041
EE 1360037741
ES 4493037003
FI 1387115261
FR 7996356615
GB 3960190613
GR 4172227743 [1]
IS 1360037741
LV 1499118641
NL 4519136003
NO 1499088261
PL 6957242141
PT 2840207112
RO 3668540933
SE 4496643343
SK 1498405681
== 868367059055
Asia/Oceania
AU 2974020902
JP 4493696853
KR 4509289423
RU 1972457562
SG 1356810941
TH 1358357761
TW 4927311704
== 215919451316
North America:
CA 7429692847
US 317491485824
== 391788414231
Middle East:
IL 1935272832
KW 1497725501
== 3432998333

Grand Total:
==15403727514 bytes107

[1] One Greek mirror was excluded from the traffic and counts, as this was their catchup sync with 7Gb of traffic after some hardware-related downtime.

As a bit of analysis, I think that more than half of our mirrors (Europe, Middle East, RU) would benefit from having a box to sync against in Europe.

Leave a comment

gentoo mirrors stats: a rsync.gentoo.org box (December 16, 2008, 21:37 UTC)

Robin Johnson

I was doing some statistics about Gentoo mirrors to see about future plans, and thought that the indirect crowd that read my blog via the various aggregators might be interested in numbers.

These are the traffic for boobie.gentoo.org, which is a newer box in the official rsync.gentoo.org box directly maintained by the Infrastructure team. Hardware specs are 2x Xeon 3050 @2.13Ghz, 4GB RAM. Disk is mostly irrelevant - the rsync workload is served purely from RAM (tail-packing reiserfs, backed via loop device pointing to a file on tmpfs).

Inbound traffic is spiky, but does not exceed 10Mbit by more than a little bit - we can the inbound rsyncs from the rsync1 master to 10Mbit. Outbound traffic varies between 4Mbit and 9Mbit, with an average around 6-7Mbit.

DateInBytesInBPSOutBytesOutBPS
2008-12-0124510353412836859523455410688928
2008-12-0223251768542691154877643699635157
2008-12-0321678292492509050850785431588550
2008-12-0422273424352577950823673845588236
2008-12-0521820142142525450558268814585165
2008-12-0620394684352360447476164351549492
2008-12-0719065284552206650327689263582496
2008-12-0821277927972462752759944753610647
2008-12-0923277314192694156661069093655799
2008-12-1022462625702599852107127647603091
2008-12-1123025726732665053602727876620401
2008-12-1220771853122404147108235487545234
2008-12-1321621937092502550807583749588050
2008-12-1416987667881966143678479520505537
2008-12-1523701326092743258353939353675392

Leave a comment

Steve Dibb a.k.a. beandog (homepage, stats, bugs)
mplayer-resume-1.6 (December 16, 2008, 20:41 UTC)

Steve Dibb

I just finished updating and pushing mplayer-resume-1.6 into the tree.  I’ve actually been sitting on the update for a long time, just never got around to releasing it.

There has only been a small change — the script will check mplayer’s exit code to see if it died on something.  I added it since I hit a bug on my frontend where some files wouldn’t resume, and it would delete my resume point anyway.  Kind of annoying.  This update fixes that, and really, that’s it.

Leave a comment

Diego Pettenò a.k.a. flameeyes (homepage, stats, bugs)
Follow up on the tests; looking for CC-BY content (December 16, 2008, 20:24 UTC)

Diego Pettenò

I’ve written about my idea of writing my own testsuite for uif2iso and using CC-BY-NC-SA music from Jamendo to do the trick, unfortunately it seems like I cannot do that:

If your project is non commercial, you can use the music without paying for a license as long as your project respects the rules of the Creative Commons (http://www.jamendo.com/en/creativecommons). It means that Gentoo has to be distributed under a Creative Commons License. If not, then you do have to buy a license, even if it is a non-commercial project.

Seems like mere aggregation becomes a problem too. Coincidently there was a poll on the Creative Commons site about the NC clause the other day, and I already expressed there my doubts about the “mere aggregation” case.

As it is, it becomes a problem for me to use the album by Countdown for my testsuite project, which actually makes me a bit sad since I really liked the album and would have liked to give the testsuite a fun character with something I liked. I guess I’ll have to find something else.

So if somebody knows where I can find some CC-BY content, I’d be glad to hear it; my idea was to use an image and an audio file, just to have two different size magnitudes. Metal would probably be more acceptable, but I guess anything at this point works. My reason for looking for those is that they can be easily used for other type of tests, for instance mp3 or OggVorbis files can be used for tag reading/writing programs for testing purposes.

On a different note, thanks a lot to the (anonymous) user who sent me The Hitchhiker’s Guide To The Galaxy radio series, tonight I have something to look forward to when I’m done filing bugs!

Steve Dibb a.k.a. beandog (homepage, stats, bugs)
planet admin stuff (December 16, 2008, 17:49 UTC)

Steve Dibb

I’m working on cleaning up some planet stuff again, and for Planet Larry I made a small policy shift — if your feed is spitting out partial feeds, I’m not gonna accept it anymore.  Currently it only affected like two people, I think (unless I missed someone), so if you wanna get back on … fix it. :)

I realize it’s a small case of me shoving my own preferences down the pipe, which is something I avoid as much as possible.

If I’m going to add on one more asinine policy requirement, I figure it’s fair that I take one away as well, though most people probably don’t notice or care about this one. I prefer to have the blogger’s name show up as the blog title, and have in just about every case (minus one) forced them to be that way.  But, I’ve changed my mind there as well, so if you want to customize the name on the feed, I can do that for you.

An example would be that my blog (wonkablog) shows up in the list as Steve Dibb (my name).  So, if you wanna change it from displaying your name to something else, just drop me an email.

And, as always, more bloggers are welcome to sign up. :)

Leave a comment