Gentoo Logo
Gentoo Logo Side
Gentoo Spaceship

Contributors:
. Alec Warner
. Alexander Gabert
. Alexis Ballier
. Ali Polatel
. Anant Narayanan
. Andreas Proschofsky
. Andrew Gaffney
. Andrew Ross
. Ben de Groot
. Benedikt Boehm
. Benjamin Smee
. Bernard Cafarelli
. Bjarke Istrup Pedersen
. Brent Baude
. Caleb Tennis
. Christian Faulhammer
. Christian Heim
. Christian Zoffoli
. Damien Krotkine
. Daniel Drake
. Daniel Gryniewicz
. Daniel Ostrow
. David Shakaryan
. Davide Italiano
. Dawid Dawid Węgliński
. Diego Pettenò
. Donnie Berkholz
. Doug Goldstein
. Fernando J. Pereda
. Gentoo News
. Grant Goodyear
. Greg KH
. Guillaume Destuynder
. Gunnar Wrobel
. Gustavo Felisberto
. Hanno Böck
. Hans de Graaff
. Ioannis Aslanidis
. Jan Kundrát
. Jeffrey Gardner
. Joe Peterson
. Joe Sapp
. Jonathan Smith
. Joseph Jezak
. Josh Saddler
. Joshua Jackson
. Joshua Nichols
. José Alberto Suárez López
. Krzysiek Pawlik
. Lance Albertson
. Lars Weiler
. Luca Barbato
. Luca Longinotti
. Luis Francisco Araujo
. Marcus Hanwell
. Marius Mauch
. Mark Kowarsky
. Mark Loeser
. Markus Ullmann
. Mart Raudsepp
. Matthias Geerdsen
. Michael Marineau
. Michal Januszewski
. Mike Doty
. Mike Pagano
. Ned Ludd
. Olivier Crête
. Otavio R. Piske
. Patrick Kursawe
. Patrick McLean
. Paul de Vrieze
. Peter Weller
. Petteri Räty
. Pierre-Yves Rofes
. Piotr Jaroszyński
. Remi Cardona
. Renat Lumpau
. Rob Cakebread
. Robert Buchholz
. Robin Johnson
. Rodrigo Lazo
. Ryan Hill
. Shyam Mani
. Steev Klimaszewski
. Stefan Knoblich
. Stefan Schweizer
. Steve Dibb
. Stuart Longland
. Sune Kloppenborg Jeppesen
. Sven Vermeulen
. Sven Wegener
. Thilo Bangert
. Timothy Redaelli
. Tiziano Müller
. Tobias Klausmann
. Tobias Scherbaum
. Yuval Yaari
. Zack Medico
. Zaheer Abbas Merali
. Zhang Le

Last updated:
May 16, 2008, 11:05 UTC

Disclaimer:
Views expressed in the content published here do not necessarily represent the views of Gentoo Linux or the Gentoo Foundation.


Bugs? Comments? Suggestions? Contact us: planet@gentoo.org

Powered by:
Planet

Welcome to Planet Gentoo, an aggregation of Gentoo-related weblog articles written by Gentoo developers. For a broader range of topics, you might be interested in Gentoo Universe.

May 16, 2008
Josh Saddler a.k.a. nightmorph (homepage, stats, bugs)
Decibel hits the tree (May 16, 2008, 06:49 UTC)

Josh Saddler

It's official! Decibel Audio Player is now in Portage. Thanks to aballier for adding it, and thanks to everyone who provided feedback & testing.

Yay for yummy audio goodness!

Leave a comment

Gentoo Foundation Reinstated (May 16, 2008, 05:20 UTC)

Josh Saddler

In case you missed it, the Gentoo Foundation has been reinstated, so we're all nice'n'legal again.

Thank you, trustees, for all your efforts!

You can view our updated paperwork with the state of New Mexico here.

Leave a comment

May 15, 2008
Hanno Böck a.k.a. hanno (homepage, stats, bugs)
Tic Tac Toe over Jabber (May 15, 2008, 22:45 UTC)

Hanno Böck Tic Tac Toe in gajimOne of the missing features in jabber most commercial instant messaging systems have are small games one can easily play over them. Till now, because just yesterday, gajim got support for Tic Tac Toe over jabber.

Which reminds me tic tac toe was one of the first »programming projects« I did in the past, some day I'll have to dig out the sources and publish them. My next wish would be something like tetrinet over jabber.

Gajim-svn ebuild is in my overlay at http://svn.hboeck.de/overlay/.

Leave a comment

Slacker Council (May 15, 2008, 20:44 UTC)

Fernando J. Pereda

As per http://archives.gentoo.org/gentoo-dev/msg_19892c04f0e6cf4c24629f13718e45cb.xml there was a meeting council scheduled for 20:00 UTC today (that's a bit more than half an hour ago).

For some reason, only amne and dberkholz showed up. As per GLEP39's Specification:

  • If any meeting has less than 50% attendance by council members, a new election for all places must be held within a month. The 'one year' is then reset from that point.

What's the council going to do? Place your bets.

— ferdy

Leave a comment

Gunnar Wrobel a.k.a. wrobel (homepage, stats, bugs)

Gunnar Wrobel

If you look at the current ebuild you might wonder what the fuss might be about... It is a pretty empty package.

But the p@rdalys project will form the core for Kolab2/Gentoo-2.2. It will certainly replace net-mail/kolabd and might include some other packages, too.

The idea is to allow you to install Kolab2/Gentoo-2.2 with two simple steps:

emerge app-admin/pardalys
pardalys

Of course there is still a certain way to go until it will actually work that way. And this easy setup is actually just meant as a nice side effect and is not the main point of starting the project. I'll start explaining this package in greater detail once I push more code into it.

For now the link to the project page will be all I can provide.

Currently it might not be clear what the package will actually be about but if people wish to contribute to the project at a later time point you should go visit the git repository on GitHub. This git repository should serve as a scratch repository used for easy sharing and patching of the code. The reference repository on the other hand will be kept in subversion on SourceForge and will be used for packaging.

More on the whole story once there is more code.

Diego Pettenò a.k.a. flameeyes (homepage, stats, bugs)

Diego Pettenò

Maybe it’s somehow not clear in the documentation, as I see this kind of mistakes happens quite a lot. The obvious reason is that sometimes it happens you oversee something, but as a lot of these problems happen on packages maintained by the same person or group, I start to think this is not clear enough. In particular I’m afraid the reasoning why you should make sure that the two variables are filled in properly is not clear enough.

First of all, at the moment we unfortunately have only two dependency variables, DEPEND and RDEPEND. Cross-compiling and proper multilib support would need two more, splitting both in executed and linked-against packages, as the first just need to be available in some executable form, while the latter need to have the same ABI as the program being built. But that’s a story for another day.

So we’re back to DEPEND and RDEPEND. The first is the set of packages that has to be available during build, as this is what we have, it has to have both tools used to build, and library linked against. RDEPEND instead is the set of packages that has to be available during execution of the program. This means it has to have the libraries linked against, if shared – which is what we want with pretty rare exceptions – and the programs that are ran by the program at runtime.

Programs like unzip and other tools used to extract the source archive as DEPEND and not RDEPEND. Unless they are executed by the software itself, like Amarok does (alas), in which case they are RDEPEND and not DEPEND. Build tools, like pmake or build systems like imake, scons or cmake, generators like flex and bison, rust or swig are DEPEND only, as they are used during source build. Software used for the tests is DEPEND only and subject to the test USE flag.

Fonts, when used by a graphical application, are most likely useful only at runtime. Rarely tools require them at buildtime, but it happens, for instance when you use gd to generate some images for the documentation. By the way, Doxygen is a build-only dependency, and should always be subject to the doc USE flag!

Most, but not all, packages for interpreted languages like Perl, Python and Ruby extensions have little or no build dependency (they are not built, most of the times) but may have lots of runtime dependencies. And might have build-time dependencies under test USE flag for the tests.

When a package install no shared library, it’s usually just one of the two between DEPEND and RDEPEND, as rarely the software checks at build-time what it will execute at runtime, which is quite good for us, as we’ll see. If the configure script for a package just checks if the tools are available, but doesn’t use them, it’s a good idea to either fool it through cache values, or change it so that the tests are not done. Again, we’ll see why.

So okay, why should we spend time to make sure the dependencies are right? Shouldn’t people install the packages anyway for both build and runtime? Well, sort of. When you prepare a filesystem image for another system through ROOT=, you usually just want the packages needed at runtime, not at buildtime. This means you don’t want autotools, you don’t want pkg-config, you don’t want lzma and so on. Often this is done through cross-compile, and this saves you cross-compiling for instance Perl – which, if you didn’t know, is not cross-compilable at this stage – but it might also apply to systems that are maintained through binary packages, as you don’t want to have to distribute extra packages, especially if the target machines don’t have as much space as you’d like them to be.

But having proper DEPEND/RDEPEND also helps testing. For instance, if I’m to check whether a given package builds after a flex bump, I don’t want to have to install all its runtime dependencies, as that would stop me from using the -B option of emerge to try just the build.

This also means that if you know for sure that the package calls the linker with a given library at build-time, but the library is never ever used (and for instance --as-needed drops it), you should be patching the package not to try to use it at build-time, or use it only conditionally, so that you don’t depend on a package that is not useful at all.

Please, when you write the dependencies of an ebuild, think a lot how to get them right, make sure that datafiles are only runtime dependencies unless they are used for build stuff too, make sure that build tools are not leaked at runtime. Basically, look at the package from every angle and make sure that you actually get the thing how it is supposed to be.

Leave a comment

May 14, 2008
Diego Pettenò a.k.a. flameeyes (homepage, stats, bugs)

Diego Pettenò

… you don’t.

I’m not referring to PAM-less systems, I sincerely don’t care much about those. I’m referring to the dependencies of packages, which sometimes require sys-libs/pam even though they work fine with any PAM implementation.

My idea at the moment is to check for packages that can be ported to virtual/pam without code changes. I already found at least two, and there probably are others. The ones not working, are usually looking for the misc_conv symbol, which is provided by the libpam_misc library “proprietary” to Linux-PAM.

This falls into two lines of work for me, the first is that I’m trying to clean the area for Seraphim Mellos, who’s working on Summer of Code to support OpenPAM, the second is for Gentoo/FreeBSD, as there Linux-PAM is not used. Additionally, the less references directly to sys-libs/pam I have in the tree, the easier it will be when I decide to move it to sys-auth/Linux-PAM as it owe to be called.

Again, I wonder why am I stuck with this task as I usually don’t need PAM at all myself. I hope that getting a token changes something, so that I’d just have to key in the PIN of the token and be done with login too, but even that it’s a bit of a stretch for a standard user.

Anyway if somebody would like to help out, it would be nice if the packages using misc_conv were ported not to need it, most software doesn’t need it anyway. At the moment I know that sys-apps/qingy, sys-apps/busybox and app-misc/away don’t support OpenPAM as they require pam_misc.

Leave a comment

Greg KH a.k.a. gregkh (homepage, stats, bugs)
what I am doing, RIGHT NOW (May 14, 2008, 21:14 UTC)

I've been watching twitter for a while now, amused at the ability for it to keep people appraised of what you are doing at the moment, if they really care. I didn't think it was really worth it.

Until I read this post last night which was linked off of some site that I forgot (probably reddit but I did think it was from the every wonderful Arachaia, which if you are a programmer, you should be paying attention to.)

I just couldn't resist...

So, if you want to see what I am doing, RIGHT NOW (well what I just did, it waits for the command to complete before sending it off to twitter), you can follow along right here.

I'm only enabling it on a few of my terminal windows for now, watching me constantly run mutt and offlineimap would get a bit boring.

I wonder how long it's going to be before I type in my password accidentally to this thing. Or until twitter bans me. Any odds on which is going to happen first?

I pity anyone who subscribes to this twit feed, they are going to start hating me very quickly, like the Portland, Oregon local feed already has...

Joshua Nichols a.k.a. nichoj (homepage, stats, bugs)

Joshua Nichols

Since I've been less active packaging for Gentoo recently, I've been trying to at least lurk in #gentoo-dev-help and give a helping hand with packaging. If I can't help them directly, at least I try to give them a gentle shove towards something that can.

As a result, I've been collecting various bits of documentation about how to package stuff. This post is a result of this, as well as some other digging around I did to flesh it out.

General development documentation

Channels of communication

Lots of projects and herds within Gentoo have their own mailing lists and irc channels.

Language-specific

Packages written in different languages often have their own way of dealing with packaging:

Herds

Same goes for herds.

Other stuff

These don't fall into any other categories, but are still useful.

Diego Pettenò a.k.a. flameeyes (homepage, stats, bugs)
Even little differences count (May 14, 2008, 16:20 UTC)

Diego Pettenò

If you ever find yourself relying for any reason on the behaviour of Microsoft Excel, make sure you never ever change its version. Seems like I had some problems with my job because the version of Microsoft Excel I bought is not the same as the one they are using at the office.

Also, if you want to shut up Valgrind’s warnings to make life easier for people developing against a given library, it’s time you hear about suppression files, rather than trying to change code you don’t know enough about.

Seems like a little difference in the patches applied by Debian (and Ubuntu) made OpenSSL vulnerable like it never was. Even my keys has to be considered compromised, so I had to change them on every service I use. This means Alioth, Gentoo’s Infra, GentooExperimental, SourceForge, Berlios, Savannah, Rubyforge, Launchpad (don’t ask), KDE, and of course my own server.

I think yesterday will be remembered for the years to come as a critical day. It’s like a city of the size of Milan starting to change the door locks for all the apartments at once. I’m surprised there hasn’t been more confusion.

Myself, I’m starting to consider some other kind of authentication, as the passphrases I use for the keys are quite long, and I’m tired of having to key it in every time I start the system. SmartCard and authentication tokens start to look quite nice, and it would be a nice test for Gentoo’s PAM infrastructure.

It’s unfortunate that Alon, who as far as I remembered managed these things, left Gentoo yesterday :/ Although I suppose that if I am to start looking into these things I might be able to lend a hand.

At the moment, I’m oriented toward an authentication token, as that would make it possible for me to have it around on the laptop too at any time. Strangely enough, I found an Italian producer, Eutronsec, and their CryptoIdentity token seems to be supported by pcsclite.

SmartCards are more likely useful in an organisation where you’d have one reader on a given system and many many cards around. Here I’d just have one reader and one smartcard if I did go that way. If the Italian ID card was a SmartCard I could have chosen that, but at the moment it doesn’t look like a convenient idea.

Unfortunately it seems like Eutronsec only sells to other companies, I’ve mailed them asking if they could sell to single entities, but I haven’t received an answer yet. I’m afraid I’ll have to start looking for alternatives if I want to proceed on this road.

On a totally different topic, but still related to “little differences”, I’m having difficulties finding cups that I can use with my espresso machine. Usual teacups that my family uses for coffee (yeah they are quite larger for coffee, we’re used to those though, a six servings Moka is good for three here) tend to be too thin at the bottom and large at the top, so the coffee spills a lot, especially when doing a hot boiling espresso. Mugs are quite better for the task, but all the ones I have here are little less than 9.5 centimetres (little more than three and a half inches), and require to be skewed to enter the espresso machine.

The idea would be an 8.5cm cup, but I can’t seem to find them in the shops around here, they are either 6 centimetres, which is way too short, or they are the “usual” mug size. In a Chinese shop around here I found some nice cups that are about 12cm actually, I was tempted to take one to hold the milk for cappuccino, actually, but they are quite too high for the espresso machine.

Leave a comment

Ben de Groot a.k.a. yngwin (homepage, stats, bugs)
Berkano overlay news (May 14, 2008, 10:46 UTC)

Recent changes in berkano overlay:

  • I cleaned out some cruft, packages that in my opinion weren’t useful anymore
  • I removed the older versions of hitchhiker-sources, because the kernel devs recommend using only the latest, for security reasons
  • I added a live git ebuild for Arora, the new Qt4 WebKit browser

Arora icon Arora needs >=qt-4.4.0, which you can get here as a tarball for use in your local overlay (if it breaks anything, you get to keep the pieces), or from one of the other overlays that provide up-to-date or development snapshots of Qt4. I’ve been told qt-4.4.0 release should be added (as hardmasked) to portage later this week.

I’m actually looking into adding more Qt4 based applications, either to the overlay or portage. My goal would be to get together an as complete as possible desktop using just (or mainly at least) Qt4 programs. The most obvious missing part is a Window Manager. Cutebox was a promising project, but has been abandoned even before its first release. Anyone that would resurrect it and bring it to a useful form, would win a guaranteed spot in my personal Hall of Heroes.

ShareThis

Leave a comment

May 13, 2008
Steve Dibb a.k.a. beandog (homepage, stats, bugs)
blu-ray dvd drives (May 13, 2008, 19:20 UTC)

Steve Dibb

An interesting post came up the other day on the Gentoo forums about how to rip Blu-Ray discs on Linux. Short summary: I have no idea if it’s possible, and the original poster is still investigating. It has gotten me thinking though. The Blu-Ray player that I want to get it is $600, and it looks like it’s being phased out of production anyway, so why not get a disc drive instead and rip the movies? It’d save me some money, and I’d eventually buy one anyway.

Well, the questions that come to mind are, will the software actually work, will the drive firmware let me do that, and am I going to have to use Windows?

I haven’t done any research at all, mostly because I can’t afford to buy a DVD drive right now, but the whole thing does have me curious. I always assumed there was no way to rip the stuff under Linux, but I haven’t gone looking for possible solutions either. The only thing I am sure about though, is that once ripped, you can play the content just fine. At least, I think so. I’m not positive about the HD audio codecs, pretty sure about the video ones though.

I tend to buy hardware first and figure out how to get it working second, but because the DRM is so finicky in this case, I really don’t want to take that approach and be out a couple hundred bucks.  In the meantime, I really wish I could at least demo the stuff at home.  That would be cool.  The only 1080p content I’ve seen so far is the movie trailers I’ve downloaded from Apple’s website.  I gotta say that stuff looks pretty good.

Leave a comment

Use(d) Debian? Check your keys! (May 13, 2008, 14:59 UTC)

Robert Buchholz

If you run any kind of server, especially Debian or Ubuntu, or grant users access to your server, you might want to read the Debian Security Advisory DSA-1571-1 or Ubuntu’s Security Notice USN-612-1 for CVE-2008-0166, and check your encryption keys:

It is strongly recommended that all cryptographic key material which has
been generated by OpenSSL versions starting with 0.9.8c-1 on Debian
systems is recreated from scratch. Furthermore, all DSA keys ever used
on affected Debian systems for signing or authentication purposes should
be considered compromised; the Digital Signature Algorithm relies on a
secret random value used during signature generation.

The first vulnerable version, 0.9.8c-1, was uploaded to the unstable
distribution on 2006-09-17, and has since propagated to the testing and
current stable (etch) distributions. The old stable distribution
(sarge) is not affected.

Affected keys include SSH keys, OpenVPN keys, DNSSEC keys, and key
material for use in X.509 certificates and session keys used in SSL/TLS
connections. Keys generated with GnuPG or GNUTLS are not affected,
though.

This vulnerability is caused by a patch shipped in Debian, Ubuntu, and other derivatives. Gentoo’s OpenSSL version is not affected, but everyone should check user-provided public keys (such as OpenSSH’s authorized_keys) using the Debian/OpenSSL Weak Key Detector.

Update: Ben Laurie of OpenSSL is making a point that Vendors Are Bad For Security, which I would not follow in that general form. What I have to grant him: Mechanisms of peer review must be employed properly and patches discussed with upstream. If you follow this philosophy, Vendors Are Good For Security.

Leave a comment

Alec Warner a.k.a. antarus (homepage, stats, bugs)
Growing Pains: Part Deux (May 13, 2008, 11:04 UTC)

Alon Bl has left the project (the mail hasn’t hit archives.gentoo.org or I’d link you). In his resignation he stated that he had problems with other developers and that was a big part of his leaving. He talks about a lack of leadership, of goals, and of user experience; of good developers leaving, and many other things.

First off, sorry to see you go Alon and I appreciate the work you did for Gentoo.

I empathize with your views that you expressed in your mail. These are difficult problems and it is not a quick change to fix them and you (nor I) may not agree with the solution. I think Gentoo has many goals; however they are poorly expressed and because of that there is no way to essentially ‘meet’ them.

The problem we previously had was that ‘everyone thought they had a say’ and that appeared to not work. The council is attempting to modify that view somewhat and it is a slow process. You can’t just elect some dictator guy to make arbitrary choices and set arbitrary goals and metrics. You need a critical mass of folks who are willing to dedicate time to meet those goals. You must also be willing to accept losses of developers who disagree with this new vision.

I know many developers who left because they thought change was impossible. I used to think that way. One of the things that getting a real job has taught me is that changes like these are not impossible, they are just not easy. You need folks dedicated to actually solving them and not just talking about it. People to set goals and actually sell them to the community as worthwhile and convince others to work towards them.

Maybe we used to have that calibur of person; maybe we never did; maybe we never will. But you are correct in one thing. Someone has to try. If everyone fears making the necessary changes than Gentoo will stagnate and probably die.

Leave a comment

Donnie Berkholz a.k.a. dberkholz (homepage, stats, bugs)
How to run an effective meeting on IRC (May 13, 2008, 09:08 UTC)

Donnie Berkholz


Since my election to the Gentoo Council, I’ve become the de facto meeting chair and secretary. Over the past 6 months or so, I’ve learned a lot about what works well in online meetings (often by virtue of doing the opposite). By no means have I mastered this, but here’s some of my discoveries along the way.

What works well:

  • Send out a draft agenda in advance (say, 1 week). This helps avoid confusion and disorganization at the meeting, and it also allows you to have no “open floor” section at all, because all topics should have come up when you posted the draft. Settle on a final agenda a couple of days in advance.
  • On the draft, say who should attend the meeting to discuss each topic.
  • Be specific about the topic, so you stay focused during the meeting.
  • For each topic, have a very specific goal of what will be accomplished at that meeting. If it’s a decision, exactly what will the vote be? If it’s a discussion, what points do we want to get out of it, and why is it happening during the meeting instead of on mailing lists?
  • Prepare. All of the information needed to make a decision should be readily available by the time the meeting comes along. To aid this, say on the draft agenda what information will be needed.
  • Stay relentlessly on topic. Cut off diverging threads early on, before everyone gets involved.
  • During the meeting, get an action plan for each topic: What’s the next step? Who’s responsible for it? When will they have it ready? Make sure the person responsible personally commits to this–don’t just assign it to them.
  • Take notes, and post a public summary. This summary informs and reminds people of the progress made and what progress needs to be made next. By being posted publicly, it also allows for discussion, clarification and correction.
  • Keep track of unresolved topics, and keep bringing them up over and over so they can’t slip through the cracks.

What works poorly:

  • Request topics on a mailing list, but don’t collate them into an agenda until after the meeting’s started.
  • Do your best to ensure that people relevant to a topic don’t even know it’s going to be discussed, or don’t tell them what information you need from them.
  • Have vague topics, so nobody’s really sure what you’re supposed to be talking about or what you want to get out of it. Feel free to branch out into anything that seems related, or really anything at all.
  • If a topic isn’t resolved by the end of the meeting, forget about it. If it’s important, it’ll come up again, right?
  • Don’t tell anyone what the results of the meeting were. If you have to release something, make it as hard to comprehend as possible, like an IRC log instead of a summary.

It took a lot of pain and wasted time for me to figure out the value of doing things right, and I’m still working on getting some of the above points right, so I want to save you that same pain.

Do you have any more points to add? Please do so in the comments. Thanks!

Josh Saddler a.k.a. nightmorph (homepage, stats, bugs)
Decibel .10 comes to Gentoo (May 13, 2008, 00:25 UTC)

Josh Saddler

The newest Decibel Audio Player is out, and it fixes several bugs and adds some new features.

The ebuild has been updated, so get it here!

Leave a comment

May 12, 2008
Steve Dibb a.k.a. beandog (homepage, stats, bugs)
planet, packages: small bugfixes (May 12, 2008, 23:21 UTC)

Steve Dibb

I took a few minutes today and cleaned up a few small bugs on Planet Larry and friends.

One thing I get asked for every now and then is if I have archives of past posts. Well, I do now. I just copy the HTML file of the last post to $date one each run. A simple and unelegant solution. I’ll be doing the same thing for Planet Gentoo soon.

Also, fixed the FeedBurner link on the main page — I didn’t even realize it was broken. While I was at it, I created one for Universe as well.

On the packages website, I finally fixed it so you can search against just packages again. That’s been annoying me for a while. By default the search is way too wide, I think. It will search the full atom, the package description and the package name. I have to do the package name twice because of regular expressions (starting with, ending with, exact matches, etc.). And there’s still no simple way to search for packages containing multiple words, which is also an annoying little bug. Advanced searches for GPNL and Packages has been something I’ve wanted to do for a long time, but have been putting off since I started the projects. Sheesh. Every time I sit down and start to poke at it, though, I realize just how big a beast it is, based on what I’d like to accomplish. I really need something for the interim, though.

Anyway, I better quit before this post gets any more boring. One last thing — we can use more users who are Gentoo users and have a blog on Planet Larry. Just drop me an e-mail and I’ll get you setup.

Leave a comment

Alec Warner a.k.a. antarus (homepage, stats, bugs)
More gStats (May 12, 2008, 11:50 UTC)

I have not worked on gstats in a while and it definately needed some improvements. I found myself struggling with the UI a lot and after chatting with Graff on irc I decided to drop the webUI entirely and go RESTful. Ewwww REST….

I actually hate REST because all of its supporters claim it is ‘Teh awesome!’ but so very few can tell me why in any sort of convincing manner. ‘It doesn’t have all the overhead of SOAP!’, ‘It uses HTTP!’…these are not convincing arguments. Anyway I figured I’d give it a try.

gStats now has no real webUI. Most of the data retrieval functionality is written (aka HTTP GET) using old data in the cache so you can do queries like /api/cp/sys-apps/portage and it will return some XML that describes sys-apps/portage’s installation history. Also available is /api/arch/x86 which will return some XML describing x86’s installation info.

I am working on writing POST methods; although I noticed I have a slight issue. Users have hosts with 100’s of packages installed. REST is kind of limiting in this regard; right now I store arches and cps (category/package strings) and via the rest interface you can POST new arch data (not really allowed at this point as it would overwrite old data in my reckoning) and you can POST new cp data (being worked on). But the cp POST model right now kind of takes a cp as a key; maybe I should drop that requirement.

Currently:

POST /api/cp/sys-apps/newpackage
some XML describing a single cp named sys-apps/newpackage

New Idea:

POST /api/cp
some XML describing multiple CPs

Leave a comment

May 11, 2008
Diego Pettenò a.k.a. flameeyes (homepage, stats, bugs)
Today's mass filings (May 11, 2008, 19:35 UTC)

Diego Pettenò

Our bugzilla today really had an hard day, I’ve started filing bugs for build tools added to runtime-depend of packages, the list is awfully long, and “My Bugs” search now lists over 300 bugs.

Why did I do this? Well, we really need to keep the tree clean on these things so that --depclean works as expected. But mostly my interest is for final system deployment. On my vserver, as well as on a final target for a cross-compiled embedded system, you don’t want packages like flex, bison or swig, and this is quite possible if the packages are not in RDEPEND, when using ROOT= or binary packages.

Now, the work today was boring, long and a huge pour of time. I did it gladly without being forced to, I’m fine with it and I’m not expecting anything in particular for what I did. If I didn’t want to do it, I could have just reported the issue, and let someone else doing the thing. This is what volunteer-based work is about.

Why do I say this? Well there seems to be people to think that even volunteer work should apply the same rules as businesses. While sometimes you might make use of strategies designed for businesses, like marketing your project better and similar, volunteer works and businesses have vast differences.

Sure there are free software projects ran as businesses, but that usually involves people paid to do their job as the core rather than full volunteer-based approaches. So please if you think to run a volunteer project as a business, well think twice or expect serious trouble from the volunteers.

So please remember that there are lots of people who are fine to work on a volunteer project that would probably start to be quite nervous if you run it as a business, okay?

Leave a comment

Donnie Berkholz a.k.a. dberkholz (homepage, stats, bugs)
Distributions in the Summer of Code (May 11, 2008, 09:34 UTC)

Donnie Berkholz


I wrote an article discussing the role of distros in Google’s Summer of Code for this week’s LWN.net. Here’s an excerpt to whet your appetite:

Aren’t they just writing packages? What would they do with a Summer of Code project?

Read on to find out.

May 09, 2008
Alexander Gabert a.k.a. pappy (homepage, stats, bugs)

okay here we go

>>> Regenerating /etc/ld.so.cache...
>>> sys-libs/gxslibc-2.6.1-r2 merged.

>>> No packages selected for removal by clean
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.
* Regenerating GNU info directory index...
* Processed 87 info files.

TMPFS chroot001 miranda ~ # export STATIC="-fstack-protector-all"; gcc-3.4.6 "${STATIC}" -fstack-protector-all -o vuln-stack vuln-stack.c && file vuln-stack && readelf -s vuln-stack | egrep "__guard|__stack_smash"; ./vuln-stack 1234567891234567; einfo "return code: ${?}"; echo; gcc-3.4.6 "${STATIC}" -fstack-protector-all -o ssp_entropy ssp_entropy.c && file ssp_entropy && ./ssp_entropy
vuln-stack: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.18, dynamically linked (uses shared libs), not stripped
2: 08049698 4 OBJECT GLOBAL DEFAULT 23 __guard@GLIBC_2.3.2 (3)
4: 00000000 30 FUNC GLOBAL DEFAULT UND __stack_smash_handler@GLIBC_2.3.2 (3)
78: 08049698 4 OBJECT GLOBAL DEFAULT 23 __guard@@GLIBC_2.3.2
80: 00000000 30 FUNC GLOBAL DEFAULT UND __stack_smash_handler@@GL
* return code: 46

ssp_entropy: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.18, dynamically linked (uses shared libs), not stripped
__guard: [[0x288a2b8c]]

TMPFS chroot001 miranda ~ # export STATIC="-static"; gcc-3.4.6 "${STATIC}" -fstack-protector-all -o vuln-stack vuln-stack.c && file vuln-stack && readelf -s vuln-stack | egrep "__guard|__stack_smash"; ./vuln-stack 1234567891234567; einfo "return code: ${?}"; echo; gcc-3.4.6 "${STATIC}" -fstack-protector-all -o ssp_entropy ssp_entropy.c && file ssp_entropy && ./ssp_entropy
vuln-stack: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.18, statically linked, not stripped
1346: 0804f810 18 FUNC GLOBAL DEFAULT 3 __stack_smash_handler
1554: 080bc370 4 OBJECT GLOBAL DEFAULT 16 __guard
* return code: 46

ssp_entropy: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.18, statically linked, not stripped
__guard: [[0xe686ece4]]

i invented return code 46 as SSP failure because i could not find a list of valid exit codes (unless segfault which is 127) at google.

-Alex

Leave a comment

Zack Medico a.k.a. zmedico (homepage, stats, bugs)

Blocker Conflicts

If one package blocks another package, the two packages conflict such that they cannot be installed simultaneously. These conflicts are often due to file collisions. In some cases, packages that block each other can be temporarily installed simultaneously. In order to resolve file collisions that occur between two blocking packages that are installed simultaneously, the overlapping files must be removed from the contents list of the package which was installed first.

Some cases may exist such that temporary simultaneous installation of blocking packages will cause some sort of problem. However, this type of solution will only be chosen for blockers that can not be satisfied in any other way, such as by simple adjustment of merge order. In addition, this type of solution will not be chosen if a blocking package will overwrite files belonging to packages from the system set, or packages that are runtime dependencies of Portage itself. These constraints serve to limit the probability that a chosen solution will cause an unforeseen problem.

gtk-doc-am vs. gtk-doc-1.8-r2

These are the packages that would be merged, in reverse order:

Calculating dependencies... done!
[blocks b     ] dev-util/gtk-doc-am (is blocking dev-util/gtk-doc-1.8-r2)
[uninstall    ]  dev-util/gtk-doc-1.8-r2  USE="emacs -debug -doc"
[nomerge      ] x11-libs/gtk+-2.12.9-r2  USE="X cups jpeg tiff xinerama -debug -doc -vim-syntax"
[ebuild  N    ]  dev-util/gtk-doc-am-1.10-r1  0 kB

Total: 1 package (1 new, 1 uninstall), Size of downloads: 0 kB
Conflict: 1 block

coreutils vs. mktemp

These are the packages that would be merged, in reverse order:

Calculating dependencies... done!
[nomerge      ] sys-apps/coreutils-6.11 [6.9-r1] USE="acl nls xattr (-selinux) -static -vanilla%"
[blocks b     ]  sys-apps/mktemp (is blocking sys-apps/coreutils-6.11)
[blocks b     ] >=sys-apps/coreutils-6.10 (is blocking sys-apps/mktemp-1.5)
[uninstall    ]  sys-apps/mktemp-1.5
[ebuild     U ] sys-apps/coreutils-6.11 [6.9-r1] USE="acl nls xattr (-selinux) -static -vanilla%" 0 kB

Total: 1 package (1 upgrade, 1 uninstall), Size of downloads: 0 kB
Conflict: 2 blocks

Leave a comment

Alexander Gabert a.k.a. pappy (homepage, stats, bugs)
the long tail (May 09, 2008, 01:05 UTC)

well it looks like i can go to bed with a smile on my face...


gcc -g -ggdb -fstack-protector-all -o ssp_entropy ssp_entropy.c
./ssp_entropy
__guard: [[0x353275b9]]


gcc -static -g -ggdb -fstack-protector-all -o ssp_entropy ssp_entropy.c && ./ssp_entropy
__guard: [[0x3687e720]]


TMPFS chroot001 miranda ~ # cat ssp_entropy.c
#include stdio.h

extern unsigned long int __guard;

int main(void) {
printf("__guard: [[0x%x]]\n", __guard);
while(1) { ; }
return(0);
}

for learning about the whole story: http://bugs.gentoo.org/show_bug.cgi?id=182231

have fun and good night!

Leave a comment

May 08, 2008
Tobias Klausmann a.k.a. klausman (homepage, stats, bugs)
Gzip, Bzip2 and Lzma compared (May 08, 2008, 14:35 UTC)

Why?

There has recently been a discussion about GNU switching from bzip2 to lzma for their distributed tarballs. They still offer gzip tarballs as an alternative. However, Gentoo has been preferring the bzip2 tarballs mostly due to the improved pack ratio of bzip2.

Unfortunately, the software for lzma is not (yet) as mature as some would like. For example, the format of files produced has changed recently (in a compatible way, though). Also, the current incarnation of the canonical binaries (lzma-utils) by default links against libstdc++.so which is a huge headache for release engineering and the like.

What and How?

How these distribution problems can/will be solved, remains to be seen. What I'm more interested in, is a comparison of the performance of the three packers. I had initially hoped to also compare the amount of I/O done and memory usage, but GNU time let me down there.

GNU time's manpage claims that it can record and output quite a few figures regarding I/O and memory usage. Unfortunately, I have not been able to make time report anything other than 0 for those interesting stats. Not wanting to debug time, I've chosen to do performance tests regarding pack ratio and execution time, instead.

I've run all tests in single user mode, so as to not disturb page caches and the like. All tests run from the page cache, to factor out disk latency and possible fragmentation of files.

Here's the data I used:

  1. lenna.tiff The compression testing poster girl
  2. lenna.png The same image as above, as a PNG, to see what the algorithms make of already-compressed data
  3. linux-2.6.25.tar The sources for the linux kernel, version 2.6.25
  4. wrnpc12.txt Leo Tolstoy: War and Peace. From Project Gutenberg. ASCII Plaintext.
  5. ImageMagick-6.4.0_modules-Q16.tar A tar file /usr/lib64/ImageMagick-6.4.0/modules-Q16/ from my amd64 system, containing many .so, .la and .a files

Results

Red indicates worst, green indicates best result in a category. All times are in seconds, compression ratio is percentage of uncompressed size.

Input dataAlgoTime (C)Time (D)Ratio
ImageMagick modulesgzip0.300.0425.28%
bzip2 0.610.2720.44%
lzma 4.970.1012.64%
 
Lena (TIFF)gzip0.050.0193.24%
bzip2 0.150.0874.31%
lzma 0.330.0680.69%
 
Lena (PNG)gzip0.020.00100.02%
bzip2 0.130.05100.32%
lzma 0.180.04100.67%
 
Linux Sourcesgzip12.981.8821.81%
bzip2 54.6114.2517.07%
lzma 287.585.4614.47%
 
War and Peacegzip0.300.0337.00%
bzip2 0.58 0.2626.96%
lzma 3.300.0928.34%
 
Overall (Average)gzip2.730.3955.47%
bzip2 11.222.9847.82%
lzma 59.271.1547.36%

Mostly, this benchmark turns out as expected (and advertised by the lzma authors). Gzip is fastest on compression; bzip2 comes second for compression times and lzma finishes last. Decompression is different: gzip is fastest again, but lzma is faster than bzip2, sometimes nearly by a factor of three. Compression ratio is where gzip shows its age: its compression ratio is worst (except when packing already compressed stuff). Bzip2 bests lzma's compression ration in three cases of five (two of four if you don't count the PNG). On average, however, lzma compresses better by a hair. Lzma's strongest lead over the others is when compressing the ImageMagick module tar.

Conclusion

Lzma is definitely worth a look due to its performance - if you unpack much more often than you pack (three times slower than already-not-so-quick bzip2!). Also, lzma works better on large files. From a distribution standpoint, if/when the library/dependency problems have been sorted out, lzma is quite preferable, if you don't do much compression yourself, yet have to watch your bandwidth usage. For Gentoo, it's reasonable - if the kinks are worked out. Another disadvantage of lzma is that while gzip and bzip2 have their own single-char decompression switch when using tar (z and j, repectively), the new guy on the block only gets a long one: --lzma (introduced in tar 1.20). This might sound minor, but it can quickly get pn your nerves. YMMV.

What else to test

I'd love to see more tests with real-life data. I also think a comparison of I/O load and memory usage for the three contenders would be interesting. If somebody wants to do all the work, comparing the different speed/compression settings for the three could be interesting - I have only used the default settings, which might not exactly be fair for gzip, which was written at a time when CPU cycles where far more expensive.

Comments

Josh "Nightmorph" Saddler writes:

Thanks for the blog and the benchies. Some interesting stuff here.

I still don't know that I'd go for lzma, because it's compression time sucks assballs. bzip2 is bad enough as it is. lzma is at the back of the pack for compression, every time.

The numbers for Linux sources aren't very encouraging. lzma takes a metric assload of time to compress it. Far, far too long. At least it's quick to decompress the same file.

I guess if we were to mostly move to lzma archives, it wouldn't be so bad on the package installation side. I mean, most of what we do as Gentoo users is unpack tarballs for installation, right? Anything to speed up this process is probably a step in the right direction ... I just wouldn't use lzma for anything else. For Gentoo, it's fine. For personal stuff, it's not.

and

One thing I didn't even think about was about postinstall compression. Diego mentioned this over at his blog — every merge operation is going to have some compression for things like docs, manpages, etc.

I'd completely forgotten about the stuff that gets compressed when installed. lzma doesn't scale particularly well to small files, not like gzip does.

I guess it's a tradeoff — you can unpack large archives like kernel sources very quickly, but you'll get (a bit?) more time added to each merge op if lzma is doing the postinstall compression.

I absolutely agree with Josh. The maturity problems aside, being a consumer of lzma archives is nice if you're not short on memory. That said, supporting lzma as a method for upstream tarballs is a nice thing to have. I won't be switching to it for my personal compression needs any time soon.

Bonus fact learned about recent tar versions

Recent version of tar autodetect the compression for bzip2 and gzip, which means you can use tar xf linux-2.6.25.tar.bz. Not only that, you can also tell it to guess compression upon creation of archives: tar caf foo.tbz2 /etc. Nifty. Doesn't work for lzma, though, which might be due to the recent change in file format. file doesn't recognize the files, either.

Bonus Bonus fact learned: bash completion for tar needs to be fixed.

I've taken the liberty of inserting the link to Diego's blog post.

Joshua Nichols a.k.a. nichoj (homepage, stats, bugs)

Joshua Nichols

One thing that kind of annoyed me about Gentoo Prefix is that you always needed to do a little work to enter the prefix:

$ ./bootstrap-prefix.sh /path/to/prefix startscript

Personally, I just want to always live in a prefixed shell. Amazingly enough, this is really easy to do.

  1. First open up /etc/shells as root
  2. Add a line like /path/to/prefix/bin/bash and save
  3. As your user, run: chsh -s /path/to/prefix/bin/bash
  4. Enter your password
  5. Done

Now, everytime you open a terminal, it will be running a bash gloriously built by Gentoo

May 07, 2008
Tobias Klausmann a.k.a. klausman (homepage, stats, bugs)
ack - Part II (May 07, 2008, 19:31 UTC)

This is a followup on my earlier post about ack. A few tips on daily usage and a few caveats.

Every day is a winding command line

Normally, ack does not search text files. This may come as a surprise since it goes a bit against the grain of what we're used from Unix and Linux commands. Seeing where ack comes from and what its intention is, it's not so unusual. Anyway, we can make it behave the way we want.

I'm grateful that ack doesn't search .svn/ and friends, I also don't mind backup files, swp files, core files etc being ignored. But I do want to search text files when I grep ack through source files. What if the elusive fixAllBugs() function I'm looking for is only mentioned in a README file?

So how does one customise ack? There are two ways: one is an environment variable (ACK_OPTIONS). It can hold all those extra options you always want. There are more environment variables (for coloring, the pager and where the .ackrc (more on that later) resides). They're all described in ack's perldoc page (which can be accessed by perldoc ack (if it's in your PATH) or ack --man.

The second way to customize ack is having an .ackrc in your home directory (or wherever the ACKRC environment variable points). It contains all the options just as you would use them on the command line, one per line. You can intersperse it with comments in the usual way. Again, more on this can be found in the perldoc.

So now what do we want? This hugely depends on preference, I would call ack like this, usually:

ack --text <other options>

Since I'm rather lazy, I'd consequently put "--text" in my .ackrc or in ACK_OPTIONS. Whenever this gets in my way, I can use --noenv to ignore all environment and config file settings.

Measurements, Unicode and all that

I have an add-on note for the performance checks (I wouldn't call them measurements, far too little diligence on my part) in my last post. It turns out that the locale setting can have a wild amount of influence on the performance of grep. This is especially true of any UTF-8 or similar multibyte encodings.

I've made new measurements since the file I used has grown since and I haven't kept the old version. Again, the file is in the page cache before I do my measurements. Also, I've done at least a dozen or so runs each and the numbers I'll show you are typical.

$ echo $LC_ALL
en_US.utf8
$ time grep 1Jsztn-000647-SL exim_main.log >/dev/null
real    0m15.995s
user    0m14.770s
sys     0m0.160s
$ time ack 1Jsztn-000647-SL exim_main.log >/dev/null
real    0m3.829s
user    0m3.570s
sys     0m0.125s
$ export LC_ALL=C
$ time grep 1Jsztn-000647-SL exim_main.log >/dev/null
real    0m0.226s
user    0m0.120s
sys     0m0.100s
$ time ack 1Jsztn-000647-SL exim_main.log >/dev/null
real    0m3.871s
user    0m3.370s
sys     0m0.120s
$ 

As you can see, with the C locale, grep is about 70 times faster than with the multibyte locale and about 17 times faster than ack, which in turn shows no speed difference between the two locales.

Now some might make the case that one shouldn't bother with ack since it's so slow. Well, I prefer the 17x penalty if I don't have to muck around with my locale every time I use grep or ack on the command lines. In scripts, things are a bit different, but ack is not aimed at that. Additionally, ack's performance advantage is also generated by not even looking at uninteresting files. For example, searching for "klausman" in a cvs checkout of the Gentoo portage tree takes 8 seconds with ack, but over thirty seconds with grep (locale set to C).

All this said, ack is one of the few tools that look like they're intended for use "just like grep" but are actually meant for humans (or programmers ;)).

So the final verdict is: both grep and ack have their places, but ack sure eats some of grep's pie.

As a side note, the developers of grep might want to look into optimizing their program for multibyte encodings. Not to the detriment of the "classic" encodings, but I think there's some untapped potential.

Update: The grep devs already know.

May 06, 2008
Zack Medico a.k.a. zmedico (homepage, stats, bugs)

A while ago I wrote a blog entry about using blockers to adjust merge order. Now, in portage-2.1.5, blockers are also resolved automatically in cases when it makes sense to uninstall a conflicting package (bug #172812). This feature should allow automatic resolution of blocker conflicts in many more cases than previously possible, so Gentoo users won't be inconvenienced with the task of resolving them manually.

gtk-doc-am vs. gtk-doc-1.8-r2

These are the packages that would be merged, in reverse order:

Calculating dependencies... done!
[nomerge      ] x11-libs/gtk+-2.12.9-r2  USE="X cups jpeg tiff xinerama -debug -doc -vim-syntax"
[ebuild  N    ]  dev-util/gtk-doc-am-1.10-r1  0 kB
[blocks b     ] dev-util/gtk-doc-am (is blocking dev-util/gtk-doc-1.8-r2)
[uninstall    ]  dev-util/gtk-doc-1.8-r2  USE="emacs -debug -doc"

Total: 1 package (1 new, 1 uninstall), Size of downloads: 0 kB
Conflict: 1 block

coreutils vs. mktemp

These are the packages that would be merged, in reverse order:

Calculating dependencies... done!
[ebuild     U ] sys-apps/coreutils-6.11 [6.9-r1] USE="acl nls xattr (-selinux) -static -vanilla%" 0 kB
[blocks b     ]  sys-apps/mktemp (is blocking sys-apps/coreutils-6.11)
[blocks b     ] >=sys-apps/coreutils-6.10 (is blocking sys-apps/mktemp-1.5)
[uninstall    ]  sys-apps/mktemp-1.5

Total: 1 package (1 upgrade, 1 uninstall), Size of downloads: 0 kB
Conflict: 2 blocks

Leave a comment

Tobias Klausmann a.k.a. klausman (homepage, stats, bugs)
ack! (thpppt!) (May 06, 2008, 14:44 UTC)

There are some tools that look like you will never replace them. One of those (for me) is grep. It does what it does very well (remarks about the shortcomings of regexen in general aside). It works reasonably well with Unicode/UTF-8 (a great opportunity to Fail Miserably for any tool, viz. a2ps).

Yet, the other day I read about ack, which claims to be "better than grep, a search tool for programmers". Woo. Better than grep? In what way?

The ack homepage lists the top ten reasons why one should use it instead of grep. Actually, it's thirteen reasons but then some are dupes. So I'd say "about ten reasons". Let's look at them in order.

  1. It's blazingly fast because it only searches the stuff you want searched.

    Wait, how does it know what I want? A DWIM-Interface at last? Not quite. First off, ack is faster than grep for simple searches. Here's an example:

    $ time ack 1Jsztn-000647-SL exim_main.log >/dev/null
    real    0m3.463s
    user    0m3.280s
    sys     0m0.180s
    $ time grep -F 1Jsztn-000647-SL exim_main.log >/dev/null
    real    0m14.957s
    user    0m14.770s
    sys     0m0.160s
    

    Two notes: first, yes, the file was in the page cache before I ran ack; second, I even made it easy for grep by telling it explicitly I was looking for a fixed string (not that it helped much, the same command without -F was faster by about 0.1s). Oh and for completeness, the exim logfile I searched has about two million lines and is 250M. I've run those tests ten times for each, the times shown above are typical.

    So yes, for simple searches, ack is faster than grep. Let's try with a more complicated pattern, then. This time, let's use the pattern (klausman|gentoo) on the same file. Note that we have to use -E for grep to use extended regexen, which ack in turn does not need, since it (almost) always uses them. Here, grep takes its sweet time: 3:56, nearly four minutes. In contrast, ack accomplished the same task in 49 seconds (all times averaged over ten runs, then rounded to integer seconds).

    As for the "being clever" side of speed, see below, points 5 and 6

  2. ack is pure Perl, so it runs on Windows just fine.

    This isn't relevant to me, since I don't use windows for anything where I might need grep. That said, it might be a killer feature for others.

  3. The standalone version uses no non-standard modules, so you can put it in your ~/bin without fear.

    Ok, this is not so much of a feature than a hard criterion. If I needed extra modules for the whole thing to run, that'd be a deal breaker. I already have tons of libraries, I don't need more undergrowth around my dependency tree.

  4. Searches recursively through directories by default, while ignoring .svn, CVS and other VCS directories.

    This is a feature, yet one that wouldn't pry me away from grep: -r is there (though it distinctly feels like an afterthought). Since ack ignores a certain set of files and directories, its recursive capabilities where there from the start, making it feel more seamless.

  5. ack ignores most of the crap you don't want to search

    To be precise:

    • VCS directories
    • blib, the Perl build directory
    • backup files like foo~ and #foo#
    • binary files, core dumps, etc.

    Most of the time, I don't want to search those (and have to exclude them with grep -v from find results). Of course, this ignore-mode can be switched off with ack (-u). All that said, it sure makes command lines shorter (and easier to read and construct). Also, this is the first spot where ack's Perl-centricism shows. I don't mind, even though I prefer that other language with P.

  6. Ignoring .svn directories means that ack is faster than grep for searching through trees.

    Dupe. See Point 5

  7. Lets you specify file types to search, as in --perl or --nohtml.

    While at first glance, this may seem limited, ack comes with a plethora of definitions (45 if I counted correctly), so it's not as perl-centric as it may seem from the example. This feature saves command-line space (if there's such a thing), since it avoids wild find-constructs. The docs mention that --perl also checks the shebang line of files that don't have a suffix, but make no mention of the other "shipped" file type recognizers doing so.

  8. File-filtering capabilities usable without searching with ack -f. This lets you create lists of files of a given type.

    This mostly is a consequence of the feature above. Even if it weren't there, you could simply search for "."

  9. Color highlighting of search results.

    While I've looked upon color in shells as kinda childish for a while, I wouldn't want to miss syntax highlighting in vim, colors for ls (if they're not as sucky as the defaults we had for years) or match highlighting for grep. It's really neat to see that yes, the pattern you grepped for indeed matches what you think it does. Especially during evolutionary construction of command lines and shell scripts.

  10. Uses real Perl regular expressions, not a GNU subset

    Again, this doesn't bother me much. I use egrep/grep -E all the time, anyway. And I'm no Perl programmer, so I don't get withdrawal symptoms every time I use another regex engine.

  11. Allows you to specify output using Perl's special variables

    This sounds neat, yet I don't really have a use case for it. Also, my perl-fu is weak, so I probably won't use it anyway. Still, might be a killer feature for you.

    The docs have an example:

    ack '(Mr|Mr?s)\. (Smith|Jones)' --output='$&'
  12. Many command-line switches are the same as in GNU grep:

    Specifically mentioned are -w, -c and -l. It's always nice if you don't have to look up all the flags every time.

  13. Command name is 25% fewer characters to type! Save days of free-time! Heck, it's 50% shorter compared to grep -r

    Okay, now we have proof that not only the ack webmaster can't count, he's also making up reasons for fun. Works for me.

Bottom line: yes, ack is an exciting new tool which partly replaces grep. That said, a drop-in replacement it ain't. While the standalone version of ack needs nothing but a perl interpreter and its standard modules, for embedded systems that may not work out (vs. the binary with no deps beside a libc). This might also be an issue if you need grep early on during boot and /usr (where your perl resides) isn't mounted yet. Also, default behaviour is divergent enough that it might yield nasty surprises if you just drop in ack instead of grep. Still, I recommend giving ack a try if you ever use grep on the command line. If you're a coder who often needs to search through working copies/checkouts, even more so.

Update

I've written a followup on this, including some tips for day-to-day usage (and an explanation of grep's sucky performance).

Comments

René "Necoro" Neumann writes (in German, translation by me):

Stumbled across your blog entry about "ack" today. I tried it and found it to be cool :). So I created two ebuilds for it:

Just wanted to let you know (there is no comment function on your blog).

'Tis well appreciated. Can't speak for the ebuilds' quality (yet), but I'm sure something will be worked out.

Daniel Drake a.k.a. dsd (homepage, stats, bugs)

My fprint fingerprint scanning efforts formed my final year Computer Science project at The University of Manchester.

The source code for this project has been available on SourceForge from early on (GPL-2/LGPL-2 licenses). I’ve now completed and submitted a comprehensive project report (similar to a dissertation) for academic assessment, and I’m making this available under a Creative Commons license. You can find the report here.

Creative Commons License

The report is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 2.0 UK: England & Wales License.

The academic project is now complete, but I plan to continue development as time permits. There is a lot to be done, and I have already made some good progress on moving libusb-1.0 closer to release.

Leave a comment

May 05, 2008
Steve Dibb a.k.a. beandog (homepage, stats, bugs)
planet larry policy update (May 05, 2008, 20:41 UTC)

Steve Dibb

I’ve made an awkward and difficult decision that I hope doesn’t make anyone feel bad: effective immediately I’ve removed any feeds from Planet Larry / Larry the Universe that were from developers who have retired from the Gentoo project.

I setup the planet feeds mainly so that regular users of Gentoo Linux could talk about and share their experiences, and I want to get back to keeping it with them primarily in mind. I tend to think that ex-developers have more weight with their posts, which tends to cause some unbalance that I was never comfortable with.

Speaking of users, I am always looking for new bloggers to get added, so if you are a Gentoo user who blogs about computer experiences, be sure to sign up.

That’s all for now.

Edit: I originally took this post down, and am now restoring it. I still haven’t decided if this is a permanent change or not. I might create a planet just for ex-developers, or reintegrate them somehow. I dunno. Comments and suggestions are welcome.

Leave a comment

Joshua Nichols a.k.a. nichoj (homepage, stats, bugs)
Managing RubyGems on Gentoo (May 05, 2008, 03:04 UTC)

Joshua Nichols

For Ruby on Gentoo, it has been asked how one should go about managing RubyGems. You have a few options:

  • Use Gentoo packages exclusively
  • Use gem exclusively
  • Mix and match and pray

Mix and match

Offhand, mixing and matching seems like a bad idea.

Doing so, you might install a gem using portage, but then you could use gem update to update it. Then you update it with portage...

Overall, seems like it could be problematic in the long run.

Gentoo packages exclusively

Being a Gentoo packager, my first take is that using packages is the way to go.

Some pluses:

  • One stop for installing/updating packages, ruby or otherwise
  • For gems with native extensions, can have dependencies on the things the native extensions use

Of course, there are a number of disadvantages.

  • There are tons of gems out there
  • Some update frequently
  • Lot of packages to be keeping up with stable keywords
  • Despite gem ebuilds being pretty straightforward, there is a bit of grunt work involved on to verify dependencies are accurate
  • Lots of gems are unpackaged, because they are usually packaged as requested or as developers need them
  • Despite being able to install multiple gems at once, packages usually aren't slotted that way

Gems exclusively

If you're a Ruby developer, the most natural way would be using RubyGem directly. Some nice things include:

  • Most direct way to have the latest and greatest gems
  • Can have as many versions installed as you want
  • Zero maintenance for Gentoo packagers (sweet!)

Of course, not without their drawbacks:

  • If a gem uses native extensions, it can be tricky to determine what the it's dependencies are
  • Problematic when other (non-ruby) packages need to depend on gems

My current practices

So where does that bring us?

Here are the practices I've taken to using for my Ruby and Rails development and deployment needs.

In development

On my development machine (4 MacBook running Leopard with Gentoo/Prefix), I've using RubyGems directly.

If I'm working on a Rails app, and it needs any gems, I've taken to vendoring everything. This type of thing will be supported out of the box for Rails 2.1.

Of course, this wouldn't work so well for things with native extensions, like hpricot.

In production

I avoid installing any gems in production as much as possible. If an app needs something, it should be vendored, like I just described.

For gems needed by my apps that have native extensions, or are otherwise needed in production (like rake and the mysql gem), again, I'm using RubyGem directly.

May 03, 2008
On cooperating and paludis vulnerability (May 03, 2008, 22:24 UTC)

Fernando J. Pereda

A serious security issue in paludis was brought to my attention recently, and I feel I should make you all aware. Apparently someone, with root access to a machine, can gain root access by installing or editing paludis config files.

For those interested, this is how it happened (times are GMT+1):

22:34 <@ferdy> bonsaikitten: can you give me any details regarding that
 security bug in paludis?
22:35 <+bonsaikitten> ferdy: it's so obvious you should have found it already
22:37 <@ferdy> bonsaikitten: I should, but I probably haven't
22:37 <+bonsaikitten> ferdy: well, as I am a moron I'm unable to coherently explain :)
22:37 <@ferdy> bonsaikitten: I mean, depends on whether we are talking about
a real security issue or about something we should document to avoid people
shooting themselves in the foot
22:39 <@ferdy> bonsaikitten: is that all you are going to tell me?
22:39 <+bonsaikitten> ferdy: come on, it's obvious. You're supposed to be smart ...
22:39 * bonsaikitten is not in a mood to explain
22:40 <@ferdy> bonsaikitten: you aren't really talking about the paludisbuild issue, are you?
22:41 <+bonsaikitten> mmh no, that's a different one
22:41 <@ferdy> k
22:41 <@ferdy> bonsaikitten: what are we talking about?
22:42 <@ferdy> bonsaikitten: you don't need to explain it... just say, in general 
terms, what the issue is
22:50 <@ferdy> bonsaikitten: so? care to give any useful hint?
22:50 <+bonsaikitten> ferdy: doesn't happen in portage compatibility mode
22:51 <+bonsaikitten> but I blame the vodka, hard to explain when *burp* *giggle*
22:52 <@ferdy> bonsaikitten: what's the impact?
22:53 <+bonsaikitten> ferdy: depends on how annoying the other person is
22:54 <+bonsaikitten> ferdy: worst case random file modification
22:58 <@ferdy> bonsaikitten: and we already agreed that we aren't talking about
the paludisbuild issue, right?
22:59 <@ferdy> bonsaikitten: if we aren't, I'll need more hints....
23:05 <@ferdy> bonsaikitten: can I get an attack vector?
23:05 <@ferdy> that shouldn't need lots of explaining... I can figure out that
part myself
23:19 <@ferdy> bonsaikitten: have you got that attack vector for me?
23:24 <+bonsaikitten> ferdy: look at configuration files, maybe you notice that
there's some exquisit code execution possible there
23:29 <@ferdy> bonsaikitten: you mean those config files that only root can
edit? I must be missing something here
23:29 <+bonsaikitten> ferdy: you are :)
23:29 <+bonsaikitten> not much, and it's basically the same flaw bashrc is
for portage
23:29 <+bonsaikitten> only that bashrc is config_protect'ed ...
23:30 <@ferdy> bonsaikitten: but for a package to clover those files, it must be
in a repo root added, right?
23:31 <+bonsaikitten> someone in the package mangler group, but yes
23:35 <@ferdy> bonsaikitten: but if you can change those files in the first place,
why clover them by adding a malicious repo with a malicious package that changes
those files?
23:35 <+bonsaikitten> ferdy: because it's very subtle
23:36 <@ferdy> moreover, if you can already do that, why not just make the
package install whatever backdoor you want?
23:37 <@ferdy> I mean, it is subtle, but why would anyone go the 'convoluted'
route? he is already able to edit those files (since he had to add that repo)
23:38 <+bonsaikitten> 'cause only paludis is affected and you will find it very
hard to trace
23:38 <+bonsaikitten> that makes it so tempting ...
23:40 <+bonsaikitten> just don't be surprised if it suddenly unmerges itself :)
23:41 <@ferdy> yeah... well...
23:41 <@ferdy> bonsaikitten: mind if I disclose this vulnerability in
 planet.gentoo.org?
23:42 <+bonsaikitten> go ahead
23:42 <@ferdy> ta
23:42 <+bonsaikitten> 't is even on the features page of the package mangler :)

This is a good lesson to learn today:

If you can edit files owned by root in a machine, you can get root access to that machine.

So the bottom line is: There is no vulnerability, if you can mangle paludis config files, you are already root so you don't need to edit a file to run any command you want. Another lesson one can learn by reading that log is how to be really cooperative.

Ah, and before someone with a need to use cheap psychology asks, the intention of this blag post is to stop the FUD.

- ferdy

Leave a comment

May 02, 2008
Remi Cardona a.k.a. remi (homepage, stats, bugs)

Replying to Ryan's post...

https://bugzilla.mozilla.org/show_bug.cgi?id=384090#c41

And I'll just quote that comment here:

Firefox-3 (now that the patch is applied) will behave like Gtk, i.e. it will read the font dpi from any standards-compliant XSETTINGS manager. If Gtk programs have the right font dpi on your system, then so will firefox.

As for the gconf key, it won't help you unless you are also running gnome-settings-daemon (the default XSETTINGS manager for the Gnome desktop).

Let's not spread unnecessary false information, shall we? ;)

Leave a comment

Better late than sorry (May 02, 2008, 09:12 UTC)

Christian Faulhammer

Or whatever the proverb was...anyway, here is my meme list:

Desktop user: 

77 cd
75 l
59 grep
48 ebuild
33 less
21 exit
21 cvs
16 repoman
15 $EDITOR
12 /bin/su

Chroot (for architecture work):

118 gatt
53 emerge
46 exit
24 rm
24 FEATURES="test
21