![]() |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![]() |
![]() Welcome to Planet Gentoo, an aggregation of Gentoo-related weblog articles written by Gentoo developers. For a broader range of topics, you might be interested in Gentoo Universe. May 16, 2008
Decibel hits the tree (May 16, 2008, 06:49 UTC)
It's official! Decibel Audio Player is now in Portage. Thanks to aballier for adding it, and thanks to everyone who provided feedback & testing. Yay for yummy audio goodness!
Gentoo Foundation Reinstated (May 16, 2008, 05:20 UTC)
In case you missed it, the Gentoo Foundation has been reinstated, so we're all nice'n'legal again. Thank you, trustees, for all your efforts! You can view our updated paperwork with the state of New Mexico here. May 15, 2008
Tic Tac Toe over Jabber (May 15, 2008, 22:45 UTC)
Slacker Council (May 15, 2008, 20:44 UTC)
As per http://archives.gentoo.org/gentoo-dev/msg_19892c04f0e6cf4c24629f13718e45cb.xml there was a meeting council scheduled for 20:00 UTC today (that's a bit more than half an hour ago). For some reason, only amne and dberkholz showed up. As per GLEP39's Specification:
What's the council going to do? Place your bets. — ferdy
app-admin/pardalys was created in the Kolab overlay (May 15, 2008, 11:02 UTC)
If you look at the current ebuild you might wonder what the fuss might be about... It is a pretty empty package. But the p@rdalys project will form the core for Kolab2/Gentoo-2.2. It will certainly replace net-mail/kolabd and might include some other packages, too. The idea is to allow you to install Kolab2/Gentoo-2.2 with two simple steps: emerge app-admin/pardalys pardalys Of course there is still a certain way to go until it will actually work that way. And this easy setup is actually just meant as a nice side effect and is not the main point of starting the project. I'll start explaining this package in greater detail once I push more code into it. For now the link to the project page will be all I can provide. Currently it might not be clear what the package will actually be about but if people wish to contribute to the project at a later time point you should go visit the git repository on GitHub. This git repository should serve as a scratch repository used for easy sharing and patching of the code. The reference repository on the other hand will be kept in subversion on SourceForge and will be used for packaging. More on the whole story once there is more code.
When you should use RDEPEND and when you should use DEPEND (May 15, 2008, 00:37 UTC)
Maybe it’s somehow not clear in the documentation, as I see this kind of mistakes happens quite a lot. The obvious reason is that sometimes it happens you oversee something, but as a lot of these problems happen on packages maintained by the same person or group, I start to think this is not clear enough. In particular I’m afraid the reasoning why you should make sure that the two variables are filled in properly is not clear enough. First of all, at the moment we unfortunately have only two dependency variables, DEPEND and RDEPEND. Cross-compiling and proper multilib support would need two more, splitting both in executed and linked-against packages, as the first just need to be available in some executable form, while the latter need to have the same ABI as the program being built. But that’s a story for another day. So we’re back to DEPEND and RDEPEND. The first is the set of packages that has to be available during build, as this is what we have, it has to have both tools used to build, and library linked against. RDEPEND instead is the set of packages that has to be available during execution of the program. This means it has to have the libraries linked against, if shared – which is what we want with pretty rare exceptions – and the programs that are ran by the program at runtime. Programs like unzip and other tools used to extract the source archive as DEPEND and not RDEPEND. Unless they are executed by the software itself, like Amarok does (alas), in which case they are RDEPEND and not DEPEND. Build tools, like pmake or build systems like imake, scons or cmake, generators like flex and bison, rust or swig are DEPEND only, as they are used during source build. Software used for the tests is DEPEND only and subject to the test USE flag. Fonts, when used by a graphical application, are most likely useful only at runtime. Rarely tools require them at buildtime, but it happens, for instance when you use gd to generate some images for the documentation. By the way, Doxygen is a build-only dependency, and should always be subject to the doc USE flag! Most, but not all, packages for interpreted languages like Perl, Python and Ruby extensions have little or no build dependency (they are not built, most of the times) but may have lots of runtime dependencies. And might have build-time dependencies under test USE flag for the tests. When a package install no shared library, it’s usually just one of the two between DEPEND and RDEPEND, as rarely the software checks at build-time what it will execute at runtime, which is quite good for us, as we’ll see. If the configure script for a package just checks if the tools are available, but doesn’t use them, it’s a good idea to either fool it through cache values, or change it so that the tests are not done. Again, we’ll see why. So okay, why should we spend time to make sure the dependencies are right? Shouldn’t people install the packages anyway for both build and runtime? Well, sort of. When you prepare a filesystem image for another system through ROOT=, you usually just want the packages needed at runtime, not at buildtime. This means you don’t want autotools, you don’t want pkg-config, you don’t want lzma and so on. Often this is done through cross-compile, and this saves you cross-compiling for instance Perl – which, if you didn’t know, is not cross-compilable at this stage – but it might also apply to systems that are maintained through binary packages, as you don’t want to have to distribute extra packages, especially if the target machines don’t have as much space as you’d like them to be. But having proper DEPEND/RDEPEND also helps testing. For instance, if I’m to check whether a given package builds after a flex bump, I don’t want to have to install all its runtime dependencies, as that would stop me from using the This also means that if you know for sure that the package calls the linker with a given library at build-time, but the library is never ever used (and for instance Please, when you write the dependencies of an ebuild, think a lot how to get them right, make sure that datafiles are only runtime dependencies unless they are used for build stuff too, make sure that build tools are not leaked at runtime. Basically, look at the package from every angle and make sure that you actually get the thing how it is supposed to be. May 14, 2008
Sometimes you do need Linux-PAM, but mostly... (May 14, 2008, 21:17 UTC)
… you don’t. I’m not referring to PAM-less systems, I sincerely don’t care much about those. I’m referring to the dependencies of packages, which sometimes require My idea at the moment is to check for packages that can be ported to virtual/pam without code changes. I already found at least two, and there probably are others. The ones not working, are usually looking for the This falls into two lines of work for me, the first is that I’m trying to clean the area for Seraphim Mellos, who’s working on Summer of Code to support OpenPAM, the second is for Gentoo/FreeBSD, as there Linux-PAM is not used. Additionally, the less references directly to Again, I wonder why am I stuck with this task as I usually don’t need PAM at all myself. I hope that getting a token changes something, so that I’d just have to key in the PIN of the token and be done with login too, but even that it’s a bit of a stretch for a standard user. Anyway if somebody would like to help out, it would be nice if the packages using
what I am doing, RIGHT NOW (May 14, 2008, 21:14 UTC)
I've been watching twitter for a while now, amused at the ability for it to keep people appraised of what you are doing at the moment, if they really care. I didn't think it was really worth it. Until I read this post last night which was linked off of some site that I forgot (probably reddit but I did think it was from the every wonderful Arachaia, which if you are a programmer, you should be paying attention to.) I just couldn't resist... So, if you want to see what I am doing, RIGHT NOW (well what I just did, it waits for the command to complete before sending it off to twitter), you can follow along right here. I'm only enabling it on a few of my terminal windows for now, watching me constantly run mutt and offlineimap would get a bit boring. I wonder how long it's going to be before I type in my password accidentally to this thing. Or until twitter bans me. Any odds on which is going to happen first? I pity anyone who subscribes to this twit feed, they are going to start hating me very quickly, like the Portland, Oregon local feed already has...
More Gentoo development links than you probably need (May 14, 2008, 20:04 UTC)
Since I've been less active packaging for Gentoo recently, I've been trying to at least lurk in #gentoo-dev-help and give a helping hand with packaging. If I can't help them directly, at least I try to give them a gentle shove towards something that can. As a result, I've been collecting various bits of documentation about how to package stuff. This post is a result of this, as well as some other digging around I did to flesh it out. General development documentationChannels of communicationLots of projects and herds within Gentoo have their own mailing lists and irc channels. Language-specificPackages written in different languages often have their own way of dealing with packaging: HerdsSame goes for herds. Other stuffThese don't fall into any other categories, but are still useful.
Even little differences count (May 14, 2008, 16:20 UTC)
If you ever find yourself relying for any reason on the behaviour of Microsoft Excel, make sure you never ever change its version. Seems like I had some problems with my job because the version of Microsoft Excel I bought is not the same as the one they are using at the office. Also, if you want to shut up Valgrind’s warnings to make life easier for people developing against a given library, it’s time you hear about suppression files, rather than trying to change code you don’t know enough about. Seems like a little difference in the patches applied by Debian (and Ubuntu) made OpenSSL vulnerable like it never was. Even my keys has to be considered compromised, so I had to change them on every service I use. This means Alioth, Gentoo’s Infra, GentooExperimental, SourceForge, Berlios, Savannah, Rubyforge, Launchpad (don’t ask), KDE, and of course my own server. I think yesterday will be remembered for the years to come as a critical day. It’s like a city of the size of Milan starting to change the door locks for all the apartments at once. I’m surprised there hasn’t been more confusion. Myself, I’m starting to consider some other kind of authentication, as the passphrases I use for the keys are quite long, and I’m tired of having to key it in every time I start the system. SmartCard and authentication tokens start to look quite nice, and it would be a nice test for Gentoo’s PAM infrastructure. It’s unfortunate that Alon, who as far as I remembered managed these things, left Gentoo yesterday :/ Although I suppose that if I am to start looking into these things I might be able to lend a hand. At the moment, I’m oriented toward an authentication token, as that would make it possible for me to have it around on the laptop too at any time. Strangely enough, I found an Italian producer, Eutronsec, and their CryptoIdentity token seems to be supported by pcsclite. SmartCards are more likely useful in an organisation where you’d have one reader on a given system and many many cards around. Here I’d just have one reader and one smartcard if I did go that way. If the Italian ID card was a SmartCard I could have chosen that, but at the moment it doesn’t look like a convenient idea. Unfortunately it seems like Eutronsec only sells to other companies, I’ve mailed them asking if they could sell to single entities, but I haven’t received an answer yet. I’m afraid I’ll have to start looking for alternatives if I want to proceed on this road. On a totally different topic, but still related to “little differences”, I’m having difficulties finding cups that I can use with my espresso machine. Usual teacups that my family uses for coffee (yeah they are quite larger for coffee, we’re used to those though, a six servings Moka is good for three here) tend to be too thin at the bottom and large at the top, so the coffee spills a lot, especially when doing a hot boiling espresso. Mugs are quite better for the task, but all the ones I have here are little less than 9.5 centimetres (little more than three and a half inches), and require to be skewed to enter the espresso machine. The idea would be an 8.5cm cup, but I can’t seem to find them in the shops around here, they are either 6 centimetres, which is way too short, or they are the “usual” mug size. In a Chinese shop around here I found some nice cups that are about 12cm actually, I was tempted to take one to hold the milk for cappuccino, actually, but they are quite too high for the espresso machine.
Berkano overlay news (May 14, 2008, 10:46 UTC)
Recent changes in berkano overlay:
I’m actually looking into adding more Qt4 based applications, either to the overlay or portage. My goal would be to get together an as complete as possible desktop using just (or mainly at least) Qt4 programs. The most obvious missing part is a Window Manager. Cutebox was a promising project, but has been abandoned even before its first release. Anyone that would resurrect it and bring it to a useful form, would win a guaranteed spot in my personal Hall of Heroes. May 13, 2008
blu-ray dvd drives (May 13, 2008, 19:20 UTC)
An interesting post came up the other day on the Gentoo forums about how to rip Blu-Ray discs on Linux. Short summary: I have no idea if it’s possible, and the original poster is still investigating. It has gotten me thinking though. The Blu-Ray player that I want to get it is $600, and it looks like it’s being phased out of production anyway, so why not get a disc drive instead and rip the movies? It’d save me some money, and I’d eventually buy one anyway. Well, the questions that come to mind are, will the software actually work, will the drive firmware let me do that, and am I going to have to use Windows? I haven’t done any research at all, mostly because I can’t afford to buy a DVD drive right now, but the whole thing does have me curious. I always assumed there was no way to rip the stuff under Linux, but I haven’t gone looking for possible solutions either. The only thing I am sure about though, is that once ripped, you can play the content just fine. At least, I think so. I’m not positive about the HD audio codecs, pretty sure about the video ones though. I tend to buy hardware first and figure out how to get it working second, but because the DRM is so finicky in this case, I really don’t want to take that approach and be out a couple hundred bucks. In the meantime, I really wish I could at least demo the stuff at home. That would be cool. The only 1080p content I’ve seen so far is the movie trailers I’ve downloaded from Apple’s website. I gotta say that stuff looks pretty good.
Use(d) Debian? Check your keys! (May 13, 2008, 14:59 UTC)
If you run any kind of server, especially Debian or Ubuntu, or grant users access to your server, you might want to read the Debian Security Advisory DSA-1571-1 or Ubuntu’s Security Notice USN-612-1 for CVE-2008-0166, and check your encryption keys:
This vulnerability is caused by a patch shipped in Debian, Ubuntu, and other derivatives. Gentoo’s OpenSSL version is not affected, but everyone should check user-provided public keys (such as OpenSSH’s authorized_keys) using the Debian/OpenSSL Weak Key Detector. Update: Ben Laurie of OpenSSL is making a point that Vendors Are Bad For Security, which I would not follow in that general form. What I have to grant him: Mechanisms of peer review must be employed properly and patches discussed with upstream. If you follow this philosophy, Vendors Are Good For Security.
Growing Pains: Part Deux (May 13, 2008, 11:04 UTC)
Alon Bl has left the project (the mail hasn’t hit archives.gentoo.org or I’d link you). In his resignation he stated that he had problems with other developers and that was a big part of his leaving. He talks about a lack of leadership, of goals, and of user experience; of good developers leaving, and many other things. First off, sorry to see you go Alon and I appreciate the work you did for Gentoo. I empathize with your views that you expressed in your mail. These are difficult problems and it is not a quick change to fix them and you (nor I) may not agree with the solution. I think Gentoo has many goals; however they are poorly expressed and because of that there is no way to essentially ‘meet’ them. The problem we previously had was that ‘everyone thought they had a say’ and that appeared to not work. The council is attempting to modify that view somewhat and it is a slow process. You can’t just elect some dictator guy to make arbitrary choices and set arbitrary goals and metrics. You need a critical mass of folks who are willing to dedicate time to meet those goals. You must also be willing to accept losses of developers who disagree with this new vision. I know many developers who left because they thought change was impossible. I used to think that way. One of the things that getting a real job has taught me is that changes like these are not impossible, they are just not easy. You need folks dedicated to actually solving them and not just talking about it. People to set goals and actually sell them to the community as worthwhile and convince others to work towards them. Maybe we used to have that calibur of person; maybe we never did; maybe we never will. But you are correct in one thing. Someone has to try. If everyone fears making the necessary changes than Gentoo will stagnate and probably die.
How to run an effective meeting on IRC (May 13, 2008, 09:08 UTC)
Since my election to the Gentoo Council, I’ve become the de facto meeting chair and secretary. Over the past 6 months or so, I’ve learned a lot about what works well in online meetings (often by virtue of doing the opposite). By no means have I mastered this, but here’s some of my discoveries along the way. What works well:
What works poorly:
It took a lot of pain and wasted time for me to figure out the value of doing things right, and I’m still working on getting some of the above points right, so I want to save you that same pain. Do you have any more points to add? Please do so in the comments. Thanks! ![]()
Decibel .10 comes to Gentoo (May 13, 2008, 00:25 UTC)
The newest Decibel Audio Player is out, and it fixes several bugs and adds some new features. The ebuild has been updated, so get it here! May 12, 2008
planet, packages: small bugfixes (May 12, 2008, 23:21 UTC)
I took a few minutes today and cleaned up a few small bugs on Planet Larry and friends. One thing I get asked for every now and then is if I have archives of past posts. Well, I do now. I just copy the HTML file of the last post to $date one each run. A simple and unelegant solution. I’ll be doing the same thing for Planet Gentoo soon. Also, fixed the FeedBurner link on the main page — I didn’t even realize it was broken. While I was at it, I created one for Universe as well. On the packages website, I finally fixed it so you can search against just packages again. That’s been annoying me for a while. By default the search is way too wide, I think. It will search the full atom, the package description and the package name. I have to do the package name twice because of regular expressions (starting with, ending with, exact matches, etc.). And there’s still no simple way to search for packages containing multiple words, which is also an annoying little bug. Advanced searches for GPNL and Packages has been something I’ve wanted to do for a long time, but have been putting off since I started the projects. Sheesh. Every time I sit down and start to poke at it, though, I realize just how big a beast it is, based on what I’d like to accomplish. I really need something for the interim, though. Anyway, I better quit before this post gets any more boring. One last thing — we can use more users who are Gentoo users and have a blog on Planet Larry. Just drop me an e-mail and I’ll get you setup.
More gStats (May 12, 2008, 11:50 UTC)
I have not worked on gstats in a while and it definately needed some improvements. I found myself struggling with the UI a lot and after chatting with Graff on irc I decided to drop the webUI entirely and go RESTful. Ewwww REST…. I actually hate REST because all of its supporters claim it is ‘Teh awesome!’ but so very few can tell me why in any sort of convincing manner. ‘It doesn’t have all the overhead of SOAP!’, ‘It uses HTTP!’…these are not convincing arguments. Anyway I figured I’d give it a try. gStats now has no real webUI. Most of the data retrieval functionality is written (aka HTTP GET) using old data in the cache so you can do queries like /api/cp/sys-apps/portage and it will return some XML that describes sys-apps/portage’s installation history. Also available is /api/arch/x86 which will return some XML describing x86’s installation info. I am working on writing POST methods; although I noticed I have a slight issue. Users have hosts with 100’s of packages installed. REST is kind of limiting in this regard; right now I store arches and cps (category/package strings) and via the rest interface you can POST new arch data (not really allowed at this point as it would overwrite old data in my reckoning) and you can POST new cp data (being worked on). But the cp POST model right now kind of takes a cp as a key; maybe I should drop that requirement. Currently: May 11, 2008
Today's mass filings (May 11, 2008, 19:35 UTC)
Our bugzilla today really had an hard day, I’ve started filing bugs for build tools added to runtime-depend of packages, the list is awfully long, and “My Bugs” search now lists over 300 bugs. Why did I do this? Well, we really need to keep the tree clean on these things so that Now, the work today was boring, long and a huge pour of time. I did it gladly without being forced to, I’m fine with it and I’m not expecting anything in particular for what I did. If I didn’t want to do it, I could have just reported the issue, and let someone else doing the thing. This is what volunteer-based work is about. Why do I say this? Well there seems to be people to think that even volunteer work should apply the same rules as businesses. While sometimes you might make use of strategies designed for businesses, like marketing your project better and similar, volunteer works and businesses have vast differences. Sure there are free software projects ran as businesses, but that usually involves people paid to do their job as the core rather than full volunteer-based approaches. So please if you think to run a volunteer project as a business, well think twice or expect serious trouble from the volunteers. So please remember that there are lots of people who are fine to work on a volunteer project that would probably start to be quite nervous if you run it as a business, okay?
Distributions in the Summer of Code (May 11, 2008, 09:34 UTC)
I wrote an article discussing the role of distros in Google’s Summer of Code for this week’s LWN.net. Here’s an excerpt to whet your appetite:
![]() May 09, 2008
all your __guard are belong to __stack_smash_handler (May 09, 2008, 23:35 UTC)
okay here we go
i invented return code 46 as SSP failure because i could not find a list of valid exit codes (unless segfault which is 127) at google. -Alex
Handling File Collisions Between Blocking Packages (May 09, 2008, 16:17 UTC)
If one package blocks another package, the two packages conflict such that they cannot be installed simultaneously. These conflicts are often due to file collisions. In some cases, packages that block each other can be temporarily installed simultaneously. In order to resolve file collisions that occur between two blocking packages that are installed simultaneously, the overlapping files must be removed from the contents list of the package which was installed first. Some cases may exist such that temporary simultaneous installation of blocking packages will cause some sort of problem. However, this type of solution will only be chosen for blockers that can not be satisfied in any other way, such as by simple adjustment of merge order. In addition, this type of solution will not be chosen if a blocking package will overwrite files belonging to packages from the system set, or packages that are runtime dependencies of Portage itself. These constraints serve to limit the probability that a chosen solution will cause an unforeseen problem. gtk-doc-am vs. gtk-doc-1.8-r2 These are the packages that would be merged, in reverse order: Calculating dependencies... done! [blocks b ] dev-util/gtk-doc-am (is blocking dev-util/gtk-doc-1.8-r2) [uninstall ] dev-util/gtk-doc-1.8-r2 USE="emacs -debug -doc" [nomerge ] x11-libs/gtk+-2.12.9-r2 USE="X cups jpeg tiff xinerama -debug -doc -vim-syntax" [ebuild N ] dev-util/gtk-doc-am-1.10-r1 0 kB Total: 1 package (1 new, 1 uninstall), Size of downloads: 0 kB Conflict: 1 block coreutils vs. mktemp These are the packages that would be merged, in reverse order: Calculating dependencies... done! [nomerge ] sys-apps/coreutils-6.11 [6.9-r1] USE="acl nls xattr (-selinux) -static -vanilla%" [blocks b ] sys-apps/mktemp (is blocking sys-apps/coreutils-6.11) [blocks b ] >=sys-apps/coreutils-6.10 (is blocking sys-apps/mktemp-1.5) [uninstall ] sys-apps/mktemp-1.5 [ebuild U ] sys-apps/coreutils-6.11 [6.9-r1] USE="acl nls xattr (-selinux) -static -vanilla%" 0 kB Total: 1 package (1 upgrade, 1 uninstall), Size of downloads: 0 kB Conflict: 2 blocks
the long tail (May 09, 2008, 01:05 UTC)
well it looks like i can go to bed with a smile on my face...
extern unsigned long int __guard; int main(void) { for learning about the whole story: http://bugs.gentoo.org/show_bug.cgi?id=182231 have fun and good night! May 08, 2008
Gzip, Bzip2 and Lzma compared (May 08, 2008, 14:35 UTC)
Why?There has recently been a discussion about GNU switching from bzip2 to lzma for their distributed tarballs. They still offer gzip tarballs as an alternative. However, Gentoo has been preferring the bzip2 tarballs mostly due to the improved pack ratio of bzip2. Unfortunately, the software for lzma is not (yet) as mature as some would like. For example, the format of files produced has changed recently (in a compatible way, though). Also, the current incarnation of the canonical binaries (lzma-utils) by default links against libstdc++.so which is a huge headache for release engineering and the like. What and How?How these distribution problems can/will be solved, remains to be seen. What I'm more interested in, is a comparison of the performance of the three packers. I had initially hoped to also compare the amount of I/O done and memory usage, but GNU time let me down there. GNU time's manpage claims that it can record and output quite a few figures regarding I/O and memory usage. Unfortunately, I have not been able to make time report anything other than 0 for those interesting stats. Not wanting to debug time, I've chosen to do performance tests regarding pack ratio and execution time, instead. I've run all tests in single user mode, so as to not disturb page caches and the like. All tests run from the page cache, to factor out disk latency and possible fragmentation of files. Here's the data I used:
ResultsRed indicates worst, green indicates best result in a category. All times are in seconds, compression ratio is percentage of uncompressed size.
Mostly, this benchmark turns out as expected (and advertised by the lzma authors). Gzip is fastest on compression; bzip2 comes second for compression times and lzma finishes last. Decompression is different: gzip is fastest again, but lzma is faster than bzip2, sometimes nearly by a factor of three. Compression ratio is where gzip shows its age: its compression ratio is worst (except when packing already compressed stuff). Bzip2 bests lzma's compression ration in three cases of five (two of four if you don't count the PNG). On average, however, lzma compresses better by a hair. Lzma's strongest lead over the others is when compressing the ImageMagick module tar. ConclusionLzma is definitely worth a look due to its performance - if you unpack much more often than you pack (three times slower than already-not-so-quick bzip2!). Also, lzma works better on large files. From a distribution standpoint, if/when the library/dependency problems have been sorted out, lzma is quite preferable, if you don't do much compression yourself, yet have to watch your bandwidth usage. For Gentoo, it's reasonable - if the kinks are worked out. Another disadvantage of lzma is that while gzip and bzip2 have their own single-char decompression switch when using tar (z and j, repectively), the new guy on the block only gets a long one: --lzma (introduced in tar 1.20). This might sound minor, but it can quickly get pn your nerves. YMMV. What else to testI'd love to see more tests with real-life data. I also think a comparison of I/O load and memory usage for the three contenders would be interesting. If somebody wants to do all the work, comparing the different speed/compression settings for the three could be interesting - I have only used the default settings, which might not exactly be fair for gzip, which was written at a time when CPU cycles where far more expensive. CommentsJosh "Nightmorph" Saddler writes:
and
I absolutely agree with Josh. The maturity problems aside, being a consumer of lzma archives is nice if you're not short on memory. That said, supporting lzma as a method for upstream tarballs is a nice thing to have. I won't be switching to it for my personal compression needs any time soon. Bonus fact learned about recent tar versionsRecent version of tar autodetect the compression for bzip2 and gzip, which means you can use tar xf linux-2.6.25.tar.bz. Not only that, you can also tell it to guess compression upon creation of archives: tar caf foo.tbz2 /etc. Nifty. Doesn't work for lzma, though, which might be due to the recent change in file format. file doesn't recognize the files, either. Bonus Bonus fact learned: bash completion for tar needs to be fixed. I've taken the liberty of inserting the link to Diego's blog post.
Using a Gentoo Prefixed shell as your login shell (May 08, 2008, 02:04 UTC)
One thing that kind of annoyed me about Gentoo Prefix is that you always needed to do a little work to enter the prefix:
Personally, I just want to always live in a prefixed shell. Amazingly enough, this is really easy to do.
Now, everytime you open a terminal, it will be running a bash gloriously built by Gentoo May 07, 2008
ack - Part II (May 07, 2008, 19:31 UTC)
This is a followup on my earlier post about ack. A few tips on daily usage and a few caveats. Every day is a winding command lineNormally, ack does not search text files. This may come as a surprise since it goes a bit against the grain of what we're used from Unix and Linux commands. Seeing where ack comes from and what its intention is, it's not so unusual. Anyway, we can make it behave the way we want. I'm grateful that ack doesn't search .svn/ and
friends, I also don't mind backup files, swp files, core files etc being
ignored. But I do want to search text files when I
So how does one customise ack? There are two ways: one is an environment variable (ACK_OPTIONS). It can hold all those extra options you always want. There are more environment variables (for coloring, the pager and where the .ackrc (more on that later) resides). They're all described in ack's perldoc page (which can be accessed by perldoc ack (if it's in your PATH) or ack --man. The second way to customize ack is having an .ackrc in your home directory (or wherever the ACKRC environment variable points). It contains all the options just as you would use them on the command line, one per line. You can intersperse it with comments in the usual way. Again, more on this can be found in the perldoc. So now what do we want? This hugely depends on preference, I would call ack like this, usually: ack --text <other options> Since I'm rather lazy, I'd consequently put "--text" in my .ackrc or in ACK_OPTIONS. Whenever this gets in my way, I can use --noenv to ignore all environment and config file settings. Measurements, Unicode and all thatI have an add-on note for the performance checks (I wouldn't call them measurements, far too little diligence on my part) in my last post. It turns out that the locale setting can have a wild amount of influence on the performance of grep. This is especially true of any UTF-8 or similar multibyte encodings. I've made new measurements since the file I used has grown since and I haven't kept the old version. Again, the file is in the page cache before I do my measurements. Also, I've done at least a dozen or so runs each and the numbers I'll show you are typical. $ echo $LC_ALL en_US.utf8 $ time grep 1Jsztn-000647-SL exim_main.log >/dev/null real 0m15.995s user 0m14.770s sys 0m0.160s $ time ack 1Jsztn-000647-SL exim_main.log >/dev/null real 0m3.829s user 0m3.570s sys 0m0.125s $ export LC_ALL=C $ time grep 1Jsztn-000647-SL exim_main.log >/dev/null real 0m0.226s user 0m0.120s sys 0m0.100s $ time ack 1Jsztn-000647-SL exim_main.log >/dev/null real 0m3.871s user 0m3.370s sys 0m0.120s $ As you can see, with the C locale, grep is about 70 times faster than with the multibyte locale and about 17 times faster than ack, which in turn shows no speed difference between the two locales. Now some might make the case that one shouldn't bother with ack since it's so slow. Well, I prefer the 17x penalty if I don't have to muck around with my locale every time I use grep or ack on the command lines. In scripts, things are a bit different, but ack is not aimed at that. Additionally, ack's performance advantage is also generated by not even looking at uninteresting files. For example, searching for "klausman" in a cvs checkout of the Gentoo portage tree takes 8 seconds with ack, but over thirty seconds with grep (locale set to C). All this said, ack is one of the few tools that look like they're intended for use "just like grep" but are actually meant for humans (or programmers ;)). So the final verdict is: both grep and ack have their places, but ack sure eats some of grep's pie. As a side note, the developers of grep might want to look into optimizing their program for multibyte encodings. Not to the detriment of the "classic" encodings, but I think there's some untapped potential. Update: The grep devs already know. May 06, 2008
Using Blockers to Automatically Uninstall Unwanted Packages (May 06, 2008, 18:29 UTC)
A while ago I wrote a blog entry about using blockers to adjust merge order. Now, in portage-2.1.5, blockers are also resolved automatically in cases when it makes sense to uninstall a conflicting package (bug #172812). This feature should allow automatic resolution of blocker conflicts in many more cases than previously possible, so Gentoo users won't be inconvenienced with the task of resolving them manually. gtk-doc-am vs. gtk-doc-1.8-r2 These are the packages that would be merged, in reverse order: Calculating dependencies... done! [nomerge ] x11-libs/gtk+-2.12.9-r2 USE="X cups jpeg tiff xinerama -debug -doc -vim-syntax" [ebuild N ] dev-util/gtk-doc-am-1.10-r1 0 kB [blocks b ] dev-util/gtk-doc-am (is blocking dev-util/gtk-doc-1.8-r2) [uninstall ] dev-util/gtk-doc-1.8-r2 USE="emacs -debug -doc" Total: 1 package (1 new, 1 uninstall), Size of downloads: 0 kB Conflict: 1 block coreutils vs. mktemp These are the packages that would be merged, in reverse order: Calculating dependencies... done! [ebuild U ] sys-apps/coreutils-6.11 [6.9-r1] USE="acl nls xattr (-selinux) -static -vanilla%" 0 kB [blocks b ] sys-apps/mktemp (is blocking sys-apps/coreutils-6.11) [blocks b ] >=sys-apps/coreutils-6.10 (is blocking sys-apps/mktemp-1.5) [uninstall ] sys-apps/mktemp-1.5 Total: 1 package (1 upgrade, 1 uninstall), Size of downloads: 0 kB Conflict: 2 blocks
ack! (thpppt!) (May 06, 2008, 14:44 UTC)
There are some tools that look like you will never replace them. One of those (for me) is grep. It does what it does very well (remarks about the shortcomings of regexen in general aside). It works reasonably well with Unicode/UTF-8 (a great opportunity to Fail Miserably for any tool, viz. a2ps). Yet, the other day I read about ack, which claims to be "better than grep, a search tool for programmers". Woo. Better than grep? In what way? The ack homepage lists the top ten reasons why one should use it instead of grep. Actually, it's thirteen reasons but then some are dupes. So I'd say "about ten reasons". Let's look at them in order.
Bottom line: yes, ack is an exciting new tool which partly replaces grep. That said, a drop-in replacement it ain't. While the standalone version of ack needs nothing but a perl interpreter and its standard modules, for embedded systems that may not work out (vs. the binary with no deps beside a libc). This might also be an issue if you need grep early on during boot and /usr (where your perl resides) isn't mounted yet. Also, default behaviour is divergent enough that it might yield nasty surprises if you just drop in ack instead of grep. Still, I recommend giving ack a try if you ever use grep on the command line. If you're a coder who often needs to search through working copies/checkouts, even more so. UpdateI've written a followup on this, including some tips for day-to-day usage (and an explanation of grep's sucky performance). CommentsRené "Necoro" Neumann writes (in German, translation by me):
'Tis well appreciated. Can't speak for the ebuilds' quality (yet), but I'm sure something will be worked out.
Fingerprint scanning project report published (May 06, 2008, 10:30 UTC)
My fprint fingerprint scanning efforts formed my final year Computer Science project at The University of Manchester. The source code for this project has been available on SourceForge from early on (GPL-2/LGPL-2 licenses). I’ve now completed and submitted a comprehensive project report (similar to a dissertation) for academic assessment, and I’m making this available under a Creative Commons license. You can find the report here. The report is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 2.0 UK: England & Wales License. The academic project is now complete, but I plan to continue development as time permits. There is a lot to be done, and I have already made some good progress on moving libusb-1.0 closer to release. May 05, 2008
planet larry policy update (May 05, 2008, 20:41 UTC)
I’ve made an awkward and difficult decision that I hope doesn’t make anyone feel bad: effective immediately I’ve removed any feeds from Planet Larry / Larry the Universe that were from developers who have retired from the Gentoo project. I setup the planet feeds mainly so that regular users of Gentoo Linux could talk about and share their experiences, and I want to get back to keeping it with them primarily in mind. I tend to think that ex-developers have more weight with their posts, which tends to cause some unbalance that I was never comfortable with. Speaking of users, I am always looking for new bloggers to get added, so if you are a Gentoo user who blogs about computer experiences, be sure to sign up. That’s all for now. Edit: I originally took this post down, and am now restoring it. I still haven’t decided if this is a permanent change or not. I might create a planet just for ex-developers, or reintegrate them somehow. I dunno. Comments and suggestions are welcome.
Managing RubyGems on Gentoo (May 05, 2008, 03:04 UTC)
For Ruby on Gentoo, it has been asked how one should go about managing RubyGems. You have a few options:
Mix and matchOffhand, mixing and matching seems like a bad idea. Doing so, you might install a gem using portage, but then you could use Overall, seems like it could be problematic in the long run. Gentoo packages exclusivelyBeing a Gentoo packager, my first take is that using packages is the way to go. Some pluses:
Of course, there are a number of disadvantages.
Gems exclusivelyIf you're a Ruby developer, the most natural way would be using RubyGem directly. Some nice things include:
Of course, not without their drawbacks:
My current practicesSo where does that bring us? Here are the practices I've taken to using for my Ruby and Rails development and deployment needs. In developmentOn my development machine (4 MacBook running Leopard with Gentoo/Prefix), I've using RubyGems directly. If I'm working on a Rails app, and it needs any gems, I've taken to vendoring everything. This type of thing will be supported out of the box for Rails 2.1. Of course, this wouldn't work so well for things with native extensions, like hpricot. In productionI avoid installing any gems in production as much as possible. If an app needs something, it should be vendored, like I just described. For gems needed by my apps that have native extensions, or are otherwise needed in production (like rake and the mysql gem), again, I'm using RubyGem directly. May 03, 2008
On cooperating and paludis vulnerability (May 03, 2008, 22:24 UTC)
A serious security issue in paludis was brought to my attention recently, and I feel I should make you all aware. Apparently someone, with root access to a machine, can gain root access by installing or editing paludis config files. For those interested, this is how it happened (times are GMT+1): 22:34 <@ferdy> bonsaikitten: can you give me any details regarding that security bug in paludis? 22:35 <+bonsaikitten> ferdy: it's so obvious you should have found it already 22:37 <@ferdy> bonsaikitten: I should, but I probably haven't 22:37 <+bonsaikitten> ferdy: well, as I am a moron I'm unable to coherently explain :) 22:37 <@ferdy> bonsaikitten: I mean, depends on whether we are talking about a real security issue or about something we should document to avoid people shooting themselves in the foot 22:39 <@ferdy> bonsaikitten: is that all you are going to tell me? 22:39 <+bonsaikitten> ferdy: come on, it's obvious. You're supposed to be smart ... 22:39 * bonsaikitten is not in a mood to explain 22:40 <@ferdy> bonsaikitten: you aren't really talking about the paludisbuild issue, are you? 22:41 <+bonsaikitten> mmh no, that's a different one 22:41 <@ferdy> k 22:41 <@ferdy> bonsaikitten: what are we talking about? 22:42 <@ferdy> bonsaikitten: you don't need to explain it... just say, in general terms, what the issue is 22:50 <@ferdy> bonsaikitten: so? care to give any useful hint? 22:50 <+bonsaikitten> ferdy: doesn't happen in portage compatibility mode 22:51 <+bonsaikitten> but I blame the vodka, hard to explain when *burp* *giggle* 22:52 <@ferdy> bonsaikitten: what's the impact? 22:53 <+bonsaikitten> ferdy: depends on how annoying the other person is 22:54 <+bonsaikitten> ferdy: worst case random file modification 22:58 <@ferdy> bonsaikitten: and we already agreed that we aren't talking about the paludisbuild issue, right? 22:59 <@ferdy> bonsaikitten: if we aren't, I'll need more hints.... 23:05 <@ferdy> bonsaikitten: can I get an attack vector? 23:05 <@ferdy> that shouldn't need lots of explaining... I can figure out that part myself 23:19 <@ferdy> bonsaikitten: have you got that attack vector for me? 23:24 <+bonsaikitten> ferdy: look at configuration files, maybe you notice that there's some exquisit code execution possible there 23:29 <@ferdy> bonsaikitten: you mean those config files that only root can edit? I must be missing something here 23:29 <+bonsaikitten> ferdy: you are :) 23:29 <+bonsaikitten> not much, and it's basically the same flaw bashrc is for portage 23:29 <+bonsaikitten> only that bashrc is config_protect'ed ... 23:30 <@ferdy> bonsaikitten: but for a package to clover those files, it must be in a repo root added, right? 23:31 <+bonsaikitten> someone in the package mangler group, but yes 23:35 <@ferdy> bonsaikitten: but if you can change those files in the first place, why clover them by adding a malicious repo with a malicious package that changes those files? 23:35 <+bonsaikitten> ferdy: because it's very subtle 23:36 <@ferdy> moreover, if you can already do that, why not just make the package install whatever backdoor you want? 23:37 <@ferdy> I mean, it is subtle, but why would anyone go the 'convoluted' route? he is already able to edit those files (since he had to add that repo) 23:38 <+bonsaikitten> 'cause only paludis is affected and you will find it very hard to trace 23:38 <+bonsaikitten> that makes it so tempting ... 23:40 <+bonsaikitten> just don't be surprised if it suddenly unmerges itself :) 23:41 <@ferdy> yeah... well... 23:41 <@ferdy> bonsaikitten: mind if I disclose this vulnerability in planet.gentoo.org? 23:42 <+bonsaikitten> go ahead 23:42 <@ferdy> ta 23:42 <+bonsaikitten> 't is even on the features page of the package mangler :) This is a good lesson to learn today: If you can edit files owned by root in a machine, you can get root access to that machine. So the bottom line is: There is no vulnerability, if you can mangle paludis config files, you are already root so you don't need to edit a file to run any command you want. Another lesson one can learn by reading that log is how to be really cooperative. Ah, and before someone with a need to use cheap psychology asks, the intention of this blag post is to stop the FUD. - ferdy May 02, 2008
Replying to Ryan's post... https://bugzilla.mozilla.org/show_bug.cgi?id=384090#c41 And I'll just quote that comment here:
Let's not spread unnecessary false information, shall we?
Better late than sorry (May 02, 2008, 09:12 UTC)
Or whatever the proverb was...anyway, here is my meme list: Desktop user: 77 cd Chroot (for architecture work): 118 gatt | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||