Planet Gentoo

Diego Pettenò: Today's mass filings

Diego Pettenò (flameeyes) — Sun, 11 May 2008 19:35:00 +0000

Our bugzilla today really had an hard day, I’ve started filing bugs for build tools added to runtime-depend of packages, the list is awfully long, and “My Bugs” search now lists over 300 bugs.

Why did I do this? Well, we really need to keep the tree clean on these things so that --depclean works as expected. But mostly my interest is for final system deployment. On my vserver, as well as on a final target for a cross-compiled embedded system, you don’t want packages like flex, bison or swig, and this is quite possible if the packages are not in RDEPEND, when using ROOT= or binary packages.

Now, the work today was boring, long and a huge pour of time. I did it gladly without being forced to, I’m fine with it and I’m not expecting anything in particular for what I did. If I didn’t want to do it, I could have just reported the issue, and let someone else doing the thing. This is what volunteer-based work is about.

Why do I say this? Well there seems to be people to think that even volunteer work should apply the same rules as businesses. While sometimes you might make use of strategies designed for businesses, like marketing your project better and similar, volunteer works and businesses have vast differences.

Sure there are free software projects ran as businesses, but that usually involves people paid to do their job as the core rather than full volunteer-based approaches. So please if you think to run a volunteer project as a business, well think twice or expect serious trouble from the volunteers.

So please remember that there are lots of people who are fine to work on a volunteer project that would probably start to be quite nervous if you run it as a business, okay?

Donnie Berkholz: Distributions in the Summer of Code

Donnie Berkholz (dberkholz) — Sun, 11 May 2008 09:34:45 +0000

I wrote an article discussing the role of distros in Google’s Summer of Code for this week’s LWN.net. Here’s an excerpt to whet your appetite:

Aren’t they just writing packages? What would they do with a Summer of Code project?

Read on to find out.

Alexander Gabert: all your guard are belong to stack_smash_handler

Alexander Gabert (pappy) — Fri, 09 May 2008 23:35:11 +0000

okay here we go

>>> Regenerating /etc/ld.so.cache...
>>> sys-libs/gxslibc-2.6.1-r2 merged.

>>> No packages selected for removal by clean
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.
* Regenerating GNU info directory index...
* Processed 87 info files.

TMPFS chroot001 miranda ~ # export STATIC="-fstack-protector-all"; gcc-3.4.6 "${STATIC}" -fstack-protector-all -o vuln-stack vuln-stack.c && file vuln-stack && readelf -s vuln-stack | egrep "__guard|__stack_smash"; ./vuln-stack 1234567891234567; einfo "return code: ${?}"; echo; gcc-3.4.6 "${STATIC}" -fstack-protector-all -o ssp_entropy ssp_entropy.c && file ssp_entropy && ./ssp_entropy
vuln-stack: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.18, dynamically linked (uses shared libs), not stripped
2: 08049698 4 OBJECT GLOBAL DEFAULT 23 __guard@GLIBC_2.3.2 (3)
4: 00000000 30 FUNC GLOBAL DEFAULT UND __stack_smash_handler@GLIBC_2.3.2 (3)
78: 08049698 4 OBJECT GLOBAL DEFAULT 23 __guard@@GLIBC_2.3.2
80: 00000000 30 FUNC GLOBAL DEFAULT UND __stack_smash_handler@@GL
* return code: 46

ssp_entropy: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.18, dynamically linked (uses shared libs), not stripped
__guard: [[0x288a2b8c]]

TMPFS chroot001 miranda ~ # export STATIC="-static"; gcc-3.4.6 "${STATIC}" -fstack-protector-all -o vuln-stack vuln-stack.c && file vuln-stack && readelf -s vuln-stack | egrep "__guard|__stack_smash"; ./vuln-stack 1234567891234567; einfo "return code: ${?}"; echo; gcc-3.4.6 "${STATIC}" -fstack-protector-all -o ssp_entropy ssp_entropy.c && file ssp_entropy && ./ssp_entropy
vuln-stack: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.18, statically linked, not stripped
1346: 0804f810 18 FUNC GLOBAL DEFAULT 3 __stack_smash_handler
1554: 080bc370 4 OBJECT GLOBAL DEFAULT 16 __guard
* return code: 46

ssp_entropy: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.18, statically linked, not stripped
__guard: [[0xe686ece4]]

i invented return code 46 as SSP failure because i could not find a list of valid exit codes (unless segfault which is 127) at google.

-Alex

Zack Medico: Handling File Collisions Between Blocking Packages

Zack Medico (zmedico) — Fri, 09 May 2008 16:17:21 +0000

Blocker Conflicts

If one package blocks another package, the two packages conflict such that they cannot be installed simultaneously. These conflicts are often due to file collisions. In some cases, packages that block each other can be temporarily installed simultaneously. In order to resolve file collisions that occur between two blocking packages that are installed simultaneously, the overlapping files must be removed from the contents list of the package which was installed first.

gtk-doc-am vs. gtk-doc-1.8-r2

These are the packages that would be merged, in reverse order:

Calculating dependencies... done!
[blocks b     ] dev-util/gtk-doc-am (is blocking dev-util/gtk-doc-1.8-r2)
[uninstall    ]  dev-util/gtk-doc-1.8-r2  USE="emacs -debug -doc"
[nomerge      ] x11-libs/gtk+-2.12.9-r2  USE="X cups jpeg tiff xinerama -debug -doc -vim-syntax"
[ebuild  N    ]  dev-util/gtk-doc-am-1.10-r1  0 kB

Total: 1 package (1 new, 1 uninstall), Size of downloads: 0 kB
Conflict: 1 block

coreutils vs. mktemp

These are the packages that would be merged, in reverse order:

Calculating dependencies... done!
[nomerge      ] sys-apps/coreutils-6.11 [6.9-r1] USE="acl nls xattr (-selinux) -static -vanilla%"
[blocks b     ]  sys-apps/mktemp (is blocking sys-apps/coreutils-6.11)
[blocks b     ] >=sys-apps/coreutils-6.10 (is blocking sys-apps/mktemp-1.5)
[uninstall    ]  sys-apps/mktemp-1.5
[ebuild     U ] sys-apps/coreutils-6.11 [6.9-r1] USE="acl nls xattr (-selinux) -static -vanilla%" 0 kB

Total: 1 package (1 upgrade, 1 uninstall), Size of downloads: 0 kB
Conflict: 2 blocks

Alexander Gabert: the long tail

Alexander Gabert (pappy) — Fri, 09 May 2008 01:05:06 +0000

well it looks like i can go to bed with a smile on my face...

gcc -g -ggdb -fstack-protector-all -o ssp_entropy ssp_entropy.c ./ssp_entropy __guard: [[0x353275b9]]

gcc -static -g -ggdb -fstack-protector-all -o ssp_entropy ssp_entropy.c && ./ssp_entropy __guard: [[0x3687e720]]

TMPFS chroot001 miranda ~ # cat ssp_entropy.c #include stdio.h

extern unsigned long int __guard;

int main(void) {
printf("__guard: [[0x%x]]\n", __guard);
while(1) { ; }
return(0);
}

for learning about the whole story: http://bugs.gentoo.org/show_bug.cgi?id=182231

have fun and good night!

Tobias Klausmann: Gzip, Bzip2 and Lzma compared

Tobias Klausmann (klausman) — Thu, 08 May 2008 14:35:13 +0000

Why?

There has recently been a discussion about GNU switching from bzip2 to lzma for their distributed tarballs. They still offer gzip tarballs as an alternative. However, Gentoo has been preferring the bzip2 tarballs mostly due to the improved pack ratio of bzip2.

Unfortunately, the software for lzma is not (yet) as mature as some would like. For example, the format of files produced has changed recently (in a compatible way, though). Also, the current incarnation of the canonical binaries (lzma-utils) by default links against libstdc++.so which is a huge headache for release engineering and the like.

What and How?

How these distribution problems can/will be solved, remains to be seen. What I'm more interested in, is a comparison of the performance of the three packers. I had initially hoped to also compare the amount of I/O done and memory usage, but GNU time let me down there.

GNU time's manpage claims that it can record and output quite a few figures regarding I/O and memory usage. Unfortunately, I have not been able to make time report anything other than 0 for those interesting stats. Not wanting to debug time, I've chosen to do performance tests regarding pack ratio and execution time, instead.

I've run all tests in single user mode, so as to not disturb page caches and the like. All tests run from the page cache, to factor out disk latency and possible fragmentation of files.

Here's the data I used:

lenna.tiff The compression testing poster girl
lenna.png The same image as above, as a PNG, to see what the algorithms make of already-compressed data
linux-2.6.25.tar The sources for the linux kernel, version 2.6.25
wrnpc12.txt Leo Tolstoy: War and Peace. From Project Gutenberg. ASCII Plaintext.
ImageMagick-6.4.0_modules-Q16.tar A tar file /usr/lib64/ImageMagick-6.4.0/modules-Q16/ from my amd64 system, containing many .so, .la and .a files

Results

Red indicates worst, green indicates best result in a category. All times are in seconds, compression ratio is percentage of uncompressed size.

Input data	Algo	Time (C)	Time (D)	Ratio
ImageMagick modules	`gzip`	0.30	0.04	25.28%
	`bzip2`	0.61	0.27	20.44%
	`lzma`	4.97	0.10	12.64%

Lena (TIFF)	`gzip`	0.05	0.01	93.24%
	`bzip2`	0.15	0.08	74.31%
	`lzma`	0.33	0.06	80.69%

Lena (PNG)	`gzip`	0.02	0.00	100.02%
	`bzip2`	0.13	0.05	100.32%
	`lzma`	0.18	0.04	100.67%

Linux Sources	`gzip`	12.98	1.88	21.81%
	`bzip2`	54.61	14.25	17.07%
	`lzma`	287.58	5.46	14.47%

War and Peace	`gzip`	0.30	0.03	37.00%
	`bzip2`	0.58	0.26	26.96%
	`lzma`	3.30	0.09	28.34%

Overall (Average)	`gzip`	2.73	0.39	55.47%
	`bzip2`	11.22	2.98	47.82%
	`lzma`	59.27	1.15	47.36%

Mostly, this benchmark turns out as expected (and advertised by the lzma authors). Gzip is fastest on compression; bzip2 comes second for compression times and lzma finishes last. Decompression is different: gzip is fastest again, but lzma is faster than bzip2, sometimes nearly by a factor of three. Compression ratio is where gzip shows its age: its compression ratio is worst (except when packing already compressed stuff). Bzip2 bests lzma's compression ration in three cases of five (two of four if you don't count the PNG). On average, however, lzma compresses better by a hair. Lzma's strongest lead over the others is when compressing the ImageMagick module tar.

Conclusion

Lzma is definitely worth a look due to its performance - if you unpack much more often than you pack (three times slower than already-not-so-quick bzip2!). Also, lzma works better on large files. From a distribution standpoint, if/when the library/dependency problems have been sorted out, lzma is quite preferable, if you don't do much compression yourself, yet have to watch your bandwidth usage. For Gentoo, it's reasonable - if the kinks are worked out. Another disadvantage of lzma is that while gzip and bzip2 have their own single-char decompression switch when using tar (z and j, repectively), the new guy on the block only gets a long one: --lzma (introduced in tar 1.20). This might sound minor, but it can quickly get pn your nerves. YMMV.

What else to test

I'd love to see more tests with real-life data. I also think a comparison of I/O load and memory usage for the three contenders would be interesting. If somebody wants to do all the work, comparing the different speed/compression settings for the three could be interesting - I have only used the default settings, which might not exactly be fair for gzip, which was written at a time when CPU cycles where far more expensive.

Comments

Josh "Nightmorph" Saddler writes:

Thanks for the blog and the benchies. Some interesting stuff here.

I still don't know that I'd go for lzma, because it's compression time sucks assballs. bzip2 is bad enough as it is. lzma is at the back of the pack for compression, every time.

The numbers for Linux sources aren't very encouraging. lzma takes a metric assload of time to compress it. Far, far too long. At least it's quick to decompress the same file.

I guess if we were to mostly move to lzma archives, it wouldn't be so bad on the package installation side. I mean, most of what we do as Gentoo users is unpack tarballs for installation, right? Anything to speed up this process is probably a step in the right direction ... I just wouldn't use lzma for anything else. For Gentoo, it's fine. For personal stuff, it's not.

and

One thing I didn't even think about was about postinstall compression. Diego mentioned this over at his blog — every merge operation is going to have some compression for things like docs, manpages, etc.

I'd completely forgotten about the stuff that gets compressed when installed. lzma doesn't scale particularly well to small files, not like gzip does.

I guess it's a tradeoff — you can unpack large archives like kernel sources very quickly, but you'll get (a bit?) more time added to each merge op if lzma is doing the postinstall compression.

I absolutely agree with Josh. The maturity problems aside, being a consumer of lzma archives is nice if you're not short on memory. That said, supporting lzma as a method for upstream tarballs is a nice thing to have. I won't be switching to it for my personal compression needs any time soon.

Bonus fact learned about recent tar versions

Recent version of tar autodetect the compression for bzip2 and gzip, which means you can use tar xf linux-2.6.25.tar.bz. Not only that, you can also tell it to guess compression upon creation of archives: tar caf foo.tbz2 /etc. Nifty. Doesn't work for lzma, though, which might be due to the recent change in file format. file doesn't recognize the files, either.

Bonus Bonus fact learned: bash completion for tar needs to be fixed.

I've taken the liberty of inserting the link to Diego's blog post.

Joshua Nichols: Using a Gentoo Prefixed shell as your login shell

Joshua Nichols (nichoj) — Thu, 08 May 2008 02:04:46 +0000

One thing that kind of annoyed me about Gentoo Prefix is that you always needed to do a little work to enter the prefix:

$ ./bootstrap-prefix.sh /path/to/prefix startscript

Personally, I just want to always live in a prefixed shell. Amazingly enough, this is really easy to do.

First open up /etc/shells as root
Add a line like /path/to/prefix/bin/bash and save
As your user, run: chsh -s /path/to/prefix/bin/bash
Enter your password
Done

Now, everytime you open a terminal, it will be running a bash gloriously built by Gentoo

Tobias Klausmann: ack - Part II

Tobias Klausmann (klausman) — Wed, 07 May 2008 19:31:43 +0000

This is a followup on my earlier post about ack. A few tips on daily usage and a few caveats.

Every day is a winding command line

Normally, ack does not search text files. This may come as a surprise since it goes a bit against the grain of what we're used from Unix and Linux commands. Seeing where ack comes from and what its intention is, it's not so unusual. Anyway, we can make it behave the way we want.

I'm grateful that ack doesn't search .svn/ and friends, I also don't mind backup files, swp files, core files etc being ignored. But I do want to search text files when I ~~grep~~ ack through source files. What if the elusive fixAllBugs() function I'm looking for is only mentioned in a README file?

So how does one customise ack? There are two ways: one is an environment variable (ACK_OPTIONS). It can hold all those extra options you always want. There are more environment variables (for coloring, the pager and where the .ackrc (more on that later) resides). They're all described in ack's perldoc page (which can be accessed by perldoc ack (if it's in your PATH) or ack --man.

The second way to customize ack is having an .ackrc in your home directory (or wherever the ACKRC environment variable points). It contains all the options just as you would use them on the command line, one per line. You can intersperse it with comments in the usual way. Again, more on this can be found in the perldoc.

So now what do we want? This hugely depends on preference, I would call ack like this, usually:

ack --text <other options>

Since I'm rather lazy, I'd consequently put "--text" in my .ackrc or in ACK_OPTIONS. Whenever this gets in my way, I can use --noenv to ignore all environment and config file settings.

Measurements, Unicode and all that

I have an add-on note for the performance checks (I wouldn't call them measurements, far too little diligence on my part) in my last post. It turns out that the locale setting can have a wild amount of influence on the performance of grep. This is especially true of any UTF-8 or similar multibyte encodings.

I've made new measurements since the file I used has grown since and I haven't kept the old version. Again, the file is in the page cache before I do my measurements. Also, I've done at least a dozen or so runs each and the numbers I'll show you are typical.

$ echo $LC_ALL
en_US.utf8
$ time grep 1Jsztn-000647-SL exim_main.log >/dev/null
real    0m15.995s
user    0m14.770s
sys     0m0.160s
$ time ack 1Jsztn-000647-SL exim_main.log >/dev/null
real    0m3.829s
user    0m3.570s
sys     0m0.125s
$ export LC_ALL=C
$ time grep 1Jsztn-000647-SL exim_main.log >/dev/null
real    0m0.226s
user    0m0.120s
sys     0m0.100s
$ time ack 1Jsztn-000647-SL exim_main.log >/dev/null
real    0m3.871s
user    0m3.370s
sys     0m0.120s
$

As you can see, with the C locale, grep is about 70 times faster than with the multibyte locale and about 17 times faster than ack, which in turn shows no speed difference between the two locales.

Now some might make the case that one shouldn't bother with ack since it's so slow. Well, I prefer the 17x penalty if I don't have to muck around with my locale every time I use grep or ack on the command lines. In scripts, things are a bit different, but ack is not aimed at that. Additionally, ack's performance advantage is also generated by not even looking at uninteresting files. For example, searching for "klausman" in a cvs checkout of the Gentoo portage tree takes 8 seconds with ack, but over thirty seconds with grep (locale set to C).

All this said, ack is one of the few tools that look like they're intended for use "just like grep" but are actually meant for humans (or programmers ;)).

So the final verdict is: both grep and ack have their places, but ack sure eats some of grep's pie.

As a side note, the developers of grep might want to look into optimizing their program for multibyte encodings. Not to the detriment of the "classic" encodings, but I think there's some untapped potential.

Update: The grep devs already know.

Zack Medico: Using Blockers to Automatically Uninstall Unwanted Packages

Zack Medico (zmedico) — Tue, 06 May 2008 18:29:58 +0000

A while ago I wrote a blog entry about using blockers to adjust merge order. Now, in portage-2.1.5, blockers are also resolved automatically in cases when it makes sense to uninstall a conflicting package (bug #172812). This feature should allow automatic resolution of blocker conflicts in many more cases than previously possible, so Gentoo users won't be inconvenienced with the task of resolving them manually.

gtk-doc-am vs. gtk-doc-1.8-r2

These are the packages that would be merged, in reverse order:

Calculating dependencies... done!
[nomerge      ] x11-libs/gtk+-2.12.9-r2  USE="X cups jpeg tiff xinerama -debug -doc -vim-syntax"
[ebuild  N    ]  dev-util/gtk-doc-am-1.10-r1  0 kB
[blocks b     ] dev-util/gtk-doc-am (is blocking dev-util/gtk-doc-1.8-r2)
[uninstall    ]  dev-util/gtk-doc-1.8-r2  USE="emacs -debug -doc"

Total: 1 package (1 new, 1 uninstall), Size of downloads: 0 kB
Conflict: 1 block

coreutils vs. mktemp

These are the packages that would be merged, in reverse order:

Calculating dependencies... done!
[ebuild     U ] sys-apps/coreutils-6.11 [6.9-r1] USE="acl nls xattr (-selinux) -static -vanilla%" 0 kB
[blocks b     ]  sys-apps/mktemp (is blocking sys-apps/coreutils-6.11)
[blocks b     ] >=sys-apps/coreutils-6.10 (is blocking sys-apps/mktemp-1.5)
[uninstall    ]  sys-apps/mktemp-1.5

Total: 1 package (1 upgrade, 1 uninstall), Size of downloads: 0 kB
Conflict: 2 blocks

Tobias Klausmann: ack! (thpppt!)

Tobias Klausmann (klausman) — Tue, 06 May 2008 14:44:51 +0000

There are some tools that look like you will never replace them. One of those (for me) is grep. It does what it does very well (remarks about the shortcomings of regexen in general aside). It works reasonably well with Unicode/UTF-8 (a great opportunity to Fail Miserably for any tool, viz. a2ps).

Yet, the other day I read about ack, which claims to be "better than grep, a search tool for programmers". Woo. Better than grep? In what way?

The ack homepage lists the top ten reasons why one should use it instead of grep. Actually, it's thirteen reasons but then some are dupes. So I'd say "about ten reasons". Let's look at them in order.

It's blazingly fast because it only searches the stuff you want searched.

Wait, how does it know what I want? A DWIM-Interface at last? Not quite. First off, ack is faster than grep for simple searches. Here's an example:
```
$ time ack 1Jsztn-000647-SL exim_main.log >/dev/null
real    0m3.463s
user    0m3.280s
sys     0m0.180s
$ time grep -F 1Jsztn-000647-SL exim_main.log >/dev/null
real    0m14.957s
user    0m14.770s
sys     0m0.160s
```
Two notes: first, yes, the file was in the page cache before I ran ack; second, I even made it easy for grep by telling it explicitly I was looking for a fixed string (not that it helped much, the same command without -F was faster by about 0.1s). Oh and for completeness, the exim logfile I searched has about two million lines and is 250M. I've run those tests ten times for each, the times shown above are typical.

So yes, for simple searches, ack is faster than grep. Let's try with a more complicated pattern, then. This time, let's use the pattern (klausman|gentoo) on the same file. Note that we have to use -E for grep to use extended regexen, which ack in turn does not need, since it (almost) always uses them. Here, grep takes its sweet time: 3:56, nearly four minutes. In contrast, ack accomplished the same task in 49 seconds (all times averaged over ten runs, then rounded to integer seconds).

As for the "being clever" side of speed, see below, points 5 and 6
ack is pure Perl, so it runs on Windows just fine.

This isn't relevant to me, since I don't use windows for anything where I might need grep. That said, it might be a killer feature for others.
The standalone version uses no non-standard modules, so you can put it in your ~/bin without fear.

Ok, this is not so much of a feature than a hard criterion. If I needed extra modules for the whole thing to run, that'd be a deal breaker. I already have tons of libraries, I don't need more undergrowth around my dependency tree.
Searches recursively through directories by default, while ignoring .svn, CVS and other VCS directories.

This is a feature, yet one that wouldn't pry me away from grep: -r is there (though it distinctly feels like an afterthought). Since ack ignores a certain set of files and directories, its recursive capabilities where there from the start, making it feel more seamless.
ack ignores most of the crap you don't want to search

To be precise:
- VCS directories
- blib, the Perl build directory
- backup files like foo~ and #foo#
- binary files, core dumps, etc.
Most of the time, I don't want to search those (and have to exclude them with grep -v from find results). Of course, this ignore-mode can be switched off with ack (-u). All that said, it sure makes command lines shorter (and easier to read and construct). Also, this is the first spot where ack's Perl-centricism shows. I don't mind, even though I prefer that other language with P.
Ignoring .svn directories means that ack is faster than grep for searching through trees.

Dupe. See Point 5
Lets you specify file types to search, as in --perl or --nohtml.

While at first glance, this may seem limited, ack comes with a plethora of definitions (45 if I counted correctly), so it's not as perl-centric as it may seem from the example. This feature saves command-line space (if there's such a thing), since it avoids wild find-constructs. The docs mention that --perl also checks the shebang line of files that don't have a suffix, but make no mention of the other "shipped" file type recognizers doing so.
File-filtering capabilities usable without searching with ack -f. This lets you create lists of files of a given type.

This mostly is a consequence of the feature above. Even if it weren't there, you could simply search for "."
Color highlighting of search results.

While I've looked upon color in shells as kinda childish for a while, I wouldn't want to miss syntax highlighting in vim, colors for ls (if they're not as sucky as the defaults we had for years) or match highlighting for grep. It's really neat to see that yes, the pattern you grepped for indeed matches what you think it does. Especially during evolutionary construction of command lines and shell scripts.
Uses real Perl regular expressions, not a GNU subset

Again, this doesn't bother me much. I use egrep/grep -E all the time, anyway. And I'm no Perl programmer, so I don't get withdrawal symptoms every time I use another regex engine.
Allows you to specify output using Perl's special variables

This sounds neat, yet I don't really have a use case for it. Also, my perl-fu is weak, so I probably won't use it anyway. Still, might be a killer feature for you.

The docs have an example:

ack '(Mr|Mr?s)\. (Smith|Jones)' --output='$&'
Many command-line switches are the same as in GNU grep:

Specifically mentioned are -w, -c and -l. It's always nice if you don't have to look up all the flags every time.
Command name is 25% fewer characters to type! Save days of free-time! Heck, it's 50% shorter compared to grep -r

Okay, now we have proof that not only the ack webmaster can't count, he's also making up reasons for fun. Works for me.

Bottom line: yes, ack is an exciting new tool which partly replaces grep. That said, a drop-in replacement it ain't. While the standalone version of ack needs nothing but a perl interpreter and its standard modules, for embedded systems that may not work out (vs. the binary with no deps beside a libc). This might also be an issue if you need grep early on during boot and /usr (where your perl resides) isn't mounted yet. Also, default behaviour is divergent enough that it might yield nasty surprises if you just drop in ack instead of grep. Still, I recommend giving ack a try if you ever use grep on the command line. If you're a coder who often needs to search through working copies/checkouts, even more so.

Update

I've written a followup on this, including some tips for day-to-day usage (and an explanation of grep's sucky performance).

Comments

René "Necoro" Neumann writes (in German, translation by me):

Stumbled across your blog entry about "ack" today. I tried it and found it to be cool :). So I created two ebuilds for it:

sys-apps/ack

dev-perl/File-Next

Just wanted to let you know (there is no comment function on your blog).

'Tis well appreciated. Can't speak for the ebuilds' quality (yet), but I'm sure something will be worked out.

Daniel Drake: Fingerprint scanning project report published

Daniel Drake (dsd) — Tue, 06 May 2008 10:30:20 +0000

My fprint fingerprint scanning efforts formed my final year Computer Science project at The University of Manchester.

The source code for this project has been available on SourceForge from early on (GPL-2/LGPL-2 licenses). I’ve now completed and submitted a comprehensive project report (similar to a dissertation) for academic assessment, and I’m making this available under a Creative Commons license. You can find the report here.

The report is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 2.0 UK: England & Wales License.

The academic project is now complete, but I plan to continue development as time permits. There is a lot to be done, and I have already made some good progress on moving libusb-1.0 closer to release.

Joshua Nichols: Managing RubyGems on Gentoo

Joshua Nichols (nichoj) — Mon, 05 May 2008 03:04:16 +0000

For Ruby on Gentoo, it has been asked how one should go about managing RubyGems. You have a few options:

Use Gentoo packages exclusively
Use gem exclusively
Mix and match and pray

Mix and match

Offhand, mixing and matching seems like a bad idea.

Doing so, you might install a gem using portage, but then you could use gem update to update it. Then you update it with portage...

Overall, seems like it could be problematic in the long run.

Gentoo packages exclusively

Being a Gentoo packager, my first take is that using packages is the way to go.

Some pluses:

One stop for installing/updating packages, ruby or otherwise
For gems with native extensions, can have dependencies on the things the native extensions use

Of course, there are a number of disadvantages.

There are tons of gems out there
Some update frequently
Lot of packages to be keeping up with stable keywords
Despite gem ebuilds being pretty straightforward, there is a bit of grunt work involved on to verify dependencies are accurate
Lots of gems are unpackaged, because they are usually packaged as requested or as developers need them
Despite being able to install multiple gems at once, packages usually aren't slotted that way

Gems exclusively

If you're a Ruby developer, the most natural way would be using RubyGem directly. Some nice things include:

Most direct way to have the latest and greatest gems
Can have as many versions installed as you want
Zero maintenance for Gentoo packagers (sweet!)

Of course, not without their drawbacks:

If a gem uses native extensions, it can be tricky to determine what the it's dependencies are
Problematic when other (non-ruby) packages need to depend on gems

My current practices

So where does that bring us?

Here are the practices I've taken to using for my Ruby and Rails development and deployment needs.

In development

On my development machine (4 MacBook running Leopard with Gentoo/Prefix), I've using RubyGems directly.

If I'm working on a Rails app, and it needs any gems, I've taken to vendoring everything. This type of thing will be supported out of the box for Rails 2.1.

Of course, this wouldn't work so well for things with native extensions, like hpricot.

In production

I avoid installing any gems in production as much as possible. If an app needs something, it should be vendored, like I just described.

For gems needed by my apps that have native extensions, or are otherwise needed in production (like rake and the mysql gem), again, I'm using RubyGem directly.

Fernando J. Pereda: On cooperating and paludis vulnerability

Fernando J. Pereda (ferdy) — Sat, 03 May 2008 22:24:10 +0000

A serious security issue in paludis was brought to my attention recently, and I feel I should make you all aware. Apparently someone, with root access to a machine, can gain root access by installing or editing paludis config files.

For those interested, this is how it happened (times are GMT+1):

22:34 <@ferdy> bonsaikitten: can you give me any details regarding that
 security bug in paludis?
22:35 <+bonsaikitten> ferdy: it's so obvious you should have found it already
22:37 <@ferdy> bonsaikitten: I should, but I probably haven't
22:37 <+bonsaikitten> ferdy: well, as I am a moron I'm unable to coherently explain :)
22:37 <@ferdy> bonsaikitten: I mean, depends on whether we are talking about
a real security issue or about something we should document to avoid people
shooting themselves in the foot
22:39 <@ferdy> bonsaikitten: is that all you are going to tell me?
22:39 <+bonsaikitten> ferdy: come on, it's obvious. You're supposed to be smart ...
22:39 * bonsaikitten is not in a mood to explain
22:40 <@ferdy> bonsaikitten: you aren't really talking about the paludisbuild issue, are you?
22:41 <+bonsaikitten> mmh no, that's a different one
22:41 <@ferdy> k
22:41 <@ferdy> bonsaikitten: what are we talking about?
22:42 <@ferdy> bonsaikitten: you don't need to explain it... just say, in general 
terms, what the issue is
22:50 <@ferdy> bonsaikitten: so? care to give any useful hint?
22:50 <+bonsaikitten> ferdy: doesn't happen in portage compatibility mode
22:51 <+bonsaikitten> but I blame the vodka, hard to explain when *burp* *giggle*
22:52 <@ferdy> bonsaikitten: what's the impact?
22:53 <+bonsaikitten> ferdy: depends on how annoying the other person is
22:54 <+bonsaikitten> ferdy: worst case random file modification
22:58 <@ferdy> bonsaikitten: and we already agreed that we aren't talking about
the paludisbuild issue, right?
22:59 <@ferdy> bonsaikitten: if we aren't, I'll need more hints....
23:05 <@ferdy> bonsaikitten: can I get an attack vector?
23:05 <@ferdy> that shouldn't need lots of explaining... I can figure out that
part myself
23:19 <@ferdy> bonsaikitten: have you got that attack vector for me?
23:24 <+bonsaikitten> ferdy: look at configuration files, maybe you notice that
there's some exquisit code execution possible there
23:29 <@ferdy> bonsaikitten: you mean those config files that only root can
edit? I must be missing something here
23:29 <+bonsaikitten> ferdy: you are :)
23:29 <+bonsaikitten> not much, and it's basically the same flaw bashrc is
for portage
23:29 <+bonsaikitten> only that bashrc is config_protect'ed ...
23:30 <@ferdy> bonsaikitten: but for a package to clover those files, it must be
in a repo root added, right?
23:31 <+bonsaikitten> someone in the package mangler group, but yes
23:35 <@ferdy> bonsaikitten: but if you can change those files in the first place,
why clover them by adding a malicious repo with a malicious package that changes
those files?
23:35 <+bonsaikitten> ferdy: because it's very subtle
23:36 <@ferdy> moreover, if you can already do that, why not just make the
package install whatever backdoor you want?
23:37 <@ferdy> I mean, it is subtle, but why would anyone go the 'convoluted'
route? he is already able to edit those files (since he had to add that repo)
23:38 <+bonsaikitten> 'cause only paludis is affected and you will find it very
hard to trace
23:38 <+bonsaikitten> that makes it so tempting ...
23:40 <+bonsaikitten> just don't be surprised if it suddenly unmerges itself :)
23:41 <@ferdy> yeah... well...
23:41 <@ferdy> bonsaikitten: mind if I disclose this vulnerability in
 planet.gentoo.org?
23:42 <+bonsaikitten> go ahead
23:42 <@ferdy> ta
23:42 <+bonsaikitten> 't is even on the features page of the package mangler :)

This is a good lesson to learn today:

If you can edit files owned by root in a machine, you can get root access to that machine.

So the bottom line is: There is no vulnerability, if you can mangle paludis config files, you are already root so you don't need to edit a file to run any command you want. Another lesson one can learn by reading that log is how to be really cooperative.

Ah, and before someone with a need to use cheap psychology asks, the intention of this blag post is to stop the FUD.

- ferdy

Remi Cardona

Remi Cardona (remi) — Fri, 02 May 2008 15:31:05 +0000

Replying to Ryan's post...

https://bugzilla.mozilla.org/show_bug.cgi?id=384090#c41

And I'll just quote that comment here:

Firefox-3 (now that the patch is applied) will behave like Gtk, i.e. it will read the font dpi from any standards-compliant XSETTINGS manager. If Gtk programs have the right font dpi on your system, then so will firefox.

As for the gconf key, it won't help you unless you are also running gnome-settings-daemon (the default XSETTINGS manager for the Gnome desktop).

Let's not spread unnecessary false information, shall we?

Christian Faulhammer: Better late than sorry

Christian Faulhammer (opfer) — Fri, 02 May 2008 09:12:04 +0000

Or whatever the proverb was...anyway, here is my meme list:

Desktop user:

77 cd
75 l
59 grep
48 ebuild
33 less
21 exit
21 cvs
16 repoman
15 $EDITOR
12 /bin/su

Chroot (for architecture work):

118 gatt
53 emerge
46 exit
24 rm
24 FEATURES="test
21 USE="doc
19 l
17 emacs
15 eix
15 dispatch-conf

Yes, I use Gatt a lot for arch work and so should all testers and stabilisers. :)

Donnie Berkholz: Updated xorg-server and mesa live-git packages

Donnie Berkholz (dberkholz) — Fri, 02 May 2008 08:31:48 +0000

Today I reworked the mesa and xorg-server ebuilds to do things the cool way with the latest git sources. Mesa added an autoconf build, thanks to Dan Nicholson, which makes the ebuild a lot simpler and less error-prone by being more standardized. There’s no XCB support yet–if nobody else adds it, I’ll probably do it eventually. It’s just autoconf, no automake, so the installation side of the ebuild stayed pretty complex. There’s a few strange things going on with what headers mesa installs that I haven’t looked into besides just deleting them.

Also, these updates incorporate the first major step to removing the extra copy of the mesa source code used to build xorg-server. The GLcore module is now built within mesa instead of xorg-server, thanks to George Sapountzis. This created a circular dependency between xorg-server and mesa, so I made a new ebuild called mesa-glcore. Unfortunately this means that libmesa.a still gets built twice. The other half of getting the mesa source out of the xorg-server build is libglx, and I’m hoping someone does it soon (hint, hint!).

This work is all available in the Gentoo x11 overlay, which you can add with the simple command `layman -a x11`.

Luis Francisco Araujo: Himerge 0.20 release

Luis Francisco Araujo (araujo) — Thu, 01 May 2008 21:10:20 +0000

After almost a year since the first Himerge version was released (May, 03, 2007) , the application has reached its version 0.20 as of today , introducing the following important changes for this release (from its Changelog):

1 - The queue operation 'add' is now available through a right-click from the browser version view for each package.

This will allow to move packages easier to any package queue located inside the down-right control panel.

2 - The queue operations 'remove/clear' are now available from each of the
control panel package queue through a right-click.

You only right-click on any package queue , and a popup-menu will let you choose between removing the selected package inside the queue or clear the whole queue.

With these two changes, which are ones of the most noticeable changes in the general feel&look since Himerge was released almost a year ago; it is intended to increase usability and improves the user experience around the different packages queues ; so now you don't have to move the mouse click over the whole window but instead you have the operations at hand for each package in the specific visual context.

3 - A backup of all the system files used by himerge will be created inside
the ~/.himerge/ directory.

This feature will keep those important configuration files used by Himerge saved inside ~/.himerge/. The application will read and copy these files every time you run a Himerge browser instance, so we don't miss any change.

4 - The USE flag browser is highly-improved. A new browser construction
mechanism speeds up the launch of the browser and shows a more organized and
coherent framework, listing only the necessary flags boxes available for a
given entity (a package or the whole system use flags).

In other words , now the integrated Himerge USE Flag browser will *only* show the flags boxes alphabetically ordered for the existing USE flags of a specific package. There won't exist USE flags categories with empty content.

Some code has been refactored for supporting this feature and the USE flag browser speed has greatly improved.

Himerge is an application written entirely in Haskell that it only acts like a front-end and it uses several other applications written in different languages for some operations; nevertheless , some users seem to have problems or not willing to compile many keyworded/masked Haskell packages for it and I have been creating binaries for x86/amd64 for some Himerge versions lately ; you can get the ebuilds from: http://dev.gentoo.org/~araujo/stuff/ebuilds/.

Be aware that these binaries will be usually out-of-dated and they are currently in testing stage (some runtime dependencies could be missing here and there) , so if you receive an error , please try to compile and use the source version of the package instead before reporting.

If you want to know more about Himerge , please visit its official web-site at http://www.haskell.org/himerge. You can also stop by #gentoo-guis@irc.freenode.net where some of us who believe in graphical interfaces inside Gentoo use to idle ;-).

Himerge is a program developed by one-man , which means, one-man ideas too, but comments and bugs reports are welcome , and much more if they come with a patch ;-)

Regards,

Steve Dibb: mplayer + dvdnav svn ebuilds

Steve Dibb (beandog) — Thu, 01 May 2008 16:57:30 +0000

There has been some recent activity again on the libdvdnav front, and so I’ve been playing around with it again. I finally created an SVN ebuild for MPlayer which I can use myself instead of just manually updating and reconfiguring the repo myself.

Generally speaking, I don’t like the idea of ebuilds that hit upstream’s SVN servers directly, so please be kind and don’t do anything asinine like upgrade every single day or something.

So now I have live subversion ebuilds for both libdvdnav and mplayer. This is the important thing to know about the libdvdnav ebuild: it has some changes to libdvdread so you will have to unmerge the one that is already uninstalled. On top of that, since it is a new API version, everything you previously compiled against libdvdread (k3b, etc.) will have to be re-emerged … and even then, there are zero guarantees that it will work at all.

In short, these ebuilds are only designed for a select few: those people who are using mplayer exclusively and want to have dvdnav support at the risk of breaking anything else that needs DVD access. Obviously, that scenario fits for someone with a media frontend, but doesn’t make sense for general desktop usage.

Also worth noting is that these are the only two ebuilds that will work together. Previously, the mplayer in the tree would have detected and worked with the libdvdnav ebuild, but that’s not the case anymore. It’s these two in the overlay, or the mplayer in the tree. Pick one set and choose.

If you have problems with the ebuilds, let me know. I’m still not an expert at layman so I can’t go about giving you instructions on how to install this stuff directly. Have fun, though.

Oh, and last but not least — I tested them, and it works for me. :) Just emerge libdvdread first, then mplayer.

Christian Faulhammer: Security on Gentoo

Christian Faulhammer (opfer) — Thu, 01 May 2008 14:55:58 +0000

A task of the Gentoo security team is to provide Gentoo Linux Security Advisories (GLSA). Some months ago we just lagged behind there, but now I must admit that the current members do a good job in getting the bugs done on time. Backpadding is wanted by everyone, so here is yours, security monkeys.

Ryan Hill: Why mozilla needs a freedesktop.org liason

Ryan Hill (dirtyepic) — Wed, 30 Apr 2008 17:38:51 +0000

So I finally got fed up with fonts on half the sites i visit (including LJ) being gargantuan in Firefox 3. I managed to track it down to this bug [https://bugzilla.mozilla.org/show_bug.cgi?id=384090].

Yay, it's fixed!

Then I read this gem:

"Well, the correct way to set font DPI on the Linux desktop is with the
/desktop/gnome/font_rendering/dpi gconf key."

*sigh*

Gunnar Wrobel: Virtual servers get converted to Gentoo

Gunnar Wrobel (wrobel) — Wed, 30 Apr 2008 12:40:15 +0000

It looks like a number of people are not too happy about running SuSE on their virtual servers. I got a good deal of feedback on my HowTo that describes the conversion of a virtual server preinstalled with SuSE. I integrated all fixes and improvements that I received from various contributors. Thanks a lot to all of them!

While the instructions were initially targeted at the servers provided by 1 and 1 it definitely also works for the servers of Strato. Anyone with a success story on the servers of other companies?

Gunnar Wrobel: Gentoo on a 1&1 vServer

Gunnar Wrobel (wrobel) — Wed, 30 Apr 2008 12:32:19 +0000

Last update: 2008/04/30

Companies like 1and1 and Strato offer virtual servers based on the Virtuozzo virtualization technology. While these machines are quite cheap and provide a full linux work environment they run SUSE by default. Not my favorite linux distribution...

I was pretty certain that I could also switch the server to Gentoo. But when I asked the customer support they told me that they have no one running Gentoo on any of these machines. And that they would have no clue if that could work.

So I tried and it is definitely possible. Just in case there are others who would like to have a Gentoo vserver on a Virtuozzo system this HowTo will provide some instructions on how to achieve that.

Do I need to give the usual warnings? You'll completely wipe the old system and if something does not work, you will have to reinitialize the server. If you don't want to take that risk, do not continue.

Cleaning up

First you will have to log into your "Virtuozzo Power Panel" in order to switch the system into repair mode. The original system now resides in /repair and you work in a safety mode.

Now log into your system via ssh and make a backup copy the old /etc/mtab (this helps to have a working df command at a later time point, reported by Gian):

     
cp /repair/etc/mtab /root/mtab.old

Now remove the old suse system:

                                                                                                                                                              
cd /repair                                                                                                                                                                         
rm -rf *

In case this results in a failure your repair directory might be mounted as read-only (reported by Ulrich):

                                                                                                                                                              
mount -o remount,rw /repair

Install the basic Gentoo system

Now (still in /repair) start to download the stage and a portage snapshot from your nearest mirror:

                                                                                                                                                              
wget ftp://linux.rz.ruhr-uni-bochum.de/gentoo-mirror/experimental/x86/vserver/stage3-i686-20060317.tar.bz2
wget ftp://linux.rz.ruhr-uni-bochum.de/gentoo-mirror/snapshots/portage-latest.tar.bz2
tar xvjpf stage3-*.tar.bz2                                                                                                                                                         
tar xvjf portage-*.tar.bz2 -C /repair/usr                                                                                                                                          
rm stage3-*.tar.bz2 portage-*.tar.bz2

The basic tools are now in place. Next we need the original network information:

                                                                                                                                                              
cp /etc/resolv.conf /repair/etc/

In addition copy the original mtab back into place:

                                                                                                                                                              
cp /root/mtab.old /repair/etc/mtab

And now we can chroot into the new Gentoo environment:

                                                                                                                                                              
mount -t proc proc /repair/proc/
mount -o bind /dev /repair/dev
chroot /repair

Time to fix the timezone information and sync the portage tree:

                                                                                                                                                              
env-update                                                                                                                                                                         
source /etc/profile                                                                                                                                                                
export PS1="(chroot) $PS1"                                                                                                                                               
cp /usr/share/zoneinfo/Europe/Berlin /etc/localtime                                                                                                                                
emerge --sync

Set a root password:

                                                                                                                                                              
passwd

Please note that this password becomes your new master password for the server!

Optional: Configure a build host

The vServers are not the most powerful machines and they definitely benefit from pulling packages from a central build host. If you have such a machine you should complete your /etc/make.conf with the following variables:

PORTAGE_BINHOST="http://buildhost.example.com/packages/i686/All"
SYNC="rsync://buildhost.example.com/portage"
EMERGE_DEFAULT_OPTS=" --usepkg --getbinpkg --getbinpkgonly"

Move to baselayout2

The old baselayout-vserver probably still works but the newer baselayout2 also copes for vServers and I recommend to use it.

First we should ensure that we link to the current Gentoo profile:

                                                                                                                                                              
rm /etc/make.profile
ln -s ../usr/portage/profiles/default-linux/x86/2007.0 /etc/make.profile

Now we unmask the newer baselayout and the OpenRC package:

                                                                                                                                                              
echo "sys-apps/baselayout ~x86" >> /etc/portage/package.keywords
echo "sys-apps/openrc ~x86" >> /etc/portage/package.keywords

In case the kernel of the system underlying your virtual server is somewhat older, you should also ensure that you do not use the newer glibc-2.4 and that nptl is disabled:

                                                                                                                                                              
echo ">sys-libs/glibc-2.5-r4" >> /etc/portage/package.mask
echo "sys-libs/glibc -nptl -nptlonly" >> /etc/portage/package.use

Time to update the system:

emerge -uND world

Configure Gentoo as a virtual server

Now you can configure the network:

                                                                                                                                                              
emerge iproute2                                                                                                                                                                    
cd /etc/init.d                                                                                                                                                                     
rm net.eth0                                                                                                                                                                        
ln -s net.lo net.venet0                                                                                                                                                            
rc-update add net.venet0 default                                                                                                                                                   
rc-update add net.lo default

You will need to provide a static definition of your network parameters in /etc/conf.d/net. In order to determine the necessary parameters, follow the steps below:

                                                                                                                                                              
# ip addr                                                                                                                                                                          
326: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue                                                                                                                               
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00                                                                                                                          
    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo                                                                                                                             
327: venet0: <BROADCAST,POINTOPOINT,NOARP,UP> mtu 1500 qdisc noqueue                                                                                                         
    link/void                                                                                                                                                                      
    inet 127.0.0.1/32 scope host venet0                                                                                                                                            
    inet 87.123.45.123/32 scope global venet0:0

From the output note the ip of the venet0 adapter. Here it is 87.123.45.123.

Now you need the routing information:

                                                                                                                                                              
# ip route                                                                                                                                                                         
191.255.255.0/24 dev venet0  scope link                                                                                                                                            
127.0.0.0/8 dev lo  scope link                                                                                                                                                     
default via 191.255.255.1 dev venet0

The necessary parameters are the first netmask and the default gateway (191.255.255.0/24 and 191.255.255.1).

Adapt the following settings to your specific parameters and echo it into your network configuration file:

                                                                                                                                                              
echo '                                                                                                                                                                             
modules="iproute2"
modules="!ifconfig"

config_venet0="87.123.45.123 netmask 255.255.255.0 broadcast 0.0.0.0"

routes_venet0="191.255.255.0/24 scope link
               default via 191.255.255.1"

' >> /etc/conf.d/net

I am not an expert on the network settings and the proper routing on a vserver but these settings did work for me. Please send me a mail if you have suggestions on how to improve the configuration.

Another comment by Ulrich:

I did put spaces between the config_venet0, routes_venet0 and the equal sign. It's not allowed to do so. Adding this as a don't to your explanation might save an hour or two to some guys out there.

Finally you need to add the ssh server to the default services so that you will be able to log into the system:

                                                                                                                                                              
rc-update add sshd default

Reboot into Gentoo

Now you should be able to end the repair mode. Log into your Virtuozzo Power Panel, select "Finish repair" and try to log into your vserver via ssh a short while later.

ChangeLog

2008/04/30: Included moving to baselayout2

Gentoo News: 2008.0_beta2 released

Gentoo News () — Tue, 29 Apr 2008 22:02:09 +0000

2008.0_beta2 is here. Thanks to you, our community, for testing beta1 and filing many bugs. You can help make 2008.0 amazing! Test out this beta and report any functionality issues you encounter. Since this is still a beta, we're looking only for bugs in functionality, not bugs in appearance such as desktop backgrounds or other artwork.

This should be the last beta and will be followed by the final 2008.0 release after further bug fixing.

Get 2008.0_beta2 from the usual places.

Discuss this!

Gunnar Wrobel: Moving to baselayout2

Gunnar Wrobel (wrobel) — Tue, 29 Apr 2008 19:36:42 +0000

I finally took the time to move my configuration to baselayout2 and openrc. It was about time since I was still using the old baselayout-vserver packages on my vservers. I admit I was afraid the move would hurt so I waited for a while.

But it was really, really smooth.

I made only one mistake and did not notice that my link to net.lo vanished in the upgrade process. So I was subsequently wondering why ping responded with connect: invalid argument when pinging my own machine. Easy enough to fix.

Excellent work from the baselayout and OpenRC devs. Nice.

Ryan Hill: note to self...

Ryan Hill (dirtyepic) — Tue, 29 Apr 2008 18:04:00 +0000

...posting to mailing lists while on significant doses of painkillers is not always the best idea.

Diego Pettenò: I consider AC_CHECK_LIB harmful

Diego Pettenò (flameeyes) — Tue, 29 Apr 2008 16:55:00 +0000

You know I always stress the point that it’s not autoconf (and automake) to be bad per-se, but rather the fact that it’s poorly documented and most people get the worst examples on how to do stuff.

One thing that I find totally misdesigned in autoconf though is the AC_CHECK_LIB macro. This macro is supposedly designed to look for certain functions in a library, to make sure that it’s linked upon if needed. The most simple case for this is using

AC_CHECK_LIB([dl], [dlopen])

to gather if the dlopen() function is in libdl or would be available directly from the C library. (Actually you usually use something more complex than this, but that’s another story).

What is the problem of this macro? Well, the problem comes when you check non-generic functions, because it has a side effect: if the function is found on the library, it is added to the LIBS variable, that is appended to every link line, causing the library to be linked in every target. This is fine when you only have one target, but it’s nasty in every other case.

This is one of the main cause of extra libraries being linked in on other libraries and executables, which are worked around with --as-needed (at least ELF side, as --as-needed does not help with libtool archives at all). It should really be avoided.

The easy way to avoid this is to reset the LIBS variable after AC_CHECK_LIB to the value it had before, it also helps if you set a new variable with the library you just detected, to use just where needed. A better way to handle this is to push on using pkg-config, which might not be the solution to all problems, but makes this problem quite easier to handle.

I know most people doing this mistake won’t be reading this, but at least I have it documented somewhere so that I can just point people at this post without repeating the reasoning. And yes, it’s problem like these that makes neon link against pam for no good reason.

Zhang Le: Old story

Zhang Le (r0bertz) — Tue, 29 Apr 2008 14:30:15 +0000

Although I don't know the direct cause of council's decision this time, it reminds me of an old story.
http://r0bertz.blogspot.com/2007/01/be-careful-when-you-are-on-paludis.html

Quote some Diego's words:

Don’t think we woke up this morning and decided to get rid of any developer. We considered this for a quite long time. Over all, I think the decision was took after a too long time.

You can criticise, you can joke, but if the people you joke upon don’t laugh with you at the joke, then apologise and stop it!

Remi Cardona: Shiny new Intel gfx drivers (2.3.0)

Remi Cardona (remi) — Tue, 29 Apr 2008 08:30:42 +0000

As it has been announced on several mailing lists, Intel released version 2.3.0 of their X driver. Being back from vacation and all, I've only just committed it to portage. Here are a few things Gentoo users might want to know about it.

This version is definitely not as disruptive as 2.2.0 (which will probably be remembered as one of the worst driver releases in all of Xorg history). Overall, it really is an incremental release on top of 2.2 with all the bug fixes.
Laptop users of i915 chipsets and newer should enjoy new xrandr options to control how the image is displayed on the LCD when not using its native resolution: the old behavior was to stretch the image in both X and Y directions, now the driver supports stretching with aspect ration constraints and not stretching the image at all to keep a 1:1 ratio.
i965 does have a video overlay and I think it's enabled by default now.
XvMC support should be much better now on i945 and newer. As I can't test XvMC (and I don't really need it anyway), some testing might be interesting. Please let me know how it works on your systems, I'd like to hear from you
Not really technical but very important IMHO: the new release process is now much much smoother. Intel decided to release a new driver every quarter and that's a great improvement. "Release early, release often". Intel now also releases beta versions of the driver. It might not seem like much, but it does make my job a lot easier and it really helps to get real users to test drivers on their systems. And it shows: only hours after each beta, bugs get reported and fixed. If only more FOSS projects could work like that...

As a whole, it's a good release that shows that even big companies such as Intel can successfully build communities that are committed to quality and openness. Big props to all who worked on it.

On the Gentoo side of things, here are some additional notes:

As it looks today, 2.3.0 is a prime candidate for stabilization. As such, I encourage all stable users to unmask it and try it on their systems. It should work properly with xorg-server-1.3. If not, please don't hesitate to open a bug.
I'd also like to remind everyone that opening a bug in Gentoo's bugzilla is a great step to get your problems solved. But in the case of the Intel driver, there's very little I can actually do to fix bugs: I don't work for Intel, I don't know their code nor how the hardware works, I just know about bugs and patches. So if I ask you to open a bug in FreeDesktop's bugzilla, it's not because I don't want to fix the bug, it's just that I can't do it myself. Users who actually take the time to do so are usually rewarded by having their issue fixed in a matter of days. So please, take the time to report bugs upstream, it's in your best interest.

Now, off to work.

Ryan Hill: mail

Ryan Hill (dirtyepic) — Tue, 29 Apr 2008 07:06:59 +0000

after about three weeks without email, i gave up on setting up a dedicated router/firewall/web/mail/media server set up and just installed getmail and dovecot on one of my machines for now. at least then i can still get email access, and i can work on getting everything else set up as i learn more.

i've installed claws-mail again and am giving it another spin. i can't remember exactly what was the showstopper last time i used it, but that was in the days where i was frequently offline for a week or two at a time and so it was probably some annoyance i had with with the offline mode. i've been happy with it so far, except for the fact that to display html messages i get to choose from two plugins - dillo, with a pantload of gtk-1 dependences, or gtkhtml, which pulls in half of gnome. *sigh*

i tried kmail again last week and it went all funky on my maildir - showing me a subjectline that when i click gives me a completely different message body. thunderbird, which is what i've used the last couple years finally accumulated enough pain in the ass annoyances (not remembering folder setting when you leave the folder, not removing newsgroup data (which is significantly large on some mailing lists i'm on) when unsubscribing, leaving me at one point with nearly a gig of useless data my profile. but the straw that finally broke TB's back was when random email and news messages would display nothing but a blank page when i opened them. To read the message i would have to save it to disk and then open it in an external editor. Doing this for every third message got old fast.

Yes, I should be using mutt. I like mutt, it seems great. It has a learning curve however, and it seems every time i start up it i end up sliding back down. Maybe a bit later.

Planet Gentoo

Diego Pettenò: Today's mass filings

Donnie Berkholz: Distributions in the Summer of Code

Alexander Gabert: all your __guard are belong to __stack_smash_handler

Zack Medico: Handling File Collisions Between Blocking Packages

Alexander Gabert: the long tail

Tobias Klausmann: Gzip, Bzip2 and Lzma compared

Why?

What and How?

Results

Conclusion

What else to test

Comments

Bonus fact learned about recent tar versions

Joshua Nichols: Using a Gentoo Prefixed shell as your login shell

Tobias Klausmann: ack - Part II

Every day is a winding command line

Measurements, Unicode and all that

Zack Medico: Using Blockers to Automatically Uninstall Unwanted Packages

Tobias Klausmann: ack! (thpppt!)

It's blazingly fast because it only searches the stuff you want searched.

ack is pure Perl, so it runs on Windows just fine.

The standalone version uses no non-standard modules, so you can put it in your ~/bin without fear.

Searches recursively through directories by default, while ignoring .svn, CVS and other VCS directories.

ack ignores most of the crap you don't want to search

Ignoring .svn directories means that ack is faster than grep for searching through trees.

Lets you specify file types to search, as in --perl or --nohtml.

File-filtering capabilities usable without searching with ack -f. This lets you create lists of files of a given type.

Color highlighting of search results.

Uses real Perl regular expressions, not a GNU subset

Allows you to specify output using Perl's special variables

Many command-line switches are the same as in GNU grep:

Command name is 25% fewer characters to type! Save days of free-time! Heck, it's 50% shorter compared to grep -r

Update

Comments

Daniel Drake: Fingerprint scanning project report published

Joshua Nichols: Managing RubyGems on Gentoo

Mix and match

Gentoo packages exclusively

Gems exclusively

My current practices

In development

In production

Fernando J. Pereda: On cooperating and paludis vulnerability

Remi Cardona

Christian Faulhammer: Better late than sorry

Donnie Berkholz: Updated xorg-server and mesa live-git packages

Luis Francisco Araujo: Himerge 0.20 release

Steve Dibb: mplayer + dvdnav svn ebuilds

Christian Faulhammer: Security on Gentoo

Ryan Hill: Why mozilla needs a freedesktop.org liason

Gunnar Wrobel: Virtual servers get converted to Gentoo

Gunnar Wrobel: Gentoo on a 1&1 vServer

Last update: 2008/04/30

Cleaning up

Install the basic Gentoo system

Optional: Configure a build host

Move to baselayout2

Configure Gentoo as a virtual server

Reboot into Gentoo

ChangeLog

Gentoo News: 2008.0_beta2 released

Gunnar Wrobel: Moving to baselayout2

Ryan Hill: note to self...

Diego Pettenò: I consider AC_CHECK_LIB harmful

Zhang Le: Old story

Remi Cardona: Shiny new Intel gfx drivers (2.3.0)

Ryan Hill: mail

Alexander Gabert: all your guard are belong to stack_smash_handler