Marius Mauch

Something that's bugged me for while in portage was the crappy namespace handling we had since whenever we moved the python modules to /usr/lib/portage/pym. Originally there was no real problem as we only had a single module portage.py, so all you needed was a 'import portage', but over time more modules were created, which Nick started to name portage_foo.py due to the lack of a "portage" python package to use as container. Also there were a number of modules without any "portage" part in the name, such as xpak, cvstree, output or the cache package, which could potentially cause a namespace collision with other packages in site-packages or even the standard library, not a very pleasant thought.
But as of today that's history, I finally fixed this annoyance and moved all the portage related code into the new "portage" package (so portage.py is now portage/__init__.py and portage_foo.py is now portage/foo.py). For now the code is mostly a 1:1 translation, but over time it hopefully gets a bit cleaner by removing redundant qualifiers. Also this now allows us to split the big portage.py (or now __init__.py) up further without fearing namespace collisions, I'll probably move the dbapi classes into their own package later this week.
But what does this all mean to you? If you're just a normal user it shouldn't affect you in any way (assuming I didn't screw up anything and Zac updates the ebuild accordingly). If you have some custom scripts or are a developer of a tool using the portage API you should prepare for updating it after portage-2.1.3 is released, though for the time being the old names should just continue to work as I've also added some symlinks to avoid a large-scale API breakage.

On another note I fully agree with Diego on the idea of splitting the tree up. I've never been a big fan of the recent overlay hype, but at this point it's still manageable. Also besides any technical problems a tree split would increase the "repo hunting" problem which we're already starting to see and is IMHO one of the major downsides of most other (rpm-based) distributions, and that's something I'd like to avoid in Gentoo.

After raising the awareness about KEYWORDS="-*" being a stupid thing to use in the last months today I decided to eliminate the remaining reason for using it (one couldn't unmask a package that had KEYWORDS="" without editing it) by adding support for a new token in package.keywords. So now when portage-2.1.3 goes live all theses live-cvs-completely-unsupported packages can stop using the broken KEYWORDS="-*" and use KEYWORDS="" instead without loosing functionality. And once we get the tree clean from those KEYWORDS="-*" abusers we can also finally fix the -* handling for package.keywords to do what it should do (act like ACCEPT_KEYWORDS).

I assume some of you have been wondering what has happened to my gentoo-stats project as there haven't been any news or updates recently. Well, unfortunately there isn't much going on, I guess I've been just a bit too frustrated with it to work on it in the last weeks/months. That frustration mainly comes from the package-filemap module and its crappy performance and the conceptual failure of the auth encryption I had planned/implemented. The latter is just frustrating simply due to the wasted time, but the former means the lack of a key feature, namely finding which packages provide a given file even for uninstalled packages. Already tried several things to get it faster but without real success so far
Now I have two more ideas how to get it still working: First is to simply reduce the amount of data to the bare minimum (e.g. just recording executables and libraries), the second is using a custom storage backend for filenames instead of using MySQL for everything (as the DBMS is the slow part). I really want to avoid the first (as it would reduce functionality and likely just delay the problem a bit) and only use it as a last resort before dropping the module completely, so a while ago I wrote a custom backend based for storing filenames efficiently, but haven't integrated it yet into the processing module. We'll see if I can find some time in the coming days/weeks to get this project back on track.

If you're interested in helping with it:
- I don't have any design for the web interface yet, so far it's just basic HTML-2.0 or so. I'm not a big designer, so this is something where I'd definitely welcome external help
- A GUI for the client would be nice (like for selecting data modules or performing complex queries), but I'm not a big fan of GUI programming (though I could help with any missing backend parts in the client)
- Wouldn't hurt to have someone else who's an expert with (My)SQL/mod_python/security have a look at the current code/db schema before this service goes into public testing.

So today I wrote a little script to store the ebuilds and associated files of installed packages into a separate overlay as a first measure to solve bug #126059. It's still far from perfect as it only adds to the overlay, isn't limited to ebuilds deleted by an `emerge --sync` (which can be seen as a benefit as well) and most importantly doesn't deal with Manifests yet.
The last point makes it rather useless for general usage, but it's just a prototype script to see if this is a viable solution to the problem of disappearing ebuilds.
So any feedback about this would be appreciated.

After venting about annoying users last week I'll try to post something a bit more useful today, maybe this will even evolve into a series about what's new in portage land.

Well, today I added a little extension to the elog subsystem to make multi-target logging a bit more useful by extending the PORTAGE_ELOG_SYSTEM syntax a bit. Now you can override PORTAGE_ELOG_CLASSES per module, so for example one might send all messages into a file (using the save-summary module added in 2.1.2) and additionally send the important ones also by mail. Another related extension is that you can now use a * wildcard whereever a loglevel is wanted to include all loglevels.
So using the example above you would put the following in your make.conf:
PORTAGE_ELOG_SYSTEM="save-summary:* mail:log,warn,error"

sometimes I just hate you. Why? Because every now and then someone you haven't heard of before comes around and asks a question, then doesn't like the answer and calls you and idiot in one way or the other. Like today when somone joined our portage IRC channel, asked how to test a package on a arch it wasn't keyworded for and then refused to understand how the keyword system works. He even called using package.keywords for anything other than ~arch a misuse of that feature (which is funny as I wrote that feature initially).
And after trying to explain it three times he still insisted that we should implement another way because the current one (with ACCEPT_KEYWORDS and package.keywords) wasn't "logical".
Instead he insisted that we add a feature to allow users to change package metadata (like KEYWORDS) with a simple config file, which isn't just redundant but as any dev will assure you quite stupid and possibly harmful.
In the end he left after some nasty remarks, without having accomplished anything other than to upset a couple of devs due to simple ignorance. Now I assume he probably feels the same about us, but sometimes you just have to accept that the other party is right, and in this case we have >99% of the userbase and several years of experience backing our way up.

Now I know that such incidents are the rare exception and in most cases dealing with users is a nice experience, but they can make you pretty upset for a while sometimes.
So what's the moral of the story? If you're in (heated) discussion, always consider that you may be wrong, try to look at the other parties arguments from a different angle. If you notice that you're hitting a wall go away for a while so everyone can cool down (and try the first advice again), and maybe try to resume the discussion at a later date. And if you're talking with someone who's an expert in a domain while you're not, don't try to beat him in that domain (e.g. don't say that you change package metadata with package.keywords when you aren't sure what package metadata is in the first place). That doesn't mean that you can't talk about it, but realize that the other person likely knows a lot more about the topic (and maybe doesn't want to explain every little detail to you when rejecting an idea).

In the end it's just for your own benefit (and nobody likes grumpy devs )