Gentoo Logo
Gentoo Logo Side
Gentoo Spaceship

Contributors:
. Aaron W. Swenson
. Agostino Sarubbo
. Alec Warner
. Alex Alexander
. Alex Legler
. Alexey Shvetsov
. Alexis Ballier
. Alexys Jacob
. Amadeusz Żołnowski
. Andreas K. Hüttel
. Andreas Proschofsky
. Andrew Gaffney
. Anthony Basile
. Arun Raghavan
. Bernard Cafarelli
. Bjarke Istrup Pedersen
. Brent Baude
. Brian Harring
. Christian Faulhammer
. Christian Ruppert
. Christopher Harvey
. Chí-Thanh Christopher Nguyễn
. Dane Smith
. Daniel Gryniewicz
. David Abbott
. Denis Dupeyron
. Detlev Casanova
. Diego E. Pettenò
. Domen Kožar
. Donnie Berkholz
. Doug Goldstein
. Eray Aslan
. Fabio Erculiani
. Gentoo Haskell Herd
. Gentoo Monthly Newsletter
. Gentoo News
. Gilles Dartiguelongue
. Greg KH
. Hanno Böck
. Hans de Graaff
. Ian Whyman
. Ioannis Aslanidis
. Jan Kundrát
. Jason Donenfeld
. Jeffrey Gardner
. Jeremy Olexa
. Joachim Bartosik
. Johannes Huber
. Jonathan Callen
. Jorge Manuel B. S. Vicetto
. Joseph Jezak
. Kenneth Prugh
. Lance Albertson
. Liam McLoughlin
. LinuxCrazy Podcasts
. Luca Barbato
. Luis Francisco Araujo
. Mark Loeser
. Markos Chandras
. Mart Raudsepp
. Matt Turner
. Matthew Marlowe
. Matthew Thode
. Matti Bickel
. Michael Palimaka
. Michal Hrusecky
. Michał Górny
. Mike Doty
. Mike Gilbert
. Mike Pagano
. Mu Qiao
. Nathan Zachary
. Ned Ludd
. Nirbheek Chauhan
. Olivier Crête
. Pacho Ramos
. Patrick Kursawe
. Patrick Lauer
. Patrick McLean
. Paul de Vrieze
. Pavlos Ratis
. Paweł Hajdan, Jr.
. Petteri Räty
. Piotr Jaroszyński
. Rafael Goncalves Martins
. Raúl Porcel
. Remi Cardona
. Richard Freeman
. Robin Johnson
. Ryan Hill
. Sean Amoss
. Sebastian Pipping
. Serkan Kaba
. Steev Klimaszewski
. Stratos Psomadakis
. Sune Kloppenborg Jeppesen
. Sven Vermeulen
. Sven Wegener
. Theo Chatzimichos
. Thomas Kahle
. Tim Sammut
. Tiziano Müller
. Tobias Heinlein
. Tobias Klausmann
. Tom Wijsman
. Tomáš Chvátal
. Torsten Veller
. Victor Ostorga
. Vikraman Choudhury
. Zack Medico
. Zhang Le

Last updated:
March 06, 2014, 23:06 UTC

Disclaimer:
Views expressed in the content published here do not necessarily represent the views of Gentoo Linux or the Gentoo Foundation.


Bugs? Comments? Suggestions? Contact us!

Powered by:
Planet Venus

Welcome to Planet Gentoo, an aggregation of Gentoo-related weblog articles written by Gentoo developers. For a broader range of topics, you might be interested in Gentoo Universe.

March 05, 2014
Michał Górny a.k.a. mgorny (homepage, bugs)

In a previous article titled ‘using deltas to speed up SquashFS ebuild repository updates’, the author has considered benefits of using binary deltas to update SquashFS images. The proposed method has proven very efficient in terms of disk I/O, memory and CPU time use. However, the relatively large size of deltas made network bandwidth a bottleneck.

The rough estimations done at the time proved that this is not a major issue for a common client with a moderate-bandwidth link such as ADSL. Nevertheless, the size is an inconvenience both to clients and to mirror providers. Assuming that there is an upper bound on disk space consumed by snapshots, the extra size reduces the number of snapshots stored on mirrors, and therefore shortens the supported update period.

The most likely cause for the excessive delta size is the complexity of correlation between input and compressed output. Changes in input files are likely to cause much larger changes in the SquashFS output that the tested delta algorithms fail to express efficiently.

For example, in the LZ family of compression algorithms, a change in input stream may affect the contents of the dictionary and therefore the output stream following it. In block-based compressors such as bzip2, a change in input may shift all the following data moving it across block boundaries. As a result, the contents of all the blocks following it change, and therefore the compressed output for each of them.

Since SquashFS splits the input into multiple blocks that are compressed separately, the scope of this issue is much smaller than in plain tarballs. Nevertheless, small changes occurring in multiple blocks are able to grow delta two to four times as large as it would be if the data was not compressed. In this paper, the author explores the possibility of introducing a transparent decompression in the delta generation process to reduce the delta size.

Read on… [PDF]

Jan Kundrát a.k.a. jkt (homepage, bugs)
Trojita 0.4 "Ukraine" is released (March 05, 2014, 14:02 UTC)

Hi all,
we are pleased to announce version 0.4 of Trojitá, a fast Qt IMAP e-mail client. For this release, a lot of changes were made under the hood, but of course there are some changes that are visible to the user as well.

Improvements:

  • Users are able to use multiple sessions, which means that it is possible to use Trojitá with multiple IMAP accounts at the same time. It can be used by invoking Trojitá with the --profile something switch. For each profile, a new instance of the application is started. Please note that this is not our final solution for the multi-accounts problem; work on this is ongoing. For details, refer to the detailed instructions.
  • In the Composer Window, users can now control whether the current message is a reply to some other message. Hopefully, this will make it easier to reply to a ton of people while starting a new thread, not lumping the unrelated conversations together.
  • Trojitá will now detect changes to the network connection state. So for example, when a user switches from a wireless connection to a wired one, Trojitá will detect that and try to reconnect automatically.
  • Trojitá gained a setting to automatically use the system proxy settings.
  • SOCKS5 and HTTP proxies are supported.
  • Memory usage has been reduced and speed has been improved. Our benchmarks indicate being ten times faster when syncing huge mailboxes, and using 38% less memory at the same time.
  • The Compose Window supports editing the "From" field with hand-picked addresses as per common user requests.

This release has been tagged in git as "v0.4". You can also download a tarball (GPG signature). Prebuilt binaries for multiple distributions are available via the OBS .

This release is dedicated to the people of all nations living in Ukraine. We are no fans of political messages in software announcements, but we also cannot remain silent when unmarked Russian troops are marching over a free country. The Trojitá project was founded in a republic formerly known as Czechoslovakia. We were "protected" by foreign aggressors twice in the 20th century — first in 1938 by the Nazi Germany, and second time in 1968 by the occupation forces of the USSR. Back in 1938, Adolf Hitler used the same rhetorics we hear today: that a national minority was oppressed. In 1968, eight people who protested against the occupation in Moscow were detained within a couple of minutes, convicted and sent to jail. In 2014, Moscowians are protesting on a bigger scale, yet we all see the cops arresting them on Youtube — including those displaying blank signs.

This is not about politics, this is about morality. What is happening today in Ukraine is a barbaric act, an occupation of an innocent country which has done nothing but stopped being attracted to their more prominent eastern neighbor. No matter what one thinks about the international politics and the Crimean independence, this is an act which must be condemned and fiercely fought against. There isn't much what we could do, so we hope that at least this symbolic act will let the Ukrainians know that the world's thoughts are with them in this dire moment. За вашу и нашу свободу, indeed!

Finally, we would like to thank Jai Luthra, Danny Rim, Benjamin Kaiser and Yazeed Zoabi, our Google Code-In students, and Stephan Platz, Karan Luthra, Tomasz Kalkosiński and Luigi Toscano, people who recently joined Trojitá, for their code contributions.

The Trojitá developers

  • Jan Kundrát
  • Yuri Chornoivan
  • Karan Luthra
  • Pali Rohár
  • Tomasz Kalkosiński
  • Christian Degenkolb
  • Jai Luthra
  • Stephan Platz
  • Thomas Lübking

March 03, 2014
Gentoo Monthly Newsletter - February 2014 (March 03, 2014, 19:02 UTC)

The February 2014 GMN issue is now available online.

This month on GMN:

  • Interview with Gentoo developer Sven Vermeulen (swift)
  • Latest Gentoo news, job openings, interesting stats and much more.

March 01, 2014
Gentoo Monthly Newsletter: February 2014 (March 01, 2014, 19:00 UTC)

Gentoo news

Interview with Gentoo developer Sven Vermeulen (swift)

(by David Abbott)

1. Hi Sven, tell us about yourself?
My name is Sven Vermeulen. Although Sven is a primarily Scandinavian name, I have no roots with Scandinavia. I’m born (and still living) in Belgium, growing up with geeky domains such as technology, math, science and computing. In 2005 I graduated as engineer and started working for KBC, one of Belgium’s leading financial institutions (bank & insurance). In it, I always kept technology close to me, first as system engineer and now as IT architect.

My interest in technology & science never faded. Although computer systems and software development are my primary hobbies (as they can be handled hands-on easily without heavy investments) I still like to learn about the progress made in other fields and give myself exercises to keep my knowledge on those fields up to date. And for some reason, that always tends to help me with my real-life work (for instance for contract optimizations I used mathematical optimization methods).

I live with my daughter close to my work (between Brussels and Antwerp) which allows me to go to work by bicycle if my presence isn’t needed elsewhere. My work does require me to go abroad from time to time, but mostly within the European Union.

In my free time I enjoy … wait, free time? Nope, don’t have that nowadays. Let me rephrase it: if I had more free time, I’d probably spend it jogging or swimming (which I currently only do to clear my mind), sitting behind my computer (programming, documenting or just playing around), watching cats do stupid things on my tv (youtube – I don’t have cable or other TV services) and playing board games with friends.

Alas, most time is spent either on work, on working in my home (renovations) or providing taxi services to my daughter.

2. How did you get involved with Linux and Open Source?
That started in ‘96 or ‘97. I got a RedHat installation to play with and thought I could become a kernel developer with it. Well, I did have lots of imagination back then ;-) But I did enjoy the difference from the previous operating systems I used (Atari and Microsoft DOS/Windows) and was quite hooked by the idea of free software (I think then it was still mostly coined as “open source”).

I never deployed anything commercial / proprietary on my own systems anymore since. The BBS’es (and later Internet) provided all the information I needed to continue with free software. And as a C programmer (not saying I’m good at it, just saying I program in it) I took on the challenge of supporting my (then unsupported) Matrox graphics card with dual output in Linux. I got good help by the Linux development community, and got in touch with Linux’ internal structures. Which I immediately embraced as a new source of knowledge, as I moved to software engineering in my studies.

All software related things I did were in the free software world, patching here and there. After a while, I stumbled upon the next challenge, which was convincing other users to use free software. A major gap in this area was documentation, so I started learning about writing good documentation (I’m still disappointed that the Darwin Information Typing Architecture (DITA) hasn’t broken through), which is about the point that I joined Gentoo Linux.

In Gentoo, I first helped with translations, then moving on to English documentation, authoring, etc. Internally, I’ve been through various roles (regular developer, project manager, top-level project lead, trustee, council) in various areas (most of them non-technical, such as documentation, PR, recruitment). After quitting and joining a few times (I seem to have ups and downs in available time) I’m now running to keep the Gentoo documentation maintained, as well as supporting SELinux through the Gentoo Hardened project.

I often bounce from one technology or software to another, depending on the needs of the day. Need to detect installed libraries (in order to track potential vulnerabilities) but can’t find a tool? I’ll write one. Want to confirm secure configurations? I’ll learn about SCAP technologies and implement that. Require a web-based question & answer application? Let’s look how HTML5 works shall we. I’m pretty fluent in learning about technologies, protocols and what not.

Almost wished I was equally fluent in languages and history, which was my major obstacle at school…

3. I read your book Linux Sea and not only was impressed, really enjoyed it. Doing a Gentoo Linux install using the book as a classroom textbook would be the kind of class I would love to take. How did the book come about, and why Gentoo?
I wanted to create a documentation resource on Linux, discussing how Linux operating systems work (the concepts and architecture, but without diving into the details and advanced usage) and to which I can refer people who have a need for understanding a particular aspect of the operating system.

As a target distribution, I choose Gentoo because there aren’t many resource on Gentoo, and because Gentoo sticks close to the implementations of the projects themselves. There are no interfaces or APIs surrounding any of the functionalities that a Linux operating system provides, so I can easily discuss the real implementations. Not completely a “Linux from scratch”, but sufficiently close.

Another advantage of using Gentoo as example distribution is that readers, who use different distributions, can still enjoy the book (as it explains how things work) and then refer to the distribution-specific information of their distribution to go further, now with the knowledge of how things work “under the hood”.

4. With your skillset you would be welcome in any project, why do you support Gentoo?
I switch between many interest fields, and Gentoo is one of the few distributions that caters for it. If you need a responsive desktop, Gentoo can offer that. You want good support for many graphical environments? Gentoo can offer that. Need to implement a secure server: yes, Gentoo can offer that. Want to run Gentoo on a very small, lightweight device? Gentoo can offer that. Want to create a Linux router? Of course Gentoo can offer that.

If I want to do something similar with another distribution, I would most likely need to use a different set of distributions depending on my needs.

A second reason is the flexibility offered by Gentoo. Many tools offered by Gentoo are meant to assist in the maintenance and use of one or more tools or services, but without limiting the configuration abilities of the underlying components. Take portage for instance: you can hook into the various phases of package deployment easily, and many ebuilds support epatch_user, allowing for customizing deployments without removing functionality offered by Gentoo.

Or OpenRC’s dependency-based service scripts. Instead of naming it with a number depending on when you want to launch it, just put in the necessary dependencies in the scripts and you’re all set. That’s not just easy. That’s what makes Gentoo unique and powerful.

5. What could we be doing better?
I think we should be focusing more on (functional) areas than package sets (herds), and looking for ways to innovate in those areas. Right now, we’re happily following along with (most) upstream projects, and doing our job as a distribution that upstreams patches and supports users.

But why not look for more innovative ideas? Be open and bold with ideas, discuss them publicly (now that we have the Gentoo wiki, this should be easy to implement), create concept code and documentation. Do things other distributions can’t.

We should dare to fail, in order to learn. Right now, it seems that we’re sometimes afraid of making the wrong choice. We’re an organization with several hundred developers and volunteers, but not bound by service agreements, contractual obligations or implied functional adherence based on financial contributions. We should leverage that and move towards more innovative fields.

A second item that I believe would improve Gentoo as a distribution would be to remove complexity. Often, we do things in a somewhat complex way because there is no other way. That’s fine. But after a while, new and simpler methods come by that should replace the functionality we implemented more simplified.

Think about how the Gentoo Handbook is currently developed. We used our own format / syntax for reasons that were, back then, correct reasons. But things move on and mature. And while there are now much better alternatives available, we can’t use it because we customized everything to our needs. Writing documentation in the Gentoo Handbook almost requires you to learn how to program, as we use keywords, conditionals, include directives, automatic link generation, string substitutions and more. This is complex, and we should focus on simplifying this.

*I* should focus on simplifying this.

I’m pretty sure other examples can be found. Are all our eclasses still fully needed? How come the ruby-ng eclass is quite different from python-r1 eclass, even though they generally want to offer the same functionality? TIMTOWTDI, but if there is a method better and more simple than the other, use it.

6. Describe to our readers the relationship between the council and the foundation?
Basically speaking, the council is for technical matters and organization with regards to the Gentoo project, whereas the foundation is for the legal and financial aspects to support the Gentoo project. The two work orthogonal to each other (I am not aware of any overlap).

7. Is this relationship working, does it need to be changed or improved?
I think this is working pretty well and see little room for changes.

8. Same question for improving our partnership with Förderverein Gentoo e.V.
The Förderverein Gentoo e.V. and Gentoo Foundation, Inc. are sort-of siblings. After the decommissioning of Gentoo Technologies, Inc. each organization took on the responsibility of protecting the Gentoo trademark and supporting the Gentoo project in their home base: Förderverein Gentoo e.V. in Germany/Europe, and Gentoo Foundation in the United States of America.

9. What about moving the Gentoo Foundation to Belgium or somewhere in Europe?
I don’t think (re)locating a company to a specific location helps if there isn’t a need to. We should focus on what matters: protection and support of the Gentoo project and its intellectual property, and then evolve towards a structure that can easily support this now and in the future.

10. What documentation is moving to the wiki?
Well, right now we want to have all GuideXML documentation (which is non-handbook formats) on the Gentoo wiki. Most of the GDP-maintained documents (those in /doc/en) have been moved already, and moved into the main name space of the wiki so that others can contribute to it. That is also one of the main motivations for the move, as the Gentoo Documentation Project, for now, has insufficient resources to maintain GDP-only documentation.

In the next phase, handbook format documents (such as the SELinux Handbook, Gentoo Security Handbook and eventually the Gentoo Handbook itself) can be moved to the wiki as well. For the Gentoo Handbook though, this is more than just a copy of the data – it will require a refactoring of the documentation into a way that we can structure. I know the wiki supports inclusions and even conditionals, but this is some complexity I want to remove from the handbook.

A second thing a3li and I will look into (when time comes) is the ability to actually generate booklets from the wiki (like wikibooks.org does). I think this is a logical consequence, as those plugins (as used by wikibooks) are made with larger documents in mind, and allow us to align the documentation development with those best practices as gently suggested by the plugins.

But to do so, I believe that the architecture-specifics will need to be cleaned out. Either an entire chapter can be written independently of an architecture, or it can’t. Having a chapter that is “mostly” for one architecture, but with parameters and variables for each architecture just to make sure it reads fine for that architecture, is probably not doable or maintainable.

I have considered moving the larger documents in DocBook format (which is the format I use for my other, non-Gentoo documents), and that is still not abandoned. I guess I’ll need to sleep over it some more.

But first make sure that our wiki is qualitatively up to the standards we once had for our documentation.

11. With the documentation moving to the wiki have you noticed more contributions from the community?
The main advantage is that there are new documents being created of good quality, which upon discovery I also mark for translations (so that our translation teams can provide the same documentation to non-English readers) and perhaps even add metadata to it (so that it is taken up in the “featured documentation” overview). The Gentoo wiki is constantly growing, and is more and more becoming a standard source of information when trying to debug or troubleshoot issues reported on our support channels or forums.

Existing documentation, which is moved to the wiki, doesn’t get as much updates as I expected. But there are many reasons why, such as documentation being quite explicit, or people being afraid of editing documents written in a particular style they are not familiar with, or people just suggesting things in the discussion pages but not in the main page, …

12. What should we be doing to get more users involved?
One thing is to make it clear to users that the wiki is open for everybody, and that we welcome all additions. Even when the change is not within the expectations of the English language (style and grammar) as we have enough people watching over to fix these styles and who do this gladly, without any remark towards the original author. Not everyone is fluent in English, and we shouldn’t restrict contributions to language puritans as the broader community has a lot more knowledge ready to be shared.

A second thing is to try and get the discussions through the discussion pages more active. Right now, many discussions are still slow-paced. We should promote this more, but also make sure that we can follow up on these discussions easily. There are two ways to do this in a wiki. One is to watch the page (and the discussions), the second one is to mark the discussions as being “open”, so they can be aggregated and viewed through the proper category in the wiki.

13. Who would you like to see recruited to become Gentoo Developers?
I’d like to see more package maintainers. There is still plenty of software without ebuilds, and that is after all what our users expect us to do the most. Even if a developer only maintains a handful of packages, that shouldn’t be a criteria to grant or deny access to the repository.

With the (eventual) implementation of git repositories, we should also be able to work with the pull request methods allowing people who don’t want to become developer to still contribute to the portage tree.

But the most important is not what technical or non-technical abilities they have, or which role they want to take in the Gentoo project, but rather their willingness to perform and work on an operating system used by several thousand users.

14. What else can we utilize the wiki for?
When the wiki was first launched, I started using it as some sort of Knowledge Base [1]. It allows for specific issues or misconfigurations to be documented and assist users in troubleshooting them. I still think this is a worthwhile set of documents to pursue, but needs a lot more content. I hope to, one day, be able to just mine the knowledge from #gentoo (i.e. historical discussions and questions) and put those in the knowledge base.

[1] https://wiki.gentoo.org/wiki/Knowledge_Base

Perhaps we can, one day, use the wiki as some sort of reference architecture for Gentoo. Such a reference architecture would explain readers how Gentoo could be used to create an integrated environment, where each component has bindings with other components, in a well-orchestrated manner.

Right now, most documents focus on a single technology implementation and there is no full picture as to what Gentoo can really offer to organizations and companies of reasonable size.

15. What would you like the main site to be used for and what framework / language should we use for the redesign?
Personally, I think it would be a good idea to focus on a small main site, using a no-nonsense interface like Bootstrap, with support for mobile devices. Keep the amount of information that is dynamic of nature on other sites, like the Gentoo wiki (perhaps in a closed category so that only privileged developers can access it, for instance if it is about the social contract) and focus on telling the reader what Gentoo is and how to get it.

Underlying, this can even be made static HTML. That’s quite powerful, well known to most people, and doesn’t need any (potentially risky) modules on the web sites.

16. As a Gentoo Developer what are some of your accomplishments?
It’s difficult to put these in any order, as their accomplishment value depends on the time ;-) Still, it would be to assist in the Gentoo Handbook, the creation of the Gentoo Foundation, improved integration of SELinux in Gentoo, the Dutch translations (now they’re fully abandoned, but were once the top translation language), package maintenance here and there, support on #gentoo and the Gentoo Forums and what not.

17. What would be your dream job?
Honestly, I have no idea what it would be. However, it would not be as much about the content, but rather the energy that it would give me to go forward. A job with responsibility (but only on areas that you can influence – not the “You’re responsible for everything that goes wrong” kind of jobs), flexibility in hours, close to home, continuous education/improvement possibilities, lots of social contact (but not necessarily in team manner) and an innovative, evolving goal (not a day-in, day-out same kind of job).

18. What are the specs of your current boxes?
I have two laptops at home (a 2-year old i5 and a recent i7 laptop), a hacked Samsung TV, a hacked Ubiquiti router and two Synology DiskStations (which I oddly haven’t modified yet).

Next to the systems at home, I also manage two Dell PowerEdge servers which both host virtual systems for various personal purposes (such as attempts to move current cloud-driven solutions, such as Google mail and calendar) towards self-hosted solutions. These servers are co-located (luckily, because they make too much noise to be in my home).

19. Can you describe your personal desktop setup (WM/DE)?
I run XFCE with 7 xterms and two browsers open. I’m more a CLI guy ;-)

My previous one was fluxbox, which I enjoyed much as well. However, I ran XFCE due to a bug that someone reported (in SELinux support) and I wanted to reproduce it. And for some reason, it stuck.

20. What gives you the most enjoyment within the Gentoo community?
The appreciation received when fixing someone’s situation or helping them get the most of their installation. Honestly, I think that’s the best thing one can receive. Not only because it gives you a warm and fuzzy feeling, but also because these users often start helping others as well. This is why #gentoo is one of the largest support channels out there.

Gentoo @ FOSDEM 2014

(by Pavlos Ratis)

Gentoo Developers @ FOSDEM 2014
Photo by jmbsvicetto

On the 1st and 2nd of February, many Gentoo users and developers attended FOSDEM, the biggest F/OSS conference in Europe. Gentoo developer and council member Donnie Berkholz(dberkholz) had a talk about the status of distribution-level package management and the latest trends. Furthermore, a Gentoo BoF took place on Saturday. There, we had the chance to meet each other and talk about our favorite distro. The day ended with a Gentoo-ish dinner and beers at city’s center.

Council News

(by Andreas K. Huettel)

First of all, Robin Johnson’s (robbat2) GnuPG key policy GLEP is progressing; it is now officially GLEP (draft) 63 [1], will be posted to the mailing list for discussion one last time soon, and be on the agenda of the next council meeting (March 2014) for final confirmation. In the meantime, we’ll be happy to receive feedback.

About EAPIs, the council decided to immediately deprecate EAPI 0 and EAPI 3, which means they should in general not be used in new ebuilds anymore and repoman gives a non-fatal warning on commit. EAPI 1 and EAPI 2, already deprecated for long, will be banned immediately, in the sense that repoman does not allow committing new ebuilds (but existing ones keep working and can also be modified).

Regarding stable keywords usage on m68k, sh, s390 some discussion about details took place. In the end, based on a suggestion by Mike Frysinger (vapier), it was decided that the profiles of these arches should all be marked as experimental; the consensus was that then package maintainers do not have to care about the keywording status on these particular arches and can e.g. remove the last stable marked or keyworded ebuild of a package at will.

The last important topic that was brought up was the policy on tree wide use of the gtk / gtk2 / gtk3 useflags, or to be more precise the clash between the documentation provided by the gnome team and the policy decided on in a recent QA team meeting. Both Chris Reffett (creffett) as QA team lead and Chí-Thanh Christopher Nguyễn (chithead) presented their viewpoints. Further discussion touched upon the question how far-reaching policy decisions the QA team may make. In the end the council members affirmed that “QA’s right to create standards in glep 48 includes flag names / functions”. Subsequent discussion encouraged QA and Gnome team to keep talking.

[1] https://wiki.gentoo.org/wiki/GLEP:63

Staffing Needs

If you are interested in helping out, please visit our staffing needs page on the Gentoo wiki.

Gentoo Developer Moves

Summary

Gentoo is made up of 252 active developers, of which 40 are currently away.
Gentoo has recruited a total of 794 developers since its inception.

Additions

The following developers have recently joined the project:
Returning Dev :D Steve Dibbs (announcement)

Portage

This section summarizes the current state of the portage tree.

Architectures 45
Categories 159
Packages 17243
Ebuilds 37610
Architecture Stable Testing Total % of Packages
alpha 3613 510 4123 23.91%
amd64 10644 6102 16746 97.12%
amd64-fbsd 0 1575 1575 9.13%
arm 2625 1628 4253 24.67%
hppa 3027 468 3495 20.27%
ia64 3187 569 3756 21.78%
m68k 585 77 662 3.84%
mips 2 2292 2294 13.30%
ppc 6870 2354 9224 53.49%
ppc64 4332 849 5181 30.05%
s390 1538 243 1781 10.33%
sh 1762 288 2050 11.89%
sparc 4138 876 5014 29.08%
sparc-fbsd 0 322 322 1.87%
x86 11404 5146 16550 95.98%
x86-fbsd 0 3224 3224 18.70%

gmn-portage-stats-2013-11

Infrastructure

The Gentoo Foundation recently received a donation of services from Rackspace. We would like to thank Rackspace for their donation and for continuing to support Open Source and Free Software Projects.

Rackspace

Security

The following GLSAs have been released by the Security Team

GLSA Package Description Bug
201402-27 x11-plugins/pidgin-knotify pidgin-knotify: Arbitrary code execution 336916
201402-26 net-libs/libssh libssh: Arbitrary code execution 444147
201402-25 dev-libs/openssl OpenSSL: Denial of Service 497838
201402-24 app-crypt/gnupg GnuPG: Multiple vulnerabilities 449546
201402-23 x11-libs/libXfont libXfont: Multiple vulnerabilities 378797
201402-22 net-analyzer/tcptrack TCPTrack: Arbitrary code execution 377917
201402-21 media-libs/tiff libTIFF: Multiple vulnerabilities 440154
201402-20 net-irc/kvirc KVIrc: Multiple vulnerabilities 326149
201402-19 dev-libs/libtar libtar: Arbitraty code execution 487420
201402-18 app-misc/mc GNU Midnight Commander: User-assisted execution of arbitrary code 436518
201402-17 app-text/xpdf Xpdf: User-assisted execution of arbitrary code 386271
201402-16 media-libs/freetype FreeType: Multiple vulnerabilities 448550
201402-15 mail-client/roundcube Roundcube: Arbitrary code execution 488954
201402-14 dev-libs/icu International Components for Unicode: Denial of Service 460426
201402-13 app-text/djvu DjVu: User-assisted execution of arbitrary code 497088
201402-12 sys-auth/pam_skey PAM S/Key: Information disclosure 482588
201402-11 www-client/links Links: Denial of Service 493138
201402-10 media-sound/pulseaudio PulseAudio: Insecure temporary file usage 313329
201402-09 www-apache/mod_fcgid Apache mod_fcgid: Arbitrary code execution 487314
201402-08 net-misc/stunnel stunnel: Arbitrary code execution 460278
201402-07 games-strategy/freeciv Freeciv: User-assisted execution of arbitrary code 329949
201402-06 www-plugins/adobe-flash Adobe Flash Player: Multiple vulnerabilities 491148
201402-05 media-sound/banshee Banshee: Arbitrary code execution 345567
201402-04 dev-perl/libwww-perl libwww-perl: Multiple vulnerabilities 329943
201402-03 x11-libs/pixman Pixman: User-assisted execution of arbitrary code 493292
201402-02 x11-drivers/nvidia-drivers NVIDIA Drivers: Privilege Escalation 493448
201402-01 net-libs/libmicrohttpd GNU libmicrohttpd: Multiple vulnerabilities 493450

Package Removals/Additions

Removals

Package Developer Date
dev-php/PEAR-HTML_BBCodeParser mabi 13 Feb 2014
dev-php/PEAR-HTML_QuickForm_ElementGrid mabi 13 Feb 2014
dev-php/PEAR-HTTP_Upload mabi 13 Feb 2014
dev-php/PEAR-HTTP_WebDAV_Server mabi 13 Feb 2014
dev-php/PEAR-Tree mabi 13 Feb 2014
dev-php/PHPUnit_Selenium mabi 13 Feb 2014
dev-php/PHPUnit_MockObject mabi 13 Feb 2014
media-plugins/vdr-xxvautotimer hd_brummy 14 Feb 2014
media-plugins/vdr-skinclassic hd_brummy 14 Feb 2014
media-plugins/vdr-sky hd_brummy 14 Feb 2014
media-plugins/vdr-skinreel hd_brummy 14 Feb 2014
net-misc/usbip ssuominen 18 Feb 2014

Additions

Package Developer Date
app-crypt/ssdeep radhermit 01 Feb 2014
sec-policy/selinux-couchdb swift 02 Feb 2014
app-text/bdf2psf floppym 10 Feb 2014
dev-python/llvmpy bicatali 10 Feb 2014
dev-python/numba bicatali 10 Feb 2014
dev-python/blz bicatali 10 Feb 2014
dev-python/datashape bicatali 10 Feb 2014
dev-libs/libdynd bicatali 10 Feb 2014
dev-python/dynd-python bicatali 11 Feb 2014
dev-python/llvmmath bicatali 11 Feb 2014
dev-python/pykit bicatali 11 Feb 2014
dev-python/blaze bicatali 11 Feb 2014
dev-util/squashdelta mgorny 11 Feb 2014
dev-util/squashmerge mgorny 11 Feb 2014
dev-python/astroid idella4 12 Feb 2014
net-im/birdie jlec 12 Feb 2014
media-libs/libomxil-bellagio chithanh 12 Feb 2014
dev-ruby/instantiator graaff 13 Feb 2014
dev-ruby/introspection graaff 13 Feb 2014
dev-ruby/multi_test graaff 13 Feb 2014
app-emacs/redo+ ulm 13 Feb 2014
sys-apps/install-xattr blueness 13 Feb 2014
dev-python/astor patrick 14 Feb 2014
dev-lang/hy patrick 14 Feb 2014
sys-block/kvpm kensington 14 Feb 2014
app-misc/mediacrush-cli maksbotan 17 Feb 2014
sec-policy/selinux-pcscd swift 17 Feb 2014
dev-vcs/git-crypt patrick 18 Feb 2014
app-emacs/mediawiki ulm 18 Feb 2014
dev-python/repoze-sphinx-autointerface radhermit 19 Feb 2014
dev-python/kazoo radhermit 19 Feb 2014
kde-misc/kdeconnect mrueg 20 Feb 2014
net-news/canto-daemon pinkbyte 20 Feb 2014
net-news/canto-curses pinkbyte 20 Feb 2014
dev-ruby/http_parser_rb graaff 21 Feb 2014
dev-ruby/http graaff 21 Feb 2014
dev-python/keyczar radhermit 22 Feb 2014
app-emacs/hexrgb ulm 23 Feb 2014
dev-python/scoop slis 23 Feb 2014

Bugzilla

The Gentoo community uses Bugzilla to record and track bugs, notifications, suggestions and other interactions with the development team.

Activity

The following tables and charts summarize the activity on Bugzilla between 27 January 2014 and 26 February 2014. Not fixed means bugs that were resolved as NEEDINFO, WONTFIX, CANTFIX, INVALID or UPSTREAM.

gmn-activity-2014-02

Bug Activity Number
New 1583
Closed 1051
Not fixed 227
Duplicates 171
Total 5480
Blocker 4
Critical 17
Major 67

Closed bug ranking

The developers and teams who have closed the most bugs during this period are as follows.

Rank Team/Developer Bug Count
1 Gentoo Security 105
2 Gentoo Linux Gnome Desktop Team 63
3 Perl Devs @ Gentoo 37
4 Robin Johnson 34
5 Gentoo KDE team 28
6 Gentoo X packagers 25
7 Gentoo Sound Team 25
8 Python Gentoo Team 21
9 Bernard Cafarelli 20
10 Others 692

gmn-closed-2014-02

Assigned bug ranking

The developers and teams who have been assigned the most bugs during this period are as follows.

Rank Team/Developer Bug Count
1 Gentoo Linux bug wranglers 104
2 Gentoo Security 88
3 Gentoo Linux Gnome Desktop Team 73
4 Gentoo KDE team 37
5 Gentoo's Team for Core System packages 35
6 Default Assignee for New Packages 34
7 Java team 34
8 Portage team 34
9 Default Assignee for Orphaned Packages 32
10 Others 1111

gmn-opened-2014-02

Tip of the month

Are you using a packages that needs a maintainer?
To find out use this python script developed by Ewoud Kohl Van Wijngaarden and Chris Stout. The script requires dev-python/beautifulsoup. Users can become maintainers for packages via the proxy-maintainer process.

Heard in the community

Problem installing net-libs/webkit-gtk:* hangs (gobject-introspection problem?) with =x11-drivers/nvidia-drivers-325.*
If you are using the nvidia proprietary driver, you may encounter a g-ir-failure as emerge will hang.
See BUG 463960
There is also a forum post about this.
Work around is:

# eselect opengl set xorg-x11
# emerge -1 webkit-gtk
# eselect opengl set nvidia

Want to emerge (update) all installed packages which depend on some given package P?

eix --deps -# -I P

That will list all packages in short that are installed and have P in their dependency variables plus the package itself.

Thanks go to gentoo-user@lists.gentoo.org for that :)

What do you do if you encounter a bug and it may have already been fixed. Search on bugzilla with this to show all the bugs even if they have been fixed and closed?

ALL category/package

Gentoo Website survey 2012 launched (March 01, 2014, 18:17 UTC)

Building a website that fits the needs of users and editors alike is hard. That's why the Gentoo PR team would like to ask you a few questions about our websites, mainly www.gentoo.org, but also about our other sites.

Take the survey now

Thanks for your time and interest in making our website experience better!

Gentoo at FOSSCOMM 2013 (March 01, 2014, 18:17 UTC)

What? FOSSCOMM 2013

Free and Open Source Software COMmunities Meeting(FOSSCOMM) 2013

When? 20th, April 2013 - 21st, April 2013

Where? Harokopio University, Athens, Greece

Website?http://hua.fosscomm.gr

FOSSCOMM 2013 is almost here, and Gentoo will be there!

We will have a booth with Gentoo promo stuff, stickers, flyers, badges, live DVD's and much more! Whether you're a developer, user, or simply curious, be sure and stop by. We are also going to represent Gentoo in a round table with other foss communities. See you there!

Pavlos Ratis contributed the draft for this announcement.

Gentoo Monthly Newsletter - November 2013 (March 01, 2014, 18:17 UTC)

The November 2013 GMN issue is now available online. This month on GMN:

  • Interview with Richard Freeman. A Gentoo developer, Council and Trustees member.
  • Gentoo as a development environment for newcomers.
  • Overall Gentoo activity status with bugzilla, portage and developer statistics.

Gentoo Monthly Newsletter - January 2014 (March 01, 2014, 18:17 UTC)

The January 2014 GMN issue is now available online.

This month on GMN:

  • Meet up with Gentoo developers and users at this years FOSDEM'14 event.
  • Give back by helping the proxy-maintainers project.
  • Latest Gentoo news, job openings, interesting stats and much more.

Gentoo at LinuxTag 2013 in Berlin (March 01, 2014, 18:17 UTC)

LinuxTag 2013 runs from May 22nd to May 25th in Berlin, Germany. With more than 10,000 visitors last year, it is one of the biggest Linux and open source events in Europe.

You will find the Gentoo booth at Hall 7.1c, Booth 179. Come and visit us! You will meet many of our developers and users, talk with us, plus get some of the Gentoo merchandise you have always wanted.

Miniconf: Gentoo on the OLPC XO-1.75 (March 01, 2014, 18:17 UTC)

At the Gentoo Miniconf 2012 in Prague we will install Gentoo on the OLPC XO-1.75, an ARM based laptop designed as an educational tool for children. If you are interested in joining us, come to the Gentoo booth and start hacking with us!

—Chí-Thanh Christopher Nguyễn

Imagination Technologies

Imagination Technologies has donated two MIPS64-based Ubiquiti ERLite-3 routers, plus resources to aid Gentoo in providing up-to-date root file system builds for MIPS.

Imagination Technologies is a global leader in multimedia, processor and communication technologies. The company creates and licenses market-leading IP solutions for graphics, video and vision, CPU/general purpose processing, multi-standard communications and connectivity, and cross-platform voice and video communications. Imagination's MIPS CPU cores and architectures range from solutions for ultra low-power 32-bit microcontrollers to high-performance 32/64-bit advanced applications and network processing. MIPS is supported by a broad ecosystem of tools and software including open source Linux distributions like Gentoo.

Gentoo Monthly Newsletter - December 2013 (March 01, 2014, 18:17 UTC)

The December 2013 GMN issue is now available online. This month on GMN:

  • Interview with Sergey Popov. A Gentoo developer and the team leader of Qt, proxy-maintainers and desktop-effects teams.
  • Overall Gentoo activity status with bugzilla, portage and developer statistics.

2012 Gentoo Screenshot Contest Results (March 01, 2014, 18:17 UTC)

Gentoo - Still alive and kicking ...

As the quantity and quality of this year's entries will attest, Gentoo is alive, well, and taking no prisoners!

We had 70 entries for the 2012 Gentoo screenshot contest, representing 11 different window managers / desktop environments. Thanks to all that participated, the judges and likewhoa for the screenshot site.

The Winners!

February 28, 2014
Patrick Lauer a.k.a. bonsaikitten (homepage, bugs)
INSTALL_MASK'ing for a better future (February 28, 2014, 07:37 UTC)

So today I was pointed at a funny one:

/etc/systemd/system/ntpdate.service.d/00gentoo.conf
Now instead of being wrongly installed in /usr/lib (whuarghllaaaaaaaawwreghhh!?!$?) there's some config files for systemd bleeding into /etc.

Apart from being inconsistent with itself this eludes all previous ways to avoid useless files from being installed. The proper response thus looks like this now:
INSTALL_MASK="/lib/systemd /lib32/systemd /lib64/systemd /usr/lib/systemd /usr/lib32/systemd /usr/lib64/systemd /etc/systemd"
And on the upside this will break udev unless you carefully move config to /etc (lolwat ur no haz EUNICHS system operation?) - which just motivated me to shift everything I can to eudev.

Reading recommendation: FHS

February 23, 2014
Alexys Jacob a.k.a. ultrabug (homepage, bugs)
py3status v1.3 (February 23, 2014, 19:23 UTC)

I’m glad to announce the release of py3status v1.3 which brings to life a feature request from @tasse and @ttyE0. Guys, I hope this one will please you !

what’s new ?

Along with a localization bug fix thanks to @zetok from Poland, the main new feature is that py3status now supports a standalone mode which you can use when you only want your own modules displayed in an i3bar !

As usual, this release is already available for my fellow Gentoo Linux users and on pypi !

Changelog is here and quick to get, enjoy !

February 22, 2014
Michał Górny a.k.a. mgorny (homepage, bugs)
A few words on lzip compressor (February 22, 2014, 17:08 UTC)

Some of you may already have noticed that sys-apps/ed and sys-fs/ddrescue packages started pulling in lzip archiver. «Is this some new fancy archiver?» you may ask. The answer is «no. It’s been around for a very long time, and it never got any real interest.»

You can read some of the background story in New Options in the World of File Compression Linux Gazette article. Long story short, lzip was created before xz as a response to the limitations of .lzma format used by lzma-utils. However, it never got any real attention and when xz-utils was released as a direct successor to lzma-utils it became practically redundant. And the two projects co-existed silently until lately…

Over the past five years, Antonio Diaz Diaz, lzip’s author, and a few project supporters were trying to convince the community that the lzip format is superior to xz. However, they were never able to provide any convincing arguments to the community, and while xz gained popularity lzip stayed in the shadow. And it was used mostly by the projects Diaz was member of.

It seems that he has finally decided that advocacy will not help his pet project in gaining popularity. Instead, he decided to take advantage of his administrator position in the mentioned GNU projects and discontinue providing non-.lz tarballs. As he says, «surely every user of ddrescue would like to know about lzip […]».

So, Gentoo user, would you like to know about lzip? Let’s try to get a few fair points here.

First of all, it should be noted that the two competing projects are two different implementations of the same compression algorithm — LZMA2, and they use incompatible file formats. Therefore, both can achieve the same compression ratio (and similar speed) but using either of them requires users to download the appropriate tool.

xz is gaining traction lately. The initial doubt period seems to be over, and growing number of projects is adapting the format. This includes both use of the compression library and distribution of sources in .tar.xz. An xz coder is implemented within the kernel and it can be used as a compressor for filesystems. Practically saying, it’s inevitable for Gentoo users since it is used to compress some of the larger sources (like kernel sources).

Lzip is a side project with minor interest. Most of our users were unaware of its existence until they were forced to install it in order to unpack some other package. In fact, since it is used in particularly small packages, the gain from using it (compared to gzip) is even smaller than size of lzip tarball. Not to mention if we compare it to xz that most of our users have installed already.

As analyzed by Ohloh, xz-utils «has a well established, mature codebase» though it is mostly maintained by a single person. lzip is developed by a single person with no officially known contributors, and no public source code repository. The author claims that lzip is mature and will be continually developed but those are only promises, compared to the community growing around xz.

Feature-wise, xz supports more fine-grained configuration of LZMA2 and additional filters (alike 7-Zip). Lzip has a recovery tool and a few extra promises.

xz-utils are written in C, with the compression library and basic utilities being public-domain. Lzip is C++, and GPLv3 (but there’s also a limited public-domain C version). xz-utils use clean autotools, lzip — custom configure script and Makefile.

To sum up: both tools are quite similar and have no strong advantages over the other. However, the popularity of xz makes it a better choice most of the time, while using lzip mostly forces users to install an extra tool for no real benefit. The continuous support and development in xz-utils is guaranteed by the community, while in lzip it’s just author’s promise.

This post would end here if not for the late events. Now lzip gained an important disadvantage: its author is simply unprofessional. While many other projects start shipping .xz compressed tarballs following its rise in popularity, Diaz is grasping at straws and abusing his position trying to force people to use his pet project instead. He has clearly more concern about the popularity of lzip than about the friendliness to users of the other projects he is administering.

February 20, 2014
Patrick Lauer a.k.a. bonsaikitten (homepage, bugs)
gentoo-x86 to git, round two (February 20, 2014, 08:32 UTC)

After my not-so-good experiments with cvs2git I was pointed at cvsps. The currently masked 3.13 release (plus the lastest ~arch version of cvs) seems to do the trick quite well. It throws a handful of warnings about timestamps that appear to be harmless to me.
What I haven't figured out yet is how to "fix" the email addresses, but that's a minor thing.
Take the raw cvs repo as in the first blogpost, then:

$ time cvsps --root :local:/var/tmp/git-test/gentoo-x86-raw/ --fast-export gentoo-x86 > git-fast-export-stream
cvsps: NOTICE: used alternate strip path /var/tmp/git-test/gentoo-x86-raw/gentoo-x86/
cvsps: broken revision date: 2003-02-18 13:46:55 +0000 -> 2003-02-18 13:46:55 file: dev-php/PEAR-Date/PEAR-HTML_Common-1.0.ebuild, repairing.

[SNIP]

real    212m56.219s
user    12m11.170s
sys     6m59.110s
So this step takes near 3h walltime, and consumes ~10GB RAM. It generates about 17GB of temporary data.
To get performance up you'd need a machine with 32GB+ RAM so that you can do that in TMPFS (and don't forget to make /tmp a tmpfs too, because tmpfile() creates lots and lots of temporary files there) - and the tmpfs needs to be >18GB

In theory you can pipe that directly into git-fast-import. To make testing easier I didn't do that..
Throwing everything into git takes "a while" (forgot to time it, about 20 minutes I think):
Alloc'd objects:    9680000
Total objects:      9675121 (    190979 duplicates                  )
      blobs  :      3020032 (    158366 duplicates    1389088 deltas of    2989578 attempts)
      trees  :      5150778 (     32613 duplicates    4633675 deltas of    4709477 attempts)
      commits:      1504311 (         0 duplicates          0 deltas of          0 attempts)
      tags   :            0 (         0 duplicates          0 deltas of          0 attempts)
Total branches:           8 (         3 loads     )
      marks:     1073741824 (   4682709 unique    )
      atoms:         431658
Memory total:        516969 KiB
       pools:         63219 KiB
     objects:        453750 KiB

pack_report: getpagesize()            =       4096
pack_report: core.packedGitWindowSize = 1073741824
pack_report: core.packedGitLimit      = 8589934592
pack_report: pack_used_ctr            =    7139457
pack_report: pack_mmap_calls          =    1976288
pack_report: pack_open_windows        =          3 /          9
pack_report: pack_mapped              = 2545679911 / 8589934592
And then run git gc (warning: Another mem-hungry operation peaking at ~8GB).
The result is about 7.2GB git repository and appears to have full history.

Files to play around with:
Raw copy of the CVS repo (~440MB)
The git-fast-importable stream created by cvsps (biiig)
The mangled compressed git repository that results from it (~6GB)
Edit:
The same repo recompressed (~1.7GB)
"git repack -a -d -f --max-pack-size=10g --depth=100 --window=250" takes ~3 CPU-hours and collapses the size nicely. Thanks, Mr.Klausmann!

February 19, 2014
Anthony Basile a.k.a. blueness (homepage, bugs)
Lilblue Linux: release 20140218 (February 19, 2014, 18:34 UTC)

I just pushed out a new release of Lilblue Linux 20140218 [1] which you can download from any Gentoo mirror [2].  For those of you who don’t know, Lilblue Linux is a security-enhanced fully featured XFCE4 desktop system for amd64.  It is built with Gentoo’s hardened toolchain [3] and uses Gentoo’s hardened-sources for the kernel which include the Grsec/PaX patches [4] for added security.  Lilblue Linux really is Gentoo, so the name is a bit pretentious, but there is one important and interesting twist: it is built using uClibc [5] as its standard C library rather than glibc, giving it some advantages of an embedded system, such as speed.

Release 20140218 is primarily a maintenance release in which I updated all the packages so as to sync up with maintream Gentoo’s stable amd64 set.  I didn’t touch the toolchain since there was no pressing need, but I did update the kernel to hardened-sources-3.12.6.  There were no known major security issue nor major bugs in the previous release.  But, there was a lot of package flux, with lots of fixes that resolved some annoying issues which plagued the earlier release.  One such annoyance was SMPlayer that used to open up a second window to play a video rather than rendering it in the main window.

If you are already running Lilblue, you can probably just do a `emerge –sync; layman -S; emerge -uvND world` and get caught up [6], but if you are starting a fresh system, the newer image cleans out those annoyances, so you want to start there.  One of the reasons I push out new images every few months is that there are always glitches when updating.  This is true of any Gentoo system but all the moreso of Lilblue because most software is developed under the assumption that we are building against glibc.  These assumptions (GNU-isms) manifest themselves in varioius ways: 1) assumptions about the availability of functions which are GNU extensions, such as secure_getenv() in systemd’s code base which eudev removes [7],  2) assumptions about header stacking, eg. using variadic functions without including stdarg.h (You can sometimes get away with this on a glibc system because it sneaks in via some other included header, but not in uClibc), [8] and 3) missing LDFLAGS like, -liconv -lintl or -largp, which are needed to find these breakout libraries [9].  There are, however, some very deep issue which require serious investigation, such as the removal of poll_waiting in glib (versions above 2.30.3) which lead to a dead lock for all applications linking against it.  It turned out that the issue there was in uClibc’s implementation of eventfd() [10].  Another interesting bug in uClibc-0.9.33.2 was the non-atomic implementation of pread() and pwrite() in terms of open() and lseek() [11].  This caused a race in git-1.8.x which does a multithreaded unpacking of the deltas and requires atomic pread/pwrite.  Mike Frysinger (vapier) had already worked out the implementation in terms of SYS_pread64/pwrite64 but these had not yet been backported.  The latest adventure was on arm architecture (yes I’m thinking of porting Lilblue to arm) where the syscall for pread/pwrite was being done using _syscall5() rather than _syscall6() and not properly aligning the 64-bit value on even register pairs.  This again broke pread/pwrite and git, but only on arm arch! [12]  Mike again had the fix and backported it.

Lilblue is built form a stage3-amd64-uclibc-hardened tarball that can be found on any gentoo mirror under /experimental/uclibc along side the Lilblue image itself [2].  I keep the build scripts on Gentoo’s releng (release engineering) git repo [13] and I run them occassionally to see if any major issues are creeping in as mainstream Gentoo evolves.  If everything goes well, then I don’t push out another release to avoid taxing the mirrors.  But when things get complicated, or a large number of packages need updating, I get the feeling I’d better push out another release.  I hope one day to have a binpkg system going where you can donwload a ~200MB seed image and then install from a binhost but this is more involved than I first suspected.

So give it a try in a virtual machine if you like.  It runs out-of-the-box on VirtualBox.  Installation instructions are on the home page [1].  Or run it as you main desktop as I do on one of my boxes at home!

References:

[1] https://wiki.gentoo.org/wiki/Project:Hardened_uClibc/Lilblue

[2] The image is named desktop-amd64-uclibc-hardened-[release].tar.bz2 and can be found at http://distfiles.gentoo.org/experimental/amd64/uclibc/

[3] https://wiki.gentoo.org/wiki/Hardened/Toolchain

[4] http://grsecurity.net/

[5] http://uclibc.org/

[6] While I’ve tried to get most of the fixes either into the main Gentoo tree, or upstream, some patches still remain and so I maintain a repository for them: http://git.overlays.gentoo.org/gitweb/?p=proj/hardened-dev.git;a=shortlog;h=refs/heads/uclibc

[7] See man 3 getenv.  While getenv conforms to SVr4, POSIX.1-2001, 4.3BSD, C89 and C99, secure_getenv() is a GNU extension.

[8] The header stacking problem works both ways.  In https://bugs.gentoo.org/show_bug.cgi?id=497200, sys-apps/kbd failed to build on uClibc because of a missing stdarg.h when trying to prototype functions with variadic parameters.  Contrast this to https://bugs.gentoo.org/show_bug.cgi?id=486782, where app-cdr/cdrtools fails to build because including stdio.h indirectly includes bits/sched.h which defines clone() (as in man 2 clone).  But this clashes with a definition of clone() in cdrtools’ readcd.c.  Upstream felt that this was a poor implementation on the part of uClibc and the stacking problem there should be fixed.  I can’t disagree, but it is a thorny issue!

BTW, for those interested, you can get a little python script I wrote to analyse header stacking from my dev space: http://dev.gentoo.org/~blueness/misc/header-stack.py

[9] In this way Lilblue is similar to Gentoo on FreeBSD.  Rather than using uClibc’s iconv which has issues, Lilblue pulls in dev-libs/libiconv. The additional LDFLAGS are added on a per package basis using /etc/portage/package.env.

[10] https://bugzilla.gnome.org/show_bug.cgi?id=691168

[11] https://bugs.gentoo.org/show_bug.cgi?id=500382

[12] https://bugs.gentoo.org/show_bug.cgi?id=500382

[13] http://git.overlays.gentoo.org/gitweb/?p=proj/releng.git;a=tree;f=tools-uclibc/desktop

Patrick Lauer a.k.a. bonsaikitten (homepage, bugs)
Thunderbird - double sending is better sending (February 19, 2014, 06:43 UTC)

So here's something brilliant I've found while debugging some PGP-issues:

0q2CYNVFEz6wXHAGYArfO/F/faOL5L6fQw9f93FurZgx7Y+iR1J7Civaa7LHxQ8h
FzstP7BYEhCx2HmEZuDf18htDsTBZAlNVGsI0DMb2wFKudCaI7hXhMHpYBQF/rdZ
=3Dw1hZ
-- --  END PGP MESSAGE     


--  -- --  --  -- 070107010101000406000609
Content Type: text/html; charset=ISO 8859 1
Content Transfer Encoding: 8bit

<html>
  <head>
    <meta content="text/html; charset=ISO 8859 1"
      http equiv="Content Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <br>
         BEGIN PGP MESSAGE      <br>
    Charset: ISO 8859 1 <br>
    Version: GnuPG v2.0.22 (MingW32) <br>
    Comment: Using GnuPG with Thunderbird   <a class="moz txt link freetext" href="http://www.enigmail.net/">http://www.enigmail.net/</a> <br>
     <br>
    hQIMA0dhXCfgRaeBAQ/+P2NCYSVE7vxW742D9eYJmJ/7g7xHSvPFuYvGSZk2gRaJ <br>
    JoZ98x+TPjSlvYVWuS+Y2Fz04ydhi4vNcK+QAqImVO0nO6dFvxUfmZiERBcYGs4C <br>
    Lhe+B/I0P/hEDl+Zu/QJ/v+SEcFoXKv2iclrXwWF6RyLlO97iu8UsLYUjLIZ7Y+r <br>
    YGqphoIdJLfVZ9bb05RIb0ZKnYX5dzunpqu6V6zRpwckWCkos7qBOZ9hfBjaFkvD <br>
    ZQAoJM78qQ0//vV6qyxSpXXFEFbDZuJjPjjDfIF+qyNbcW657bDHQH2ctcyvdcTf <br>
(Modulo some dashes, but you get the idea)

So, uhm, there's a multipart-mime mail, with a PGP-encrypted attachment, and then there's a properly quoted HTML attachment, CONTAINING the same PGP attachment BASE64 encoded. Or something. The funny thing is that Thunderbird itself fails to display the body directly, but displays it in the editor window when you reply.
In vino veritas, and tonight I will need lots of veritas to unremember this madness.

February 18, 2014
Alexys Jacob a.k.a. ultrabug (homepage, bugs)
mongoDB v2.4.9/v2.2.7, rabbitMQ v3.2.3 (February 18, 2014, 16:11 UTC)

Quick post about some recent bumps.

mongodb-2.4.9 & mongodb-2.2.7

IMPORTANT : These versions fix a mongos bug which could lead it to report a write as successful when it was not. This affects all versions of MongoDB prior to and including v2.4.8.

Stay tuned on mongoDB, the next post will probably talk about the release of pymongo v2.7 which supports some neat futures from the upcoming mongoDB v2.6 series.

rabbitMQ-3.2.3

I skipped a bump post when releasing the v3.2.2 so you should check out the v3.2.3 changelog as well if you’re willing to know more about those bug fix releases.

Patrick Lauer a.k.a. bonsaikitten (homepage, bugs)
Converting gentoo-x86 to git, first attempt (February 18, 2014, 05:14 UTC)

A first attempt at cvs-to-git conversion of gentoo-x86; not yet complete. Needs: ~4GB storage for cvs repo, a few GB for temporary files, and a few GB for the git repo

Where possible using tmpfs is recommended as this whole operation is very IO-heavy.
Aquire a complete (server-side) copy of the CVS repo:

mkdir cvs; cd cvs
mkdir CVSROOT
rsync anoncvs.gentoo.org::vcs-public-cvsroot/gentoo-x86 . -r --stats
WIP: Use ferringb's modifications to cvs2git to transform the repo
git clone git://pkgcore.org/git-conversion-tools
I haven't figured out why it fails for me yet, but that would make the whole thing a lot easier.

Naive cvs2git run on one category to demonstrate that it works in theory:
cvs2git --encoding=utf_8 --fallback-encoding=ascii 
        --trunk-only --blobfile=./blob --dumpfile=./dump 
        --username=derp cvs/gentoo-x86/app-emulation/
This does work, but it's really slow and doesn't do things like rewrite committer names etc.etc.
cvs2svn Statistics:

Total CVS Files:              6569
Total CVS Revisions:         37696
Total CVS Branches:              0
Total CVS Tags:                  0
Total Unique Tags:               0
Total Unique Branches:           0
CVS Repos Size in KB:        37135
Total SVN Commits:           11385
First Revision Date:    Thu Oct 26 15:02:06 2000
Last Revision Date:     Mon Feb 10 06:58:17 2014

Timings (seconds):

   6   pass1    CollectRevsPass
   0   pass2    CleanMetadataPass
   0   pass3    CollateSymbolsPass
1100   pass4    FilterSymbolsPass
   0   pass5    SortRevisionsPass
   0   pass6    SortSymbolsPass
   2   pass7    InitializeChangesetsPass
   2   pass8    BreakRevisionChangesetCyclesPass
   2   pass9    RevisionTopologicalSortPass
   0   pass10   BreakSymbolChangesetCyclesPass
   2   pass11   BreakAllChangesetCyclesPass
   1   pass12   TopologicalSortPass
   3   pass13   CreateRevsPass
   0   pass14   SortSymbolOpeningsClosingsPass
   0   pass15   IndexSymbolsPass
   3   pass16   OutputPass
1121   total
This creates some temporary files which we feed to git fast-import:
$ git init --bare git-test; cd git-test
$ git fast-import --export-marks=../cvs2git-tmp/git-marks.dat <../blob                    
[snip]
$ git fast-import --import-marks=../cvs2git-tmp/git-marks.dat <../dump 
git-fast-import statistics:

Alloc'd objects:      70000
Total objects:        40469 (       253 duplicates                  )
      blobs  :            0 (         0 duplicates          0 deltas of          0 attempts)
      trees  :        29085 (       253 duplicates      26014 deltas of      26667 attempts)
      commits:        11384 (         0 duplicates          0 deltas of          0 attempts)
      tags   :            0 (         0 duplicates          0 deltas of          0 attempts)
Total branches:           1 (         1 loads     )
      marks:     1073741824 (     43450 unique    )
      atoms:           5742
Memory total:          5454 KiB
       pools:          2173 KiB
     objects:          3281 KiB

pack_report: getpagesize()            =       4096
pack_report: core.packedGitWindowSize = 1073741824
pack_report: core.packedGitLimit      = 8589934592
pack_report: pack_used_ctr            =      28072
pack_report: pack_mmap_calls          =          2
pack_report: pack_open_windows        =          2 /          2
pack_report: pack_mapped              =   19836762 /   19836762
And there's our converted category. Runtime of the cvs2git step is ~1200sec = 20min, the git fast-import steps both take ~5 seconds.
There's still a lot left to figure out, but this should be enough information to allow others to attempt to do this reliably.

February 16, 2014
The future of Gnome 2 (February 16, 2014, 18:49 UTC)

Following forum [1], IRC and mailing list reading, I wanted to clarify Gnome team position on what will happen now that Gnome 3.8 has moved to stable.

Gnome 2 is going to be removed.

I think it cannot be more clear and there are multiple reasons for that. Let’s write a bit about those so people do not try to invent conspiracy theories:

  • Gnome 2 is not maintained anymore, nothing will make this fact go away.
  • the team is understaffed, many of our talented contributors are too busy with real life or simply quit the (Gentoo) project.
  • Bug reports are still flowing for Gnome 2 but none of us in the team are running it anymore because we do not have the extent of time needed for that.

So, yes, Gnome 3 does not suit everyone’s tastes, but most of us still love it, yes, it depends on systemd and most of us would rather keep our good ol’ openrc that did the work just fine but Gnome 2 is going away and nothing will change that.

What we recommend to people who loved Gnome 2 is to switch to alternatives like XFCE, MATE or Cinnamon because there is no point in living in the past.

[1] http://forums.gentoo.org/viewtopic-t-977288.html

February 08, 2014
Gentoo Haskell Herd a.k.a. haskell (homepage, bugs)
7.8.1-rc1 Gentoo experience (February 08, 2014, 18:34 UTC)

A week ago Austin Seipp and GHC team announced first release candidate from 7.8 branch.

As a packager I was especially interested in following features:

  1. GHCi (and dynamic linking) on unregisterised arches, like ia64 and powerpc64
  2. jobs argument for ghc make. Parallel builds for free.
  3. what did seriously break, what was fixed?

First off, -rc1 is packaged in gentoo-haskell overlay (not keyworded as quite a bit of packages fail to build against ghc-7.8).

GHCi (and dynamic linking)

Dynamic linking works like a charm! GHCi loads binaries noticeaby faster. Let’s test it! Simplest synthetic test: how fast do you get prompt from interpreter?

# ghc-7.6:
$ time { echo '1+1' | ghci -package yesod-core >/dev/null; }
real    0m0.626s
user    0m0.550s
sys     0m0.074s
# ghc-7.8:
$ time { echo '1+1' | ghci -package yesod-core >/dev/null; }
real    0m0.209s
user    0m0.172s
sys     0m0.034s

It’s a case, when files are cached in RAM. 3-4 times faster. The same boost should apply every time when you compile something template-haskell related.

jobs argument for ghc make

I’ve went ahead and tried to enable it for all ebuilds.

For some reason ghc eats a lot of system time in that mode. Likely jobs without arguments is not very good idea and i’ll need to limit it by minimum of MAKEOPTS value and some N (Cabal picked 64).

Even in this mode 2-time speedup is visible on large packages.

So what did break?

Not _that_ much, actually.

alex and happy generated parsers

All package maintainers who ship lexers generated by alex and parsers generated by happy are strongly advised to update those tools locally and reissue hackage update, as old parsers do not compile against ghc-7.8.

If you have happened to use low-level

(==#) :: Int# -> Int# -> Bool

primitives, you might need to port your code a bit, as how their type is a bit different:

(==#) :: Int# -> Int# -> Int#

Here is our example fix for arithmoi.

Type inference changed a bit.

Traditionally darcs needed a patch :] In that big mostly dumb patch most interesting bit is explicit assignment:

- where copyRepo =
+ where copyRepo :: IO ()
+ copyRepo =

Even more amusing breakage was in shake, where error was in inability to infer Addr# argument. No idea was it a bug or feature.

Unsafe removals

As we’ve seen in darcs patch many unsafe${something} functions went away from Foreign modules down to their Unsafe counterparts.

Typeable

Typeable representation did change in a substantial way, thus advanced generic stuff will break. I have no example fix, but have a few of broken packages, like dependent-sum.

Hashtable gone from base

Example of fix for frag package. By the way, ghc-7.6 used to eat 8GBs of RAM compiling frag. For ghc-7.8 it was enough 700MBs even for 8 building threads.

Compiler itself

The thing I expected to try didn’t compile: unregisterised arches and GHCi on them.

I’ve hacked-up a workaround to make them build, but in threaded RTS mode it still SIGSEGVs.

STG gurus are welcome to help me :]

I have fundamental questions like:

  • can unregisterised builds support SMP in theory? (via __thread attribute for example)
  • did UNREG ever produce working threaded runtime?
$ cat __foo/foo.hs 
main = print 1
# non-threaded works, as always been
$ inplace/bin/ghc-stage1 --make __foo/foo.hs -threaded -debug -fforce-recomp
#
$ gdb --args ./__foo/foo +RTS -D{s,i,w,g,G,b,S,t,p,a,l,m,z,c,r}
...
(gdb) run
...
7ffff7fb9700: resuming capability 0
7ffff7fb9700: cap 0: created thread 1
7ffff7fb9700: new bound thread (1)
7ffff7fb9700: cap 0: schedule()
7ffff7fb9700: cap 0: running thread 1 (ThreadRunGHC)
Jumping to 0x7ec17f
#
Program received signal SIGSEGV, Segmentation fault.
0x00000000007ec1a2 in stg_returnToStackTop ()
(gdb) bt
#0  0x00000000007ec1a2 in stg_returnToStackTop ()
#1  0x00000000007d26d9 in StgRun (f=0x7ec17f , basereg=0xca0648) at rts/StgCRun.c:81
#2  0x00000000007c7a30 in schedule (initialCapability=0xca0630, task=0xcc3b30) at rts/Schedule.c:463
#3  0x00000000007ca2c4 in scheduleWaitThread (tso=0x7ffff6b05390, ret=0x0, pcap=0x7fffffffd218) at rts/Schedule.c:2346
#4  0x00000000007c0162 in rts_evalIO (cap=0x7fffffffd218, p=0xb61450 , ret=0x0) at rts/RtsAPI.c:459
#5  0x00000000007e04c3 in ioManagerStartCap (cap=0x7fffffffd218) at rts/posix/Signals.c:184
#6  0x00000000007e04f6 in ioManagerStart () at rts/posix/Signals.c:194
#7  0x00000000007d1d5d in hs_init_ghc (argc=0xc96570 , argv=0xc96578 , rts_config=...) at rts/RtsStartup.c:262
#8  0x00000000007d000b in real_main () at rts/RtsMain.c:47
#9  0x00000000007d0122 in hs_main (argc=17, argv=0x7fffffffd418, main_closure=0xb527a0 , rts_config=...) at rts/RtsMain.c:114
#10 0x0000000000404df1 in main ()

Looks like CurrentTSO is complete garbage. Should not happen :]

Conclusion

The experience is positive. I already get bored, when see single-threaded make of ghc-7.6 and want to update a compiler.

Things like yesod, darcs, hoogle, pandoc and xmonad build fine, thus you can get working environment very fast.

Package authors are more eager to fix stuff for this release: it turns bug lookup and benchmarking into very interactive process.

I want to thank All Of You to make push haskell forward!

Thank you!


February 07, 2014
Anthony Basile a.k.a. blueness (homepage, bugs)
The Gentoo Profile Stacking Problem (February 07, 2014, 21:40 UTC)

I thought I’d write a bit about a long standing problem that the hardened team has been facing with Gentoo’s profile system.  Ever since I joined the team around 2009, we’ve had to deal with the “profile stacking problem”.  Most users and devs just merily go along using `eselect profile` to pick the profile closest to the type of system they want and then tweak the various files under /etc/portage, adding a USE flag here, and keywording or unmasking a package there, until they get the “perfect” system.  What I want to do in this post is expose just what goes into designing the profiles that we publicly export.

I was inpsired to write this because of bug #492312.  There we want to re-introduce the hardened desktop profile for amd64, x86 and arm.  I say “re-introduce” because we had to remove it and its sibling profiles /server and /developer.  So what was going on there?

To start, let me give you a nice pice of python code:

import portage
for p in portage.settings.profiles:
    print("%s" % p)

What this little snippet does is print out the profile stack as the directories inherited from one another via the parent file.  Its a useful tool because profile stacking can get very hard to follow.  When the parent file has something simple like just “..” then the inheritance is easy and that directory just inherits all the package.mask, package.unmask etc of the parent directory, as you would expect from the shell meaning of “..”  But what happens when the parent file looks like this:

../../../base
../../../default/linux
../../../arch/amd64
..

as it does for hardened/linux/amd64?  Well then we get some interesting behavior. The first line says, inherit from base. Easy enough since base inherits from nothing else you get all of base’s settings. The second line says inherit from default/linux, which aslo doesn’t inherit from anything.  These setting just add and override those from base.  Easy enough. Ah! But now we come to arch/amd64, where the parent file says

../base
../../features/multilib/lib32

and the inheritance continues to those directories in order. Finally “..” in hardened/linux/amd64 means inherit hardened/linux which sets most of hardened’s needs via make.defaults, package.mask, use.mask and friends. But alas, hardened/linux has its own parent file which reads

../../releases/13.0

and the trip down the rabbit hole continues!  If you are starting to get a little lost, don’t feel bad. It is hard to wrap your brain around stacking, which is why that little script above is so useful.  But the difficulty in following profiles stacking is not the real problem. If you’re like me, you’re too proud to admit you can’t get your head around any complexity ;)   No, the real problem is that you can’t control the stacking order.

To demonstrate, let me refer again to bug #492312.  There we’d like to have a profile which reads

hardened/linux/amd64/desktop

Okay, but what should we put for its parent file?  We’ll need “..” in there to inherit all of hardened/amd64 settings, but we also would like targets/desktop.  So let’s try a parent file that looks like this

..
../../../../targets/desktop

In that case, our little script tells us that our profile stack as follows:

/usr/portage/profiles/base
/usr/portage/profiles/default/linux
/usr/portage/profiles/arch/base
/usr/portage/profiles/features/multilib
/usr/portage/profiles/features/multilib/lib32
/usr/portage/profiles/arch/amd64
/usr/portage/profiles/releases
/usr/portage/profiles/eapi-5-files
/usr/portage/profiles/releases/13.0
/usr/portage/profiles/hardened/linux
/usr/portage/profiles/hardened/linux/amd64
/usr/portage/profiles/targets/desktop
/usr/portage/profiles/hardened/linux/amd64/destkop

And if you switch the order of .. and targets/desktop, you get

/usr/portage/profiles/targets/desktop
/usr/portage/profiles/base
/usr/portage/profiles/default/linux
/usr/portage/profiles/arch/base
/usr/portage/profiles/features/multilib
/usr/portage/profiles/features/multilib/lib32
/usr/portage/profiles/arch/amd64
/usr/portage/profiles/releases
/usr/portage/profiles/eapi-5-files
/usr/portage/profiles/releases/13.0
/usr/portage/profiles/hardened/linux
/usr/portage/profiles/hardened/linux/amd64
/usr/portage/profiles/hardened/linux/amd64/destkop

The problem with the first ordering is that targets/desktop overrides hardened/linux/amd64 and so any USE flags that we may turn off or on in hardened can get reverse in desktop.  The example here is the jit flag — Just-In-Time compilers write executable code on the fly in areas of memory which must be both writeable and executable.  But a PaX hardened kernel will not allow WX mmap-ings because this is an obvious exploit vector.  Rather, in hardened, we prefer slower and safer methods for compiling/interpreting code on the fly than JIT.

Okay, so what about the second ordering.  It may look strange to have target/desktop before base, but that in itself is not an issue.  Here we have the same problem as above but in an even more subtle way!  (See my comment #9 of bug #492312.)  Consider a fairly important package like dev-libs/libxml2.  In the current state of the tree, `emerge -vp dev-libs/libxml2` would give

 [ebuild   R    ] dev-libs/libxml2-2.9.1-r1:2  USE="ipv6 python* ...

for both stacking choices. But if at some point in the future, someone added the following to profiles/default/linux/package.use

#Python support causes problems on xyz
#Don't pull it in if we don't neeed it
dev-libs/libxml2  -python

The vanilla profile default/linux/amd64/13.0/desktop and our hardened profile with targets/desktop last would not change since they have “dev-libs/libxml2 python” in package.use near the bottom of the stack, but our proposed hardened profile with targets/desktop on top would give

 dev-libs/libxml2-2.9.1-r1:2  USE="ipv6 readline ... -python ...

So, both choices for orderings of “..” and “targets/desktop” in our parent file for hardened/linux/amd64/desktop lead to situations where we can’t control what packages get what use flags. What we would like is a stacking that looks something like this

...
/usr/portage/profiles/targets/desktop
/usr/portage/profiles/hardened/linux/amd64
/usr/portage/profiles/hardened/linux/amd64/destkop

but how do we get that with our current inheritance mechanism? One idea that Magnus (Zorry) had was to gutt out this portion of portage and replace the parsing of the parent file with something along the lines of openrc’s depend() { … } clause. Then we just locally say what has to come before/after what and we let the algorithm figure it out. It sounds like an interesting problem if there were two of me and if there were a good chance that it would actually get implemented. In the mean time, we limp along with what we have and do ad hoc fixes as changes in one part of the profiles means we have to adjust other things. Since we are all responsible for different areas of the tree’s profiles, inevitably we cause one another breakage even with the best of intentions. For example, a few days ago, Mike (vapier) removed a masking on the uclibc USE flag in the base profile.  Doing so makes perfect sense. He didn’t tell me, and why should he have to?, but this lead to a small breakage in hardened/linux/uclibc/amd64 and friends where I had to relax that masking.  I only discovered this upon a catalyst run which is a bit annoying.

February 06, 2014
Michał Górny a.k.a. mgorny (homepage, bugs)
Using deltas to speed up SquashFS updates (February 06, 2014, 08:11 UTC)

The ebuild repository format that is used by Gentoo generally fits well in the developer and power user work flow. It has a simple design that makes reading, modifying and adding ebuilds easy. However, the large number of separate small files with many similarities do not make it very space efficient and often impacts performance. The update (rsync) mechanism is relatively slow compared to distributions like Arch Linux, and is only moderately bandwidth efficient.

There were various attempts at solving at least some of those issues. Various filesystems were used in order to reduce the space consumption and improve performance. Delta updates were introduced through the emerge-delta-webrsync tool to save bandwidth. Sadly, those solutions usually introduce other inconveniences.

Using a separate filesystem for the repositories involves additional maintenance. Using a read-only filesystem makes updates time-consuming. Similarly, the delta update mechanism — while saving bandwidth — usually takes more time than plain rsync update.

In this article, the author proposes a new solution that aims both to save disk space and reduce update time significantly, bringing Gentoo closer to the features of binary distributions. The ultimate goal of this project would be to make it possible to use the package manager efficiently without having to perform additional administrative tasks such as designating an extra partition.

Read on… [PDF]

February 04, 2014
Richard Freeman a.k.a. rich0 (homepage, bugs)
Quick EC2 Backups with Duplicity (February 04, 2014, 22:07 UTC)

I’ve been doing online EC2 backups on my Gentoo box for a while, but switched to Duplicity a few months ago and have been very happy with the results.  Setting this up took some trial and error, so I figured I’d share my config in case others find it useful.  But first, here’s why I switched…

Duplicity is based on librsync, and is designed to be a simple command-line-based tool.  Due to librsync the incremental backups it generates are VERY small – if you change one byte in a 2GB file it sends only a few bytes, just like rsync.  It supports encryption and EC2 upload (and a bunch of other options) natively, which means I can ditch a bunch of shell scripting.  It also stores encrypted manifests on the backup destination, which means that if your local index gets out of sync it can quickly synchronize and continue sending incrementals.

My configuration makes use of an alternates-filename patch which you can find in the duplicity bugzilla, or obtain from the rich0 Gentoo overlay.  v0.6.24 of Duplicity will actually implement this in a slightly different manner, so if you do use this patch be aware that you’ll need to do new full backups after you upgrade.  You can just drop the –alternate-filenames option and not use the patch, but if you do so you won’t be able to configure Amazon S3 to archive your difftar files to Glacier.

Once you install duplicity from either Gentoo portage or using my overlay, you’ll need your EC2 credentials.  Then to execute a backup you can run:

AWS_ACCESS_KEY_ID=foo
AWS_SECRET_ACCESS_KEY=bar
TMPDIR=/lots-of-space/tmp
duplicity --encrypt-key ABCDEF01 --archive-dir /var/cache/duplicity \
--exclude-if-present .noonlinebackup \
--exclude-filelist /etc/duplicity-configs/dup-online1-exclude \
--include /etc --include /home --include /root \
--exclude '**' --full-if-older-than 60D --volsize 500 --asynchronous-upload
--alternate-filenames / s3+http://bucket/path/

You should substitute your AWS credentials.  Setting TMPDIR is optional, but if you’re running tmpfs you might not have space for all the stuff you’re backing up.  Setting archive-dir is also optional, but the archives are expendable and I don’t want it in my home getting backed up – if they get deleted duplicity will automatically fetch them back from EC2 (which will cost you money – so don’t delete them needlessly).

Exclude-if-present means that if you touch .noonlinebackup in a directory it won’t back up that directory.  The exclude list is just a file with one path per row which will be excluded.  You can have as many include options as you like.  The exclude/include/exclude followed by a backup on / was the only way I could get it to back up paths relative to root while excluding files that were otherwise in the include paths.  Otherwise the default include/exclude order wasn’t terribly helpful, but I might not be grokking the intent.

Full-if-older-than tells duplicity to create a new full backup every 60d.  You can separately run a variation of this to do cleanup:

AWS_ACCESS_KEY_ID=foo
AWS_SECRET_ACCESS_KEY=bar
TMPDIR=/lots-of-space/tmp 
duplicity --encrypt-key ABCDEF01 --archive-dir /var/cache/duplicity \ 
remove-all-but-n-full 2 --force s3+http://bucket/path/

This will delete all but the last two full backups, and any incrementals that depend on them.

As far as how it performs, this log speaks for itself:

Reading filelist /etc/duplicity-configs/dup-online1-exclude
Sorting filelist /etc/duplicity-configs/dup-online1-exclude
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Tue Jan 21 03:11:45 2014
--------------[ Backup Statistics ]--------------
StartTime 1391501490.67 (Tue Feb 4 03:11:30 2014)
EndTime 1391502084.80 (Tue Feb 4 03:21:24 2014)
ElapsedTime 594.13 (9 minutes 54.13 seconds)
SourceFiles 332755
SourceFileSize 32503585361 (30.3 GB)
NewFiles 456
NewFileSize 22332746 (21.3 MB)
DeletedFiles 11
ChangedFiles 76
ChangedFileSize 2181064980 (2.03 GB)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 543
RawDeltaSize 110980087 (106 MB)
TotalDestinationSizeChange 38473179 (36.7 MB)
Errors 0
-------------------------------------------------

30GB of source data with 2GB of files that changed, and this was protected by transferring 36MB of data with the whole operation completed in 10 minutes.

By using alternative filenames you can set a prefix in S3 to archive your difftar files to glacier.  I wouldn’t archive anything else – the manifests do not account for much space and if you lose your local copy it will re-download them (which is very expensive for glacier if not carefully managed).  The future Duplicity release will allow you to specify separate prefixes for each file type created so that you can just put your difftars in a separate directory.


Filed under: foss, gentoo, linux

February 01, 2014
LinuxCrazy Podcasts a.k.a. linuxcrazy (homepage, bugs)
Podcast 99 | Gentoo Monthly Newsletter (February 01, 2014, 22:33 UTC)

Links

Gentoo Monthly Newsletter
http://blogs.gentoo.org/news/
All about Choice!

LC


Download

ogg

January 31, 2014
Gentoo Monthly Newsletter: January 2014 (January 31, 2014, 19:00 UTC)

Gentoo News

FOSDEM 2014

(by Markos Chandras) By the time you read this, a few of us will be heading to the FOSDEM 2014 event. As usual, FOSDEM takes place the first weekend of February in Brussels. Quite a few Gentoo developers will be there so come and look for us if you want to meet us in person or discuss something that you want to see improved in our favorite distribution. Yes, we accept bribes if you want your bug fixed ASAP ;-) Chances are most of us will be lurking at the Distribution devroomDonnie Berkholz is scheduled to give a talk titled “Is distribution-level package management obsolete?” on Saturday afternoon. Do not miss it!

Tracking orphaned packages

(by Markos Chandras) Orphaned packages is not an uncommon thing in the portage tree. Nearly 6.45% of the available packages lack a maintainer. However, not having a maintainer is not always a bad thing. Actually, most of these packages still work flawlessly. However, looking at the history of orphaned packages (Figure 1) one may observe that their number grew significantly over the past year.

Figure 1

Figure 1

AFRAID NOT! It is not as bad as it seems ;-) Truth is, the reason for the high number of unmaintained packages is the outstanding retirements that happened last year. The retirement team has been actively tracking developer and herd activity removing those who have been inactive for a long time. However, this only justifies the increased number of packages since 2010. On the other hand, the absolute number of packages is definitely something to worry about. Nobody is going to remove unmaintained packages from the tree for no good reason. However, if one of them breaks at some point, then chances are the package will go away if nobody steps up to pick up the pieces. If you are using any of these packages, you can easily help us maintain it through the proxy-maintainers project.

Council News

One first agenda topic concerned the EAPI of the profile directories. Since  all non-deprecated profiles require EAPI=5 support already for a year, the  council decided to give an additional 30 days notice and then switch the  whole profile tree to EAPI=5. This also means that the deprecated 10.0 profiles will  be removed. Next, the move of the Gentoo Linux Enhancement Proposals (GLEPs) to the wiki  and improvements to the GLEP submission process were addressed. Without much discussion, the decision was to follow the suggestions by Chris Reffett (creffett) and update GLEP 1 (which defines the procedures) and GLEP 2 (an example text) accordingly. Summarizing the most important new points, GLEP proposals are now submitted on Bugzilla, can be  discussed on the gentoo-project mailing list instead of gentoo-dev if appropriate, are written in MediaWiki markup and stored on wiki.gentoo.org, and are licensed CC-BY-SA 3.0. Regarding the status of the PGP key requirements GLEP that has been in the works for a while, it will be the first test case for the new procedures, and we’re waiting for Robin Johnson (robbat2) to finalize the text. Finally, during open floor discussion the question of architecture teams lagging behind in stabilizations came up again. The main question here was whether similar rules as already in place for alpha and ia64 should be put in place for all stable arches (maintainers may remove the last stable version of a package if the stablerequest is delayed without reason for more than 90 days). Any decision was deferred; discussion on the mailing lists should take place first.

Catalyst News

After a long period on “life support”, the catalyst repository is going to have major changes introduced to master in the next few days. The work done in the rewrite branch by Brian Dolbec, is finally going to be merged into master through the pending branch. Anyone using catalyst to produce stages is advised to use the latest release (currently 2.0.16). If you need to track the stable branch, please use the catalyst 2.0.9999 ebuild that tracks the 2.X branch. Anyone wanting to help with catalyst development and testing is encouraged to use the 9999 version and report issues to the catalyst team, pending the understanding that master may be broken during the next few months. Please report any issues to our bugzilla with Component: Catalyst. You can always find us in the #gentoo-releng irc channel of freenode. To be clear, these changes will only affect catalyst-9999 and the master branch of the repository. If you’re not using either, this doesn’t affect you.

 Job Openings

The following job openings have been posted since 2014-01-01:

Role Project Requirements
Gentoo-keys Developer Gentoo-keys Good python skills and or gpg key creation, verification knowledge
Web Developer Recruiters Web development knowledge, Ruby on Rails, Bootstrap, basic database knowledge
PyPy hacker Python Moderate ebuild knowledge (we can help with that). Understanding of Python integration within Gentoo. Ability to hack on PyPy's source code. We can provide the infrastructure capable of building PyPy if necessary.

You can see all job openings in the Gentoo Wiki.

Gentoo Developer Moves

Summary

Gentoo is made up of 251 active developers, of which 38 are currently away. Gentoo has recruited a total of 794 developers since its inception.

Moves

The following developers have recently changed roles

Zac Medico, the Lead developer of the Portage package manager announced that is he stepping down from portage development. As a result of which, the team had to ask for help, and after a very short period of time, the team now comprises 18 contributors. Please take a moment to thank Zac for his hard work all these years, and for all of the new contributors for keeping our package manager alive :)

Additions

The following developers have recently joined the project: Yixun Lan (announcement) Samuel Damashek (announcement) Alexander Berntsen (announcement)

Infrastructure

New SSL Certificates

(by Robin H. Johnson) The Gentoo Infrastructure team would like announce that almost all of the public Gentoo services with SSL have been migrated away from CACert. We would like to extend thanks to the certificate authorities that have provided our new certificates: GlobalSign (*.bugs.gentoo.org), and DigiCert (all other certificates). We would also like to thank CACert for their longstanding support.

Fortune is Fickle: Restoring overlays.gentoo.org

(by Alex Legler) This month, Gentoo saw the biggest service outage it has had for a long time. On Friday, January 10, the machine powering overlays.gentoo.org went down. The same day, we reached out to our sponsor who is providing the machine. Unfortunately, the email was only received and acted upon the following Monday where a remote reboot command was issued that sadly could not resolve the issue. Thus, a datacenter technician was dispatched to assess the state of the machine. He found out the mainboard has died. We had hoped that we could restore service by plugging the disks into another machine provided by the same sponsor only to find out that they were in fact still good old IDE drives. Don’t believe me? Here they are:

IDE drives from the old overlays.gentoo.org machine

IDE drives from the old overlays.gentoo.org machine

Thanks to the tireless efforts of our sponsor’s contact, Vassilis, we were able to finally get the overlays data on Thursday (as well as the great picture above). After importing the data into a new, empty overlays setup provisioned by our configuration management and a quick test of a few repositories, I was glad to be able to announce the service restoration. Sadly, the bad patch we’ve been going through wasn’t over yet: Several of the repositories showed corruption which forced us to start looking into the backup and merge the recovered live state with a backup taken a few hours before the outage. Having suffered from all these little setbacks, on Saturday we were able to finally fully restore the service. What have we learned during this outage?

  • First and foremost: Redundancy would have spared us almost a week of downtime. Thus, we’re looking into preparing a second machine to host Overlays.
  • Very important as well: Keeping up an information flow. The incident marked the baptism by fire for our recently launched Infrastructure Status web site. We were glad to have this site at our disposal to update the community on developments and the status of the service. We’re hoping that next time (let’s hope not too soon though) even more people know about this site and use it.
  • The decision to restore from backup should have been made earlier. In the end, we ascertained only a couple of hours of work were lost and could easily be re-pushed onto the server.

Special thanks again to Vassilis and his colleagues for their help and to you, our community, for bearing with us during the outage as well as countless offers of help with hardware and hosting.

Portage

This section summarizes the current state of the portage tree.

Architectures 45
Categories 159
Packages 17189
Ebuilds 37614
Architecture Stable Testing Total % of Packages
alpha 3606 517 4123 23.99%
amd64 10636 6050 16686 97.07%
amd64-fbsd 0 1573 1573 9.15%
arm 2604 1598 4202 24.45%
hppa 3022 464 3486 20.28%
ia64 3162 573 3735 21.73%
m68k 548 68 616 3.58%
mips 0 2285 2285 13.29%
ppc 6865 2357 9222 53.65%
ppc64 4323 856 5179 30.13%
s390 1548 230 1778 10.34%
sh 1767 279 2046 11.90%
sparc 4128 884 5012 29.16%
sparc-fbsd 0 322 322 1.87%
x86 11390 5111 16501 96.00%
x86-fbsd 0 3219 3219 18.73%

gmn-portage-stats-2013-11

Security

The following GLSAs have been released by the Security Team

GLSA Package Description Bug
201401-33 perl-core/digest-base Perl Digest-Base module: Arbitrary code execution 385487
201401-32 mail-mta/exim Exim: Multiple vulnerabilities 322665
201401-31 app-emacs/cedet CEDET: Privilege escalation 398227
201401-30 None Oracle JRE/JDK: Multiple vulnerabilities 404071
201401-29 media-libs/vips VIPS: Privilege Escalation 344561
201401-28 app-misc/tomboy Tomboy: Privilege escalation 356583
201401-27 app-office/texmacs GNU TeXmacs: Privilege escalation 337532
201401-26 net-analyzer/zabbix Zabbix: Shell command injection 493250
201401-25 net-libs/ldns ldns: Arbitrary code execution 384249
201401-24 net-nntp/inn INN: Man-in-the-middle attack 432002
201401-23 app-admin/sudo sudo: Privilege escalation 459722
201401-22 dev-ruby/activerecord Active Record: SQL injection 449826
201401-21 app-text/poppler Poppler: Multiple vulnerabilities 489720
201401-20 net-analyzer/cacti Cacti: Multiple vulnerabilities 324031
201401-19 dev-libs/gmime GMime: Arbitrary code execution 308051
201401-18 dev-libs/opensc OpenSC: Arbitrary code execution 349567
201401-17 sys-apps/pcsc-lite PCSC-Lite: Arbitrary code execution 349561
201401-16 app-crypt/ccid CCID: Arbitrary code execution 349559
201401-15 net-misc/asterisk Asterisk: Multiple vulnerabilities 449828
201401-14 net-misc/curl cURL: Multiple vulnerabilities 456074
201401-13 app-emulation/virtualbox VirtualBox: Multiple Vulnerabilities 434872
201401-12 gnustep-base/gnustep-base GNUstep Base library: Multiple vulnerabilities 325577
201401-11 dev-lang/perl Locale Maketext Perl module: Multiple vulnerabilities 384887
201401-10 media-libs/libexif exif: Multiple vulnerabilities 426366
201401-09 net-misc/openswan Openswan: User-assisted execution of arbitrary code 483204
201401-08 net-misc/ntp NTP: Traffic amplification 496776
201401-07 dev-libs/libxslt libxslt: Denial of Service 433603
201401-06 dev-vcs/git Git: Privilege escalation 335891
201401-05 net-misc/dhcp ISC DHCP: Denial of Service 463848
201401-04 dev-lang/python Python: Multiple vulnerabilities 325593
201401-03 net-analyzer/nagstamon Nagstamon: Information disclosure 476538
201401-02 net-im/gajim Gajim: Information disclosure 442860
201401-01 dev-dotnet/libgdiplus Libgdiplus: Arbitrary code execution 334101

Package Removals/Additions

Removals

Package Developer Date
dev-php/DBUnit mabi 06 Jan 2014
dev-php/PEAR-File_PDF mabi 06 Jan 2014
dev-java/jdictrayapi mr_bones_ 08 Jan 2014
app-office/rabbit mrueg 13 Jan 2014
app-i18n/rskkserv mrueg 13 Jan 2014
dev-ruby/postgres mrueg 13 Jan 2014
dev-ruby/radiant mrueg 17 Jan 2014
dev-ruby/actionwebservice graaff 18 Jan 2014
dev-ruby/gettext_activerecord graaff 18 Jan 2014
dev-ruby/gettext_rails graaff 18 Jan 2014
kde-base/solid kensington 20 Jan 2014
kde-base/kuiviewer kensington 20 Jan 2014
kde-base/kstartperf kensington 20 Jan 2014
kde-base/kdesdk-scripts kensington 20 Jan 2014
kde-base/kdesdk-misc kensington 20 Jan 2014
kde-base/kdegraphics-strigi-analyzer kensington 20 Jan 2014
games-board/capitalism hasufell 23 Jan 2014
games-board/CapiCity ulm 23 Jan 2014
dev-ruby/sqlite-ruby mrueg 24 Jan 2014
dev-ruby/dbd-sqlite3 mrueg 24 Jan 2014
dev-ruby/dbd-sqlite mrueg 24 Jan 2014
dev-ruby/dbd-pg mrueg 24 Jan 2014
dev-ruby/dbd-odbc mrueg 24 Jan 2014
dev-ruby/dbd-mysql mrueg 24 Jan 2014
dev-ruby/dbi mrueg 24 Jan 2014

Additions

Package Developer Date
sci-libs/vtkdata jlec 02 Jan 2014
dev-util/icemon scarabeus 02 Jan 2014
media-libs/hupnp-ng pinkbyte 02 Jan 2014
dev-java/jlibeps mrueg 03 Jan 2014
dev-python/mox3 idella4 03 Jan 2014
dev-vcs/git-merge-changelog ulm 04 Jan 2014
dev-perl/MediaWiki-API dilfridge 04 Jan 2014
net-misc/libreswan floppym 05 Jan 2014
dev-python/bcrypt idella4 05 Jan 2014
lxde-base/lxappearance-obconf nullishzero 05 Jan 2014
app-text/openlp anarchy 05 Jan 2014
kde-base/calendarjanitor creffett 06 Jan 2014
kde-base/contactthemeeditor dilfridge 06 Jan 2014
dev-php/phpcov mabi 06 Jan 2014
dev-vcs/gitinspector jlec 06 Jan 2014
net-misc/stuntman chainsaw 07 Jan 2014
sci-libs/magma bicatali 07 Jan 2014
kde-misc/redshift-plasmoid mrueg 08 Jan 2014
dev-util/igprof maksbotan 08 Jan 2014
dev-php/phpDocumentor mabi 08 Jan 2014
app-text/XML-Schema-learner mjo 09 Jan 2014
app-admin/cdist hwoarang 09 Jan 2014
dev-python/tmdb3 floppym 11 Jan 2014
dev-perl/Statistics-Distributions civil 12 Jan 2014
dev-perl/Statistics-TTest civil 12 Jan 2014
dev-perl/Getopt-Tabular civil 12 Jan 2014
dev-perl/Benchmark-Timer civil 12 Jan 2014
dev-java/disruptor ercpe 12 Jan 2014
net-misc/leapcast vapier 12 Jan 2014
dev-java/jackson-annotations ercpe 12 Jan 2014
dev-java/jackson-databind ercpe 12 Jan 2014
games-rpg/to-the-moon hasufell 12 Jan 2014
sci-libs/chemkit jlec 13 Jan 2014
dev-libs/rapidxml jlec 13 Jan 2014
dev-java/cal10n ercpe 13 Jan 2014
dev-java/slf4j-ext ercpe 13 Jan 2014
sci-libs/lemon jlec 13 Jan 2014
sci-libs/coinor-sample bicatali 14 Jan 2014
sci-libs/coinor-utils bicatali 14 Jan 2014
sci-libs/coinor-osi bicatali 14 Jan 2014
sci-libs/coinor-vol bicatali 14 Jan 2014
sci-libs/coinor-dylp bicatali 14 Jan 2014
sci-libs/scalapack bicatali 14 Jan 2014
sci-libs/mumps bicatali 14 Jan 2014
sci-libs/coinor-clp bicatali 14 Jan 2014
sci-libs/coinor-cgl bicatali 14 Jan 2014
sci-libs/coinor-cbc bicatali 14 Jan 2014
sci-libs/coinor-alps bicatali 14 Jan 2014
sci-libs/coinor-netlib bicatali 14 Jan 2014
sci-libs/coinor-bcp bicatali 14 Jan 2014
sci-libs/coinor-bcps bicatali 14 Jan 2014
sci-libs/coinor-blis bicatali 14 Jan 2014
sci-libs/coinor-csdp bicatali 14 Jan 2014
sci-libs/coinor-dip bicatali 14 Jan 2014
sci-libs/coinor-flopcpp bicatali 14 Jan 2014
sci-libs/coinor-mp bicatali 14 Jan 2014
sci-libs/coinor-smi bicatali 14 Jan 2014
sci-libs/coinor-symphony bicatali 14 Jan 2014
sci-libs/coinor-bonmin bicatali 15 Jan 2014
sci-libs/coinor-couenne bicatali 15 Jan 2014
sci-libs/coinhsl bicatali 15 Jan 2014
sci-libs/ipopt bicatali 15 Jan 2014
sci-libs/coinor-cppad bicatali 15 Jan 2014
sci-libs/coinor-os bicatali 15 Jan 2014
sci-libs/avogadrolibs jlec 16 Jan 2014
sys-cluster/libcircle ottxor 18 Jan 2014
app-emulation/armv8-fast-model vapier 18 Jan 2014
dev-db/lmdb eras 18 Jan 2014
www-client/google-chrome-beta floppym 19 Jan 2014
www-client/google-chrome-unstable floppym 19 Jan 2014
dev-ml/pa_bench aballier 19 Jan 2014
dev-ml/typerep aballier 19 Jan 2014
dev-ml/pa_test aballier 19 Jan 2014
dev-ml/re2 aballier 19 Jan 2014
dev-ml/async_kernel aballier 19 Jan 2014
dev-ml/faillib aballier 19 Jan 2014
sec-policy/selinux-cachefilesd swift 19 Jan 2014
net-libs/libmbim alexxy 20 Jan 2014
dev-java/jortho sera 20 Jan 2014
app-misc/asciinema kensington 20 Jan 2014
net-libs/libnftnl chainsaw 20 Jan 2014
sys-firmware/iwl7260-ucode gienah 23 Jan 2014
media-gfx/librecad slis 23 Jan 2014
sys-apps/getent blueness 23 Jan 2014
games-board/CapiCity hasufell 23 Jan 2014
games-board/capicity ulm 23 Jan 2014
x11-libs/gtk-mac-integration grobian 23 Jan 2014
x11-misc/sxhkd radhermit 24 Jan 2014
x11-wm/bspwm radhermit 24 Jan 2014
app-crypt/scrypt radhermit 24 Jan 2014
net-firewall/nftables chainsaw 24 Jan 2014
sys-firmware/iwl3160-ucode gienah 25 Jan 2014
sys-firmware/iwl3160-7260-bt-ucode gienah 25 Jan 2014
dev-libs/efl tommy 25 Jan 2014
dev-python/sphinx-better-theme floppym 25 Jan 2014
dev-python/backports radhermit 26 Jan 2014
dev-python/backports-ssl-match-hostname radhermit 26 Jan 2014
app-leechcraft/lc-rosenthal maksbotan 26 Jan 2014

Bugzilla

The Gentoo community uses Bugzilla to record and track bugs, notifications, suggestions and other interactions with the development team.

Activity

The following tables and charts summarize the activity on Bugzilla between 29 December 2013 and 28 January 2014. Not fixed means bugs that were resolved as NEEDINFO, WONTFIX, CANTFIX, INVALID or UPSTREAM.

gmn-activity-2014-01

Bug Activity Number
New 1653
Closed 1298
Not fixed 233
Duplicates 186
Total 5427
Blocker 5
Critical 19
Major 68

Closed bug ranking

The developers and teams who have closed the most bugs during this period are as follows.

Rank Team/Developer Bug Count
1 Gentoo Security 95
2 Gentoo's Team for Core System packages 60
3 Perl Devs @ Gentoo 43
4 Default Assignee for Orphaned Packages 42
5 Gentoo Linux Gnome Desktop Team 32
6 Robin Johnson 31
7 Gentoo KDE team 30
8 Gentoo Sound Team 29
9 Python Gentoo Team 28
10 Others 907

gmn-closed-2014-01

Assigned bug ranking

The developers and teams who have been assigned the most bugs during this period are as follows.

Rank Team/Developer Bug Count
1 Gentoo Linux bug wranglers 145
2 Gentoo Security 65
3 Gentoo Linux Gnome Desktop Team 59
4 Gentoo's Team for Core System packages 55
5 Portage team 40
6 Default Assignee for New Packages 38
7 Gentoo KDE team 38
8 media-video herd 34
9 Default Assignee for Orphaned Packages 30
10 Others 1148

gmn-opened-2014-01

Tip of the month

(by Pavlos Ratis) Many of us are using overlays every day. Overlays vary from very small to big enough in size. As a result they slow down the majority of Portage operations. That happens because overlays do not contain metadata cache. The cache is used to speed up searches and the building of dependency trees. A neat trick is to generate local metadata cache after syncing overlays.

# layman -S
# emerge --regen

This trick also works in conjunction with eix. eix-update can use metadata cache generated by emerge –regen to speed up things. To enable this, add the following variable in /etc/eixrc. OVERLAY_CACHE_METHOD="assign"

Bonus: Fun tips

  1. Have you mooed today?: emerge --moo
  2. Emerge games-misc/doge and/or games-misc/cowsay to beautify your motd ;)

Alexys Jacob a.k.a. ultrabug (homepage, bugs)
keepalived v1.2.11 & glusterfs v3.4.2 (January 31, 2014, 16:46 UTC)

Quick post for two quick bumps related to clustering.

glusterfs-3.4.2

  • quite a lot of bug fixes and improvements
  • contains a backport for libgfapi support for integrating with NFS Ganesha
  • nfs/mount3: fix crash in subdir resolution

keepalived-1.2.11

  • autoconf: better libnl3 detection
  • Fix memory allocation for MD5 digest
  • Quite some nice memory leak fixes on different components
  • vrrp: dont try to load ip_vs module when not needed
  • Pim van den Berg work on libipvs-2.6 to sync with libipvs from ipvsadm 1.27
  • vrrp: extend ip parser to support default and default6
  • vrrp: fix/extend gratuitous ARP handling (multiple people reported issues where MASTER didnt recover properly after outage due to no gratuitous ARP sent)
  • Multiple fixes to genhash
  • vrrp: fix vrrp socket sync while leaving FAULT state (old old bug here)
  • Full changelog here

January 30, 2014
Luca Barbato a.k.a. lu_zero (homepage, bugs)
Fosdem! (January 30, 2014, 16:58 UTC)

About 26h before Fosdem (yes, the beer event is the glorious start of the conference)!

What

I’ll be around bearing chocolate and chocolate for friends and fellow members of the communities I belong to (no beers this time, sorry guys!), hopefully we’ll find some space to discuss anything you’d like to discuss with me.

Topics

  • Libav (We should also have a room to discuss some more Libav10 and Libav11 planned releases)
  • VLC (Probably most discussions during the meeting, where Felix will stab me for not having done hwaccel2)
  • Gentoo/Sabayon (Complaints and rants welcome only during the beer event)
  • Any of my other many projects (contributions welcome btw!)
  • Anything else.

Where

There might be a room to discuss for about 1 hour about Libav10 Sunday, I’ll be around the Gentoo BoF Saturday and obviously I’ll be around attending some of the events.

See you there! (hopefully)

January 29, 2014
Michael Palimaka a.k.a. kensington (homepage, bugs)
app-misc/asciinema – new package (January 29, 2014, 10:41 UTC)

I recently added app-misc/asciinema to the tree, a tool used to record and share terminal sessions. Everything is text-based, so it’s much more lightweight than video recording approaches.

Let me know if you do anything useful with it. :-)

ASCII Solitaire

January 28, 2014
Robin Johnson a.k.a. robbat2 (homepage, bugs)

Munin is commonly used to graph lots of systems stuff, however it lacks a common piece of functionality: 95th percentile.

The Munin bug tracker has ticket #443 sitting open for 7 years now, asking for this, and proving a not-great patch for it.

I really wanted to add 95th percentile to one of my complicated graphs (4 base variables, and 3 derived variables deep), but I didn't like the above patch either. Reading the Munin source to consider implementing VDEF properly, I noticed an undocumented setting: graph_args_after. It was introduced by ticket #1032, as a way of passing things directly to rrdtool-graph.

Clever use of this variable can pass in ANYTHING else to rrdtool-graph, including VDEF! So without further ado, here's how to put 95th percentile into individual Munin graphs, relatively easily.

# GRAPHNAME is the name of the graph you want to render on.
# VARNAME is the name of the new variable to call the Percentile line.
# DEF_VAR is the name of the CDEF or DEF variable from earlier in your graph definition.
# LEGEND is whatever legend you want to display on the graph for the line.
#   FYI Normal rrdtool escaping rules apply for legend (spaces, pound, slash).
${GRAPHNAME}.graph_args_after \
  VDEF:${VARNAME}=gcdef${DEF_VAR},95,PERCENT \
  LINE1:${VARNAME}\#999999:${LEGEND}:dashes \
  GPRINT:${VARNAME}:\%6.2lf\%s\\j
# Example of the above I'm using
bandwidth1.graph_args_after \
  VDEF:totalperc=gcdeftotal,95,PERCENT \
  LINE1:totalperc\#999999:95th\ Percentile\ (billable\):dashes \
  GPRINT:totalperc:\%6.2lf\%s\\j

January 27, 2014
David Abbott a.k.a. dabbott (homepage, bugs)
Ambiance-Gentoo Theme Buttons (January 27, 2014, 21:50 UTC)

Just installed the theme Ambiance-Gentoo from the package x11-themes/light-themes for my Gnome3 desktop. I just had to get rid of those orange buttons. Unpack this file and read the readme.

Thats it :)

Luca Barbato a.k.a. lu_zero (homepage, bugs)
Gentoo on Macbookpro 2013 – part2 (January 27, 2014, 04:19 UTC)

Go read here for the first part.

What changed

  • The linux 3.13 still doesn’t sport support for the wireless yet. The closed source driver works almost great beside when conman crashes horribly due some bad interaction, luckily doesn’t happen often, sadly I do not have time to debug it.
  • Using grub with the patch pointed in the comments here does make appear the intel gpu and you can enjoy using it for hardware decoding using QSV, patches for Libav availabe in the usual place.
  • vga switcheroo doesn’t let me switch properly, apparently nouveau takes the console framebuffer and does not really wants to release it.
  • the nvidia closed source driver works but you lose the access to the gmux, so you can’t change the brightness, and your console framebuffer is gone as well.
  • bbswitch seems confused enough to give up, I might ask upstream to help me figure out since seems almost everything is there post-grub-setup.
  • My pommed patches are waiting for more testers, then I’ll bake a release for everybody. Here it is working nicely.

What next

  • Probably I’ll try to figure out better how get the intel gpu work fully, since nouveau works mostly fine but Blizzard games on wine do not play at all.
  • Pommed will see more cleanups.
  • Hopefully I’ll play more with the displayports.

So far I’m still quite happy about this model even if the mentioned quirks.

January 24, 2014
Patrick Lauer a.k.a. bonsaikitten (homepage, bugs)
NTP: Working around bad hardware since 1842 (January 24, 2014, 06:00 UTC)

After my recent troubles with NTP and excessive time drift things have settled down.

For reasons unknown to me the time drift on the problem server changed from -330ppm to +2.3ppm. I'm not quite sure how to interpret that.
Comparing some other machines I have access to:

Old P4:         -292.238
Dell R510:        12.428
Another R510:     13.232
Random amd64:    -28.438
Another amd64:   -23.323
A newish Xeon:    -7.296
So the general trend seems to be older = more time drift. And machines with "same" hardware appear to have similar drift factors.
The stability of <30ppm means a drift of about 0.1sec/day. Without extra correction that's tolerable (a few seconds a month), but still unsatisfactory.

The old P4 drifting at 300ppm means it'll be getting close to 3 minutes a week away from "real time" - that's enough to cause problems if you rely on it.

I think the lesson in this is "manufacturers use the cheapest they can get away with", so every computer should have a time correction mechanism (NTP, DCF-77, GPS - doesn't matter as long as you correct it). And there's a reasonable assumption that environmental factors (heat, hardware aging, change in the provided voltage, ...) will randomly change the time drift.

And I thought timekeeping was a problem solved two centuries ago ...

January 21, 2014
Mike Pagano a.k.a. mpagano (homepage, bugs)

I did a minor bump to fix a compile error for the fbcondecor patch. If you don’t use fbcondecor,  there is no need to upgrade. That is the only change.

January 20, 2014
Paweł Hajdan, Jr. a.k.a. phajdan.jr (homepage, bugs)
System update report (last update a year ago) (January 20, 2014, 22:32 UTC)

I have some systems that I update more rarely than the others. Of course I highly recommend keeping up to date, especially for the security fixes. I've also written similar update reports in the past. You may want to read them and compare the experience:

Another 5-month update (December 2011)
Another report from rarely updated system (September 2012)

Read more »

January 17, 2014
Alexys Jacob a.k.a. ultrabug (homepage, bugs)
uWSGI v2.0 (January 17, 2014, 11:09 UTC)

Yesterday was a big day for the famous application container uWSGI. We released the brand new version 2.0 LTS along with quite a huge bump of the ebuild, closing 6 bugs at once. I thought I’d give some input about the ebuild changes and some quick notes about uWSGI. Many thanks again to @dev-zero !

New plugins selection : UWSGI_PLUGINS

We introduced a new USE_EXPAND named UWSGI_PLUGINS so that you can now select which plugins to build individually. This is a great step as it makes the compilation more clear and lets you fine tune your uWSGI installation.

Along this work, we had to describe each plugin which was also quite a challenge. To my knownledge, this has not been done anywhere else so here it is. Please ping me if you have something to add or if we failed to describe a plugin correctly.

Migration note : You will need to change your package.use configuration to switch to using UWSGI_PLUGINS. As an example, where you had the USE flag spooler enabled you’ll now need to use uwsgi_plugins_spooler.

uWSGI v2.0 highlights

These are my biased favorites, go check for more, it’s huge !

Patrick Lauer a.k.a. bonsaikitten (homepage, bugs)
Unexpected fun with NTP (January 17, 2014, 03:24 UTC)

This morning I had to fix an unexpected dovecot "failure" by restarting it. Apparently it only tolerates time jumps of less than seven seconds.
The trigger of this oopsie is NTP:

Jan 16 23:52:53 stupidserver ntpd[27668]: synchronized to 202.112.10.36, stratum 3
Jan 16 23:52:45 stupidserver ntpd[27668]: time reset -7.732856 s
Riiight. That's not nice, but why does it jump around so much? Looks like the time behaviour worsened over the last days:
Jan 15 19:34:18 stupidserver ntpd[27668]: no servers reachable
Jan 15 19:59:56 stupidserver ntpd[27668]: synchronized to 202.112.10.36, stratum 2
Jan 15 20:06:22 stupidserver ntpd[27668]: time reset +0.533773 s
...
Jan 16 11:47:33 stupidserver ntpd[27668]: synchronized to 202.112.10.36, stratum 2
Jan 16 11:47:30 stupidserver ntpd[27668]: time reset -2.966137 s
...
Jan 16 18:14:28 stupidserver ntpd[27668]: synchronized to 202.112.10.36, stratum 2
Jan 16 18:15:27 stupidserver ntpd[27668]: time reset -4.223295 s
...
Jan 16 23:52:53 stupidserver ntpd[27668]: synchronized to 202.112.10.36, stratum 3
Jan 16 23:52:45 stupidserver ntpd[27668]: time reset -7.732856 s
That's an offset of more than 1sec/h, and that's with ntpd correcting at around 330 PPM. The docs say: "The capture range of the loop is 500 PPM at an interval of 64s decreasing by a factor of two for each doubling of interval." (PPM = parts-per-million)
In other words, if the drift is above 500 PPM it may force a clock reset because it can't drift fast enough. And it looks like this situation was either a failing mainboard RTC clock, or a screwed up ntp server (since it always sync'ed to the same one).

I've tried two things to avoid this time skipping:
1) Change the ntp servers used to something more "local" - the global pool.ntp.org may not be as reliable as servers geographically close you
2) Remove the drift file to force the system to re-learn

The results, at first glance, look promising:
Jan 17 10:48:37 stupidserver ntpd[3059]: kernel time sync status 0040
Jan 17 10:52:55 stupidserver ntpd[3059]: synchronized to 202.120.2.101, stratum 3
Jan 17 10:52:50 stupidserver ntpd[3059]: time reset -5.023639 s
Jan 17 10:57:54 stupidserver ntpd[3059]: synchronized to 202.120.2.101, stratum 3
Jan 17 11:01:08 stupidserver ntpd[3059]: synchronized to 202.73.36.32, stratum 1
Jan 17 11:05:34 stupidserver ntpd[3059]: kernel time sync enabled 0001
So after an initial 5-second skip it managed to sync twice without abnormal drift. Let's hope that it's going to stay sane ...

January 16, 2014
Patrick Lauer a.k.a. bonsaikitten (homepage, bugs)
EAPI usage in tree (January 16, 2014, 06:36 UTC)

Total number of ebuilds: 37807

EAPI 0:  5959  15.78%
EAPI 1:   370   0.98%
EAPI 2:  3335   8.82%
EAPI 3:  3005   7.95%
EAPI 4: 12385  32.76%
EAPI 5: 12746  33.72%
That looks quite good: EAPI5 has grown very well, EAPI1 is almost gone.

EAPI0 is still needlessly common, and EAPI 2+3 should be deprecated.

Update: Now running as a cronjerb, Output here, History here

January 15, 2014
Greg KH a.k.a. gregkh (homepage, bugs)
kdbus details (January 15, 2014, 20:57 UTC)

Now that linux.conf.au is over, there has been a bunch of information running around about the status of kdbus and the integration of it with systemd. So, here’s a short summary of what’s going on at the moment.

Lennart Poettering gave a talk about kdbus at linux.conf.au. The talk can be viewed here, and the slides are here. Go read the slides and watch the talk, odds are, most of your questions will be answered there already.

For those who don’t want to take the time watching the talk, lwn.net wrote up a great summary of the talk, and that article is here. For those of you without a lwn.net subscription, what are you waiting for? You’ll have to wait two weeks before it comes out from behind the paid section of the website before reading it, sorry.

There will be a systemd hack-fest a few days before FOSDEM, where we should hopefully pound out the remaining rough edges on the codebase and get it ready to be merged. Lennart will also be giving his kdbus talk again at FOSDEM if anyone wants to see it in person.

The kdbus code can be found in two places, both on google code, and on github, depending on where you like to browse things. In a few weeks we’ll probably be creating some patches and submitting it for inclusion in the main kernel, but more testing with the latest systemd code needs to be done first.

If you want more information about the kdbus interface, and how it works, please see the kdbus.txt file for details.

Binder vs. kdbus

A lot of people have asked about replacing Android’s binder code with kdbus. I originally thought this could be done, but as time has gone by, I’ve come to the conclusion that this will not happen with the first version of kdbus, and possibly can never happen.

First off, go read that link describing binder that I pointed to above, especially all of the links to different resources from that page. That should give you more than you ever wanted to know about binder.

Short answer

Binder is bound to the CPU, D-Bus (and hence kdbus), is bound to RAM.

Long answer

Binder

Binder is an interface that Android uses to provide synchronous calling (CPU) from one task to a thread of another task. There is no queueing involved in these calls, other than the caller process is suspended until the answering process returns. RAM is not interesting besides the fact that it is used to share the data between the different callers. The fact that the caller process gives up its CPU slice to the answering process is key for how Android works with the binder library.

This is just like a syscall, and it behaves a lot like a mutex. The communicating processes are directly connected to each other. There is an upper limit of how many different processes can be using binder at once, and I think it’s around 16 for most systems.

D-Bus

D-Bus is asynchronous, it queues (RAM) messages, keeps the messages in order, and the receiver dequeues the messages. The CPU does not matter at all other than it is used to do the asynchronous work of passing the RAM around between the different processes.

This is a lot like network communication protocols. It is a very “disconnected” communication method between processes. The upper limit of message sizes and numbers is usually around 8Mb per connection and a normal message is around 200-800 bytes.

Binder

The model of Binder was created for a microkernel-like device (side note, go read this wonderful article about the history of Danger written by one of the engineers at that company for a glimpse into where the Android internals came from, binder included.) The model of binder is very limited, inflexible in its use-cases, but very powerful and extremely low-overhead and fast. Binder ensures that the same CPU timeslice will go from the calling process into the called process’s thread, and then come back into the caller when finished. There is almost no scheduling involved, and is much like a syscall into the kernel that does work for the calling process. This interface is very well suited for cheap devices with almost no RAM and very low CPU resources.

So, for systems like Android, binder makes total sense, especially given the history of it and where it was designed to be used.

D-Bus

D-Bus is a create-store-forward, compose reply and then create-store-forward messaging model which is more complex than binder, but because of that, it is extremely flexible, versatile, network transparent, much easier to manage, and very easy to let fully untrusted peers take part of the communication model (hint, never let this happen with binder, or bad things will happen…) D-Bus can scale up to huge amounts of data, and with the implementation of kdbus it is possible to pass gigabytes of buffers to every connection on the bus if you really wanted to. CPU-wise, it is not as efficient as binder, but is a much better general-purpose solution for general-purpose machines and workloads.

CPU vs. RAM

Yes, it’s an over simplification of a different set of complex IPC methods, but these 3 words should help you explain the differences between binder and D-Bus and why kdbus isn’t going to be able to easily replace binder anytime soon.

Never say never

Ok, before you start to object to the above statements, yes, we could add functionality to kdbus to have some blocking ioctl calls that implement something like: write question -> block for reply and read reply one answer for the request side, and then on the server side do: write answer -> block in read That would get kdbus a tiny bit closer to the binder model, by queueing stuff in RAM instead of relying on a thread pool.

That might work, but would require a lot of work on the binder library side in Android, and as a very limited number of people have write access to that code (they all can be counted on one hand), and it’s a non-trivial amount of work for a core function of Android that is working very well today, I don’t know if it will ever happen.

But anything is possible, it’s just software you know…

Thanks

Many thanks to Kay Sievers who came up with the CPU vs. RAM description of binder and D-Bus and whose email I pretty much just copied into this post. Also thanks to Kay and Lennart for taking the time and energy to put up with my silly statements about how kdbus could replace binder, and totally proving me wrong, sorry for having you spend so much time on this, but I now know you are right.

Also thanks to Daniel Mack and Kay for doing so much work on the kdbus kernel code, that I don’t think any of my original implementation is even present anymore, which is probably a good thing. Also thanks to Tejun Heo for help with the memfd implementation and cgroups help in kdbus.

January 12, 2014
Alexys Jacob a.k.a. ultrabug (homepage, bugs)
py3status v1.2 (January 12, 2014, 21:52 UTC)

I’m glad to announce a new release of py3status with an exciting main new feature giving the ability to modify any of i3status’ module output from any of your modules !

feature

changelog

  • new module dpms.py allowing activation and deactivation of DPMS thx to André Doser
  • order i3status output updates to prevent it from overwritting any modification made on i3status json list by a user module, this avoids a possible user filter flapping on i3status modules
  • fix delay on first execution of each module which could be equal to py3status interval time before being executed : your modules get executed and displayed immediately no matter py3status’ interval
  • the real i3status thread output json list is passed to all modules as the i3status_output_json parameter, this allows any user module to change any of the i3status output by simply altering the given json on the list, inspired thx to @drestebon on issue #23
  • add validation for the position parameter
  • add cpu usage info to sysdata script, by Patrick Shan

contributors

Many thanks to all contributors for their work and inspiration.

  • Patrick Shan, @patrickshan
  • @drestebon
  • André Doser, @tasse

January 10, 2014
Alexys Jacob a.k.a. ultrabug (homepage, bugs)
Tuning pacemaker for large clusters (January 10, 2014, 11:19 UTC)

We’ve been running quite a lot of production clusters using pacemaker/corosync for a while. Some of them are large, handling more than 200 resources across multiple nodes and we’ve exceeded some limits on pacemaker’s CIB size.

I thought I’d share how to tune your cluster to handle such a bunch of resources since there are some default limits on the IPC buffer size which can lead to problems when your resources (and thus CIB) grows too much.

Hitting the IPC limit

When running a large cluster you may hit the following problem :

error: crm_ipc_prepare: Could not compress the message into less than the configured ipc limit (51200 bytes).Set PCMK_ipc_buffer to a higher value (2071644 bytes suggested)

Evaluating the buffer size

Have a look at the size of your current CIB :

# cibadmin -Ql > cib.xml
# ls -l cib.xml
# bzip2 cib.xml
# ls -l cib.xml.bz2

The CIB is compressed on the wire using bzip2 so you have to compare the compressed cib.xml.bz2 with the IPC default buffer size of 51200 and you’ll find the sufficient PCMK_ipc_buffer value for you (take more just to be safe).

Setting the environment variables

On Gentoo Linux, you’ll have to create the /etc/env.d/90pacemaker file containing :

PCMK_ipc_type=shared-mem
PCMK_ipc_buffer=2071644
  • PCMK_ipc_buffer : you may need to increase this depending on your cluster size and needs
  • PCMK_ipc_type : the shared-mem one is the default now, other values are socket|posix|sysv

You will also need to set these env. vars in your .bashrc so that the crm CLI doesn’t break :

export PCMK_ipc_type=shared-mem
export PCMK_ipc_buffer=2071644

Future

Finally, I wanted to let you know that the upcoming Pacemaker v1.1.11 should come with a feature which will allow the IPC layer to adjust the PCMK_ipc_buffer automagically !

Hopefully you shouldn’t need this blog post anymore pretty soon :)

EDIT, Jan 16 2014

Following this blog post, I had a very interesting comment from @beekhof (lead dev of pacemaker)

beekhof> Ultrabug: regarding large clusters, the cib in 1.1.12 will be O(2) faster than 1.1.11.
Ultrabug> beekhof: that's great news mate ! when is it scheduled to be released ?
beekhof> 30th of Feb

January 09, 2014
Sven Vermeulen a.k.a. swift (homepage, bugs)

Sounds like a stupid question, as the answer is already in the title. If a company has only RedHat Enterprise Linux as allowed / supported Linux platform (be it for a support model requirement, ISV certification, management tooling support or what not) how could or would Gentoo still play a role in it.

But the answer is, surprisingly, that Gentoo can still be made available in the company architecture. One of the possible approaches is a virtual appliance role.

Virtual appliances are entire operating systems, provided through VM images (be it VMDK or in an OVF package), which offer a well defined service to its consumers. More and more products are being presented as virtual appliances. Why not – in the past, they would be in sealed hardware appliances (but still running some form of Linux on it) but nowadays the hypervisor and other infrastructure is strong and powerful enough to handle even the most intensive tasks in a virtual guest.

Gentoo is extremely powerful as a meta-distribution. You can tweak, update, enhance and tune a Gentoo Linux installation to fulfill whatever requirement you have. And in the end, you can easily create a virtual image from it, and have it run as a virtual appliance in the company.

An example could be to offer a web-based password management suite. A Gentoo Linux deployment could be created, fully hardened of course, with a MAC such as SELinux active. On it, a properly secured web server with the password management suite, with underlying database (of course only listening on localhost – don’t want to expose the database to the wider network). Through a simple menu, the various administrative services needed to integrate the “appliance” in a larger environment can be configured: downloading an SSL certificate request (and uploading the signed one), (encrypted) backup/restore routines, SNMP configuration and more.

If properly designed, all configuration data could be easily exported and imported (or provided through a secundary mount) so that updates on the appliance are as simple as booting a new image and uploading/mounting the configuration data.

Building such a virtual appliance can be simplified with Gentoo Prefix, multi-tenancy on the web application level through the webapp-config tool while all necessary software is readily available in the Portage tree.

All you need is some imagination…

January 06, 2014
Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)
Git and MediaWiki (January 06, 2014, 00:18 UTC)

I'm again and again amazed at the modular archtecture of Git, the distributed version control system. One feature that I've just discovered is its MediaWiki integration. In short, you can clone the contents of a wiki into a local git repository, pull, make changes in the wikitext, commit them, and even push them back onto the wiki. Amazingly, this is all possible without any special server-side support; the normal MediaWiki API is sufficient.
If you want to try, you'll need dev-vcs/git-1.8.5.2 or later with the mediawiki useflag enabled. (Because of missing dependencies, at the moment the useflag is masked on all arches except amd64, but that will hopefully change soon.) The documentation for the module can be found on github.
Here's a small demonstration. Since cloning a whole wiki is timeconsuming and also (if done repeatedly) not very nice towards the wiki operators, let's clone only all pages of the "Desktop" category on the Gentoo wiki:

git clone -c remote.origin.categories='Desktop' \
mediawiki::http://wiki.gentoo.org
This creates a directory wiki.gentoo.org, we change directory in there and call "git log --stat", and obtain a version history:
commit d10a150c78e743d1e62c39f5a7feadf81c552e28
Author: Tox2ik <Tox2ik@wiki.gentoo.org>
Date: Wed Jan 1 14:52:24 2014 +0000

Broke up a long sentence.

Fontconfig.mw | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

commit 1da83052b8062239e265458c538b2efb32ca25a8
Author: El Salmon <El Salmon@wiki.gentoo.org>
Date: Tue Dec 3 19:05:37 2013 +0000

Checking configuration

Fontconfig.mw | 4 ++++
1 file changed, 4 insertions(+)

commit 9583e17c82508d166d9a30733765d3bdfd9d7f3e
Author: Emery <Emery@wiki.gentoo.org>
Date: Fri Oct 25 04:53:35 2013 +0000

Tense changes (changes to tense).

Tallscreen_Monitor.mw | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)

...
The files in the Git working copy are raw wikitext files, and can easily be edited by hand. So, we edit a file and commit that change locally into Git:
commit 6af2bfa0bba4cf825d4e52242709ffccfe341223
Author: Andreas K. Huettel (dilfridge) <dilfridge@gentoo.org>
Date:   Sat Jan 4 23:47:55 2014 +0100

    Add klayout to the scientific applications

 Recommended_applications.mw | 1 +
 1 file changed, 1 insertion(+)
To be able to push the change back into the wiki, we need to log in there. So, we tell Git our wiki username:
git config remote.origin.mwLogin Dilfridge
Afterwards, a simple "git push" is enough to publish our change; Git will ask for the wiki password.
While definitely a cool feature, there are some limitations. The most obvious one that I have come across is the following: if different people clone from the wiki, they obtain different git histories, i.e. the commits in their histories will have different hashes. This means sharing changesets via git is only possible if the cloning from MediaWiki takes place only once, and then the generated git repository is cloned in turn. In addition, I suspect the author of a change in the wiki is set by the username when pushing the change, not by the author of the git commit.
Anyway, I think this might be useful.

December 30, 2013
Gentoo Monthly Newsletter: December 2013 (December 30, 2013, 19:44 UTC)

Gentoo News

Interview with Sergey Popov

(by David Abbott)

Sergey Popov  is a Gentoo developer and the team leader of Qt, proxy-maintainers and desktop-effects teams.

Who is Sergey Popov?

In short: system administrator, Linux fan.
System administration is my job. I work in geographically distributed company, a technical university, with departments all over the region. Also, I am open source contributor and Gentoo Developer, (surprise! :-) ) and I really enjoy that role.

How did you get involved with Linux and Open Source, and what was the path that lead to you to Gentoo?

Well, first of all, I first interacted with Linux when I began to work in my alma mater as junior system administrator, when I was a 2nd year student. Senior admins were mostly undergraduates and thus they were busy with diplomas. So, me and my colleague began to study *NIX systems, cause we have 3 servers, running Fedora Core 5, if i remember correctly.
I was aware of Linux, but only had a little expirience with Debian on VPS in high school.
Some of my colleague’s had been charmed by the power of FreeBSD, but I decided to stay with Linux.
After some experiments I came to Gentoo. God, how awesome it was, and still is, compared to other binary distro’s. Soon, we got rid of Fedora entirely, as it was replaced on our servers by Gentoo :-)

What aspects of Gentoo do you feel the developers and maintainers have got right?

First of all, Gentoo is about choice. When somebody tells me that it’s not about choice, rather that it’s about flexibility, I think it does not matter how it sounds, only what it means. For me it means near unlimited possibilities of customization.
So, me and our fellow developers provide choice for users. And this is main thing that we are doing right, I think.

What is it about Gentoo you would like to see improved?

Portage, while it is one of the best package managers I have ever seen, sometimes can be really slow :-( .
Also, I think we should focus on tightening user-developer interaction, because it is our source for new developers, which in turn bring new software to the tree and improves the support for existing software.

What are some of the projects within Gentoo that you enjoy contributing to?

Well, arch teams and security is my primary focus lately, so thats what I am spending most of my time on. But I have changeable personality, so it usually shifts after some time.

How can users get involved with proxy maintaining?

Well, we are always looking for enthusiastic users that have, or want to learn, skills in ebuild maintained and who wants a package to be integrated within the Gentoo ecosystem. It’s quite simple to pick a package and become a proxy maintainer, the process is described on our project page.

Describe some of the challenges being a team lead?

Well, first of all, team lead is organizational duty. So, you do not need to be the the most skilled in your team, but as team lead you should know about direction of development and define it. So, the main challenge for me was to see the whole project from the position of leader, to understand this position properly. And I hope that I am doing this right :-)

What arch teams are you involved with, and describe the process and any special problems in keeping packages stabilized?

I am member of amd64, arm and mips arch teams. Working with amd64 is simplest one – easy access to hardware, a major arch, so compatibility problems – near zero, but some old software from 200x or even 19xx, that still exists in portage tree, can have problems. arm – harder one, because of the slower  hardware(Raspberry Pi) for testing packages(but qemu-user chroot saves me from endless waiting for compilations ;-) ), compatibility problems – presents, but rarely. mips is the hardest from one side(different ABIs, endianness, etc) and specific problems(e.g. aligning), but from the other side – it is unstable-only arch – so, it ease things.

MIPS is testing only, why is that?

Well, let me give you some technical background. Let’s took amd64 as example. It’s major arch(according to last GMN we have more keyword coverage for it then even for x86, nice!). It has 3 supported ABIs in Gentoo – 32,64 and x32(which is less supported due to many breakages in vast amount of software, but that’s not our topic). We can have multilib or use only natively compiled binaries, it does not matter.
Now, let’s talk about mips. What do we have here? 3 supported ABIs – n32, o32 and n64. Same as for x86, so what differs? And here goes Endianness. We can have those ABIs either Big Endian(BE) or Little Endian(LE). So, we have much more combinations that can break software. And, as our resources(both manpower and hardware) are limited, we just can not afford maintaining two branches(stable/unstable) for that arch.

What was the process you went through to become involved with the security team?

Well, to be honest, security is not my strong side(I for example, have very limited installations of Hardened Gentoo, but I am sure – it will grow), but I always cared about it. That was mixed feelings – I imagined that all security team members are gurus in exploits, shell code stuff and such, while I am not. But, no matter, I decided to try to become at least GLSA coordinator, cause I thought that I can help with GLSA release process and, well, if I will stuck somewhere – ask for help from senior members. At that moment I was aware that recruitment process differs and now, from the inside, I understand why. Security is one of the key points, cause we, as developers should provide programs and different solutions for our users, but they should be, well, ‘secure’. And this can be very time-consuming activity – to get information from security mail lists, handle it properly(either in a form of simple bug report, upstream interaction or patch) and bring fixes to tree. And again, and again – never ending fun :-) . That’s why, for proper training, we have opportunities for ordinary users to become security scouts and padawans(more details – on our project page). As I was already Gentoo developer I passed through this training right to full team membership in two months.

What is your programming background? 

I began with Pascal in high school, then I was charmed by Assembler. After that was C/C++ and PHP. Have some basic reading skills in Perl and Python.

Which open source programs would you like to see developed?

First of all, Linux kernel, primarily in networking and visualization. Network and socket tools(I am system administrator, first of all): nmap, netcat, tcpdump, wireshark, socat. Portage becomes nicer and nicer with each release.

What resources have you found most helpful when troubleshooting within Gentoo and Linux in general?

Well, if sort them in order of absolute amount of knowledge acquired, that would be:

  1. gentoo-wiki.info(ex gentoo-wiki.com)
  2. gentoo.org (handbook, project docs, forums, wiki, etc.)
  3. gentoo.ru
  4. other resources, mostly found via Google :-)

What can users do to improve Gentoo and how can we get users and developers working more closely?

Well, first and the most valuable aspect is closer interaction between users and developers. Filing bugs to bugzilla, talking in IRC, etc. If a user wants to participate in improving Gentoo there are many opportunities for them. Making a personal overlay public and register it in layman maybe one way. Another opportunity – contributing to sunrise overlay or directly to main tree through Proxy maintainers. Last two requires not only basics of ebuild writing but some knowledge of QA standards and guidelines.

What advice do you have for people wanting to become Gentoo Developers?

Learn the developer documentation. Do not be scared of the quizzes. Improve your skills. Last one is a constant process, you can not relax when you become a Gentoo developer – it’s just the beginning for your future progress.

Tell us your mentoring experiences, what do you get out of it?
Well, I could not say that I am person who can teach others, but my mentee was really persistent, so I decided to try. And it was successful after all, my mentee beat the quizzes and passed review sessions. And I… well… I revised my position about teaching others – when they are really motivated it is not hard to help them, it is a pleasure.

What needs to be improved, changed, fixed in the recruitment process?
Quizzes should be updated(some updates happened already and it’s good) to include some questions about subslots, for example. Situation with recruiters and mentors seems fine now, so we just should keep things as they are.

You are currently the Gentoo Qt lead, tell us about that
Well, it was my first experience as team lead. Our team keeps regular meetings to discuss some major problems(bugs, integration questions, etc.), so I need to learn how to hold a meeting. And, thanks to yngwin(previous lead), I have learned it quickly. The main topic now is inclusion of Qt5 in main tree. There is some work that has already been done, but there is more work ahead.

Where do you see Gentoo 5 years from now?
Well, that’s hard to predict, honestly. I hope that we continue to move on to our goals and develop our tools for easing users’ life.

Can you describe your personal desktop setup (WM/DE)?
Currently I have 2 desktops – one at home, one at work. Both running Gentoo Linux(mostly stable, with few things in package.accept_keywords). I use KDE 4 as DE on them. Home desktop has Compiz as WM replacement for default kwin.

What are the specs of your current boxes and describe your home network?
My home LAN is divided into some segments. First of all – main segment, where all wired devices are connected. Here are my PC router(Pentium IV, 2.8Ghz, 1 core with HyperThreading, uClibc as C library), desktop(Intel Core i7, 3Ghz, 4 cores with HyperThreading, multi-seat setup with 2 complements of VGA/Keyboard/Mouse/Monitor, both VGAs are NVIDIA) and recently bought MIPS router(Cavium Octeon, 500Mhz, 2 cores). Then – Wi-Fi segment, shared through PC router and PCI Wi-Fi card(Atheros chipset, very easy setup). Persistent client on this network is my Raspberry Pi model B with USB Wi-Fi adapter. All of listed devices are running, of course, Gentoo Linux :-) . There are also three virtual segments in my desktop for virtualization purposes(KVM/Libvirt). PC router are linked with work desktop through OpenVPN and I utilize Quagga to redistribute routes to/from it.

What gives you the most enjoyment within the Open Source community?
Contribution to such project as Gentoo, first of all: knowledge that you fixes will ease life of users is really encouraging. Chatting with interesting people in IRC with different areas of interests and skills is a fun too.

How did you get the nick “pinkbyte”?
Origins come from the “Tron” movie and character “Bit” that can transform yourself into red figure when answering ‘No’.

Gentoo Council News

One thing on the agenda of this month’s council meeting was once more the modernization of the Gentoo Code of Conduct. Our decision was to make some minimal changes that basically adapt the wording to the status quo and remove mention of long gone projects such as the proctors. The second agenda topic was improvement of GLEP 48, which defines the role of the QA team. The GLEP was amended such that the QA lead is elected by the team members but has to be confirmed by the council, with a term of one year. If the QA team lead position remains vacant, the council may appoint an interim lead.

Gentoo Developer Moves

Summary

Gentoo is made up of 250 active developers, of which 42 are currently away.
Gentoo has recruited a total of 791 developers since its inception

Moves

The following developers have recently changed roles

Additions

The following developers have recently joined the project

Nicolas Bock (announcement)

Michael Orlitzky (announcement)

Portage

This section summarizes the current state of the portage tree.

Architectures 44
Categories 159
Packages 17111
Ebuilds 38053
Architecture Stable Testing Total % of Packages
alpha 3576 540 4116 24.05%
amd64 10607 5985 16592 96.97%
amd64-fbsd 0 1572 1572 9.19%
arm 2583 1596 4179 24.42%
hppa 3022 456 3478 20.33%
ia64 3117 595 3712 21.69%
m68k 515 95 610 3.56%
mips 0 2266 2266 13.24%
ppc 6859 2375 9234 53.97%
ppc64 4317 870 5187 30.31%
s390 1613 156 1769 10.34%
sh 1834 210 2044 11.95%
sparc 4094 909 5003 29.24%
sparc-fbsd 0 325 325 1.90%
x86 11390 5032 16422 95.97%
x86-fbsd 0 3199 3199 18.70%

gmn-portage-stats-2013-11

Security

The following GLSAs have been released by the Security Team

GLSA Package Description Bug
201312-16 media-gfx/xfig Xfig: Arbitrary code execution 348344
201312-15 net-proxy/tinyproxy Tinyproxy: Denial of Service 432046
201312-14 media-libs/libsndfile libsndfile: Arbitrary code execution 375125
201312-13 net-analyzer/wireshark Wireshark: Multiple vulnerabilities 484582
201312-12 app-crypt/mit-krb5 MIT Kerberos 5: Multiple vulnerabilities 429324
201312-11 media-libs/win32codecs Win32 Codecs: User-assisted execution of arbitrary code 232999
201312-10 net-libs/libsmi libsmi: Arbitrary code execution 342127
201312-09 app-arch/cabextract cabextract: Multiple vulnerabilities 329891
201312-08 media-libs/libwebp WebP: User-assisted execution of arbitrary code 442152
201312-07 media-libs/openexr OpenEXR: Multiple Vulnerabilities 277202
201312-06 app-accessibility/festival Festival: Arbitrary code execution 386319
201312-05 dev-lang/swi-prolog SWI-Prolog : Multiple vulnerabilities 450284
201312-04 media-libs/libtheora libtheora: Arbitrary code execution 298039
201312-03 dev-libs/openssl OpenSSL: Multiple Vulnerabilities 369753
201312-02 sys-apps/busybox BusyBox: Multiple vulnerabilities 379857
201312-01 sys-libs/glibc GNU C Library: Multiple vulnerabilities 350744

Package Removals/Additions

Removals

Package Developer Date
app-arch/xarchiver hwoarang 02 Dec 2013
kde-misc/kio-upnp-ms johu 04 Dec 2013
kde-misc/qtrans johu 04 Dec 2013
sys-apps/pcfclock pinkbyte 09 Dec 2013
dev-python/python-subunit idella4 12 Dec 2013
app-text/gsview mr_bones_ 14 Dec 2013
mail-client/gbuffy mr_bones_ 14 Dec 2013
net-print/pup mr_bones_ 14 Dec 2013
dev-libs/libsmtp mr_bones_ 14 Dec 2013
net-analyzer/traffic-vis mr_bones_ 14 Dec 2013
dev-libs/pwlib moult 15 Dec 2013
net-libs/openh323 moult 15 Dec 2013
app-emulation/qenv moult 15 Dec 2013
dev-lang/v8cgi phajdan.jr 18 Dec 2013
dev-lang/v8 phajdan.jr 18 Dec 2013
media-sound/omptagger graaff 29 Dec 2013
dev-ruby/id3lib-ruby graaff 29 Dec 2013

Additions

Package Developer Date
games-misc/sound-of-sorting blueness 02 Dec 2013
dev-python/sure idella4 02 Dec 2013
dev-python/misaka idella4 02 Dec 2013
dev-python/steadymark idella4 02 Dec 2013
dev-python/httpretty idella4 02 Dec 2013
dev-python/libvirt-python cardoe 02 Dec 2013
dev-util/spec-cleaner scarabeus 03 Dec 2013
net-mail/postfix-logwatch mjo 03 Dec 2013
app-leechcraft/lc-htthare pinkbyte 03 Dec 2013
sys-libs/libapparmor kensington 03 Dec 2013
sys-apps/apparmor kensington 03 Dec 2013
sys-apps/apparmor-utils kensington 03 Dec 2013
sec-policy/apparmor-profiles kensington 03 Dec 2013
net-misc/vrrpd robbat2 03 Dec 2013
dev-python/XenAPI idella4 05 Dec 2013
dev-ruby/ruby-clutter-gstreamer naota 05 Dec 2013
dev-lang/moarvm patrick 06 Dec 2013
dev-python/queuelib patrick 06 Dec 2013
dev-libs/boost-numpy heroxbd 06 Dec 2013
app-emacs/visual-basic-mode ulm 07 Dec 2013
dev-python/pysrt tomwij 07 Dec 2013
sys-apps/epoch tomwij 07 Dec 2013
dev-python/retry-decorator vapier 09 Dec 2013
sys-block/blocks jlec 10 Dec 2013
dev-python/python-subunit idella4 10 Dec 2013
dev-haskell/connection gienah 11 Dec 2013
dev-haskell/control-monad-loop gienah 11 Dec 2013
dev-haskell/free gienah 11 Dec 2013
dev-haskell/http-client gienah 11 Dec 2013
dev-haskell/http-client-conduit gienah 11 Dec 2013
dev-haskell/http-client-multipart gienah 11 Dec 2013
dev-haskell/http-client-tls gienah 11 Dec 2013
dev-haskell/keys gienah 11 Dec 2013
dev-haskell/monad-loops gienah 11 Dec 2013
dev-haskell/mono-traversable gienah 11 Dec 2013
dev-haskell/process-conduit gienah 11 Dec 2013
dev-haskell/stm-chans gienah 11 Dec 2013
dev-haskell/vector-instances gienah 11 Dec 2013
dev-haskell/pointed gienah 11 Dec 2013
dev-haskell/warp-tls gienah 11 Dec 2013
xfce-extra/xfce4-windowck-plugin ssuominen 11 Dec 2013
games-emulation/pcsxr mgorny 11 Dec 2013
dev-haskell/tasty-quickcheck gienah 11 Dec 2013
dev-ruby/blankslate mrueg 12 Dec 2013
dev-ruby/parslet mrueg 13 Dec 2013
dev-ruby/mercenary mrueg 13 Dec 2013
dev-ruby/slim mrueg 13 Dec 2013
dev-ruby/memoizable mrueg 13 Dec 2013
dev-ruby/toml mrueg 13 Dec 2013
dev-ruby/asciidoctor mrueg 13 Dec 2013
dev-ruby/org-ruby mrueg 13 Dec 2013
dev-ruby/hipchat mrueg 13 Dec 2013
dev-ruby/settingslogic mrueg 13 Dec 2013
dev-ruby/gemoji mrueg 13 Dec 2013
dev-ruby/equalizer mrueg 13 Dec 2013
dev-ruby/buftok mrueg 13 Dec 2013
dev-ruby/adhearsion-loquacious mrueg 13 Dec 2013
dev-ruby/http-cookie mrueg 13 Dec 2013
dev-ruby/turbolinks mrueg 13 Dec 2013
dev-ruby/seed-fu mrueg 13 Dec 2013
dev-ruby/d3_rails mrueg 13 Dec 2013
dev-ruby/modernizr mrueg 13 Dec 2013
dev-ruby/ffaker mrueg 13 Dec 2013
dev-ruby/letter_opener mrueg 13 Dec 2013
sci-chemistry/freeon nicolasbock 13 Dec 2013
sys-cluster/charmdebug nicolasbock 13 Dec 2013
sys-cluster/projections nicolasbock 13 Dec 2013
dev-python/babelfish tomwij 14 Dec 2013
dev-libs/libmongo-client vadimk 14 Dec 2013
dev-python/Yamlog idella4 15 Dec 2013
dev-python/Bcryptor idella4 15 Dec 2013
games-emulation/mupen64plus-core mgorny 15 Dec 2013
games-emulation/mupen64plus-audio-sdl mgorny 15 Dec 2013
games-emulation/mupen64plus-input-sdl mgorny 15 Dec 2013
games-emulation/mupen64plus-rsp-hle mgorny 15 Dec 2013
games-emulation/mupen64plus-video-rice mgorny 15 Dec 2013
games-emulation/mupen64plus-video-glide64mk2 mgorny 15 Dec 2013
games-emulation/mupen64plus-ui-console mgorny 15 Dec 2013
games-emulation/m64py mgorny 15 Dec 2013
games-arcade/mrrescue hasufell 15 Dec 2013
app-admin/eselect-metasploit zerochaos 15 Dec 2013
dev-ruby/pcaprub zerochaos 15 Dec 2013
dev-ruby/sdoc zerochaos 15 Dec 2013
dev-ruby/packetfu zerochaos 15 Dec 2013
dev-ruby/rjb zerochaos 15 Dec 2013
dev-embedded/cpik rafaelmartins 15 Dec 2013
sec-policy/selinux-rngd swift 16 Dec 2013
net-misc/ssh-chain ottxor 18 Dec 2013
kde-base/libkomparediff2 johu 18 Dec 2013
x11-apps/radeontop tomwij 19 Dec 2013
net-mail/amavis-logwatch mjo 20 Dec 2013
perl-core/CPAN zlogene 21 Dec 2013
games-util/lutris hasufell 22 Dec 2013
dev-java/logback ercpe 23 Dec 2013
dev-ruby/rails-observers graaff 24 Dec 2013
dev-python/argcomplete jlec 24 Dec 2013
net-misc/gnome-online-miners pacho 24 Dec 2013
media-sound/gnome-music pacho 24 Dec 2013
sci-geosciences/gnome-maps pacho 24 Dec 2013
gnome-extra/gnome-boxes pacho 24 Dec 2013
dev-ruby/github_api graaff 25 Dec 2013
dev-ruby/permutation naota 25 Dec 2013
dev-perl/Sys-Mmap dilfridge 25 Dec 2013
dev-embedded/pikdev rafaelmartins 25 Dec 2013
dev-ruby/watch naota 25 Dec 2013
games-sports/dustrac hasufell 26 Dec 2013
dev-ruby/redis mrueg 26 Dec 2013
dev-ruby/json_pure naota 27 Dec 2013
dev-util/freecode-submit radhermit 27 Dec 2013
dev-ruby/dbf graaff 27 Dec 2013
app-leechcraft/lc-scroblibre maksbotan 27 Dec 2013
app-antivirus/clamav-unofficial-sigs mjo 27 Dec 2013
net-analyzer/speedtest-cli zx2c4 27 Dec 2013
net-p2p/datacoin-hp blueness 28 Dec 2013
dev-db/wxsqlite3 jlec 28 Dec 2013
dev-vcs/cvs-fast-export slyfox 28 Dec 2013
sec-policy/selinux-mandb swift 29 Dec 2013
dev-util/qbs pesa 29 Dec 2013

Bugzilla

The Gentoo community uses Bugzilla to record and track bugs, notifications, suggestions and other interactions with the development team.

Activity

The following tables and charts summarize the activity on Bugzilla between 29 November 2013 and 29 December 2013. Not fixed means bugs that were resolved as NEEDINFO, WONTFIX, CANTFIX, INVALID or UPSTREAM.
gmn-activity-2013-12

Bug Activity Number
New 1810
Closed 1160
Not fixed 231
Duplicates 158
Total 5291
Blocker 5
Critical 16
Major 68

Closed bug ranking

The developers and teams who have closed the most bugs during this period are as follows.

Rank Team/Developer Bug Count
1 Gentoo Security 84
2 Perl Devs @ Gentoo 66
3 Gentoo's Team for Core System packages 41
4 Gentoo Games 36
5 Gentoo Linux Gnome Desktop Team 35
6 Robin Johnson 29
7 Gentoo KDE team 27
8 Sven Vermeulen 25
9 Gentoo Ruby Team 24
10 Others 792

gmn-closed-2013-12

Assigned bug ranking

The developers and teams who have been assigned the most bugs during this period are as follows.

Rank Team/Developer Bug Count
1 Gentoo Linux bug wranglers 118
2 Gentoo Security 91
3 Perl Devs @ Gentoo 87
4 Gentoo Linux Gnome Desktop Team 68
5 Python Gentoo Team 64
6 Gentoo's Team for Core System packages 58
7 Gentoo KDE team 49
8 Default Assignee for Orphaned Packages 36
9 Gentoo Games 36
10 Others 1202

gmn-opened-2013-12

Tip of the month

Search packages in Portage by regular expressions:
#emerge -s "%^python$"

Getting Involved?

Interested in helping out? The GMN relies on volunteers and members of the community for content every month. If you are interested in writing for the GMN or thinking of another way to contribute, please send an e-mail to gmn@gentoo.org.

December 29, 2013
Sven Vermeulen a.k.a. swift (homepage, bugs)
Upgrading old Gentoo installations (December 29, 2013, 12:18 UTC)

Today I got “pinged” on bug #463240 about the difficulty of upgrading a Gentoo Linux deployment after a long time of inactivity on the system. We already have an Upgrading Gentoo article on the Gentoo wiki that describes in great detail how upgrades can be accomplished. But one of the methods I personally suggest most is to do small, incremental upgrades.

Say you have a system from early 2009. Not too long ago, but also not that recent anymore. If you would upgrade that system using the regular approach, your system would probably be using a non-existing profile (the /etc/make.profile symlink would point to a non-existing location), and if you switch the profile to an existing one, you might have to deal with problems like the profile requiring certain features (or EAPI version) that the software currently available on your system doesn’t support.

This problem is mentioned in the upgrade guide through the following:

Make sure your Portage is updated before performing any profile changes.

However, it does not tell how to update Portage. In my opinion the best way forward is to install an older Portage tree snapshot (somewhat more recent than your own deployment) and upgrade at least portage, perhaps also the packages belonging to the system set. So for a system that has not been updated since January 2009, you might want to try the portage tree snapshot of July 2009, then January 2010, then July 2010, etc. until you have a recent deployment again.

All that is left for you to do is to find such a snapshot (as the Portage tree snapshots from the mirrors are cleaned out after a few months). I try to keep a set of Portage tree snapshots available with a 2-month period which should be sufficient for most users to gradually upgrade their systems.

Considering I’ve used this method a few times already I’m going to add them to the upgrading instructions as well.

December 28, 2013
Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)
foomatic is moving into cups-filters (December 28, 2013, 17:58 UTC)

Part of the printing packages on Gentoo are very similar to, err, well, a dumpster on a hot summer afternoon. Nobody really wants to lift the lid and look inside, and beware of poking around in it... version numbers starting with 2007 and self-generated distfiles that noone knows how to regenerate. Yes I'm trying but so far my insights are limited.
What news is happening? The foomatic filters, an awesome printer driver system, originally supported not just CUPS but also LPRng, LPD, and many other prehistoric printing systems. Turns out, every backend except cups is now unmaintained. As a result, after a Google summer of code project, the actual filter binary is now upstream moving into cups-filters, to have a more active development and be perfectly integrated into the CUPS printing system. (Yes this means foomatic support for LPRng will not see further updates and may have to go away one day in the future due to bitrot.) For Gentoo this means that newer versions of net-print/cups-filters cannot be installed together with net-print/foomatic-filters but replace that package.
As a result we get to today's call for testers, preferably ~arch users: If you feel adventurous and if you are using foomatic-filters together with cups, please keyword newest cups-filters, remove foomatic-filters from your world file, and upgrade:

echo '=net-print/cups-filters-1* **' > /etc/portage/package.keywords/cupsfilters
emerge --deselect net-print/foomatic-filters
emerge -uDNav world
If this leads to blockers with one of the following packages, you might want to upgrade that to newest ~arch or re-emerge it first:  net-print/foo2zjs, net-print/foomatic-db-engine, net-print/foomatic-filters-ppds, net-print/hplip, net-print/pnm2ppa
Then, test if your printers still work! Any feedback is appreciated, either on a tracking bug or here in the comments section.

December 26, 2013
Rafael Goncalves Martins a.k.a. rafaelmartins (homepage, bugs)
Gentoo: News for PIC developers (December 26, 2013, 03:31 UTC)

Some weeks ago I decided to restart working with PIC microcontrollers, just for fun, and bought some electronic components, tools, etc. After having everything in hands, I started looking at the state of the PIC development tools in Gentoo, and found some outdated packages. I updated the packages I wanted to use (dev-embedded/gputils and dev-embedded/gpsim; dev-embedded/sdcc still needs some work), and added some other packages (dev-embedded/cpik, a C compiler for PIC 18F, and re-added dev-embedded/pikdev, a simple graphic IDE for the development of PIC-based applications, that was previously removed due to the usage of kdelibs3, and now is a Qt4-only application).

I'll be putting some effort on packaging the MPLAB X IDE and the XC8 compiler in the next weeks, if permitted by their licenses. I'm not sure yet.

That's all for now.

Thanks.

December 24, 2013
Alexys Jacob a.k.a. ultrabug (homepage, bugs)
rsyslog v7.4.7 (December 24, 2013, 13:33 UTC)

I’ve been slacking on this package for some months but slowly got the chance to catch-up with the multiple bugs open. So today I’m glad to say to make this blog post not only for a version bump release but also for some nice added features and ebuild improvements fixing 6 bugs in a row.

ebuild

  • added support for the mongoDB output template module thanks to Vadim Kuznetsov
  • added support for sub-slot operators for json-c and libgcrypt dependencies using EAPI5 thanks to Thomas D.
  • libgcrypt is now a required dependency of rsyslog as I didn’t want to add another USE flag for such a widely spread dependency
  • I’ve also bumped net-libs/czmq with approval of @jlec which fixes dependencies when trying to build rsyslog with zeromq support, thanks to Allen Parker for his report and valuable debugging

rsyslog v7.4.7

This is a bugfix release, see the full changelog here.