Gentoo Logo
Gentoo Logo Side
Gentoo Spaceship

Contributors:
. Aaron W. Swenson
. Agostino Sarubbo
. Alexey Shvetsov
. Alexis Ballier
. Alexys Jacob
. Alice Ferrazzi
. Andreas K. Hüttel
. Anthony Basile
. Arun Raghavan
. Bernard Cafarelli
. Brian Harring
. Christian Ruppert
. Chí-Thanh Christopher Nguyễn
. Denis Dupeyron
. Detlev Casanova
. Diego E. Pettenò
. Domen Kožar
. Doug Goldstein
. Eray Aslan
. Erik Mackdanz
. Fabio Erculiani
. Gentoo Haskell Herd
. Gentoo Miniconf 2016
. Gentoo Monthly Newsletter
. Gentoo News
. Gilles Dartiguelongue
. Greg KH
. Göktürk Yüksek
. Hanno Böck
. Hans de Graaff
. Ian Whyman
. Jan Kundrát
. Jason A. Donenfeld
. Jeffrey Gardner
. Joachim Bartosik
. Johannes Huber
. Jonathan Callen
. Jorge Manuel B. S. Vicetto
. Kristian Fiskerstrand
. Liam McLoughlin
. Luca Barbato
. Marek Szuba
. Mart Raudsepp
. Matt Turner
. Matthew Thode
. Michael Palimaka
. Michal Hrusecky
. Michał Górny
. Mike Doty
. Mike Gilbert
. Mike Pagano
. Nathan Zachary
. Pacho Ramos
. Patrick Kursawe
. Patrick Lauer
. Patrick McLean
. Paweł Hajdan, Jr.
. Petteri Räty
. Piotr Jaroszyński
. Rafael G. Martins
. Remi Cardona
. Richard Freeman
. Robin Johnson
. Sean Amoss
. Sebastian Pipping
. Steev Klimaszewski
. Sven Vermeulen
. Sven Wegener
. Tom Wijsman
. Tomáš Chvátal
. Yury German
. Zack Medico

Last updated:
March 25, 2017, 00:03 UTC

Disclaimer:
Views expressed in the content published here do not necessarily represent the views of Gentoo Linux or the Gentoo Foundation.


Bugs? Comments? Suggestions? Contact us!

Powered by:
Planet Venus

Welcome to Planet Gentoo, an aggregation of Gentoo-related weblog articles written by Gentoo developers. For a broader range of topics, you might be interested in Gentoo Universe.

March 21, 2017
Jason A. Donenfeld a.k.a. zx2c4 (homepage, bugs)
WireGuard in Google Summer of Code (March 21, 2017, 18:52 UTC)

WireGuard is participating in Google Summer of Code 2017. If you're a student who would like to be funded this summer for writing interesting kernel code, studying cryptography, building networks, or working on a wide variety of interesting problems, then this might be appealing. The program opened to students on March 20th. If you're applying for WireGuard, choose "Linux Foundation" and state in your proposal that you'd like to work on WireGuard with "Jason Donenfeld" as your mentor.

March 17, 2017
Michał Górny a.k.a. mgorny (homepage, bugs)

You should know already that you are not supposed to rely on Portage internals in ebuilds — all variables, functions and helpers that are not defined by the PMS. You probably know that you are not supposed to touch various configuration files, vdb and other Portage files as well. What most people don’t seem to understand, you are not supposed to make any assumptions about the ebuild repository either. In this post, I will expand on this and try to explain why.

What PMS specifies, what you can rely on

I think the first confusing point is that PMS actually defines the repository format pretty thoroughly. However, it does not specify that you can rely on that format being visible from within ebuild environment. It just defines a few interfaces that you can reliably use, some of them in fact quite consistent with the repository layout.

You should really look as the PMS-defined repository format as an input specification. This is the format that the developers are supposed to use when writing ebuilds, and that all basic tools are supposed to support. However, it does not prevent the package managers from defining and using other package formats, as long as they provide the environment compliant with the PMS.

In fact, this is how binary packages are implemented in Gentoo. The PMS does not define any specific format for them. It only defines a few basic rules and facilities, and both Portage and Paludis implement their own binary package formats. The package managers expose APIs required by the PMS, and can use them to run the necessary pkg_* phases.

However, the problem is not limited to two currently used binary package formats. This is a generic goal of being able to define any new package format in the future, and make it work out of the box with existing ebuilds. Imagine just a few possibilities: more compact repository formats (i.e. not requiring hundreds of unpacked files), fetching only needed ebuild files…

Sadly, none of this can even start being implemented if developers continuosly insist to rely on specific repository layout.

The *DIR variables

Let’s get into the details and iterate over the few relevant variables here.

First of all, FILESDIR. This is the directory where ebuild support files are provided throughout src_* phases. However, there is no guarantee that this will be exactly the directory you created in the ebuild repository. The package manager just needs to provide the files in some directory, and this directory may not actually exist before the first src_* phase. This implies that the support files may not even exist at all when installing from a binary package, and may be created (copied, unpacked) later when doing a source build.

The next variable listed by the PMS is DISTDIR. While this variable is somewhat similar to the previous one, some developers are actually eager to make the opposite assumption. Once again, the package manager may provide the path to any directory that contains the downloaded files. This may be a ‘shadow’ directory containing only files for this package, or it can be any system downloads directory containing lots of other files. Once again, you can’t assume that DISTDIR will exist before src_*, and that it will exist at all (and contain necessary files) when the build is performed using a binary package.

The two remaining variables I would like to discuss are PORTDIR and ECLASSDIR. Those two are a cause of real mayhem: they are completely unsuited for a multi-repository layout modern package managers use and they enforce a particular source repository layout (they are not available outside src_* phases). They pretty much block any effort on improvement, and sadly their removal is continuously blocked by a few short-sighted developers. Nevertheless, work on removing them is in progress.

Environment saving

While we’re discussing those matters, a short note on environment saving is worth being written. By environment saving we usually mean the magic that causes the variables set in one phase function to be carried to a phase function following it, possibly over a disjoint sequence of actions (i.e. install followed by uninstall).

A common misunderstanding is to assume the Portage model of environment saving — i.e. basically dumping a whole ebuild environment including functions into a file. However, this is not sanctioned by the PMS. The rules require the package manager to save only variables, and only those that are not defined in global scope. If phase functions define functions, there is no guarantee that those functions will be preserved or restored. If phases redefine global variables, there is no guarantee that the redefinition will be preserved.

In fact, the specific wording used in the PMS allows a completely different implementation to be used. The package manager may just snapshot defined functions after processing the global scope, or even not snapshot them at all and instead re-read the ebuild (and re-inherit eclasses) every time the execution continues. In this case, any functions defined during phase function are lost.

Is there a future in this?

I hope this clears up all the misunderstandings on how to write ebuilds so that they will work reliably, both for source and binary builds. If those rules are followed, our users can finally start expecting some fun features to come. However, before that happens we need to fix the few existing violations — and for that to happen, we need a few developers to stop thinking only of their own convenience.

Marek Szuba a.k.a. marecki (homepage, bugs)
Gentoo Linux in a Docker container (March 17, 2017, 14:31 UTC)

I have been using Docker for ebuild development for quite a while and absolutely love it, mostly because how easy it is to manipulate filesystem state with it. Work on several separate ebuilds in parallel? Just spin up several containers. Clean up once I’m done? Happens automatically when I close the container. Come back to something later? One docker commit invocation and I’m done. I could of course do something similar with virtual machines (and indeed I have to for cross-platform work) – but for native amd64 is is extremely convenient.

There is, however, one catch. By default processes running in a Docker container are fairly restricted privilege-wise and the Gentoo sandbox uses ptrace(). Result? By default, certain ebuilds (sys-libs/glibc and dev-libs/gobject-introspection , to name just two) will fail to emerge. One can of course set FEATURES=”-sandbox -usersandbox” for such ebuilds but it is an absolute no-no for both new ebuilds and any stabilisation work.

In the past working around this issue required messing with Docker security policies, which at least I found rather awkward. Fortunately since version 1.13.0 there has been a considerably easier way – simply pass

--cap-add=SYS_PTRACE

to docker-run. Done! Sandbox can now use ptrace() to its heart’s content.

Big Fat Warning: The reason why by default Docker restricts CAP_SYS_PTRACE is that a malicious program can use ptrace() to break out of the container it runs in. Do not grant this capability to containers unless you know what you are doing. Seriously.

Unfortunately the above is not the end of the story because at least as of version 1.13.0, Docker does not allow to enhance the capabilities of a docker-build job. Why is this a problem? For my own work I use a custom image which extends somewhat the official gentoo/stage3-amd64-hardened . One of the things my Dockerfile does is rsync the Portage tree and update @world so that my image contains a fully up-to-date stage3 even when the official base image does not. You can guess what happens when Docker tries to emerge an ebuild requiring the sandbox to use ptrace()… and remember, one of the packages containing such ebuilds is sys-libs/glibc . To my current knowledge the only way around this is to spin up a ptrace-enabled container using the latest good intermediate image left behind by docker-build and execute the remaining build steps manually. Not fun… Hope they will fix this some day.

 

Possibly the simplest way of changing the passhprase protecting a SSH key imported into gpg-agent is to use the Assuan passwd command:

echo passwd foo | gpg-connect-agent

where foo is the keygrip of your SSH key, which one can obtain from the file $GNUPGHOME/sshcontrol [1]. So far so good – but how does one know which of the keys listed in that file is the right one, especially if your sshcontrol list is fairly long? Here are the options I am aware of at this point:

Use the key comment. If you remember the contents of the comment field of the SSH key in question you can simply grep for it in all the files stored in $GNUPGHOME/private-keys-v1.d/ . Take the name of the file that matches, strip .key from the end and you’re set! Note that these are binary files so make sure your grep variant does not skip over them.

Use the MD5 fingerprint and the key comment. If for some reason you would rather not do the above you can take advantage of the fact that for SSH keys imported into gpg-agent the normal way, each keygrip line in sshcontrol is preceded by comment lines containing, among other things, the MD5 fingerprint of the imported key. Just tell ssh-add to print MD5 fingerprints for keys known to the agent instead of the default SHA256 ones:

ssh-add -E md5 -l

locate the fingerprint corresponding to the relevant key comment, then find the corresponding keygrip in sshcontrol .

Use the MD5 fingerprint and the public key. A slightly more complex variant of the above can be used if your SSH key pair in question has no comment but you still have the public key lying around. Start by running

ssh-add -L

and note the number of the line in which the public key in question shows up. The output of ssh-add -L and ssh-add -l is in the same order so you should have no trouble locating the corresponding MD5 fingerprint.

Bottom line: use meaningful comments for your SSH keys. It can really simplify key management in the long run.

[1] https://lists.gnupg.org/pipermail/gnupg-users/2007-July/031482.html

March 08, 2017
Marek Szuba a.k.a. marecki (homepage, bugs)
Hello world! (March 08, 2017, 02:12 UTC)

Welcome to Gentoo Blogs. This is your first post. Edit or delete it, then start blogging!

March 06, 2017
Sven Vermeulen a.k.a. swift (homepage, bugs)
Handling certificates in Gentoo Linux (March 06, 2017, 21:20 UTC)

I recently created a new article on the Gentoo Wiki titled Certificates which talks about how to handle certificate stores on Gentoo Linux. The write-up of the article (which might still change name later, because it does not handle everything about certificates, mostly how to handle certificate stores) was inspired by the observation that I had to adjust the certificate stores of both Chromium and Firefox separately, even though they both use NSS.

Certificates?

Well, when a secure communication is established from a browser to a site (or any other interaction that uses SSL/TLS, but let's stay with the browser example for now) part of the exchange is to ensure that the target site is actually the site it claims to be. Don't want someone else to trick you into giving your e-mail credentials do you?

To establish this, the certificate presented by the remote site is validated (alongside other handshake steps). A certificate contains a public key, as well as information about what the certificate can be used for, and who (or what) the certificate represents. In case of a site, the identification is (or should be) tied to the fully qualified domain name.

Of course, everyone could create a certificate for accounts.google.com and try to trick you into leaving your credentials. So, part of the validation of a certificate is to verify that it is signed by a third party that you trust to only sign certificates that are trustworthy. And to validate this signature, you hence need the certificate of this third party as well.

So, what about this certificate? Well, turns out, this one is also often signed by another certificate, and so on, until you reach the "top" of the certificate tree. This top certificate is called the "root certificate". And because we still have to establish that this certificate is trustworthy, we need another way to accomplish this.

Enter certificate stores

The root certificates of these trusted third parties (well, let us call them "Certificate Authorities" from now onward, because they sometimes will lose your trust) need to be reachable by the browser. The location where they are stored in is (often) called the truststore (a naming that I came across when dealing with Java and which stuck).

So, what I wanted to accomplish was to remove a particular CA certificate from the certificate store. I assumed that, because Chromium and Firefox both use NSS as the library to support their cryptographic uses, they would also both use the store location at ~/.pki/nssdb. That was wrong.

Another assumption I had was that NSS also uses the /etc/pki/nssdb location as a system-wide one. Wrong again (not that NSS doesn't allow this, but it seems that it is very much up to, and often ignored by, the NSS-implementing applications).

Oh, and I also assumed that there wouldn't be a hard-coded list in the application. Yup. Wrong again.

How NSS tracks root CA

Basically, NSS has a hard-coded root CA list inside the libnssckbi.so file. On Gentoo, this file is provided by the dev-libs/nss package. Because it is hard-coded, it seemed like there was little I could do to remove it, yet still through the user interfaces offered by Firefox and Chromium I was able to remove the trust bits from the certificate.

Turns out that Firefox (inside ~/.mozilla/firefox/*.default) and Chromium (inside ~/.pki/nssdb) store the (modified) trust bits for those locations, so that the hardcoded list does not need to be altered if all I want to do was revoke the trust on a specific CA. And it isn't that this hard-coded list is a bad list: Mozilla has a CA Certificate Program which controls the CAs that are accepted inside this store.

Still, I find it sad that the system-wide location (at /etc/pki/nssdb) is not by default used as well (or I have something wrong on my system that makes it so). On a multi-user system, administrators who want to have some control over the certificate stores might need to either use login scripts to manipulate the user certificate stores, or adapt the user files directly currently.

February 28, 2017
Denis Dupeyron a.k.a. calchan (homepage, bugs)
Gentoo is accepted to GSoC 2017 (February 28, 2017, 00:07 UTC)

There was good news in my mailbox today. The Gentoo Foundation was accepted to be a mentor organization for Google Summer of Code 2017!

What this means is we need you as a mentor, backup mentor or expert mentor. Whether you are a Gentoo developer and have done GSoC before does not matter at this point.

A mentor is somebody who will help during the selection of students, and will mentor a student during the summer. This should take at most one hour of your time on weekdays when student actually work on their project. What’s in it for you, you ask? A pretty exclusive Google T-shirt, a minion who does things you wouldn’t have the time or energy to do, but most importantly gratification and a lot of fun.

Backup mentors are for when the primary mentor of a student becomes unavailable for an extended period, typically for medical or family reasons. It rarely happens but it does happen. But a backup mentor can also be an experienced mentor (i.e., have done it at least once) who assists a primary mentor who is doing it for the first time.

Expert mentors have a very specific knowledge and are contacted on an as-needed basis to help with technical decisions.

You can be any combination of all that. However, our immediate need in the coming weeks is for people (again, not necessarily mentors or devs) who will help us evaluate student proposals.

If you’re a student, it’s the right time now to start thinking about what project idea you would want to work on during the summer. You can find ideas on our dedicated page, or you can come up with yours (these are the best!). One note though: you are going to be working on this full-time (i.e., 8 hours a day, we don’t allow for another even part-time job next to GSoC, although we do accommodate students who have a limited amount of classes or exams) for 3 months, so make sure your idea can keep you busy for this long. Whether you pick one of our ideas or come up with yours, it is strongly recommended to start discussing it with us on IRC.

As usual, we’d love to chat with you or answer your questions in #gentoo-soc on Freenode IRC. Make sure you stay long enough in the channel and give us enough time to respond to you. We are all volunteers and can’t maintain a 24/7 presence. It can take up to a few hours for one of us to see your request.

February 18, 2017
Sebastian Pipping a.k.a. sping (homepage, bugs)

Hi!

Just a quick tip on how to easily create a Fedora chroot environment from (even a non-Fedora) Linux distribution.

I am going to show the process on Debian stretch but it’s not be much different elsewhere.

Since I am going to leverage pip/PyPI, I need it available — that and a few non-Python widespread dependencies:

# apt install python-pip db-util lsb-release rpm yum
# pip install image-bootstrap pychroot

Now for the actual chroot creation, process and usage is very close to debootstrap of Debian:

# directory-bootstrap fedora --release 25 /var/lib/fedora_25_chroot

Done. Now let’s prove we have actual Fedora 25 in there. For lsb_release we need package redhat-lsb here, but the chroot was is functional before that already.

# pychroot /var/lib/fedora_25_chroot dnf -y install redhat-lsb
# pychroot /var/lib/fedora_25_chroot lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch:[..]:printing-4.1-noarch
Distributor ID: Fedora
Description:    Fedora release 25 (Twenty Five)
Release:        25
Codename:       TwentyFive

Note the use of pychroot which does bind mounts of /dev and friends out of the box, mainly.

directory-bootstrap is part of image-bootstrap and, besides Fedora, also supports creation of chroots for Arch Linux and Gentoo.

See you 🙂

February 09, 2017
Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)

So, FOSDEM 2017 is over, and as every year it was both fun and interesting. There will for sure be more blog posts, e.g., with photographs from talks by our developers, the booth, the annual Gentoo dinner, or (obviously) the beer event. The Gentoo booth, centrally located just opposite to KDE and Gnome and directly next to CoreOS, was quite popular; it's always great to hear from all the enthusiastic Gentoo fans. Many visitors also prepared, compiled, and installed their own Gentoo buttons at our button machine.
In addition we had a new Gentoo LiveDVD as handout - the "Crispy Belgian Waffle" FOSDEM 2017 edition. For those of you who couldn't make it to Brussels, you can still get it! Download the ISO here and burn it on a DVD or copy it on a USB stick - all done. Many thanks to Fernando Reyes (likewhoa) for all his work!
Finally, for those who are wondering, the "Gentoo Ecosystem" poster from our table can be downloaded as PDF here. It is based on work by Daniel Robbins and mitzip from Funtoo; the source files are available on Github. Of course this poster is continous work in progress, so tell me if you find something missing!

Matthew Thode a.k.a. prometheanfire (homepage, bugs)
Gentoo at Fosdem (February 09, 2017, 06:00 UTC)

At the stand

It was nice to meet everyone and hang out as well. There was an interview with Hacker Public Radio which you can find HERE as well.

Just a short one this time, but it was nice to meet everyone.

February 07, 2017
Sven Vermeulen a.k.a. swift (homepage, bugs)
I missed FOSDEM (February 07, 2017, 16:06 UTC)

I sadly had to miss out on the FOSDEM event. The entire weekend was filled with me being apathetic, feverish and overall zombie-like. Yes, sickness can be cruel. It wasn't until today that I had the energy back to fire up my laptop.

Sorry for the crew that I promised to meet at FOSDEM. I'll make it up, somehow.

February 06, 2017
Bernard Cafarelli a.k.a. voyageur (homepage, bugs)

Tesseract is one of the best open-source OCR software available, and I recently took over ebuilds maintainership for it. Current development is still quite active, and since last stable release they added a new OCR engine based on LSTM neural networks. This engine is available in an alpha release, and initial numbers show a much faster OCR pass, with fewer errors.

Sounds interesting? If you want to try it, this alpha release is now in tree (along with a live ebuild). I insist on the alpha tag, this is for testing, not for production; so the ebuild masked by default, and you will have to add to your package.unmask file:
=app-text/tesseract-4.00.00_alpha*
The ebuild also includes some additional changes, like current documentation generated with USE=doc (available in stable release too), and updated linguas.

Testing with paperwork

The initial reason I took over tesseract is that I also maintain paperwork ebuilds, a personal document manager, to handle scanned documents and PDFs (which is heavy tesseract user). It recently got a new 1.1 release, if you want to give it a try!

Denis Dupeyron a.k.a. calchan (homepage, bugs)
Google Summer of Code 2017 is starting! (February 06, 2017, 01:53 UTC)

(A previous version of this post recommended #gentoo-soc-mentors on Freenode as the preferred discussion channel for GSoC, please use #gentoo-soc instead as the former is invite-only or ask us to invite you to it)

It’s time to send us your GSoC ideas whether you can/want to mentor or not. We need as many good ideas as possible to make sure Google will select us as an organization again this year. Experience has shown us that we’re not automatically selected. You can submit them yourself on the wiki or let us do it. Don’t waste any time because some polishing typically needs to occur before the deadline (February 27th). You can discuss your ideas with us on Freenode in #gentoo-soc (preferred), or by email at soc-mentors@gentoo.org.

If you’re potentially interested in being a mentor, only want to help during the early phases of discussing and reviewing projects, or are just curious and want to see what goes on there, please let us know and we’ll add you to the mail alias. Everybody from last year was removed so don’t assume you’ll be on the alias because you were last year. Note that you do not have to be a Gentoo developer to be a mentor or help us with GSoC in any way.

Finally, if you’re a student it’s not quite time yet to ask us about projects. Please be patient, we’ll let you know.

Now go and submit that idea!

February 03, 2017
Nathan Zachary a.k.a. nathanzachary (homepage, bugs)

Important!

My tech articles—especially Linux ones—are some of the most-viewed on The Z-Issue. If this one has helped you, please consider a small donation to The Parker Fund by using the top widget at the right. Thanks!

Recently I was on a mission to make my audio experience on my main desktop more enjoyable. I had previously just used some older Bose AE2 headphones from 2010 plugged in directly to the 3.5mm audio output on the back of my desktop. The sound quality was mediocre at best, and I knew that a combination of a Digital-to-Analogue Converter (DAC) and some better headphones would certainly improve the experience. I also knew that the DAC would probably yield the most noticeable improvements, so I purchased the Big Ego USB DAC from one of my favourite audiophile-grade manufacturers, Emotiva. I have several of their monoblock amplifiers and use their amazing XMC1 for my preamp/processor in my home audio system, so I knew that the quality would be outstanding, especially for the price.

Emotiva Big Ego DAC and V-Moda Crossfade M-100 headphones

Now, the Big Ego FAQ on the Emotiva website indicates that it should work with all modern computing devices:

Q: What devices can I use the Ego DACs with?
A: The Ego DACs are basically designed to work with any modern “computer device” which can be used
with an external USB sound card, which includes:
1) All modern Apple computers
2) All modern Windows computers (Windows XP, Vista, 7, 8.0, 8.1, and Windows 10)
3) Many Linux computers (as long as they support USB Audio Class 1 or 2)
4) Some Android tablets and phones (as long as they support UAC1 or UAC2)
5) Apple iPhone 5 and iPhone 6 (with the lightning to USB camera adapter)

For many Linux users, the Big Ego probably works without any manual intervention. However, if it doesn’t, it shouldn’t be that difficult to get it working properly, and I hope that this guide helps if you are running into trouble.

Firstly, let’s get something out of the way, and that’s USB Audio Class 2 (UAC2) support within Linux. With all modern distributions (>=2.6 kernel), UAC2 is readily available. It can be validated by looking at the audio-v2.h file within the kernel source:

# grep 'From the USB Audio' /usr/src/linux/include/linux/usb/audio-v2.h
* From the USB Audio spec v2.0:

Feel free to look at the full file to see the references to the UAC2 specification.

Kernel support:

Secondly, and also speaking to the kernel, if your distribution doesn’t even show the device, you are likely lacking the one needed kernel driver. To see if your system recognises the Emotiva Big Ego, try the following command and look for similar output:

$ lsusb -v | grep 'Emotiva Big Ego'
...
iProduct 3 Emotiva Big Ego
...

The full identifier (Vendor ID and Product ID) from lsusb is 20ee:0021, even though it doesn’t have a description:

# grep -A 4 /var/log/messages
kernel: usb 9-1: New USB device found, idVendor=20ee, idProduct=0021
kernel: usb 9-1: New USB device strings: Mfr=1, Product=3, SerialNumber=2
kernel: usb 9-1: Product: Emotiva Big Ego
kernel: usb 9-1: Manufacturer: Emotiva

$ lsusb | grep '20ee:0021'
Bus 009 Device 005: ID 20ee:0021

If you don’t get similar output, then you’re lacking kernel support for the Big Ego. The one driver in the kernel that you need is the “USB Audio/MIDI driver” which can be found in the make menuconfig hierarchy as:

Device Drivers --->
  <*> Sound card support --->
    <*> Advanced Linux Sound Architecture --->
      [*] USB sound devices --->
        <*> USB Audio/MIDI driver

You can also check your kernel .config for it, or if you have it as a module, load it:

$ grep -i snd_usb_audio /usr/src/linux/.config
CONFIG_SND_USB_AUDIO=y

OR

# modprobe snd-usb-audio

Emotiva Big Ego DAC and V-Moda Crossfade M-100 headphones

ALSA configurations:

Thirdly, and now that you have the appropriate kernel support, let’s move on to configuring and using the Big Ego with ALSA. You can see a list of device names by using aplay -l, and it’s best to address the device by name instead of number (because the numbering could possibly change upon reboot). This one-liner should show you precisely how it is named (note that your output may be different based on the available sound output devices on your system):

$ aplay -l | awk -F \: '/,/{print $2}' | awk '{print $1}' | uniq
Intel
NVidia
Ego

With that information, you are ready to set the Big Ego as your default sound output device by editing either .asoundrc (in your home directory, for a per-user directive) or within the system-wide /etc/asound.conf (which is the one that I would recommend for most situations). I tried various configurations for my ALSA configuration, but would end up with various oddities. For instance, I ran into a problem where I had sound in applications like Audacious, mpv, and even ALSA’s own speaker-test, but had no sound in other terminal applications like ogg123 or, more importantly, web browsers like Firefox and Chromium. The only configuration that worked fully for me was:

$ cat /etc/asound.conf
defaults.pcm.!card Ego
defaults.pcm.!device 0
defaults.ctl.!card Ego
defaults.ctl.!device 0

After changing your ALSA configuration, you need to reload it, and the method for doing so varies based on your distribution and init system. For me, using Gentoo Linux with OpenRC, I just issued, (as root), /etc/init.d/alsasound restart and it reloaded. Worst case, just reboot your system to test the changes.

Now that you have it set as the default card, applications like alsamixer and such should automatically choose the Big Ego for your levels and mixing. One thing that I noticed with alsamixer is that there are two adjustable level sliders:

alsamixer with the Emotiva Big Ego USB DAC

What I am guessing is that, even though they are labelled “Emotiva Big Ego” and “Emotiva Big Ego 1”, they actually correspond to the output that you are using on the DAC. For instance, I am using the 3.5mm headphone jack on the front, and that corresponds to the “Emotiva Big Ego 1” slider, whereas if I were using the line out jack on the back of the DAC (those rhymes are fun 😛 ), I would adjust it using the slider for “Emotiva Big Ego”.

Additional configurations:

Now that we have configured ALSA to use our USB DAC as the default sound card, there are some additional things that I would like for my convenience. I prefer to not use a full desktop environment (DE), but instead favour a more minimalistic approach. I just use the Openbox window manager (WM). One of the things that I like about Openbox is the ability to set my own key bindings. In this case, I would like to be able to control the volume by using the designated keys on my keyboard, regardless of the application that is using the USB DAC. Here are my key bindings, which are added to ~/.config/openbox/rc.xml:


    <!-- Keybinding for increasing Emotiva Big Ego volume by 1 -->
    <keybind key="XF86AudioRaiseVolume">
      <action name="execute">
        <command>amixer set 'Emotiva Big Ego',1 1+</command>
      </action>
    </keybind>
    <!-- Keybinding for decreasing Emotiva Big Ego volume by 1 -->
    <keybind key="XF86AudioLowerVolume">
      <action name="execute">
        <command>amixer set 'Emotiva Big Ego',1 1-</command>
      </action>
    </keybind>
    <!-- Keybinding for muting/unmuting volume -->
    <keybind key="XF86AudioMute">
      <action name="execute">
        <command>amixer set 'Emotiva Big Ego',1 toggle</command>
      </action>
    </keybind>

Take note that the subdevice is ‘1’ (bold in the code above). That is because, like I showed in the alsamixer output, I’m using the headphone jack (so it corresponds to the secondary volume slider).

Further troubleshooting:

I hope that these instructions help you get your USB DAC working under Linux, but if they don’t, feel free to leave me a comment here. We’ll see what we can do to get it working for you. One last note is that I experienced some rather severe popping and other undesirable sounds when I had the Big Ego plugged into one of the USB2 ports on the back of my tower. Swapping it to its own non-shared USB3 port fixed that problem. So, if you have it plugged into a USB hub or something similar, try isolating it. Remember, it is a sensitive piece of audio equipment, and special considerations may need to be made. 🙂

Cheers,
Zach

February 02, 2017

FOSDEM 2017 logo

As FOSDEM 2017 approaches we are happy to announce there are a total of five Gentoo developers scheduled to give talks!

Developers and their talks include:

Only a few hours remain until the event kicks off. See you at FOSDEM!

February 01, 2017

Description:
pax-utils is a set of tools that check files for security relevant properties.

A fuzz on scanelf exposed an out-of bound read. It was reported to vapier which fixed the issue immediately.
Unfortunately I can’t get a symbolized ASan stacktrace, so I will show only the useful part of both asan and gdb.

# scanelf -s '*' -axetrnibSDIYZB $FILE
==32758==ERROR: AddressSanitizer: unknown-crash on address 0x7f8f9fa252dc at pc 0x00000053c6a0 bp 0x7ffe93a19910 sp 0x7ffe93a19908 
READ of size 4 at 0x7f8f9fa252dc thread T0                                                                                                                                                                                                                                      
   #0 0x53c69f  (/usr/bin/scanelf+0x53c69f) 
   #1 0x51d649  (/usr/bin/scanelf+0x51d649) 
   #2 0x51b97e  (/usr/bin/scanelf+0x51b97e) 
   #3 0x51ad43  (/usr/bin/scanelf+0x51ad43) 
   #4 0x51922e  (/usr/bin/scanelf+0x51922e) 
   #5 0x7f8f9e7fd61f  (/lib64/libc.so.6+0x2061f) 
   #6 0x41a008  (/usr/bin/scanelf+0x41a008) 

(gdb) bt
#8  0x000000000053c6a0 in scanelf_file_get_symtabs (elf=, sym=0x7fffffffcc00, str=0x7fffffffcc20) at scanelf.c:357
#9  0x000000000051d64a in scanelf_file_sym (elf=0x60700000de60, found_sym=) at scanelf.c:1327
#10 scanelf_elfobj (elf=) at scanelf.c:1547
#11 0x000000000051b97f in scanelf_elf (filename=0x7fffffffe50e "1.crashes", fd=, len=) at scanelf.c:1612
#12 scanelf_fileat (dir_fd=, filename=, st_cache=) at scanelf.c:1679
#13 0x000000000051ad44 in scanelf_dirat (dir_fd=, path=) at scanelf.c:1713
#14 0x000000000051922f in scanelf_dir (path=) at scanelf.c:1763
#15 parseargs (argc=5, argv=0x7fffffffe258) at scanelf.c:2273
#16 main (argc=5, argv=) at scanelf.c:2361

Affected version:
1.2

Fixed version:
1.2.1

Commit fix:
https://github.com/gentoo/pax-utils/commit/95e5489534ac9e9324c5096286899b688e19ae00

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
N/A

Reproducer:
https://github.com/asarubbo/poc/blob/master/00131-pax-utils-scanelf-oobread-scanelf_file_get_symtabs

Timeline:
2017-01-23: bug discovered and reported to upstream
2017-01-24: upstream realeased a patch and 1.2.1
2017-02-01: blog post about the issue

Note:
This bug was found with American Fuzzy Lop.
I’d suggest to go to 1.2.2 because of a functionality bug(s) in 1.2.1

Permalink:
https://blogs.gentoo.org/ago/2017/02/01/pax-utils-scanelf-out-of-bounds-read-in-scanelf_file_get_symtabs-scanelf-c

Description:
pax-utils is a set of tools that check files for security relevant properties.

A fuzz on scanelf exposed an out-of bound read. It was reported to vapier which fixed the issue immediately.
Unfortunately I can’t get a symbolized ASan stacktrace, so I will show only the useful part of both asan and gdb.

# scanelf -s '*' -axetrnibSDIYZB $FILE
==1853==ERROR: AddressSanitizer: unknown-crash on address 0x7f4099d25008 at pc 0x00000053586e bp 0x7fff335cb8b0 sp 0x7fff335cb8a8
READ of size 8 at 0x7f4099d25008 thread T0
    #0 0x53586d  (/usr/bin/scanelf+0x53586d)
    #1 0x51f526  (/usr/bin/scanelf+0x51f526)
    #2 0x51b97e  (/usr/bin/scanelf+0x51b97e)
    #3 0x51ad43  (/usr/bin/scanelf+0x51ad43)
    #4 0x51922e  (/usr/bin/scanelf+0x51922e)
    #5 0x7f4098afd61f  (/lib64/libc.so.6+0x2061f)
    #6 0x41a008  (/usr/bin/scanelf+0x41a008) 

(gdb) bt
#8  0x000000000053586e in scanelf_file_textrel (elf=, found_textrel=) at scanelf.c:560
#9  0x000000000051f527 in scanelf_elfobj (elf=) at scanelf.c:1536
#10 0x000000000051b97f in scanelf_elf (filename=0x7fffffffe50e "/tmp/afl/scanelf/report/crashes/2.crashes", fd=, len=) at scanelf.c:1612
#11 scanelf_fileat (dir_fd=, filename=, st_cache=) at scanelf.c:1679
#12 0x000000000051ad44 in scanelf_dirat (dir_fd=, path=) at scanelf.c:1713
#13 0x000000000051922f in scanelf_dir (path=) at scanelf.c:1763
#14 parseargs (argc=5, argv=0x7fffffffe258) at scanelf.c:2273
#15 main (argc=5, argv=) at scanelf.c:2361

Affected version:
1.2

Fixed version:
1.2.1

Commit fix:
https://github.com/gentoo/pax-utils/commit/95e5489534ac9e9324c5096286899b688e19ae00

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
N/A

Reproducer:
https://github.com/asarubbo/poc/blob/master/00132-pax-utils-scanelf-oobread-scanelf_file_textrel

Timeline:
2017-01-23: bug discovered and reported to upstream
2017-01-24: upstream realeased a patch and 1.2.1
2017-02-01: blog post about the issue

Note:
This bug was found with American Fuzzy Lop.
I’d suggest to go to 1.2.2 because of a functionality bug(s) in 1.2.1

Permalink:
https://blogs.gentoo.org/ago/2017/02/01/pax-utils-scanelf-out-of-bounds-read-in-scanelf_file_textrel-scanelf-c

January 27, 2017
Yury German a.k.a. blueknight (homepage, bugs)
WordPress Blogs Maintenance (January 27, 2017, 22:12 UTC)

Changes for blogs.gentoo.org

With the update of the WordPress to 4.7.1 a few plug-ins have created instability to the platform.

We have disabled the WordPress Mobile Site Plugin and the Picasa Album update.

  • WordPress Mobile Site is causing all sorts of issues and an update just came out today. We will push the update and enable it for some testing.  if you were one of the users that was using it please let us know so that you can test it when we update it.
  • The Picasa Album is not working and is disabled pending updates.
If you have any questions please feel free to contact me on irc @blueknight

Alexys Jacob a.k.a. ultrabug (homepage, bugs)
py3status v3.4 (January 27, 2017, 09:29 UTC)

Another community driven and incredible update of py3status has been released !

Our contributor star for this release is without doubt @lasers who is showing some amazing energy with challenging ideas and some impressive modules QA clean ups !

Thanks a lot as usual to @tobes who is basically leading the development of py3status now days with me being in a merge button mode most of the time.

By looking at the issues and pull requests I can already say that the 3.5 release will be grand !

Highlights

  • support of python 3.6 thanks to @tobes
  • a major effort in modules standardization, almost all of them support the format parameter now thanks to @lasers
  • modules documentation has been cleaned up
  • new do_not_disturb module to toggle notifications, by @maximbaz
  • new rss_aggregator module to display your unread feed items, by @raspbeguy
  • whatsmyip module: added geolocation support using ip-api.com, by @vicyap with original code from @neutronst4r

See the full changelog here.

Thank you guys !

January 16, 2017

I don’t know if a news will be sent. A possibile data corruption was found on zlib 1.2.10.
Please update your zlib to 1.2.11 and make sure you restart all services that are linked to zlib (a reboot may be an easy way).

Gentoo bug:
https://bugs.gentoo.org/show_bug.cgi?id=605888

Upstream bug:
https://github.com/madler/zlib/issues/198

Upstream commit:
https://github.com/madler/zlib/commit/4c7c90768308587884fab6159d93a4695a5ab1f0</a

January 15, 2017
Gentoo at FOSDEM 2017 (January 15, 2017, 00:00 UTC)

FOSDEM 2017 logo

On February, 4th and 5th, Gentoo will be attending FOSDEM 2017 in Brussels, Belgium.

This year one of our own, Jason A Donenfeld (zx2c4), will be speaking on WireGuard: a next generation secure kernel network tunnel.

Similar to last year, the event will be hosted at Université libre de Bruxelles. Gentoo developers will be taking rotating shifts at the Gentoo stand with gadgets, swag, and a new 2017 LiveDVD. You can visit this wiki article to see which developer will be manning the stand when you drop by.

We are looking forward to seeing those in the community who have been hard at work on their quizzes!

December 31, 2016
Domen Kožar a.k.a. domen (homepage, bugs)
Reflecting on 2016 (December 31, 2016, 18:00 UTC)

Haven't blogged in 2016, but a lot has happened.

A quick summary of highlighted events:

2016 was a functional programming year as I've planned by end of 2015.

I greatly miss Python community and in that spirit, I've attended EuroPython 2016 and helped organize DragonSprint in Ljubljana. I don't think there's a place for me in OOP anymore, but I'll surely attend community events as nostalgia will kick in.

2017 seems extremely exciting, plans will unveil as I go, starting with some exciting news in January for Nix community.

December 22, 2016
Sven Vermeulen a.k.a. swift (homepage, bugs)
SELinux System Administration, 2nd Edition (December 22, 2016, 18:26 UTC)

While still working on a few other projects, one of the time consumers of the past half year (haven't you noticed? my blog was quite silent) has come to an end: the SELinux System Administration - Second Edition book is now available. With almost double the amount of pages and a serious update of the content, the book can now be bought either through Packt Publishing itself, or the various online bookstores such as Amazon.

With the holidays now approaching, I hope to be able to execute a few tasks within the Gentoo community (and of the Gentoo Foundation) and get back on track. Luckily, my absence was not jeopardizing the state of SELinux in Gentoo thanks to the efforts of Jason Zaman.

December 08, 2016
Mike Pagano a.k.a. mpagano (homepage, bugs)

Just a quick note that I am walking the patch for CVE-2016-8655 down the gentoo-sources kernels.

Yesterday, I released the following kernels with the patch backported:

sys-kernel/gentoo-sources-4.8.12-r1
sys-kernel/gentoo-sources-4.4.36
sys-kernel/gentoo-sources-4.1.36-r1

Updated: 12/08
Also patched:
sys-kernel/gentoo-sources-3.18.45-r1
sys-kernel/gentoo-sources-3.12.68-r1

Updated 12/09
sys-kernel/gentoo-sources-3.10.104-r1
sys-kernel/gentoo-sources-3.4.113-r1

Updated 12/11
sys-kernel/gentoo-sources-3.14.79-r1

If Alice does not get to the others before me, I will continue to walk down the versions until all of them are patched.

Done.

November 21, 2016
Alexys Jacob a.k.a. ultrabug (homepage, bugs)
py3status v3.3 (November 21, 2016, 12:40 UTC)

Ok I slacked by not posting for v3.1 and v3.2 and I should have since those previous versions were awesome and feature rich.

But v3.3 is another major milestone which was made possible by tremendous contributions from @tobes as usual and also greatly thanks to the hard work of @guiniol and @pferate who I’d like to mention and thank again !

Also, I’d like to mention that @tobes has become the first collaborator of the py3status project !

Instead of doing a changelog review, I’ll highlight some of the key features that got introduced and extended during those versions.

The py3 helper

Writing powerful py3status modules have never been so easy thanks to the py3 helper !

This magical object is added automatically to modules and provides a lot of useful methods to help normalize and enhance modules capabilities. This is a non exhaustive list of such methods:

  • format_units: to pretty format units (KB, MB etc)
  • notify_user: send a notification to the user
  • time_in: to handle module cache expiration easily
  • safe_format: use the extended formatter to handle the module’s output in a powerful way (see below)
  • check_commands: check if the listed commands are available on the system
  • command_run: execute the given command
  • command_output: execute the command and get its output
  • play_sound: sound notifications !

Powerful control over the modules’ output

Using the self.py3.safe_format helper will unleash a feature rich formatter that one can use to conditionally select the output of a module based on its content.

  • Square brackets [] can be used. The content of them will be removed from the output if there is no valid placeholder contained within. They can also be nested.
  • A pipe (vertical bar) | can be used to divide sections the first valid section only will be shown in the output.
  • A backslash \ can be used to escape a character eg \[ will show [ in the output.
  • \? is special and is used to provide extra commands to the format string, example \?color=#FF00FF. Multiple commands can be given using an ampersand & as a separator, example \?color=#FF00FF&show.
  • {<placeholder>} will be converted, or removed if it is None or empty. Formatting can also be applied to the placeholder eg {number:03.2f}.

Example format_string:

This will show artist - title if artist is present, title if title but no artist, and file if file is present but not artist or title.

"[[{artist} - ]{title}]|{file}"

More code and documentation tests

A lot of efforts have been put into py3status automated CI and feature testing allowing more confidence in the advanced features we develop while keeping a higher standard on code quality.

This is such as even modules’ docstrings are now tested for bad formatting 🙂

Colouring and thresholds

A special effort have been put in normalizing modules’ output colouring with the added refinement of normalized thresholds to give users more power over their output.

New modules, on and on !

  • new clock module to display multiple times and dates informations in a flexible way, by @tobes
  • new coin_balance module to display balances of diverse crypto-currencies, by Felix Morgner
  • new diskdata module to shows both usage data and IO data from disks, by @guiniol
  • new exchange_rate module to check for your favorite currency rates, by @tobes
  • new file_status module to check the presence of a file, by @ritze
  • new frame module to group and display multiple modules inline, by @tobes
  • new gpmdp module for Google Play Music Desktop Player by @Spirotot
  • new kdeconnector module to display information about Android devices, by @ritze
  • new mpris module to control MPRIS enabled music players, by @ritze
  • new net_iplist module to display interfaces and their IPv4 and IPv6 IP addresses, by @guiniol
  • new process_status module to check the presence of a process, by @ritze
  • new rainbow module to enlight your day, by @tobes
  • new tcp_status module to check for a given TCP port on a host, by @ritze

Changelog

The changelog is very big and the next 3.4 milestone is very promising with amazing new features giving you even more power over your i3bar, stay tuned !

Thank you contributors

Still a lot of new timer contributors which I take great pride in as I see it as py3status being an accessible project.

  • @btall
  • @chezstov
  • @coxley
  • Felix Morgner
  • Gabriel Féron
  • @guiniol
  • @inclementweather
  • @jakubjedelsky
  • Jan Mrázek
  • @m45t3r
  • Maxim Baz
  • @pferate
  • @ritze
  • @rixx
  • @Spirotot
  • @Stautob
  • @tjaartvdwalt
  • Yuli Khodorkovskiy
  • @ZeiP

November 19, 2016

Git bisect is absolutely powerful, but sometimes is more comfortable use emerge instead of compile the software outside the package manager.

That was my case with media-libs/jasper which I’m picking as example for this ‘howto’

So basically, you are running Gentoo, you can install a live ebuild (9999) and you want to find which commit id fixes an issue. Let’s see step-by-step what to do.

1) Clone the repository to obtain the commit ids and put them in a file

ago@blackgate ~ $ cd /tmp
ago@blackgate /tmp $ git clone https://github.com/mdadams/jasper.git
ago@blackgate /tmp $ cd jasper/
ago@blackgate /tmp/jasper $ git --no-pager log --format=%H > /tmp/commitlist.txt

The file should contain the git commit ids, for example:

883f85876a463019a16b6d38dd9afc022d1f07cf
de4e3953fd3ef9d539c5187b7988e8750b3d67c9
f9ccc661fd1094c8d1c3df38b51295677d268dbf

2) Use a simple script which runs emerge and the command you need to test.

#!/bin/bash
for COMMIT_ID in $( cat /tmp/commitlist.txt )
do
      echo "Testing with the commit id: "${COMMIT_ID}""
      EGIT_COMMIT="${COMMIT_ID}" emerge -q media-libs/jasper
      imginfo -f /tmp/myjpg.jpg
      echo -ne "\n\n\n"
done

With the EGIT_COMMIT variable from the git-* eclass, we can emerge the live ebuild at a specific commit id.
imginfo is in my case the command I need and then I print some blank lines to better separate the output of the commands and understand what is happening.

Now you need to wait and just check what is the output of the specified command.

SOME IMPORTANT NOTES:
– This howto looks to be valid when the project you are building is small; running this script with e.g. libreoffice will take months.
– This howto looks to be valid when you know that the problem is near to the commit master and will take few emerge cycles to found the issue.
– If you know that the problem is fixed e.g. a year ago, you can manually edit the commitlist.txt file and delete some recent ids, to have a specified and minor range of commits.

That’s all.

November 10, 2016
Alice Ferrazzi a.k.a. alicef (homepage, bugs)

Open Source Conference 2016 Tokyo

Many people came to the Gentoo booth,
mainly students and Open Source users
asking for Gentoo information.

We gave away around 200 flyers, and
many many stickers during the two days.

Unfortunately the sticker we ordered
from unixsticker had some SVG problem.

We had also in exposition some esoteric
enviroment like the IS01 sharp,
off course mounting Gentoo as Native and
as prefix.
Of course one of the first things we tried
was the 5 minutes long Gentoo sl command.



image from: @NTSC_J

We also had a Gentoo notebook
running wayland (the one in the middle).

It was an amazing event and I would
like to thanks everyone that came to
the Gentoo booth, everyone that helped
making the Gentoo booth and all the
amazing Gentoo community.

November 07, 2016
Denis Dupeyron a.k.a. calchan (homepage, bugs)
SCALE 15x CFP is closing soon (November 07, 2016, 04:07 UTC)

Just a quick reminder that the deadline for proposing a talk to SCALE 15x is on November 15th. More information, including topics of interest, is available on the SCALE website.

SCALE 15x is to be held on March 2-5, 2017 at the Pasadena Convention Center in Pasadena, California, near Los Angeles. This is the same venue as last year and is much nicer than the original one form the years before.

I’ll see you there.

November 06, 2016
Alice Ferrazzi a.k.a. alicef (homepage, bugs)
2016-10-01 Gentoo Study Meeting (November 06, 2016, 18:24 UTC)

Gentoo Study Meeting talking (English Summary):  
homepage: http://gentoo.connpass.com/event/40906/  
Live broadcast: https://www.youtube.com/watch?v=j0SzulXKFwI  

Introduction:  
    First Gentoo Study Meeting Tokyo with https://www.youtube.com/watch?v=j0SzulXKFwI  
    How to become Gentoo Developer introduction talk.  
        Developer:  
            Contributing Ebuild:  
                - sending Git pull request  
                - Searching for a Mentor on proxy-maint  
                - Asking in #gentoo-proxy-maint  
                - Using https://bugs.gentoo.org/  
        Non committer developer:  
            - Contributing in Gentoo projects, with work that 
              not need gentoo git repository access.  
            - Contributing in the wiki (Not with translation also if 
              translator need the wiki translator permisson)  
    How to get help in Japanese:  
        - #gentoo-ja Freenode  
        - http://gentoo.slack.com http://slackin.aliceinwire.net  
        - https://github.com/gentoojp/issues  
        - ?forum  
        - http://www.gentoo.jp   
        - Gentoo勉強会 (Gentoo Study Meeting)  
        - https://wiki.gentoo.org/wiki/Handbook:Main_Page/ja  
    Gentoo News update:  
        Talk about Future EAPI 7 ulm slide  
            Question: When new EAPI are released?  
                I think there is not a setted release date for EAPI  
            New feature  
                - Runtime-switchable useflag  
                - eqwarn  
            Banned  
                - dohtml  
                - package.provided in profiles  
                - DESTTREE and INSDESTTREE  
    Talk about presence of Gentoo booth at Open Source Conference 
    2016 Tokyo http://www.ospn.jp/osc2016-fall/ :  
        - Stickers http://www.unixstickers.com/stickers/linux_os_distribution_stickers/gentoo-linux-os-badge-sticker  
        ask to fondation:  
        - Banner  
            size and format  
        - Table cover  
            size and format  
    Presentation:  
        Presenter: Matsuu san  
        Slide: Isucon 6 http://isucon.net/  
            - Team tuning speed contest  
            - This time was tuning speed contest on azure.microsoft.com  
                Only distribution with company can give sapport on azure.  
                Debian have a third party company that is supporting azure.  
                Gentoo also need something similar.  
            - Is good to do past problem for have more score on isucon.  
                vagrant is nice to use for the doing previous problem  
            - Go language, varnish+ESI, mysql  
            - https://github.com/matsuu/kataribe access log 
              analyzer for isucon/tuning  
            - sshrchttps://github.com/Russell91/sshrc 
              bring your .bashrc, .vimrc, etc. with you when you ssh.  
            - Matsu san as been choosen for become staff at the 
              isucon presentation in the future.  
        Presenter: @tkshnt  
        Slide: Report on last update  
            - let's make Gentoo goods shop for Gentoo-JP  
                previous OSC item:  
                    - t-shirt (@matsuu, @naota)  
                    - stickers (@matsuu)  
                next item:  
                    - Gentoo Tenugui (手拭い) https://goo.gl/SVeDHQ  
                OSC booth:  
                    - presentation  
                    - flyer  
                Design repository:  
                    - Github  
                        - project management  
                        - simple file upload  
        Presenter: @d_aki  
        Slide: my chaotic /etc/portage  
            - package.use can become chaotic  
            - /var/lib/portage/world difficult to 
              remeber when you added something and why  
            - let's use package.use directory and put file 
              name about what you are installing  
            - not what but why you installed the package  
        Presenter: alicef  
        Slide: How to contribute on Gentoo Github  
            - recently Gentoo CVS repository as been converted to Git  
            - Using the Github mirror is possible to send pull request.  
            - Good point of pull request:  
                - Code comment and review from more than one developer  
                - fast way to send ebuild patch upstream  
                - QA automatic check   
            - Bad point of pull request:  
                - the review are open to see to everyone  
                - basic git knowledge is needed  
            When we clone Gentoo repository:   
                Use git clone --depth=50   
                for fast pull request with less log information  
                git clone and git clone --depth=50 time difference:  
                    http://paste.ubuntu.com/23259104/  
        Presenter: @usaturn  
        Slide: systemd-nspawn & btrfs  
            - On Gentoo using systemd-nspawn  
            btrfs:   
                - copy on write  
                - using subvolume we can make snapshot  
                - compress is possible  
                - cannot make swapfile  
            Systemd:   
                - unit is the process file manager  
                - using systemd stage 3 is simple to install  
                - not using syslog but journald  
                - network setting by networkd  
                - instead of cron there is timer  
                - instead of ntp there is systemd-timesyncd  
                - grub is not needed, instead systemd-boot 
                  (ex gummi boot) work as bootloader  
                - docker is not needed, instead systemd-nspawn 
                  using machinectl command (good for testing gentoo package)  
        Presenter: @naota344  
        Slide: automatically resolving conflict   
            Gentoo developer, btrfs, linux kernel, emacs, T-code  
            portage:  
                resolving conflict  
                    - when USE flag is needed it will ask to 
                      add a USE flag.  
                    - when circulation dependency is detected it will 
                      ask to remove a USE flag, for example  
                Why there is a conflict:  
                    - Before installing a new package, we 
                      have a package (for example perl-5.20) with all 
                      the the dependency package setted  
                    - when we are goin to update world and we get a 
                      new package update (for example perl-5.22),
                       also some dependency of perl-5.22 get new update  
                    - in this situation can happen that some dependency
                      linked 
                      to perl-5.20 get in conflict with perl-5.22  
                How can we fix such situation:  
                    - we have a option to add --reinstall-atoms="Y"
                      to emerge command (Y= name of the dependency
                      package that is causing problem)  
                    - this command will instead of just update the
                      package it will reinstall the package as if 
                      they are not installed, solving such dependency
                      problem.  
                Why package is anyway deciding to automatically 
                not fixing dependency  
                    maybe because trying to fix all the dependecy will not
                    work correctly  
                When portage have conflict for many package  
                    it became more complicated and we will have a command
                    similar to this:  
                    --reinstall-atoms="A B C D E F G H I L M N ..."  
                For solving such problem there is emerge --reinstall-atoms
                wrapper  
                    https://github.com/naota/emerge-wrapper  
                    - automatically fixing circle dependecy  
                    - showing dependency graph  
                    - there is also a function for make try the 
                      dependency graph in a container  
                    - emerge analyzer tool  


October 25, 2016
Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
Gentoo Miniconf 2016 (October 25, 2016, 18:02 UTC)

Gentoo Miniconf, Prague, October 2016

As I noted when I resurrected the blog, part of the reason why I managed to come back to “active duty” within Gentoo Linux is because Robin and Amy helped me set up my laptop and my staging servers for singing commits with GnuPG remotely.

And that happened because this year I finally managed to go to the Gentoo MiniConf hosted as part of LinuxDays in Prague, Czech Republic.

The conference track was fairly minimal; Robin gave us an update on the Foundation and on what Infra is doing — I’m really looking forward to the ability to send out changes for review, instead of having to pull and push Git directly. After spending three years using code reviews with a massive repository I feel I like it and want to see significantly more of it.

Ulrich gave us a nice presentation on the new features coming with EAPI 7, which together with Michal’s post on EAPI 6 made it significantly easier to pick up Gentoo again.

And of course, I managed to get my GnuPG key signed by some of the developers over there, so that there is proof that who’s committing those changes is really.

But the most important part for me has been seeing my colleagues again, and meeting the new ones. Hopefully this won’t be the last time I get to the Miniconf, although fitting this together with the rest of my work travel is not straightforward.

I’m hoping to be at 33C3 — I have a hotel reservation and flight tickets, but no ticket for the conference yet. If any of you, devs or users, is there, feel free to ping me over Twitter or something. I’ll probably be at FOSDEM next year too, although that is not a certain thing, because I might have some scheduling conflicts with ENIGMA (unless I can get Delta to give me the ticket I have in mind.)

So once again thank you for CVU and LinuxDays for hosting us, and hopefully see you all in the future!

October 17, 2016
Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
GnuPG Agent Forwarding with OpenPGP cards (October 17, 2016, 22:02 UTC)

Finally, after many months (a year?) absence, I’m officially back as a Gentoo Linux developer with proper tree access. I have not used my powers much yet, but I wanted to at least point out why it took me so long to make it possible for me to come back.

There are two main obstacles that I was facing, the first was that the manifest signing key needed to be replaced for a number of reasons, and I had no easy access to the smartcard with my main key which I’ve been using since 2010. Instead I set myself up with a separate key on a “token”: a SIM-sized OpenPGP card installed into a Gemalto fixed-card reader (IDBridge K30.) Unfortunately this key was not cross-signed (and still isn’t, but we’re fixing that.)

The other problem is that for many (yet not all) packages I worked on, I would work on a remote system, one of the containers in my “testing server”, which also host(ed) the tinderbox. This means that the signing needs to happen on the remote host, although the key cannot leave the smartcard on the local laptop. GPG forwarding is not very simple but it has sort-of-recently become possible without too much intrusion.

The first thing to know is that you really want GnuPG 2.1; this is because it makes your life significantly easier as the key management is handed over to the Agent in all cases, which means there is no need for the “stubs” of the private key to be generated in the remote home. The other improvement in GnuPG 2.1 is that there is better sockets’ handling: on systemd it uses the /run/user path, and in general it uses a standard named socket with no way to opt-out. It also allows you to define an extra socket that is allowed to issue signature requests, but not modify the card or secret keys, which is part of the defence in depth when allowing remote access to the key.

There are instructions which should make it easier to set up, but they don’t quite work the way I read them, in particular because they require a separate wrapper to set up the connection. Instead, together with Robin we managed to figure out how to make this work correctly with GnuPG 2.0. Of course, since that Sunday, GnuPG 2.1 was made stable, and so it stopped working, too.

So, without further ado, let’s see what is needed to get this to work correctly. In the following example we assume we have two hosts, “local” and “remote”; we’ll have to change ~/.gnupg/gpg-agent.conf and ~/.ssh/config on “local”, and /etc/ssh/sshd_config on “remote”.

The first step is to ask GPG Agent to listen to an “extra socket”, which is the restricted socket that we want to forward. We also want for it to keep the display information in memory, I’ll get to explain that towards the end.

# local:~/.gnupg/gpg-agent.conf

keep-display
extra-socket ~/.gnupg/S.gpg-agent.remote

This is particularly important for systemd users because the normal sockets would be in /run and so it’s a bit more complicated to forward them correctly.

Secondly, we need to ask OpenSSH to forward this Unix socket to the remote host; for this to work you need at least OpenSSH 6.7, but since that’s now quite old, we can be mostly safe to assume you are using that. Unlike GnuPG, SSH does not correctly expand tilde for home, so you’ll have to know the actual paths we want to write the unix at the right path.

# local:~/.ssh/config

Host remote
RemoteForward /home/remote-user/.gnupg/S.gpg-agent /home/local-user/.gnupg/S.gpg-agent.remote
ExitOnForwardFailure yes

Note that the paths need to be fully qualified and are in the order remote, local. The ExitOnForwardFailure option ensures that you don’t get a silent failure to listen to the socket and fight for an hour trying to figure out what’s going on. Yes, I had that problem. By the way, you can combine this just fine with the now not so unknown SSH tricks I spoke about nearly six years ago.

Now is the slightly trickier part. Unlike the original gpg-agent, OpenSSH will not clean up the socket when it’s closed, which means you need to make sure it gets overwritten. This is indeed the main logic behind the remote-gpg script that I linked earlier, and the reason for that is that the StreamLocalBindUnlink option, which seems like the most obvious parameter to set, does not behave like most people would expect it to.

The explanation for that is actually simple: as the name of the option says, this only works for local sockets. So if you’re using the LocalForward it works exactly as intended, but if you’re using RemoteForward (as we need in this case), the one on the client side is just going to be thoroughly ignored. Which means you need to do this instead:

# remote:/etc/sshd/config

StreamLocalBindUnlink yes

Note that this applies to all the requests. You could reduce the possibility of bugs by using the Match directive to reduce them to the single user you care about, but that’s left up to you as an exercise.

At this point, things should just work: GnuPG 2.1 will notice there is a socket already so it will not start up a new gpg-agent process, and it will still start up every other project that is needed. And since as I said the stubs are not needed, there is no need to use --card-edit or --card-status (which, by the way, would not be working anyway as they are forbidden by the extra socket.)

However, if you try at this point to sign anything, it’ll just fail because it does not know anything about the key; so before you use it, you need to fetch a copy of the public key for the key id you want to use:

gpg --recv-key ${yourkeyid}
gpg -u ${yourkeyid} --clearsign --stdin

(It will also work without -u if that’s the only key it knows about.)

So what is about keep-display in local:~/.gnupg/gpg-agent.conf? One of the issues I faced with Robin was gpg failing saying something about “file not found”, though obviously the file I was using was found. A bit of fiddling later found these problems:

  • before GnuPG 2.1 I would start up gpg-agent with the wrapper script I wrote, and so it would usually be started by one of my Konsole session;
  • most of the time the Konsole session with the agent would be dead by the time I went to SSH;
  • the PIN for the card has to be typed on the local machine, not remote, so the pinentry binary should always be started locally; but it would get (some of) the environment variables from the session in which gpg is running, which means the shell on “remote”;
  • using DISPLAY=:0 gpg would make it work fine as pinentry would be told to open the local display.

A bit of sniffing around the source code brought up that keep-display option, which essentially tells pinentry to ignore the session where gpg is running and only consider the DISPLAY variable when gpg-agent is started. This works for me, but it has a few drawbacks. It would not work correctly if you tried to use GnuPG out of the X11 session, and it would not work correctly if you have multiple X11 sessions (say through X11 forwarding.) I think this is fine.

There is another general drawback on this solution: if two clients connect to the same SSH server with the same user, the last one connecting is the one that actually gets to provide its gpg-agent. The other one will be silently overruled. I”m afraid there is no obvious way to fix this. The way OpenSSH itself handles this for the SSH Agent forwarding is to provide a randomly-named socket in /tmp, and set the environment variable to point at it. This would not work for GnuPG anymore because it now standardised the socket name, and removed support for passing it in environment variables.

October 16, 2016
Patrick Lauer a.k.a. bonsaikitten (homepage, bugs)
Fixing gtk behaviour (October 16, 2016, 13:26 UTC)

Recently I've noticed all gtk2 apps becoming quite ... what's the word ... derpy?
Things like scrollbars not working and stuff. And by "not working" I mean the gtk3 behaviour of not showing up/down arrows and being a grey smudge of stupid.

So accidentally I stumbled over an old gentoo bug where it was required to deviate from defaults to have, like, icons and stuff.
That sounds pretty reasonable to me, but with gtk upstream crippling the Ad-Waiter, err, adwaita theme, because gtk3, this is a pretty sad interaction. And unsurprisingly by switching to the upstream default theme, Raleigh, gtk2 apps start looking a lot better.(Like, scrollbars and stuff)

The change might make sense to apply to Gentoo globally, locally for each user it is simply:

$ cat ~/.gtkrc-2.0
gtk-theme-name = "Raleigh"
gtk-cursor-theme-name = "Raleigh"
I'm still experimenting with 'gtk-icon-theme-name' and 'gtk-fallback-icon-theme', maybe that should change too. And as a benefit we can remove the Ad-Waiter from dependencies, possibly drop gnome-themes too, and restore a fair amount of sanity to gtk2.

Changing console fontsize (October 16, 2016, 10:09 UTC)

Recently I accidentally aquired some "HiDPI" hardware. While it is awesome to use it quickly becomes irritating to be almost unable to read the bootup messages or work in a VT.
The documentation on fixing this is surprisingly sparse, but luckily it is very easy:

  • Get a font that comes in the required sizes. media-fonts/terminus-font was the first choice I found, there may be others that are nice to use. Since terminus works well enough I didn't bother to check.
  • Test the font with "setfont". The default path is /usr/share/consolefonts, and the font 'name' is just the filename without the .psf.gz suffix. If you break things you can revert to sane defaults by just calling "setfont" or rebooting the machine (ehehehehehe)
  • Set the font in /etc/conf.d/consolefont. For a 210dpi notebook display I chose 'ter-v24b', but I'm considering going down a font size or two, maybe 'ter-v20b'? It's all very subjective ...
  • On reboot the consolefont init script will set the required font.
Now I'm wondering if such fonts can be embedded into the kernel so that on boot it directly switches to a 'nice' font, but just being able to read the console output is a good start ...

October 13, 2016
Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
The end of an era, the end of the tinderbox (October 13, 2016, 19:02 UTC)

I’m partly sad, but for the most part this is a weight that goes away from my shoulders, so I can’t say I’m not at least in part joyful of it, even though the context in which this is happening is not exactly what I expected.

I turned off the Gentoo tinderbox, never to come back. The S3 storage of logs is still running, but I’ve asked Ian to see if he can attach everything at his pace, so I can turn off the account and be done with it.

Why did this happen? Well, it’s a long story. I already stopped running it for a few months because I got tired of Mike behaving like a child, like I already reported in 2012 by closing my bugs because the logs are linked (from S3) rather than attached. I already made my position clear that it’s a silly distinction as the logs will not disappear in the middle of nowhere (indeed I’ll keep the S3 bucket for them running until they are all attached to Bugzilla), but as he keeps insisting that it’s “trivial” to change the behaviour of the whole pipeline, I decided to give up.

Yes, it’s only one developer, and yes, lots of other developers took my side (thanks guys!), but it’s still aggravating to have somebody who can do whatever he likes without reporting to anybody, ignoring Council resolutions, QA (when I was the lead) and essentially using Gentoo as his personal playground. And the fact that only two people (Michał and Julian) have been pushing for a proper resolution is a bit disappointing.

I know it might feel like I’m taking my toys and going home — well, that’s what I’m doing. The tinderbox has been draining on my time (little) and my money (quite more), but those I was willing to part with — draining my motivation due to assholes in the project was not in the plans.

In the past six years that I’ve been working on this particular project, things evolved:

  • Originally, it was a simple chroot with a looping emerge, inspected with grep and Emacs, running on my desktop and intended to catch --as-needed failures. It went through lots of disks, and got me off XFS for good due to kernel panics.
  • It was moved to LXC, which is why the package entered the Gentoo tree, together with the OpenRC support and the first few crude hacks.
  • When I started spendig time in Los Angeles for a customer, Yamato under my desk got replaced with Excelsior which was crowdfounded and hosted, for two years straight, by my customer at the time.
  • This is where the rewrite happened, from attaching logs (which I could earlier do with more or less ease, thanks to NFS) to store them away and linking instead. This had to do mostly with the ability to remote-manage the tinderbox.
  • This year, since I no longer work for the company in Los Angeles, and instead I work in Dublin for a completely different company, I decided Excelsior was better off on a personal space, and rented a full 42 unit cabinet with Hurricane Electric in Fremont, where the server is still running as I type this.

You can see that it’s not that ’m trying to avoid spending time to engineer solutions. It’s just that I feel that what Mike is asking is unreasonable, and the way he’s asking it makes it unbearable. Especially when he feigns to care about my expenses — as I noted in the previously linked post, S3 is dirty cheap, and indeed it now comes down to $1/month given to Amazon for the logs storage and access, compared to $600/month to rent the cabinet at Hurricane.

Yes, it’s true that the server is not doing only tinderboxing – it also is running some fate instances, and I have been using it as a development server for my own projects, mostly open-source ones – but that’s the original use for it, and if it wasn’t for it I wouldn’t be paying so much to rent a cabinet, I’d be renting a single dedicated server off, say, Hetzner.

So here we go, the end of the era of my tinderbox. Patrick and Michael are still continuing their efforts so it’s not like Gentoo is left without integration test, but I’m afraid it’ll be harder for at least some of the maintainers who leveraged the tinderbox heavily in the past. My contract with Hurricane expires in April; at that point I’ll get the hardware out of the cabinet, and will decide what to do with it — it’s possible I’ll donate the server (minus harddrives) to Gentoo Foundation or someone else who can use it.

My involvement in Gentoo might also suffer from this; I hopefully will be dropping one of the servers I maintain off the net pretty soon, which will be one less system to build packages for, but I still have a few to take care of. For the moment I’m taking a break: I’ll soon send an email that it’s open season on my packages; I locked my bugzilla account already to avoid providing harsher responses in the bug linked at the top of this post.

I have been trying my best not to comment on systemd one way or another for a while. For the most part because I don’t want to have a trollfest on my blog, because moderating it is something I hate and I’m sure would be needed. On the other hand it seems like people start to bring me in the conversation now from time to time.

What I would like to point out at this point is that both extreme sides of the vision are, in my opinion, behaving childishly and being totally unprofessional. Whether it is name-calling of the people or the software, death threats, insults, satirical websites, labeling of 300 people for a handful of them, etc.

I don’t think I have been as happy to have a job that allows me not to care about open source as much as I did before as in the past few weeks as things keep escalating and escalating. You guys are the worst. And again I refer to both supporters and detractors, devs of systemd, devs of eudev, Debian devs and Gentoo devs, and so on so forth.

And the reason why I say this is because you both want to bring this to extremes that I think are totally uncalled for. I don’t see the world in black and white and I think I said that before. Gray is nuanced and interesting, and needs skills to navigate, so I understand it’s easier to just take a stand and never revise your opinion, but the easy way is not what I care about.

Myself, I decided to migrate my non-server systems to systemd a few months ago. It works fine. I’ve considered migrating my servers, and I decided for the moment to wait. The reason is technical for the most part: I don’t think I trust the stability promises for the moment and I don’t reboot servers that often anyway.

There are good things to the systemd design. And I’m sure that very few people will really miss sysvinit as is. Most people, especially in Gentoo, have not been using sysvinit properly, but rather through OpenRC, which shares more spirit with systemd than sysv, either by coincidence or because they are just the right approach to things (declarativeness to begin with).

At the same time, I don’t like Lennart’s approach on this to begin with, and I don’t think it’s uncalled for to criticize the product based on the person in this case, as the two are tightly coupled. I don’t like moderating people away from a discussion, because it just ends up making the discussion even more confrontational on the next forum you stumble across them — this is why I never blacklisted Ciaran and friends from my blog even after a group of them started pasting my face on pictures of nazi soldiers from WW2. Yes I agree that Gentoo has a good chunk of toxic supporters, I wish we got rid of them a long while ago.

At the same time, if somebody were to try to categorize me the same way as the people who decided to fork udev without even thinking of what they were doing, I would want to point out that I was reproaching them from day one for their absolutely insane (and inane) starting announcement and first few commits. And I have not been using it ever, since for the moment they seem to have made good on the promise of not making it impossible to run udev without systemd.

I don’t agree with the complete direction right now, and especially with the one-size-fit-all approach (on either side!) that tries to reduce the “software biodiversity”. At the same time there are a few designs that would be difficult for me to attack given that they were ideas of mine as well, at some point. Such as the runtime binary approach to hardware IDs (that Greg disagreed with at the time and then was implemented by systemd/udev), or the usage of tmpfs ACLs to allow users at the console to access devices — which was essentially my original proposal to get rid of pam_console (that played with owners instead, making it messy when having more than one user at console), when consolekit and its groups-fiddling was introduced (groups can be used for setgid, not a good idea).

So why am I posting this? Mostly to tell everybody out there that if you plan on using me for either side point to be brought home, you can forget about it. I’ll probably get pissed off enough to try to prove the exact opposite, and then back again.

Neither of you is perfectly right. You both make mistake. And you both are unprofessional. Try to grow up.

Edit: I mistyped eudev in the original article and it read euscan. Sorry Corentin, was thinking one thing and typing another.

New devbox running (October 13, 2016, 19:02 UTC)

I announced it in February that Excelsior, which ran the Tinderbox, was no longer at Hurricane Electric. I have also said I’ll start on working on a new generation Tinderbox, and to do that I need a new devbox, as the only three Gentoo systems I have at home are the laptops and my HTPC, not exactly hardware to run compilation all the freaking time.

So after thinking of options, I decided that it was much cheaper to just rent a single dedicated server, rather than a full cabinet, and after asking around for options I settled for Online.net, because of price and recommendation from friends. Unfortunately they do not support Gentoo as an operating system, which makes a few things a bit more complicated. They do provide you with a rescue system, based on Ubuntu, which is enough to do the install, but not everything is easy that way either.

Luckily, most of the configuration (but not all) was stored in Puppet — so I only had to rename the hosts there, changed the MAC addresses for the LAN and WAN interfaces (I use static naming of the interfaces as lan0 and wan0, which makes many other pieces of configuration much easier to deal with), changed the IP addresses, and so on. Unfortunately since I didn’t start setting up that machine through Puppet, it also meant that it did not carry all the information to replicate the system, so it required some iteration and fixing of the configuration. This also means that the next move is going to be easier.

The biggest problem has been setting up correctly the MDRAID partitions, because of GRUB2: if you didn’t know, grub2 has an automagic dependency on mdadm — if you don’t install it it won’t be able to install itself on a RAID device, even though it can detect it; the maintainer refused to add an USE flag for it, so you have to know about it.

Given what can and cannot be autodetected by the kernel, I had to fight a little more than usual and just gave up and rebuilt the two (/boot and / — yes laugh at me but when I installed Excelsior it was the only way to get GRUB2 not to throw up) arrays as metadata 0.90. But the problem was being able to tell what the boot up errors were, as I have no physical access to the device of course.

The Online.net server I rented is a Dell server, that comes with iDRAC for remote management (Dell’s own name for IPMI, essentially), and Online.net allows you to set up connections to through your browser, which is pretty neat — they use a pool of temporary IP addresses and they only authorize your own IP address to connect to them. On the other hand, they do not change the default certificates, which means you end up with the same untrustable Dell certificate every time.

From the iDRAC console you can’t do much, but you can start up the remove, JavaWS-based console, which reminded me of something. Unfortunately the JNLP file that you can download from iDRAC did not work on either Sun, Oracle or IcedTea JREs, segfaulting (no kidding) with an X.509 error log as last output — I seriously thought the problem was with the certificates until I decided to dig deeper and found this set of entries in the JNLP file:

 <resources os="Windows" arch="x86">
   <nativelib href="https://idracip/software/avctKVMIOWin32.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLWin32.jar" download="eager"/>
 </resources>
 <resources os="Windows" arch="amd64">
   <nativelib href="https://idracip/software/avctKVMIOWin64.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLWin64.jar" download="eager"/>
 </resources>
 <resources os="Windows" arch="x86_64">
   <nativelib href="https://idracip/software/avctKVMIOWin64.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLWin64.jar" download="eager"/>
 </resources>
  <resources os="Linux" arch="x86">
    <nativelib href="https://idracip/software/avctKVMIOLinux32.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLLinux32.jar" download="eager"/>
  </resources>
  <resources os="Linux" arch="i386">
    <nativelib href="https://idracip/software/avctKVMIOLinux32.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLLinux32.jar" download="eager"/>
  </resources>
  <resources os="Linux" arch="i586">
    <nativelib href="https://idracip/software/avctKVMIOLinux32.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLLinux32.jar" download="eager"/>
  </resources>
  <resources os="Linux" arch="i686">
    <nativelib href="https://idracip/software/avctKVMIOLinux32.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLLinux32.jar" download="eager"/>
  </resources>
  <resources os="Linux" arch="amd64">
    <nativelib href="https://idracip/software/avctKVMIOLinux64.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLLinux64.jar" download="eager"/>
  </resources>
  <resources os="Linux" arch="x86_64">
    <nativelib href="https://idracip/software/avctKVMIOLinux64.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLLinux64.jar" download="eager"/>
  </resources>
  <resources os="Mac OS X" arch="x86_64">
    <nativelib href="https://idracip/software/avctKVMIOMac64.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLMac64.jar" download="eager"/>
  </resources>

Turns out if you remove everything but the Linux/x86_64 option, it does fetch the right jar and execute the right code without segfaulting. Mysteries of Java Web Start I guess.

So after finally getting the system to boot, the next step is setting up networking — as I said I used Puppet to set up the addresses and everything, so I had working IPv4 at boot, but I had to fight a little longer to get IPv6 working. Indeed IPv6 configuration with servers, virtual and dedicated alike, is very much an unsolved problem. Not because there is no solution, but mostly because there are too many solutions — essentially every single hosting provider I ever used had a different way to set up IPv6 (including none at all in one case, so the only option was a tunnel) so it takes some fiddling around to set it up correctly.

To be honest, Online.net has a better set up than OVH or Hetzner, the latter being very flaky, and a more self-service one that Hurricane, which was very flexible, making it very easy to set up, but at the same time required me to just mail them if I wanted to make changes. They document for dibbler, as they rely on DHCPv6 with DUID for delegation — they give you a single /56 v6 net that you can then split up in subnets and delegate independently.

What DHCPv6 in this configuration does not give you is routing — which kinda make sense, as you can use RA (Route Advertisement) for it. Unfortunately at first I could not get it to work. Turns out that, since I use subnets for the containerized network, I enabled IPv6 forwarding, through Puppet of course. Turns out that Linux will ignore Route Advertisement packets when forwarding IPv6 unless you ask it nicely to — by setting accept_ra=2 as well. Yey!

Again this is the kind of problems that finding this information took much longer than it should have been; Linux does not really tell you that it’s ignoring RA packets, and it is by far not obvious that setting one sysctl will disable another — unless you go and look for it.

Luckily this was the last problem I had, after that the server was set up fine and I just had to finish configuring the domain’s zone file, and the reverse DNS and the SPF records… yes this is all the kind of trouble you go through if you don’t just run your whole infrastructure, or use fully cloud — which is why I don’t consider self-hosting a general solution.

What remained is just bits and pieces. The first was me realizing that Puppet does not remove the entries from /etc/fstab by default, so I noticed that the Gentoo default /etc/fstab file still contains the entries for CD-ROM drives as well as /dev/fd0. I don’t remember which was the last computer with a floppy disk drive that I used, let alone owned.

The other fun bit has been setting up the containers themselves — similarly to the server itself, they are set up with Puppet. Since the server used to be running a tinderbox, it used to also host a proper rsync mirror, it was just easier, but I didn’t want to repeat that here, and since I was unable to find a good mirror through mirrorselect (longer story), I configured Puppet to just provide to all the containers with distfiles.gentoo.org as their sync server, which did not work. Turns out that our default mirror address does not have any IPv6 hosts on it ­– when I asked Robin about it, it seems like we just don’t have any IPv6-hosted mirror that can handle that traffic, it is sad.

So anyway, I now have a new devbox and I’m trying to set up the rest of my repositories and access (I have not set up access to Gentoo’s repositories yet which is kind of the point here.) Hopefully this will also lead to more technical blogging in the next few weeks as I’m cutting down on the overwork to relax a bit.

What does #shellshock mean for Gentoo? (October 13, 2016, 19:02 UTC)

Gentoo Penguins with chicks at Jougla Point, Antarctica
Photo credit: Liam Quinn

This is going to be interesting as Planet Gentoo is currently unavailable as I write this. I’ll try to send this out further so that people know about it.

By now we have all been doing our best to update our laptops and servers to the new bash version so that we are safe from the big scare of the quarter, shellshock. I say laptop because the way the vulnerability can be exploited limits the impact considerably if you have a desktop or otherwise connect only to trusted networks.

What remains to be done is to figure out how to avoid this repeats. And that’s a difficult topic, because a 25 years old bug is not easy to avoid, especially because there are probably plenty of siblings of it around, that we have not found yet, just like this last week. But there are things that we can do as a whole environment to reduce the chances of problems like this to either happen or at least avoid that they escalate so quickly.

In this post I want to look into some things that Gentoo and its developers can do to make things better.

The first obvious thing is to figure out why /bin/sh for Gentoo is not dash or any other very limited shell such as BusyBox. The main answer lies in the init scripts that still use bashisms; this is not news, as I’ve pushed for that four years ago, while Roy insisted on it even before that. Interestingly enough, though, this excuse is getting less and less relevant thanks to systemd. It is indeed, among all the reasons, one I find very much good in Lennart’s design: we want declarative init systems, not imperative ones. Unfortunately, even systemd is not as declarative as it was originally supposed to be, so the init script problem is half unsolved — on the other hand, it does make things much easier, as you have to start afresh anyway.

If either all your init scripts are non-bash-requiring or you’re using systemd (like me on the laptops), then it’s mostly safe to switch to use dash as the provider for /bin/sh:

# emerge eselect-sh
# eselect sh set dash

That will change your /bin/sh and make it much less likely that you’d be vulnerable to this particular problem. Unfortunately as I said it’s mostly safe. I even found that some of the init scripts I wrote, that I checked with checkbashisms did not work as intended with dash, fixes are on their way. I also found that the lsb_release command, while not requiring bash itself, uses non-POSIX features, resulting in garbage on the output — this breaks facter-2 but not facter-1, I found out when it broke my Puppet setup.

Interestingly it would be simpler for me to use zsh, as then both the init script and lsb_release would have worked. Unfortunately when I tried doing that, Emacs tramp-mode froze when trying to open files, both with sshx and sudo modes. The same was true for using BusyBox, so I decided to just install dash everywhere and use that.

Unfortunately it does not mean you’ll be perfectly safe or that you can remove bash from your system. Especially in Gentoo, we have too many dependencies on it, the first being Portage of course, but eselect also qualifies. Of the two I’m actually more concerned about eselect: I have been saying this from the start, but designing such a major piece of software – that does not change that often – in bash sounds like insanity. I still think that is the case.

I think this is the main problem: in Gentoo especially, bash has always been considered a programming language. That’s bad. Not only because it only has one reference implementation, but it also seem to convince other people, new to coding, that it’s a good engineering practice. It is not. If you need to build something like eselect, you do it in Python, or Perl, or C, but not bash!

Gentoo is currently stagnating, and that’s hard to deny. I’ve stopped being active since I finally accepted stable employment – I’m almost thirty, it was time to stop playing around, I needed to make a living, even if I don’t really make a life – and QA has obviously taken a step back (I still have a non-working dev-python/imaging on my laptop). So trying to push for getting rid of bash in Gentoo altogether is not a good deal. On the other hand, even though it’s going to be probably too late to be relevant, I’ll push for having a Summer of Code next year to convert eselect to Python or something along those lines.

Myself, I decided that the current bashisms in the init scripts I rely upon on my servers are simple enough that dash will work, so I pushed that through puppet to all my servers. It should be enough, for the moment. I expect more scrutiny to be spent on dash, zsh, ksh and the other shells in the next few months as people migrate around, or decide that a 25 years old bug is enough to think twice about all of them, o I’ll keep my options open.

This is actually why I like software biodiversity: it allows to have options to select different options when one components fail, and that is what worries me the most with systemd right now. I also hope that showing how bad bash has been all this time with its closed development will make it possible to have a better syntax-compatible shell with a proper parser, even better with a proper librarised implementation. But that’s probably hoping too much.

TG4: Tinderbox Generation 4 (October 13, 2016, 19:02 UTC)

Everybody’s a critic: the first comment I received when I showed other Gentoo developers my previous post about the tinderbox was a question on whether I would be using pkgcore for the new generation tinderbox. If you have understood what my blog post was about, you probably understand why I was not happy about such a question.

I thought the blog post made it very clear that my focus right now is not to change the way the tinderbox runs but the way the reporting pipeline works. This is the same problem as 2009: generating build logs is easy, sifting through them is not. At first I thought this was hard just for me, but the fact that GSoC attracted multiple people interested in doing continuous build, but not one interested in logmining showed me this is just a hard problem.

The approach I took last time, with what I’ll start calling TG3 (Tinderbox Generation 3), was to: highlight the error/warning messages; provide a list of build logs for which a problem was identified (without caring much for which kind of problem), and just showing up broken builds or broken tests in the interface. This was easy to build up, and to a point to use, but it had a lots of drawbacks.

Major drawbacks in that UI is that it relies on manual work to identify open bugs for the package (and thus make sure not to report duplicate bugs), and on my own memory not to report the same issue multiple time, if the bug was closed by some child as NEEDINFO.

I don’t have my graphic tablet with me to draw a mock of what I have in mind yet, but I can throw in some of the things I’ve been thinking of:

  • Being able to tell what problem or problems a particular build is about. It’s easy to tell whether a build log is just a build failure or a test failure, but what if instead it has three or four different warning conditions? Being able to tell which ones have been found and having a single-click bug filing system would be a good start.
  • Keep in mind the bugs filed against a package. This is important because sometimes a build log is just a repeat of something filed already; it may be that it failed multiple times since you started a reporting run, so it might be better to show that easily.
  • Related, it should collapse failures for packages so not to repeat the same package multiple times on the page. Say you look at the build failures every day or two, you don’t care if the same package failed 20 times, especially if the logs report the same error. Finding out whether the error messages are the same is tricky, but at least you can collapse the multiple logs in a single log per package, so you don’t need to skip it over and over again.
  • Again related, it should keep track of which logs have been read and which weren’t. It’s going to be tricky if the app is made multi-user, but at least a starting point needs to be there.
  • It should show the three most recent bugs open for the package (and a count of how many other open bugs) so that if the bug was filed by someone else, it does not need to be filed again. Bonus points for showing the few most recently reported closed bugs too.

You can tell already that this is a considerably more complex interface than the one I used before. I expect it’ll take some work with JavaScript at the very least, so I may end up doing it with AngularJS and Go mostly because that’s what I need to learn at work as well, don’t get me started. At least I don’t expect I’ll be doing it in Polymer but I won’t exclude that just yet.

Why do I spend this much time thinking and talking (and soon writing) about UI? Because I think this is the current bottleneck to scale up the amount of analysis of Gentoo’s quality. Running a tinderbox is getting cheaper — there are plenty of dedicated server offers that are considerably cheaper than what I paid for hosting Excelsior, let alone the initial investment in it. And this is without going to look again at the possible costs of running them on GCE or AWS at request.

Three years ago, my choice of a physical server in my hands was easier to justify than now, with 4-core HT servers with 48GB of RAM starting at €40/month — while I/O is still the limiting factor, with that much RAM it’s well possible to have one tinderbox building fully in tmpfs, and just run a separate server for a second instance, rather than sharing multiple instances.

And even if GCE/AWS instances that are charged for time running are not exactly interesting for continuous build systems, having a cloud image that can be instructed to start running a tinderbox with a fixed set of packages, say all the reverse dependencies of libav, would make it possible to run explicit tests for code that is known to be fragile, while not pausing the main tinderbox.

Finally, there are different ideas of how we should be testing packages: all options enabled, all options disabled, multilib or not, hardened or not, one package at a time, all packages together… they can all share the same exact logmining pipeline, as all it needs is the emerge --info output, and the log itself, which can have markers for known issues to look out for or not. And then you can build the packages however you desire, as long as you can submit them there.

Now my idea is not to just build this for myself and run analysis over all the people who want to submit the build logs, because that would be just about as crazy. But I think it would be okay to have a shared instance for Gentoo developers to submit build logs from their own personal instances, if they want to, and then have them look at their own accounts only. It’s not going to be my first target but I’ll keep that in mind when I start my mocks and implementations, because I think it might prove successful.

When the Google Online Security blog announced earlier this week the general availability of Security Key, everybody at the office was thrilled, as we’ve been waiting for the day for a while. I’ve been using this for a while already, and my hope is for it to be easy enough for my mother and my sister, as well as my friends, to start using it.

While the promise is for a hassle-free second factor authenticator, it turns out it might not be as simple as originally intended, at least on Linux, at least right now.

Let’s start with the hardware, as there are four different options of hardware that you can choose from:

  • Yubico FIDO U2F which is a simple option only supporting the U2F protocol, no configuration needed;
  • Plug-up FIDO U2F which is a cheaper alternative for the same features — I have not witnessed whether it is as sturdy as the Yubico one, so I can’t vouch for it;
  • Yubikey NEO which provides multiple interface, including OTP (not usable together with U2F), OpenPGP and NFC;
  • Yubikey NEO-n the same as above, without NFC, and in a very tiny form factor designed to be left semi-permanently in a computer or laptop.

I got the NEO, but mostly to be used with LastPass ­– the NFC support allows you to have 2FA on the phone without having to type it back from a computer – and a NEO-n to leave installed on one of my computers. I already had a NEO from work to use as well. The NEO requires configuration, so I’ll get back at it in a moment.

The U2F devices are accessible via hidraw, a driverless access protocol for USB devices, originally intended for devices such as keyboards and mice but also leveraged by UPSes. What happen though is that you need access to the device, that the Linux kernel will make by default accessible only by root, for good reasons.

To make the device accessible to you, the user actually at the keyboard of the computer, you have to use udev rules, and those are, as always, not straightforward. My personal hacky choice is to make all the Yubico devices accessible — the main reason being that I don’t know all of the compatible USB Product IDs, as some of them are not really available to buy but come from instance from developer mode devices that I may or may not end up using.

If you’re using systemd with device ACLs (in Gentoo, that would be sys-apps/systemd with acl USE flag enabled), you can do it with a file as follows:

# /etc/udev/rules.d/90-u2f-securitykey.rules
ATTRS{idVendor}=="1050", TAG+="uaccess"
ATTRS{idVendor}=="2581", ATTRS{idProduct}=="f1d0", TAG+="uaccess"

If you’re not using systemd or ACLs, you can use the plugdev group and instead do it this way:

# /etc/udev/rules.d/90-u2f-securitykey.rules
ATTRS{idVendor}=="1050", GROUP="plugdev", MODE="0660"
ATTRS{idVendor}=="2581", ATTRS{idProduct}=="f1d0", GROUP="plugdev", MODE="0660"

-These rules do not include support for the Plug-up because I have no idea what their VID/PID pairs are, I asked Janne who got one so I can amend this later.- Edit: added the rules for the Plug-up device. Cute their use of f1d0 as device id.

Also note that there are properly less hacky solutions to get the ownership of the devices right, but I’ll leave it to the systemd devs to figure out how to include in the default ruleset.

These rules will not only allow your user to access /dev/hidraw0 but also to the /dev/bus/usb/* devices. This is intentional: Chrome (and Chromium, the open-source version works as well) use the U2F devices in two different modes: one is through a built-in extension that works with Google assets, and it accesses the low-level device as /dev/bus/usb/*, the other is through a Chrome extension which uses /dev/hidraw* and is meant to be used by all websites. The latter is the actually standardized specification and how you’re supposed to use it right now. I don’t know if the former workflow is going to be deprecated at some point, but I wouldn’t be surprised.

For those like me who bought the NEO devices, you’ll have to enable the U2F mode — while Yubico provides the linked step-by-step guide, it was not really completely correct for me on Gentoo, but it should be less complicated now: I packaged the app-crypt/yubikey-neo-manager app, which already brings in all the necessary software, including the latest version of app-crypt/ccid required to use the CCID interface on U2F-enabled NEOs. And if you already created the udev rules file as I noted above, it’ll work without you using root privileges. Just remember that if you are interested in the OpenPGP support you’ll need the pcscd service (it should auto-start with both OpenRC and systemd anyway).

I’ll recount separately the issues with packaging the software. In the mean time make sure you keep your accounts safe, and let’s all hope that more sites will start protecting your accounts with U2F — I’ll also write a separate opinion piece on why U2F is important and why it is better than OTP, this is just meant as documentation, howto set up the U2F devices on your Linux systems.

You might have seen the word TEXTREL thrown around security or hardening circles, or used in Gentoo Linux installation warnings, but one thing that is clear out there is that the documentation around this term is not very useful to understand why they are a problem. so I’ve been asked to write something about it.

Let’s start with taking apart the terminology. TEXTREL is jargon for “text relocation”, which is once again more jargon, as “text” in this case means “code portion of an executable file.” Indeed, in ELF files, the .text section is the one that contains all the actual machine code.

As for “relocation”, the term is related to dynamic loaders. It is the process of modifying the data loaded from the loaded file to suit its placement within memory. This might also require some explanation.

When you build code into executables, any named reference is translated into an address instead. This includes, among others, variables, functions, constants and labels — and also some unnamed references such as branch destinations on statements such as if and for.

These references fall into two main typologies: relative and absolute references. This is the easiest part to explain: a relative reference takes some address as “base” and then adds or subtracts from it. Indeed, many architectures have a “base register” which is used for relative references. In case of executable code, particularly with the reference to labels and branch destinations, relative references translate into relative jumps, which are relative to the current instruction pointer. An absolute reference is instead a fully qualified pointer to memory, well at least to the address space of the running process.

While absolute addresses are kinda obvious as a concept, they are not very practical for a compiler to emit in many cases. For instance, when building shared objects, there is no way for the compiler to know which addresses to use, particularly because a single process can load multiple objects, and they need to all be loaded at different addresses. So instead of writing to the file the actual final (unknown) address, what gets written by the compiler first – and by the link editor afterwards – is a placeholder. It might sound ironic, but an absolute reference is then emitted as a relative reference based upon the loading address of the object itself.

When the loader takes an object and loads it to memory, it’ll be mapped at a given “start” address. After that, the absolute references are inspected, and the relative placeholder resolved to the final absolute address. This is the process of relocation. Different types of relocation (or displacements) exists, but they are not the topic of this post.

Relocations as described up until now can apply to both data and code, but we single out code relocations as TEXTRELs. The reason for this is to be found in mitigation (or hardening) techniques. In particular, what is called W^X, NX or PaX. The basic idea of this technique is to disallow modification to executable areas of memory, by forcing the mapped pages to either be writable or executable, but not both (W^X reads “writable xor executable”.) This has a number of drawbacks, which are most clearly visible with JIT (Just-in-Time) compilation processes, including most JavaScript engines.

But beside JIT problem, there is the problem with relocations happening in code section of an executable, too. Since the relocations need to be written to, it is not feasible (or at least not easy) to provide an exclusive writeable or executable access to those. Well, there are theoretical ways to produce that result, but it complicates memory management significantly, so the short version is that generally speaking, TEXTRELs and W^X techniques don’t go well together.

This is further complicated by another mitigation strategy: ASLR, Address Space Layout Randomization. In particular, ASLR fully defeats prelinking as a strategy for dealing with TEXTRELs — theoretically on a system that allows TEXTREL but has the address space to map every single shared object at a fixed address, it would not be necessary to relocate at runtime. For stronger ASLR you also want to make sure that the executables themselves are mapped at different addresses, so you use PIE, Position Independent Executable, to make sure they don’t depend on a single stable loading address.

Usage of PIE was for a long while limited to a few select hardened distributions, such as Gentoo Hardened, but it’s getting more common, as ASLR is a fairly effective mitigation strategy even for binary distributions where otherwise function offsets would be known to an attacker.

At the same time, SELinux also implements protection against text relocation, so you no longer need to have a patched hardened kernel to provide this protection.

Similarly, Android 6 is now disallowing the generation of shared objects with text relocations, although I have no idea if binaries built to target this new SDK version gain any more protection at runtime, since it’s not really my area of expertise.

LibreSSL and the bundled libs hurdle (October 13, 2016, 19:02 UTC)

It was over five years ago that I ranted about the bundling of libraries and what that means for vulnerabilities found in those libraries. The world has, since, not really listened. RubyGems still keep insisting that “vendoring” gems is good, Go explicitly didn’t implement a concept of shared libraries, and let’s not even talk about Docker or OSv and their absolutism in static linking and bundling of the whole operating system, essentially.

It should have been obvious how this can be a problem when Heartbleed came out, bundled copies of OpenSSL would have needed separate updates from the system libraries. I guess lots of enterprise users of such software were saved only by the fact that most of the bundlers ended up using older versions of OpenSSL where heartbeat was not implemented at all.

Now that we’re talking about replacing the OpenSSL libraries with those coming from a different project, we’re going to be hit by both edges of the proprietary software sword: bundling and ABI compatibility, which will make things really interesting for everybody.

If you’ve seen my (short, incomplete) list of RAND_egd() users which I posted yesterday. While the tinderbox from which I took this is out of date and needs cleaning, it is a good starting point to figure out the trends, and as somebody already picked up, the bundling is actually strong.

Software that bundled Curl, or even Python, but then relied on the system copy of OpenSSL, will now be looking for RAND_egd() and thus fail. You could be unbundling these libraries, and then use a proper, patched copy of Curl from the system, where the usage of RAND_egd() has been removed, but then again, this is what I’ve been advocating forever or so. With caveats, in the case of Curl.

But now if the use of RAND_egd() is actually coming from the proprietary bits themselves, you’re stuck and you can’t use the new library: you either need to keep around an old copy of OpenSSL (which may be buggy and expose even more vulnerability) or you need a shim library that only provides ABI compatibility against the new LibreSSL-provided library — I’m still not sure why this particular trick is not employed more often, when the changes to a library are only at the interface level but still implements the same functionality.

Now the good news is that from the list that I produced, at least the egd functions never seemed to be popular among proprietary developers. This is expected as egd was vastly a way to implement the /dev/random semantics for non-Linux systems, while the proprietary software that we deal with, at least in the Linux world, can just accept the existence of the devices themselves. So the only problems have to do with unbundling (or replacing) Curl and possibly the Python SSL module. Doing so is not obvious though, as I see from the list that there are at least two copies of libcurl.so.3 which is the older ABI for Curl — although admittedly one is from the scratchbox SDKs which could just as easily be replaced with something less hacky.

Anyway, my current task is to clean up the tinderbox so that it’s in a working state, after which I plan to do a full build of all the reverse dependencies on OpenSSL, it’s very possible that there are more entries that should be in the list, since it was built with USE=gnutls globally to test for GnuTLS 3.0 when it came out.

October 11, 2016
Matthew Thode a.k.a. prometheanfire (homepage, bugs)
Openstack Newton Update (October 11, 2016, 05:00 UTC)

The short of it

Openstack Newton was packaged early last week (when rc2 was still going on upstream) and the tags for the major projects were packaged the day they released (nova and the like).

I've updated the openstack-meta package to 2016.2.9999 and would recommend people use that …

September 27, 2016
Sven Vermeulen a.k.a. swift (homepage, bugs)
We do not ship SELinux sandbox (September 27, 2016, 18:47 UTC)

A few days ago a vulnerability was reported in the SELinux sandbox user space utility. The utility is part of the policycoreutils package. Luckily, Gentoo's sys-apps/policycoreutils package is not vulnerable - and not because we were clairvoyant about this issue, but because we don't ship this utility.

September 22, 2016

Description:
mupdf is a lightweight PDF viewer and toolkit written in portable C.

A fuzzing though mutool revealed an infinite loop in gatherresourceinfo if mutool try to get info from a crafted pdf.

The output is a bit cutted because the original is ~1300 lines (because of the loop)

# mutool info $FILE
[cut here]
warning: not a font dict (0 0 R)
ASAN:DEADLYSIGNAL
=================================================================
==8763==ERROR: AddressSanitizer: stack-overflow on address 0x7ffeb34e6f6c (pc 0x7f188e685b2e bp 0x7ffeb34e7410 sp 0x7ffeb34e6ea0 T0)
    #0 0x7f188e685b2d in _IO_vfprintf /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/stdio-common/vfprintf.c:1266
    #1 0x7f188e6888c0 in buffered_vfprintf /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/stdio-common/vfprintf.c:2346
    #2 0x7f188e685cd4 in _IO_vfprintf /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/stdio-common/vfprintf.c:1292
    #3 0x49927f in __interceptor_vfprintf /var/tmp/portage/sys-devel/llvm-3.8.0-r3/work/llvm-3.8.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1111
    #4 0x499352 in fprintf /var/tmp/portage/sys-devel/llvm-3.8.0-r3/work/llvm-3.8.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1156
    #5 0x7f188f70f03c in fz_flush_warnings /var/tmp/portage/app-text/mupdf-1.9a/work/mupdf-1.9a/source/fitz/error.c:18:3
    #6 0x7f188f70f03c in fz_throw /var/tmp/portage/app-text/mupdf-1.9a/work/mupdf-1.9a/source/fitz/error.c:168
    #7 0x7f188fac98d5 in pdf_parse_ind_obj /var/tmp/portage/app-text/mupdf-1.9a/work/mupdf-1.9a/source/pdf/pdf-parse.c:565:3
    #8 0x7f188fb5fe6b in pdf_cache_object /var/tmp/portage/app-text/mupdf-1.9a/work/mupdf-1.9a/source/pdf/pdf-xref.c:2029:13
    #9 0x7f188fb658d2 in pdf_resolve_indirect /var/tmp/portage/app-text/mupdf-1.9a/work/mupdf-1.9a/source/pdf/pdf-xref.c:2155:12
    #10 0x7f188fbc0a0d in pdf_is_dict /var/tmp/portage/app-text/mupdf-1.9a/work/mupdf-1.9a/source/pdf/pdf-object.c:268:2
    #11 0x53ea6a in gatherfonts /var/tmp/portage/app-text/mupdf-1.9a/work/mupdf-1.9a/source/tools/pdfinfo.c:257:8
    #12 0x53ea6a in gatherresourceinfo /var/tmp/portage/app-text/mupdf-1.9a/work/mupdf-1.9a/source/tools/pdfinfo.c:595
    #13 0x53f31b in gatherresourceinfo /var/tmp/portage/app-text/mupdf-1.9a/work/mupdf-1.9a/source/tools/pdfinfo.c:603:5
    [cut here]
    #253 0x53f31b in gatherresourceinfo /var/tmp/portage/app-text/mupdf-1.9a/work/mupdf-1.9a/source/tools/pdfinfo.c:603:5

SUMMARY: AddressSanitizer: stack-overflow /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/stdio-common/vfprintf.c:1266 in _IO_vfprintf
==8763==ABORTING
1152.crashes:
PDF-1.4
Pages: 1
Retrieving info from pages 1-1...

Affected version:
1.9a

Fixed version:
N/A

Commit fix:
http://git.ghostscript.com/?p=mupdf.git;h=fdf71862fe929b4560e9f632d775c50313d6ef02

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

Timeline:
2016-08-05: bug discovered
2016-08-05: bug reported to upstream
2016-09-22: upstream released a patch
2016-09-22: blog post about the issue

Note:
This bug was found with American Fuzzy Lop.

Permalink:

mupdf: mutool: stack-based buffer overflow after an infinite loop in gatherresourceinfo (pdfinfo.c)

September 11, 2016
Gentoo Miniconf 2016 a.k.a. miniconf-2016 (homepage, bugs)
4 Weeks Left to Gentoo Miniconf (September 11, 2016, 17:50 UTC)

4 weeks are left until LinuxDays and Gentoo Miniconf 2016 in Prague.

Are you excited to see Gentoo in action? Here is something to look forward to:

  • Gentoo amd64-fbsd on an ASRock E350M1
  • Gentoo arm64 on a Raspberry Pi 3 (yes, running a 64 bit kernel and userland) with systemd
  • Gentoo mipsel on a MIPS Creator CI20 with musl libc
  • Gentoo arm on an Orange Pi PC, with Clang as system compiler

dsc00015

September 07, 2016

Description:
Graphicsmagick is an Image Processing System.

A fuzzing revealed a NULL pointer access in the TIFF parser.

The complete ASan output:

# gm identify $FILE
ASAN:DEADLYSIGNAL
=================================================================
==19028==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7fbd36dd6c3c bp 0x7ffe3c007090 sp 0x7ffe3c006d10 T0)
    #0 0x7fbd36dd6c3b in MagickStrlCpy /var/tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/utility.c:4567:3
    #1 0x7fbd2b714c26 in ReadTIFFImage /var/tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/coders/tiff.c:2048:13
    #2 0x7fbd36ac506a in ReadImage /var/tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/constitute.c:1607:13
    #3 0x7fbd36ac46ac in PingImage /var/tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/constitute.c:1370:9
    #4 0x7fbd36a165a0 in IdentifyImageCommand /var/tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/command.c:8372:17
    #5 0x7fbd36a1affb in MagickCommand /var/tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/command.c:8862:17
    #6 0x7fbd36a6fee3 in GMCommandSingle /var/tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/command.c:17370:10
    #7 0x7fbd36a6eb78 in GMCommand /var/tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/command.c:17423:16
    #8 0x7fbd3597761f in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/csu/libc-start.c:289
    #9 0x4188d8 in _init (/usr/bin/gm+0x4188d8)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /var/tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/utility.c:4567:3 in MagickStrlCpy
==19028==ABORTING

Affected version:
1.3.24 (and maybe past)

Fixed version:
1.3.25

Commit fix:
http://hg.code.sf.net/p/graphicsmagick/code/rev/eb58028dacf5

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
N/A

Timeline:
2016-08-17: bug discovered
2016-08-18: bug reported privately to upstream
2016-08-19: upstream released a patch
2016-09-05: upstream released 1.3.25
2016-09-07: blog post about the issue

Note:
This bug was found with American Fuzzy Lop.

Permalink:

graphicsmagick: NULL pointer dereference in MagickStrlCpy (utility.c)

September 06, 2016

Description:
ettercap is a comprehensive suite for man in the middle attacks.

Etterlog, which is part of the package, fails to read malformed data produced from the fuzzer and then it overflows.

Since there are three issues, to make it short, I’m cutting a bit the ASan output.

# etterlog $FILE
==10077==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60e00000dac0 at pc 0x00000047b8cf bp 0x7ffdc6850580 sp 0x7ffdc684fd30                                                      
READ of size 43780 at 0x60e00000dac0 thread T0                                                                                                                                                 
    #0 0x47b8ce in memcmp /var/tmp/temp/portage/sys-devel/llvm-3.8.0-r2/work/llvm-3.8.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:418            
    #1 0x50e26e in profile_add_info /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_profiles.c:106:12                                                             
    #2 0x4f3d3a in create_hosts_list /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_analyze.c:128:7                                                              
    #3 0x4fcf0c in display_info /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_display.c:218:4                                                                   
    #4 0x4fcf0c in display /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_display.c:52                                                                           
    #5 0x507818 in main /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_main.c:94:4                                                                               
    #6 0x7faa8656161f in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/csu/libc-start.c:289                                                                        
    #7 0x41a408 in _start (/usr/bin/etterlog+0x41a408)                                                                                                                                         

0x60e00000dac0 is located 0 bytes to the right of 160-byte region [0x60e00000da20,0x60e00000dac0)                                                                                              
allocated by thread T0 here:                                                                                                                                                                   
    #0 0x4c1ae0 in calloc /var/tmp/temp/portage/sys-devel/llvm-3.8.0-r2/work/llvm-3.8.0.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:66                                              
    #1 0x50e215 in profile_add_info /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_profiles.c:99:4                                                               


==10144==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60600000efa0 at pc 0x00000047b8cf bp 0x7ffe22881090 sp 0x7ffe22880840
READ of size 256 at 0x60600000efa0 thread T0
    #0 0x47b8ce in memcmp /var/tmp/temp/portage/sys-devel/llvm-3.8.0-r2/work/llvm-3.8.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:418
    #1 0x50ff6f in update_user_list /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_profiles.c:251:12
    #2 0x50e555 in profile_add_info /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_profiles.c:91:10
    #3 0x4f3d3a in create_hosts_list /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_analyze.c:128:7
    #4 0x4fcf0c in display_info /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_display.c:218:4
    #5 0x4fcf0c in display /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_display.c:52
    #6 0x507818 in main /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_main.c:94:4
    #7 0x7f53a781461f in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/csu/libc-start.c:289
    #8 0x41a408 in _start (/usr/bin/etterlog+0x41a408)

0x60600000efa0 is located 0 bytes to the right of 64-byte region [0x60600000ef60,0x60600000efa0)
allocated by thread T0 here:
    #0 0x4c1ae0 in calloc /var/tmp/temp/portage/sys-devel/llvm-3.8.0-r2/work/llvm-3.8.0.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:66
    #1 0x50ffea in update_user_list /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_profiles.c:256:4


==10212==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60e00000de40 at pc 0x00000047b8cf bp 0x7ffe0a6b9460 sp 0x7ffe0a6b8c10
READ of size 37636 at 0x60e00000de40 thread T0
    #0 0x47b8ce in memcmp /var/tmp/temp/portage/sys-devel/llvm-3.8.0-r2/work/llvm-3.8.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:418
    #1 0x50e1a3 in profile_add_info /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_profiles.c:89:12
    #2 0x4f3d3a in create_hosts_list /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_analyze.c:128:7
    #3 0x4fcf0c in display_info /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_display.c:218:4
    #4 0x4fcf0c in display /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_display.c:52
    #5 0x507818 in main /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_main.c:94:4
    #6 0x7f562119261f in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/csu/libc-start.c:289
    #7 0x41a408 in _start (/usr/bin/etterlog+0x41a408)

0x60e00000de40 is located 0 bytes to the right of 160-byte region [0x60e00000dda0,0x60e00000de40)
allocated by thread T0 here:
    #0 0x4c1ae0 in calloc /var/tmp/temp/portage/sys-devel/llvm-3.8.0-r2/work/llvm-3.8.0.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:66
    #1 0x50e215 in profile_add_info /tmp/portage/net-analyzer/ettercap-9999/work/ettercap-9999/utils/etterlog/el_profiles.c:99:4

Affected version:
0.8.2

Fixed version:
N/A

Commit fix:
N/a

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
N/A

Timeline:
2016-08-10: bug discovered
2016-08-11: bug reported to upstream
2016-09-06: blog post about the issue

Note:
This bug was found with American Fuzzy Lop.

Permalink:

ettercap: etterlog: multiple (three) heap-based buffer overflow (el_profiles.c)

Greg KH a.k.a. gregkh (homepage, bugs)
4.9 == next LTS kernel (September 06, 2016, 07:59 UTC)

As I briefly mentioned a few weeks ago on my G+ page, the plan is for the 4.9 Linux kernel release to be the next “Long Term Supported” (LTS) kernel.

Last year, at the Linux Kernel Summit, we discussed just how to pick the LTS kernel. Many years ago, we tried to let everyone know ahead of time what the kernel version would be, but that caused a lot of problems as people threw crud in there that really wasn’t ready to be merged, just to make it easier for their “day job”. That was many years ago, and people insist they aren’t going to do this again, so let’s see what happens.

I reserve the right to not pick 4.9 and support it for two years, if it’s a major pain because people abused this notice. If so, I’ll possibly drop back to 4.8, or just wait for 4.10 to be released. I’ll let everyone know by updating the kernel.org releases page when it’s time (many months from now.)

If people have questions about this, email me and I will be glad to discuss it.

September 02, 2016
Alexys Jacob a.k.a. ultrabug (homepage, bugs)
RethinkDB on Gentoo Linux (September 02, 2016, 06:56 UTC)

2014-11-05-cat-instagram

It was about time I added a new package to portage and I’m very glad it to be RethinkDB and its python driver !

  • dev-db/rethinkdb
  • dev-python/python-rethinkdb

For those of you who never heard about this database, I urge you to go about their excellent website and have a good read.

Packaging RethinkDB

RethinkDB has been under my radar for quite a long time now and when they finally got serious enough about high availability I also got serious about using it at work… and obviously “getting serious” + “work” means packaging it for Gentoo Linux 🙂

Quick notes on packaging for Gentoo Linux:

  • This is a C++ project so it feels natural and easy to grasp
  • The configure script already offers a way of using system libraries instead of the bundled ones which is in line with Gentoo’s QA policy
  • The only grey zone about the above statement is the web UI which is used precompiled

RethinkDB has a few QA violations which the ebuild is addressing by modifying the sources:

  • There is a configure.default which tries to force some configure options
  • The configure is missing some options to avoid auto installing some docs and init scripts
  • The build system does its best to guess the CXX compiler but it should offer an option to set it directly
  • The build system does not respect users’ CXXFLAGS and tries to force the usage of -03

Getting our hands into RethinkDB

At work, we finally found the excuse to get our hands into RethinkDB when we challenged ourselves with developing a quizz game for our booth as a sponsor of Europython 2016.

It was a simple game where you were presented a question and four possible answers and you had 60 seconds to answer as much of them as you could. The trick is that we wanted to create an interactive game where the participant had to play on a tablet but the rest of the audience got to see who was currently playing and follow their score progression + their ranking for the day and the week in real time on another screen !

Another challenge for us in the creation of this game is that we only used technologies that were new to us and even switched jobs so the backend python guys would be doing the frontend javascript et vice et versa. The stack finally went like this :

  • Game quizz frontend : Angular2 (TypeScript)
  • Game questions API : Go
  • Real time scores frontend : Angular2 + autobahn
  • Real time scores API : python 3.5 asyncio + autobahn
  • Database : RethinkDB

As you can see on the stack we chose RethinkDB for its main strength : real time updates pushed to the connected clients. The real time scores frontend and API were bonded together using autobahn while the API was using the changefeeds (realtime updates coming from the database) and broadcasting them to the frontend.

What we learnt about RethinkDB

  • We’re sure that we want to use it in production !
  • The ReQL query language is a pipeline so its syntax is quite tricky to get familiar with (even more when coming from mongoDB like us), it is as powerful as it can be disconcerting
  • Realtime changefeeds have limitations which are sometimes not so easy to understand/find out (especially the order_by / secondary index part)
  • Changefeeds limitations is a constraint you have to take into account in your data modeling !
  • Changefeeds + order_by can do the ordering for you when using the include_offsets option, this is amazing
  • The administration web UI is awesome
  • The python 3.5 asyncio proper support is still not merged, this is a pain !

Try it out

Now that you can emerge rethinkdb I encourage you to try this awesome database.

Be advised that the ebuild also provides a way of configuring your rethinkdb instance by running emerge –config dev-db/rethinkdb !

I’ll now try to get in touch with upstream to get Gentoo listed on their website.

August 29, 2016
potrace: memory allocation failure (August 29, 2016, 17:23 UTC)

Description:
potrace is a utility that transforms bitmaps into vector graphics.

A crafted image, through a fuzz testing, causes the memory allocation to fail.

Since the ASan symbolyzer didn’t start up correctly, I’m reporting only what it prints at the end.

# potrace $FILE
potrace: warning: 2.hangs: premature end of file
==13660==ERROR: AddressSanitizer failed to allocate 0x200003000 (8589946880) bytes of LargeMmapAllocator (error code: 12)
==13660==AddressSanitizer CHECK failed: /var/tmp/portage/sys-devel/llvm-3.8.1/work/llvm-3.8.1.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_common.cc:183 "((0 && "unable to mmap")) != (0)" (0x0, 0x0)

Affected version:
1.13

Fixed version:
N/A

Commit fix:
N/A

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
N/A

Timeline:
2016-08-26: bug discovered
2016-08-27: bug reported privately to upstream
2016-08-29: blog post about the issue

Note:
This bug was found with American Fuzzy Lop.

Permalink:

potrace: memory allocation failure