Gentoo Logo
Gentoo Logo Side
Gentoo Spaceship

Contributors:
. Aaron W. Swenson
. Agostino Sarubbo
. Alexey Shvetsov
. Alexis Ballier
. Alexys Jacob
. Alice Ferrazzi
. Alice Ferrazzi
. Andreas K. Hüttel
. Anthony Basile
. Arun Raghavan
. Bernard Cafarelli
. Brian Harring
. Christian Ruppert
. Chí-Thanh Christopher Nguyễn
. Denis Dupeyron
. Detlev Casanova
. Diego E. Pettenò
. Domen Kožar
. Doug Goldstein
. Eray Aslan
. Fabio Erculiani
. Gentoo Haskell Herd
. Gentoo Miniconf 2016
. Gentoo Monthly Newsletter
. Gentoo News
. Gilles Dartiguelongue
. Greg KH
. Göktürk Yüksek
. Hanno Böck
. Hans de Graaff
. Ian Whyman
. Jan Kundrát
. Jason A. Donenfeld
. Jeffrey Gardner
. Joachim Bartosik
. Johannes Huber
. Jonathan Callen
. Jorge Manuel B. S. Vicetto
. Kristian Fiskerstrand
. Lance Albertson
. Liam McLoughlin
. Luca Barbato
. Marek Szuba
. Mart Raudsepp
. Matt Turner
. Matthew Thode
. Michael Palimaka
. Michal Hrusecky
. Michał Górny
. Mike Doty
. Mike Gilbert
. Mike Pagano
. Nathan Zachary
. Pacho Ramos
. Patrick Kursawe
. Patrick Lauer
. Patrick McLean
. Paweł Hajdan, Jr.
. Petteri Räty
. Piotr Jaroszyński
. Rafael G. Martins
. Remi Cardona
. Richard Freeman
. Robin Johnson
. Ryan Hill
. Sean Amoss
. Sebastian Pipping
. Steev Klimaszewski
. Stratos Psomadakis
. Sven Vermeulen
. Sven Wegener
. Tom Wijsman
. Tomáš Chvátal
. Yury German
. Zack Medico

Last updated:
June 29, 2017, 01:05 UTC

Disclaimer:
Views expressed in the content published here do not necessarily represent the views of Gentoo Linux or the Gentoo Foundation.


Bugs? Comments? Suggestions? Contact us!

Powered by:
Planet Venus

Welcome to Gentoo Universe, an aggregation of weblog articles on all topics written by Gentoo developers. For a more refined aggregation of Gentoo-related topics only, you might be interested in Planet Gentoo.

June 28, 2017

Description:
xar is an easily extensible archive format.

The complete ASan output of the issue:

# xar -t -f $FILE
==5525==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7f075cfb35f6 bp 0x7fff705167b0 sp 0x7fff70515f38 T0)
==5525==The signal is caused by a READ memory access.
==5525==Hint: address points to the zero page.
    #0 0x7f075cfb35f5 in strlen /tmp/portage/sys-libs/glibc-2.23-r3/work/glibc-2.23/string/../sysdeps/x86_64/strlen.S:76
    #1 0x45f5ef in __strdup /tmp/portage/sys-libs/compiler-rt-sanitizers-4.0.0/work/compiler-rt-4.0.0.src/lib/asan/asan_interceptors.cc:562
    #2 0x7f075decebc8 in xar_get_path /var/tmp/portage/app-arch/xar-1.6.1-r1/work/xar-1.6.1/lib/util.c:95:8
    #3 0x523f93 in print_file /var/tmp/portage/app-arch/xar-1.6.1-r1/work/xar-1.6.1/src/xar.c:214:16
    #4 0x513f07 in list /var/tmp/portage/app-arch/xar-1.6.1-r1/work/xar-1.6.1/src/xar.c:1524:4
    #5 0x513f07 in main /var/tmp/portage/app-arch/xar-1.6.1-r1/work/xar-1.6.1/src/xar.c:2666
    #6 0x7f075cf55680 in __libc_start_main /tmp/portage/sys-libs/glibc-2.23-r3/work/glibc-2.23/csu/../csu/libc-start.c:289
    #7 0x41af38 in _init (/usr/bin/xar+0x41af38)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /tmp/portage/sys-libs/glibc-2.23-r3/work/glibc-2.23/string/../sysdeps/x86_64/strlen.S:76 in strlen
==5525==ABORTING

Affected version:
1.6.1

Fixed version:
N/A

Commit fix:
N/A

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
N/A

Reproducer:
https://github.com/asarubbo/poc/blob/master/00287-xar-nullptr-xar_get_path

Timeline:
2017-06-17: bug discovered and reported to upstream
2017-06-28: blog post about the issue

Note:
This bug was found with American Fuzzy Lop.

Permalink:

xar: NULL pointer dereference in xar_get_path (util.c)

Description:
xar is an easily extensible archive format.

The complete ASan output of the issue:

# xar -t -f $FILE
==7615==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000008 (pc 0x7f71a859ebd6 bp 0x7fffd8ace150 sp 0x7fffd8acde80 T0)
==7615==The signal is caused by a WRITE memory access.
==7615==Hint: address points to the zero page.
    #0 0x7f71a859ebd5 in xar_unserialize /var/tmp/portage/app-arch/xar-1.6.1-r1/work/xar-1.6.1/lib/archive.c:1767:27
    #1 0x7f71a859ebd5 in xar_open /var/tmp/portage/app-arch/xar-1.6.1-r1/work/xar-1.6.1/lib/archive.c:340
    #2 0x5139ee in list /var/tmp/portage/app-arch/xar-1.6.1-r1/work/xar-1.6.1/src/xar.c:1492:6
    #3 0x5139ee in main /var/tmp/portage/app-arch/xar-1.6.1-r1/work/xar-1.6.1/src/xar.c:2666
    #4 0x7f71a76a2680 in __libc_start_main /tmp/portage/sys-libs/glibc-2.23-r3/work/glibc-2.23/csu/../csu/libc-start.c:289
    #5 0x41af38 in _init (/usr/bin/xar+0x41af38)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /var/tmp/portage/app-arch/xar-1.6.1-r1/work/xar-1.6.1/lib/archive.c:1767:27 in xar_unserialize
==7615==ABORTING

Affected version:
1.6.1

Fixed version:
N/A

Commit fix:
N/A

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
N/A

Reproducer:
https://github.com/asarubbo/poc/blob/master/00288-xar-nullptr-xar_unserialize

Timeline:
2017-06-17: bug discovered and reported to upstream
2017-06-28: blog post about the issue

Note:
This bug was found with American Fuzzy Lop.

Permalink:

xar: NULL pointer dereference in xar_unserialize (archive.c)

June 27, 2017
Alice Ferrazzi a.k.a. alicef (homepage, bugs)
Google-Summer-of-Code-day20 (June 27, 2017, 22:02 UTC)

Google Summer of Code day 20

What was my plan for today?

  • work on the livepatch downloader and make the kpatch creator flexible

What i did today?

  • Created .travis.yml for validating changes https://github.com/aliceinwire/elivepatch/blob/master/.travis.yml
  • Finished making the live patch downloader https://github.com/aliceinwire/elivepatch/commit/6eca2eec3572cad0181b3ce61f521ff40fa85ec1
  • Testing elivepatch

The POC generally works but I had a problem with building the Linux kernel 4.9.29 on my notebook One problem with the POC is that still some variable are hard coded.

WARNING: Skipping gcc version matching check (not recommended) Skipping cleanup Using source directory at /usr/src/linux-4.9.29-gentoo Testing patch file checking file fs/exec.c Hunk #1 succeeded at 238 (offset -5 lines). Reading special section data Building original kernel Building patched kernel Extracting new and modified ELF sections /usr/libexec/kpatch/create-diff-object: ERROR: exec.o: find_local_syms: 136: find_local_syms for exec.c: found_none ERROR: 1 error(s) encountered. Check /root/.kpatch/build.log for more details.

the function find_local_syms https://github.com/dynup/kpatch/blob/master/kpatch-build/lookup.c#L80

Now i'm rebuilding everything with debug options for see some more useful information I'm also thinking to add a debug option to the elivepatch server

One question is if can be useful to work on making a feature for getting the kernel version from the Kernel configuration file header.

like this:

.config #

Automatically generated file; DO NOT EDIT.

Linux/x86 4.9.29-gentoo Kernel Configuration

#

like parsing this for get the version file without need to give it manually.

Another option is to passing it by rest as command line option.

something like -g 4.9.29

Interesting thing is that as now kernel-build have already embedded some way of dealing with most problems, and works better with distribution like ubuntu or fedora.

like for example is already copying the .config file and building the kernel with the option that we are giving from the rest api. cp -f /home/alicef/IdeaProjects/elivepatch/elivepatch_server/config /usr/src/linux-4.9.29-gentoo/.config

and the patch cp /home/alicef/IdeaProjects/elivepatch/elivepatch_server/1.patch kpatch.patch

Is also checking the .config for missing configurations. grep -q CONFIG_DEBUG_INFO_SPLIT=y /home/alicef/IdeaProjects/elivepatch/elivepatch_server/config

what i will do next time?
* Testing elivepatch * Getting the kernel version dynamically * Updating kpatch-build for work with Gentoo better

Open Source Summit Japan-2017 (June 27, 2017, 13:57 UTC)

Open Source Summit Japan 2017 summary

OSS Japan 2017 was a really great experience.

I sended my paper proposal and waited for a replay, some week after I got a
invite to partecipate at the Kernel Keynote.
I thought partecipating at the Kernel Keynote as mentor and doing a presentation
was a good way to talk about Gentoo Kernel Project and how to contribute in the
Linux Kernel and Gentoo Kernel Project.
Also my paper got accepted so I could join OSS Japan 2017 as speaker.
It was three really nice days.

Presentation:

Fast Releasing and Testing of Gentoo Kernel Packages and Future Plans of the Gentoo Kernel Project

My talk was manly about the Gentoo Kernel related Projects past and future
specifically about the Gentoo Kernel Continuos Integreting system we are creating:
https://github.com/gentoo/Gentoo_kernelCI

Why is needed:

  • We need some way for checking the linux-patches commits automatically, can also check pre-commit by pushing to a sandbox branch
  • Check the patches signatures
  • Checking the ebuild committed to https://github.com/gentoo/gentoo/commits/master/sys-kernel
  • Checking the kernel eclass commits
  • Checking the pull request to the sys-kernel/*
  • Use Qemu for testing kernel vmlinux correct execution

For any issue or contribution feel free to send here:
https://github.com/gentoo/Gentoo_kernelCI

For see Gentoo Kernel CI in action:
http://kernel1.amd64.dev.gentoo.org:8010

slides:
http://schd.ws/hosted_files/ossjapan2017/39/Gentoo%20Kernel%20recent%20and%20Future%20project.pdf

Open Source Summit Japan 2017
Keynote: Linux Kernel Panel - Moderated by Alice Ferrazzi, Gentoo Kernel Project Leader

The keynote was with:
Greg Kroah-Hartman - Fellow, Linux Foundation
Steven Rostedt - VMware
Dan Williams - Intel Open Source Technology Center
Alice Ferrazzi - Gentoo Kernel Project Leader, Gentoo

One interesting part was about how to contribute to the Linux Kernel.
After some information about Linux Kernel contribution numbers the talk moved on
ho to contribute in the Linux Kernel.
For contribute in the Linux Kernel there is need of some understanding of C
and running test in the Linux Kernel.
Like fuego, kselftest, coccinelle, and many others.
And also a good talk from Steven Rostedt about working with Real-Time patch.

Who can find the Gentoo logo in this image:

Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)

For someone that has spent the past thirteen years defining himself as a developer of a Linux distribution (whether I really am still a Gentoo Linux developer or not is up for debate I’m sure), having to write a title like this is obviously hard. But from the day I started working on open source software to now I have grown a lot, and I have realized I have been wrong about many things in the past.

One thing that I realized recently is that nowadays, distributions lost the war. As the title of this post says, difference is our strength, but at the same time, it is also the seed of our ruin. Take distributions: Gentoo, Fedora, Debian, SuSE, Archlinux, Ubuntu. They all look and act differently, focusing on different target users, and because of this they differ significantly in which software they make available, which versions are made available, and how much effort is spent on testing, both the package itself and the system integration.

While describing it this way, there is nothing that scream «Conflict!», except at this point we all know that they do conflict, and the solutions from many different communities, have been to just ignore distributions: developers of libraries for high level languages built their own packaging (Ruby Gems, PyPI, let’s not even talk about Go), business application developers started by using containers and ended up with Docker, and user application developers have now started converging onto Flatpak.

Why the conflicts? A lot of time the answer is to be found in bickering among developers of different distributions and the «We are better than them!» attitude, which often turned to «We don’t need your help!». Sometimes this went all the way to the negative side to the point of «Oh it’s a Gentoo [or other] developer complaining, it’s all their fault and their problem, ignore them.» And let’s not forget of the enmity between forks (like Gentoo, Funtoo and Exherbo), in which both sides are trying to prove being better than the other. A lot of conflict all over the place.

There were of course at least two main attempts to standardise parts of how a distribution works: the Linux Standard Base and FreeDesktop.org. The former is effectively a disaster, the latter is more or less accepted, but the problem lies there: in the more-or-less. Let’s look at these two separately.

The LSB was effectively a commercial effort, which was aimed at pleasing (effectively) only the distributors of binary packages. It really didn’t make much of an assurance of the environment you could build things in, and it never invited non-commercial entities to discuss the reasoning behind the standard. In an environment like open source, the fact that the LSB became an ISO standard is not a badge of honour, but rather a worry that it’s over-specified and over-complicated. Which I think most people agree it is. There is also quite an overreach of specifying the presence of binary libraries, rather than being a set of guidelines for distributions to follow.

And yes, although technically LSB is still out there working, the last release I could find described in Wikipedia is from 2015, and I couldn’t even find at first search whether they certified any distribution version. Also, because of the nature of certifications, it’s impossible to certify a rolling-release distribution, which as it happens are becoming much more widespread than they used to.

I think that one of the problem of LSB, both from the adoption and usefulness point of views, is that it focused entirely too much on providing a base platform for binary and commercial application. Back when it was developed, it seemed like the future of Linux (particularly on the desktop) relied entirely on the ability for proprietary software applications to be developed that could run on it, the way they do on Windows and OS X. Since many of the distributions didn’t really aim to support this particular environment, convincing them to support LSB was clearly pointless.

FreeDesktop.org is in a much better state in this regard. They point out that whatever they write is not standards, but de-facto specifications. Because of the de-facto character of these, they started by effectively writing down whatever GNOME and RedHat were doing, but then grown to be significantly more cross-desktop, thanks to KDE and other communities. Because of the nature of the open source community, FD.o specifications are much more widely adopted than the “standards”.

Again, if you compare with what I said above, FD.o provides specifications that make it easier to write, rather than run, applications. It provides you with guarantees of where you should be looking for your file and which icons should be rendering, and which interfaces are exposed. Instead of trying to provide an environment where a in-house written application will keep running for the next twenty years (which, admittedly, Windows has provided for a very long time), it provides you building blocks interfaces so that you can create whatever the heck you want and integrate with the rest of the desktop environments.

As it happens, Lennart and his systemd ended up standardizing distributions a lot more than LSB or FD.o ever did, if nothing else by taking over one of the biggest customization points of them all: the init system. Now, I have complained about this before that it probably could have been a good topic for a standard even before systemd, and independently from it, that developers should have been following, but that’s another problem. At the end of the day, there is at least some cross-distribution way to provide init system support, and developers know that if they build their daemon in a certain way, then they can provide the init system integration themselves, rather than relying on the packagers.

I feel that we should have had much more of that. When I worked on ruby-ng.eclass and fakegem.eclass, I tried getting the Debian Ruby team, who had similar complaints before, to join me on a mailing list so that we could discuss a common interface between Gems developers and Linux distributions, but once again, that did not actually happen. My afterthought is that we should have had a similar discussion for CPAN, CRAN, PyPI, Cargo and so on… and that would probably have spared us the mess that is go packaging.

The problem is not only getting the distributions to overcome their differences, both in technical direction and marketing, but that it requires sitting at a table with the people who built and use those systems, and actually figuring out what they are trying to achieve. Because in particular in the case of Gems and the other packaging systems, who you should talk with is not only your distribution’s users, but most importantly the library authors (whose main interest is shipping stuff so that people can use them) and the developers who use them (whose main interest is being able to fetch and use a library without waiting for months). The distribution users are, for most of the biggest projects, sysadmins.

This means you have a multi-faceted problem to solve, with different roles of people, and different needs of them. Finding a solution that does not compromise, and covers 100% of the needs of all the roles involved, and requires no workflow change on any one’s part is effectively impossible. What you should be doing is focusing on choosing the very important features for the roles critical to the environment (in the example above, the developers of the libraries, and the developers of the apps using those libraries), requiring the minimum amount of changes to their workflow (but convince them to change the workflow where it really is needed, as long as it’s not more cumbersome than it was before for no advantage), and figuring out what can be done to satisfy or change the requirements of the “less important” roles (distribution maintainers usually being that role).

Again going back to the example of Gems: it is clear by now that most of the developers never cared of getting their libraries to be carried onto distributions. They cared about the ability to push new releases of their code fast, seamlessly and not have to learn about distributions at all. The consumers of these libraries don’t and should not care about how to package them for their distributions or how they even interact with it, they just want to be able to deploy their application with the library versions they tested. And setting aside their trust in distributions, sysadmin only care to have a sane handling of dependencies and being able to tell which version of which library is running on their production, to upgrade them in case of a security issues. Now, the distribution maintainers can become the nexus for all these problems, and solve it once and for all… but they will have to be the ones making the biggest changes in their workflow – which is what we did with ruby-ng – otherwise they will just become irrelevant.

Indeed, Ruby Gems and Bundler, PyPI and VirtualEnv, and now Docker itself, are expressions of that: distribution themselves became a major risk and cost point, by being too different between each other and not providing an easy way to just provide one working library, and use one working library. These roles are critical to the environment: if nobody publish libraries, consumers have no library to use; if nobody consumes libraries, there is no point in publishing them. If nobody packages libraries, but there are ways to publish and consume them, the environment still stands.

What would I do if I could go back in time, be significantly more charismatic, and change the state of things? (And I’m saying this for future reference, because if it ever becomes relevant to my life again, I’ll do exactly that.)

  • I would try to convince people that even on divergence of technical direction, discussing and collaborating is a good thing to do. No idea is stupid, idiotic or any other random set of negative words. The whole point of that is that you need to make sure that even if you don’t agree on a given direction, you can agree on others, it’s not a zero-sum game!
  • Speaking of, “overly complicated” is a valid reason to not accept one direction and take another; “we always did it this way” is not a good reason. You can keep using it, but then you’ll end up with Solaris a very stagnant project.
  • Talk with the stakeholders of the projects that are bypassing distributions, and figure out why they are doing that. Provide “standard” tooling, or at least a proposal on how to do things in such a way that the distributions are still happy, without causing undue burden.
  • Most importantly, talk. Whether it is by organizing mailing lists, IRC channels, birds of a feather at conferences, or whatever else. People need to talk and discuss the issues at hand in clear, in front of the people building the tooling and making the decisions.

I have not done any of that in the past. If I ever get in front of something like this, I’ll do my best to, instead. Unfortunately, this is a position that, in the current universe we’re talking about, would have required more privilege than I had before. Not only for my personal training and experience to understand what should have been done, but because it requires actually meeting with people and organizing real life summits. And while nowadays I did become a globetrotter, I could never have afforded that before.

June 26, 2017
Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)

With the release of Lab::Measurement 3.550 we've switched to Dist::Zilla as maintenance tool. If you're not involved in hacking Lab::Measurement, you should essentially not notice this change. However, for the authors of the package, Dist::Zilla makes it much easier to keep track of dependencies, prepare new releases, and eventually also improve and unify the documentation... At the side we've also fixed Issue 4 and Lab::Measurement should now work out of the box with recent Gnuplot on Windows again.

June 25, 2017
Alice Ferrazzi a.k.a. alicef (homepage, bugs)
Open Source Summit Japan 2017 (June 25, 2017, 17:59 UTC)

Open Source Summit Japan 2017 summary

OSS Japan 2017 was a really great experience.

I sended my paper proposal and waited for a replay, some week after I got a invite to partecipate at the Kernel Keynote. I thought partecipating at the Kernel Keynote was a good way to talk about Gentoo Kernel Project and how to contribute in the Linux Kernel and Gentoo Kernel Project. Also my paper got accepted so I could join OSS Japan 2017 as speaker.

It was three really nice days.

[[!tags draft]]

Google-Summer-of-Code-summary week04 (June 25, 2017, 17:59 UTC)

Google Summer of Code summary week 04

What I did in this week 04 summary:

elivepatch:

  • Created the elivepatch client command line argument parser
  • Added function for sending patch and configuration files
  • Divided the call for sending (patch, config) and the call for building the livepatch
  • made send_file function more generic for sending all kind of files using RESTful api
  • Cleaned code following pep8
  • Decided to use only SSL and to don't use basic auth
  • Sending informations about the kernel version when requesting a livepatch build
  • We can now build livepatch using the RESTful API
  • Returning information about the livepatch building status

Kpatch:

  • Working on making kpatch-build working also with gentoo with all the features (As now kpatch-build can only automatically build livepatch for Ubuntu, Debian, Red Hat, Fedora)

Others:

  • Ask infra for a server for install the elivepatch server

What I need to do next time:

  • Finish the function for download the livepatch to the client
  • Testing elivepatch
  • Implementing the CVE patch uploader
  • Installing elivepatch to the Gentoo server
  • Fix kpatch-build for automatically work with gentoo-sources
  • Add more features to elivepatch

Google-Summer-of-Code-day18 (June 25, 2017, 17:59 UTC)

Google Summer of Code day 18

What was my plan for today?

  • going on with the code for retriving the livepatch and installing it

What i did today?

checked about kpatch-build required folder.

kpatch-build find_dirs function:

find_dirs() {
  if [[ -e "$SCRIPTDIR/create-diff-object" ]]; then
      # git repo
      TOOLSDIR="$SCRIPTDIR"
      DATADIR="$(readlink -f $SCRIPTDIR/../kmod)"
  elif [[ -e "$SCRIPTDIR/../libexec/kpatch/create-diff-object" ]]; then
      # installation path
      TOOLSDIR="$(readlink -f $SCRIPTDIR/../libexec/kpatch)"
      DATADIR="$(readlink -f $SCRIPTDIR/../share/kpatch)"
  else
      return 1
  fi
}

$SCRIPTDIR is the kpatch-build directory. kpatch-build is installed in /usr/bin/ so /usr/kmod /usr/libexe are all under such directory.

error "CONFIG_FUNCTION_TRACER, CONFIG_HAVE_FENTRY, CONFIG_MODULES, CONFIG_SYSFS, CONFIG_KALLSYMS_ALL kernel config options are required" Require by kmod/core.c: https://github.com/dynup/kpatch/blob/master/kmod/core/core.c#L62

We probably need someway for check that this setting are configured in the kernel we are going to build.

Updating kpatch-build for work automatically with gentoo (as now fedora for example can automatically download the kernel rpm and install it, we could do similar thing with gentoo): https://github.com/aliceinwire/kpatch/commits/gentoo

Starting to write the live patch downloader: https://github.com/aliceinwire/elivepatch/commit/d26611fb898223f2ea2dcf323078347ca928cbda

Now the elivepatch server can call and build the livepatch with kpatch:

sudo kpatch-build -s /usr/src/linux-4.10.14-gentoo/ -v /usr/src/linux-4.10.14-gentoo//vmlinux -c config 1.patch --skip-gcc-check
ERROR: kpatch build failed. Check /root/.kpatch/build.log for more details.
127.0.0.1 - - [25/Jun/2017 05:27:06] "POST /elivepatch/api/v1.0/build_livepatch HTTP/1.1" 201 -
WARNING: Skipping gcc version matching check (not recommended)
Using source directory at /usr/src/linux-4.10.14-gentoo
Testing patch file
checking file fs/exec.c
Hunk #1 succeeded at 259 (offset 16 lines).
Reading special section data
Building original kernel

Fixed some minor pep8

what i will do next time?
* work on the livepatch downloader and make the kpatch creator flexible

June 24, 2017
Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)

I’m still not sure how it is that over the past two years I consider myself a big expert of 2FA, it probably has to do with having wanted to implement this for a very long time for the work I did before my current employer, and the fact that things keeps changing, and bad advices keep being sent out. But as it happens here is something more.

A couple of months ago, LastPass announced Cloud Backup for LastPass Authenticator, which effectively means you can upload your TOTP shared key (which is used for the generation of the one-time codes you see in the authenticator app) to LastPass so that you can log in on a different device (phone) and have access to all your previously set up 2FA credentials.

It’s marketed as a solution to the TOTP dance of changing devices which is always a definite pain in the neck, and it’s not alone. Authy (which many may know as the authentication system for Humble Bundle or Twilio) has a similar feature for their app. I have honestly wondered why Google Authenticator still does not have a similar feature either, but let’s not go there for now.

The obvious question that came to me, and probably to anyone with a little bit of security knowledge, when this was announced is «who in their sane mind would use this?». But before we get to that question, let me explain something that, it turns out, is not obvious to many people.

2-factor authentication, also referred to 2-step verification, – which refer to slightly different semantics but I’m not here splitting hairs – is the idea that you need “something you have” (or “something you have access to”) as well as “something you know”. Your username would be effectively the zeroth factor, followed by your password as first factor, and the magical second factor. For the most part at first the reference to this “something you have” related to physical devices, such as the RSA SecurID and other “tamper proof” tokens that display a one-time code, either changing every given amount of seconds or changing every time you pressed a button.

Nowadays, the majority of 2FA users use an authenticator app, such as the already mentioned LastPass Authenticator, Authy and Google Authenticator apps. These run on Android or iOS and provide TOTP-based one-time codes that are used for second factor authentication. These are not really as secure as one would like, and indeed they are still phishable. For proper security, services should really just use U2F.

So if you’re using an authenticator app, any authenticator app, you have your second factor, which is not the device itself! Rather, the second factor is the shared key, that the system gave you through the QR code (in most cases, at least), and that is combined with the current timestamp to give you a one time code (which is why it’s called TOTP: Time-based One Time Password). This is a very important distinction because, as I showed in a previous post, at least one of the services will not re-key the shared key when you set up authenticator on a new device, which means people with access to your old device still have your “factor”.

One obvious problem one might think of is that your phone is likely connected to your Google account, so if you have your authenticator on it, you may get an impression that factors 0, 1 and 2 are all on the same device. Of course that is not the case, because you can’t retrieve the password from a logged in Google account on a mobile device. In place of a password, a single-use token is provided (that is true not only for Google, but for almost every single app out there), which is tied to the device in some form. And that token effectively replaces both factors (password and TOTP). So you really have factors 0 and 2.

Why do I bring up this problem? Well, here’s the thing with LastPass: while all LastPass’s assurances are believable, and they may not have made huge mistakes with their authenticator app as they did for their extensions in the past, you have a basket with multiple eggs on it already, do you want to add all of them for real? Assume I put a TOTP key onto LastPass Authenticator with Cloud Backup, and the username and password for that account are also stored in LastPass, and that someone manages to log into my LastPass (how, that’s a different problem, but just assume they do). Now they have all three factors at their disposal.

Now it is true that LastPass forces you to use 2FA yourself for an account that has the Authenticator Cloud Backup enabled, but it is not clear to me if that means that you can’t use a physical YubiKey for that and whether the Cloud Backup would become unrestorable if you decide to disable 2FA, for instance because your device got stolen (in which case, well, you just lost all your 2FAs). Having had to disable 2FA for LastPass once already (I was in China without my main Authenticator device), it effectively relies on you having access to your registered email address… which for someone who may gain access to my LastPass password, would probably be a cakewalk. It’s also not clear to me whether having the ability to request arbitrary information from the extension (as Tavis managed to do at least once) would allow you to fetch the Cloud Backup shared keys too. In which case, the whole security sandcastle is gone.

An astute reader could ask why am I even considering this a valid option and spending my time figuring out how insecure this is. The answer to that is a bit complicated but I’ll still talk about my reasoning. 2FA is good, and everybody should be using it. On the other hand, 2FA is also cumbersome and it feels it’s not worth the hassle. For services that support U2F, it’s very very easy: you just register a couple of U2F devices (e.g. one you keep at home in a safe, and one you keep with your housekeys), and you would have access to at least one at any time. Minimal amount of work needed, major security.

But would you really do the TOTP registration dance I talked above for things like (and I’m checking my own history here), IFTTT? Maybe, but probably not. It’s more likely that you decide that for those, it’s easier to get SMS as second factor, and you prove you have access to your phone number. Unfortunately this is both a problem, for instance when you fly a lot. It’s also not secure at all, as people have by now repeatedly demonstrated, both because intercepting SMS is getting more and more trivial, and because lots of telcos can be easily manipulated into giving up access to a phone number, particularly in the US, where the phone is tied to a device, rather than to a SIM card.

I also do something like this, except the phone number is a Google Voice number (similar to Google Project Fi), which makes it easier for me to receive the codes wherever I am in the world, and also to receive them when I’m only connected to the Internet and not to cell service. And since my Google account is protected by U2F, and the password is not in LastPass, I feel safe… enough for most of the smaller services. But not all the services allow me to send the code to Google Voice (some of the bridges still fail there), so I have been hoping for some cloud backup option to at least have an easy way to move around a subset of my TOTP shared keys.

Because accounts like EA, AutoDesk (I have no licenses for them) or WordPress (I have no sites there), are probably low-targets enough that I wouldn’t mind using an imperfect (security-wise) solution such as LastPass Authenticator Cloud Backup… except if it defeated the whole purpose of 2FA, which it clearly does, when their password is also there. This is funny because it means I’d feel safer to keep a copy of my Google account TOTP key in LastPass as its password is not in LastPass.

As I noted above, Authy does have something like that as well. Their main API, used by Humble Bundle and Twitch, relies on server-side hosting of the keys, with extra seeds that are given to each device if you install their app and use it. Setting up an account is trivial as it relies on a phone number as username and a one-time code they send to that phone number — effectively it’s a single factor authenticator. This does mean that its base security is not any better than the security of an SMS. Which makes it hardly useful, except for being easier to access when you somehow have no access to SMS. Or, as Twitch will tell you, to “cut down on those expensive SMS” — yeah I always forget in the US you pay to receive text messages. Strange country.

So while I don’t think Authy is a very sensible or secure option, as it has the same vulnerability as SMS do, their system does have an option to backup your TOTP shared keys, which includes an extra password in addition to the SMS auth. Of course by virtue of what I Was saying earlier, you should not be storing that password in LastPass, or you’ll have put effectively one basket into the other.

What does this mean at the end of day? Well, it’s obvious people are finding 2FA hard, and they want to provide better solutions that still work for users and allow them to be protected by 2FA. Unfortunately it does not seem the current options available, except for U2F are very secure or reliable, and U2F is far from widespread.

My personal perfect end state would be for websites to allow me to log in based on Facebook or Google account, the passwords of which are not stored in LastPass, and then asking me for a 2FA token that could be stored in LastPass then. In that case, LastPass would still only have one factor, same as they do right now with the password, but since the first factor is not “password” but “access to {other} account”, that is one less thing to keep in mind.

We’ll see what the future brings ups.

June 23, 2017
Alice Ferrazzi a.k.a. alicef (homepage, bugs)
Google-Summer-of-Code-day17 (June 23, 2017, 23:12 UTC)

Google Summer of Code day 16

What was my plan for today?

  • going on with the code for retriving the livepatch and installing it
  • Ask infra for a server where to install elivepatch sever

What i did today?
Sended request for the server that will offer the elivepatch service as talked with my mentor. https://bugs.gentoo.org/show_bug.cgi?id=622476

Fixed some pep8 warnings.

Livepatch server is now returning information about the livepatch building status.

Removed basic auth as we will go with SSL.

The client is now sending information about the kernel version when requesting a new build.

The kernel directory under the server is now a livepatch class variable.

what i will do next time?

  • going on with the code for retriving the livepatch and installing it

Google-Summer-of-Code-day16 (June 23, 2017, 23:12 UTC)

Google Summer of Code day 16

What was my plan for today?

  • Divide call for sending (patch, config) and the call for build the livepatch
  • Make the livepatch call more flexible (as now is hardcoded)
  • Ask infra for a server where to install elivepatch sever

What i did today?

Added patch file path argument to the elivepatch server API and added patch call to elivepatch client.

Adding way for dividing the call for sending the configuration with a POST call sending the patch with a POST calland than start the livepatch build and getting the result.

patch sended work and working on calling livepatch.

Added docstring to the build patch function.

Cleaned GetLive dispatcher function.

Added call from client to build livepatch of the server API.

made send_file function more generic for send all kind of file.

what i will do next time?

  • going on with the code for retriving the livepatch and installing it

June 21, 2017
Alice Ferrazzi a.k.a. alicef (homepage, bugs)
2017-06-21 Google-Summer-of-Code-day14 (June 21, 2017, 17:57 UTC)

Google Summer of Code day 15

What was my plan for today?
working on sending the configuration file on RESTful api,
and starting to work on making the patch.ko file in the server.

What i did today?
using wekzeug.datastructures.FileStorage in elivepatch_server,
I could receive the file from the elivepatch_client POST request
using the RESTful API.

def post(self):
    parse = reqparse.RequestParser()
    parse.add_argument('file', type=werkzeug.datastructures.FileStorage, location)

so as now we can get the kernel configuration file, extract if is .gz filename and send it to the elivepatch server.

elivepatch server need to read the configuration, compare it with the
current kernel configuration and if different recompile the kernel.
After we can start making the livepatch with kpatch-build.

This is the example of using kpatch-build:

kpatch-build/kpatch-build -s /usr/src/linux-4.9.16-gentoo/ -v /usr/src/linux-4.9.16-gentoo/vmlinux examples/test.patch --skip-gcc-check
gsoc-2017 kpatch (gentoo) # kpatch-build/kpatch-build --help
usage: kpatch-build [options] <patch file>
            -h, --help         Show this help message
            -r, --sourcerpm    Specify kernel source RPM
            -s, --sourcedir    Specify kernel source directory
            -c, --config       Specify kernel config file
            -v, --vmlinux      Specify original vmlinux
            -t, --target       Specify custom kernel build targets
            -d, --debug        Keep scratch files in /tmp
            --skip-cleanup     Skip post-build cleanup
            --skip-gcc-check   Skip gcc version matching check
                               (not recommended)

This command is called automatically by the elivepatch server after receiving the configuration file.

we need also to send the patch file.

what i will do next time?

  • Divide call for sending (patch, config) and the call for build the livepatch
  • Make the livepatch call more flexible (as now is hardcoded)
  • Ask infra for a server where to install elivepatch sever

Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)

You probably have noticed by now that I write about glucometers quite a bit, not only reviewing them as an user, but also reverse engineering to figure out their protocols. This all started four years ago when I needed to send my glucometer readings to my doctor and I ended up having to write my own tool.

That tool started almost as a joke, particularly given I wrote it in Python, which at the time I was not an expert in at all (I have since learnt a lot more about it, and at work I got to be more of an expert than I’d ever expected to be). But I always known that it would be for the most part just a proof of concept. Not only exporting CSV is mostly useless, but the most important part in diabetes management software is the analysis and I don’t have any clue how to do analysis.

At first I thought I could reuse some of the implementation to expand Xavier’s OpenGlucose but it turned out that it’s not really easy for those meters that are using serial adapters or other USB devices beside the HID ones that he implemented already. Of course this does mean it would probably work fine for things like the FreeStyle Libre which I appear to have written the only Linux software to download from, but even in that case, things are more complicated.

Indeed, as I have noted here and there previously, we need a better format to export glucometer data, and in particular the data from continuous or mixed meters like the Libre. My current out format for it only includes the raw glucose readings from the meter that are not marked as errors; it does provide an unstructured text comment that tells you whether the reading is coming from the background sensor, an explicit scan or a blood sample, but it does not provide all the level of details of the original readings. And it does not expose ketone readings at all, despite the fact that most of the FreeStyle-line devices support them and I even documented how to get them. But this is a topic for a different post, I think.

On the other hand, over the past four years, the number of meters increased significantly, and I even have a few more that I only have partially reversed and not published yet. Currently there are 9 drivers, covering over a dozen meters (some meters share the same driver, either because they are just rebranded versions or simply because they share the same protocol). One is for the InsuLinx, which also loses a bunch of details, and is based off Xavier’s own reverse engineering — I did that mostly because all the modern FreeStyle devices appear to share the same basic protocol, and so writing new drivers for them is actually fairly trivial.

This would make the project an interesting base if someone feels like writing a proper UI for it. If I ever tried to look into that, I may end up just starting an HTTP server and provide everything over HMTL for the browser to render. After all that’s actually how OpenGlucose is doing things, except there is no server, and the browser is embedded. Alternatively one could just write an HTML report file out, the same way Accu-Chek Mobile does using data URLs and JavaScript bundles.

One of the most important usability changes I have added recently, though, is allowing the user not to specify the device path. When I started writing the tool, I started by looking at serial adapter based devices, which usually come with their own cable, and you just access it. The next driver was for the LBA-over-SCSI used int he OneTouch Verio, which I could have auto-detected but didn’t, and the following ones, mostly based off HID, I just expected to be given an hidraw path.

But all of this is difficult, and indeed I had more than a few people asking me which device are they meant to use, so over the past few months I adapter the drivers to try auto-detecting the devices. For the serial port based meters, the auto-detection targets the original manufacturer’s cable, so if you have a custom one, you should still pass them a path. For HID based devices, you also need the Python hidapi library because I couldn’t bother writing my own HID bus parsing on Linux…

… and the library also brings another important feature: it works on non-Linux operating systems. Indeed I now have not one but two confirmed users that managed to use the tool on Windows, for two separate FreeStyle devices (at the time of writing, the only ones implementing HID-based protocols, although I have another one in the pipeline.

Supposedly, all of this should work fine on macOS (I almost called it OS X), though the one person who contacted me trying to have it working there has been having trouble with it — I think the problem has something to do with the Python version available (I’m targetting Python 3 because I had a very hard time to do the right processing with Python 2.7). So if you want to give it a try feel free.

And yes, I’m very happy to receive pull request, so if you want to implement that HTML output I talked above about, you’re awesome and I’m looking forward to your pull request. I’m afraid I won’t be much help with the visualisation though.

Nathan Zachary a.k.a. nathanzachary (homepage, bugs)
The Book of Henry film review (June 21, 2017, 03:06 UTC)

Notice

Though I make every effort to not include spoilers in my reviews (personal opinions and biases, however, are fair game 🙂 ), some of the linked pages may contain them.

The Book of Henry opening title

This past weekend I went to the theatre to see the newly-released film The Book of Henry (WARNING: the IMDb and Wikipedia pages contain massive spoilers) starring Jaeden Lieberher, Jacob Tremblay, and Naomi Watts. I’ve been anxiously awaiting the release of the film because 1) the trailer piqued my interest, and 2) the cast is filled with some outstanding talent.

Without giving too much away, the basic premise is that Henry (Lieberher) is an incredibly intelligent 11-year-old who essentially plays the “head of household” role for the family. His mother Susan (Watts) has come to rely on him for many adult-oriented tasks, such as managing the finances, setting ground rules, and taking care of his younger brother Peter (Tremblay). When Henry learns that a girl in his class (who is also his neighbour) is being abused by her stepfather, he devises a plan to rescue her and his mother is tasked with helping execute that plan.

Brothers Peter and Henry Carpenter in The Book of HenryBrothers Peter and Henry Carpenter

I went into this film not knowing all that much about it (which is what I prefer: I would rather form my own impressions than start viewing it with preconceived notions and expectations). Based on the opening credits, I anticipated being introduced to a precocious kid that was wise beyond his years. I thought that it may turn into a dramatic representation of his sociocultural interactions at school and how they impacted his outlook on life. Prepared for that type of somewhat slower-paced progression, I was pleasantly surprised to find a stronger emphasis on the family dynamics between Henry, Peter, Susan, and Susan’s coworker and friend, Sheila (Sarah Silverman). As the film continues, we do get to see some of the philosophically advanced aspects of Henry’s worldview, but more importantly we see—through his focus on making the world a better place and his desire for justice—the love that he has for other people.

 

Our legacy isn’t what we write on a résumé or how many commas we have in our bank account. It’s who we’re lucky enough to have in our lives, and what we can leave them with. The one thing we do know: we’re here now, so I say we do the best we can while we’re on this side of the dirt.—Henry Carpenter

Henry and his mother discuss helping othersHenry and his mother discussing helping others

Is that what sets The Book of Henry apart from other films? No, not at all. There are plenty of examples of films where a child protagonist is longing to make a difference in the world (including another one of Liberher’s recent roles [and Watts’s, for that matter] in St. Vincent). Instead, what makes this film special is the way in which it is able to elicit the full gamut of human emotion in such a strong, sweeping fashion. For instance, in one scene we’re reminded of the frivolity of childhood by seeing Henry and Peter playing and inventing in their elaborate fort in the woods behind their home. Thereafter, we see some of the strain on Susan as she tries to balance her own needs and wants with doing the best that she in can in raising her two boys (as illustrated in a scene where she may have slightly overindulged with her friend Sheila 😉 ). Following that somewhat more serious tone comes a tender moment with Susan putting Henry and Peter to bed by singing a song to them and playing the ukulele. It’s this juxtaposition of emotional evocation that elevates The Book of Henry to levels above others in the genre.

Henry and Peter in their backyard fortHenry and Peter in their backyard fort

Focusing again on emotionality, there have been very few movies over the years that have actually made me cry (for the curious, Radio Flyer, Life is Beautiful, and The Fountain are a few). I’ve been misty-eyed before, but there were a few scenes in this film that actually brought me to tears. One in particular was so gut-wrenching for me that I thought I might have to briefly excuse myself from the theatre (thankfully, though, I toughened up). Will everyone react in the same way to these types of scenes? No, but that’s one of the beautiful bonds between well-made films and its viewers.

Even with the moments that were extremely difficult to watch due to their forlorn sentiments, there were plenty of convivial parts that provided an overall undertone of warmth throughout the movie. One key example that sticks out in my mind is at the Calvary Elementary School talent show where Peter performs a magic trick. He comes out in full magician garb (reminiscent of one of Tremblay’s first roles in The Magic Ferret [which, by the way, is a cute short film that can be purchased through director Alison Parker’s website]), and wows the audience all the while smiling ear-to-ear. These types of scenes provided the much needed comic relief from the overarching sentimentality.

The Book of Henry - Jacob Tremblay as Peter the GreatJacob Tremblay as Peter the Great

It is likely this roller coaster ride of emotions (or some of the unusual/unresolved details) that led to the overwhelmingly negative Rotten Tomatoes Critics’ Consensus that the movie “deserves a few points for ambition, but its tonal juggling act—and a deeply maudlin twist—may leave viewers gaping in disbelief rather than choking back tears”. Again, that’s the beauty of the world of artistic expression (which, of course, includes film): it can speak to different people in vastly different ways based on what they individually bring to the table. I, for one, thought that it was beautifully done and in no way “maudlin”. Instead of relying solely on the reviews (including this one) of professional critics or viewers, though, I urge you to go see it for yourself and make your own assessment.

8 out of 10 stars:
Filled starFilled starFilled starFilled starFilled starFilled starFilled starFilled starUnfilled starUnfilled star

Cheers,
Zach

June 19, 2017
Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
Free Idea: a QEMU Facedancer fuzzer (June 19, 2017, 01:04 UTC)

This post is part of a series of free ideas that I’m posting on my blog in the hope that someone with more time can implement. It’s effectively a very sketched proposal that comes with no design attached, but if you have time you would like to spend learning something new, but no idea what to do, it may be a good fit for you.

Update (2017-06-19): see last paragraph.

You may already be familiar with the Facedancer, the USB fuzzer originally designed and developed by Travis Goodspeed of PoC||GTFO fame. If you’re not, in no so many words, it’s a board (and framework) that allows you to simulate the behaviour of any USB device. It works thanks to a Python framework which supports a few other board and could – theoretically – be expanded to support any device with gadgetfs support (such as the BeagleBone Black I have at home, but I digress).

I have found about this through Micah’s videos, and I have been thinking for a while to spend some time to do the extension to gadgetfs so I can use it to simulate glucometers with their original software — as in particular it should allow me to figure out which value represent what, by changing what is reported to the software and see how it behaves. While I have had no time to do that yet, this is anyway a topic for a different post.

The free idea I want to give instead is to integrate, somehow, the Facedancer framework with QEMU, so that you can run the code behind a Facedancer device as if it was connected to a QEMU guest, without having to use any hardware at all. A “Virtualdancer”, which would not only obviate the need for hardware in the development phase (if the proof of concept of a facedancer were to require an off-site usage) but also would integrate more easily into fuzzing projects such as Bochspwn or TriforceAFL.

In particular, I have interest not only in writing simulated glucometers for debugging purposes (although a testsuite that requires qemu and a simulated device may be a bit overkill), but also in simulating HID devices. You may remember that recently I had to fix my ELECOM trackball, and this is not the first time I have to deal with broken HID descriptor. I have spent some more time looking into the Linux HID subsystem and I’m trying to figure out if I can make some simplifications here and there (again, topic for another time), so having a way to simulate an HID device with strange behaviour and see if my changes fix it or not would be extremely beneficial.

Speaking of HID, and report descriptors in particular, Alex Ionescu (of ReactOS fame) at REcon pointed out that there appear to be very few reported security issues with HID report descriptor parsing, in particular for Windows, which seems strange given how parsing those descriptor is very hard, and in particular there are very seriously broken descriptors out there. This would be another very interesting surface for a QEMU-based dancer software, to run through a number of broken HID report descriptor and send data to see how the system behaves. I would be very surprised if there is no bug in particular on the many small and random drives that apply workarounds such as the one I did for ELECOM.

Anyway, as I said I haven’t even had time to make the (probably minor) modifications to the framework to support BBB (which I already have access to), so you can imagine I’m not going to be working on this any time soon, but if you feel like working on some USB code, why not?

Update (2017-07-19): I pointed Travis at this post over Twitter, and he showed me vUSBf. While this does not have the same interface as facedancer, it proves that there is a chance to provide a virtual USB device implemented in Python.

As a follow up, Binyamin Sharet linked to Umap2 which supports fuzzing on top of facedancer, but does not support qemu as it is.

While it’s not quite (yet) what I had in mind, it proves that it is a feasible goal, and that there is already some code out there getting very close!

June 18, 2017
Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
I'm moving… to London (June 18, 2017, 01:04 UTC)

There is something about me and writing blog posts while sitting in airport lounges around the globe, or at the very least in New York and Paris…

Four years and change after I announced my first move I’m moving again, though this time the jump is significantly shorter, and even the downsizing is downsized, particularly as the plug I’d be using is the same. You could say this is a bold move, particularly with the ghost of Brexit floating above us as a semi-transparent Damocle’s Sword, but it’s a decision I made, not over night, but through many months of deliberation and a good number of arguments back and forth. And I decided this before the hilarious hung parlament results.

So, why am I doing this move, and what does it actually include? I’m staying at the same company, in the same role and in a team not really far away (on a management, if not service, chain) from the one I was at before. What really matters in all of this is the place I live in, rather than the work I’m doing. Work provided an excuse and a spark for me to make the jump, simply because my previous team has been folded into another.

When I flew onto Dublin, for the second time after the interview, four years ago, I knew next to nothing about Ireland, except for the stereotypes and the worrying stories of the Troubles. I effectively knew only one person in the country, and I was fairly scared about the whole process, which went significantly smoother than I would have expected. As a friend of mine told me before I left, Dublin is the capital that still feels like a little town, and I can definitely see that.

I grew up in a relatively small town, with next to no public transport, but a significant amount of malls and stores, and a stupid but functional airport nearby. Dublin beat that significantly, except for the lack of malls, and the fact that unless you have a car to go outside the city centre, it is likely that whatever you need you’ll order on Amazon anyway. Except there is no Amazon Ireland, so you’ll be looking at either Amazon UK, Italy, Germany or (if you can manage) France. Because the prices will be different all over, and some of the Amazon stores will refuse to send things to you in Ireland, and you have to use one of the many services that provide you with an UK address, and then reships the parcel to you (most of which use addresses in Northern Ireland, because it’s close enough, and counts as UK for most of those limitation).

But it also has limitations that a small provincial town has, and I’m not talking about the song from Beauty and the Beast, though it does come close. At least for the native Irish, the stereotypes is that they all know each other. It is not true of course, as it’s a stereotype, but it comes awfully close. In particular, according to a former manager, Irish people make friends in two phases in life: during school and during their kids’ school years. If you happen to get to Ireland when you’re clearly too old to be a student, but single, with no kids (and no intention to have any), making friends is not quite easy.

I have tried, maybe not with just as much energy as others have, but still tried. Unfortunately, almost the full set of social gatherings in Ireland involve pubs. I don’t drink alcohol, alcohol could kill me, and definitely makes me mean. But I could go to a pub and not drink of course. I did that in Italy, where the pub favourite by my group of friends offered as many pages of menu to coffee, chocolate, the and infusions drinks as of beers, plus a good selection of other desserts. That is not the case in Ireland. Effectively, the only non-alcoholic drinks you find at the pub are there to not live thirsty the designated drivers, and they are bloody expensive. €5 for a 150ml bottle of Diet Coke! — I’m told that Coca Cola by the pint is cheaper, unfortunately because of diabetes that would be almost as bad as alcohol. I know at least one pub in which I can order a pot of tea, and not get the stink eye by the waiters, but that’s about it.

For the first couple of years I didn’t really feel this being a big problem: for the first few months, Enrico (from #gentoo-it) was around as well, before moving on with his life in a different country, I made a number of friends at the office, and a few people who I knew before, such as Stefano from automake, came to Dublin. Then something happened, that made me question the idea of mixing colleagues and friends. Nothing Earth-shattering, but enough of a demonstration for me that I started isolating myself from most of the people at the office: rejected invitations for social events, mass de-friended people from Facebook, and so on. You may think this sounds a bit extreme, but I have my reasons.

I turned again to trying to find interesting groups of people in Ireland, and that’s where the introvert and awkward part of me just gave up and filled me with dread: I’m not particularly good with meeting people already, but it just felt too much work to join an established group of people who all know each other without even having one person you know to introduce you. I had better luck on that with meeting people at Eastercon and Nine Worlds, thanks to Gianni. And all the groups I could find in Dublin that didn’t have to do with “going out for pints”, were effectively groups of people who have known each other for years. They may be friendly but not easy to make friends, at least not for me.

Of course there is a group of people that I knew already, and I considered: the people who left the office! Unfortunately, almost the full amount of people who I knew that stayed in Ireland did so because they have a family there. They are in a different “time of life” than me, and that makes for an awkward time — most wouldn’t be interested in meeting to go to a geek store on the weekend, or a Comicon or similar venue. You can go and see them at home (depending on how friendly the terms you’re in are), and see the kids. Heh, no thanks.

So after four years, a lot of the people I knew and was/am friend with leaving Ireland for other countries, offices or jobs, I found myself lonely, and with a need to move somewhere else. And just to be clear, loneliness is not just about being single. That is obviously a part of it, but it’s in general the inability to say to any group of friends “Let’s meet tonight for movies, boardgames or a match of Street Fighter” (or whatever beat ‘em up the cool kids play nowadays). I could do that in Italy, I even still managed to do that in Italy last time I flew in, with some of my best friends now living together and expecting a baby. But I can’t do that in Dublin because there are very few people with who this proposition is not totally awkward.

Heck, I can do that in Paris! I have friends in France who I just had to say “There is this concert at this date, want to go?” and six months later I’m flying a DUB-CDG/CDG-YUL/YUL-DUB ticket to see Joe Hisaishi at Le Palais des Congrés. (Montréal is for a conference, it was the week after so it was easy to just go from one to the other without passing by Dublin.)

But most of this awkwardness is on me, so why do I expect things to be any different in London? Well, to begin with I know more people there (outside of work) than I did in Dublin, or that I even do now. A friend of mine from junior high moved to London last year with her boyfriend, so at the last I know them, and we’re close enough that the similar proposition of spending an evening in playing Scrabble is totally feasible (actually, we did just that when I went to see them last year). And these are people who can make the general awkwardness of entering a new group of people easier for me.

Most importantly, though, I think London has a different structure that should make it possible for me to end up meeting people even randomly. Because though I am clearly awkward and scared of joining big groups that already know each other, I don’t mind the random words with a stranger in the queue for a flight or while waiting for coffee. After all that’s give or take how I end up meeting people at conferences. In London there are just too many people for them to all know each other. And in particular, there are the places where these things can happen.

Part of the reason why almost all the social gatherings in Dublin revolve around pubs is that they are the only venue you can go to spend some time on “neutral grounds” (i.e. not in someone’s home, not at an office). I don’t know of a coffee shop in Dublin that is open after 1900 on a weekday, except for the Starbucks in front of St Stephen Green and a few Insomnias that are embedded within 24hrs convenience stores. London has plenty, although they are not all in the most convenient of places, if anything you can find a Nero open until 2200 almost every other corner. No, Nero does not stay open that much in Dublin either.

There is more. If you’re a geek you probably have at least heard of Forbidden Planet. It’s a megastore of comics, manga, books, and anything geek effectively. It’s very commercial and mainstream, but it’s also big and they run book signings there, which makes it an interesting place to go just to see if something may be going on. It’s effectively in the centre of life if London, effectively in SoHo, and if you meet there with friends you can geek out with friends on their latest purchases in one of the many cafes around the area. Or browse, buy something and go to dinner.

There is a Forbidden Planet in Dublin which is not as big but it still has a decent selection. Unfortunately all the venues around it are effectively pubs, as you’re on the verge of Temple Bar, which appears to be the Irish national treasure. Unless you manage to get out well before 5pm and get coffee to the shop just behind it, which of course closes at 5pm all week long. And even to be fair and consider Sub City comics which in my opinion is usually better stocked, it’s all pubs around it except for Bewley’s and Krüst down the street in front of it… which requires you to cross one of the most trafficked streets in Dublin city centre.

I’m not saying that there is an absolute certainty that I will meet people in London, I’m just saying that there are a few things that are not stacked against me, like they are in Dublin. And I really need to make some change that I don’t feel full of dread every time I come back to my apartment after weeks travelling, like I did coming back from my recent trip across Asia, when all of the four people I hang out with at all were not in town.

Nothing else really changes even, but you may notice that the targets of my rants will change from Ireland to UK, and possibly to the process of actually moving, filing taxes and so on. Actually, I know you’ll get at least another post on payment cards because I started looking into them and it looks like the foreign transaction fees in the UK are really bad.

June 17, 2017
Sebastian Pipping a.k.a. sping (homepage, bugs)

Expat 2.2.1 has been released. It’s a security release with a variety of security fixes, for instance: An infinite loop denial-of-service fix (that Rhodri James wrote more about), introduction of SipHash against sophisticated hash flooding, use of OS-specific high quality entropy providers like getrandom, integer overflow fixes, and more. We also got better code coverage, moved all but the downloads from SourceForge to GitHub, … but maybe have a look at the detailed change log yourself 🙂

So if you control copies of Expat somewhere, please get them updated.

Let me use the occasion to point out that we are looking for help with a few things Expat. There are tickets with details up here. If you can help, please get in touch.

Thanks and best

 

Sebastian

lame: two UBSAN crashes (June 17, 2017, 14:52 UTC)

Description:
lame is a high quality MPEG Audio Layer III (MP3) encoder licensed under the LGPL.

Few notes before the details of this bug. Time ago a fuzz was done by Brian Carpenter and Jakub Wilk which posted the results on the debian bugtracker. In cases like this, when upstream is not active and people do not post on the upstream bugzilla is easy discover duplicates, so I downloaded all available testcases, and noone of the bug you will see on my blog is a duplicate of an existing issue. Upstream seems a bit dead, latest release was into 2011, so this blog post will probably forwarded on the upstream bugtracker just for the record.

The complete ASan output of the issue:

# lame -f -V 9 $FILE out.wav
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/brhist.c:204:60: runtime error: signed integer overflow: 953447384 + 1908859798 cannot be represented in type 'int'

Reproducer:
https://github.com/asarubbo/poc/blob/master/00298-lame-signintoverflow-brhist.c
CVE:
N/A

#######################

# lame -f -V 9 $FILE out.wav
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:1234:21: runtime error: value -nan is outside the range of representable values of type 'int'

Reproducer:
https://github.com/asarubbo/poc/blob/master/00299-lame-outside-int-get_audio.c
CVE:
N/A

#######################

Affected version:
3.99.5

Fixed version:
N/A

Commit fix:
N/A

Credit:
These bugs were discovered by Agostino Sarubbo of Gentoo.

Timeline:
2017-06-01: bug discovered
2017-06-17: blog post about the issue

Note:
These bugs were found with American Fuzzy Lop.

Permalink:

lame: two UBSAN crashes

lame: multiple left shift (June 17, 2017, 14:52 UTC)

Description:
lame is a high quality MPEG Audio Layer III (MP3) encoder licensed under the LGPL.

Few notes before the details of this bug. Time ago a fuzz was done by Brian Carpenter and Jakub Wilk which posted the results on the debian bugtracker. In cases like this, when upstream is not active and people do not post on the upstream bugzilla is easy discover duplicates, so I downloaded all available testcases, and noone of the bug you will see on my blog is a duplicate of an existing issue. Upstream seems a bit dead, latest release was into 2011, so this blog post will probably forwarded on the upstream bugtracker just for the record.

The complete ASan output of the issue:

# lame -f -V 9 $FILE out.wav
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:263:5: runtime error: left shift of negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:265:5: runtime error: left shift of negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:266:5: runtime error: left shift of negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:267:5: runtime error: left shift of negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:268:5: runtime error: left shift of negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:269:5: runtime error: left shift of negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:271:5: runtime error: left shift of negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:272:5: runtime error: left shift of negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:273:5: runtime error: left shift of negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:274:5: runtime error: left shift of negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:276:5: runtime error: left shift of negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:277:5: runtime error: left shift of negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:278:5: runtime error: left shift of negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:279:5: runtime error: left shift of negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:280:5: runtime error: left shift of negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:845:48: runtime error: left shift of negative value -18
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:848:52: runtime error: left shift of negative value -10

Reproducer:
https://github.com/asarubbo/poc/blob/master/00295-lame-leftshift1
CVE:
N/A

#######################################

# lame -f -V 9 $FILE out.wav
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:848:52: runtime error: left shift of negative value -29398
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/bitstream.c:181:50: runtime error: left shift of 45389699 by 6 places cannot be represented in type 'int'

Reproducer:
https://github.com/asarubbo/poc/blob/master/00296-lame-leftshift2
CVE:
N/A

#######################################

# lame -f -V 9 $FILE out.wav
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:1195:52: runtime error: left shift of 255 by 24 places cannot be represented in type 'int'

Reproducer:
https://github.com/asarubbo/poc/blob/master/00297-lame-leftshift3
CVE:
N/A

#######################################

Affected version:
3.99.5

Fixed version:
N/A

Commit fix:
N/A

Credit:
These bugs were discovered by Agostino Sarubbo of Gentoo.

Timeline:
2017-06-01: bug discovered
2017-06-17: blog post about the issue

Note:
These bugs were found with American Fuzzy Lop.

Permalink:

lame: multiple left shift

Description:
lame is a high quality MPEG Audio Layer III (MP3) encoder licensed under the LGPL.

Few notes before the details of this bug. Time ago a fuzz was done by Brian Carpenter and Jakub Wilk which posted the results on the debian bugtracker. In cases like this, when upstream is not active and people do not post on the upstream bugzilla is easy discover duplicates, so I downloaded all available testcases, and noone of the bug you will see on my blog is a duplicate of an existing issue. Upstream seems a bit dead, latest release was into 2011, so this blog post will probably forwarded on the upstream bugtracker just for the record.

The complete ASan output of the issue:

# lame -f -V 9 $FILE out.wav
==30801==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffe82a515a0 at pc 0x7f56d24c9df7 bp 0x7ffe82a4ffb0 sp 0x7ffe82a4ffa8
WRITE of size 4 at 0x7ffe82a515a0 thread T0
    #0 0x7f56d24c9df6 in III_dequantize_sample /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/layer3.c
    #1 0x7f56d24a664f in decode_layer3_frame /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/layer3.c:1738:17
    #2 0x7f56d24733ca in decodeMP3_clipchoice /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/interface.c:615:13
    #3 0x7f56d2470c13 in decodeMP3 /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/interface.c:696:12
    #4 0x7f56d2431092 in decode1_headersB_clipchoice /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/mpglib_interface.c:149:11
    #5 0x7f56d243694a in hip_decode1_headersB /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/mpglib_interface.c:436:16
    #6 0x7f56d243694a in hip_decode1_headers /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/mpglib_interface.c:379
    #7 0x51e984 in lame_decode_fromfile /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:2089:11
    #8 0x51e984 in read_samples_mp3 /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:877
    #9 0x51e984 in get_audio_common /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:785
    #10 0x51e4fa in get_audio /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:688:16
    #11 0x50f776 in lame_encoder_loop /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:456:17
    #12 0x50f776 in lame_encoder /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:531
    #13 0x50c43f in lame_main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:707:15
    #14 0x510793 in c_main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/main.c:470:15
    #15 0x510793 in main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/main.c:438
    #16 0x7f56d1029680 in __libc_start_main /tmp/portage/sys-libs/glibc-2.23-r3/work/glibc-2.23/csu/../csu/libc-start.c:289
    #17 0x41c998 in _init (/usr/bin/lame+0x41c998)

Address 0x7ffe82a515a0 is located in stack of thread T0 at offset 5024 in frame
    #0 0x7f56d24a548f in decode_layer3_frame /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/layer3.c:1659

  This frame has 4 object(s):
    [32, 344) 'scalefacs'
    [416, 5024) 'hybridIn' 0x1000505422b0: 00 00 00 00[f2]f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2
  0x1000505422c0: f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2
  0x1000505422d0: f2 f2 f2 f2 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000505422e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000505422f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x100050542300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==30801==ABORTING

Affected version:
3.99.5

Fixed version:
N/A

Commit fix:
N/A

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
CVE-2017-9872

Reproducer:
https://github.com/asarubbo/poc/blob/master/00294-lame-stackoverflow-III_dequantize_sample

Timeline:
2017-06-01: bug discovered
2017-06-17: blog post about the issue
2017-06-25: CVE assigned

Note:
This bug was found with American Fuzzy Lop.

Permalink:

lame: stack-based buffer overflow in III_dequantize_sample (layer3.c)

Description:
lame is a high quality MPEG Audio Layer III (MP3) encoder licensed under the LGPL.

Few notes before the details of this bug. Time ago a fuzz was done by Brian Carpenter and Jakub Wilk which posted the results on the debian bugtracker. In cases like this, when upstream is not active and people do not post on the upstream bugzilla is easy discover duplicates, so I downloaded all available testcases, and noone of the bug you will see on my blog is a duplicate of an existing issue. Upstream seems a bit dead, latest release was into 2011, so this blog post will probably forwarded on the upstream bugtracker just for the record.

The complete ASan output of the issue:

# lame -f -V 9 $FILE out.wav
==30141==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffd587a7600 at pc 0x7f95cdaf0f34 bp 0x7ffd587a6250 sp 0x7ffd587a6248
WRITE of size 4 at 0x7ffd587a7600 thread T0
    #0 0x7f95cdaf0f33 in III_i_stereo /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/layer3.c:1236:28
    #1 0x7f95cdaf0f33 in decode_layer3_frame /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/layer3.c:1753
    #2 0x7f95cdaaa3ca in decodeMP3_clipchoice /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/interface.c:615:13
    #3 0x7f95cdaa7c13 in decodeMP3 /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/interface.c:696:12
    #4 0x7f95cda68092 in decode1_headersB_clipchoice /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/mpglib_interface.c:149:11
    #5 0x7f95cda6d94a in hip_decode1_headersB /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/mpglib_interface.c:436:16
    #6 0x7f95cda6d94a in hip_decode1_headers /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/mpglib_interface.c:379
    #7 0x51efa6 in lame_decode_fromfile /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:2099:19
    #8 0x51efa6 in read_samples_mp3 /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:877
    #9 0x51efa6 in get_audio_common /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:785
    #10 0x51e4fa in get_audio /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:688:16
    #11 0x50f776 in lame_encoder_loop /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:456:17
    #12 0x50f776 in lame_encoder /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:531
    #13 0x50c43f in lame_main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:707:15
    #14 0x510793 in c_main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/main.c:470:15
    #15 0x510793 in main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/main.c:438
    #16 0x7f95cc660680 in __libc_start_main /tmp/portage/sys-libs/glibc-2.23-r3/work/glibc-2.23/csu/../csu/libc-start.c:289
    #17 0x41c998 in _init (/usr/bin/lame+0x41c998)

Address 0x7ffd587a7600 is located in stack of thread T0 at offset 5024 in frame
    #0 0x7f95cdadc48f in decode_layer3_frame /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/layer3.c:1659

  This frame has 4 object(s):
    [32, 344) 'scalefacs'
    [416, 5024) 'hybridIn' 0x10002b0ecec0:[f2]f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2
  0x10002b0eced0: f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2
  0x10002b0ecee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10002b0ecef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10002b0ecf00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10002b0ecf10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==30141==ABORTING

Affected version:
3.99.5

Fixed version:
N/A

Commit fix:
N/A

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
CVE-2017-9871

Reproducer:
https://github.com/asarubbo/poc/blob/master/00293-lame-stackoverflow-III_i_stereo

Timeline:
2017-06-01: bug discovered
2017-06-17: blog post about the issue
2017-06-25: CVE assigned

Note:
This bug was found with American Fuzzy Lop.

Permalink:

lame: stack-based buffer overflow in III_i_stereo (layer3.c)

Description:
lame is a high quality MPEG Audio Layer III (MP3) encoder licensed under the LGPL.

Few notes before the details of this bug. Time ago a fuzz was done by Brian Carpenter and Jakub Wilk which posted the results on the debian bugtracker. In cases like this, when upstream is not active and people do not post on the upstream bugzilla is easy discover duplicates, so I downloaded all available testcases, and noone of the bug you will see on my blog is a duplicate of an existing issue. Upstream seems a bit dead, latest release was into 2011, so this blog post will probably forwarded on the upstream bugtracker just for the record.

The complete ASan output of the issue:

# lame -f -V 9 $FILE out.wav
==29263==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60c00000003c at pc 0x7f60ef5a8c12 bp 0x7ffe7420b940 sp 0x7ffe7420b938
READ of size 4 at 0x60c00000003c thread T0
    #0 0x7f60ef5a8c11 in fill_buffer_resample /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/util.c
    #1 0x7f60ef5a8c11 in fill_buffer /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/util.c:677
    #2 0x7f60ef47866b in lame_encode_buffer_sample_t /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/lame.c:1736:9
    #3 0x7f60ef47866b in lame_encode_buffer_template /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/lame.c:1891
    #4 0x7f60ef47e83a in lame_encode_buffer /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/lame.c:1902:12
    #5 0x7f60ef47e83a in lame_encode_flush /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/lame.c:2134
    #6 0x50fa2c in lame_encoder_loop /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:487:16
    #7 0x50fa2c in lame_encoder /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:531
    #8 0x50c43f in lame_main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:707:15
    #9 0x510793 in c_main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/main.c:470:15
    #10 0x510793 in main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/main.c:438
    #11 0x7f60ee1c7680 in __libc_start_main /tmp/portage/sys-libs/glibc-2.23-r3/work/glibc-2.23/csu/../csu/libc-start.c:289
    #12 0x41c998 in _init (/usr/bin/lame+0x41c998)

0x60c00000003c is located 4 bytes to the left of 128-byte region [0x60c000000040,0x60c0000000c0)
allocated by thread T0 here:
    #0 0x4d2540 in calloc /tmp/portage/sys-libs/compiler-rt-sanitizers-4.0.0/work/compiler-rt-4.0.0.src/lib/asan/asan_malloc_linux.cc:74
    #1 0x7f60ef5a3575 in fill_buffer_resample /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/util.c:558:29
    #2 0x7f60ef5a3575 in fill_buffer /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/util.c:677
    #3 0x7f60ef47866b in lame_encode_buffer_sample_t /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/lame.c:1736:9
    #4 0x7f60ef47866b in lame_encode_buffer_template /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/lame.c:1891
    #5 0x7f60ef47e83a in lame_encode_buffer /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/lame.c:1902:12
    #6 0x7f60ef47e83a in lame_encode_flush /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/lame.c:2134
    #7 0x50fa2c in lame_encoder_loop /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:487:16
    #8 0x50fa2c in lame_encoder /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:531
    #9 0x50c43f in lame_main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:707:15
    #10 0x510793 in c_main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/main.c:470:15
    #11 0x510793 in main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/main.c:438
    #12 0x7f60ee1c7680 in __libc_start_main /tmp/portage/sys-libs/glibc-2.23-r3/work/glibc-2.23/csu/../csu/libc-start.c:289

SUMMARY: AddressSanitizer: heap-buffer-overflow /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/util.c in fill_buffer_resample
Shadow bytes around the buggy address:
  0x0c187fff7fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c187fff7fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c187fff7fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c187fff7fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c187fff7ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c187fff8000: fa fa fa fa fa fa fa[fa]00 00 00 00 00 00 00 00
  0x0c187fff8010: 00 00 00 00 00 00 00 00 fa fa fa fa fa fa fa fa
  0x0c187fff8020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c187fff8030: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x0c187fff8040: 00 00 00 00 00 00 00 00 fa fa fa fa fa fa fa fa
  0x0c187fff8050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==29263==ABORTING

Affected version:
3.99.5

Fixed version:
N/A

Commit fix:
N/A

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
CVE-2015-9101

Reproducer:
https://github.com/asarubbo/poc/blob/master/00292-lame-heapoverflow-fill_buffer_resample

Timeline:
2017-06-01: bug discovered
2017-06-17: blog post about the issue
2017-06-25: CVE assigned

Note:
This bug was found with American Fuzzy Lop.
Mitre decided that this bug can share the same CVE id of https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=777161

Permalink:

lame: heap-based buffer overflow in fill_buffer_resample (util.c)

Description:
lame is a high quality MPEG Audio Layer III (MP3) encoder licensed under the LGPL.

Few notes before the details of this bug. Time ago a fuzz was done by Brian Carpenter and Jakub Wilk which posted the results on the debian bugtracker. In cases like this, when upstream is not active and people do not post on the upstream bugzilla is easy discover duplicates, so I downloaded all available testcases, and noone of the bug you will see on my blog is a duplicate of an existing issue. Upstream seems a bit dead, latest release was into 2011, so this blog post will probably forwarded on the upstream bugtracker just for the record.

The complete ASan output of the issue:

# lame -f -V 9 $FILE out.wav
==28403==ERROR: AddressSanitizer: global-buffer-overflow on address 0x7fecc4b7eb6c at pc 0x7fecc489accc bp 0x7fff525972d0 sp 0x7fff525972c8
READ of size 4 at 0x7fecc4b7eb6c thread T0
    #0 0x7fecc489accb in III_i_stereo /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/layer3.c:1149:26
    #1 0x7fecc489accb in decode_layer3_frame /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/layer3.c:1753
    #2 0x7fecc48543ca in decodeMP3_clipchoice /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/interface.c:615:13
    #3 0x7fecc4851c13 in decodeMP3 /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/interface.c:696:12
    #4 0x7fecc4812092 in decode1_headersB_clipchoice /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/mpglib_interface.c:149:11
    #5 0x7fecc481794a in hip_decode1_headersB /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/mpglib_interface.c:436:16
    #6 0x7fecc481794a in hip_decode1_headers /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/mpglib_interface.c:379
    #7 0x51e984 in lame_decode_fromfile /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:2089:11
    #8 0x51e984 in read_samples_mp3 /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:877
    #9 0x51e984 in get_audio_common /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:785
    #10 0x51e4fa in get_audio /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:688:16
    #11 0x50f776 in lame_encoder_loop /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:456:17
    #12 0x50f776 in lame_encoder /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:531
    #13 0x50c43f in lame_main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:707:15
    #14 0x510793 in c_main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/main.c:470:15
    #15 0x510793 in main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/main.c:438
    #16 0x7fecc340a680 in __libc_start_main /tmp/portage/sys-libs/glibc-2.23-r3/work/glibc-2.23/csu/../csu/libc-start.c:289
    #17 0x41c998 in _init (/usr/bin/lame+0x41c998)

0x7fecc4b7eb6c is located 20 bytes to the left of global variable 'pow2_1' defined in '/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/layer3.c:128:28' (0x7fecc4b7eb80) of size 128
0x7fecc4b7eb6c is located 12 bytes to the right of global variable 'pow1_1' defined in '/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/layer3.c:128:13' (0x7fecc4b7eae0) of size 128
SUMMARY: AddressSanitizer: global-buffer-overflow /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/layer3.c:1149:26 in III_i_stereo
Shadow bytes around the buggy address:
  0x0ffe18967d10: 00 00 00 00 00 00 00 00 00 00 00 00 f9 f9 f9 f9
  0x0ffe18967d20: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 00 00 00 00
  0x0ffe18967d30: 00 00 00 00 f9 f9 f9 f9 00 00 00 00 00 00 00 00
  0x0ffe18967d40: f9 f9 f9 f9 00 00 00 00 00 00 00 00 f9 f9 f9 f9
  0x0ffe18967d50: 00 00 00 00 00 00 00 00 f9 f9 f9 f9 00 00 00 00
=>0x0ffe18967d60: 00 00 00 00 00 00 00 00 00 00 00 00 f9[f9]f9 f9
  0x0ffe18967d70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ffe18967d80: f9 f9 f9 f9 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ffe18967d90: 00 00 00 00 f9 f9 f9 f9 00 00 00 00 00 00 00 00
  0x0ffe18967da0: 00 00 00 00 00 00 00 00 f9 f9 f9 f9 00 00 00 00
  0x0ffe18967db0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==28403==ABORTING

Affected version:
3.99.5

Fixed version:
N/A

Commit fix:
N/A

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
CVE-2017-9870

Reproducer:
https://github.com/asarubbo/poc/blob/master/00291-lame-globaloverflow-III_i_stereo

Timeline:
2017-06-01: bug discovered
2017-06-17: blog post about the issue
2017-06-25: CVE assigned

Note:
This bug was found with American Fuzzy Lop.

Permalink:

lame: global-buffer-overflow in III_i_stereo (layer3.c)

Description:
lame is a high quality MPEG Audio Layer III (MP3) encoder licensed under the LGPL.

Few notes before the details of this bug. Time ago a fuzz was done by Brian Carpenter and Jakub Wilk which posted the results on the debian bugtracker. In cases like this, when upstream is not active and people do not post on the upstream bugzilla is easy discover duplicates, so I downloaded all available testcases, and noone of the bug you will see on my blog is a duplicate of an existing issue. Upstream seems a bit dead, latest release was into 2011, so this blog post will probably forwarded on the upstream bugtracker just for the record.

The complete ASan output of the issue:

# lame -f -V 9 $FILE out.wav
==27479==ERROR: AddressSanitizer: global-buffer-overflow on address 0x7f598d317f20 at pc 0x7f598d2b246b bp 0x7ffe780cf310 sp 0x7ffe780cf308
READ of size 2 at 0x7f598d317f20 thread T0
    #0 0x7f598d2b246a in II_step_one /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/layer2.c:144:36
    #1 0x7f598d2b246a in decode_layer2_frame /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/layer2.c:375
    #2 0x7f598d29b377 in decodeMP3_clipchoice /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/interface.c:611:13
    #3 0x7f598d298c13 in decodeMP3 /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/interface.c:696:12
    #4 0x7f598d259092 in decode1_headersB_clipchoice /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/mpglib_interface.c:149:11
    #5 0x7f598d25e94a in hip_decode1_headersB /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/mpglib_interface.c:436:16
    #6 0x7f598d25e94a in hip_decode1_headers /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/mpglib_interface.c:379
    #7 0x51e984 in lame_decode_fromfile /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:2089:11
    #8 0x51e984 in read_samples_mp3 /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:877
    #9 0x51e984 in get_audio_common /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:785
    #10 0x51e4fa in get_audio /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:688:16
    #11 0x50f776 in lame_encoder_loop /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:456:17
    #12 0x50f776 in lame_encoder /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:531
    #13 0x50c43f in lame_main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:707:15
    #14 0x510793 in c_main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/main.c:470:15
    #15 0x510793 in main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/main.c:438
    #16 0x7f598be51680 in __libc_start_main /tmp/portage/sys-libs/glibc-2.23-r3/work/glibc-2.23/csu/../csu/libc-start.c:289
    #17 0x41c998 in _init (/usr/bin/lame+0x41c998)

0x7f598d317f20 is located 0 bytes to the right of global variable 'alloc_2' defined in '/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/l2tables.h:118:24' (0x7f598d317de0) of size 320
SUMMARY: AddressSanitizer: global-buffer-overflow /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/mpglib/layer2.c:144:36 in II_step_one
Shadow bytes around the buggy address:
  0x0febb1a5af90: 00 00 00 00 00 00 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
  0x0febb1a5afa0: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
  0x0febb1a5afb0: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 00 00 00 00
  0x0febb1a5afc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0febb1a5afd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0febb1a5afe0: 00 00 00 00[f9]f9 f9 f9 f9 f9 f9 f9 00 00 00 00
  0x0febb1a5aff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0febb1a5b000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0febb1a5b010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0febb1a5b020: 00 00 00 00 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
  0x0febb1a5b030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==27479==ABORTING

Affected version:
3.99.5

Fixed version:
N/A

Commit fix:
N/A

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
CVE-2017-9869

Reproducer:
https://github.com/asarubbo/poc/blob/master/00290-lame-globaloverflow-II_step_one

Timeline:
2017-06-01: bug discovered
2017-06-17: blog post about the issue
2017-06-25: CVE assigned

Note:
This bug was found with American Fuzzy Lop.

Permalink:

lame: global-buffer-overflow in II_step_one (layer2.c)

Description:
lame is a high quality MPEG Audio Layer III (MP3) encoder licensed under the LGPL.

Few notes before the details of this bug. Time ago a fuzz was done by Brian Carpenter and Jakub Wilk which posted the results on the debian bugtracker. In cases like this, when upstream is not active and people do not post on the upstream bugzilla is easy discover duplicates, so I downloaded all available testcases, and noone of the bug you will see on my blog is a duplicate of an existing issue. Upstream seems a bit dead, latest release was into 2011, so this blog post will probably forwarded on the upstream bugtracker just for the record.

EDIT:
This bug was already discovered by Brian Carpenter. There wasn’t mentioned because it is in the frontend, but I guess that many webapplications which offer audio conversion, may run the lame command-line tool directly.

The complete ASan output of the issue:

# lame -f -V 9 $FILE out.wav
==26565==ERROR: AddressSanitizer: FPE on unknown address 0x000000519c13 (pc 0x000000519c13 bp 0x7ffd4d07a0d0 sp 0x7ffd4d079e20 T0)                   
    #0 0x519c12 in parse_wave_header /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:1454:54                       
    #1 0x519c12 in parse_file_header /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:1683                          
    #2 0x519c12 in open_wave_file /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:1754                             
    #3 0x519c12 in init_infile /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:620                                 
    #4 0x50f273 in init_files /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:151:9                                
    #5 0x50befc in lame_main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/lame_main.c:674:16                                
    #6 0x510793 in c_main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/main.c:470:15                                        
    #7 0x510793 in main /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/main.c:438                                             
    #8 0x7f14a792b680 in __libc_start_main /tmp/portage/sys-libs/glibc-2.23-r3/work/glibc-2.23/csu/../csu/libc-start.c:289                           
    #9 0x41c998 in _init (/usr/bin/lame+0x41c998)                                                                                                    
                                                                                                                                                     
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: FPE /var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:1454:54 in parse_wave_header
==26565==ABORTING

Affected version:
3.99.5

Fixed version:
N/A

Commit fix:
N/A

Credit:
This bug was initially discovered by Brian Carpenter.

CVE:
N/A

Reproducer:
https://github.com/asarubbo/poc/blob/master/00289-lame-fpe-get_audio.c.wav

Timeline:
2017-06-01: bug discovered
2017-06-17: blog post about the issue

Note:
This bug was found with American Fuzzy Lop.

Permalink:

lame: divide-by-zero in parse_wave_header (get_audio.c)

June 16, 2017
Nathan Zachary a.k.a. nathanzachary (homepage, bugs)

Recently, Gentoo Linux put GCC 6.3 (released in December 2016) into the testing branch. For a source-based distribution like Gentoo, GCC is a critical part of the toolchain, and sometimes can lead to packages failing to compile or run. I recently ran into just this problem with Audacity. The error that I hit was not during compilation, but during runtime:


$ audacity
Fatal Error: Mismatch between the program and library build versions detected.
The library used 3.0 (wchar_t,compiler with C++ ABI 1009,wx containers,compatible with 2.8),
and your program used 3.0 (wchar_t,compiler with C++ ABI 1010,wx containers,compatible with 2.8).
Aborted

The Gentoo Wiki has a nice, detailed page on Upgrading GCC, and explicitly calls out ABI changes. In this particular case of Audacity, the problematic library is referenced in the error above: “wx containers”. WX containers are handled by the wxGTK package. So, I simply needed to rebuild the currently-installed version of wxGTK to fix this particular problem.

Hope that helps.

Cheers,
Zach

June 15, 2017
Hanno Böck a.k.a. hanno (homepage, bugs)
Don't leave Coredumps on Web Servers (June 15, 2017, 09:20 UTC)

CoreCoredumps are a feature of Linux and other Unix systems to analyze crashing software. If a software crashes, for example due to an invalid memory access, the operating system can save the current content of the application's memory to a file. By default it is simply called core.

While this is useful for debugging purposes it can produce a security risk. If a web application crashes the coredump may simply end up in the web server's root folder. Given that its file name is known an attacker can simply download it via an URL of the form http://example.org/core. As coredumps contain an application's memory they may expose secret information. A very typical example would be passwords.

PHP used to crash relatively often. Recently a lot of these crash bugs have been fixed, in part because PHP now has a bug bounty program. But there are still situations in which PHP crashes. Some of them likely won't be fixed.

How to disclose?

With a scan of the Alexa Top 1 Million domains for exposed core dumps I found around 1.000 vulnerable hosts. I was faced with a challenge: How can I properly disclose this? It is obvious that I wouldn't write hundreds of manual mails. So I needed an automated way to contact the site owners.

Abusix runs a service where you can query the abuse contacts of IP addresses via a DNS query. This turned out to be very useful for this purpose. One could also imagine contacting domain owners directly, but that's not very practical. The domain whois databases have rate limits and don't always expose contact mail addresses in a machine readable way.

Using the abuse contacts doesn't reach all of the affected host operators. Some abuse contacts were nonexistent mail addresses, others didn't have abuse contacts at all. I also got all kinds of automated replies, some of them asking me to fill out forms or do other things, otherwise my message wouldn't be read. Due to the scale I ignored those. I feel that if people make it hard for me to inform them about security problems that's not my responsibility.

I took away two things that I changed in a second batch of disclosures. Some abuse contacts seem to automatically search for IP addresses in the abuse mails. I originally only included affected URLs. So I changed that to include the affected IPs as well.

In many cases I was informed that the affected hosts are not owned by the company I contacted, but by a customer. Some of them asked me if they're allowed to forward the message to them. I thought that would be obvious, but I made it explicit now. Some of them asked me that I contact their customers, which again, of course, is impractical at scale. And sorry: They are your customers, not mine.

How to fix and prevent it?

If you have a coredump on your web host, the obvious fix is to remove it from there. However you obviously also want to prevent this from happening again.

There are two settings that impact coredump creation: A limits setting, configurable via /etc/security/limits.conf and ulimit and a sysctl interface that can be found under /proc/sys/kernel/core_pattern.

The limits setting is a size limit for coredumps. If it is set to zero then no core dumps are created. To set this as the default you can add something like this to your limits.conf:

* soft core 0

The sysctl interface sets a pattern for the file name and can also contain a path. You can set it to something like this:

/var/log/core/core.%e.%p.%h.%t

This would store all coredumps under /var/log/core/ and add the executable name, process id, host name and timestamp to the filename. The directory needs to be writable by all users, you should use a directory with the sticky bit (chmod +t).

If you set this via the proc file interface it will only be temporary until the next reboot. To set this permanently you can add it to /etc/sysctl.conf:

kernel.core_pattern = /var/log/core/core.%e.%p.%h.%t

Some Linux distributions directly forward core dumps to crash analysis tools. This can be done by prefixing the pattern with a pipe (|). These tools like apport from Ubuntu or abrt from Fedora have also been the source of security vulnerabilities in the past. However that's a separate issue.

Look out for coredumps

My scans showed that this is a relatively common issue. Among popular web pages around one in a thousand were affected before my disclosure attempts. I recommend that pentesters and developers of security scan tools consider checking for this. It's simple: Just try download the /core file and check if it looks like an executable. In most cases it will be an ELF file, however sometimes it may be a Mach-O (OS X) or an a.out file (very old Linux and Unix systems).

Image credit: NASA/JPL-Université Paris Diderot

Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
Glucometer Review: Contour Next One (June 15, 2017, 01:04 UTC)

In my current quest for documenting glucometer protocols and implementing tooling to download data I ended up ordering myself a Contour Next One glucometer, currently marketed by Ascensia Diabetes Care, but originally developed by Bayer (to the point that devices and manuals still mostly refer to Bayer).

The Contour Next One is marketed as a smart glucometer that can connect to an app on a smartphone, as it supports Bluetooth Low Energy. I have not tried that yet because I already have an app I care about on the phone. In addition to that the physical device is very small, but includes a nice, complete and bright display, a three way buttons, a coloured LED for the strip port device, and a beeper.

The size of the device reminds me closely the OneTouch Ultra Mini, which is quite favourable, as every other device I have is significantly bulkier, and for the most part using a different form factor that I don’t particularly enjoy.

The strip port has a coloured LED as I said, and it’ll be white to tell you where to insert the strip, and then turn either green or red depending on the value. You can associate one out of three states for meal information, represented by an apple (why is it always an apple?) crossed out (fasting), full (before meal) or taken two bites of (after meal).

If there is one thing that I can complain about is that the first-time setup is a bit tricky. I have received the meter a couple of months ago, but since I’ve been travelling so much, I have not managed to try it out until earlier this month. When I finally put the strip into the device, and provided blood to it, I found it wanted to confirm date, time and settings… and did not actually give me a result, oops!

On the practicality, the device is really the kind of size I’d like to keep around, if it wasn’t for the fact I switched to the Libre. While it is not all-in-one like the Accu-Chek Mobile, it is small and compact, and looks sleek for a device.

I have installed the Irish app – it seems they have separate app entries for multiple countries, although it does not appear to have country limitations, as I could ask for it to be installed on my Italian SIM device – the first thing it asked me was the country or region, and Ireland was not in the visible list. Turns out the problem is that the list is actually very long and the app’s scrolling is misconfigured, as scrolling up appears to only scroll a third of a row at a time, so scrolling down to Ireland takes quite a while. For United Kingdom I don’t even want to consider the idea. Looks like Ascensia does have some rough edges with testing the first-time setup of their systems.

The setup process is a bit bothersome, if what you want to do is just downloading the data. It tries to convince you to set up an account to back up your data (this appears to be more and more common), and then it goes through give or take the same first-time setup as happens on the meter. In addition to the meter’s own settings, it also allows you to define time bands to divide your meals into breakfast, lunch and dinner, which I liked: instead of just assuming, it asks you to confirm the expected ranges.

There is also a request to configure two phone numbers: the local emergency number, and a personal emergency contact, so that if you have a low-blood-sugar event you can make a call directly. While the idea is interesting on principle, since it already asked me for my region I wonder why it can’t just default to 999. Finding the contact by name in the list is also extremely hard because of the scrolling issue above. I ended up just typing my own name and phone number, as there is never anyone who could actually help me if I have a sugar low, and the request is unskippable.

The app itself appears very simple, showing by default your last reading and a small graph of your recent readings. If you turn your phone or click on the graph, a more detailed (but still rough) graph appears:

The probably most interesting thing about this the app is that if you take a reading on your meter it will be “pushed” to the device, and appear in real-time. It also allows you to set the markings and a number of comments straight from the app, well exceeding the amount of details that the device can set for you, particularly given it’s a three-buttons device.

It reminds me of the iHealth Align but in a much less clumsy way. It also requires significantly less bother with the strips, and there is no validation of when the bottle was open or whether the strips expired, which makes it probably a better choice for those people who only test occasionally.

On more technical sides, the device uses two CR2032 batteries, which while beefier than the iHealth, need to sustain a full BLE connection – which by the way I couldn’t figure out how to turn off at first try – and are at the same time easier to find replacements for (though not as easy as the AAA batteries used by more clunky meters). It has a micro USB connector on the back, behind a rubber seal, and it uses an HID-based protocol to communicate. I’m halfway through reversing and documenting it of course.

June 13, 2017
Alice Ferrazzi a.k.a. alicef (homepage, bugs)
Moving back to ikiwiki (June 13, 2017, 03:40 UTC)

I was trying to use blogs.gentoo.org/alicef/ Gentoo official blog
based on wordpress.
As far as I could like the draft feature, it had some big drawback.
Most big one I couldn't post any syntax highlighted code.
And wordpress maintenance takes lots of time, in particular managing plugins.
Also because I cannot change Gentoo blog plugins without admin privilege, is bit too much to have to ask every time I have problem with plugins or I need a new one.

So I decided to come back to ikiwiki.
Also in ikiwiki I can make some sort of draft function where post tagged as draft dosen't come up on the blog list.
This is simply done by using this on the blog.mdwn
pages="page(blog/) and !/Discussion and !*/local.css and !tagged(draft)"
That will remove the blog page tagged with draft from the blog view.

For the syntax highlight I used a plugin for pygments.py made by
tylercipriani.com
That you can find here pygments.pm

Pygments.pm output example:

#include <stdio.h>

int main(void) {
    // your code goes here
    return 0;
}

And last one, on Ikiwiki comments, I decided to delegate comments spam to disqus, also if I don't like so much to use a private business plugin, do is job of managing comments well enough.
I also just discovered that disqus is part of Y combinator, the company behind hacker news.

[2017 05] Gentoo activity summary (June 13, 2017, 01:49 UTC)

June 12, 2017
Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)

I've just uploaded Lab::Measurement 3.544 to CPAN. This is our first release containing support for Zurich Instruments equipment, in particular the ZI MFLI digital lock-in amplifier, via Simon's Lab::Zhinst package. Enjoy!

Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)

In my previous rant I have noted that in Ireland it’s next to impossible to buy unlocked phones. Indeed when I went to look for a phone to travel to China at Carphone Warehouse (which at least in the UK is owned by Samsung), while they had plenty of selections for the phones, they all came with contracts.

Contracts are useful for most people, since effectively the carrier is giving you a discount on a phone so that you commit to stay their customer for a certain amount of time. When you do this, they lock you to their network, so that you can’t just switch to another carrier without either giving them their due in subscriptions or paying back the discount they gave you on the phone. In general, I see this approach as reasonable, although it has clearly created a bit of a mess to the market, particularly on the cheaper phone scale.

I have to admit that I have not paid enough attention to this in Ireland up to now simply because I have been using my company-provided phone for most of my day to day travel. Except in China, where it would not be really appropriate. So when I had to go back to Shanghai, I found myself in need of a new phone. I ended up buying one at Argos because they could source one for me by the following day, which is what I needed, and they also had last year’s Sony flagship device (Xperia X) at a decent discount, particularly when compared to the not-much-better Xperia XZ. Alternatively, Amazon would have worked, but that would have taken too long, and the price was actually lower at Argos, for this particular model.

As it is usual for most Android phones, the device started running through a number of system software updates as it was turned up. Indeed, after three cycles the device, which started off with Android 6.0, ended up on 7.0. Not only that, but by now I know that Sony appears to care about the device quite a bit. While they have not updated to 7.1, they have pushed a new system software — I noticed because my phone started downloading it while in Changi airport, in Singapore, while connected to a power pack and the Airport’s WiFi! With this update, the phone is running Android security update as of May 1st 2017.

That made me compare it with the Xperia XA, the locked phone I bought from Three, and that I now managed to unlock. The phone came “branded” by Three Ireland, which for the most part appeared to just mean it splashed their custom logo at boot. Unlocking the phone did not make it update to a newer version, or de-brand itself. But despite being the cheaper version of the X, and theoretically the same generation, it was still stuck on Android 6.0.

Indeed, before the last update, probably released at the same time as the latest Xperia X firmware, the security patch level was reported as April 1st 2016, over a year ago! Fortunately the latest update at least brings it to this year, as now the patch level is January 5th, 2017. As it turns out, even the non-branded versions of the phone is only available up to Android 6.0. At least I should say hat tip to Sony for actually caring about users, at least enough to provide these updates. My Samsung Tab A is at security level 1st June 2016, and it had no software updates in nearly as much time.

There is officially no way to de-brand a phone, but there are of course a number of options out there on how to do that otherwise, although a significant amount of them relied on CyanogenMod and nowadays they will rely on… whatever the name of the new project that forked from that is. I did manage to bring the phone to a clean slate with somewhat sketchy instructions, but as I said even the debranded version did not update to Android 7.0 and I’m not sure if now I would have to manually manage software update. But since the phone does not seem to remember that the phone ever was branded, and there is no Three logo, I guess it might be alright. And since I did not have to unlock the bootloader, I’m relatively safe that the firmware was signed by Sony to begin with.

What I found that is interesting in from using the tool to download Sony’s firmware, is that most of their phones are indeed sold in Ireland, but there is no unbranded Irish firmware. There are, though, a number of unbranded firmwares for other countries, including UK. My (unbranded, unlocked) Xperia X is indeed marked down as a UK firmware. Effectively it looks like that Ireland is once again acting like “UK lite” by not having its own devices, and instead relying on the UK versions. Because who would invest time and energy to cather to the 4.5M people market we have here? Sigh.

June 11, 2017
Sebastian Pipping a.k.a. sping (homepage, bugs)

About rel=noopener (mathiasbynens.github.io)

June 10, 2017
Matthew Thode a.k.a. prometheanfire (homepage, bugs)
Gentoo Puppet Portage Package Provider (June 10, 2017, 05:00 UTC)

Why do this

The previus built in puppet portage package provider (I'm just going to shorten it to PPPP) only supported very simplistic package interactions. Mainly package name (with slot) install and uninstall. This has proven fairly limiting, if you want to install a specific version of a package and lock it down you were forced to call out to exec or editing package.{mask,unmask,keywords} files.

The new provider (which will be built into puppet in 5.0 or puppet-agent-2.0) supports all the package provider attributes.

How do I get this awesome thing

Emerge puppet or puppet-agent with the experimental use flag.

What it can do

You can use the following attributes with the new PPPP.

  • Name - The full package atom works now, using qatom on the backend.
  • ensure - now allowing a package purge as well (CONFIG_PROTECT="-*").
  • install_options - you can now pass options to emerge (--deep or --usepkgonly for example).
  • uninstall_options - just like install_options

Being able to call out specific versions and per package install options will give much greater flexability.

fin

Here is the pull request that upstream puppet merged.

If you have any questions I'm on freenode as prometheanfire.

June 09, 2017
Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
My horrible experience with Three Ireland (June 09, 2017, 22:04 UTC)

I have not ranted about the ineptitude of companies for a while, but this time I have to go back to it. Most of the people who follow me on Twitter are probably already fully aware of what’s going on, so if you want to skip on reading this, feel free.

When I moved to Ireland in 2013, I evaluated quickly the mobile providers available and decided to become a customer of Three Ireland. I was already a customer of Three back in Italy, and they had the same offer here than they had there, which involved the ability to be “Three like at home”, roaming on foreign Three networks for free, using the same allowance of calls and data that you have on your own country. Since my expectation was to go home more often than I actually did, roaming to Three Italy sounded like a good deal.

Fast forward four years, and I ended up having to give up and moved to a new provider altogether. This all precipitated since Three Ireland took effectively four months to fix up my account so I could actually use it, but let’s take one step at a time.

Back in January this year, my Tesco credit card got used fraudulently. Given I have been using Revolut for most of my later trips to the States, I can narrow down where my card was skimmed as one of three places, but it looks like the MIT Coop store is the most likely culprit. This is a different story, and luckily Tesco Bank managed to catch the activity right away, cancelled my card and issued me a new one. This is something I talked about previously.

The main problem was migrating whatever was still attached to that card onto a different one. I managed to convert most of my automated debits onto my Ulster Bank MasterCard (except Vodafone Italy, that’s a longer story), but then I hit a snag. My Three Ireland account was set up to auto-top-up from my Tesco Bank card €20 every month. This was enough to enable the “All you can eat data” offer, which gave me virtually unlimited data in Ireland, UK, Italy, and a few other countries. Unfortunately when I went to try editing my card, their management webapp (My3) started throwing errors at me.

Or rather, not even throwing errors. Whenever I would go to list my payment cards to remove the now-cancelled card, it would send me back to the service’s homepage. So I called them, and I’ll remind you this is January, to ask if they could look into it, and advised I won’t be able to take call because I was about to leave for the USA.

The problem was clearly not solved when I got back to Ireland, and I called them again, told me I would be contacted back from their tech support and they will give me an update. They called me, of course always at awkward times, and the first thing they asked me was for a screenshot of the error I was shown, except I was shown no error. So they had to go back and forth a couple of times with them, both on the phone and over Twitter (both publicly and over direct messages).

At some point during this exchange they asked me for my password. Now, I use LastPass so the password is not actually sensitive information by itself, but you would expect that they would have built something in place where they can act as one of their customers, for debugging purposes, or at least be able to override the password, and just ask me to change it afterwards. Since the second auto-top-up failed and required me to make a manual payment, I decided to give up, and send them screenshot of both the loading page and the following landing page, and send it to them as requested.

Aside note here: the reason why these auto-top-up are important, is that without these, you get charged for every megabyte you use. And you don’t get any notification that your all-you-can-eat expired, you only get a notification after you spent between €5 and €10 in data, as that’s what law requires. So if the auto-top-up failed, you end up just using your credit. Since I used to spend the credit on Google Play instead (particularly to pay for Google Play Music All Access — my, what a mouthful!), this was not cool.

By end of March, when the third auto-top-up failed, and I ended up wasting €15 for not noticing it. I called them again, and I managed to speak to the only person in this whole ordeal who actually treated me decently. She found the ticket closed because they did not receive my screenshot, so she asked me to send them directly to her address and she attached them to the ticket herself. This reopened the ticket, but turned out not to help.

At this point I’ve also topped up the €130 that were required to request an unlock code for my Sony Xperia XA phone, so I decide to request that in parallel while I fight with trying to be able to configure my payment cards. Since the phone is Sony, the unlock code comes directly from them and Three advises is going to take up to 21 working days. When I send the request, I get an email back telling me the unlock request was not successful, and to contact the customer support. Since I was already bothering them on Twitter, I do so there, and they reassure me that they took care of it and sent the request through.

Also, this time I give up and give them my password, too. Which became even funnier, because as I was dictating it to them I go to “ampersand” and they reply “No that’s impossible, it’s not a valid character for the password!” — as it happens it is indeed not a valid character, now. When I set my password it was valid, but now it is not. I found out after they fixed the problem, because of course by then I wanted to change my password, and LastPass generated another one with the & character.

It took another month for them to finally figure out the problem, and another three or four requests for screenshots, despite them knowing my password. And a couple of times asking me to confirm my email address, despite it already being in the system and all. But at least that part got fixed.

Now remember the unlock code request above? 21 working days in most cases mean around a month. So a month after my unlock code request I call them, and they inform me that the 21 working days would expire the next day, a Friday. The reason is to be found in Easter and bank holidays being present, reducing the number of working days in the month. Fair enough, I still ask them what’s going to happen if the 21 days promise is breached, and the guy on the phone denies it is even possible. Of course the day after I got to chat with them again, and they realize that there was no update whatsoever and there should have been at least one.

They decide to request an urgent unlock, since on the Thursday I would be leaving for China, and they promise me the unlock code would be there by Monday. Goes without saying it didn’t work. When I called on Monday they told me that only Sony can provide the unlock, and since it was a long weekend they were not going to answer until at least the day after (May 1st was bank holiday, too). At this point I was pissed and asked to speak with a manager.

Unfortunately the person at the phone was not actually human, but rather was replaced by one of those call center scripts kind of drone and not only kept telling me that they had nothing personal against me, which I did not care if they did, to be honest, but refused to redirect me to a manager when I pointed out that this was ludicrous after fighting four months to get the other problem solved. They kept saying that since the ticket is closed, there was nothing they could talk to me about. They also insisted that since the unlock code hasn’t arrived they couldn’t even offer me a trade-in with an unlocked phone, as that is only available if the unlock code fails to work.

I ended up having to buy myself a new phone, because I could not risk going to China with a locked phone again. Which turned out to be an interesting experience as it looks like in Ireland, the only places to buy unlocked phones are either corner shops selling Chinese phones, or Argos. I ended up buying an Xperia X from Argos, and I’m very happy of the result, although I did not intend to spend that money. But that’s a story for another day, too. Of course the unlock code arrived the day after I bought my new phone, or should I say the day after I gave up on Three Ireland, and moved to Tesco Mobile.

Because at that point, the drone got me so angry that I decided to just spend all of my credit (minus €20 because I hit the usage limit) buying movies and books on Google Play, and when I picked up the phone on Tuesday, I also picked up a SIM for Tesco Mobile. I found out that MNP in Ireland takes less than an hour and just involves a couple of confirmation codes, rather than having to speak with people and fill in forms. And I’m indeed happy on Tesco Mobile right now.

Why am I so riled up? Because I think Three Ireland lost a big opportunity to keep a customer, the moment when I expressed my dissatisfaction with the service and with the lack of unlock code. They could have offered me a trade-in of the current phone. They could have given me the credit I spent because of their issue back. They could have even offered me a new, any new phone, locked to their network, to make it harder for me to leave them. Instead they went the road of saying that since the problem has been solved at all, there was never any problem.

I found this particularly stupid particularly compared to the way Virgin Media and Sky Ireland seem to have it down to practice: when I called Sky to ask them if they had any better offer than Virgin, back when I used TV service, they told me they couldn’t do better broadband, but they would offer me a lower price on the TV package so that I could unbundle it from Virgin. When I called Virgin to remove the TV package (because at the time they were going to increase the monthly fee), they offered to lower their price for a year to make it still more convenient for me.

June 07, 2017
Sven Vermeulen a.k.a. swift (homepage, bugs)
Structuring infrastructural deployments (June 07, 2017, 18:40 UTC)

Many organizations struggle with the all-time increase in IP address allocation and the accompanying need for segmentation. In the past, governing the segments within the organization means keeping close control over the service deployments, firewall rules, etc.

Lately, the idea of micro-segmentation, supported through software-defined networking solutions, seems to defy the need for a segmentation governance. However, I think that that is a very short-sighted sales proposition. Even with micro-segmentation, or even pure point-to-point / peer2peer communication flow control, you'll still be needing a high level overview of the services within your scope.

In this blog post, I'll give some insights in how we are approaching this in the company I work for. In short, it starts with requirements gathering, creating labels to assign to deployments, creating groups based on one or two labels in a layered approach, and finally fixating the resulting schema and start mapping guidance documents (policies) toward the presented architecture.

As always, start with the requirements

From an infrastructure architect point of view, creating structure is one way of dealing with the onslaught in complexity that is prevalent within the wider organizational architecture. By creating a framework in which infrastructural services can be positioned, architects and other stakeholders (such as information security officers, process managers, service delivery owners, project and team leaders ...) can support the wide organization in its endeavor of becoming or remaining competitive.

Structure can be provided through various viewpoints. As such, while creating such framework, the initial intention is not to start drawing borders or creating a complex graph. Instead, look at attributes that one would assign to an infrastructural service, and treat those as labels. Create a nice portfolio of attributes which will help guide the development of such framework.

The following list gives some ideas in labels or attributes that one can use. But be creative, and use experienced people in devising the "true" list of attributes that fits the needs of your organization. Be sure to describe them properly and unambiguously - the list here is just an example, as are the descriptions.

  • tenant identifies the organizational aggregation of business units which are sufficiently similar in areas such as policies (same policies in use), governance (decision bodies or approval structure), charging, etc. It could be a hierarchical aspect (such as organization) as well.
  • location provides insight in the physical (if applicable) location of the service. This could be an actual building name, but can also be structured depending on the size of the environment. If it is structured, make sure to devise a structure up front. Consider things such as regions, countries, cities, data centers, etc. A special case location value could be the jurisdiction, if that is something that concerns the organization.
  • service type tells you what kind of service an asset is. It can be a workstation, a server/host, server/guest, network device, virtual or physical appliance, sensor, tablet, etc.
  • trust level provides information on how controlled and trusted the service is. Consider the differences between unmanaged (no patching, no users doing any maintenance), open (one or more admins, but no active controlled maintenance), controlled (basic maintenance and monitoring, but with still administrative access by others), managed (actively maintained, no privileged access without strict control), hardened (actively maintained, additional security measures taken) and kiosk (actively maintained, additional security measures taken and limited, well-known interfacing).
  • compliance set identifies specific compliance-related attributes, such as the PCI-DSS compliancy level that a system has to adhere to.
  • consumer group informs about the main consumer group, active on the service. This could be an identification of the relationship that consumer group has with the organization (anonymous, customer, provider, partner, employee, ...) or the actual name of the consumer group.
  • architectural purpose gives insight in the purpose of the service in infrastructural terms. Is it a client system, a gateway, a mid-tier system, a processing system, a data management system, a batch server, a reporting system, etc.
  • domain could be interpreted as to the company purpose of the system. Is it for commercial purposes (such as customer-facing software), corporate functions (company management), development, infrastructure/operations ...
  • production status provides information about the production state of a service. Is it a production service, or a pre-production (final testing before going to production), staging (aggregation of multiple changes) or development environment?

Given the final set of labels, the next step is to aggregate results to create a high-level view of the environment.

Creating a layered structure

Chances are high that you'll end up with several attributes, and many of these will have multiple possible values. What we don't want is to end in an N-dimensional infrastructure architecture overview. Sure, it sounds sexy to do so, but you want to show the infrastructure architecture to several stakeholders in your organization. And projecting an N-dimensional structure on a 2-dimensional slide is not only challenging - you'll possibly create a projection which leaves out important details or make it hard to interpret.

Instead, we looked at a layered approach, with each layer handling one or two requirements. The top layer represents the requirement which the organization seems to see as the most defining attribute. It is the attribute where, if its value changes, most of its architecture changes (and thus the impact of a service relocation is the largest).

Suppose for instance that the domain attribute is seen as the most defining one: the organization has strict rules about placing corporate services and commercial services in separate environments, or the security officers want to see the commercial services, which are well exposed to many end users, be in a separate environment from corporate services. Or perhaps the company offers commercial services for multiple tenants, and as such wants several separate "commercial services" environments while having a single corporate service domain.

In this case, part of the infrastructure architecture overview on the top level could look like so (hypothetical example):

Top level view

This also shows that, next to the corporate and commercial interests of the organization, a strong development support focus is prevalent as well. This of course depends on the type of organization or company and how significant in-house development is, but in this example it is seen as a major decisive factor for service positioning.

These top-level blocks (depicted as locations, for those of you using Archimate) are what we call "zones". These are not networks, but clearly bounded areas in which multiple services are positioned, and for which particular handling rules exist. These rules are generally written down in policies and standards - more about that later.

Inside each of these zones, a substructure is made available as well, based on another attribute. For instance, let's assume that this is the architectural purpose. This could be because the company has a requirement on segregating workstations and other client-oriented zones from the application hosting related ones. Security-wise, the company might have a principle where mid-tier services (API and presentation layer exposures) are separate from processing services, and where data is located in a separate zone to ensure specific data access or more optimal infrastructure services.

This zoning result could then be depicted as follows:

Detailed top-level view

From this viewpoint, we can also deduce that this company provides separate workstation services: corporate workstation services (most likely managed workstations with focus on application disclosure, end user computing, etc.) and development workstations (most likely controlled workstations but with more open privileged access for the developer).

By making this separation explicit, the organization makes it clear that the development workstations will have a different position, and even a different access profile toward other services within the company.

We're not done yet. For instance, on the mid-tier level, we could look at the consumer group of the services:

Mid-tier explained

This separation can be established due to security reasons (isolating services that are exposed to anonymous users from customer services or even partner services), but one can also envision this to be from a management point of view (availability requirements can differ, capacity management is more uncertain for anonymous-facing services than authenticated, etc.)

Going one layer down, we use a production status attribute as the defining requirement:

Anonymous user detail

At this point, our company decided that the defined layers are sufficiently established and make for a good overview. We used different defining properties than the ones displayed above (again, find a good balance that fits the company or organization that you're focusing on), but found that the ones we used were mostly involved in existing policies and principles, while the other ones are not that decisive for infrastructure architectural purposes.

For instance, the tenant might not be selected as a deciding attribute, because there will be larger tenants and smaller tenants (which could make the resulting zone set very convoluted) or because some commercial services are offered toward multiple tenants and the organizations' strategy would be to move toward multi-tenant services rather than multiple deployments.

Now, in the zoning structure there is still another layer, which from an infrastructure architecture point is less about rules and guidelines and more about manageability from an organizational point of view. For instance, in the above example, a SAP deployment for HR purposes (which is obviously a corporate service) might have its Enterprise Portal service in the Corporate Services > Mid-tier > Group Employees > Production zone. However, another service such as an on-premise SharePoint deployment for group collaboration might be in Corporate Services > Mid-tier > Group Employees > Production zone as well. Yet both services are supported through different teams.

This "final" layer thus enables grouping of services based on the supporting team (again, this is an example), which is organizationally aligned with the business units of the company, and potentially further isolation of services based on other attributes which are not defining for all services. For instance, the company might have a policy that services with a certain business impact assessment score must be in isolated segments with no other deployments within the same segment.

What about management services

Now, the above picture is missing some of the (in my opinion) most important services: infrastructure support and management services. These services do not shine in functional offerings (which many non-IT people generally look at) but are needed for non-functional requirements: manageability, cost control, security (if security can be defined as a non-functional - let's not discuss that right now).

Let's first consider interfaces - gateways and other services which are positioned between zones or the "outside world". In the past, we would speak of a demilitarized zone (DMZ). In more recent publications, one can find this as an interface zone, or a set of Zone Interface Points (ZIPs) for accessing and interacting with the services within a zone.

In many cases, several of these interface points and gateways are used in the organization to support a number of non-functional requirements. They can be used for intelligent load balancing, providing virtual patching capabilities, validating content against malware before passing it on to the actual services, etc.

Depending on the top level zone, different gateways might be needed (i.e. different requirements). Interfaces for commercial services will have a strong focus on security and manageability. Those for the corporate services might be more integration-oriented, and have different data leakage requirements than those for commercial services.

Also, inside such an interface zone, one can imagine a substructure to take place as well: egress interfaces (for communication that is exiting the zone), ingress interfaces (for communication that is entering the zone) and internal interfaces (used for routing between the subzones within the zone).

Yet, there will also be requirements which are company-wide. Hence, one could envision a structure where there is a company-wide interface zone (with mandatory requirements regardless of the zone that they support) as well as a zone-specific interface zone (with the mandatory requirements specific to that zone).

Before I show a picture of this, let's consider management services. Unlike interfaces, these services are more oriented toward the operational management of the infrastructure. Software deployment, configuration management, identity & access management services, etc. Are services one can put under management services.

And like with interfaces, one can envision the need for both company-wide management services, as well as zone-specific management services.

This information brings us to a final picture, one that assists the organization in providing a more manageable view on its deployment landscape. It does not show the 3rd layer (i.e. production versus non-production deployments) and only displays the second layer through specialization information, which I've quickly made a few examples for (you don't want to make such decisions in a few hours, like I did for this post).

General overview

If the organization took an alternative approach for structuring (different requirements and grouping) the resulting diagram could look quite different:

Alternative general overview

Flows, flows and more flows

With the high-level picture ready, it is not a bad idea to look at how flows are handled in such an architecture. As the interface layer is available on both company-wide level as well as the next, flows will cross multiple zones.

Consider the case of a corporate workstation connecting to a reporting server (like a Cognos or PowerBI or whatever fancy tool is used), and this reporting server is pulling data from a database system. Now, this database system is positioned in the Commercial zone, while the reporting server is in the Corporate zone. The flows could then look like so:

Flow example

Note for the Archimate people: I'm sorry that I'm abusing the flow relation here. I didn't want to create abstract services in the locations and then use the "serves" or "used by" relation and then explaining readers that the arrows are then inverse from what they imagine.

In this picture, the corporate workstation does not connect to the reporting server directly. It goes through the internal interface layer for the corporate zone. This internal interface layer can offer services such as reverse proxies or intelligent load balancers. The idea here is that, if the organization wants, it can introduce additional controls or supporting services in this internal interface layer without impacting the system deployments themselves much.

But the true flow challenge is in the next one, where a processing system connects to a data layer. Here, the processing server will first connect to the egress interface for corporate, then through the company-wide internal interface, toward the ingress interface of the commercial and then to the data layer.

Now, why three different interfaces, and what would be inside it?

On the corporate level, the egress interface could be focusing on privacy controls or data leakage controls. On the company-wide internal interface more functional routing capabilities could be provided, while on the commercial level the ingress could be a database activity monitoring (DAM) system such as a database firewall to provide enhanced auditing and access controls.

Does that mean that all flows need to have at least three gateways? No, this is a functional picture. If the organization agrees, then one or more of these interface levels can have a simple pass-through setup. It is well possible that database connections only connect directly to a DAM service and that such flows are allowed to immediately go through other interfaces.

The importance thus is not to make flows more difficult to provide, but to provide several areas where the organization can introduce controls.

Making policies and standards more visible

One of the effects of having a better structure of the company-wide deployments (i.e. a good zoning solution) is that one can start making policies more clear, and potentially even simple to implement with supporting tools (such as software defined network solutions).

For instance, a company might want to protect its production data and establish that it cannot be used for non-production use, but that there are no restrictions for the other data environments. Another rule could be that web-based access toward the mid-tier is only allowed through an interface.

These are simple statements which, if a company has a good IP plan, are easy to implement - one doesn't need zoning, although it helps. But it goes further than access controls.

For instance, the company might require corporate workstations to be under heavy data leakage prevention and protection measures, while developer workstations are more open (but don't have any production data access). This not only reveals an access control, but also implies particular minimal requirements (for the Corporate > Workstation zone) and services (for the Corporate interfaces).

This zoning structure does not necessarily make any statements about the location (assuming it isn't picked as one of the requirements in the beginning). One can easily extend this to include cloud-based services or services offered by third parties.

Finally, it also supports making policies and standards more realistic. I often see policies that make bold statements such as "all software deployments must be done through the company software distribution tool", but the policies don't consider development environments (production status) or unmanaged, open or controlled deployments (trust level). When challenged, the policy owner might shrug away the comment with "it's obvious that this policy does not apply to our sandbox environment" or so.

With a proper zoning structure, policies can establish the rules for the right set of zones, and actually pin-point which zones are affected by a statement. This is also important if a company has many, many policies. With a good zoning structure, the policies can be assigned with meta-data so that affected roles (such as project leaders, architects, solution engineers, etc.) can easily get an overview of the policies that influence a given zone.

For instance, if I want to position a new management service, I am less concerned about workstation-specific policies. And if the management service is specific for the development environment (such as a new version control system) many corporate or commercially oriented policies don't apply either.

Conclusion

The above approach for structuring an organization is documented here in a high-level manner. It takes many assumptions or hypothetical decisions which are to be tailored toward the company itself. In my company, a different zoning structure is selected, taking into account that it is a financial service provider with entities in multiple countries, handling several thousand of systems and with an ongoing effort to include cloud providers within its infrastructure architecture.

Yet the approach itself is followed in an equal fashion. We looked at requirements, created a layered structure, and finished the zoning schema. Once the schema was established, the requirements for all the zones were written out further, and a mapping of existing deployments (as-is) toward the new zoning picture is on-going. For those thinking that it is just slideware right now - it isn't. Some of the structures that come out of the zoning exercise are already prevalent in the organization, and new environments (due to mergers and acquisitions) are directed to this new situation.

Still, we know we have a large exercise ahead before it is finished, but I believe that it will benefit us greatly, not only from a security point of view, but also clarity and manageability of the environment.

June 06, 2017
Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
Let's have a serious talk about Ads (June 06, 2017, 12:04 UTC)

I have already expressed my opinion on Internet Ads months ago, so I would suggest you all to start reading from that, as I don’t want to have to repeat myself on this particular topic. What I want to talk right now is whether Ads actually work at all for things like my blog, or Autotools Mythbuster.

I’ll start by describing the “monetization” options that I use, and then talk a bit about how much they make, look into the costs and then take a short tour of what else I’m still using.

Right now, there are two sources of ads that I use on this blog: Google AdSense and Amazon Native Ads. Autotools Mythbuster only has AdSense, because the Amazon ads don’t really fit it well. On mobile platform, the only thing you really see is AdSense, as the Native Ads are all the way to the bottom (they don’t do page-level ads as far as I can tell), on desktop you only get to see the Amazon ads.

AdSense pays you both for clicks and for views of the ads on your site, although of course the former gives you significantly higher revenue. Amazon Native Ads only pays you for the items people actually buy, after clicking on the Ads on your site, as it is part of the Amazon Affiliate program. I have started using the Amazon Native Ads as an experiment over April and May, mostly out of curiosity of how they would perform.

The reason why I was curious of the performance is that AdSense, while it used to mostly make decision on which ads to show based on the content of the page, it has been mostly doing remarketing, which appears to creep some people app (I will make no further comments on this), while the idea that Amazon could show ads for content relevant to what I talked about appealed to me. It also turned out to have been an interesting way to find a bug with Amazon’s acapbot, because of course crawlers are hard. As it turns out, the amount of clicks coming from Amazon Native Ads is near zero, and the buying rate is even lower, but it still stacks fairly against AdSense.

To understand what I mean I need to give you some numbers, which is something people don’t seem to be very comfortable with in general. Google AdSense, overall, brings in a gross between €3 and €8 a month, with a very few rare cases in which it went all the way up to a staggering €12. Amazon Affiliates (which as I’ll get to does not only include Native Ads) varies very widely month after month, as it even reaches $50. Do note that all of this is still pre-tax, so you have to just about cut it in half to estimate (it’s actually closer to 35% but that’s a longer story).

I would say that between the two sources, over the past year I probably got around €200 before tax, so call it €120 net. I would have considered that not bad when I was self-employed, but nowadays I have different expectations, too. Getting the actual numbers of how much the domains cost me per year is a bit complicated, as some of those, including flameeyes.eu, are renewed for a block of years at the same time, but I can give you makefile.am as a point of reference (and yes that is an alias for Autotools Mythbuster) as €65.19 a year. The two servers (one storing configuration for easy re-deployment, and the other actually being the server you read the blog from) cost me €7.36/month (for the two of them), and the server I use for actually building stuff costs me €49/month. This already exceeds the gross revenue of the two advertising platforms. Oops.

Of course there is another consideration to make. Leaving aside my personal preferences on lifestyle, and thus where I spend my budget for things like entertainment and food, there is one expense I’m okay with sharing, and that is my current standing donations. My employer not only makes it possible to match donations, but it also makes it very easy to just set up a standing donation that gets taken directly at payroll time. Thanks to making this very simple, I have a standing €90/month donation, spread between Diabetes Ireland, EFF and Internet Archive, and a couple others, that I rotate every few months. And then there are Patreons I subscribe to.

This means that even if I were to just put all the revenue from those ads into donations, it would barely make an impact. Which is why by the time you read this post, my blog will have no ads left on (Autotools Mythbuster will continue for a month or two just so that the next payment is processed and not left in the system). They would be okay to be left there even if they make effectively no money, except that they still require paperwork to be filed for taxes, and this is why I have considered using Amazon Native Ads.

As I said, Amazon Native Ads are part of their Affiliate program, and while you can see in the reports how much revenue is coming from ads versus links, the payments, and thus the tax paperwork, is merged with the rest of the affiliate program. And I have been using affiliate links in a number of places, not just my blog, because in that case there is no drawback: Amazon is not tracking you any more or less than before, and it’s not getting in your way at all. The only places in which I actually let Amazon count the impression (view) is when I’m actually reviewing a product (book, game, hardware device), and even that is fairly minimal, and not any different from me just providing the image and a link to it — except I don’t have to deal with the images and the link breakage connected with that.

There is another reason why I am keeping the affiliates: while they require people to actually spend money to get me anything, they give you a percentages of the selling price of what was sold. Not what you linked to specifically, but what is sold in the session that the user initiated when they clicked on your link. This makes it interesting and ironic when people click on the link to buy Autotools Mythbuster and end up buying instead Autotools by Calcote.

Do I think this experience is universal or generally applicable? I doubt so. My blog does not get that many views anyway, and it went significantly down since I stopped blogging daily, and in particular it went down since I no longer talk about Gentoo that much. I guess part of the problem with that is that beside for people looking for particular information finding me on Google, the vast majority of the people end up on my blog either because they read it already, or follow me on various social media. I have an IFTTT recipe to post most of my entries on Twitter and LinkedIn (not Google+ because there is no way to do that automatically), and I used to have it auto-post new entries that would go to Planet Gentoo on /r/gentoo (much as I hate Reddit myself).

There is also the target audience problem: as most of the people reading this blog are geeks, it is very likely that they will have an adblocker installed, and they do not want to see the ads. I think uBlock may even make the affiliate links broken, while at it. They do block things like Skymiles Shooping and similar affiliate aggregators, because “privacy” (though there is not really any privacy problem there).

So at the end of the day, there is no gain for me to keep ads running, and there is some discomfort for my readers, thus I took them down. If I could, I would love to be able to just have ads for charities only, with no money going to me at all, but reminding people of the importance of donating, even a little bit, to organizations such as Internet Archive, which saved my bacon multiple times as I fixed the links from this blog to other sites, that in the mean time moved without redirects, or just got turned down. But that’s a topic for another day, I think.

June 04, 2017
Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
Chinese Glucometer Review: Sannuo (June 04, 2017, 12:04 UTC)

As I start this draft I’m in Shanghai (mainland China), for a work trip. I have visited Shanghai before, but this time I have some more random time around, and while I was browsing stores with a colleague also visiting the city, we ended up in a big (three, four stories) pharmacy. We would have left right away if it wasn’t that I noticed a big sign advertising a Countour by Bayer glucometer, and I decided to peek around.

I found indeed a whole floor dedicated to health hardware, including, but clearly not limited to, glucometers. It had a long series of Omron hardware, blood pressure measurements, thermometers, etc. And a few desks of glucometer, some from brands that are established and known in the West, and a few I never heard of.

I looked around for the prices, and the meters are more expensive than in Europe, but about on par with the US, between ¥150 and ¥400. I asked if any of the ones they had that I did not recognize would allow downloading to the computer, and they showed me one for ¥258 (around €35), branded Sannuo and manufactured by Sinocare. I decided to buy it for the sake of figuring out how things differ in China for diabetes.

Before getting to the device itself, a few words of the act of buying one. First of all, as it appears to be common in China, or at least in Shanghai, buying something is a bit of a trip around: you select what you want, they send you to the cashier, you pay, and then you go back to the clerk who you chose the item with. If you’re lucky, you’ll be able to pay with card, and if you’re really lucky, they will accept yours. In this case the store accepted my Tesco Visa, but not my Revolut MasterCard, which means it probably costed me closer to €40, but it’s okay.

After I paid for the device, the clerks assisted me not only by giving me the box of the meter, but configuring it up, particularly date and time. You’d think this would be obvious to do (and it is) but one of the things that kept surprising me in Shanghai is that every time you buy something, the seller will configure it for you, and make sure it works at least to a point, the same was true when I bought a SIM card at the airport. They did not ask me to take blood there and then, but they showed me how to set up the date and time.

Finally, they gave me a box of two bottles of strips, told me there were two bottles in the box, and even warned me to use them one bottle at a time (valid warning for glucometers’ reactive strips). And just to make a point here: only one of the two clerks who assisted me spoke any English, and even she didn’t speak it very well. They still did quite a bit to make me understand and explain how to use it.

Now to go back to the meter itself, the €40 got me a fairly clunky meter, a box of lancets and 50 test strips, which declare themselves having a fairly wide range (from 1.1 mmol/l). In the box with the device came the usual (by now) carrying case, and a lancing device. The lancets appear to be “standard” or at least as close as that word as can possibly be used for lancets and lancing devices.

What became very obvious both on the box of the meter, and on the wall ads of all the other meters, is that China, like the UK and Ireland, use mmol/l measurement for blood sugar. I honestly thought that was just a UK (and Ireland, Australia) thing, but clearly it is much more common. Since I’m writing this before getting back to Europe, I cannot tell whether the meter uses this in the wire protocol or, like all the western meters, uses mg/dL internally.

The meter itself feels clunky and it’s fairly big, just shy of the size of an Accu-Chek Mobile. It uses two AAA batteries, and that has only three buttons: power and up/down arrows. The display is a monochrome LCD, but it also has two LEDs, red and green, to tell you whether you’re in range or not. Oh, and it speaks.

It felt funny when I arrived at the hotel and tried it with a blood sample (it seems consistent with the variation of other meters), and it started announcing… something. I don’t speak Chinese so I have not understood anything, I should probably start Google Translate next time I try it. Part of the reason why this feels funny is because it reminded me of the i-ching calculator from Dirk Gently: The Long Dark Tea-Time of the Soul radio series, which was effectively a calculator that spoke the results out loud. I did not make much of a parallel until this, but it then reminded me of the devices in the taxis I took last year, too.

Unfortunately even though I’m now at home, I have not been able to start on the reverse engineering, because I can’t seem to get the device to work on my laptop. According to Linux, the device is not accepting the assigned address. I should wire it up to the proper logic analyzer for that.

June 03, 2017
Matthew Thode a.k.a. prometheanfire (homepage, bugs)
Gentoo portage templates (June 03, 2017, 05:00 UTC)

Why do this

Gentoo is known for being somewhat complex to manage, making clusters of gentoo machines even more complex in most scenarios. Using the following methods the configuration becomes easier.

By the end of this you should be able to have a default hiera configuration for Gentoo while still being able to override it for specific use cases. What makes the method I chose particularly powerful is the ability to delete default vales entirely, not just setting them to something else.

Most of these methods came from my experience with chef that I thought would apply well to other config engines. While some don't like shoving logic to the configuration template engine, I'm open to suggestions.

Requirements

  • Puppet 4.x or puppet-agent with hiera support.
  • Puppet's stdlib installed (specifically for delete_values stuff).
  • (optional) use puppetserver instead of running this oneoff.
  • Hiera configured to use the following configuration.

Hiera config

:merge_behavior: deeper
:deep_merge_options:
  :merge_hash_arrays: true

Basic Setup

  • Convert the common portage configuraitons to templates.
  • Convert the data in those templates to a datastructure.
  • Use hiera to write the defaults / node overrides.
  • Call the templates via a puppet module.

Datastructures

The easiest way of explaing how this works is to say that the only data stored in the 'deepest' value is ever going to be True or False. The reason for this is a because using deep_merge in hiera is an additive process and we need a flag to remove things further down the line.

The datastructure itself is fairly simple, here is an excerpt from my setup.

make_conf:
  emerge_default_opts:
    "--quiet-build": true
    "--changed-use": true

If I wanted to disable --quiet-build down the line you would just set the value to False in a higher precidence (the specific node config instead of the general location.

make_conf:
  emerge_default_opts:
    "--quiet-build": false

Configuration Templates

The templates themselves are epp based, not erb (the old method).

package.keywords

For this one I'll also supply how I auto-set the right archetecture, works for amd64 at least.

Hiera data

"app-admin/paxtest ~%{facts.architecture}": true

Template

<%- |$packages| -%>
# THIS FILE WAS GENERATED BY PUPPET, CHANGES WILL BE OVERWRITTEN

<%- keys(delete_values($packages, false)).each |$package| { -%>
<%= "$package" %>
<%- } -%>

This one is the simplest, if a value for the key (paxtest in this case) is set to false, don't use it, the remaining keys are then set as plan text.

package.use

<%- |$packages| -%>
# THIS FILE WAS GENERATED BY PUPPET, CHANGES WILL BE OVERWRITTEN

<%- keys(delete_values($packages, false)).each |$package| { -%>
  <%- if ! empty(keys(delete_values($packages[$package], false))) { -%>
<%= "$package" %> <%= join(keys(delete_values($packages[$package], false)), ' ') %>
  <%- } -%>
<%- } -%>

This one is fairly straight forward as well, for each package that isn't disabled, if there are keys for the package (signifying use flags, needed because we remove the unset flags) then set them. This combines the flags set from all levels in hiera.

make.conf

This will be the most complicated one, but it's also likely to be the most important. I'll explain a bit about it after the paste.

<%- |$config| -%>
# THIS FILE WAS GENERATED BY PUPPET, CHANGES WILL BE OVERWRITTEN

CFLAGS="<%= join(keys(delete_values($config['cflags'], false)), ' ') %>"
CXXFLAGS="<%= join(keys(delete_values($config['cxxflags'], false)), ' ') %>"
CHOST="<%= $config['chost'] %>"
MAKEOPTS="<%= join(keys(delete_values($config['makeopts'], false)), ' ') %>"
CPU_FLAGS_X86="<%= join(keys(delete_values($config['cpu_flags_x86'], false)), ' ') %>"
ABI_X86="<%= join(keys(delete_values($config['abi_x86'], false)), ' ') %>"

USE="<%= join(keys(delete_values($config['use'], false)), ' ') %>"

GENTOO_MIRRORS="<%= join(keys(delete_values($config['gentoo_mirrors'], false)), ' ') %>"
<% if has_key($config, 'portage_binhost') { -%>
  <%- if $config['portage_binhost'] != false { -%>
PORTAGE_BINHOST="<%= $config['portage_binhost'] %>"
  <%- } -%>
<% } -%>

FEATURES="<%= join(keys(delete_values($config['features'], false)), ' ') %>"
EMERGE_DEFAULT_OPTS="<%= join(keys(delete_values($config['emerge_default_opts'], false)), ' ') %>"
PKGDIR="<%= $config['pkgdir'] %>"
PORT_LOGDIR="<%= $config['port_logdir'] %>"
PORTAGE_GPG_DIR="<%= $config['portage_gpg_dir'] %>"
PORTAGE_GPG_KEY='<%= $config['portage_gpg_key'] %>'

GRUB_PLATFORMS="<%= join(keys(delete_values($config['grub_platforms'], false)), ' ') %>"
LINGUAS="<%= join(keys(delete_values($config['linguas'], false)), ' ') %>"
L10N="<%= join(keys(delete_values($config['l10n'], false)), ' ') %>"

PORTAGE_ELOG_CLASSES="<%= join(keys(delete_values($config['portage_elog_classes'], false)), ' ') %>"
PORTAGE_ELOG_SYSTEM="<%= join(keys(delete_values($config['portage_elog_system'], false)), ' ') %>"
PORTAGE_ELOG_MAILURI="<%= $config['portage_elog_mailuri'] %>"
PORTAGE_ELOG_MAILFROM="<%= $config['portage_elog_mailfrom'] %>"

<% if has_key($config, 'accept_licence') { -%>
ACCEPT_LICENSE="<%= join(keys(delete_values($config['accept_licence'], false)), ' ') %>"
<%- } -%>
POLICY_TYPES="<%= join(keys(delete_values($config['policy_types'], false)), ' ') %>"
PAX_MARKINGS="<%= join(keys(delete_values($config['pax_markings'], false)), ' ') %>"

USE_PYTHON="<%= join(keys(delete_values($config['use_python'], false)), ' ') %>"
PYTHON_TARGETS="<%= join(keys(delete_values($config['python_targets'], false)), ' ') %>"
RUBY_TARGETS="<%= join(keys(delete_values($config['ruby_targets'], false)), ' ') %>"
PHP_TARGETS="<%= join(keys(delete_values($config['php_targets'], false)), ' ') %>"

<% if has_key($config, 'source') { -%>
source <%= join(keys(delete_values($config['source'], false)), ' ') %>
<%- } -%>

The basic idea of this is that you pass in the full make.conf datastructre you will generate as a single variable. Everything else is pulled (or elemated from that).

Each option that is selected already has all the options merged, but this could mean both the disabled versions of a given value could be still there, this is removed using the delete_values($config['foo'], false) bit.

The puppet module itself

It's fairly easy to call, just make sure the template is in the template location and do it as follows.

file { '/etc/portage/make.conf':
  content => epp('portage/portage-make_conf.epp', {'config' => hiera_hash('portage')['make_conf']})
}

fin

If you have any questions I'm on freenode as prometheanfire.

June 02, 2017
Alexys Jacob a.k.a. ultrabug (homepage, bugs)
Load balancing Hadoop Hive with F5 BIG-IP (June 02, 2017, 12:51 UTC)

In our quest to a highly available HiveServer2, we faced so many problems and a clear lack of documentation when it came to do it with F5 BIG-IP load balancers that I think it’s worth a blog post to help around.

We are using the Cloudera Hadoop distribution but this applies whatever your distribution.

Hive HA configuration

This appears to be well documented at a first glance but the HiveServer2 (HS2) documentation vanished at the time of writing.

Anyway, using Cloudera Manager to set up HS2 HA is not hard but there are a few gotchas that I want to highlight and that you will need to be careful with:

  • As for every Keberos based service, make sure you have a dedicated IP for the HiveServer2 Load Balancer URL and that it’s reverse DNS is setup properly. Else you will get GSSAPI errors.
  • When running a secure cluster with Kerberos, the HiveServer2 Load Balancer URL is to be used as your connection host (obvious) AND in your Kerberos principal connection string (maybe less obvious).

Example beeline connection string before HA:

!connect jdbc:hive2://hive-server:10000/default;principal=hive/_HOST@REALM.COM

and with HA (notice we changed also the _HOST):

!connect jdbc:hive2://ha-hive-fqdn:10000/default;principal=hive/ha-hive-fqdn@REALM.COM

We found out the kerberos principal gotcha the hard way… The reason behind this is that the _HOST is basically a macro that will get resolved to the client host name which will then be used to validate the kerberos ticket. When running in load balanced/HA mode , the actual source IP will be replaced by the load balancer’s IP (SNAT) and the kerberos reverse DNS lookup will then fail!

So if you do not use the HS2 HA URL in the kerberos principal string, you will get Kerberos GSSAPI errors when the load balanding SNAT will be used (see next chapter).

This will require you to update all your jobs using HS2 to reflect these changes before load balancing HS2 with F5 BIG-IP.

Load balancing HiveServer2 with F5

Our pals at Cloudera have brought a good doc for Impala HA with F5 and they instructed we followed it to set up HS2 HA too because they had nothing better.

Kerberos GSSAPI problem

When we applied it the first time and tried to switch to using the F5, all our jobs failed because of the kerberos _HOST principal problem mentioned on the previous chapter. This one is not that hard to find out and debug with a google search and explained on Cloudera community forums.

We then migrated (again) all our jobs to update the principal connection strings before migrating again to the F5 load balancers.

Connection Reset problems

After our next migration to F5 load balancers, we had most of our jobs running well and the Kerberos problems vanished but we faced a new problem: some of our jobs failed with Connection Reset errors on HiveServer2:

java.sql.SQLException: org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset

After some debugging and traffic analysis we found out that the F5 were actually responsible for those connection reset but we struggled to understand why.

It turned out that the Protocol Profile set up on the Virtual Server was the root cause of the problem and specifically its idle timeout setting default of 300s:

Note the Reset on Timeout setting as well which is responsible for the actual Reset packet sent by the F5 to the client.

This could also be proven by the Virtual Server statistics showing an increasing Connection Expires count.

The solution is to create a new Protocol Profile based on the fastL4 with a higher Idle Timeout setting and update our Virtual Server to use this profile instead of the default one.

It seemed sensible in our case to increase the 5 minutes expiration to 1 day, so let’s call our new profile fastL4-24h-idle-timeout:

Then change the Hive Virtual Server configuration to use this Protocol Profile:

You will see no more expired connections on the Virtual Server statistics!

Job design consideration

We could argue whether or not a default 5 minutes idle timeout is reasonable or not for Hive or any other Hadoop component but it is important to point out that the jobs which were affected also had sub-optimal design pattern in the first place. This also explains why most of our jobs (including also long running ones) were not affected.

The affected jobs allowed were Talend jobs where the Hive connection was established at the beginning of the job, used at that time and then the job went on doing other things before using the Hive connection again.

When those in between computation took more than 300s, the remaining of the job failed because the initial Hive connection got reset by the F5:

This is clearly not a good job design for long processing jobs and you should refrain from doing it. Instead open a connection to Hive when you need it, use it and close it properly. Shall you need it later in your job, open a new connection to Hive and use that one.

This will also have the benefit of not keeping open idle connections to Hive itself and favour resources allocation fairness across your jobs.

I hope this will be of help to anyone facing these kind of issues.

May 19, 2017
Hanno Böck a.k.a. hanno (homepage, bugs)

Today the OCSP servers from Let’s Encrypt were offline for a while. This has caused far more trouble than it should have, because in theory we have all the technologies available to handle such an incident. However due to failures in how they are implemented they don’t really work.

We have to understand some background. Encrypted connections using the TLS protocol like HTTPS use certificates. These are essentially cryptographic public keys together with a signed statement from a certificate authority that they belong to a certain host name.

CRL and OCSP – two technologies that don’t work

Certificates can be revoked. That means that for some reason the certificate should no longer be used. A typical scenario is when a certificate owner learns that his servers have been hacked and his private keys stolen. In this case it’s good to avoid that the stolen keys and their corresponding certificates can still be used. Therefore a TLS client like a browser should check that a certificate provided by a server is not revoked.

That’s the theory at least. However the history of certificate revocation is a history of two technologies that don’t really work.

One method are certificate revocation lists (CRLs). It’s quite simple: A certificate authority provides a list of certificates that are revoked. This has an obvious limitation: These lists can grow. Given that a revocation check needs to happen during a connection it’s obvious that this is non-workable in any realistic scenario.

The second method is called OCSP (Online Certificate Status Protocol). Here a client can query a server about the status of a single certificate and will get a signed answer. This avoids the size problem of CRLs, but it still has a number of problems. Given that connections should be fast it’s quite a high cost for a client to make a connection to an OCSP server during each handshake. It’s also concerning for privacy, as it gives the operator of an OCSP server a lot of information.

However there’s a more severe problem: What happens if an OCSP server is not available? From a security point of view one could say that a certificate that can’t be OCSP-checked should be considered invalid. However OCSP servers are far too unreliable. So practically all clients implement OCSP in soft fail mode (or not at all). Soft fail means that if the OCSP server is not available the certificate is considered valid.

That makes the whole OCSP concept pointless: If an attacker tries to abuse a stolen, revoked certificate he can just block the connection to the OCSP server – and thus a client can’t learn that it’s revoked. Due to this inherent security failure Chrome decided to disable OCSP checking altogether. As a workaround they have something called CRLsets and Mozilla has something similar called OneCRL, which is essentially a big revocation list for important revocations managed by the browser vendor. However this is a weak workaround that doesn’t cover most certificates.

OCSP Stapling and Must Staple to the rescue?

There are two technologies that could fix this: OCSP Stapling and Must-Staple.

OCSP Stapling moves the querying of the OCSP server from the client to the server. The server gets OCSP replies and then sends them within the TLS handshake. This has several advantages: It avoids the latency and privacy implications of OCSP. It also allows surviving short downtimes of OCSP servers, because a TLS server can cache OCSP replies (they’re usually valid for several days).

However it still does not solve the security issue: If an attacker has a stolen, revoked certificate it can be used without Stapling. The browser won’t know about it and will query the OCSP server, this request can again be blocked by the attacker and the browser will accept the certificate.

Therefore an extension for certificates has been introduced that allows us to require Stapling. It’s usually called OCSP Must-Staple and is defined in https://tools.ietf.org/html/rfc7633 RFC 7633 (although the RFC doesn’t mention the name Must-Staple, which can cause some confusion). If a browser sees a certificate with this extension that is used without OCSP Stapling it shouldn’t accept it.

So we should be fine. With OCSP Stapling we can avoid the latency and privacy issues of OCSP and we can avoid failing when OCSP servers have short downtimes. With OCSP Must-Staple we fix the security problems. No more soft fail. All good, right?

The OCSP Stapling implementations of Apache and Nginx are broken

Well, here come the implementations. While a lot of protocols use TLS, the most common use case is the web and HTTPS. According to Netcraft statistics by far the biggest share of active sites on the Internet run on Apache (about 46%), followed by Nginx (about 20 %). It’s reasonable to say that if these technologies should provide a solution for revocation they should be usable with the major products in that area. On the server side this is only OCSP Stapling, as OCSP Must Staple only needs to be checked by the client.

What would you expect from a working OCSP Stapling implementation? It should try to avoid a situation where it’s unable to send out a valid OCSP response. Thus roughly what it should do is to fetch a valid OCSP response as soon as possible and cache it until it gets a new one or it expires. It should furthermore try to fetch a new OCSP response long before the old one expires (ideally several days). And it should never throw away a valid response unless it has a newer one. Google developer Ryan Sleevi wrote a detailed description of what a proper OCSP Stapling implementation could look like.

Apache does none of this.

If Apache tries to renew the OCSP response and gets an error from the OCSP server – e. g. because it’s currently malfunctioning – it will throw away the existing, still valid OCSP response and replace it with the error. It will then send out stapled OCSP errors. Which makes zero sense. Firefox will show an error if it sees this. This has been reported in 2014 and is still unfixed.

Now there’s an option in Apache to avoid this behavior: SSLStaplingReturnResponderErrors. It’s defaulting to on. If you switch it off you won’t get sane behavior (that is – use the still valid, cached response), instead Apache will disable Stapling for the time it gets errors from the OCSP server. That’s better than sending out errors, but it obviously makes using Must Staple a no go.

It gets even crazier. I have set this option, but this morning I still got complaints that Firefox users were seeing errors. That’s because in this case the OCSP server wasn’t sending out errors, it was completely unavailable. For that situation Apache has a feature that will fake a tryLater error to send out to the client. If you’re wondering how that makes any sense: It doesn’t. The “tryLater” error of OCSP isn’t useful at all in TLS, because you can’t try later during a handshake which only lasts seconds.

This is controlled by another option: SSLStaplingFakeTryLater. However if we read the documentation it says “Only effective if SSLStaplingReturnResponderErrors is also enabled.” So if we disabled SSLStapingReturnResponderErrors this shouldn’t matter, right? Well: The documentation is wrong.

There are more problems: Apache doesn’t get the OCSP responses on startup, it only fetches them during the handshake. This causes extra latency on the first connection and increases the risk of hitting a situation where you don’t have a valid OCSP response. Also cached OCSP responses don’t survive server restarts, they’re kept in an in-memory cache.

There’s currently no way to configure Apache to handle OCSP stapling in a reasonable way. Here’s the configuration I use, which will at least make sure that it won’t send out errors and cache the responses a bit longer than it does by default:

SSLStaplingCache shmcb:/var/tmp/ocsp-stapling-cache/cache(128000000)
SSLUseStapling on
SSLStaplingResponderTimeout 2
SSLStaplingReturnResponderErrors off
SSLStaplingFakeTryLater off
SSLStaplingStandardCacheTimeout 86400


I’m less familiar with Nginx, but from what I hear it isn’t much better either. According to https://blog.crashed.org/nginx-stapling-busted/ this blogpost it doesn’t fetch OCSP responses on startup and will send out the first TLS connections without stapling even if it’s enabled. Here’s a blog post that recommends to work around this by connecting to all configured hosts after the server has started.

To summarize: This is all a big mess. Both Apache and Nginx have OCSP Stapling implementations that are essentially broken. As long as you’re using either of those then enabling Must-Staple is a reliable way to shoot yourself in the foot and get into trouble. Don’t enable it if you plan to use Apache or Nginx.

Certificate revocation is broken. It has been broken since the invention of SSL and it’s still broken. OCSP Stapling and OCSP Must-Staple could fix it in theory. But that would require working and stable implementations in the most widely used server products.

May 18, 2017
Sven Vermeulen a.k.a. swift (homepage, bugs)
Matching MD5 SSH fingerprint (May 18, 2017, 16:20 UTC)

Today I was attempting to update a local repository, when SSH complained about a changed fingerprint, something like the following:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
SHA256:p4ZGs+YjsBAw26tn2a+HPkga1dPWWAWX+NEm4Cv4I9s.
Please contact your system administrator.
Add correct host key in /home/user/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /home/user/.ssh/known_hosts:9
ECDSA host key for 192.168.56.101 has changed and you have requested strict checking.
Host key verification failed.

May 16, 2017
Alexys Jacob a.k.a. ultrabug (homepage, bugs)

In my previous blog post, I demonstrated how to use the PIV feature of a Yubikey to add a 2nd factor authentication to SSH.

Careful readers such as Grzegorz Kulewski pointed out that using the GPG capability of the Yubikey was also a great, more versatile and more secure option on the table (I love those community insights):

  • GPG keys and subkeys are indeed more flexible and can be used for case-specific operations (signing, encryption, authentication)
  • GPG is more widely used and one could use their Yubikey smartcard for SSH, VPN, HTTP auth and code signing
  • The Yubikey 4 GPG feature supports 4096 bit keys (limited to 2048 for PIV)

While I initially looked at the GPG feature, its apparent complexity got me to discard it for my direct use case (SSH). But I couldn’t resist the good points of Grzegorz and here I got back into testing it. Thank you again Grzegorz for the excuse you provided 😉

So let’s get through with the GPG feature of the Yubikey to authenticate our SSH connections. Just like the PIV method, this one has the  advantage to allow a 2nd factor authentication while using the public key authentication mechanism of OpenSSH and thus does not need any kind of setup on the servers.

Method 3 – SSH using Yubikey and GPG

Acknowledgement

The first choice you have to make is to decide whether you allow your master key to be stored on the Yubikey or not. This choice will be guided by how you plan to use and industrialize your usage of the GPG based SSH authentication.

Consider this to choose whether to store the master key on the Yubikey or not:

  • (con) it will not allow the usage of the same GPG key on multiple Yubikeys
  • (con) if you loose your Yubikey, you will have to revoke your entire GPG key and start from scratch (since the secret key is stored on the Yubikey)
  • (pro) by storing everything on the Yubikey, you won’t necessary have to have an offline copy of your master key (and all the process that comes with it)
  • (pro) it is easier to generate and store everything on the key and is then a good starting point for new comers or rare GPG users

Because I want to demonstrate and enforce the most straightforward way of using it, I will base this article on generating and storing everything on a Yubikey 4. You can find useful links at the end of the article pointing to reference on how to do it differently.

Tools installation

For this to work, we will need some tools on our local machine to setup our Yubikey correctly.

Gentoo users should install those packages:

emerge -av dev-libs/opensc sys-auth/ykpers app-crypt/ccid sys-apps/pcsc-tools app-crypt/gnupg

Gentoo users should also allow the pcscd service to be hotplugged (started automatically upon key insertion) by modifying their /etc/rc.conf and having:

rc_hotplug="pcscd"

Yubikey setup

The idea behind the Yubikey setup is to generate and store the GPG keys directly on our Yubikey and to secure them via a PIN code (and an admin PIN code).

  • default PIN code: 123456
  • default admin PIN code: 12345678

First, insert your Yubikey and let’s change its USB operating mode to OTP+U2F+CCID with MODE_FLAG_EJECT flag.

ykpersonalize -m86
Firmware version 4.3.4 Touch level 783 Program sequence 3

The USB mode will be set to: 0x86

Commit? (y/n) [n]: y

NOTE: if you have an older version of Yubikey (before Sept. 2014), use -m82 instead.

Then, we can generate a new GPG key on the Yubikey. Let’s open the smartcard for edition.

gpg --card-edit --expert

Reader ...........: Yubico Yubikey 4 OTP U2F CCID (0005435106) 00 00
Application ID ...: A7560001240102010006054351060000
Version ..........: 2.1
Manufacturer .....: Yubico
Serial number ....: 75435106
Name of cardholder: [not set]
Language prefs ...: [not set]
Sex ..............: unspecified
URL of public key : [not set]
Login data .......: [not set]
Signature PIN ....: forced
Key attributes ...: rsa2048 rsa2048 rsa2048
Max. PIN lengths .: 127 127 127
PIN retry counter : 3 0 3
Signature counter : 0
Signature key ....: [none]
Encryption key....: [none]
Authentication key: [none]
General key info..: [none]

Then switch to admin mode.

gpg/card> admin
Admin commands are allowed

We can start generating the Signature, Encryption and Authentication keys on the Yubikey. During the process, you will be prompted alternatively for the admin PIN and PIN.

gpg/card> generate 
Make off-card backup of encryption key? (Y/n) 

Please note that the factory settings of the PINs are
   PIN = '123456'     Admin PIN = '12345678'
You should change them using the command --change-pin

I advise you say Yes to the off-card backup of the encryption key.

Yubikey 4 users can choose a 4096 bits key, let’s go for it for every key type.

What keysize do you want for the Signature key? (2048) 4096
The card will now be re-configured to generate a key of 4096 bits
Note: There is no guarantee that the card supports the requested size.
      If the key generation does not succeed, please check the
      documentation of your card to see what sizes are allowed.
What keysize do you want for the Encryption key? (2048) 4096
The card will now be re-configured to generate a key of 4096 bits
What keysize do you want for the Authentication key? (2048) 4096
The card will now be re-configured to generate a key of 4096 bits

Then you’re asked for the expiration of your key. I choose 1 year but it’s up to you (leave 0 for no expiration).

Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 1y
Key expires at mer. 15 mai 2018 21:42:42 CEST
Is this correct? (y/N) y

Finally you give GnuPG details about your user ID and you will be prompted for a passphrase (make it strong).

GnuPG needs to construct a user ID to identify your key.

Real name: Ultrabug
Email address: ultrabug@nospam.com
Comment: 
You selected this USER-ID:
    "Ultrabug <ultrabug@nospam.com>"

Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.

If you chose to make an off-card backup of your key, you will also get notified of its location as well the revocation certificate.

gpg: Note: backup of card key saved to '/home/ultrabug/.gnupg/sk_8E407636C9C32C38.gpg'
gpg: key 22A73AED8E766F01 marked as ultimately trusted
gpg: revocation certificate stored as '/home/ultrabug/.gnupg/openpgp-revocs.d/A1580FD98C0486D94C1BE63B22A73AED8E766F01.rev'
public and secret key created and signed.

Make sure to store that backup in a secure and offline location.

You can verify that everything went good and take this chance to note the public key ID.

gpg/card> verify

Reader ...........: Yubico Yubikey 4 OTP U2F CCID (0001435106) 00 00
Application ID ...: A7560001240102010006054351060000
Version ..........: 2.1
Manufacturer .....: Yubico
Serial number ....: 75435106
Name of cardholder: [not set]
Language prefs ...: [not set]
Sex ..............: unspecified
URL of public key : [not set]
Login data .......: [not set]
Signature PIN ....: forced
Key attributes ...: rsa4096 rsa4096 rsa4096
Max. PIN lengths .: 127 127 127
PIN retry counter : 3 0 3
Signature counter : 4
Signature key ....: A158 0FD9 8C04 86D9 4C1B E63B 22A7 3AED 8E76 6F01
 created ....: 2017-05-16 20:43:17
Encryption key....: E1B6 7009 907D 1D94 B200 37D7 8E40 7636 C9C3 2C38
 created ....: 2017-05-16 20:43:17
Authentication key: AAED AB8E E055 41B2 EFFF 62A4 164F 873A 75D2 AD6B
 created ....: 2017-05-16 20:43:17
General key info..: pub rsa4096/22A73AED8E766F01 2017-05-16 Ultrabug <ultrabug@nospam.com>
sec> rsa4096/22A73AED8E766F01 created: 2017-05-16 expires: 2018-05-16
 card-no: 0001 05435106
ssb> rsa4096/164F873A75D2AD6B created: 2017-05-16 expires: 2018-05-16
 card-no: 0001 05435106
ssb> rsa4096/8E407636C9C32C38 created: 2017-05-16 expires: 2018-05-16
 card-no: 0001 05435106

You’ll find the public key ID on the “General key info” line (22A73AED8E766F01):

General key info..: pub rsa4096/22A73AED8E766F01 2017-05-16 Ultrabug <ultrabug@nospam.com>

Quit the card edition.

gpg/card> quit

It is then convenient to upload your public key to a key server, whether public or on your own web server (you can also keep it to be used and imported directly from an USB stick).

Export the public key:

gpg --armor --export 22A73AED8E766F01 > 22A73AED8E766F01.asc

Then upload it to your http server or a public server (needed if you want to be able to easily use the key on multiple machines):

# Upload it to your http server
scp 22A73AED8E766F01.asc user@server:public_html/static/22A73AED8E766F01.asc

# OR upload it to a public keyserver
gpg --keyserver hkps://hkps.pool.sks-keyservers.net --send-key 22A73AED8E766F01

Now we can finish up the Yubikey setup. Let’s edit the card again:

gpg --card-edit --expert

Reader ...........: Yubico Yubikey 4 OTP U2F CCID (0001435106) 00 00
Application ID ...: A7560001240102010006054351060000
Version ..........: 2.1
Manufacturer .....: Yubico
Serial number ....: 75435106
Name of cardholder: [not set]
Language prefs ...: [not set]
Sex ..............: unspecified
URL of public key : [not set]
Login data .......: [not set]
Signature PIN ....: forced
Key attributes ...: rsa4096 rsa4096 rsa4096
Max. PIN lengths .: 127 127 127
PIN retry counter : 3 0 3
Signature counter : 4
Signature key ....: A158 0FD9 8C04 86D9 4C1B E63B 22A7 3AED 8E76 6F01
 created ....: 2017-05-16 20:43:17
Encryption key....: E1B6 7009 907D 1D94 B200 37D7 8E40 7636 C9C3 2C38
 created ....: 2017-05-16 20:43:17
Authentication key: AAED AB8E E055 41B2 EFFF 62A4 164F 873A 75D2 AD6B
 created ....: 2017-05-16 20:43:17
General key info..: pub rsa4096/22A73AED8E766F01 2017-05-16 Ultrabug <ultrabug@nospam.com>
sec> rsa4096/22A73AED8E766F01 created: 2017-05-16 expires: 2018-05-16
 card-no: 0001 05435106
ssb> rsa4096/164F873A75D2AD6B created: 2017-05-16 expires: 2018-05-16
 card-no: 0001 05435106
ssb> rsa4096/8E407636C9C32C38 created: 2017-05-16 expires: 2018-05-16
 card-no: 0001 05435106
gpg/card> admin

Make sure that the Signature PIN is forced to request that your PIN is entered when your key is used. If it is listed as “not forced”, you can enforce it by entering the following command:

gpg/card> forcesig

It is also good practice to set a few more settings on your key.

gpg/card> login
Login data (account name): ultrabug

gpg/card> lang
Language preferences: en

gpg/card> name 
Cardholder's surname: Bug
Cardholder's given name: Ultra

Now we need to setup the PIN and admin PIN on the card.

gpg/card> passwd 
gpg: OpenPGP card no. A7560001240102010006054351060000 detected

1 - change PIN
2 - unblock PIN
3 - change Admin PIN
4 - set the Reset Code
Q - quit

Your selection? 1
PIN changed.

1 - change PIN
2 - unblock PIN
3 - change Admin PIN
4 - set the Reset Code
Q - quit

Your selection? 3
PIN changed.

1 - change PIN
2 - unblock PIN
3 - change Admin PIN
4 - set the Reset Code
Q - quit

Your selection? Q

If you uploaded your public key on your web server or a public server, configure it on the key:

gpg/card> url
URL to retrieve public key: http://ultrabug.fr/keyserver/22A73AED8E766F01.asc

gpg/card> quit

Now we can quit the gpg card edition, we’re done on the Yubikey side!

gpg/card> quit

SSH client setup

This is the setup on the machine(s) where you will be using the GPG key. The idea is to import your key from the card to your local keyring so you can use it on gpg-agent (and its ssh support).

You can skip the fetch/import part below if you generated the key on the same machine than you are using it. You should see it listed when executing gpg -k.

Plug-in your Yubikey and load the smartcard.

gpg --card-edit --expert

Then fetch the key from the URL to import it to your local keyring.

gpg/card> fetch

Then you’re done on this part, exit gpg and update/display& your card status.

gpg/card> quit

gpg --card-status

You can verify the presence of the key in your keyring:

gpg -K
sec>  rsa4096 2017-05-16 [SC] [expires: 2018-05-16]
      A1580FD98C0486D94C1BE63B22A73AED8E766F01
      Card serial no. = 0001 05435106
uid           [ultimate] Ultrabug <ultrabug@nospam.com>
ssb>  rsa4096 2017-05-16 [A] [expires: 2018-05-16]
ssb>  rsa4096 2017-05-16 [E] [expires: 2018-05-16]

Note the “Card serial no.” showing that the key is actually stored on a smartcard.

Now we need to configure gpg-agent to enable ssh support, edit your ~/.gnupg/gpg-agent.conf configuration file and make sure that the enable-ssh-support is present:

default-cache-ttl 7200
max-cache-ttl 86400
enable-ssh-support

Then you will need to update your ~/.bashrc file to automatically start gpg-agent and override ssh-agent’s environment variables. Add this at the end of your ~/.bashrc file (or equivalent).

# start gpg-agent if it's not running
# then override SSH authentication socket to use gpg-agent
pgrep -l gpg-agent &>/dev/null
if [[ "$?" != "0" ]]; then
 gpg-agent --daemon &>/dev/null
fi
SSH_AUTH_SOCK=/run/user/$(id -u)/gnupg/S.gpg-agent.ssh
export SSH_AUTH_SOCK

To simulate a clean slate, unplug your card then kill any running gpg-agent:

killall gpg-agent

Then plug back your card and source your ~/.bashrc file:

source ~/.bashrc

Your GPG key is now listed in you ssh identities!

ssh-add -l
4096 SHA256:a4vsJM6Sw1Rt8orvPnI8nvNUwHbRQ67ylnoTxruozK9 cardno:000105435106 (RSA)

You will now be able to get the SSH public key hash to copy to your remote servers using:

ssh-add -L
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCVDq24Ld/bOzc3yNnY6fF7FNfZnb6wRVdN2xMo1YiA5pz20y+2P1ynq0rb6l/wsSip0Owq4G6fzaJtT1pBUAkEJbuMvZBrbYokb2mZ78iFZyzAkCT+C9YQtvPClFxSnqSL38jBpunZuuiFfejM842dWdMNK3BTcjeTQdTbmY+VsVOw7ppepRh7BWslb52cVpg+bHhB/4H0BUUd/KHZ5sor0z6e1OkqIV8UTiLY2laCCL8iWepZcE6n7MH9KItqzX2I9HVIuLCzhIib35IRp6d3Whhw3hXlit4/irqkIw0g7s9jC8OwybEqXiaeQbmosLromY3X6H8++uLnk7eg9RtCwcWfDq0Qg2uNTEarVGgQ1HXFr8WsjWBIneC8iG09idnwZNbfV3ocY5+l1REZZDobo2BbhSZiS7hKRxzoSiwRvlWh9GuIv8RNCDss9yNFxNUiEIi7lyskSgQl3J8znDXHfNiOAA2X5kVH0s6AQx4hQP9Dl1X2Em4zOz+yJEPFnAvE+XvBys1yuUPq1c3WKMWzongZi8JNu51Yfj7Trm74hoFRn+CREUNpELD9JignxlvkoKAJpWVLdEu1bxJ7jh7kcMQfVEflLbfkEPLV4nZS4sC1FJR88DZwQvOudyS69wLrF3azC1Gc/fTgBiXVVQwuAXE7vozZk+K4hdrGq4u7Gw== cardno:000105435106

This is what ends up in ~/.ssh/authorized_keys on your servers.

When connecting to your remote server, you will be prompted for the PIN!

Conclusion

Using the GPG feature of your Yubikey is very convenient and versatile. Even if it is not that hard after all, it is interesting and fair to note that the PIV method is indeed more simple to implement.

When you need to maintain a large number of security keys in an organization and that their usage is limited to SSH, you will be inclined to stick with PIV if 2048 bits keys are acceptable for you.

However, for power users and developers, usage of GPG is definitely something you need to consider for its versatility and enhanced security.

Useful links

You may find those articles useful to setup your GPG key differently and avoid having the secret key tied to your Yubikey.

Sebastian Pipping a.k.a. sping (homepage, bugs)

Hi!

When I started fetchcommandwrapper about 6 years ago it was a proof of concept: It plugged into portage replacing wget for downloads, facilitating ${GENTOO_MIRRORS} and aria2 to both download faster and distribute loads across mirrors. A hack for sure, but with some potential.

Back then public interest was non-existent, fetchcommandwrapper had some issues — e.g. metadata.xsd downloads failed and some sites rejected downloading before it made aria2 dress like wget — and I stopped using it myself, eventually.

With the latest bug reports, bugfixes and release of version 0.8 in Gentoo, fetchcommandwrapper is ready for general use now. To give it a shot, you emerge app-portage/fetchcommandwrapper and append source /usr/share/fetchcommandwrapper/make.conf to /etc/portage/make.conf. Done.

If you have extra options that you would like to pass to aria2c, put them in ${FETCHCOMMANDWRAPPER_EXTRA}, or ${FETCHCOMMANDWRAPPER_OPTIONS} for fetchcommendwrapper itself; for example

FETCHCOMMANDWRAPPER_OPTIONS="--link-speed=600000"

tells fetchcommandwrapper that my download link has 600KB/s only and makes aria2 in turn drop connections to mirrors that cannot keep up with at least a third of that, so that faster mirrors get a chance to take their place.

For non-ebuild bugs, feel free to use https://github.com/gentoo/fetchcommandwrapper/issues to report.

Best, Sebastian

May 12, 2017
Alexys Jacob a.k.a. ultrabug (homepage, bugs)

In my previous blog post, I demonstrated how to use a Yubikey to add a 2nd factor (2FA) authentication to SSH using pam_ssh and pam_yubico.

In this article, I will go further and demonstrate another method using Yubikey’s Personal Identity Verification (PIV) capability.

This one has the huge advantage to allow a 2nd factor authentication while using the public key authentication mechanism of OpenSSH and thus does not need any kind of setup on the servers.

Method 2 – SSH using Yubikey and PIV

Yubikey 4 and NEO also act as smartcards supporting the PIV standard which allows you to store a private key on your security key through PKCS#11. This is an amazing feature which is also very good for our use case.

Tools installation

For this to work, we will need some tools on our local machines to setup our Yubikey correctly.

Gentoo users should install those packages:

emerge dev-libs/opensc sys-auth/ykpers sys-auth/yubico-piv-tool sys-apps/pcsc-lite app-crypt/ccid sys-apps/pcsc-tools sys-auth/yubikey-personalization-gui

Gentoo users should also allow the pcscd service to be hotplugged (started automatically upon key insertion) by modifying their /etc/rc.conf and having:

rc_hotplug="pcscd"

Yubikey setup

The idea behind the Yubikey setup is to generate and store a private key in our Yubikey and to secure it via a PIN code.

First, insert your Yubikey and let’s change its USB operating mode to OTP+CCID.

ykpersonalize -m2
Firmware version 4.3.4 Touch level 783 Program sequence 3

The USB mode will be set to: 0x2

Commit? (y/n) [n]: y

Then, we will create a new management key:

key=`dd if=/dev/random bs=1 count=24 2>/dev/null | hexdump -v -e '/1 "%02X"'`
echo $key
D59E46FE263DDC052A409C68EB71941D8DD0C5915B7C143A

Replace the default management key (if prompted, copy/paste the key printed above):

yubico-piv-tool -a set-mgm-key -n $key --key 010203040506070801020304050607080102030405060708

Then change the default PIN code and PUK code of your Yubikey

yubico-piv-tool -a change-pin -P 123456 -N <NEW PIN>

yubico-piv-tool -a change-puk -P 12345678 -N <NEW PUK>

Now that your Yubikey is secure, let’s proceed with the PCKS#11 certificate generation. You will be prompted for your management key that you generated before.

yubico-piv-tool -s 9a -a generate -o public.pem -k

Then create a self-signed certificate (only used for libpcks11) and import it in the Yubikey:

yubico-piv-tool -a verify-pin -a selfsign-certificate -s 9a -S "/CN=SSH key/" -i public.pem -o cert.pem
yubico-piv-tool -a import-certificate -s 9a -i cert.pem

Here you are! You can now export your public key to use with OpenSSH:

ssh-keygen -D opensc-pkcs11.so -e
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCWtqI37jwxYMJ9XLq9VwHgJlhZViPVAGIUfMm8SAlfs6cka4Cj570lkoGK04r8JAVJFy/iKfhGpL9N9XuartfIoq6Cg/6Qvg3REupuqs51V2cBaC/gnWIQ7qZqlzBulvcOvzNfHFD/lX42J58+E8tWnYg6GzIsImFZQVpmI6SxNfSmVQIqxIufInrbQaI+pKXntdTQC9wyNK5FAA8TXAdff5ZDnmetsOTVble9Ia5m6gqM7MnxNZ56uDpn+6lCxRZSW+Ln2PDE7sivVcST4qpfwY4P4Lrb3vrjCGODFg4xmGNKXsLi2+uZbs5rW7bg4HFO50kKDucPV1M+rBWA9999

Copy to your servers your SSH public key to your usual ~/.ssh/authorized_keys file in your $HOME.

Testing PIV secured SSH

Plug-in your Yubikey, and then SSH to your remote server using the opensc-pkcs11 library. You will be prompted for your PIN and then successfully logged in 🙂

ssh -I opensc-pkcs11.so cheetah
Enter PIN for 'PIV_II (PIV Card Holder pin)':

You can then configure SSH to use it by default for all your hosts in your ~/.ssh/config

Host=*
PKCS11Provider /usr/lib/opensc-pkcs11.so

Using PIV with ssh-agent

You can also use ssh-agent to avoid typing your PIN every time.

When asked for the passphrase, enter your PIN:

ssh-add -s /usr/lib/opensc-pkcs11.so
Enter passphrase for PKCS#11: 
Card added: /usr/lib/opensc-pkcs11.so

You can verify that it worked by listing the available keys in your ssh agent:

ssh-add -L
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCWtqI37jwxYMJ9XLq9VwHgJlhZViPVAGIUfMm8SAlfs6cka4Cj570lkoGK04r8JAVJFy/iKfhGpL9N9XuartfIoq6Cg/6Qvg3REupuqs51V2cBaC/gnWIQ7qZqlzBulvcOvzNfHFD/lX42J58+E8tWnYg6GzIsImFZQVpmI6SxNfSmVQIqxIufInrbQaI+pKXntdTQC9wyNK5FAA8TXAdff5ZDnmetsOTVble9Ia5m6gqM7MnxNZ56uDpn+6lCxRZSW+Ln2PDE7sivVcST4qpfwY4P4Lrb3vrjCGODFg4xmGNKXsLi2+uZbs5rW7bg4HFO50kKDucPV1M+rBWA9999 /usr/lib64/opensc-pkcs11.so

Enjoy!

Now you have a flexible yet robust way to authenticate your users which you can also extend by adding another type of authentication on your servers using PAM.

I recently worked a bit at how we could secure better our SSH connections to our servers at work.

So far we are using the OpenSSH public key only mechanism which means that there is no password set on the servers for our users. While this was satisfactory for a time we think that this still suffers some disadvantages such as:

  • we cannot enforce SSH private keys to have a passphrase on the user side
  • the security level of the whole system is based on the protection of the private key which means that it’s directly tied to the security level of the desktop of the users

This lead us to think about adding a 2nd factor authentication to SSH and about the usage of security keys.

Meet the Yubikey

Yubikeys are security keys made by Yubico. They can support multiple modes and work with the U2F open authentication standard which is why they got my attention.

I decided to try the Yubikey 4 because it can act as a smartcard while offering these interesting features:

  • Challenge-Response
  • OTP
  • GPG
  • PIV

Method 1 – SSH using pam_ssh + pam_yubico

The first method I found satisfactory was to combine pam_ssh authentication module along with pam_yubico as a 2nd factor. This allows server side passphrase enforcement on SSH and the usage of the security key to login.

TL;DR: two gotchas before we begin

ADVISE: keep a root SSH session to your servers while deploying/testing this so you can revert any change you make and avoid to lock yourself out of your servers.

Setup pam_ssh

Use pam_ssh on the servers to force usage of a passphrase on a private key. The idea behind pam_ssh is that the passphrase of your SSH key serves as your SSH password.

Generate your SSH key pair with a passphrase on your local machine.

ssh-keygen -f identity
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in identity.
Your public key has been saved in identity.pub.
The key fingerprint is:
SHA256:a2/HNCe28+bpMZ2dIf9bodnBwnmD7stO5sdBOV6teP8 alexys@yazd
The key's randomart image is:
+---[RSA 2048]----+
|                 |
|                 |
|                o|
|            . ++o|
|        S    BoOo|
|         .  B %+O|
|        o  + %+*=|
|       . .. @ .*+|
|         ....%B.E|
+----[SHA256]-----+

You then must copy your private key (named identity with no extension) to your servers under  the ~/.ssh/login-keys.d/ folder.

In your $HOME on the servers, you will get something like this:

.ssh/
├── known_hosts
└── login-keys.d
    └── identity

Then you can enable the pam_ssh authentication. Gentoo users should enable the pam_ssh USE flag for sys-auth/pambase and re-install.

Add this at the beginning of the file /etc/pam.d/ssh

auth    required    pam_ssh.so debug

The debug flag can be removed after you tested it correctly.

Disable public key authentication

Because it takes precedence over the PAM authentication mechanism, you have to disable OpenSSH PubkeyAuthentication authentication on /etc/ssh/sshd_config:

PubkeyAuthentication no

Enable PAM authentication on /etc/ssh/sshd_config

ChallengeResponseAuthentication no
PasswordAuthentication no
UsePAM yes

Test pam_ssh

Now you should be prompted for your SSH passphrase to login through SSH.

➜  ~ ssh cheetah
SSH passphrase:

Setup 2nd factor using pam_yubico

Now we will make use of our Yubikey security key to add a 2nd factor authentication to login through SSH on our servers.

Because the Yubikey is not physically plugged on the server, we cannot use an offline Challenge-Response mechanism, so we will have to use a third party to validate the challenge. Yubico gracefully provide an API for this and the pam_yubico module is meant to use it easily.

Preparing your account using your Yubikey (on your machine)

First of all, you need to get your Yubico API key ID from the following URL:

You will get a Client ID (this you will use) and Secret Key (this you will keep safe).

Then you will need to create an authorization mapping file which basically link your account to a Yubikey fingerprint (modhex). This is equivalent to saying “this Yubikey belongs to this user and can authenticate him”.

First, get your modhex:

Using this modhex, create your mapping file named authorized_yubikeys which will be copied to ~/.yubico/authorized_yubikeys on the servers (replace LOGIN_USERNAME with your actual account login name).

LOGIN_USERNAME:xxccccxxuuxx

NOTE: this mapping file can be a centralized one (in /etc for example) to handle all the users from a server. See the the authfile option on the doc.

Setting up OpenSSH (on your servers)

You must install pam_yubico on the servers. For Gentoo, it’s as simple as:

emerge sys-auth/pam_yubico

Copy your authentication mapping file to your home under the .yubico folder on all servers. You should get this:

.yubico/
└── authorized_yubikey

Configure pam to use pam_yubico. Add this after the pam_ssh on the file /etc/pam.d/ssh which should look like this now:

auth    required    pam_ssh.so
auth    required    pam_yubico.so id=YOUR_API_ID debug debug_file=/var/log/auth.log

The debug and debug_file flags can be removed after you tested it correctly.

Testing pam_yubico

Now you should be prompted for your SSH passphrase and then for your Yubikey OTP to login through SSH.

➜  ~ ssh cheetah
SSH passphrase: 
YubiKey for `ultrabug':

About the Yubico API dependency

Careful readers will notice that using pam_yubico introduces a strong dependency on the Yubico API availability. If the API becomes unreachable or your internet connection goes down then your servers would be unable to authenticate you!

The solution I found to this problem is to instruct pam to ignore the Yubikey authentication when pam_yubico is unable to contact the API.

In this case, the module will return a AUTHINFO_UNAVAIL code to PAM which we can act upon using the following syntax. The /etc/pam.d/ssh first lines should be changed to this:

auth    required    pam_ssh.so
auth    [success=done authinfo_unavail=ignore new_authtok_reqd=done default=die]    pam_yubico.so id=YOUR_API_ID debug debug_file=/var/log/auth.log

Now you can be sure to be able to use your Yubikey even if the API is down or unreachable 😉

May 08, 2017
Sebastian Pipping a.k.a. sping (homepage, bugs)

What’s Wrong with Disqus? / Replacing Disqus with Github Comments (donw.io)

Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)

If you're in Regensburg, please come to the physics colloquium next week (15.5., 16:00st, lecture hall H34)! See the linked PDF for the full talk announcement.

"Millikelvin transport experiments on carbon nanotubes - nanoelectromechanics, spectroscopy, and more"

May 05, 2017
Luca Barbato a.k.a. lu_zero (homepage, bugs)
Contributing to x264 (May 05, 2017, 18:50 UTC)

Another project I contribute to is x264. As per the previous post on the topic I’ll try to summarize how things work.

Overview

Coding style

x264 has a coding style and on review you will get asked to follow it, sadly the sources do not contain a document about it, you have to look around the code and match what is there.

Testing

x264 has an amazing test harness that doubles as benchmark harness to add support for additional architecture-specific optimizations. checkasm is really good in helping you write new code for this kind of purpose and make sure it is really faster.

It is automatic to use if you are adding a function already implemented in other architectures, if you want to extend the coverage for something new it is moderately difficult, mainly because you have to read the code since no documentation is available otherwise.

Submitting patches

Submitting code to x264 requires you to sign a cla, the process is sort of manual and involves exchanging emails with the person in charge to provide and collect the cla pdf once you signed it.

Once you are done on that you should rebase your changes over the sandbox branch, that’s somehow similar to the next branch on other projects and send them to the developer mailing list using git send-email.

Interaction

The review process will happen in the mailing list and you are supposed to be subscribed to it and interact with the reviewers there.

TL;DR

  • Mimic the coding style used in the project and hope you get it right
  • Peruse checkasm to make sure what you are doing works as intended and it is fast
  • Subscribe to the developer mailing list and learn how to use git send-email.
  • Be patient and wait for review comments in the mail box.