Gentoo Logo
Gentoo Logo Side
Gentoo Spaceship

Contributors:
. Aaron W. Swenson
. Agostino Sarubbo
. Alec Warner
. Alex Alexander
. Alex Legler
. Alexey Shvetsov
. Alexis Ballier
. Alexys Jacob
. Amadeusz Żołnowski
. Andreas K. Hüttel
. Andreas Proschofsky
. Andrew Gaffney
. Anthony Basile
. Arun Raghavan
. Bernard Cafarelli
. Bjarke Istrup Pedersen
. Brent Baude
. Brian Harring
. Christian Ruppert
. Chí-Thanh Christopher Nguyễn
. Dane Smith
. Daniel Gryniewicz
. David Abbott
. Denis Dupeyron
. Detlev Casanova
. Diego E. Pettenò
. Domen Kožar
. Donnie Berkholz
. Doug Goldstein
. Eray Aslan
. Fabio Erculiani
. Gentoo Haskell Herd
. Gentoo News
. Gilles Dartiguelongue
. Greg KH
. Hanno Böck
. Hans de Graaff
. Ian Whyman
. Ioannis Aslanidis
. Jan Kundrát
. Jason Donenfeld
. Jeffrey Gardner
. Jeremy Olexa
. Joachim Bartosik
. Johannes Huber
. Jonathan Callen
. Jorge Manuel B. S. Vicetto
. Joseph Jezak
. Josh Saddler
. Kenneth Prugh
. Lance Albertson
. Liam McLoughlin
. LinuxCrazy Podcasts
. Luca Barbato
. Luis Francisco Araujo
. Mark Loeser
. Markos Chandras
. Mart Raudsepp
. Matt Turner
. Matthew Marlowe
. Matthew Thode
. Matti Bickel
. Michal Hrusecky
. Michał Górny
. Mike Doty
. Mike Gilbert
. Mike Pagano
. Mu Qiao
. Nathan Zachary
. Ned Ludd
. Nirbheek Chauhan
. Ole Markus With
. Olivier Crête
. Pacho Ramos
. Patrick Kursawe
. Patrick Lauer
. Patrick McLean
. Paul de Vrieze
. Pavlos Ratis
. Paweł Hajdan, Jr.
. Petteri Räty
. Piotr Jaroszyński
. Rafael Goncalves Martins
. Raúl Porcel
. Remi Cardona
. Richard Freeman
. Robin Johnson
. Ryan Hill
. Sean Amoss
. Sebastian Pipping
. Serkan Kaba
. Steev Klimaszewski
. Stratos Psomadakis
. Sune Kloppenborg Jeppesen
. Sven Vermeulen
. Sven Wegener
. Tim Sammut
. Tiziano Müller
. Tobias Heinlein
. Tobias Klausmann
. Tomáš Chvátal
. Torsten Veller
. Vikraman Choudhury
. Zack Medico
. Zhang Le

Last updated:
May 30, 2013, 23:05 UTC

Disclaimer:
Views expressed in the content published here do not necessarily represent the views of Gentoo Linux or the Gentoo Foundation.


Bugs? Comments? Suggestions? Contact us!

Powered by:
Planet Venus

Welcome to Gentoo Universe, an aggregation of weblog articles on all topics written by Gentoo developers. For a more refined aggregation of Gentoo-related topics only, you might be interested in Planet Gentoo.

May 30, 2013
Sebastian Pipping a.k.a. sping (homepage, bugs)

Humble Indie Bundle 8 includes these games and (most of) their soundtracks: (links go to YouTube videos)

  1. Hotline Miami
  2. Proteus
  3. Little Inferno
  4. Awesomenauts
  5. Capsized
  6. Thomas Was Alone
  7. Dear Esther

Hotline Miami

Proteus

Little Inferno

Awesomenauts

Capsized

Thomas Was Alone

 

 

Dear Esther

Sven Vermeulen a.k.a. swift (homepage, bugs)

After using a default set of directories to watch, and allowing admins to mark other types as such as well, let’s consider another approach for making the policy more flexible: booleans. The idea now is that a boolean called incron_notify_non_security_files enables incrond to be notified on changes on all possible non-security related files (the latter is merely an approach, you can define other sets as well if you want, including all possible files).

Booleans in SELinux policy can be generated in the incron.te file as follows:

## <desc>
## <p>
##      Determine whether incron can watch all non-security
##      file types
## </p>
## </desc>
gen_tunable(incron_notify_non_security_files, false)

With this boolean in place, the policy can be enhanced with code like the following:

tunable_policy(`incron_notify_non_security_files',`
        files_read_non_security_files(incrond_t)
        files_read_all_dirs_except(incrond_t)
')

This code tells SELinux that, if the incron_notify_non_security_files boolean is set (which by default is not the case), then incrond_t is able to read non security files.

Let’s try to watch for changes in the AIDE log directory:

# tail audit.log
type=AVC msg=audit(1368777675.597:28611): avc:  denied  { search } for  pid=11704 comm="incrond" name="log" dev="dm-4" ino=13 scontext=system_u:system_r:incrond_t tcontext=system_u:object_r:var_log_t tclass=dir
type=AVC msg=audit(1368777675.597:28612): avc:  denied  { search } for  pid=11704 comm="incrond" name="log" dev="dm-4" ino=13 scontext=system_u:system_r:incrond_t tcontext=system_u:object_r:var_log_t tclass=dir

# tail cron.log
May 17 10:01:15 test incrond[11704]: access denied on /var/log/aide - events will be discarded silently

# getsebool incron_notify_non_security_files
incron_notify_non_security_files --> off

Let’s enable the boolean and try again:

# setsebool incron_notify_non_security_files on

Reloading the incrontab tables now works, and the notifications work as well.

As you can see, once a policy is somewhat working, policy developers are considering the various “use cases” of an application, trying to write down policies that can be used by the majority of users, without granting too many rights automatically.

May 29, 2013
Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)

Last week we had our monthly Gentoo KDE team meeting; here are few details that are probably worth sharing.

  • So far we've provided the useflag "semantic-desktop" which in particular controls the nepomuk functionality. Some components of KDE require this functionality unconditionally, and if you try to build without it, bugs and build failures may occur. In addition, by now it is easily and reliably possible to disable e.g. the file indexer at runtime. So, we've decided that starting with KDE 4.11 we will remove the useflag and hard-enable the functionality and the required dependencies in the ebuilds. The changes are being done already in the KDE overlay in the live ebuilds (which build upstream git master and form the templates for the upcoming 4.11 releases).
  • After recent experiences the plan to drop kdepim-4.4 is off the table again. We will keep it in the portage tree as alternative version and try to support it until it finally breaks.
  • In the meantime we (well, mainly Chris Reffett) have started in the KDE overlay to package Plasma Active, the tablet Plasma workspace environment. Since Gentoo ARM support is already excellent, this may become a highly valuable addition. Unfortunately, it's not really ready yet for the main tree and general use, but packaging work will continue in the overlay- what we need most is testing and bug reporting!
Independent of the meeting, a stabilization request has already been filed for KDE 4.10.3;  thanks to the work of the kde stable testers, we can keep everyone uptodate. And as a final note, my laptop is back to kmail1... Cheers!

Sebastian Pipping a.k.a. sping (homepage, bugs)

Originial title (and link): GNµ 11 – LinuxTag Berlin 2013 – Teil 2: Gentoo, Fedora, Mageia

English subtitles available, audio is German.

Patrick Lauer a.k.a. bonsaikitten (homepage, bugs)
Having fun with cross-compiling (May 29, 2013, 09:48 UTC)

$ file build/root-filesystem-*/bin/mksh
build/root-filesystem-armv4tl/bin/mksh:       ELF 32-bit LSB executable, ARM, EABI4 version 1 (SYSV), dynamically linked (uses shared libs), stripped
build/root-filesystem-armv6l/bin/mksh:        ELF 32-bit LSB executable, ARM, EABI4 version 1 (SYSV), dynamically linked (uses shared libs), stripped
build/root-filesystem-i486/bin/mksh:          ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), stripped
build/root-filesystem-i586/bin/mksh:          ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), stripped
build/root-filesystem-i686/bin/mksh:          ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), stripped
build/root-filesystem-mips/bin/mksh:          ELF 32-bit MSB executable, MIPS, MIPS-I version 1 (SYSV), dynamically linked (uses shared libs), stripped
build/root-filesystem-mips64/bin/mksh:        ELF 64-bit MSB executable, MIPS, MIPS64 version 1 (SYSV), dynamically linked (uses shared libs), stripped
build/root-filesystem-mipsel/bin/mksh:        ELF 32-bit LSB executable, MIPS, MIPS-I version 1 (SYSV), dynamically linked (uses shared libs), stripped
build/root-filesystem-powerpc-440fp/bin/mksh: ELF 32-bit MSB executable, PowerPC or cisco 4500, version 1 (SYSV), dynamically linked (uses shared libs), stripped
build/root-filesystem-powerpc/bin/mksh:       ELF 32-bit MSB executable, PowerPC or cisco 4500, version 1 (SYSV), dynamically linked (uses shared libs), stripped
build/root-filesystem-sh4/bin/mksh:           ELF 32-bit LSB executable, Renesas SH, version 1 (SYSV), dynamically linked (uses shared libs), stripped
build/root-filesystem-sparc/bin/mksh:         ELF 32-bit MSB executable, SPARC version 1 (SYSV), dynamically linked (uses shared libs), stripped
build/root-filesystem-x86_64/bin/mksh:        ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), stripped
... that is the result of an afternoon of hacking on Aboriginal Linux to include mksh support.
Why? eh ... why not. And for such a crude hack it works surprisingly well - only two of the arm crosscompile targets failed.

Sven Vermeulen a.k.a. swift (homepage, bugs)

In the previous post we made incrond able to watch public_content_t and public_content_rw_t types. However, this is not scalable, so we might want to be able to update the policy more dynamically with additional types. To accomplish this, we will make types eligible for watching through an attribute.

So how does this work? First, we create an attribute called incron_notify_type (we can choose the name we want of course) and grant incrond_t the proper rights on all types that have been assigned the incron_notify_type attribute. Then, we create an interface that other modules (or admins) can use to mark specific types eligible for watching, called incron_notify_file. This interface will assign the incron_notify_type attribute to the provided type.

First, the attribute and its associated privileges:

attribute incron_notify_type;
...
allow incrond_t incron_notify_type:dir list_dir_perms;
allow incrond_t incron_notify_type:file read_file_perms;
allow incrond_t incron_notify_type:lnk_file read_lnk_file_perms;

That’s it. For now, this won’t do much as there are no types associated with the incron_notify_type attribute, so let’s change that by introducing the interface:

########################################
## <summary>
##      Make the specified type a file or directory
##      that incrond can watch on.
## </summary>
## <param name="file_type">
##      <summary>
##      Type of the file to be allowed to watch
##      </summary>
## </param>
#
interface(`incron_notify_file',`
        gen_require(`
                attribute incron_notify_type;
        ')

        typeattribute $1 incron_notify_type;
')

That’s it! If you want incrond to watch user content, one can now do something like:

incron_notify_file(home_root_t)
incron_notify_file(user_home_dir_t)
incron_notify_file(user_home_t)

Moreover, we can now also easily check what additional types have been marked as such:

$ seinfo -aincron_notify_type -x
   incron_notify_type
      user_home_dir_t
      user_home_t
      home_root_t

This attribute approach is commonly used for such setups and is becoming more and more a “standard” approach.

In the next post, we’ll cover a boolean-triggered approach where incrond will be eligible for watching all (non-security) content.

May 28, 2013
Sven Vermeulen a.k.a. swift (homepage, bugs)
A SELinux policy for incron: default set (May 28, 2013, 01:50 UTC)

I finished the last post a bit with a cliffhanger as incrond is still not working properly, and we got a few denials that needed to be resolved; here they are again for your convenience:

type=AVC msg=audit(1368734110.912:28353): avc:  denied  { getattr } for  pid=9716 comm="incrond" path="/home/user/test2" dev="dm-0" ino=16 scontext=system_u:system_r:incrond_t tcontext=user_u:object_r:user_home_t tclass=dir
type=AVC msg=audit(1368734110.913:28354): avc:  denied  { read } for  pid=9716 comm="incrond" name="test2" dev="dm-0" ino=16 scontext=system_u:system_r:incrond_t tcontext=user_u:object_r:user_home_t tclass=dir

The permission we are looking for here is userdom_list_user_home_content, but this is only for when we want to watch a user home directory. What if we want to watch a server upload directory? Or a cache directory? We might need to have incrond have the proper accesses on all directories. But then again, all does sound a bit… much, doesn’t it? So let’s split it up in three waves:

  1. The incrond_t domain will support a minimal set of types that it can watch, based on common approaches
  2. I will introduce an interface that allows other modules to mark specific types as being “watch-worthy”
  3. A boolean will be set to allow incrond_t to watch a very large set of types (just in case the admin trusts it sufficiently)

Let’s first consider a decent minimal set. Within most SELinux policies, two types are often used for public access (or for uploading of data). These types are public_content_t and public_content_rw_t, and is used for instance for FTP definitions (upload folders), HTTP servers and such. So we introduce the proper rights to watch that data. There is an interface available called miscfiles_read_public_files but let’s first see if that interface isn’t too broad (after all, watching might not be the same as reading).

# This is only to temporarily check if the rights of the interface are too broad or not
# You can set this using "selocal" or in a module (in which case you'll need to 'require'
# the two types)
allow incrond_t public_content_t:dir { read getattr };

After editing the incrontab to watch a directory labeled with public_content_t, we now get the following:

# tail cron.log
May 17 08:46:12 test incrond[9716]: (user) CMD (/usr/local/bin/test)
May 17 08:46:12 test incrond[11281]: cannot exec process: Operation not permitted
May 17 08:46:12 test incrond[9716]: cannot send SIGCHLD token to notification pipe

# tail audit.log
type=AVC msg=audit(1368773172.313:28386): avc:  denied  { setgid } for  pid=11281 comm="incrond" capability=6  scontext=system_u:system_r:incrond_t tcontext=system_u:system_r:incrond_t tclass=capability
type=AVC msg=audit(1368773172.314:28387): avc:  denied  { read } for  pid=9716 comm="incrond" path="pipe:[14027]" dev="pipefs" ino=14027 scontext=system_u:system_r:incrond_t tcontext=system_u:system_r:incrond_t tclass=fifo_file
type=AVC msg=audit(1368773172.315:28388): avc:  denied  { write } for  pid=9716 comm="incrond" path="pipe:[14027]" dev="pipefs" ino=14027 scontext=system_u:system_r:incrond_t tcontext=system_u:system_r:incrond_t tclass=fifo_file

As the incrontab is a user incrontab, we can expect incrond_t to require setuid and setgid privileges. Also, the fifo_file access is after forking (notice the difference in PID values) and most likely to communicate to the master process. So let’s allow those:

allow incrond_t self:capability { setuid setgid };
allow incrond_t self:fifo_file { read write };

With that set, we get the following upon triggering a file write:

# tail cron.log
May 17 08:52:46 test incrond[9716]: (user) CMD (/usr/local/bin/test)
May 17 08:52:46 test incrond[11338]: cannot exec process: Permission denied

# tail audit.log
type=AVC msg=audit(1368773566.606:28394): avc:  denied  { read } for  pid=11338 comm="incrond" name="ngroups_max" dev="proc" ino=5711 scontext=system_u:system_r:incrond_t tcontext=system_u:object_r:sysctl_kernel_t tclass=file
type=AVC msg=audit(1368773566.607:28395): avc:  denied  { search } for  pid=11338 comm="incrond" name="bin" dev="dm-3" ino=1048578 scontext=system_u:system_r:incrond_t tcontext=system_u:object_r:bin_t tclass=dir

The ngroups_max pseudo-file (in /proc/sys/kernel) returns the maximum number of supplementary group IDs per process, and is consulted through the initgroups() method provided by a system library, so it might make sense to allow it. For now though, I will not enable it (as reading sysctl_kernel_t exposes a lot of other system information) but I might be forced to do so later if things don’t work out well. The search privilege on bin_t is needed to find the script that I have prepared (/usr/local/bin/test) to be executed, so I add in a corecmd_search_bin and retry.

# tail cron.log
May 17 09:02:55 test incrond[9716]: (user) CMD (/usr/local/bin/test)
May 17 09:02:55 test incrond[11427]: cannot exec process: Permission denied

# tail audit.log
type=AVC msg=audit(1368774175.646:28441): avc:  denied  { read } for  pid=11427 comm="incrond" name="sh" dev="dm-2" ino=131454 scontext=system_u:system_r:incrond_t tcontext=root:object_r:bin_t tclass=lnk_file

Still not there yet apparently. The incrond forked process wants to execute the script, but to do so it has to follow a symbolic link labeled bin_t. This is because the script points to #!/bin/sh which is a symlink to the system shell. We need to follow this link before the execution can occur; only after execution will the transition from incrond_t to system_cronjob_t be done.

corecmd_read_bin_symlinks(incrond_t)

With that set in the policy, the watch works, incrond properly launches the command and the command properly transitions into system_cronjob_t as we defined earlier (I check this by echo’ing the output of id -Z into a temporary file).

So we are left with the (temporary) rights we granted on public_content_t. Consider the rules we had versus the rules applied with miscfiles_read_public_files:

allow incrond_t public_content_t:dir { read getattr };

# miscfiles_read_public_files
allow $1 { public_content_t public_content_rw_t }:dir list_dir_perms;
read_files_pattern($1, { public_content_t public_content_rw_t }, { public_content_t public_content_rw_t })
read_lnk_files_pattern($1, { public_content_t public_content_rw_t }, { public_content_t public_content_rw_t })

The rights here seem to bemore than what we need. Playing around a bit with the directories reveals that incrond requires a bit more. For instance, when you create additional directories (subdirectories) and want to match multiple ones:

# tail cron.log
May 17 09:16:08 test incrond[11704]: access denied on /var/www/test/* - events will be discarded silently
May 17 09:16:08 test incrond[11704]: cannot create watch for user user: (13) Permission denied

# tail audit.log
type=AVC msg=audit(1368774968.416:28504): avc:  denied  { search } for  pid=11704 comm="incrond" name="test" dev="dm-4" ino=1488 scontext=system_u:system_r:incrond_t tcontext=root:object_r:public_content_t tclass=dir
type=AVC msg=audit(1368774968.416:28505): avc:  denied  { search } for  pid=11704 comm="incrond" name="test" dev="dm-4" ino=1488 scontext=system_u:system_r:incrond_t tcontext=root:object_r:public_content_t tclass=dir

Similarly if you want to watch on a particular file:

type=AVC msg=audit(1368775274.655:28552): avc:  denied  { getattr } for  pid=11704 comm="incrond" path="/var/www/test/testfile" dev="dm-4" ino=1709 scontext=system_u:system_r:incrond_t tcontext=root:object_r:public_content_t tclass=file
type=AVC msg=audit(1368775274.655:28553): avc:  denied  { read } for  pid=11704 comm="incrond" name="testfile" dev="dm-4" ino=1709 scontext=system_u:system_r:incrond_t tcontext=root:object_r:public_content_t tclass=file

So it looks like miscfiles_read_public_files isn’t that bad after all.

All we are left with is the access to ngroups_max. We can ignore the calls and make sure they don’t show up in standard auditing using kernel_dontaudit_read_kernel_sysctls or we can allow it with kernel_read_kernel_sysctls. I’m going to take the former approach for my system, but your own idea might be different.

I tested all this with user incrontabs (as those are the “most” advanced) but one can easily test with system incrontabs as well (placing one in /etc/incron.d). Just be aware that incrond will take the first match and will not seek other matches. So if a system incrontab watches /var/www and another line (or user incrontab) watches /var/www/localhost/upload it is very well possible that only the /var/www watch is triggered.

So right now, our incrond_t policy looks like so:

###########################################
#
# incrond policy
#

allow incrond_t self:capability { setgid setuid };

allow incrond_t incron_spool_t:dir list_dir_perms;
allow incrond_t incron_spool_t:file read_file_perms;

allow incrond_t self:fifo_file { read write };

allow incrond_t incrond_var_run_t:file manage_file_perms;
files_pid_filetrans(incrond_t, incrond_var_run_t, file)

kernel_dontaudit_read_kernel_sysctls(incrond_t)

corecmd_read_bin_symlinks(incrond_t)
corecmd_search_bin(incrond_t)

files_search_spool(incrond_t)

logging_send_syslog_msg(incrond_t)

auth_use_nsswitch(incrond_t)

miscfiles_read_localization(incrond_t)
miscfiles_read_public_files(incrond_t)

Next on the agenda is another interface to make other types “watch-worthy”.

May 27, 2013
Michał Górny a.k.a. mgorny (homepage, bugs)
The pointless art of subslots (May 27, 2013, 18:00 UTC)

The sub-slots feature of EAPI 5 was announced as if it was the ultimate solution to the problem of SONAME changes on library upgrades. However, the longer I see it, the more I believe that it is not really a good solution, and that it misses the actual issue targeting somewhere nearby.

The issue is likely well-known by most of the Gentoo users. Every time a library changes its ABI, it changes the SONAME (the filename programs link to) to avoid breaking existing programs. When the package is upgraded, the new version is installed under the new name, and the old one is removed. As a direct result, all applications linking to the old version become broken and need to be rebuilt.

The classic way of handling this is to run the revdep-rebuild tool. It takes a while to scan the system with it but it supposedly finds all broken executables and initiates a rebuild of them. Of course, the system is in broken state until all relevant packages are rebuilt, and sometimes they just fail to build…

As you can guess, this is far from being perfect. That’s why people tried to find a better solution, and a few solutions were actually implemented. I’d like to describe them in a quasi-chronological order.

Using slots with slot-operator deps

A perfect solution that has been long advocated by Exherbo developers. I’m not aware, though, if they ever used it themselves. I didn’t see an exact explanation of how they expect it to be done, therefore I am mostly explaining here how I think it could be done.

The idea is that every SONAME-version of the library uses a different slot. That is, every time the SONAME changes, you change slot as well. Using different slots for each SONAME means that the incompatible versions of the library can be installed in parallel until all applications are rebuilt. This has a few requirements though.

First of all, only the newest slot may install development files such as headers. This requires that every version bump is accompanied by a revision bump of the older version, dropping the development files. On each upgrade, user builds not only the new version but also rebuilds the older version.

To handle the upgrades without a small moment of breakage (and risk of longer breakage if a build fails), the package manager would need to build both packages before starting the merge process. I doubt that enforcing this is really possible right now.

Secondly, the ebuilds installing development files would need to block the older versions (in other slots) doing the same while keeping the versions lacking development files non-blocked.

To explain this better: let’s assume that we have: foo-1, foo-1-r1, foo-2, foo-2-r1, foo-3, … The -r0 versions have development files and -r1 versions don’t have them (they are just the upgrade compatibility ebuilds). Now, the blocker in foo-3 would need to block all the older -r0 versions and not -r1 ones.

In a real-life situation, there will likely be differing revision numbers as well. And I don’t know any way of handling this other than explicitly listing all blocked versions, one by one.

And in the end, reverse dependencies need to use a special slot-dependency operator which binds the dependency to the slot that was used during the package build. But it’s least of the problems, I believe.

The solution of preserved-libs

An another attempt of solving the issue was developed in portage-2.2. Although it is available in mainstream portage nowadays, it is still disabled by default due to a few bugs and the fact that some people believe it’s a hack.

The idea of preserved-libs is for the package manager to actually trace library linkage within installed programs and automatically preserve old versions of libraries as long as the relevant programs are not rebuilt to use the newer versions. As complex and as simple as that.

Preserving libraries this way doesn’t require any specific action from the package maintainer. Portage detects itself that a library with a new SONAME has been installed during an upgrade and preserves the old one. It also keeps track of all the consumers that link against the old version and remove it after the last one is rebuilt.

Of course it is not perfect. It can’t handle all kinds of incompatibilities, it won’t work outside the traditional executable-library linkage and the SONAME tracking is not perfect. But I believe this is the best solution we can have.

The nothing-new in sub-slots

Lately, a few developers who believed that preserved-libs is not supposed to go mainstream decided to implemented a different solution. After some discussion, the feature was quickly put into EAPI 5 and then started to be tested on the tree.

The problem is that it’s somehow a solution to the wrong problem. As far as I am concerned, the major issue with SONAMEs changing is that the system is broken between package rebuilds. Tangentially to this, sub-slots mostly address having to call tools like revdep-rebuild which is not a solution to the problem.

Basically all sub-slots do is forcing rebuild on a given set of reverse dependencies when the sub-slot of package changes. The rebuilds are pulled into the same dependency graph as the upgrade to be forced immediately after it.

I can agree that sub-slots have their uses. For example, xorg-server modules definitely benefit from them, and so may other cases which weren’t handled by preserved-libs already. For other cases the sub-slots are either not good enough (virtuals), redundant (regular libraries) or even broken (packages installing multiple libraries).

Aside from the xorg module benefit, I don’t see much use of sub-slots. On systems not having preserved-libs enabled, they may eventually remove the need for revdep-rebuild. On systems having preserved-libs, it can only result in needless or needlessly hurried rebuilds.

A short summary

So, we’re having two live solutions right now: one in preserved-libs, and other in sub-slots. The former addresses the issue of system being broken mid-upgrade, the latter removes (partially?) the need for calling an external tool. The former allows you to rebuild the affected packages at any convenient time, the latter forces you to do it right away.

What really worries me is that people are so opposed to preserved-libs, and at the same time accept a partial, mis-designed work called sub-slots that easily. Then advertise it without thoroughly explaining how and when to use it, and what are the problems with it. And, for example, unnecessarily rebuilding webkit-gtk regularly would be an important issue.

A particular result of that was visible when sub-slot support was introduced into app-text/poppler. That package installs a core library with quite an unstable ABI and a set of interface libraries with stable ABIs. External packages usually link with the latter.

When sub-slot support was enabled on poppler, all reverse dependencies were desired to use sub-slot matching. As a result, every poppler upgrade required needlessly rebuilding half of the system. The rev-deps were reverted but this only made people try to extend the sub-slots into a more complex and even less maintainable idea.

Is this really what we all want? Does it benefit us? And why the heck people reinvented library preservation in eclasses?!

Sven Vermeulen a.k.a. swift (homepage, bugs)

With incrontab_t (hopefully) complete, let’s look at the incrond_t domain. As this domain will also be used to execute the user (and system) commands provided through the incrontabs, we need to consider how we are going to deal with this wide range of possible permissions that it might take. One would be to make incrond_t quite powerful, and extend its privileges as we go further. But in my opinion, that’s not a good way to deal with it.

Another would be to support a small set of permissions, and introduce an interface that other modules can use to create a transition when incrond_t executes a script properly labeled for a transition. For instance, a domain foo_t might have an executable type foo_exec_t. Most modules support an interface similar to foo_domtrans (and foo_role if roles are applicable as well), but that assumes that the incron policy is modified every time a new target module is made available (since we then need to add the proper *_domtrans rules to the incron policy. Instead, we might want to make this something that the foo SELinux module can decide.

It is that approach that we are going to take here. To do so, we will create a new interface called incron_entry, taken a bit from the cron_system_entry interface already in place for the regular cron domain (the following comes in incron.if):

## <summary>
##      Make the specified program domain
##      accessible from the incrond job.
## </summary>
## <param name="domain">
##      <summary>
##      The type of the process to transition to
##      </summary>
## </param>
## <param name="entrypoint">
##      <summary>
##      The type of the file used as an entrypoint to this domain
##      </summary>
## </param>
#
interface(`incron_entry',`
        gen_require(`
                type incrond_t;
        ')

        domtrans_pattern(incrond_t, $2, $1)
')

With this in place, the foo SELinux module can call incron_entry(foo_t, foo_exec_t) so that, the moment incrond_t executes a file with label foo_exec_t, the resulting process will run in foo_t. I am going to test (and I stress that it is only for testing) this by assigning incron_entry(system_cronjob_t, shell_exec_t), making every shell script being called run in system_cronjob_t domain (for instance in the localuser.te file that already assigned incron_role to the user_t domain.

With that in place, it’s time to start our iterations again.

# run_init rc-service incrond start
* start-stop-daemon: failed to start '/usr/sbin/incrond' [ !! ]
* ERROR: incrond failed to start

# tail audit.log
type=AVC msg=audit(1368732494.275:28319): avc:  denied  { read } for  pid=9282 comm="incrond" name="localtime" dev="dm-2" ino=393663 scontext=system_u:system_r:incrond_t tcontext=system_u:object_r:locale_t tclass=file
type=AVC msg=audit(1368732494.275:28320): avc:  denied  { create } for  pid=9282 comm="incrond" scontext=system_u:system_r:incrond_t tcontext=system_u:system_r:incrond_t tclass=unix_dgram_socket
type=AVC msg=audit(1368732494.276:28321): avc:  denied  { read } for  pid=9282 comm="incrond" name="incron.d" dev="dm-2" ino=394140 scontext=system_u:system_r:incrond_t tcontext=root:object_r:incron_spool_t tclass=dir
type=AVC msg=audit(1368732494.276:28322): avc:  denied  { create } for  pid=9282 comm="incrond" scontext=system_u:system_r:incrond_t tcontext=system_u:system_r:incrond_t tclass=unix_dgram_socket
type=AVC msg=audit(1368732494.276:28323): avc:  denied  { create } for  pid=9282 comm="incrond" scontext=system_u:system_r:incrond_t tcontext=system_u:system_r:incrond_t tclass=unix_dgram_socket

Ignoring the unix_dgram_socket for now, we need to allow incrond_t to read locale information, and to read the files in the /var/spool/incron location (this goes in incron.te again):

###########################################
#
# incrond policy
#

read_files_pattern(incrond_t, incron_spool_t, incron_spool_t)

files_search_spool(incrond_t)

miscfiles_read_localization(incrond_t)

The next run fails again, with the following denials:

type=AVC msg=audit(1368732806.757:28328): avc:  denied  { create } for  pid=9419 comm="incrond" scontext=system_u:system_r:incrond_t tcontext=system_u:system_r:incrond_t tclass=unix_dgram_socket
type=AVC msg=audit(1368732806.757:28329): avc:  denied  { read } for  pid=9419 comm="incrond" name="incron.d" dev="dm-2" ino=394140 scontext=system_u:system_r:incrond_t tcontext=root:object_r:incron_spool_t tclass=dir
type=AVC msg=audit(1368732806.757:28330): avc:  denied  { create } for  pid=9419 comm="incrond" scontext=system_u:system_r:incrond_t tcontext=system_u:system_r:incrond_t tclass=unix_dgram_socket
type=AVC msg=audit(1368732806.757:28331): avc:  denied  { create } for  pid=9419 comm="incrond" scontext=system_u:system_r:incrond_t tcontext=system_u:system_r:incrond_t tclass=unix_dgram_socket

So although incrond_t has search rights on the incron_spool_t directories (through the read_files_pattern), we need to grant it list_dir_perms as well (which contains the read permission). As list_dir_perms contains search anyhow, we can just update the line with:

allow incrond_t incron_spool_t:dir list_dir_perms;
allow incrond_t incron_spool_t:file read_file_perms;

Now the startup seems to work, but we still get denials:

# run_init rc-service incrond start
* Starting incrond... [ ok ]

# ps -eZ | grep incrond
# tail /var/log/cron.log
(nothing)

# tail audit.log
type=AVC msg=audit(1368733443.799:28340): avc:  denied  { create } for  pid=9551 comm="incrond" scontext=system_u:system_r:incrond_t tcontext=system_u:system_r:incrond_t tclass=unix_dgram_socket
type=AVC msg=audit(1368733443.802:28341): avc:  denied  { write } for  pid=9552 comm="incrond" name="/" dev="tmpfs" ino=1970 scontext=system_u:system_r:incrond_t tcontext=system_u:object_r:var_run_t tclass=dir
type=AVC msg=audit(1368733443.806:28342): avc:  denied  { create } for  pid=9552 comm="incrond" scontext=system_u:system_r:incrond_t tcontext=system_u:system_r:incrond_t tclass=unix_dgram_socket
type=AVC msg=audit(1368733443.806:28343): avc:  denied  { create } for  pid=9552 comm="incrond" scontext=system_u:system_r:incrond_t tcontext=system_u:system_r:incrond_t tclass=unix_dgram_socket

Those unix_dgram_sockets are here again. But seeing that cron.log is empty, and logging_send_syslog_msg is one of the interfaces that would enable it, we might want to do just that so that we get more information about why incrond doesn’t properly start. Also, it tries to write into var_run_t labeled directories, probably for its PID file, so add in a proper file transition as well as manage rights:

type incrond_var_run_t;
files_pid_file(incrond_var_run_t)
...
allow incrond_t incrond_var_run_t:file manage_file_perms;
files_pid_filetrans(incrond_t, incrond_var_run_t, file)
...
logging_send_syslog_msg(incrond_t)

With that in place:

# run_init rc-service incrond start
* Starting incrond... [ ok ]

# ps -eZ | grep incron
system_u:system_r:incrond_t      9648 ?        00:00:00 incrond

# tail /var/log/cron.log 
May 16 21:51:34 test incrond[9647]: starting service (version 0.5.10, built on May 16 2013 12:11:29)
May 16 21:51:34 test incrond[9648]: loading system tables
May 16 21:51:34 test incrond[9648]: loading user tables
May 16 21:51:34 test incrond[9648]: table for invalid user user found (ignored)
May 16 21:51:34 test incrond[9648]: ready to process filesystem events

# tail audit.log
type=AVC msg=audit(1368733894.641:28347): avc:  denied  { read } for  pid=9648 comm="incrond" name="nsswitch.conf" dev="dm-2" ino=393768 scontext=system_u:system_r:incrond_t tcontext=system_u:object_r:etc_t tclass=file
type=AVC msg=audit(1368733894.645:28349): avc:  denied  { read } for  pid=9648 comm="incrond" name="passwd" dev="dm-2" ino=394223 scontext=system_u:system_r:incrond_t tcontext=system_u:object_r:etc_t tclass=file

It looks like we’re getting there. Similar as with incrontab we allow auth_use_nsswitch as well, and then get:

# tail cron.log
May 16 21:55:10 test incrond[9715]: starting service (version 0.5.10, built on May 16 2013 12:11:29)
May 16 21:55:10 test incrond[9716]: loading system tables
May 16 21:55:10 test incrond[9716]: loading user tables
May 16 21:55:10 test incrond[9716]: loading table for user user
May 16 21:55:10 test incrond[9716]: access denied on /home/user/test2 - events will be discarded silently
May 16 21:55:10 test incrond[9716]: cannot create watch for user user: (13) Permission denied
May 16 21:55:10 test incrond[9716]: ready to process filesystem events

# tail audit.log
type=AVC msg=audit(1368734110.912:28353): avc:  denied  { getattr } for  pid=9716 comm="incrond" path="/home/user/test2" dev="dm-0" ino=16 scontext=system_u:system_r:incrond_t tcontext=user_u:object_r:user_home_t tclass=dir
type=AVC msg=audit(1368734110.913:28354): avc:  denied  { read } for  pid=9716 comm="incrond" name="test2" dev="dm-0" ino=16 scontext=system_u:system_r:incrond_t tcontext=user_u:object_r:user_home_t tclass=dir

What happens is that incrond read the (user) crontab, found that it had to “watch” /home/user/test2 but fails because SELinux doesn’t allow it to do so. We could just allow that, but we might do it a bit better by looking into what we want it to do in a flexible manner… next time ;-)

May 26, 2013
Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)

Gnupg is an excellent tool for encryption and signing, however, while breaking encryption or forging signatures of large key size is likely somewhere between painful and impossible even for agencies on significant budget, all this is always only as safe as your private key. Let's insert the obvious semi-relevant xkcd reference here, but someone hacking your computer, installing a keylogger and grabbing the key file is more likely. While there are no preventive measures that work for all conceivable attacks, you can at least make things as hard as possible. Be smart, use a smartcard. You'll get a number of additional bonuses on the way. I'm writing up here my personal experiences, as a kind of guide. Also, I am picking a compromise between ultra-security and convenience. Please do not complain if you find guides on the web on how to do things "better".

The smart cards

Obviously, you will need one or more OpenPGP-compatible smart cards and a reader device. I ordered my cards from kernel concepts since that shop is referred in the GnuPG smartcard howto. These are the cards developed by g10code, which is Werner Koch's company (he is the principal author of GnuPG). The website says "2048bit RSA capable", the text printed on the card says "3072bit RSA capable", but at least the currently sold cards support 4096bit RSA keys just fine. (You will need at least app-crypt/gnupg-2.0.19-r2 for encryption keys bigger than 3072bit, see this link and this portage commit.)

The readers

While the GnuPG smartcard howto provides a list of supported reader devices, that list (and indeed the whole document) is a bit stale. The best source of information that I found was the page on the Debian Wiki; Yutaka Niibe, who edits that page regularly, is also one of the code contributors to the smartcard part of GnuPG. In general there are two types of readers, those with a stand-alone pinpad and those without. The extra pinpad takes care that for normal operations like signing and encryption the pin for unlocking the keys is never entering the computer itself- so without tampering with the reader hardware it is impossible pretty hard to sniff it. I bought a SCM SPG532 reader, one of the devices supported ever first by GnuPG, however it's not produced anymore and you may have to resort to newer models soon.

Drivers and software

Now, you'll want to activate the USE flag "smartcard" and maybe "pkcs11", and rebuild app-crypt/gnupg. Afterwards, you may want to log out and back in again, since you may need the gpg-agent from the new emerge.
Several different standards for card reader access exist. One particular is the USB standard for integrated circuit card interface devices, short CCID; the driver for that one is directly built into GnuPG, and the SCM SPG532 is such a device. Another set of drivers is provided by sys-apps/pcsc-lite; that will be used by GnuPG if the built-in stuff fails, but requires a daemon to be running (pcscd, just add it to the default runlevel and start it). The page on the Debian Wiki also lists the required drivers.
These drivers do not need much (or any) configuration, but should work in principle out of the box. Testing is easy, plug in the reader, insert a card, and issue the command
gpg --card-status
If it works, you should see a message about (among other things) manufacturer and serial number of your card. Otherwise, you'll just get an uninformative error. The first thing to check is then (especially for CCID) if the device permissions are OK; just repeat above test as root. If you can now see your card, you know you have permission trouble.
Fiddling with the device file permissions was a serious pain, since all online docs are hopelessly outdated. Please forget about the files linked in the GnuPG smartcard howto. (One cannot be found anymore, the other does not work alone and tries to do things in unnecessarily complicated ways.) At some point in time I just gave up on things like user groups and told udev to hardwire the device to my user account: I created the following file into /etc/udev/rules.d/gnupg-ccid.rules:
ACTION=="add", SUBSYSTEM=="usb", ENV{PRODUCT}=="4e6/e003/*", OWNER:="huettel", MODE:="600"
ACTION=="add", SUBSYSTEM=="usb", ENV{PRODUCT}=="4e6/5115/*", OWNER:="huettel", MODE:="600"
With similar settings it should in principle be possible to solve all the permission problems. (You may want to change the USB id's and the OWNER for your needs.) Then, a quick
udevadm control --reload-rules
followed by unplugging and re-plugging the reader. Now you should be able to check the contents of your card.
If you still have problems, check the following: for accessing the cards, GnuPG starts a background process, the smart card daemon (scdaemon). scdaemon tends to hang every now and then after removing a card. Just kill it (you need SIGKILL)
killall -9 scdaemon
and try again accessing the card afterwards; the daemon is re-started by gnupg. A lot of improvements in smart card handling are scheduled for gnupg-2.0.20; I hope this will be fixed as well.
Here's how a successful card status command looks like on a blank card:
huettel@pinacolada ~ $ gpg --card-status
Application ID ...: D276000124010200000500000AFA0000
Version ..........: 2.0
Manufacturer .....: ZeitControl
Serial number ....: 00000AFA
Name of cardholder: [not set]
Language prefs ...: de
Sex ..............: unspecified
URL of public key : [not set]
Login data .......: [not set]
Signature PIN ....: forced
Key attributes ...: 2048R 2048R 2048R
Max. PIN lengths .: 32 32 32
PIN retry counter : 3 0 3
Signature counter : 0
Signature key ....: [none]
Encryption key....: [none]
Authentication key: [none]
General key info..: [none]
huettel@pinacolada ~ $

That's it for now, part 2 will be about setting up the basic card data and gnupg functions, then we'll eventually proceed to ssh and pam...

Edit: You can find part 2 here.

This is part 2 of a tutorial on OpenPGP smartcard use with Gentoo. Part 1 can be found in an earlier blog post. This time, we assume that you already have a smart card and a functioning reader, and continue setting up the card. Then we'll make everything ready for use with GnuPG by setting up a key pair. As already stated, I am picking a compromise between ultra-security and convenience. Please do not complain if you find guides on the web on how to do things "better". All information here is provided as a best effort, however I urge you to read up on your own. Even if you follow this guide to the last letter- if things break, it is your own responsibility.

Setting the AdminPIN and the PIN

OK, let's start. We insert a blank card into the card reader. The card should come with some paper documentation, stating the initial values of the PIN and the AdminPIN- these we will need in a moment. Now, we want to edit the card properties. We can do this with the command "gpg --card-edit".
jones@pinacolada ~ $ gpg --card-edit 

Application ID ...: D276000124010200000500000AFA0000
Version ..........: 2.0
Manufacturer .....: ZeitControl
Serial number ....: 00000AFA
Name of cardholder: [not set]
Language prefs ...: de
Sex ..............: unspecified
URL of public key : [not set]
Login data .......: [not set]
Signature PIN ....: forced
Key attributes ...: 2048R 2048R 2048R
Max. PIN lengths .: 32 32 32
PIN retry counter : 3 0 3
Signature counter : 0
Signature key ....: [none]
Encryption key....: [none]
Authentication key: [none]
General key info..: [none]

gpg/card> help
quit       quit this menu
admin      show admin commands
help       show this help
list       list all available data
fetch      fetch the key specified in the card URL
passwd     menu to change or unblock the PIN
verify     verify the PIN and list all data
unblock    unblock the PIN using a Reset Code
This menu is not really that helpful yet. However, a lot more commands are hidden below the "admin" keyword:
gpg/card> admin
Admin commands are allowed

gpg/card> help
quit       quit this menu
admin      show admin commands
help       show this help
list       list all available data
name       change card holder's name
url        change URL to retrieve key
fetch      fetch the key specified in the card URL
login      change the login name
lang       change the language preferences
sex        change card holder's sex
cafpr      change a CA fingerprint
forcesig   toggle the signature force PIN flag
generate   generate new keys
passwd     menu to change or unblock the PIN
verify     verify the PIN and list all data
unblock    unblock the PIN using a Reset Code
First of all we change the AdminPIN and the PIN from the manufacturer defaults to some nice random-looking values that only we know.
gpg/card> passwd
gpg: OpenPGP card no. D276000124010200000500000AFA0000 detected

1 - change PIN
2 - unblock PIN
3 - change Admin PIN
4 - set the Reset Code
Q - quit

Your selection? 3
At this point a window from gpg-agent pops up (same as when asking for a passphrase), requests the old AdminPIN and twice the new AdminPIN. Make sure you remember the new AdminPIN or write it down somewhere safe. The AdminPIN allows to change the card parameters (from name of cardholder to stored keys and PIN) and can be used to reset the PIN if you have forgotten it or mistyped it three times. However, if you mistype the AdminPIN three times, your card locks up completely and is basically trash. Note that changing the PINs cannot be done via a reader keypad yet.

PIN changed.

1 - change PIN
2 - unblock PIN
3 - change Admin PIN
4 - set the Reset Code
Q - quit

Your selection? 1
PIN changed.

1 - change PIN
2 - unblock PIN
3 - change Admin PIN
4 - set the Reset Code
Q - quit

Your selection? q

gpg/card>

Setting the cardholder data

Now, let's enter the cardholder data. With the first change you will be prompted for the AdminPIN.
gpg/card> nameCardholder's surname: Jones
Cardholder's given name: Henry W.

gpg/card> lang
Language preferences: en

gpg/card> sex
Sex ((M)ale, (F)emale or space): M

gpg/card> quit
jones@pinacolada ~ $
What are the remaining commands good for? Well...
  • "url" sets an URL where to retrieve the public keys. We will use that later on. 
  • "login" sets a log-in data field. Here you could store your username for e.g. network authentication. 
  • "forcesig" toggles a flag inside the card that has been introduced because of German legislative requirements for some smartcard applications. Normally, once you have inserted the card into the reader, you enter the PIN once for unlocking e.g. the encryption or the signature key, and then the key remains open for the moment. If the signature PIN is "forced", you will have to reenter the PIN again each time you want to make a signature.
  • "generate" generates a RSA key pair directly on the card. This is the "high security option"; the generated private key will and can never leave the card, which enhances its security but also makes backups of the key impossible.
Which leaves the "reset code" to be explained. Imagine you are issued a card by e.g. your employer. The card will be preset with your name, login, and keys, and you should not be able to change that. So, you will not know the AdminPIN. If you enter your user PIN wrong three times in a row, it is invalidated. Now the reset code instead of the AdminPIN can also be used to reset the PIN. Basically this is the same functionality as the PUK for mobile phone SIM cards. The definitive source on all this functionality is the OpenPGP Card 2.0 specification.

Generating GnuPG keypairs

As mentioned in the beginning, there are many different ways to proceed. A keypair can be generated on the card or in the computer. Different types of keys or parts of keys can be uploaded to the card. I'm now presenting the following use case:
  • We generate the GnuPG keys not on the card but on the trusted computer, and then copy them to the card. This makes backups of the keys possible, and you can also upload them later to a second card should the first one accidentally drop into the document shredder.
  • We upload the whole key, not just subkeys as described in some howtos. This makes it possible to access the entire GnuPG functionality from the card- decrypting, signing, and also especially certifying (i.e. signing keys). Of course this means that your primary key is on the card, too.
In general, before you generate a GnuPG keyset you may want to read up on GnuPG best practices; see e.g. this mailing list post of our Gentoo Infra team lead robbat2 for information and further pointers.
Enough talk. We use GPG to generate a 4096bit RSA primary key for signing and certifying with an 4096bit RSA encryption subkey. Note that for all the following steps you need in Gentoo at least app-crypt/gnupg-2.0.19-r2; I strongly recommend app-crypt/gnupg-2.0.20 since there smartcard handling has improved a lot.
jones@pinacolada ~ $ gpg --gen-key
gpg (GnuPG) 2.0.19; Copyright (C) 2012 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
Your selection? 1
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 3y

Key expires at Tue May 24 23:26:58 2016 CEST
Is this correct? (y/N) y

GnuPG needs to construct a user ID to identify your key.

Real name: Henry W. Jones Jr.
Email address: henry.w.jones@uchicago.edu
Comment:
You selected this USER-ID:
    "Henry W. Jones Jr. <henry.w.jones@uchicago.edu>"

Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
You need a Passphrase to protect your secret key.

We need to generate a lot of random bytes. It is a good idea to perform

some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
gpg: /home/jones/.gnupg/trustdb.gpg: trustdb created

gpg: key 14ED37BC marked as ultimately trusted
public and secret key created and signed.

gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: next trustdb check due at 2016-05-24
pub   4096R/14ED37BC 2013-05-25 [expires: 2016-05-24]
      Key fingerprint = 3C94 3AC9 713D E3E3 B3C6  BF73 3898 61DB 14ED 37BC
uid                  Henry W. Jones Jr. <henry.w.jones@uchicago.edu>
sub   4096R/345D5ECB 2013-05-25 [expires: 2016-05-24]

jones@pinacolada ~ $
Got it. Now we do something unusual- in addition to the sign/certify (SC) main key and the encryption (E) subkey, we add a second subkey, an authentication (A) key (for later on). We edit the just generated key with the --expert option:
jones@pinacolada ~ $ gpg --expert --edit 14ED37BC
gpg (GnuPG) 2.0.19; Copyright (C) 2012 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Secret key is available.

pub  4096R/14ED37BC  created: 2013-05-25  expires: 2016-05-24  usage: SC  
                     trust: ultimate      validity: ultimate
sub  4096R/345D5ECB  created: 2013-05-25  expires: 2016-05-24  usage: E   
[ultimate] (1). Henry W. Jones Jr. <henry.w.jones@uchicago.edu>

gpg> addkey
Please select what kind of key you want:

   (3) DSA (sign only)
   (4) RSA (sign only)
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)
   (7) DSA (set your own capabilities)
   (8) RSA (set your own capabilities)
Your selection? 8
We select to add an RSA key where we set the capabilities ourselves. Now we disable Sign and Encrypt, and enable Authenticate instead.
Possible actions for a RSA key: Sign Encrypt Authenticate
Current allowed actions: Sign Encrypt

   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished

Your selection? s

Possible actions for a RSA key: Sign Encrypt Authenticate
Current allowed actions: Encrypt

   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished

Your selection? e

Possible actions for a RSA key: Sign Encrypt Authenticate
Current allowed actions:

   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished

Your selection? a

Possible actions for a RSA key: Sign Encrypt Authenticate
Current allowed actions: Authenticate

   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished

Your selection? q
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 3y
Key expires at Tue May 24 23:39:55 2016 CEST
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.

pub  4096R/14ED37BC  created: 2013-05-25  expires: 2016-05-24  usage: SC  
                     trust: ultimate      validity: ultimate
sub  4096R/345D5ECB  created: 2013-05-25  expires: 2016-05-24  usage: E   
sub  4096R/808D3DB3  created: 2013-05-25  expires: 2016-05-24  usage: A   
[ultimate] (1). Henry W. Jones Jr. <henry.w.jones@uchicago.edu>

gpg> save
jones@pinacolada ~ $
This additional key cannot be used directly by GnuPG, but it is stored in the keyring and will come in handy later on.

Copying the keys to the smartcard

Now we copy the secret keys to the smartcard.
jones@pinacolada ~ $ gpg --edit 14ED37BC
gpg (GnuPG) 2.0.19; Copyright (C) 2012 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Secret key is available.

pub  4096R/14ED37BC  created: 2013-05-25  expires: 2016-05-24  usage: SC  
                     trust: ultimate      validity: ultimate
sub  4096R/345D5ECB  created: 2013-05-25  expires: 2016-05-24  usage: E   
sub  4096R/808D3DB3  created: 2013-05-25  expires: 2016-05-24  usage: A   
[ultimate] (1). Henry W. Jones Jr. <henry.w.jones@uchicago.edu>
With "toggle" we switch from public key to secret key view.
gpg> toggle

sec  4096R/14ED37BC  created: 2013-05-25  expires: 2016-05-24
ssb  4096R/345D5ECB  created: 2013-05-25  expires: never     
ssb  4096R/808D3DB3  created: 2013-05-25  expires: never     
(1)  Henry W. Jones Jr. <henry.w.jones@uchicago.edu>
We select the authentication key and move it to the card (we need the AdminPIN for that):
gpg> key 2

sec  4096R/14ED37BC  created: 2013-05-25  expires: 2016-05-24
ssb  4096R/345D5ECB  created: 2013-05-25  expires: never     
ssb* 4096R/808D3DB3  created: 2013-05-25  expires: never     
(1)  Henry W. Jones Jr. <henry.w.jones@uchicago.edu>

gpg> keytocard
Signature key ....: [none]
Encryption key....: [none]
Authentication key: [none]

Please select where to store the key:
   (3) Authentication key
Your selection? 3

sec  4096R/14ED37BC  created: 2013-05-25  expires: 2016-05-24
ssb  4096R/345D5ECB  created: 2013-05-25  expires: never     
ssb* 4096R/808D3DB3  created: 2013-05-25  expires: never     
                     card-no: 0005 00000AFA
(1)  Henry W. Jones Jr. <henry.w.jones@uchicago.edu>
Then, we select the encryption key and deselect the authentication key; same procedure follows.
gpg> key 1

sec  4096R/14ED37BC  created: 2013-05-25  expires: 2016-05-24
ssb* 4096R/345D5ECB  created: 2013-05-25  expires: never     
ssb* 4096R/808D3DB3  created: 2013-05-25  expires: never     
                     card-no: 0005 00000AFA
(1)  Henry W. Jones Jr. <henry.w.jones@uchicago.edu>

gpg> key 2

sec  4096R/14ED37BC  created: 2013-05-25  expires: 2016-05-24
ssb* 4096R/345D5ECB  created: 2013-05-25  expires: never     
ssb  4096R/808D3DB3  created: 2013-05-25  expires: never     
                     card-no: 0005 00000AFA
(1)  Henry W. Jones Jr. <henry.w.jones@uchicago.edu>

gpg> keytocard
Signature key ....: [none]
Encryption key....: [none]
Authentication key: 8474 2310 057F 1D64 056F  5903 F15B 3DEE 808D 3DB3

Please select where to store the key:
   (2) Encryption key
Your selection? 2

sec  4096R/14ED37BC  created: 2013-05-25  expires: 2016-05-24
ssb* 4096R/345D5ECB  created: 2013-05-25  expires: never     
                     card-no: 0005 00000AFA
ssb  4096R/808D3DB3  created: 2013-05-25  expires: never     
                     card-no: 0005 00000AFA
(1)  Henry W. Jones Jr. <henry.w.jones@uchicago.edu>
Finally we deselect the encryption key, so no subkey is selected anymore, and move the primary (signature/certification) key.
gpg> key 1

sec  4096R/14ED37BC  created: 2013-05-25  expires: 2016-05-24
ssb  4096R/345D5ECB  created: 2013-05-25  expires: never     
                     card-no: 0005 00000AFA
ssb  4096R/808D3DB3  created: 2013-05-25  expires: never     
                     card-no: 0005 00000AFA
(1)  Henry W. Jones Jr. <henry.w.jones@uchicago.edu>

gpg> keytocard
Really move the primary key? (y/N) y
Signature key ....: [none]
Encryption key....: 2050 EC35 2F6C 3EB0 223C  C551 279A 16D7 345D 5ECB
Authentication key: 8474 2310 057F 1D64 056F  5903 F15B 3DEE 808D 3DB3

Please select where to store the key:
   (1) Signature key
   (3) Authentication key
Your selection? 1

sec  4096R/14ED37BC  created: 2013-05-25  expires: 2016-05-24
                     card-no: 0005 00000AFA
ssb  4096R/345D5ECB  created: 2013-05-25  expires: never     
                     card-no: 0005 00000AFA
ssb  4096R/808D3DB3  created: 2013-05-25  expires: never     
                     card-no: 0005 00000AFA
(1)  Henry W. Jones Jr. <henry.w.jones@uchicago.edu>
Now we leave GnuPG, and it's important that we leave without saving. Otherwise, the secret key would be deleted on-disk and only remain on the card. (Of course, this may also be desired.)
gpg> quit
Save changes? (y/N) n
Quit without saving? (y/N) y
jones@pinacolada ~ $
Now, the card is basically ready for use. Let's have a look at its contents once more:
jones@pinacolada ~ $ gpg --card-status
Application ID ...: D276000124010200000500000AFA0000
Version ..........: 2.0
Manufacturer .....: ZeitControl
Serial number ....: 00000AFA
Name of cardholder: Henry W. Jones
Language prefs ...: en
Sex ..............: male
URL of public key : [not set]
Login data .......: [not set]
Signature PIN ....: forced
Key attributes ...: 4096R 4096R 4096R
Max. PIN lengths .: 32 32 32
PIN retry counter : 3 0 3
Signature counter : 0
Signature key ....: 3C94 3AC9 713D E3E3 B3C6  BF73 3898 61DB 14ED 37BC
      created ....: 2013-05-25 21:30:56
Encryption key....: 2050 EC35 2F6C 3EB0 223C  C551 279A 16D7 345D 5ECB
      created ....: 2013-05-25 21:30:56
Authentication key: 8474 2310 057F 1D64 056F  5903 F15B 3DEE 808D 3DB3
      created ....: 2013-05-25 21:39:35
General key info..: pub  4096R/14ED37BC 2013-05-25 Henry W. Jones Jr. <henry.w.jones@uchicago.edu>
sec   4096R/14ED37BC  created: 2013-05-25  expires: 2016-05-24
ssb   4096R/345D5ECB  created: 2013-05-25  expires: 2016-05-24
ssb   4096R/808D3DB3  created: 2013-05-25  expires: 2016-05-24
jones@pinacolada ~ $
We'll discuss how to exactly use the card next time (but that's not really hard to figure out either :). Cheers!

Sven Vermeulen a.k.a. swift (homepage, bugs)

So I’ve shown the iterative approach used to develop policies. Again, please be aware that this is my way of developing policies, other policy developers might have a different approach. We were working on the incrontab command, so let’s continue with trying to create a new user incrontab:

$ incrontab -e
cannot create temporary file: Permission denied

# tail audit.log
type=AVC msg=audit(1368709633.285:28211): avc:  denied  { setgid } for  pid=8159 comm="incrontab" capability=6  scontext=user_u:user_r:incrontab_t tcontext=user_u:user_r:incrontab_t tclass=capability
type=AVC msg=audit(1368709633.285:28212): avc:  denied  { setuid } for  pid=8159 comm="incrontab" capability=7  scontext=user_u:user_r:incrontab_t tcontext=user_u:user_r:incrontab_t tclass=capability
type=AVC msg=audit(1368709633.287:28213): avc:  denied  { search } for  pid=8159 comm="incrontab" name="/" dev="tmpfs" ino=3927 scontext=user_u:user_r:incrontab_t tcontext=system_u:object_r:tmp_t tclass=dir

The requests for the setuid and setgid capabilities are needed for the application to safely handle the user incrontabs. Note that SELinux does not “remove” the setuid bit on the binary itself, but does govern the related capabilities. Since this is required, we will add these capabilities to the policy. We also notice that incrontab searched in the /tmp location.

allow incrontab_t self:capability { setuid setgid };
...
files_search_tmp(incrontab_t)

In the next round of iteration, we notice the same error message with the following denial:

type=AVC msg=audit(1368728433.521:28215): avc:  denied  { write } for  pid=8913 comm="incrontab" name="/" dev="tmpfs" ino=3927 scontext=user_u:user_r:incrontab_t tcontext=system_u:object_r:tmp_t tclass=dir

It is safe to assume here that the process wants to create a temporary file (if it is a directory, we will find out later and can adapt). But when temporary files are created, we better make those files a specific type, like incrontab_tmp_t. So we define that on top of the policy:

type incrontab_tmp_t;
files_tmp_file(incrontab_tmp_t)

Also, we need to allow the incrontab_t domain write privileges into the tmp_t labeled directory, but with an automatic file transition towards incrontab_tmp_t for every file written. This is done through the files_tmp_filetrans method:

files_tmp_filetrans(incrontab_t, incrontab_tmp_t, file)

What this sais is that, if a domain incrontab_t wants to create a file inside tmp_t, then this file is automatically labeled incrontab_tmp_t. With SELinux, you can make this more precise: if you know what the file name would be, then you can add that as a fourth argument. However, this does not seem necessary now since we definitely want all files created in tmp_t to become incrontab_tmp_t. All that rests us is to allow incrontab to actually manage those files:

allow incrontab_t incrontab_tmp_t:file manage_file_perms;

With those in place, let’s look at the outcome:

$ incrontab -e
editor finished with error: No such file or directory

# tail audit.log
type=AVC msg=audit(1368729268.465:28217): avc:  denied  { search } for  pid=8981 comm="incrontab" name="bin" dev="dm-3" ino=524289 scontext=user_u:user_r:incrontab_t tcontext=system_u:object_r:bin_t tclass=dir

Considering that here, incrontab is going to launch the users $EDITOR application to allow him (or her) to create an incrontab, we need to allow incrontab_t not only search privileges inside bin_t directories, but also execute rights: corecmd_exec_bin(incrontab_t). We choose here to execute the editor inside the existing domain (incrontab_t) instead of creating a different domain for the editor for the following reasons:

  • If we would create a separate domain for the editor, the editor would eventually need to have major permissions, depending on when it is used. Editors can be used to modify the sudoers files, passwd files, the /etc/selinux/config file, etc. Instead, it makes much more sense to just be able to launch the editor in the current domain (which is much more confined to its specific purpose)
  • The additional privileges needed to launch the editor are usually very slim, or even nonexistent. It generally only makes sense if, by executing it, the existing domain would need many more privileges, because then a new (confined) domain keeps the privileges for the current domain low.

Let’s see if things work now:

$ incrontab -e
(Editor opened, so I added in an incrontab line. Upon closing:)
cannot move temporary table: Permission denied

# tail audit.log
type=AVC msg=audit(1368729825.673:28237): avc:  denied  { dac_read_search } for  pid=9030 comm="incrontab" capability=2  scontext=user_u:user_r:incrontab_t tcontext=user_u:user_r:incrontab_t tclass=capability
type=AVC msg=audit(1368729825.673:28237): avc:  denied  { dac_override } for  pid=9030 comm="incrontab" capability=1  scontext=user_u:user_r:incrontab_t tcontext=user_u:user_r:incrontab_t tclass=capability

From a quick look through ps, I notice that the application runs as the user (luckily, otherwise I could use the editor to escape and get a root shell) after which it tries to do something. Of course, it makes sense that it wants to move this newly created incrontab file somewhere in /var/spool/incron so we grant it the permission to dac_read_search (which is lower than dac_override as explained before):

allow incrontab_t self:capability { dac_read_search setuid setgid };

On to the next failure:

$ incrontab -e 
cannot move temporary table: Permission denied

# tail audit.log
type=AVC msg=audit(1368730155.706:28296): avc:  denied  { write } for  pid=9088 comm="incrontab" name="incron" dev="dm-4" ino=19725 scontext=user_u:user_r:incrontab_t tcontext=root:object_r:incron_spool_t tclass=dir

Now the application wants to write this file there. Now remember we already have search_dir_perms permissions into incron_spool_t? We need to expand those with read/write permissions into the directory, and manage permissions on files (manage because users should be able to create, modify and delete their files). These two permissions are combined in the manage_files_pattern interface, and makes the search one obsolete:

manage_files_pattern(incrontab_t, incron_spool_t, incron_spool_t)
$ incrontab -e
...
table updated

Finally! And looking at the other options in incrontab, it seems that the policy for incrontab_t is finally complete, and looks like so:

###########################################
#
# incrontab policy
#

allow incrontab_t self:capability { setuid setgid dac_read_search };

manage_files_pattern(incrontab_t, incron_spool_t, incron_spool_t)

allow incrontab_t incrontab_tmp_t:file manage_file_perms;
files_tmp_filetrans(incrontab_t, incrontab_tmp_t, file)

corecmd_exec_bin(incrontab_t)

domain_use_interactive_fds(incrontab_t)

files_search_spool(incrontab_t)
files_search_tmp(incrontab_t)

auth_use_nsswitch(incrontab_t)

userdom_use_user_terminals(incrontab_t)

Next on the agenda: the incrond_t domain.

May 25, 2013
Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
You call it privacy invasion, I don't. (May 25, 2013, 22:51 UTC)

So it looks like the paranoid came to my last post about loyalty cards complaining about the invasion of privacy that these cards come with. Maybe they expected that the myth of the Free Software developer who’s against all big corporation, who wants to be off the grid, and all that kind of stuff that comes out when you think of Stallman. Well, too bad as I’m not like that, while still considering myself a left-winger, but a realist one that cannot see how you can get workers happy by strangling the companies (the alternative to which is not, contrarily to what most people seem to think, just accepting whatever the heck they want).

But first an important disclaimer. What I’m writing here is my personal opinion and in no way that of my employer. Even if my current employer could be considered involved in what I’m going to write, this is an opinion I maintained for years — lu_zero can confirm it.

So, we’ve been told about the evil big brother of loyalty card since I can remember, when I was still a little boy. They can track what you buy, they can profile you, thus they will do bad things to you. But honestly I don’t see that like it has happened at all. Yes, they can track what you buy, they might even profile you, but about the evil things they do to you, I still have not heard of anything — and before you start with the Government (capital and evil G), if you don’t trust your government, a loyalty card programme is the last thing you should be worried in.

Let’s have a look first at the situation presented by the Irish Times article which I referred to in my first post on the topic. At least, they have been close to reality enough, so instead of going the paranoia of the Big Brother, they simply noted that marketeers will know about your life, although they do portray it as only negative.


Before long, he had come up with a list of 25 products which, if bought in certain amounts and in a certain sequence, allowed him to tell if a shopper was pregnant and when her due date was.

In his book, Duhigg tells the story of a man who goes into a branch of Target near Minneapolis. He is not happy as he wants to know why the retailer has suddenly started to send his high school-going daughter coupons for baby clothes and cribs. He asks the manager if the shop is trying to encourage very young girls, such as his daughter, to get pregnant.


The manager is bemused but promises to look into it, which he does. He finds that this girl had indeed been targeted with all manner of promos for baby products so he calls the father several days later to convey his apologies and his confusion.


That’s when the man tells him that when he raised the issue with his daughter, she told him she was pregnant. The retailer took a lot of flak when the details of its data mining emerged but the controversy blew over.

So first I would say I find it utterly ludicrous that sending coupons for “baby clothes and cribs” would “encourage very young girls […] to get pregnant”. I would also suggest that if the girl is so young that it’s scandalous that she could get pregnant, then it might indeed be too soon for her to have a loyalty card. In Italy for instance you have to be 18 before you can get a loyalty card for any program — why? Because you expect that a minor still does not have an absolutely clear idea of what his or her choices are going to mold their future as.

Then let’s see what the problem is about privacy here… if the coupons are sent by mail, one would expect that they are seen only by the addressee — if you have no expectation of privacy on personal mail, it’s hard to blame it strongly on the loyalty programmes. In this case, if you would count the profiling as a violation of privacy of the girl, then you would expect that her father looking at the coupons would be a bigger invasion still. That would be like reading a diary. If you argue that the father has a right to know as she’s a minor, I would answer that then she shouldn’t have the card to begin with.

Then there is the (anonymous, goes without saying) comment on my post, where they try to paint loyalty schemes in an even grimmer light, first by stating that data is sold to third party companies at every turn… well, turns out that’s illegal in most of Europe if you don’t provide a way for the customer not to have his data sold. And turns out that’s one of the few things I do take care of, but simply because I don’t want junk mail from a bunch of companies I don’t really care about. So using the “they’ll sell your detail” scare, to me, sounds like the usual bull.

Then it goes on to say that “Regularly purchasing alcohol and buying in the wrong neighbourhoods will certainly decrease your score to get loans.” — well, so what? The scores are statistical analysis of the chance of recovering or defaulting on a loan, I don’t blame banks for trying to make them more accurate. And maybe it’s because I don’t drink but I don’t see a problem with profiling as an alcoholic a person that would be buying four kegs of beer a day — either that or they have a bar.

Another brought point? A scare on datamining. Okay the term sounds bad, but data mining at the end is just a way for businesses to get better at what they do. If you want to blame them for doing so, it’s your call, but I think you’re out of your mind. There are obvious bad cases for data mining, but that is not the default case. As Jo pointed out on Twitter, we “sell” our shopping habits to the store chains, and what we get back are discounts, coupons and the like. It’s a tit-for-tat scenario, which to me is perfectly fine And applies to more than just loyalty card schemes.

Among others, this is why I have been blocking a number of webrobots on my ModSecurity Ruleset — those that try to get data without giving anything back, for me, are just bad companies. If you want to get something, give something bad back.

And finally, the comment twice uses the phrase, taken from the conspirationists’ rulebook, “This is only the beginning”. Sorry guys, you’ve been saying that this is the beginning for the past thirty years. I start to think you’re not smarter than me, just much more paranoid, too much.

To sum it up, I’m honestly of the opinion that all the people in countries that are in all effect free and democratic that complain about “invasion of privacy”, are only complaining because they want to keep hiding their bad sides, be it bad habits, false statements, or previous errors. Myself, as you can see from this blog, i tend to be fairly open. There is very little I would be embarrassed by, probably only the fact that I do have a profile on a dating site, but even in that, well, I’ve been as honest as a person can be. Did I do something stupid in my past? I think quite a few things. On the other hand, I don’t really care.

So, there you go, this is my personal opinion about all the paranoids who think that they have to live off the grid to be free. Unless you’re in a country that is far from democratic, I’d just say you’re a bunch of crybabies. As I said, places where your Government can’t be trusted, have much bigger problems than loyalty schemes or profiling.

Alexys Jacob a.k.a. ultrabug (homepage, bugs)
San Francisco : street art (May 25, 2013, 12:00 UTC)

000042

000055

000005

Sven Vermeulen a.k.a. swift (homepage, bugs)

Now that our regular user is allowed to execute incrontab, let’s fire it up and look at the denials to build up the policy.

$ incrontab --help

That doesn’t show much does it? Well, if you look into the audit.log (or avc.log) file, you’ll notice a lot of denials. If you are developing a policy, it is wise to clear the entire log and reproduce the “situation” so you get a proper idea of the scope.

# cd /var/log/audit
# > audit.log
# tail -f audit.log | grep AVC

Now let’s run incrontab –help again and look at the denials:

type=AVC msg=audit(1368707274.429:28180): avc:  denied  { read write } for  pid=7742 comm="incrontab" path="/dev/tty2" dev="devtmpfs" ino=1042 scontext=user_u:user_r:incrontab_t tcontext=user_u:object_r:user_tty_device_t tclass=chr_file
type=AVC msg=audit(1368707274.429:28180): avc:  denied  { use } for  pid=7742 comm="incrontab" path="/dev/tty2" dev="devtmpfs" ino=1042 scontext=user_u:user_r:incrontab_t tcontext=system_u:system_r:getty_t tclass=fd
type=AVC msg=audit(1368707274.429:28180): avc:  denied  { use } for  pid=7742 comm="incrontab" path="/dev/tty2" dev="devtmpfs" ino=1042 scontext=user_u:user_r:incrontab_t tcontext=system_u:system_r:getty_t tclass=fd
type=AVC msg=audit(1368707274.429:28180): avc:  denied  { use } for  pid=7742 comm="incrontab" path="/dev/tty2" dev="devtmpfs" ino=1042 scontext=user_u:user_r:incrontab_t tcontext=system_u:system_r:getty_t tclass=fd

You can start piping this information into audit2allow to generate policy statements, but I personally prefer not to use audit2allow for building new policies. For one, it is not intelligent enough to deduce if a denial should be fixed by allowing it, or by relabeling or even by creating a new type. Instead, it always grants it. Second, it does not know if a denial is cosmetic (and thus can be ignored) or not.

This latter is also why I don’t run domains in permissive mode to see the majority of denials first and to build from those: you might see denials that are actually never triggered when running in enforcing mode. So let’s look at the access to /dev/tty2. Given that this is a user application where we expect output to the screen, we want to grant it the proper access. With sefindif as documented before, we can look for the proper interfaces we need. I look for user_tty_device_t with rw (commonly used for read-write):

$ sefindif user_tty_device_t.*rw
system/userdomain.if: template(`userdom_base_user_template',`
system/userdomain.if:   allow $1_t user_tty_device_t:chr_file { setattr rw_chr_file_perms };
system/userdomain.if: interface(`userdom_use_user_ttys',`
system/userdomain.if:   allow $1 user_tty_device_t:chr_file rw_term_perms;
system/userdomain.if: interface(`userdom_use_user_terminals',`
system/userdomain.if:   allow $1 user_tty_device_t:chr_file rw_term_perms;
system/userdomain.if: interface(`userdom_dontaudit_use_user_terminals',`
system/userdomain.if:   dontaudit $1 user_tty_device_t:chr_file rw_term_perms;
system/userdomain.if: interface(`userdom_dontaudit_use_user_ttys',`
system/userdomain.if:   dontaudit $1 user_tty_device_t:chr_file rw_file_perms;

Two of these look interesting: userdom_use_user_ttys and userdom_use_user_terminals. Looking at the API documentation (or the rules defined therein using seshowif) reveals that userdom_use_user_terminals is needed if you also want the application to work when invoked through a devpts terminal, which is probably also something our user(s) want to do, so we’ll add that. The second one – using the file descriptor that has the getty_t context – is related to this, but not granted through the userdom_use_user_ttys. We could grant getty_use_fds but my experience tells me that domain_use_interactive_fds is more likely to be needed: the application inherits and uses a file descriptor currently owned by getty_t but it could be from any of the other domains that has such file descriptors. For instance, if you grant the incron_role to sysadm_r, then a user that switched roles through newrole will see denials for using a file descriptor owned by newrole_t.

Experience is an important aspect in developing policies. If you would go through with getty_use_fds it would work as well, and you’ll probably hit the above mentioned experience later when you try the application through a few different paths (such as within a screen session or so). When you think that the target context (in this case getty_t) could be a placeholder (so other types are likely to be needed as well), make sure you check which attributes are assigned to the type:

# seinfo -tgetty_t -x
   getty_t
      privfd
      mcssetcats
      mlsfileread
      mlsfilewrite
      application_domain_type
      domain

Of the above ones, privfd is the important one:

$ sefindif privfd.*use
kernel/domain.if: interface(`domain_use_interactive_fds',`
kernel/domain.if:       allow $1 privfd:fd use;
kernel/domain.if: interface(`domain_dontaudit_use_interactive_fds',`
kernel/domain.if:       dontaudit $1 privfd:fd use;

So let’s update incron.te accordingly:

...
type incron_spool_t;
files_type(incron_spool_t)

###########################################
#
# incrontab policy
#

userdom_use_user_terminals(incrontab_t)
domain_use_interactive_fds(incrontab_t)

Rebuild the policy and load it in memory.

If we now run incrontab we get the online help as we expected. Let’s now look at the currently installed incrontabs (there shouldn’t be any of course):

$ incrontab -l
cannot determine current user

In the denials, we notice:

type=AVC msg=audit(1368708632.060:28192): avc:  denied  { create } for  pid=7968 comm="incrontab" scontext=user_u:user_r:incrontab_t tcontext=user_u:user_r:incrontab_t tclass=unix_stream_socket
type=AVC msg=audit(1368708632.060:28194): avc:  denied  { read } for  pid=7968 comm="incrontab" name="nsswitch.conf" dev="dm-2" ino=393768 scontext=user_u:user_r:incrontab_t tcontext=system_u:object_r:etc_t tclass=file
type=AVC msg=audit(1368708632.062:28196): avc:  denied  { read } for  pid=7968 comm="incrontab" name="passwd" dev="dm-2" ino=394223 scontext=user_u:user_r:incrontab_t tcontext=system_u:object_r:etc_t tclass=file

Let’s first focus on nsswitch.conf and passwd. Although both require read access to etc_t files, it might be wrong to just add in files_read_etc (which is what audit2allow is probably going to suggest). For nsswitch, there is a special interface available: auth_use_nsswitch. It is very, very likely that you’ll need this one, especially if you want to share the policy with others who might not have all of the system databases in local files (as etc_t files).

...
domain_use_interactive_fds(incrontab_t)
auth_use_nsswitch(incrontab_t)

Let’s retry:

$ incrontab -l
cannot read table for 'user': Permission denied

# tail audit.log
type=AVC msg=audit(1368708893.260:28199): avc:  denied  { search } for  pid=7997 comm="incrontab" name="spool" dev="dm-4" ino=20 scontext=user_u:user_r:incrontab_t tcontext=system_u:object_r:var_spool_t tclass=dir

So we need to grant search privileges on var_spool_t. This is offered through files_search_spool. Add it to the policy, rebuild and retry.

$ incrontab -l
cannot read table for 'user': Permission denied

# tail audit.log
type=AVC msg=audit(1368709146.426:28201): avc:  denied  { search } for  pid=8046 comm="incrontab" name="incron" dev="dm-4" ino=19725 scontext=user_u:user_r:incrontab_t tcontext=root:object_r:incron_spool_t tclass=dir

For this one, no interface exists yet. We might be able to create one for ourselves, but as long as other domains don’t need it, we can just add it locally in our policy:

allow incrontab_t incron_spool_t:dir search_dir_perms;

Adding raw allow rules in a policy is, according to the refpolicy styleguide, only allowed if the policy module defines both the source and the destination type of the rule. If you look into other policies you might also find that you can use the search_dirs_patter call. However, that one only makes sense if you need to do this on top of another directory – just look at the definition of search_dirs_pattern. So with this permission set, let’s retry.

$ incrontab -l
no table for user

Great, we have successfully updated the policy until the commands worked. In the next post, we’ll enhance it even further while creating new incrontabs.

May 24, 2013
Sven Vermeulen a.k.a. swift (homepage, bugs)

The next step after having a basic skeleton is to get incrontab running. We know however that everything invoked from the main daemon will be running with the rights of the daemon context (unless we would patch the source code, but that is beyond the scope of this set of posts). As a result, we probably do not want everyone to be able to launch commands through this application.

What we want to do is to limit who can invoke incrontab and, as such, limit who can decide what is invoked through incrond. First of all, we define a role attribute called incrontab_roles. Every role that gets this attribute assigned will be able to transition to the incrontab_t domain.

We can accomplish this by editing the incron.te file:

policy_module(incron, 0.2)

# Declare the incrontab_roles attribute
attribute_role incrontab_roles;

...
type incrontab_t;
type incrontab_exec_t;
application_domain(incrontab_t, incrontab_exec_t)
# Allow incrontab_t for all incrontab_roles 
role incrontab_roles types incrontab_t;

Next, we need something where we can allow user domains to call incrontab. This will be done through an interface. Let’s look at incron.if with one such interface in it: the incron_role interface.

## inotify-based cron-like daemon

#########################################
## <summary>
##      Role access for incrontab
## </summary>
## <param name="role">
##      <summary>
##      Role allowed access.
##      </summary>
## </param>
## <param name="domain">
##      <summary>
##      User domain for the role.
##      </summary>
## </param>
#
interface(`incron_role',`
        gen_require(`
                attribute_role incrontab_roles;
                type incrontab_exec_t, incrontab_t;
        ')

        roleattribute $1 incrontab_roles;

        domtrans_pattern($2, incrontab_exec_t, incrontab_t)

        ps_process_pattern($2, incrontab_t)
        allow $2 incrontab_t:process signal;
')

The comments in the file are somewhat special: if the comments start with two hashes (##) then it is taken into account while building the policy documentation in /usr/share/doc/selinux-base-*. The interface itself, incron_role, grants a user role and domain the necessary privileges to transition to the incrontab_t domain as well as read process information (as used through ps, hence the name of the pattern being ps_process_pattern) and send a standard signal to it. Most of the time, you can use signal_perms here but from looking at the application we see that the application is setuid root, so we don’t want to grant too many privileges by default if they are not needed.

With this interface file created, we can rebuild the module and load it.

# make -f /usr/share/selinux/strict/include/Makefile incron.pp
# semodule -i incron.pp

But how to assign this interface to users? Well, what we want to do is something like the following:

incron_role(user_r, user_t)

When interfaces are part of the policy provided by the distribution, the definitions of it are stored in the proper location and you can easily add it. For instance, in Gentoo, if you want to allow the user_r role and user_t domain the cron_role access (and assuming it doesn’t have so already), then you can call selocal as follows:

# selocal -a "cron_role(user_r, user_t)" -c "Granting user_t cron access" -Lb

However, because the interface is currently not known yet, we need to create a second small policy that does this. Create a file (called localuser.te or so) with the following content:

policy_module(localuser, 0.1)

gen_require(`
        type user_t;
        role user_r;
')

incron_role(user_r, user_t)

Now build the policies and load them. We’ll now just build and load all the policies in the current directory (which will be the incron and localuser ones):

# make -f /usr/share/selinux/strict/include/Makefile
# semodule -i *.pp

You can now verify that the user is allowed to transition to the incrontab_t domain:

# seinfo -ruser_r -x | grep incron
         incrontab_t
# sesearch -s user_t -t incrontab_exec_t -AdCTS
Found 1 semantic av rules:
   allow user_t incrontab_exec_t : file { read getattr execute open } ; 

Found 1 semantic te rules:
   type_transition user_t incrontab_exec_t : process incrontab_t; 

Great, let’s get to our first failure to resolve… in the next post ;-)

May 23, 2013
Sven Vermeulen a.k.a. swift (homepage, bugs)

So, in the previous post I talked about incron and why I think moving it into the existing cron policy would not be a good idea. It works, somewhat, but is probably not that future-proof. So we’re going to create our own policy for it.

In SELinux, policies are generally written through 3 files:

  1. a type enforcement file that contains the SELinux rules applicable to the domain(s) related to the application (in our example, incron)
  2. a file context file that tells the SELinux utilities how the files and directories offered by the application should be labeled
  3. an interface definition file that allows other SELinux policy modules to gain rights offered through the (to be written) incron policy

We now need to create a skeleton for the policy. This skeleton will define the types related to the application. Such types can be the domains for the processes (the context of the incrond and perhaps also incrontab applications), the contexts for the directories (if any) and files, etc.

So let’s take a look at the content of the incron package. On Gentoo, we can use qlist incron for this. In the output of qlist, I added comments to show you how contexts can be (easily) deduced.

# Application binary for managing user crontabs. We want to give this a specific
# context because we want the application (which will manage the incrontabs in
# /var/spool/incron) in a specific domain
/usr/bin/incrontab  ## incrontab_exec_t

# General application information files, do not need specific attention
# (the default context is fine)
/usr/share/doc/incron-0.5.10/README.bz2
/usr/share/doc/incron-0.5.10/TODO.bz2
/usr/share/doc/incron-0.5.10/incron.conf.example.bz2
/usr/share/doc/incron-0.5.10/CHANGELOG.bz2
/usr/share/man/man8/incrond.8.bz2
/usr/share/man/man5/incron.conf.5.bz2
/usr/share/man/man5/incrontab.5.bz2
/usr/share/man/man1/incrontab.1.bz2

# Binary for the incrond daemon. This definitely needs its own context, since
# it will be launched from an init script and we do not want it to run in the
# initrc_t domain.
/usr/sbin/incrond ## incrond_exec_t

# This is the init script for the incrond daemon. If we want to allow 
# some users the rights to administer incrond without needing to grant
# those users the sysadm_r role, we need to give this file a different
# context as well.
/etc/init.d/incrond ## incrond_initrc_exec_t

With this information at hand, and the behavior of the application we know from the previous post, can lead to the following incron.fc file, which defines the file contexts for the application.

/etc/incron.d(/.*)?     gen_context(system_u:object_r:incron_spool_t,s0)

/etc/rc\.d/init\.d/incrond      --      gen_context(system_u:object_r:incrond_initrc_exec_t,s0)

/usr/bin/incrontab      --      gen_context(system_u:object_r:incrontab_exec_t,s0)

/usr/sbin/incrond       --      gen_context(system_u:object_r:incrond_exec_t,s0)

/var/spool/incron(/.*)?         gen_context(system_u:object_r:incron_spool_t,s0)

The syntax of this file closely follows the syntax that semanage fcontext takes – at least for the regular expressions in the beginning. The last column is specifically for policy development to generate a context based on the policies’ requirements: an MCS/MLS enabled policy will get the trailing sensitivity with it, but when MCS/MLS is disabled then it is dropped. The middle column is to specify if the label should only be set on regular files (--), directories (-d), sockets (-s), symlinks (-l), etc. If it is omitted, it matches whatever class the path matches.

The second file needed for the skeleton is the incron.te file, which would look like so. I added in inline comments here to explain why certain lines are prepared, but generally this is omitted when the policy is upstreamed.

policy_module(incron, 0.1)
# The above line declares that this file is a SELinux policy file. Its name
# is incron, so the file should saved as incron.te

# First, we declare the incrond_t domain, used for the "incrond" process.
# Because it is launched from an init script, we tell the policy that
# incrond_exec_t (the context of incrond), when launched from init, should
# transition to incrond_t.
#
# Basically, the syntax here is:
# type 
# type 
# 
type incrond_t;
type incrond_exec_t;
init_daemon_domain(incrond_t, incrond_exec_t)

# Next we declare that the incrond_initrc_exec_t is an init script context
# so that init can execute it (remember, SELinux is a mandatory access control
# system, so if we do not tell that init can execute it, it won't).
type incrond_initrc_exec_t;
init_script_file(incrond_initrc_exec_t)

# We also create the incrontab_t domain (for the "incrontab" application), which
# is triggered through the incrontab_exec_t labeled file. This again follows a bit
# the syntax as we used above, but now the interface call is "application_domain".
type incrontab_t;
type incrontab_exec_t;
application_domain(incrontab_t, incrontab_exec_t)

# Finally we declare the spool type as well (incron_spool_t) and tell SELinux that
# it will be used for regular files.
type incron_spool_t;
files_type(incron_spool_t)

Knowing which interface calls, like init_daemon_domain and application_domain, we should use is not obvious at first. Most of this can be gathered from existing policies. Other frequently occurring interfaces to be used immediately at the skeleton side are (examples for a foo_t domain):

  • logging_log_file(foo_log_t) to inform SELinux that the context is used for logging purposes. This allows generic log-related daemons to do “their thing” with the file.
  • files_tmp_file(foo_tmp_t) to identify the context as being used for temporary files
  • files_tmpfs_file(foo_tmpfs_t) for tmpfs files (which could be shared memory)
  • files_pid_file(foo_var_run_t) for PID files (and other run metadata files)
  • files_config_file(foo_conf_t) for configuration files (often within /etc)
  • files_lock_file(foo_lock_t) for lock files (often within /run/lock)

We might be using these later as we progress with the policy (for instance, the PID file is a very high candidate for needing to be included). However, with the information currently at hand, we have our first policy module ready for building. Save the type enforcement rules in incron.te and the file contexts in incron.fc and you can then build the SELinux policy:

# make -f /usr/share/selinux/strict/include/Makefile incron.pp
# semodule -i incron.pp

On Gentoo, you can then relabel the files and directories offered through the package using rlpkg:

# rlpkg incron

Next is to start looking at the incrontab application.

May 22, 2013
Sven Vermeulen a.k.a. swift (homepage, bugs)

In this series of posts, we’ll go through the creation of a SELinux policy for incron, a simple inotify based cron-like application. I will talk about the various steps that I would take in the creation of this policy, and give feedback when certain decisions are taken and why. At the end of the series, we’ll have a hopefully well working policy.

The first step in developing a policy is to know what the application does and how/where it works. This allows us to check if its behavior matches an existing policy (and as such might be best just added to this policy) or if a new policy needs to be written. So, what does incron do?

From the documentation, we know that incron is a cron-like application that, unlike cron, works with file system notification events instead of time-related events. Other than that, it uses a similar way of working:

  • A daemon called incrond is the run-time application that reads in the incrontab files and creates the proper inotify watches. When a watch is triggered, it will execute the matching rule.
  • The daemon looks at two definitions (incrontabs): one system-wide (in /etc/incron.d) and one for users (in /var/spool/incron).
  • The user tabfiles are managed through incrontab (the command)
  • Logging is done through syslog
  • User commands are executed with the users’ privileges (so the application calls setuid() and setgid())

With this, one can create a script to be executed when a file is uploaded (or deleted) to/from a file server, or when a process coredump occurred, or whatever automation you want to trigger when some file system event occurred. Events are plenty and can be found in /usr/include/sys/inotify.h.

So, with this information, it is safe to assume that we might be able to push incron in the existing cron policy. After all, it defines the contexts for all these and probably doesn’t need any additional tweaking. And this seems to work at first, but a few tests reveal that the behavior is not that optimal.

# chcon -t crond_exec_t /usr/sbin/incrond
# chcon -t crontab_exec_t /usr/bin/incrontab
# chcon -R -t system_cron_spool_t /etc/incron.d
# chcon -t cron_log_t /var/log/cron.log
# chcon -R -t cron_spool_t /var/spool/incron

System tables work somewhat, but all commands are executed in the crond_t domain, not in a system_cronjob_t or related domain.
User tables fail when dealing with files in the users directories, since these too run in crond_t and thus have no read access to the user home directories.

The problems we notice come from the fact that the application is very simple in its code: it is not SELinux-aware (so it doesn’t change the runtime context) as most cron daemons are, and when it changes the user id it does not call PAM, so we cannot trigger pam_selinux.so to handle context changes either. As a result, the entire daemon keeps running in crond_t.

This is one reason why a separate domain could be interesting: we might want to extend the rights of the daemon domain a bit, but don’t want to extend these rights to the other cron daemons (who also run in crond_t). Another reason is that the cron policy has a few booleans that would not affect the behavior at all, making it less obvious for users to troubleshoot. As a result, we’ll go for the separate policy instead – which will be for the next post.

May 21, 2013
Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)

My original post about loyalty cards missed the supermarkets that I’m actually using nowadays, because they are conveniently located just behind my building (for one) and right on the way back home from my office (for the other). Both of them are part of the EuroSpar chain and have the added convenience of being open respectively 24/7 and 7-22.

Mangled bill from EuroSpar

So, when I originally asked the store if they had any loyalty card, I was told they didn’t. I checked the website anyway and found the name of their loyalty program, which is “SuperEasy”, and the next time, I asked about it explicitly, and they gave me the card and a form to fill in; after filling almost all of it, I found that I could also do it online, so I trashed the paper form. They can’t get my name right anywhere here when I spell it.

On the website, strangely enough they even accept my surname as it should be, wow that’s a miracle, I thought… until I went to use the card at the shop and got back the bill that you see on the left. Yes that’s UTF-8 converted to some other 8-bit codepage which is not Latin-1. Indeed it reminds me of CP850 at the time of MS-DOS. Okay I give up, but the funniest part was getting the bill tonight, the one on the right.

The other mangled bill from EuroSpar

But beside them mangling my name in many different possible ways, is there anything that makes EuroSpar special enough for me to write a follow-up post on a topic that I don’t really care about or, honestly, have experience in? Yes of course. Compared with the various rewards I have been talking about last time, this seems to be mostly the same: one point per euro spent, and one cent per point redeemed.

The big difference here is that the points are accrued to the cent, rather than to the lower euro threshold! Not too shabby, considering that unlike Dunnes they do not round their prices to full euros most of the time. And the other one is that even though they have a single loyalty scheme for all the stores.. the cards are per-store, or so they proclaim. The two here are probably owned by the same person so they are actually linked and they work on each.

Another interesting point is that while both EuroSpar host an Insomnia café, neither accept Insomnia’s own loyalty card (ZapaTag) — instead they offer something similar in the sense that you get the 10th drink free. A similar offer is present at the regular Insomnia shops, but there, while you can combine the 10th drink offer with the ZapaTag points, you cannot combine it with other offers such as my usual coffee and brownie for €3,75 (the coffee alone is €3,25 while the brownie is €2,25)… at EuroSpar instead this is actually combinable, but of course if I use the free coffee while getting a brownie, I still have to pay almost as much as the coffee.. but sometimes I can skip on the pastry.

So yes, I think it was worth noting the differences about EuroSpar. And as a final note I’ll just say that even the pharmacy on the way to work has a loyalty card… and it’s the usual discount one, or as they call it “PayBack Card”. I have to see what Tesco does, but they somehow blacklisted my apartment in their delivery service.

Alexys Jacob a.k.a. ultrabug (homepage, bugs)
rabbitMQ : v3.1.1 released (May 21, 2013, 13:04 UTC)

EDIT: okay, they just released v3.1.1 so here it goes on portage as well !

highlights

  • relax validation of x-match binding to headers exchange for compatibility with brokers < 3.1.0
  • fix bug in ack handling for transactional channels that could cause queues to crash
  • fix race condition in cluster autoheal that could lead to nodes failing to re-join the cluster

3.1.1 changelog is here.

I’ve bumped the rabbitMQ message queuing server on portage. This new version comes with quite a nice bunch of bugfixes and features.

highlights

  • eager synchronisation of slaves by policy (manual & automatic)
  • cluster “autoheal” mode to automatically choose nodes to restart when a partition has occurred
  • cluster “pause minority” mode to prefer partition tolerance over availability
  • improved statistics (including charts) in the management plugin
  • quite a bunch of performance improvements
  • some nice memory leaks fixes

Read the full changelog.

Sven Vermeulen a.k.a. swift (homepage, bugs)

If you notice that a process is running in the unlabeled_t domain, the first question to ask is how it got there.

Well, one way is to have a process running in a known domain, like screen_t, after which the SELinux policy module that provides this domain is removed from the system (or updated and the update does not contain the screen_t definition anymore):

test ~ # ps -eZ | grep screen
root:sysadm_r:sysadm_screen_t    5047 ?        00:00:00 screen
test ~ # semodule -r screen
test ~ # ps -eZ | grep screen
system_u:object_r:unlabeled_t    5047 ?        00:00:00 screen

In permissive mode, this will be visible easily; in enforcing mode, the domains you are running in might not be allowed to do anything with unlabeled_t files, directories and processes, so ps might not show it even though it still exists:

test audit # ps -eZ | grep 5047
test audit # ls -dZ /proc/5047
ls: cannot access /proc/5047: Permission denied
test audit # tail audit.log | grep unlabeled
type=AVC msg=audit(1368698097.494:27806): avc:  denied  { getattr } for  pid=4137 comm="bash" path="/proc/5047" dev="proc" ino=6677 scontext=root:sysadm_r:sysadm_t tcontext=system_u:object_r:unlabeled_t tclass=dir

Notice that, if you reload the module, the process becomes visible again. That is because the process context itself (screen_t) is retained, but because the policy doesn’t know it anymore, it shows it as unlabeled_t.

Basically, the moment the policy doesn’t know how a label would be (should be), it uses unlabeled_t. The SELinux policy then defines how this unlabeled_t domain is handled. Processes getting into unlabeled_t is not that common though as there is no supported transition to it. The above one is one way that this still can occur.

May 19, 2013
Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)

I was very sceptic for a long time. Then, I slowly started to trust the kmail2/akonadi combination. I've been using it on my office desktop for a long time, and it works well and is very stable and fast there. (Might be related to the fact that the IMAP server is just across the lawn.) Some time ago, when I deemed things solid enough I even upgraded my laptop again, despite earlier problems. In Gentoo, we've been keeping kdepim-4.4 around all the time, and as you may have read, internal discussions led indeed to the decision to finally drop it some time ago.
What happened in the meantime?
1) One of the more annoying bugs mentioned in my last blog post was fixed with some help from Kevin Kofler. Seems like Debian stumbled into the same issue long ago.
2) I was on vacation. Which was fun, but mostly unrelated to the issue at hand. None of my Gentoo colleagues went ahead with the removal in the meantime. A lot of e-mails accumulated in my account.
3) Coming back, I was on the train with my laptop, sorting the mail. The train was full, the onboard WLAN slightly overstressed, the 4G network just about more reliable. Network comes and goes sometime with a tunnel, no problem. Or so I thought.
4) Half an hour before arriving back home I realized that silently a large part of the e-mails that I had (I though) moved (using kmail2-4.10.3 / akonadi-1.9.2) from one folder to another over ~3 hours had disappeared on one side, and not re-appeared on the other. Restarting kmail2 and akonadi did not help. A quick check of the webmail interface of my provider confirmed that also on the IMAP server the mails were gone in both folders. &%(/&%(&/$/&%$§&/
I wasn't happy. Luckily there were daily server backup snapshots, and after a few days delay I had all the documents back. Nevertheless... Now, I am considering what to do next. (Needless to say, in my opinion we should forget dropping kmail1 in Gentoo for now.) Options...
a) migrate the laptop back to kmail1, which is way more resistant to dropped connections and flaky internet connection - doable but takes a bit of time
b) install OfflineIMAP and Dovecot on the laptop, and let kmail2/akonadi access the localhost Dovecot server - probably the most elegant solution but for the fact that OfflineIMAP seems to have trouble mirroring our Novell Groupwise IMAP server
c) other e-mail client? I've heard good things about trojita...
Summarizing... no idea still how to go ahead, no good solution available. And I actually like the kdepim integration idea, so I'll never be the first one to completely migrate away from it! I am sincerely sorry for the sure fact that this post is disheartening to all the people who put a lot of effort into improving kmail2 and akonadi. It has become a huge lot better. However, I am just getting more and more convinced that the complexity of this combined system is too much to handle and that kmail should never have gone the akonadi way.

May 18, 2013
Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)
Gentoo CUPS-1.6 status (May 18, 2013, 21:02 UTC)

We've had CUPS 1.6 in the Gentoo portage tree for a while now already. It has even been keyworded by most of the arches (hooray!), and from the bug reports quite some people use it. Sometime in the intermediate future we'll stabilize it, however until then quite some bugs still have to be resolved.
CUPS 1.6 brings changes. The move to Apple has messed up the project priorities, and backward compatibility was kicked out of the window with a bang. As I've already detailed in a short previous blog post, per se, CUPS 1.6 does not "talk" the printer browsing protocol of previous versions anymore but solely relies on zeroconf (which is implemented in Gentoo by net-dns/avahi). Some other features were dropped as well...
Luckily, CUPS was and is open source, and that the people at Apple removed the code from the main CUPS distribution did not mean that it was actually gone. In the end, all these feature just made their way from the main CUPS package to a new package net-print/cups-filters maintained at The Linux Foundation. There, the code is evolving fast, bugs are fixed and features are introduced. Even network browsing with the CUPS-1.5 protocol has been restored by now; cups-filters includes a daemon called cups-browsed which can generate print queues on the fly and accepts configuration directives similar to CUPS-1.5. As far as we in Gentoo (and any other Linux distribution) are concerned, we can get along without zeroconf just fine.
The main thing that is hindering CUPS-1.6 stabilization a the moment is that the CUPS website is down, kind of. Their server had a hardware failure, and since nearly a month (!!!) only minimal, static pages are up. In particular, what's missing is the CUPS bugtracker (no I won't sign up for an Apple ID to submit CUPS bugs) and access to the Subversion repository of the source. (Remind me to git-svn clone the code history as soon as it's back and push it to gitorious.)
So... feel free to try out CUPS-1.6, testing and submitting bugs for sure helps. However, it may take some time to get these fixed...

May 17, 2013
Alexys Jacob a.k.a. ultrabug (homepage, bugs)

It is a common request in squid to have it block downloading certain files based on their extension in the url path. A quick look at google’s results on the subject apparently gives us the solution to get this done easily by squid.

The common solution is to create an ACL file listing regular expressions of the extensions you want to block and then apply this to your http_access rules.

blockExtensions.acl

\.exe$

squid.conf

acl blockExtensions urlpath_regex -i "/etc/squid/blockExtensions.acl"

[...]

http_access allow localnet !blockExtensions

Unfortunately this is not enough to prevent users from downloading .exe files. The mistake here is that we assume that the URL will strictly finish by the extension we want to block, consider the two examples below :

http://download.com/badass.exe     // will be DENIED as expected

http://download.com/badass.exe?    // WON'T be denied as it does not match the regex !

Squid uses the extended regex processor which is the same as egrep. So we need to change our blockExtensions.acl file to handle the possible ?whatever string which may be trailing our url_path. Here’s the solution to handle all the cases :

blockExtensions.acl

\.exe(\?.*)?$
\.msi(\?.*)?$
\.msu(\?.*)?$
\.torrent(\?.*)?$

You will still be hated for limiting people’s need to download and install shit on their Windows but you implemented it the right way and no script kiddie can brag about bypassing you ;)

Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
Life in the new city (May 17, 2013, 19:54 UTC)

Okay so now it’s over a month I’ve been staying in Dublin, it’s actually over a month I’m at my new job, and it is shaping up as a very good new experience for me. But even more than the job, the new experiences come with having an apartment. Last year I was leaving within the office where I was working, and before that I’ve been living with my mother, so finally having a place of mine is a new world entirely. Well, I’ll admit it: only partially.

Even though I’ve been living with my mother, like the stereotype of Italian guys suggests, it’s not like I’ve bee a parasite. Indeed, I’ve been paying all the bills for the past four years, and still I’m paying them from here. I’ve also been doing my share of grocery shopping, cleaning and maintenance tasks, but at least I did avoid the washing machine most of the time. So yeah, it wasn’t a complete revolution for my life, but it was a partial one. So right now I do feel slightly worse for wear, especially because I had a very bad experience with the kitchen, which was not cleaned before I moved in.

Thankfully, Ikea exists everywhere. And their plastic mats for drawers and cabinets are a lifesaver. Too bad I already finished the roll and I’ve not completed half the kitchen yet. I think I’ll go back to Ikea in two weeks (not next week because my sister’s visiting). With this time I bought the same identical lamp three times. Originally in Italy, then again in Los Angeles, and now in Dublin — only difference is that the American version has a loop to be able to orient it, probably because health and safety does not require having enough common sense as to not touch the hot cone…

The end line is that I’m very happy about having moved to Dublin. I love the place, and I love the people. My new job is also quite interesting, even if not as open-source focused as my previous ones (which does not mean it is completely out of the way of open source anyway), and the colleagues are terrific… hey some even read my blog before, thanks guys!

While settling down took most of my time and left me no time to do real Gentoo contributions or blogging (luckily Sven seems to have taken my place on Planet Gentoo), things are getting much better (among others I finally have a desk in the apartment, and tomorrow I’m going to get a TV as well, which I know will boost my ability to keep the house clean — because it won’t require me to stick to the monitor to watch something). So expect more presence from me soon enough!

Greg KH a.k.a. gregkh (homepage, bugs)

A few years ago, I gave a history of the 2.6.32 stable kernel, and mentioned the previous stable kernels as well. I'd like to apologize for not acknowledging the work of Adrian Bunk in maintaining the 2.6.16 stable kernel for 2 years after I gave up on it, allowing it to be used by many people for a very long time.

I've updated the previous post with this information in it at the bottom, for the archives. Again, many apologies, I never meant to ignore the work of this developer.

May 16, 2013
Alexys Jacob a.k.a. ultrabug (homepage, bugs)
Fujifilm GF670W (May 16, 2013, 20:12 UTC)

It’s been so long since I switched to film-only photography that I decided a few months ago to sell all my digital equipment. I already own a Nikon FM2 camera which I love but I’ve to admit that I was and still am totally amazed by the pictures taken by my girlfriend’s Rolleiflex 3.5F. The medium format is the kind of rendering I was craving to get and that sooner or later I’d step into the medium format world. Well, I didn’t have to wait as when we were in Tokyo to celebrate new year 2013 I fell in love with what was the perfect match between my love for wide angles and medium format film photography : the Fujifilm GF670W !

For my soon to come birthday, I got myself my new toy in advance so I could use it in my upcoming roadtrip around France (I’ll talk about it soon, it was awesome). Oddly, the only places in the world where you can get this camera is in the UK and in Japan so I bought it from the very nice guys at Dale photographic. Here is the beast (literally) :

IMG_20130412_215344

Yes, this is a big camera and it comes with a very nice leather case and a lens hood. This is a telemetric camera with a comfortable visor, it accepts 120 and 220 films and is capable of shooting in standard 6×6 and 6×7 !

In the medium format world, the 55mm lens is actually a wide angle one as it is comparable to a 28mm in the usual 24×36 world. Its performances are not crazy on paper with a 4.5 aperture and a shutter speed going from 4s to 1/500s (as fast as a 1956 Rolleiflex) but the quality is just stunning as it’s sharp and offers a somewhat inexistant chromatic abberation.

Want proof ? These are some of my first roll’s shoots uploaded at full resolution :

07760003

07760006

May 14, 2013
Michal Hrusecky a.k.a. miska (homepage, bugs)
Spring Europen 2013 (May 14, 2013, 17:00 UTC)

Europen talkThis Monday I was the first time guest and speaker at (contrary to it’s name) local Czech conference Europen. It was interesting experience. And I would like to share a bit of what I experienced. What made it different from conferences I usually speak at was the audience. Not many Linux guys and quite some Windows guys. I was told that this conference is for various IT professionals and people from academia interested in Open Source.

I was asked to speak there about something techy, low-levelly, genericy, and not SUSE only stuff. I offered OBS and Studio introduction as these are crown jewels of openSUSE environment, but I was told that they would prefer something more generic and little bit more hardcore. So in the end I decided to speak about packaging as that is something I do that since a long time ago. And to make it nor a workshop nor SUSE specific talk, I put in two more packaging systems that I worked with apart from rpm – Portage (from Gentoo) and BitBake (from Open Embedded).

Whenever I visit open source event in Czech Republic, I always know quite some people there already. I know the most prominent people from Linux magazines, other distributions and some other people who are big open source enthusiasts. On this conference, I knew something like six attendees in total (and all of them were there to give a talk and not sure what to expect from audience). Almost everybody was running MS Windows with few MacOS exceptions. Really quite different world.

As I said, in the end I spoke about why do we do software packages in Linux and how do we do it. I spoke about rpm and spec files, about Portage and BitBake showing how nice it is to have inheritance. And in the end I put in part about how great OBS is anyway.

From the almost a day I was at the conference, most questions and feedback got LibUCW library, but Martin Mareš gave amazing presentation and he had a really interesting topic. LibUCW is cool. If I’ll find a free time, I’ll write something about it separately. Otherwise audience was quite calm and quiet. For my presentation, I got question about cross-compilation of rpms, so in the end after the talk I could recommend OBS once more ;-)

It was definitely interesting experience as these people were mostly out of our usual scope. If you are interested in browsing the slides, you can, sources are on my github, but they contain quite some pages of example recipes that I was commenting on the spot.

I use it since a long time, so since it works pretty good for me, I want to share how to handle the spam for your @gentoo.org address with procmail.

First, you need to say that procmail will filter your email(s):
echo "| /usr/bin/procmail" > /home/${USER}/.forward

Then create a simple /home/${USER}/.procmailrc with this content:
:0:
* ^X-Spam-Status: Yes
/dev/null

:0:
* ^X-Spam-Level: \*\*\*
/dev/null/

:0:
* ! ^List-Id
* ^X-Spam-Level: \*\*
/dev/null/

:0:
* ^Subject:.*viagra*
/dev/null

:0:
* ^Subject:.*cialis*
/dev/null

:0:
* ^Subject:.*money*
/dev/null

:0:
* ^Subject:.*rolex*
/dev/null

:0:
* ^Subject:.*scount*
/dev/null

:0:
* ^Subject:.*Viagra*
/dev/null

:0:
* ^Subject:.*Cialis*
/dev/null

:0:
* ^Subject:.*Marketing*
/dev/null

:0:
* ^Subject:.*marketing*
/dev/null

:0:
* ^Subject:.*Money*
/dev/null

:0:
* ^Subject:.*Rolex*
/dev/null

:0:
* ^Subject:.*Scount*
/dev/null

:0:
* ^Subject:.*glxgug*
/dev/null

:0:
* ^Subject:.*offizielle sieger*
/dev/null

:0:
* ^Subject:.*educational*
/dev/null

:0 B:
* $ content-[^:]+:${WS}*.+(\<)*(file)?name${WS}*=${WS}*\/.+\.(pif|scr|com|cpl|vbs|mim|hqx|bhx|uue|uu|b64)\"?$
/dev/null

:0 B:
* ^Content-Type: .*;$[ ]*(file)?name=\"?.*\.(pif|scr|com|cpl|vbs)\"?$
/dev/null

:0 B:
* ^Content-Type: .*; [ ]*(file)?name=\"?.*\.(pif|scr|com|cpl|vbs)\"?$
/dev/null

With the filter for X-Spam-Status and X-Spam-Level you will avoid the majority of the incoming spam.
Some mails that does not have any Spam flag, contains subject like viagra, cialis ( which I absolutely don’t need :D ), rolex and scount.
Yes, I could you the (c|C)ase syntax, but I had problems, so I prefer to write twice the rules instead of have any sort of troubles.
Note: with this email address I’m not subscribed to any newsletter or any sort of offers/catalogs so I filtered scount, markerting, money.

Sometimes I receive mails from people that are not spammer, with the X-Spam-Level flag with one star, so I decided to move these email into a folder, they will be double-checked with naked eye:

:0:
* ^X-Spam-Level: \*
/home/ago/.maildir/.INBOX.pspam/

To avoid confusion I always prefer to use a complete path here.

After a stabilization you will always see the annoying mail from the bugzilla which contains ${arch} stable, so if you want to drop them:

:0 B
* ^*(alpha|amd64|arm|hppa|ia64|m68k|ppc|ppc64|s390|sh|sparc|x86) stable*
/dev/null

Now, if you are using more email clients, on more computers, you may need to set the filters here instead of on all clients you are using, so for example:

:0
* ^From.*bugzilla-daemon@gentoo.org
* ^TO.*amd64@gentoo.org
/home/ago/.maildir/.INBOX.amd64/

And so on….
These, hints obviously are valid on all postfix-based mailserver; if you are using e.g. qmail, you need to move the .procmailrc, but this is still valid.
I hope this will help :)

EDIT:
If you need a particular set of rules, you can write it if you take a look at the source/header of the message, so If for example I don’t like to see the mails from bugzilla of the bugs that I reported:

the header says: X-Bugzilla-Reporter: ago@gentoo.org
so:

:0
* ^From.*bugzilla-daemon@gentoo.org
* ^X-Bugzilla-Reporter.*ago@gentoo.org
/dev/null

May 12, 2013
Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)
Lab::Measurement 3.11 released (May 12, 2013, 10:34 UTC)

Lab::Measurement 3.11 has been uploaded to CPAN. This is a minor maintenance release, with small bug fixes in the voltage source handling (gate protect and sweep functionality) and the Yokogawa drivers (output voltage range settings).

May 11, 2013
Sebastian Pipping a.k.a. sping (homepage, bugs)

When working on (the still on-going) migration of the Gentoo java project repositories from SVN to Git I ran into bugs with svn2git 1.0.8 and my own svneverever 1.2.1.

The bug with svn2git 1.0.8 was a regression that broke support for (non-ASCII) UTF-8 author names in identity maps. That’s fixed in dev-vcs/svn2git-1.0.8-r1 in Gentoo. I sent the patch upstream and to the Debian package maintainer, too.

For svneverever, a directory that re-appeared after deletion was reported to only live once, e.g. the output was

(2488; 9253)  /projects
(2490; 9253)      /java-config-2
(2490; 2586)          /trunk

if directory /projects/java-config-2/trunk/ got deleted at revision 2586, no matter if was re-created later. With 9253 revisions in total, the correct output (with svneverever 1.2.2) is:

(2488; 9253)  /projects
(2490; 9253)      /java-config-2
(2490; 9253)          /trunk

That’s fixed in svneverever 1.2.2.

If svneverever is of help to you, please support me on Flattr. Thanks!

May 10, 2013
Gentoo at LinuxTag 2013 in Berlin (May 10, 2013, 01:03 UTC)

LinuxTag 2013 runs from May 22nd to May 25th in Berlin, Germany. With more than 10,000 visitors last year, it is one of the biggest Linux and open source events in Europe.

You will find the Gentoo booth at Hall 7.1c, Booth 179. Come and visit us! You will meet many of our developers and users, talk with us, plus get some of the Gentoo merchandise you have always wanted.

May 07, 2013
Jan Kundrát a.k.a. jkt (homepage, bugs)
On Innovation, NIH, Trojita and KDE PIM (May 07, 2013, 08:03 UTC)

Jos wrote a blog post yesterday commenting on the complexity of the PIM problem. He raises an interesting concern about whether we would be all better if there was no Trojitá and I just improved KMail instead. As usual, the matter is more complicated than it might seem on a first sight.

Executive Summary: I tried working with KDEPIM. The KDEPIM IMAP stack required a total rewrite in order to be useful. At the time I started, Akonadi did not exist. The rewrite has been done, and Trojitá is the result. It is up to the Akonadi developers to use Trojitá's IMAP implementation if they are interested; it is modular enough.

People might wonder why Trojitá exists at all. I started working on it because I wasn't happy with how the mail clients performed back in 2006. The supported features were severely limited, the speed was horrible. After studying the IMAP protocol, it became obvious that the reason for this slowness is the rather stupid way in which the contemporary clients treated the remote mail store. Yes, it's really a very dumb idea to load tens of thousands of messages when opening a mailbox for the first time. Nope, it does not make sense to block the GUI until you fetch that 15MB mail over a slow and capped cell phone connection. Yes, you can do better with IMAP, and the possibility has been there for years. The problem is that the clients were not using the IMAP protocol in an efficient manner.

It is not easy to retrofit a decent IMAP support into an existing client. There could be numerous code paths which just assume that everything happens synchronously and block the GUI when the data are stuck on the wire for some reason. Doing this properly, fetching just the required data and doing all that in an asynchronous manner is not easy -- but it's doable nonetheless. It requires huge changes to the overall architecture of the legacy applications, however.

Give Trojitá a try now and see how fast it is. I'm serious here -- Trojitá opens a mailbox with tens of thousands of messages in a fraction of second. Try to open a big e-mail with vacation pictures from your relatives over a slow link -- you will see the important textual part pop up immediately with the images being loaded in the background, not disturbing your work. Now try to do the same in your favorite e-mail client -- if it's as fast as Trojitá, congratulations. If not, perhaps you should switch.

Right now, the IMAP support in Trojitá is way more advanced than what is shipped in Geary or KDE PIM -- and it is this solid foundation which leads to Trojitá's performance. What needs work now is polishing the GUI and making it play well with the rest of a users' system. I don't care whether this polishing means improving Trojitá's GUI iteratively or whether its IMAP support gets used as a library in, say, KMail -- both would be very succesfull outcomes. It would be terrific to somehow combine the nice, polished UI of the more established e-mail clients with the IMAP engine from Trojitá. There is a GSoC proposal for integrating Trojitá into KDE's Kontact -- but for it to succeed, people from other projects must get involved as well. I have put seven years of my time into making the IMAP support rock; I would not be able to achieve the same if I was improving KMail instead. I don't need a fast KMail, I need a great e-mail client. Trojitá works well enough for me.

Oh, and there's also a currently running fundraiser for better address book integration in Trojitá. We are not asking for $ 100k, we are asking for $ 199. Let's see how many people are willing to put the money where their mouth is and actually do something to help the PIM on a free desktop. Patches and donations are both equally welcome. Actually, not really -- great patches are much more appreciated. Because Jos is right -- it takes a lot of work to produce great software, and things get better when there are more poeple working towards their common goal together.

Update: it looks like my choice of kickstarter platform was rather poor, catincan apparently doesn't accept PayPal :(. There's the possiblity of direct donations over SourceForge/PayPal -- please keep in mind that these will be charged even if less donors pledge to the idea.

May 05, 2013

Since a long time I realized that is a pita every time that I keyword, receive a repoman failure for dependency.bad(mostly) that does not regard the arch that I’m changing.
So, checking in the repoman manual, I realized that –ignore-arches looks bad for my case and I decided to request a new feature: –include-arches.
This feature, as explained in the bug, checks only for the arches that you write as argument and should be used only when you are keywording/stabilizing.

Some examples/usage:

First, it saves time, the following example will try to run repoman full in the kdelibs directory:
$ time repoman full > /dev/null 2>&1
real 0m12.434s

$ time repoman full --include-arches "amd64" > /dev/null 2>&1
real 0m3.880s

Second, kdelibs suffers for a dependency.bad on amd64-fbsd, so:
$ repoman full
RepoMan scours the neighborhood...
>>> Creating Manifest for /home/ago/gentoo-x86/kde-base/kdelibs
dependency.bad 2
kde-base/kdelibs/kdelibs-4.10.2.ebuild: PDEPEND: ~amd64-fbsd(default/bsd/fbsd/amd64/9.0) ['>=kde-base/nepomuk-widgets-4.10.2:4[aqua=]']

$ repoman full --include-arches "amd64"
RepoMan scours the neighborhood...
>>> Creating Manifest for /home/ago/gentoo-x86/kde-base/kdelibs

Now when I will keyword the packages I can check for specific arches and skip the unuseful checks since they causes, in this case, only a waste of time.
Thanks to Zac for the work on it.

May 03, 2013
Sebastian Pipping a.k.a. sping (homepage, bugs)
May 3rd = Day Against DRM (May 03, 2013, 14:23 UTC)

Learn more at dayagainstdrm.org (and drm.info).

Nirbheek Chauhan a.k.a. nirbheek (homepage, bugs)

Almost a year ago, I worked with Pooja on transliterating a Hindi poem to Bharati Braille for a Type installation at Amar Jyoti School; an institute for the visually-impaired in Delhi. You can read more about that on her blog post about it. While working on that, we were surprised to discover that there were no free (or open source) tools to do the conversion! All we could find were expensive proprietary software, or horribly wrong websites. We had to sit down and manually transliterate each character while keeping in mind the idiosyncrasies of the conversion.

Now, like all programmers who love what they do, I have an urge to reduce the amount of drudgery and repetitive work in my life with automation ;). In addition, we both felt that a free tool to do such a transliteration would be useful for those who work in this field. And so, we decided to work on a website to convert from Devanagari (Hindi & Marathi) to Bharati Braille.

Now, after tons of research and design/coding work, we are proud to announce the first release of our Devanagari to Bharati Braille converter! You can read more about the converter here, and download the source code on Github.

If you know anyone who might find this useful, please tell them about it!

May 01, 2013
Donnie Berkholz a.k.a. dberkholz (homepage, bugs)

If you’re a university student, time is running out! You could get paid to hack on Gentoo or other open-source software this summer, but you’ve gotta act now. The deadline to apply for the Google Summer of Code is this Friday.

If this sounds like your dream come true, you can find some Gentoo project ideas here and Gentoo’s GSoC homepage here. For non-Gentoo projects, you can scan through the GSoC website to find the details.


Tagged: gentoo, gsoc

April 28, 2013
Raúl Porcel a.k.a. armin76 (homepage, bugs)
The new BeagleBone Black and Gentoo (April 28, 2013, 18:02 UTC)

Hi all, long time no see.

Some weeks ago I got an early version of the BeagleBone Black from the people at Beagleboard.org to create the documentation I always create with every device I get.

Like always i’d like to announce the guide for installing Gentoo in the BeagleBone Black. Have a look at: http://dev.gentoo.org/~armin76/arm/beagleboneblack/install.xml . Feel free to send any corrections my way.

This board is a new version of the original BeagleBone, known in the community as BeagleBone white, for which I wrote a post for it: http://armin762.wordpress.com/2012/01/01/beaglebone-and-gentoo/

This new version differs in some aspects with the previous version:

  • Cheaper: 45$ vs 89$ of the BeagleBone white
  • 512MB DDR3L RAM vs 256MB DDR2 RAM of the BeagleBone white
  • 1GHz of processor speed vs 720MHz of the BeagleBone white, both when using an external PSU for power

Also it has more features which the old BeagleBone didn’t had

  • miniHDMI output
  • 2GB eMMC

However the new version has missing:

  • Serial port and JTAG through the miniUSB interface

The reason for missing this feature is cost cutting measures, as can be read in the Reference manual.

The full specs of the BeagleBone Black are:
# ARMv7-A 1GHz TI AM3358/9 ARM Cortex-A8 processor
# 512MB DDR3L RAM
# SMSC LAN8710 Ethernet card
#
# 1x microSDHC slot
# 1x USB 2.0 Type-A port
# 1x mini-USB 2.0 OTG port
# 1x RJ45
# 1x 6 pin 3.3V TTL Header for serial
#
# Reset, power and user-defined button

More info about the specs in BeagleBone Black’s webpage.

For those curious as me, here’s the bootlog and the cpuinfo.

I’ve found two issues while working on it:

  1. The USB port doesn’t have a working hotplug detection. That means that if you plug an USB device in the USB port, it will be only detected once, if you remove the USB device, the USB port will stop working. I’ve been told that they are working on it. I haven’t been able to find a workaround for it.
  2. The BeagleBone Black doesn’t detect an microSD card when plugged in when its been booted from the eMMC. If you want to use a microSD card for additional storage, it must be inserted before it boots.

I’d like to thank the people at Beagleboard.org for providing me a Beaglebone Black to document this.

Have fun!


April 26, 2013
Alexys Jacob a.k.a. ultrabug (homepage, bugs)
mongoDB and Pacemaker recent bumps (April 26, 2013, 14:23 UTC)

mongoDB 2.4.3

Yet another bugfix release, this new stable branch is surely one of the most quickly iterated I’ve ever seen. I guess we’ll wait a bit longer at work before migrating to 2.4.x.

pacemaker 1.1.10_rc1

This is the release of pacemaker we’ve been waiting for, fixing among other things, the ACL problem which was introduced in 1.1.9. Andrew and others are working hard to get a proper 1.1.10 out soon, thanks guys.

Meanwhile, we (gentoo cluster herd) have been contacted by @Psi-Jack who has offered his help to follow and keep some of our precious clustering packages up to date, I wish our work together will benefit everyone !

All of this is live on portage, enjoy.

Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
My time abroad: loyalty cards (April 26, 2013, 12:56 UTC)

Compared to most people around me now, and probably most of the people who read my blog, my life is not that extraordinary, in the terms of travel and moving around. I’ve been, after all, scared of planes for years, and it wasn’t until last year that I got out of the continent — in an year, though, I more than doubled the number of flights I’ve been on, with 18 last year, and more than doubled the number of countries I’ve been to, counting Luxembourg even though I only landed there and got on a bus to get back to Brussels after Alitalia screwed up.

On the other hand, compared to most of the people I know in Italy, I’ve been going around quite a bit, as I spent a considerable amount of time last year in Los Angeles, and I’ve now moved to Dublin, Ireland. And there are quite a few differences between these places and Italy. I’ve already written a bit about the differences I found during my time in the USA but this time I want to focus on something which is quite a triviality, but still is a remarkable difference between the three countries I got to know up to now. As the title suggest I’m referring to stores’ loyalty cards.

Interestingly enough, there was just this week an article on the Irish Times about the “privacy invasion” of loyalty cards.. I honestly don’t see it as big a deal as many others. Yes, they do profile your shopping habits. Yes, if you do not keep private the kind of offers they sent you, they might tell others something about you as well — the newspaper actually brought up the example of a father who discovered the pregnancy of the daughter because of the kind of coupons the supermarket was sending, based on her change of spending habits; I’m sorry but I cannot really feel bad about it. After all, absolute privacy and relevant offers are kinda at the opposite sides of a range.. and I’m usually happy enough when companies are relevant to me.

So of course stores want to know the habits of a single person, or of a single household, and for that they give you loyalty cards… but for you to use them, they have to give you something in return, don’t they? This is where the big difference on this topic appears clearly, if you look at the three countries:

  • in both Italy and Ireland, you get “points” with your shopping; in the USA, instead, the card gives you immediate discounts; I’m pretty sure that this gives not-really-regular-shoppers a good reason to get the card as well: you can easily save a few dollars on a single grocery run by getting the loyalty card at the till;
  • in Italy you redeem the points to get prizes – this works not so differently than with airlines after all – sometimes by adding a contribution, sometimes for free; in my experience the contribution is never worth it, so either you get something for free or just forget about it;
  • in Ireland I still haven’t seen a single prize system; instead they work with coupons: you get a certain amount of points each euro you spend (usually, one point per euro), and then when you get to a certain amount of points, they get a value (usually, one cent per point), and a coupon redeemable for the value is sent you.

Of course, the “European” method (only by contrast with American, since I don’t know what other countries do), is a real loyalty scheme: you need a critical mass of points for them to be useful, which means that you’ll try to get on the same store as much as you can. This is true for airlines as well, after all. On the other hand, people who shop occasionally are less likely to request the card at all, so even if there is some kind of data to be found in their shopping trends, they will be completely ignored by this kind of scheme.

I’m honestly not sure which method I prefer, at this point I still have one or two loyalty cards from my time in Los Angeles, and I’m now collecting a number of loyalty cards here in Dublin. Some are definitely a good choice for me, like the Insomnia card (I love getting coffee at a decent place where I can spend time to read, in the weekends), others, like Dunnes, make me wonder.. the distance from the supermarket to where I’m going to live is most likely offsetting the usefulness of their coupons compared to the (otherwise quite more expensive) Spar at the corner.

At any rate, I just want to write my take on the topic, which is definitely not of interest to most of you…

April 25, 2013
Bernard Cafarelli a.k.a. voyageur (homepage, bugs)

Recently, I have been toying around with GateOne, a web-based SSH client/terminal emulator. However, installing it on my server proved to be a bit challenging: it requires tornado as a webserver, and uses websockets, while I have an Apache 2.2 instance already running with a few sites on it (and my authentication system configured for my tastes)

So, I looked how to configure a reverse proxy for GateOne, but websockets were not officially supported by Apache... until recently! Jim Jagielski added the proxy_wstunnel module in trunk a few weeks ago. From what I have seen on the mailing list, backporting to 2.4 is easy to do (and was suggested as an official backport), but 2.2 required a few additional changes to the original patch (and current upstream trunk).

A few fixes later, I got a working patch (based on Apache 2.2.24), available here: http://cafarelli.fr/gentoo/apache-2...

Recompile with this patch, and you will get a nice and shiny mod_proxy_wstunnel.so module file!

Now just load it (in /etc/apache2/httpd.conf in Gentoo):
<IfDefine PROXY>
LoadModule proxy_wstunnel_module modules/mod_proxy_wstunnel.so
</IfDefine>

and add a location pointing to your GateOne installation:

<Location /gateone/ws>
    ProxyPass wss://127.0.0.1:1234/gateone/ws
    ProxyPassReverse wss://127.0.0.1:1234/gateone/ws
</Location>

<Location /gateone>
    Order deny,allow
    Deny from all
    Allow from #your favorite rule

    ProxyPass http://127.0.0.1:1234/gateone
    ProxyPassReverse http://127.0.0.1:1234/gateone
</Location>


Reload Apache, and you now have Gateone running behind your Apache server :) If it does not work, first check GateOne log and configuration, especially the "origins" variable

For other websocket applications, Jim Jagielski comments here :

ProxyPass /whatever ws://websocket-srvr.example/com/

Basically, the new submodule adds the 'ws' and 'wss' scheme to the allowed protocols between the client and the backend, so you tell Apache that you'll be talking 'ws' with the backend (same as ajp://whatever sez that httpd will be talking ajp to the backend).

Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
Tarsnap and backup strategies (April 25, 2013, 13:52 UTC)

After having had a quite traumatic experience with a customer’s service running on one of the virtual servers I run last November, I made sure to have a very thorough backup for all my systems. Unfortunately, it turns out to be a bit too thorough, so let me explore with you what was going on.

First of all, the software I use to run the backup is tarsnap — you might have heard of it or not, but it’s basically a very smart service, that uses an open-source client, based upon libarchive, and then a server system that stores content (de-duplicated, compressed and encrypted with a very flexible key system). The author is a FreeBSD developer, and he’s charging an insanely small amount of money.

But the most important part to know when you use tarsnap is that you just always create a new archive: it doesn’t really matter what you changed, just get everything together, and it will automatically de-duplicate the content that didn’t change, so why bother? My first dumb method of backups, which is still running as of this time, is to simply, every two hours, dump a copy of the databases (one server runs PostgreSQL, the other MySQL — I no longer run MongoDB but I start to wonder about it, honestly), and then use tarsnap to generate an archive of the whole /etc, /var and a few more places where important stuff is. The archive is named after date and time of the snapshot. And I haven’t deleted any snapshot since I started, for most servers.

It was a mistake.

The moment when I went to recover the data out of earhart (the host that still hosts this blog, a customer’s app, and a couple more sites, like the assets for the blog and even Autotools Mythbuster — but all the static content, as it’s managed by git, is now also mirrored and served active-active from another server called pasteur), the time it took to extract the backup was unsustainable. The reason was obvious when I thought about it: since it has been de-duplicating for almost an year, it would have to scan hundreds if not thousands of archives to get all the small bits and pieces.

I still haven’t replaced this backup system, which is very bad for me, especially since it takes a long time to delete the older archives even after extracting them. On the other hand it’s probably a lot of a matter of tradeoff in the expenses as well, as going through all the older archives to remove the old crap drained my credits with tarsnap quickly. Since the data is de-duplicated and encrypted, the archives’ data needs to be downloaded to be decrypted, before it can be deleted.

My next preference is going to be to set it up so that the script is executed in different modes: 24 times in 48 hours (every two hours), 14 times in 14 days (daily), and 8 times in two months (weekly). The problem is actually doing the rotation properly with a script, but I’ll probably publish a Puppet module to take care of that, since it’s the easiest thing for me to do, to make sure it executes as intended.

The essence of this post is basically to warn you all that, no matter whether it’s cheap to keep around the whole set of backups since the start of time, it’s still a good idea to just rotate them.. especially for content that does not change that often! Think about it even when you set up any kind of backup strategy…

April 24, 2013
Alexys Jacob a.k.a. ultrabug (homepage, bugs)
Hello Gentoo Planet (April 24, 2013, 08:51 UTC)

Hey Gentoo folks !

I finally followed a friend’s advice and stepped into the Gentoo Planet and Universe feeds. I hope my modest contributions will help and be of interest to some of you readers.

As you’ll see, I don’t talk only about Gentoo but also about photography and technology more generally. I also often post about the packages I maintain or I have an interest in to highlight their key features or bug fixes.

April 21, 2013

Those of you who don't live under a rock will have learned by now that AMD has published VDPAU code to use the Radeon UVD engine for accelerated video decode with the free/open source drivers.

In case you want to give it a try, mesa-9.2_pre20130404 has been added (under package.mask) to the portage tree for your convenience. Additionally you will need a patched kernel and new firmware.

Kernel

For kernel 3.9, grab the 10 patches from the dri-devel mailing list thread (recommended) [UPDATE]I put the patches into a tarball and attached to Gentoo bug 466042[/UPDATE]. For kernel 3.8 I have collected the necessary patches here, but be warned that kernel 3.8 is not officially supported. It works on my Radeon 6870, YMMV.

Firmware

The firmware is part of radeon-ucode-20130402, but has not yet reached the linux-firmware tree. If you require other firmware from the linux-firmware package, remove the radeon files from the savedconfig file and build the package with USE="savedconfig" to allow installation together with radeon-ucode. [UPDATE]linux-firmware-20130421 now contains the UVD firmware, too.[/UPDATE]

The new firmware files are
radeon/RV710_uvd.bin: Radeon 4350-4670, 4770.
radeon/RV770_uvd.bin: Not useful at this time. Maybe later for 4200, 4730, 4830-4890.
radeon/CYPRESS_uvd.bin: Evergreen cards.
radeon/SUMO_uvd.bin: Northern Islands cards and Zacate/Llano APUs.
radeon/TAHITI_uvd.bin: Southern Islands cards and Trinity APUs.

Testing it

If your kernel is properly patched and finds the correct firmware, you will see this message at boot:
[drm] UVD initialized successfully.
If mesa was correctly built with VDPAU support, vdpauinfo will list the following codecs:
Decoder capabilities:

name level macbs width height
-------------------------------------------
MPEG1 16 1048576 16384 16384
MPEG2_SIMPLE 16 1048576 16384 16384
MPEG2_MAIN 16 1048576 16384 16384
H264_BASELINE 16 9216 2048 1152
H264_MAIN 16 9216 2048 1152
H264_HIGH 16 9216 2048 1152
VC1_SIMPLE 16 9216 2048 1152
VC1_MAIN 16 9216 2048 1152
VC1_ADVANCED 16 9216 2048 1152
MPEG4_PART2_SP 16 9216 2048 1152
MPEG4_PART2_ASP 16 9216 2048 1152
If mplayer and its dependencies were correctly built with VDPAU support, running it with "-vc ffh264vdpau," parameter will output something like the following when playing back a H.264 file:
VO: [vdpau] 1280x720 => 1280x720 H.264 VDPAU acceleration
To make mplayer use acceleration by default, uncomment the [vo.vdpau] section in /etc/mplayer/mplayer.conf

Gallium3D Head-up display

Another cool new feature is the Gallium3D HUD (link via Phoronix), which can be enabled with the GALLIUM_HUD environment variable. This supposedly works with all the Gallium drivers (i915g, radeon, nouveau, llvmpipe).

An example screenshot of Supertuxkart using GALLIUM_HUD="cpu0+cpu1+cpu2:100,cpu:100,fps;draw-calls,requested-VRAM+requested-GTT,pixels-rendered"

If you have any questions or problems setting up UVD on Gentoo, stop by #gentoo-desktop on freenode IRC.

Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)

This is a follow-up on my last post for autotools introduction. I’m trying to keep these posts bite sized both because it seems to work nicely, and because this way I can avoid leaving the posts rotting in the drafts set.

So after creating a simple autotools build system in the previous now you might want to know how to build a library — this is where the first part of complexity kicks in. The complexity is not, though, into using libtool, but into making a proper library. So the question is “do you really want to use libtool?”

Let’s start from a fundamental rule: if you’re not going to install a library, you don’t want to use libtool. Some projects that only ever deal with programs still use libtool because that way they can rely on .la files for static linking. My suggestion is (very simply) not to rely on them as much as you can. Doing it this way means that you no longer have to care about using libtool for non-library-providing projects.

But in the case you are building said library, using libtool is important. Even if the library is internal only, trying to build it without libtool is just going to be a big headache for the packager that looks into your project (trust me I’ve seen said projects). Before entering the details on how you use libtool, though, let’s look into something else: what you need to make sure you think about, in your library.

First of all, make sure to have an unique prefix to your public symbols, be them constants, variables or functions. You might also want to have one for symbols that you use within your library on different translation units — my suggestion in this example is going to be that symbols starting with foo_ are public, while symbols starting with foo__ are private to the library. You’ll soon see why this is important.

Reducing the amount of symbols that you expose is not only a good performance consideration, but it also means that you avoid the off-chance to have symbol collisions which is a big problem to debug. So do pay attention.

There is another thing that you should consider when building a shared library and that’s the way the library’s ABI is versioned but it’s a topic that, in and by itself, takes more time to discuss than I want to spend in this post. I’ll leave that up to my full guide.

Once you got these details sorted out, you should start by slightly change the configure.ac file from the previous post so that it initializes libtool as well:

AC_INIT([myproject], [123], [flameeyes@flameeyes.eu], [http://blog.flameeyes.eu/tag/autotoolsmythbuster])
AM_INIT_AUTOMAKE([foreign no-dist-gz dist-xz])
LT_INIT

AC_PROG_CC

AC_OUTPUT([Makefile])

Now it is possible to provide a few options to LT_INIT for instance to disable by default the generation of static archives. My personal recommendation is not to touch those options in most cases. Packagers will disable static linking when it makes sense, and if the user does not know much about static and dynamic linking, they are better off getting everything by default on a manual install.

On the Makefile.am side, the changes are very simple. Libraries built with libtool have a different class than programs and static archives, so you declare them as lib_LTLIBRARIES with a .la extension (at build time this is unavoidable). The only real difference between _LTLIBRARIES and _PROGRAMS is that the former gets its additional links from _LIBADD rather than _LDADD like the latter.

bin_PROGRAMS = fooutil1 fooutil2 fooutil3
lib_LTLIBRARIES = libfoo.la

libfoo_la_SOURCES = lib/foo1.c lib/foo2.c lib/foo3.c
libfoo_la_LIBADD = -lz
libfoo_la_LDFLAGS = -export-symbols-regex &apos^foo_[^_]&apos

fooutil1_LDADD = libfoo.la
fooutil2_LDADD = libfoo.la
fooutil3_LDADD = libfoo.la -ldl

pkginclude_HEADERS = lib/foo1.h lib/foo2.h lib/foo3.h

The _HEADERS variable is used to define which header files to install and where. In this case, it goes into ${prefix}/include/${PACKAGE}, as I declared it a pkginclude install.

The use of -export-symbols-regex ­– further documented in the guide – ensures that only the symbols that we want to have publicly available are exported and does so in an easy way.

This is about it for now — one thing that I haven’t added in the previous post, but which I’ll expand in the next iteration or the one after, is that the only command you need to regenerate autotools is autoreconf -fis and that still applies after introducing libtool support.

April 18, 2013
Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)

Bitrot is accumulating, and while we've tried to keep kdpim-4.4 running in Gentoo as long as possible, the time is slowly coming to say goodbye. In effect this is triggered by annoying problems like these:

There are probably many more such bugs around, where incompatibilities between kdepim-4.4 and kdepimlibs of more recent releases occur or other software updates have led to problems. Slowly it's getting painful, and definitely more painful than running a recent kdepim-4.10 (which has in my opinion improved quite a lot over the last major releases).
Please be prepared for the following steps:
  • end of april 2013, all kdepim-4.4 packages in the Gentoo portage tree will be package.masked 
  • end of may 2013, all kdepim-4.4 packages in the Gentoo portage tree will be removed
  • afterwards, we will finally be able to simplify the eclasses a lot by removing the special handling
We still have the kdepim-4.7 upgrade guide around, and it also applies to the upgrade from kdepim-4.4 to any later version. Feel free to improve it or suggest improvements.

R.I.P. kmail1.

Alexys Jacob a.k.a. ultrabug (homepage, bugs)
py3status v0.9 (April 18, 2013, 17:41 UTC)

First of all py3status is on pypi ! You can now install it with the simple and usual :

$ pip install py3status

This new version features my first pull request from @Fandekasp who kindly wrote a pomodoro module which helps this technique’s adepts by having a counter on their bar. I also fixed a few glitches on module injection and some documentation.

Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)

I’ve been asked over on Twitter if I had any particular tutorial for an easy one-stop-shop tutorial for Autotools newbies… the answer was no, but I will try to make up for it by writing this post.

First of all, with the name autotools, we include quite a bit of different tools. If you have a very simple program (not hellow-simple, but still simple), you definitely want to use at the very least two: autoconf and automake. While you could use the former without the latter, you really don’t want to. This means that you need two files: configure.ac and Makefile.am.

The first of the two files (configure.ac) is processed to produce a configure script which the user will be executing at build time. It is also the bane of most people because, if you look at one for a complex project, you’ll see lots of content (and logic) and next to no comments on what things do. Lots of it is cargo-culting and I’m afraid I cannot help but just show you a possible basic configure.ac file:

AC_INIT([myproject], [123], [flameeyes@flameeyes.eu], [http://blog.flameeyes.eu/tag/autotoolsmythbuster])
AM_INIT_AUTOMAKE([foreign no-dist-gz dist-xz])

AC_PROG_CC

AC_OUTPUT([Makefile])

Let me explain. The first two lines are used to initialize autoconf and automake respectively. The former is being told the name and version of the project, the place to report bugs, and an URL for the package to use in documentation. The latter is told that we’re not a GNU project (seriously, this is important — you wouldn’t believe how many tarballs I find with 0-sized files just because they are mandatory in the default GNU layout; even though I found at least one crazy package lately that wanted to have a 0-sized NEWS file), and that we want a .tar.xz tarball and not a .tar.gz one (which is the default).

After initializing the tools, you need to, at the very least, ask for a C compiler. You could have asked for a C++ compiler as well, but I’ll leave that as an exercise to the reader. Finally, you got to tell it to output Makefile (it’ll use Makefile.in but we’ll create Makefile.am instead soon).

To build a program, you need then to create a Makefile.am similar to this:

bin_PROGRAMS = hellow

dist_doc_DATA = README

Here we’re telling automake that we have a program called hellow (which sources are by default hellow.c) which has to be installed in the binary directory, and a README file that has to be distributed in the tarball and installed as a documentation piece. Yes this is really enough as a very basic Makefile.am.

If you were to have two programs, hellow and hellou, and a convenience library between the two you could do it this way:

bin_PROGRAMS = hellow hellou

hellow_SOURCES = src/hellow.c
hellow_LDADD = libhello.a

hellou_SOURCES = src/hellou.c
hellow_LDADD = libhello.a

noinst_LIBRARIES = libhello.a
libhello_a_SOURCES = lib/libhello.c lib/libhello.h

dist_doc_DATA = README

But then you’d have to add AC_PROG_RANLIB to the configure.ac calls. My suggestion is that if you want to link things statically and it’s just one or two files, just go for building it twice… it can actually makes it faster to build (one less serialization step) and with the new LTO options it should very well improve the optimization as well.

As you can see, this is really easy when done on the basis… I’ll keep writing a few more posts with easy solutions, and probably next week I’ll integrate all of this in Autotools Mythbuster and update the ebook with an “easy how to” as an appendix.

Alexys Jacob a.k.a. ultrabug (homepage, bugs)
mongoDB v2.4.2 released (April 18, 2013, 10:53 UTC)

After the security issue related bumps of the previous releases which happened last weeks it was about time 10gen released a 2.4.x fixing the following issues:

  • Fix for upgrading sharded clusters
  • TTL assertion on replica set secondaries
  • Several V8 memory leak and performance fixes
  • High volume connection crash

I guess everything listed above would have affected our cluster at work so I’m glad we’ve been patient on following-up this release :) See the changelog for details.

Jeremy Olexa a.k.a. darkside (homepage, bugs)
I’ve been in Australia for two months (April 18, 2013, 08:05 UTC)

Well, the title says it. I’ve now been here for two months. I’m working at Skydive Maitland, which is 40 minutes from the coast and 2+ hours from Sydney. So far, I’ve broke even on my Australian travel/living expenses AND I’m skydiving 3-4 days a week, what could be better? I did 99 jumps in March, normally I do 400 per year. Australia is pretty nice, it is easy to live here and there is plenty to see but it is hard to get places since the country is so big and I need a few days break to go someplace.

How did I end up here? I knew I would goto Australia at some point during my trip since I would be passing by and it is a long way from home. (Sidenote: Of all the travelers at hostels in Europe, about 40-50% that I met were Aussie). In December, I bought my right to work in Australia by getting a working holiday visa. That required $270 and 10 minutes to fill out a form on the internet, overnight I had my approval. So, that was settled, I could now work for 12 months in Australia and show up there within a year. I knew I would be working in Australia because it is a rather expensive country to live/travel in. I thought about picking fruit in an orchard since they always hire backpackers, but skydiving sounded more fun in the end (of course!). So, in January, I emailed a few dropzones stating that I would be in Australia in the near future and looking for work. Crickets… I didn’t hear back from anyone. Fair enough, most businesses will have adequate staffing in the middle of the busy season. But, one place did get back to me some weeks later. Then, it took one Skype convo to come to a friendly agreement and I was looking for flights after. Due to some insane price scheming, there was one flight in two days that was 1/2 price of the others (thank you skyscanner.net). That sealed my decision, and I was off…

Onward looking, full time instructor for March and April then become part time in May and June so I can see more of Australia. I have a few road trips in the works, I just need my own vehicle to make that happen. Working on it. After Australia, I’m probably going to Japan or SE Asia like I planned.

Since my sister already asked, Yes, I do see kangaroos nearly everyday..