Welcome to Gentoo Universe, an aggregation of weblog articles on all topics written by Gentoo developers. For a more refined aggregation of Gentoo-related topics only, you might be interested in Planet Gentoo.

Disclaimer:
Views expressed in the content published here do not necessarily represent the views of Gentoo Linux or the Gentoo Foundation.
   
August 12, 2019
Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)

We are happy to be able to announce that our manuscript "Coulomb Blockade Spectroscopy of a MoS2 Nanotube" has been accepted for publication by pssRRL Rapid Research Letters.

Everybody is talking about novel semiconductor materials, and in particular the transition metal dichalcogenides (TMDCs), "layer materials" similar to graphene. With a chemical composition of TX2, where the transition metal T is, e.g., tungsten W or molybdenum Mo, and the chalcogenide X is, e.g., sulphur S or selenium Se, a wide range of interesting properties is expected.

What's by far not so well known is that many of these materials also form  nanotubes, similar to carbon nanotubes in structure but with distinct properties inherited from the planar system. Here, we present first low temperature transport measurements on a quantum dot in a MoS2 nanotube. The metallic contacts to the nanotube still require a lot of improvements, but the  nanotube between them acts as clean potential well for electrons.

Also, our measurements show possible traces of quantum confined behaviour. This is something that has not been achieved yet in planar, lithographically designed devices - since these have by their very geometric nature larger length scales. It means that via transport spectroscopy we can learn about the material properties and its suitability for quantum electronics devices.

A lot of complex physical phenomena have been predicted for MoS2, including spin filtering and intrinsic, possibly topologic superconductivity - a topic of high interest for the quantum computing community, where larger semiconductor nanowires are used at the moment. So this is the start of an exciting project!

"Coulomb Blockade Spectroscopy of a MoS2 Nanotube"
S. Reinhardt, L. Pirker, C. Bäuml, M. Remskar, and A. K. Hüttel
Physica Status Solidi RRL, doi:10.1002/pssr.201900251 (2019); arXiv:1904.05972 (PDF)

August 11, 2019
AArch64 (arm64) profiles are now stable! (August 11, 2019, 00:00 UTC)

Packet.com logo

The ARM64 project is pleased to announce that all ARM64 profiles are now stable.

While our developers and users have contributed significantly in this accomplishment, we must also thank our Packet sponsor for their contribution. Providing the Gentoo developer community with access to bare metal hardware has accelerated progress in achieving the stabilization of the ARM64 profiles.

About Packet.com

This access has been kindly provided to Gentoo by bare metal cloud Packet via their Works on Arm project. Learn more about their commitment to supporting open source here.

About Gentoo

Gentoo Linux is a free, source-based, rolling release meta distribution that features a high degree of flexibility and high performance. It empowers you to make your computer work for you, and offers a variety of choices at all levels of system configuration.

As a community, Gentoo consists of approximately two hundred developers and over fifty thousand users globally.

July 21, 2019
Thomas Raschbacher a.k.a. lordvan (homepage, bugs)

So I had been wondering why my roundcube had lost all the contacts and settings for my users (I changed DB host after  a server outage - see my older post).

Checked the DB and seen this:

roundcubemail=# select user_id, username, mail_host, created  from users;
 user_id | username | mail_host |            created
---------+----------+-----------+-------------------------------
       3 | User3    | localhost | 2016-04-06 14:32:28.637887+02
       2 | User2    | localhost | 2013-09-26 15:32:08.848301+02
       4 | User2    | 127.0.0.1 | 2019-06-17 14:21:18.059167+02
       5 | User1    | 127.0.0.1 | 2019-07-19 14:26:41.113583+02
       1 | User1    | localhost | 2013-08-29 10:39:47.995082+02
(5 rows)

(I changd the actual usernames to User1,2,3 ..

Turns out whlie I was changing stuff I had changed the config from

$config['default_host'] = 'localhost';

to

$config['default_host'] = '127.0.0.1';

Which resulted in roundcube thinking this is a different host - apparently it doesn't check/resolve hostnames to IP, but instead just assumes each is their own.

The solution was simple: changing it back to 'localhost' from '127.0.0.1' -- hope this maybe helps someone else too. Took me a bit to figure it out.

July 18, 2019
Thomas Raschbacher a.k.a. lordvan (homepage, bugs)
(Suspected) Arson at Kyoto Animation Studio (July 18, 2019, 07:47 UTC)

Latest info 18:03 (JST) 

Already 12 dead at Kyoto Animation Studio .. I hope that this number won't go up anymore ! :

I will probably add more links and information later as I find out.

Article at NHK world

Article at NHK world in japanese - update 2019-07-18 17:19 (JST) -- 13 people confirmed dead :(

Article on bbc.com

Our thoughts and prayers are with the victims and their families & friends!

July 09, 2019
Michał Górny a.k.a. mgorny (homepage, bugs)

Gentoo elections are conducted using a custom software called votify. During the voting period, the developers place their votes in their respective home directories on one of the Gentoo servers. Afterwards, the election officials collect the votes, count them, compare their results and finally announce them.

The simplified description stated above suggests two weak points. Firstly, we rely on honesty of election officials. If they chose to conspire, they could fake the result. Secondly, we rely on honesty of all Infrastructure members, as they could use root access to manipulate the votes (or the collection process).

To protect against possible fraud, we make the elections transparent (but pseudonymous). This means that all votes cast are public, so everyone can count them and verify the result. Furthermore, developers can verify whether their personal vote has been included. Ideally, all developers would do that and therefore confirm that no votes were manipulated.

Currently, we are pretty much implicitly relying on developers doing that, and assuming that no protest implies successful verification. However, this is not really reliable, and given the unfriendly nature of our scripts I have reasons to doubt that the majority of developers actually verify the election results. In this post, I would like to shortly explain how Gentoo elections work, how they could be manipulated and introduce Votrify — a tool to explicitly verify election results.

Gentoo voting process in detail

Once the nomination period is over, an election official sets the voting process up by creating control files for the voting scripts. Those control files include election name, voting period, ballot (containing all vote choices) and list of eligible voters.

There are no explicit events corresponding to the beginning or the end of voting period. The votify script used by developers reads election data on each execution, and uses it to determine whether the voting period is open. During the voting period, it permits the developer to edit the vote, and finally to ‘submit’ it. Both draft and submitted vote are stored as appropriate files in the developer’s home directory, ‘submitted’ votes are not collected automatically. This means that the developer can still manually manipulate the vote once voting period concludes, and before the votes are manually collected.

Votes are collected explicitly by an election official. When run, the countify script collects all vote files from developers’ home directories. An unique ‘confirmation ID’ is generated for each voting developer. All votes along with their confirmation IDs are placed in so-called ‘master ballot’, while mapping from developer names to confirmation IDs is stored separately. The latter is used to send developers their respective confirmation IDs, and can be discarded afterwards.

Each of the election officials uses the master ballot to count the votes. Afterwards, they compare their results and if they match, they announce the election results. The master ballot is attached to the announcement mail, so that everyone can verify the results.

Possible manipulations

The three methods of manipulating the vote that I can think of are:

  1. Announcing fake results. An election result may be presented that does not match the votes cast. This is actively prevented by having multiple election officials, and by making the votes transparent so that everyone can count them.
  2. Manipulating votes cast by developers. The result could be manipulated by modifying the votes cast by individual developers. This is prevented by including pseudonymous vote attribution in the master ballot. Every developer can therefore check whether his/her vote has been reproduced correctly. However, this presumes that the developer is active.
  3. Adding fake votes to the master ballot. The result could be manipulated by adding votes that were not cast by any of the existing developers. This is a major problem, and such manipulation is entirely plausible if the turnout is low enough, and developers who did not vote fail to check whether they have not been added to the casting voter list.

Furthermore, the efficiency of the last method can be improved if the attacker is able to restrict communication between voters and/or reliably deliver different versions of the master ballot to different voters, i.e. convince the voters that their own vote was included correctly while manipulating the remaining votes to achieve the desired result. The former is rather unlikely but the latter is generally feasible.

Finally, the results could be manipulated via manipulating the voting software. This can be counteracted through verifying the implementation against the algorithm specification or, to some degree, via comparing the results a third party tool. Robin H. Johnson and myself were historically working on this (or more specifically, on verifying whether the Gentoo implementation of Schulze method is correct) but neither of us was able to finish the work. If you’re interested in the topic, you can look at my election-compare repository. For the purpose of this post, I’m going to consider this possibility out of scope.

Verifying election results using Votrify

Votrify uses a two-stage verification model. It consists of individual verification which is performed by each voter separately and produces signed confirmations, and community verification that uses the aforementioned files to provide final verified election result.

The individual verification part involves:

  1. Verifying that the developer’s vote has been recorded correctly. This takes part in detecting whether any votes have been manipulated. The positive result of this verification is implied by the fact that a confirmation is produced. Additionally, developers who did not cast a vote also need to produce confirmations, in order to detect any extraneous votes.
  2. Counting the votes and producing the election result. This produces the election results as seen from the developer’s perspective, and therefore prevents manipulation via announcing fake results. Furthermore, comparing the results between different developers helps finding implementation bugs.
  3. Hashing the master ballot. The hash of master ballot file is included, and comparing it between different results confirms that all voters received the same master ballot.

If the verification is positive, a confirmation is produced and signed using developer’s OpenPGP key. I would like to note that no private data is leaked in the process. It does not even indicate whether the dev in question has actually voted — only that he/she participates in the verification process.

Afterwards, confirmations from different voters are collected. They are used to perform community verification which involves:

  1. Verifying the OpenPGP signature. This is necessary to confirm the authenticity of the signed confirmation. The check also involves verifying that the key owner was an eligible voter and that each voter produced only one confirmation. Therefore, it prevents attempts to~fake the verification results.
  2. Comparing the results and master ballot hashes. This confirms that everyone participating received the same master ballot, and produced the same results.

If the verification for all confirmations is positive, the election results are repeated, along with explicit quantification of how trustworthy they are. The number indicates how many confirmations were used, and therefore how many of the votes (or non-votes) in master ballot were confirmed. The difference between the number of eligible voters and the number of confirmations indicates how many votes may have been altered, planted or deleted. Ideally, if all eligible voters produced signed confirmations, the election would be 100% confirmed.

Thomas Raschbacher a.k.a. lordvan (homepage, bugs)
Autoclicker for Linux (July 09, 2019, 14:02 UTC)

So I wanted an autoclicker for linux - for one of my browser based games that require a lot of clicking.

Looked around and tried to find something useful, but all i could find was old pages outdated download links,..

In the end I stumbled upon something simple yet immensely more powerful:xdotool (github) or check out the xdootool website

As an extra bonus it is in the Gentoo repository so a simple

emerge xdotool

Got it installed. it also has minimal dependencies which is nice.

The good part, but also a bit of a downside is that there is no UI (maybe I'll write one when I get a chance .. just as a wrapper).

anyway to do what I wanted was simply this:

xdotool click --repeat 1000 --delay 100 1

Pretty self explainatory, but here's a short explaination anyway:

  • click .. simulate a mouse click
  • --repeat 1000 ... repeat 1000 times
  • --delay 100 ... wait 100ms between clicks
  • 1  .. mouse button 1

The only problem is I need to know how many clicks I need beforehand - which can also be a nice feature of course.

There is one way to stop it if you have the terminal you ran this command from visible (which i always have - and set it to always on top): click with your left mouse button - this stops the click events being registered since it is mouse-down and waits for mouse-up i guess .. but not sure if that is the reason. then move to the terminal and either close it or ctrl+c abort the command -- or just wait for the program to exit after finishing the requested number of clicks. -- On a side note if you don't like that way of stopping it you could always just ctrl+alt+f1 (or whatever terminal you want to use) and log in there and kill the xdotool process (either find thepid and kill it or just killall xdotool - which will of course kill all, but i doubt you'll run more than one at once)

July 05, 2019
Thomas Raschbacher a.k.a. lordvan (homepage, bugs)
openvpn client on raspberry PI (July 05, 2019, 05:56 UTC)

Needing OpenVPN on my raspberry PI caused me to have some .. unexpected issues. But first a very quick run-down on what I did:

apt-get install openvpn

(I did an upgrade and dist-upgrade to buster too since my install was quite old already, but that is a different story).

then create a .conf file in /etc/openvpn:

Here's a simple example that I am using (I am using the "embedded" style config since I don't like to have loads of files in that folder:

# OpenVPN CLient Configuration

client
dev tun

proto udp
remote <SERVER NAME OR IP> <PORT>

resolv-retry infinite
nobind

user nobody
group nogroup

persist-key
persist-tun

# if you want to save your certificates in seperate files then use this:
# ca <path/to/your/ca.crt>
# cert <path/to/your/client.crt>
# key <path/to/your/client.key>

ns-cert-type server
comp-lzo
verb 3

# I like to just embed the keys and certificates in the conf file
# useful also for the android client,..

# paste contents of ca.crt
<ca>
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
</ca>
# paste contents of client.key
<key>
-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----
</key>
# paste contents of client.cert
<cert>
Certificate:
    Data:
...
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
</cert>

Then just testing it by running sudo openvpn /etc/openvpn/client.conf.  If you didn't make any mistakes it will look something like this:

Fri Jul  5 07:21:33 2019 OpenVPN 2.4.7 arm-unknown-linux-gnueabihf [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Feb 20 2019
Fri Jul  5 07:21:33 2019 library versions: OpenSSL 1.1.1c  28 May 2019, LZO 2.10
Fri Jul  5 07:21:33 2019 WARNING: --ns-cert-type is DEPRECATED.  Use --remote-cert-tls instead.
Fri Jul  5 07:21:33 2019 TCP/UDP: Preserving recently used remote address: [AF_INET]SERVER_NAME_IP:SERVER_PORT
Fri Jul  5 07:21:33 2019 Socket Buffers: R=[163840->163840] S=[163840->163840]
Fri Jul  5 07:21:33 2019 UDP link local: (not bound)
Fri Jul  5 07:21:33 2019 UDP link remote: [AF_INET]SERVER_NAME_IP:SERVER_PORT
Fri Jul  5 07:21:33 2019 NOTE: UID/GID downgrade will be delayed because of --client, --pull, or --up-delay
Fri Jul  5 07:21:33 2019 TLS: Initial packet from [AF_INET]SERVER_NAME_IP:SERVER_PORT, sid=60ead8c7 c03a7c1d
Fri Jul  5 07:21:33 2019 VERIFY OK: depth=1, C=AT, ST=LOCATION_COUNTY, L=LOCATION, O=ORGANIZATION, CN=SERVER_NAME_IP, name=SERVER_NAME_IP, emailAddress=webmaster@SERVER_NAME_IP
Fri Jul  5 07:21:33 2019 VERIFY OK: nsCertType=SERVER
Fri Jul  5 07:21:33 2019 VERIFY OK: depth=0, C=AT, ST=LOCATION_COUNTY, L=LOCATION, O=ORGANIZATION, CN=SERVER_NAME_IP, name=SERVER_NAME_IP, emailAddress=webmaster@SERVER_NAME_IP
Fri Jul  5 07:21:34 2019 Control Channel: TLSv1.2, cipher TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384, 1024 bit RSA
Fri Jul  5 07:21:34 2019 [SERVER_NAME_IP] Peer Connection Initiated with [AF_INET]SERVER_NAME_IP:SERVER_PORT
Fri Jul  5 07:21:35 2019 SENT CONTROL [SERVER_NAME_IP]: 'PUSH_REQUEST' (status=1)
Fri Jul  5 07:21:35 2019 PUSH: Received control message: 'PUSH_REPLY,route VPN_SUBNET 255.255.255.0,topology net30,ping 10,ping-restart 120,ifconfig YOUR_VPN_IP YOUR_VPN_ROUTER,peer-id 1,cipher AES-256-GCM'
Fri Jul  5 07:21:35 2019 OPTIONS IMPORT: timers and/or timeouts modified
Fri Jul  5 07:21:35 2019 OPTIONS IMPORT: --ifconfig/up options modified
Fri Jul  5 07:21:35 2019 OPTIONS IMPORT: route options modified
Fri Jul  5 07:21:35 2019 OPTIONS IMPORT: peer-id set
Fri Jul  5 07:21:35 2019 OPTIONS IMPORT: adjusting link_mtu to 1625
Fri Jul  5 07:21:35 2019 OPTIONS IMPORT: data channel crypto options modified
Fri Jul  5 07:21:35 2019 Data Channel: using negotiated cipher 'AES-256-GCM'
Fri Jul  5 07:21:35 2019 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
Fri Jul  5 07:21:35 2019 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
Fri Jul  5 07:21:35 2019 ROUTE_GATEWAY LOCAL_GATEWAY/255.255.255.0 IFACE=eth0 HWADDR=MAC_ADDR
Fri Jul  5 07:21:35 2019 TUN/TAP device tun0 opened
Fri Jul  5 07:21:35 2019 TUN/TAP TX queue length set to 100
Fri Jul  5 07:21:35 2019 /sbin/ip link set dev tun0 up mtu 1500
Fri Jul  5 07:21:35 2019 /sbin/ip addr add dev tun0 local YOUR_VPN_IP peer YOUR_VPN_ROUTER
Fri Jul  5 07:21:35 2019 /sbin/ip route add VPN_SUBNET/24 via YOUR_VPN_ROUTER
Fri Jul  5 07:21:35 2019 GID set to nogroup
Fri Jul  5 07:21:35 2019 UID set to nobody
Fri Jul  5 07:21:35 2019 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
Fri Jul  5 07:21:35 2019 Initialization Sequence Completed

If you run ifconfig it should now include an entry for your new VPN device - similar to this:

tun0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>  mtu 1500
        inet YOUR_VPN_IP netmask 255.255.255.255  destination YOUR_VPN_ROUTER
        inet6 YOUR_VPN_IPV6_IP  prefixlen 64  scopeid 0x20<link>
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 100  (UNSPEC)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 9  bytes 432 (432.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Or if you prefer ip addr show:

5: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 100
    link/none
    inet YOUR_VPN_IP peer YOUR_VPN_ROUTER/32 scope global tun0
       valid_lft forever preferred_lft forever
    inet6 YOUR_VPN_IPV6_IP/64 scope link stable-privacy
       valid_lft forever preferred_lft forever

Then just stop it with <ctrl>+c or otherwise kill the process. The output will be like this:

Fri Jul  5 07:22:15 2019 event_wait : Interrupted system call (code=4)
Fri Jul  5 07:22:15 2019 /sbin/ip route del VPN_SUBNET/24
RTNETLINK answers: Operation not permitted
Fri Jul  5 07:22:15 2019 ERROR: Linux route delete command failed: external program exited with error status: 2
Fri Jul  5 07:22:15 2019 Closing TUN/TAP interface
Fri Jul  5 07:22:15 2019 /sbin/ip addr del dev tun0 local YOUR_VPN_IP peer YOUR_VPN_ROUTER
RTNETLINK answers: Operation not permitted
Fri Jul  5 07:22:15 2019 Linux ip addr del failed: external program exited with error status: 2
Fri Jul  5 07:22:15 2019 SIGTERM[hard,] received, process exiting

So I was having a little .. issue with getting my openvpn client to start (and then to start on boot).

Turns out both were easy to solve issues:

First I had this error:

ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such device (errno=19)

That was just because I forgot to reboot after a dist-upgrade (which included a new kernel). So Reboot and done :)

The second issue is because I am using Gentoo usually - and without systemd that is .. so I was like how the hell do I get it to start my vpn. Just running /etc/init.d/openvpn start did not start it for one. Changing the init.d file AUTOSTART="all" to AUTOSTART="client" did not do anything either. After looking it up on google a bit I found what I needed:

systemctl start openvpn@lv_new.service

And then you can check again with ifconfig or ip addr show that you have your device up. So then how to get it to autostart? Turns out that is fairly similar:

systemctl enable openvpn@lv_new.service

Turns out most of it was rather simple and the "biggest" issue for me was lack of systemd knowledge .. maybe I should install Gentoo on it after all ;)

July 04, 2019
Michał Górny a.k.a. mgorny (homepage, bugs)

The recent key poisoning attack on SKS keyservers shook the world of OpenPGP. While this isn’t a new problem, it has not been exploited on this scale before. The attackers have proved how easy it is to poison commonly used keys on the keyservers and effectively render GnuPG unusably slow. A renewed discussion on improving keyservers has started as a result. It also forced Gentoo to employ countermeasures. You can read more on them in the ‘Impact of SKS keyserver poisoning on Gentoo’ news item.

Coincidentally, the attack happened shortly after the launch of keys.openpgp.org, that advertises itself as both poisoning-resistant and GDPR-friendly keyserver. Naturally, many users see it as the ultimate solution to the issues with SKS. I’m afraid I have to disagree — in my opinion, this keyserver does not solve any problems, it merely cripples OpenPGP in order to avoid being affected by them, and harms its security in the process.

In this article, I’d like to shortly explain what the problem is, and which of the different solutions proposed so far to it (e.g. on gnupg-users mailing list) make sense, and which make things even worse. Naturally, I will also cover the new Hagrid keyserver as one of the glorified non-solutions.

The attack — key poisoning

OpenPGP uses a distributed design — once the primary key is created, additional packets can be freely appended to it and recombined on different systems. Those packets include subkeys, user identifiers and signatures. Signatures are used to confirm the authenticity of appended packets. The packets are only meaningful if the client can verify the authenticity of their respective signatures.

The attack is carried through third-party signatures that normally are used by different people to confirm the authenticity of the key — that is, to state that the signer has verified the identity of the key owner. It relies on three distinct properties of OpenPGP:

  1. The key can contain unlimited number of signatures. After all, it is natural that very old keys will have a large number of signatures made by different people on them.
  2. Anyone can append signatures to any OpenPGP key. This is partially keyserver policy, and partially the fact that SKS keyserver nodes are propagating keys one to another.
  3. There is no way to distinguish legitimate signatures from garbage. To put it other way, it is trivial to make garbage signatures look like the real deal.

The attacker abuses those properties by creating a large number of garbage signatures and sending them to keyservers. When users fetch key updates from the keyserver, GnuPG normally appends all those signatures to the local copy. As a result, the key becomes unusually large and causes severe performance issues with GnuPG, preventing its normal usage. The user ends up having to manually remove the key in order to fix the installation.

The obvious non-solutions and potential solutions

Let’s start by analyzing the properties I’ve listed above. After all, removing at least one of the requirements should prevent the attack from being possible. But can we really do that?

Firstly, we could set a hard limit on number of signatures or key size. This should obviously prevent the attacker from breaking user systems via huge keys. However, it will make it entirely possible for the attacker to ‘brick’ the key by appending garbage up to the limit. Then it would no longer be possible to append any valid signatures to the key. Users would suffer less but the key owner will lose the ability to use the key meaningfully. It’s a no-go.

Secondly, we could limit key updates to the owner. However, the keyserver update protocol currently does not provide any standard way of verifying who the uploader is, so it would effectively require incompatible changes at least to the upload protocol. Furthermore, in order to prevent malicious keyservers from propagating fake signatures we’d also need to carry the verification along when propagating key updates. This effectively means an extension of the key format, and it has been proposed e.g. in ‘Abuse-Resistant OpenPGP Keystores’ draft. This is probably a wortwhile option but it will take time before it’s implemented.

Thirdly, we could try to validate signatures. However, any validation can be easily worked around. If we started requiring signing keys to be present on the keyserver, the attackers can simply mass-upload keys used to create garbage signatures. If we went even further and e.g. started requiring verified e-mail addresses for the signing keys, the attackers can simply mass-create e-mail addresses and verify them. It might work as a temporary solution but it will probably cause more harm than good.

There were other non-solutions suggested — most notably, blacklisting poisoned keys. However, this is even worse. It means that every victim of poisoning attack would be excluded from using the keyserver, and in my opinion it will only provoke the attackers to poison even more keys. It may sound like a good interim solution preventing users from being hit but it is rather short-sighted.

keys.openpgp.org / Hagrid — a big non-solution

A common suggestion for OpenPGP users — one that even Gentoo news item mentions for lack of alternative — is to switch to keys.openpgp.org keyserver, or switch keyservers to their Hagrid software. It is not vulnerable to key poisoning attack because it strips away all third-party signatures. However, this and other limitations make it a rather poor replacement, and in my opinion can be harmful to security of OpenPGP.

Firstly, stripping all third-party signatures is not a solution. It simply avoids the problem by killing a very important portion of OpenPGP protocol — the Web of Trust. Without it, the keys obtained from the server can not be authenticated otherwise than by direct interaction between the individuals. For example, Gentoo Authority Keys can’t work there. Most of the time, you won’t be able to tell whether the key on keyserver is legitimate or forged.

The e-mail verification makes it even worse, though not intentionally. While I agree that many users do not understand or use WoT, Hagrid is implicitly going to cause users to start relying on e-mail verification as proof of key authenticity. In other words, people are going to assume that if a key on keys.openpgp.org has verified e-mail address, it has to be legitimate. This makes it trivial for an attacker that manages to gain unauthorized access to the e-mail address or the keyserver to publish a forged key and convince others to use it.

Secondly, Hagrid does not support UID revocations. This is an entirely absurd case where GDPR fear won over security. If your e-mail address becomes compromised, you will not be able to revoke it. Sure, the keyserver admins may eventually stop propagating it along with your key, but all users who fetched the key before will continue seeing it as a valid UID. Of course, if users send encrypted mail the attacker won’t be able to read it. However, the users can be trivially persuaded to switch to a new, forged key.

Thirdly, Hagrid rejects all UIDs except for verified e-mail-based UIDs. This is something we could live with if key owners actively pursue having their identities verified. However, this also means you can’t publish a photo identity or use keybase.io. The ‘explicit consent’ argument used by upstream is rather silly — apparently every UID requires separate consent, while at the same time you can trivially share somebody else’s PII as the real name of a valid e-mail address.

Apparently, upstream is willing to resolve the first two of those issues once satisfactory solutions are established. However, this doesn’t mean that it’s fine to ignore those problems. Until they are resolved, and necessary OpenPGP client updates are sufficiently widely deployed, I don’t believe Hagrid or its instance at keys.openpgp.org are good replacements for SKS and other keyservers.

So what are the solutions?

Sadly, I am not aware of any good global solution at the moment. The best workaround for GnuPG users so far is the new self-sigs-only option that prevents it from importing third-party signatures. Of course, it shares the first limitation of Hagrid keyserver. The future versions of GnuPG will supposedly fallback to this option upon meeting excessively large keys.

For domain-limited use cases such as Gentoo’s, running a local keyserver with restricted upload access is an option. However, it requires users to explicitly specify our keyserver, and effectively end up having to specify multiple different keyservers for each domain. Furthermore, WKD can be used to distribute keys. Sadly, at the moment GnuPG uses it only to locate new keys and does not support refreshing keys via WKD (gemato employs a cheap hack to make it happen). In both cases, the attack is prevented via isolating the infrastructure and preventing public upload access.

The long-term solution probably lies in the ‘First-party-attested Third-party Certifications‘ section of the ‘Abuse-Resistant OpenPGP Keystores’ draft. In this proposal, every third-party signature must be explicitly attested by the key owner. Therefore, only the key owner can append additional signatures to the key, and keyservers can reject any signatures that were not attested. However, this is not currently supported by GnuPG, and once it is, deploying it will most likely take significant time.

July 03, 2019
Marek Szuba a.k.a. marecki (homepage, bugs)
Case label for Pocket Science Lab V5 (July 03, 2019, 17:28 UTC)

tl;dr: Here (PDF, 67 kB) is a case label for Pocket Science Lab version 5 that is compatible with the design for a laser-cut case published by FOSSAsia.


In case you haven’t heard about it, Pocket Science Lab [1] is a really nifty board developed by the FOSSAsia community which combines a multichannel, megahertz-range oscilloscope, a multimeter, a logic probe, several voltage sources and a current source, several wave generators, UART and I2C interfaces… and all of this in the form factor of an Arduino Mega, i.e. only somewhat larger than that of a credit card. Hook it up over USB to a PC or an Android device running the official (free and open source, of course) app and you are all set.

Well, not quite set yet. What you get for your 50-ish EUR is just the board itself. You will quite definitely need a set of probe cables (sadly, I have yet to find even an unofficial adaptor allowing one to equip PSLab with standard industry oscilloscope probes using BNC connectors) but if you expect to lug yours around anywhere you go, you will quite definitely want to invest in a case of some sort. While FOSSAsia does not to my knowledge sell PSLab cases, they provide a design for one [2]. It is meant to be laser-cut but I have successfully managed to 3D-print it as well, and for the more patient among us it shouldn’t be too difficult to hand-cut one with a jigsaw either.

Of course in addition to making sure your Pocket Science Lab is protected against accidental damage it would also be nice to have all the connectors clearly labelled. Documentation bundled with PSLab software does show not a few “how to connect instrument X” diagrams but unfortunately said diagrams picture a version 4 of the board and the current major version, V5, features radically different pinout (compare [3] with [4]/[5] and you will see immediately what I mean), not to mention that having to stare at a screen while wiring your circuit isn’t always optimal. Now, all versions of the board feature a complete set of header labels (along with LEDs showing the device is active) on the front side and at least the more recent ones additionally show more detailed descriptions on the back, clearly suggesting the optimal way to go is to make your case our of transparent material. But what if looking at the provided labels directly is not an option, for instance because you have gone eco-friendly and made your case out of wood? Probably stick a label to the front of the case… which brings us back to the problem of the case label from [5] not being compatible with recent versions of the board.

Which brings me to my take on adapting the design from [5] to match the header layout and labels of PSLab V5.1 as well as the laser-cut case design from [2]. It could probably be more accurate but having tried it out, it is close enough. Bluetooth and ICSP-programmer connectors near the centre of the board are not included because the current case design does not provide access to them and indeed, they haven’t even got headers soldered in. Licence and copyright: same as the original.


[1]
https://pslab.io/

[2]
https://github.com/fossasia/pslab-case

[3]
https://github.com/fossasia/pslab-hardware/raw/master/docs/images/PSLab_v5_top.png

[4]
https://github.com/fossasia/pslab-hardware/raw/master/docs/images/pslab_version_previews/PSLab_v4.png

[5]
https://github.com/fossasia/pslab-hardware/raw/master/docs/images/pslabdesign.png

Impact of SKS keyserver poisoning on Gentoo (July 03, 2019, 00:00 UTC)

The SKS keyserver network has been a victim of certificate poisoning attack lately. The OpenPGP verification used for repository syncing is protected against the attack. However, our users can be affected when using GnuPG directly. In this post, we would like to shortly summarize what the attack is, what we did to protect Gentoo against it and what can you do to protect your system.

The certificate poisoning attack abuses three facts: that OpenPGP keys can contain unlimited number of signatures, that anyone can append signatures to any key and that there is no way to distinguish a legitimate signature from garbage. The attackers are appending a large number of garbage signatures to keys stored on SKS keyservers, causing them to become very large and cause severe performance issues in GnuPG clients that fetch them.

The attackers have poisoned the keys of a few high ranking OpenPGP people on the SKS keyservers, including one Gentoo developer. Furthermore, the current expectation is that the problem won’t be fixed any time soon, so it seems plausible that more keys may be affected in the future. We recommend users not to fetch or refresh keys from SKS keyserver network (this includes aliases such as keys.gnupg.net) for the time being. GnuPG upstream is already working on client-side countermeasures and they can be expected to enter Gentoo as soon as they are released.

The Gentoo key infrastructure has not been affected by the attack. Shortly after it was reported, we have disabled fetching developer key updates from SKS and today we have disabled public key upload access to prevent the keys stored on the server from being poisoned by a malicious third party.

The gemato tool used to verify the Gentoo ebuild repository uses WKD by default. During normal operation it should not be affected by this vulnerability. Gemato has a keyserver fallback that might be vulnerable if WKD fails, however gemato operates in an isolated environment that will prevent a poisoned key from causing permanent damage to your system. In the worst case; Gentoo repository syncs will be slow or hang.

The webrsync and delta-webrsync methods also support gemato, although it is not used by default at the moment. In order to use it, you need to remove PORTAGE_GPG_DIR from /etc/portage/make.conf (if it present) and put the following values into /etc/portage/repos.conf:

[gentoo]
sync-type = webrsync
sync-webrsync-delta = true  # false to use plain webrsync
sync-webrsync-verify-signature = true

Afterwards, calling emerge --sync or emaint sync --repo gentoo will use gemato key management rather than the vulnerable legacy method. The default is going to be changed in a future release of Portage.

When using GnuPG directly, Gentoo developer and service keys can be securely fetched (and refreshed) via:

  1. Web Key Directory, e.g. gpg --locate-key developer@gentoo.org
  2. Gentoo keyserver, e.g. gpg --keyserver hkps://keys.gentoo.org ...
  3. Key bundles, e.g.: active devs, service keys

Please note that the aforementioned services provide only keys specific to Gentoo. Keys belonging to other people will not be found on our keyserver. If you are looking for them, you may try keys.openpgp.org keyserver that is not vulnerable to the attack, at the cost of stripping all signatures and unverified UIDs.

July 02, 2019
Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)

The Nature Index 2019 Annual Tables have been published, and there is a valuable new addition: the tables now include a "normalized ranking", where the quality of a university's research output, and not its quantity counts. If we look at the world-wide natural sciences ranking, University of Regensburg is at spot 44, best of all universities in Germany, and in a similar ranking range as, e.g., University of Oxford, University of Tokyo, or University of California San Francisco! Cheers and congratulations!

July 01, 2019
Luca Barbato a.k.a. lu_zero (homepage, bugs)

I presented cargo-c at the rustlab 2019, here is a longer followup of this.

Mixing Rust and C

One of the best selling point for rust is being highly interoperable with the C-ABI, in addition to safety, speed and its amazing community.

This comes really handy when you have well optimized hand-crafted asm kernels you’d like to use as-they-are:

  • They are small and with a clear interface, usually strict boundaries on what they read/write by their own nature.
  • You’d basically rewrite them as they are using some inline assembly for dubious gains.
  • Both cc-rs and nasm-rs make the process of building and linking relatively painless.

Also, if you plan to integrate in a foreign language project some rust component, it is quite straightforward to link the staticlib produced by cargo in your main project.

If you have a pure-rust crate and you want to export it to the world as if it were a normal C (shared/dynamic) library, it gets quite gory.

Well behaved C-API Library structure

Usually when you want to use a C-library in your own project you should expect it to provide the following:

  • A header file, telling the compiler which symbols it should expect
  • A static library
  • A dynamic library
  • A pkg-config file giving you direction on where to find the header and what you need to pass to the linker to correctly link the library, being it static or dynamic

Header file

In C you usually keep a list of function prototypes and type definitions in a separate file and then embed it in your source file to let the compiler know what to expect.

Since you rely on a quite simple preprocessor to do that you have to be careful about adding guards so the file does not get included more than once and, in order to avoid clashes you install it in a subdirectory of your include dir.

Since the location of the header could be not part of the default search path, you store this information in pkg-config usually.

Static Libraries

Static libraries are quite simple in concept (and execution):

  • they are an archive of object code files.
  • the linker simply reads them as it would read just produced .os and link everything together.

There is a pitfall though:

  • In some platforms even if you want to make a fully static binary you end up dynamically linking some system library for a number of reasons.

    The worst offenders are the pthread libraries and in some cases the compiler builtins (e.g. libgcc_s)

  • The information on what they are is usually not known

rustc comes to the rescue with --print native-static-libs, it isn’t the best example of integration since it’s a string produced on stderr and it behaves as a side-effect of the actual building, but it is still a good step in the right direction.

pkg-config is the de-facto standard way to preserve the information and have the build systems know about it (I guess you are seeing a pattern now).

Dynamic Libraries

A shared or dynamic library is a specially crafted lump of executable code that gets linked to the binary as it is being executed.
The advantages compared to statically linking everything are mainly two:

  • Sparing disk space: since without link-time pruning you end up carrying multiple copies of the same library with every binary using it.
  • Safer and simpler updates: If you need to update say, openssl, you do that once compared to updating the 100+ consumers of it existing in your system.

There is some inherent complexity and constraints in order to get this feature right, the most problematic one is ABI stability:

  • The dynamic linker needs to find the symbols the binary expects and have them with the correct size
  • If you change the in-memory layout of a struct or how the function names are represented you should make so the linker is aware.

Usually that means that depending on your platform you have some versioning information you should provide when you are preparing your library. This can be as simple as telling the compile-time linker to embed the version information (e.g. Mach-O dylib or ELF) in the library or as complex as crafting a version script.

Compared to crafting a staticlib it there are more moving parts and platform-specific knowledge.

Sadly in this case rustc does not provide any help for now: even if the C-ABI is stable and set in stone, the rust mangling strategy is not finalized yet, and it is a large part of being ABI stable, so the work on fully supporting dynamic libraries is yet to be completed.

Dynamic libraries in most platforms have a mean to store which other dynamic libraries they reliy on and which are the paths in which to look for. When the information is incomplete, or you are storing the library in a non-standard path, pkg-config comes to the rescue again, helpfully storing the information for you.

Pkg-config

It is your single point of truth as long your build system supports it and the libraries you want to use craft it properly.
It simplifies a lot your life if you want to keep around multiple versions of a library or you are doing non-system packaging (e.g.: Homebrew or Gentoo Prefix).
Beside the search path, link line and dependency information I mentioned above, it also stores the library version and inter-library compatibility relationships.
If you are publishing a C-library and you aren’t providing a .pc file, please consider doing it.

Producing a C-compatible library out of a crate

I explained what we are expected to produce, now let see what we can do on the rust side:

  • We need to export C-ABI-compatible symbols, that means we have to:
  • Decorate the data types we want to export with #[repr(C)]
  • Decorate the functions with #[no_mangle] and prefix them with export "C"
  • Tell rustc the crate type is both staticlib and cdylib
  • Pass rustc the platform-correct link line so the library produced has the right information inside.
    > NOTE: In some platforms beside the version information also the install path must be encoded in the library.
  • Generate the header file so that the C compiler knows about them.
  • Produce a pkg-config file with the correct information

    NOTE: It requires knowing where the whole lot will be eventually installed.

cargo does not support installing libraries at all (since for now rust dynamic libraries should not be used at all) so we are a bit on our own.

For rav1e I did that the hard way and then I came up an easy way for you to use (and that I used for doing the same again with lewton spending about 1/2 day instead of several ~~weeks~~months).

The hard way

As seen in crav1e, you can explore the history there.

It isn’t the fully hard way since before cargo-c there was already nice tools to avoid some time consuming tasks: cbindgen.
In a terse summary what I had to do was:

  • Come up with an external build system since cargo itself cannot install anything nor have direct knowledge of the install path information. I used Make since it is simple and sufficiently widespread, anything richer would probably get in the way and be more time consuming to set up.
  • Figure out how to extract the information provided in Cargo.toml so I have it at Makefile level. I gave up and duplicated it since parsing toml or json is pointlessly complicated for a prototype.
  • Write down the platform-specific logic on how to build (and install) the libraries. It ended up living in the build.rs and the Makefile. Thanks again to Derek for taking care of the Windows-specific details.
  • Use cbindgen to generate the C header (And in the process smooth some of its rough edges
  • Since we already have a build system add more targets for testing and continuous integration purpose.

If you do not want to use cargo-c I spun away the cdylib-link line logic in a stand alone crate so you can use it in your build.rs.

The easier way

Using a Makefile and a separate crate with a customized build.rs works fine and keeps the developers that care just about writing in rust fully shielded from the gory details and contraptions presented above.

But it comes with some additional churn:

  • Keeping the API in sync
  • Duplicate the release work
  • Have the users confused on where to report the issues or where to find the actual sources. (The users tend to miss the information presented in the obvious places such as the README way too often)

So to try to minimize it I came up with a cargo applet that provides two subcommands:

  • cbuild to build the libraries, the .pc file and header.
  • cinstall to install the whole lot, if already built or to build and then install it.

They are two subcommands since it is quite common to build as user and then install as root. If you are using rustup and root does not have cargo you can get away with using --destdir and then sudo install or craft your local package if your distribution provides a mean to do that.

All I mentioned in the hard way happens under the hood and, beside bugs in the current implementation, you should be completely oblivious of the details.

Using cargo-c

As seen in lewton and rav1e.

  • Create a capi.rs with the C-API you want to expose and use #[cfg(cargo_c)] to hide it when you build a normal rust library.
  • Make sure you have a lib target and if you are using a workspace the first member is the crate you want to export, that means that you might have to add a "." member at the start of the list.
  • Remember to add a cbindgen.toml and fill it with at least the include guard and probably you want to set the language to C (it defaults to C++)
  • Once you are happy with the result update your documentation to tell the user to install cargo-c and do cargo cinstall --prefix=/usr --destdir=/tmp/some-place or something along those lines.

Coming next

cargo-c is a young project and far from being complete even if it is functional for my needs.

Help in improving it is welcome, there are plenty of rough edges and bugs to find and squash.

Thanks

Thanks to est31 and sdroege for the in-depth review in #rust-av and kodabb for the last minute edits.

June 26, 2019
Sergei Trofimovich a.k.a. slyfox (homepage, bugs)
An old linux kernel tty/vt bug (June 26, 2019, 00:00 UTC)

trofi's blog: An old linux kernel tty/vt bug

An old linux kernel tty/vt bug

This post is another one in series of obscure bugs. This time elusive bug manifested on my desktop for years until it was pinned down by luck.

The Bug

Initial bug manifested in a very magical way: I boot up my desktop, start a window manager, use it for a week and then at some point when I press Ctrl-F1 my machine reboots gracefully. System logs say I pressed power button. I did not though :)

That kept happening once in a few months and was very hard to say what changed.

I was not sure how to debug that. My only clue was the following message in boot logs:

Mar 29 19:22:42 sf systemd-logind[413]: Power key pressed
<graceful shutdown goes here>

To workaround the effect I made poweroff a no-op in systemd. I hever use “power” button.

The patch still kept messages popping up in the logs but did not shutdown my machine any more. This allowed me to track frequency of these events without distracting actual work on the machine.

But how one would find out how to track it down to a faulty component? Was it my hardware (keyboard, USB host, etc.) losing mind for a second or some obscure software bug?

I tried to track it down backwards from “Power key pressed” in systemd down to a source that registered generated the event.

Apparently all systemd does is reading /dev/input/event<N> device for power keypress and reacts accordingly. That means kernel itself sends those signals as code=KEY_POWER and code=KEY_POWER2 values of struct input_event. I was not able to trace it down to my keyboard driver at that time.

The clue

A few years passed. I forgot about the local systemd patch.

And one day I got a very scary kernel backtraces when my system booted:

Apr 29 13:12:24 sf kernel: BUG: unable to handle kernel paging request at ffffa39b3b117000
Apr 29 13:12:24 sf kernel: #PF error: [PROT] [WRITE]
Apr 29 13:12:24 sf kernel: PGD 5e4a01067 P4D 5e4a01067 PUD 5e4a06067 PMD 7f7d0f063 PTE 80000007fb117161
Apr 29 13:12:24 sf kernel: Oops: 0003 [#1] PREEMPT SMP
Apr 29 13:12:24 sf kernel: CPU: 7 PID: 423 Comm: loadkeys Tainted: G         C        5.1.0-rc7 #98
Apr 29 13:12:24 sf kernel: Hardware name: Gigabyte Technology Co., Ltd. To be filled by O.E.M./H77M-D3H, BIOS F12 11/14/2013
Apr 29 13:12:24 sf kernel: RIP: 0010:__memmove+0x81/0x1a0
Apr 29 13:12:24 sf kernel: Code: 4c 89 4f 10 4c 89 47 18 48 8d 7f 20 73 d4 48 83 c2 20 e9 a2 00 00 00 66 90 48 89 d1 4c 8b 5c 16 f8 4c 8d 54 17 f8 48 c1 e9 03 <f3> 48 a5 4d 89 1a e9 0c 01 00 00 0f 1f 40 00 48 89 d1 4c $
Apr 29 13:12:24 sf kernel: RSP: 0018:ffffc0c3c0c7fd08 EFLAGS: 00010203
Apr 29 13:12:24 sf kernel: RAX: ffffa39b39c9b08c RBX: 0000000000000019 RCX: 00000b8c90633fcb
Apr 29 13:12:24 sf kernel: RDX: 00005c648461bdcd RSI: ffffa39b3b116ffc RDI: ffffa39b3b116ffc
Apr 29 13:12:24 sf kernel: RBP: ffffa39b3ac04400 R08: ffffa39b3b802f00 R09: 00000000fffff73b
Apr 29 13:12:24 sf kernel: R10: ffffffffbe2b6e51 R11: 00505b1b004d5b1b R12: 0000000000000000
Apr 29 13:12:24 sf kernel: R13: ffffa39b39c9b087 R14: 0000000000000018 R15: ffffa39b39c9b08c
Apr 29 13:12:24 sf kernel: FS:  00007f84c341e580(0000) GS:ffffa39b3f1c0000(0000) knlGS:0000000000000000
Apr 29 13:12:24 sf kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 29 13:12:24 sf kernel: CR2: ffffa39b3b117000 CR3: 00000007e9d42003 CR4: 00000000000606e0
Apr 29 13:12:24 sf kernel: Call Trace:
Apr 29 13:12:24 sf kernel:  vt_do_kdgkb_ioctl+0x352/0x450
Apr 29 13:12:24 sf kernel:  vt_ioctl+0xba3/0x1190
Apr 29 13:12:24 sf kernel:  ? __bpf_prog_run32+0x39/0x60
Apr 29 13:12:24 sf kernel:  ? trace_hardirqs_on+0x31/0xe0
Apr 29 13:12:24 sf kernel:  tty_ioctl+0x23f/0x920
Apr 29 13:12:24 sf kernel:  ? preempt_count_sub+0x98/0xe0
Apr 29 13:12:24 sf kernel:  ? __seccomp_filter+0xc2/0x450
Apr 29 13:12:24 sf kernel:  ? __handle_mm_fault+0x7b0/0x1530
Apr 29 13:12:24 sf kernel:  do_vfs_ioctl+0xa2/0x6a0
Apr 29 13:12:24 sf kernel:  ? syscall_trace_enter+0x126/0x280
Apr 29 13:12:24 sf kernel:  ksys_ioctl+0x3a/0x70
Apr 29 13:12:24 sf kernel:  __x64_sys_ioctl+0x16/0x20
Apr 29 13:12:24 sf kernel:  do_syscall_64+0x54/0xe0
Apr 29 13:12:24 sf kernel:  entry_SYSCALL_64_after_hwframe+0x49/0xbe
Apr 29 13:12:24 sf kernel: RIP: 0033:0x7f84c334a3b7
Apr 29 13:12:24 sf kernel: Code: 00 00 00 75 0c 48 c7 c0 ff ff ff ff 48 83 c4 18 c3 e8 dd d2 01 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d a9 ca 0c 00 f7 d8 64 $
Apr 29 13:12:24 sf kernel: RSP: 002b:00007ffed2cc88f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
Apr 29 13:12:24 sf kernel: RAX: ffffffffffffffda RBX: 0000000000000018 RCX: 00007f84c334a3b7
Apr 29 13:12:24 sf kernel: RDX: 00007ffed2cc8910 RSI: 0000000000004b49 RDI: 0000000000000003
Apr 29 13:12:24 sf kernel: RBP: 00007ffed2cc8911 R08: 00007f84c3417c40 R09: 0000561cb25db4a0
Apr 29 13:12:24 sf kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 0000561cb25d32b0
Apr 29 13:12:24 sf kernel: R13: 00007ffed2cc8910 R14: 0000000000000018 R15: 0000000000000003
Apr 29 13:12:24 sf kernel: Modules linked in: sit tunnel4 ip_tunnel snd_hda_codec_hdmi snd_hda_codec_via snd_hda_codec_generic snd_hda_intel snd_hda_codec r8712u(C) snd_hwdep ath9k_htc snd_hda_core ath9k_common ath9k_h$
Apr 29 13:12:24 sf kernel: CR2: ffffa39b3b117000
Apr 29 13:12:24 sf kernel: ---[ end trace 9c4dbd36dd993d54 ]---
Apr 29 13:12:24 sf kernel: RIP: 0010:__memmove+0x81/0x1a0
Apr 29 13:12:24 sf kernel: Code: 4c 89 4f 10 4c 89 47 18 48 8d 7f 20 73 d4 48 83 c2 20 e9 a2 00 00 00 66 90 48 89 d1 4c 8b 5c 16 f8 4c 8d 54 17 f8 48 c1 e9 03 <f3> 48 a5 4d 89 1a e9 0c 01 00 00 0f 1f 40 00 48 89 d1 4c $
Apr 29 13:12:24 sf kernel: RSP: 0018:ffffc0c3c0c7fd08 EFLAGS: 00010203
Apr 29 13:12:24 sf kernel: RAX: ffffa39b39c9b08c RBX: 0000000000000019 RCX: 00000b8c90633fcb
Apr 29 13:12:24 sf kernel: RDX: 00005c648461bdcd RSI: ffffa39b3b116ffc RDI: ffffa39b3b116ffc
Apr 29 13:12:24 sf kernel: RBP: ffffa39b3ac04400 R08: ffffa39b3b802f00 R09: 00000000fffff73b
Apr 29 13:12:24 sf kernel: R10: ffffffffbe2b6e51 R11: 00505b1b004d5b1b R12: 0000000000000000
Apr 29 13:12:24 sf kernel: R13: ffffa39b39c9b087 R14: 0000000000000018 R15: ffffa39b39c9b08c
Apr 29 13:12:24 sf kernel: FS:  00007f84c341e580(0000) GS:ffffa39b3f1c0000(0000) knlGS:0000000000000000
Apr 29 13:12:24 sf kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 29 13:12:24 sf kernel: CR2: ffffa39b3b117000 CR3: 00000007e9d42003 CR4: 00000000000606e0
Apr 29 13:12:24 sf kernel: BUG: sleeping function called from invalid context at include/linux/percpu-rwsem.h:34
Apr 29 13:12:24 sf kernel: in_atomic(): 0, irqs_disabled(): 1, pid: 423, name: loadkeys
Apr 29 13:12:24 sf kernel: CPU: 7 PID: 423 Comm: loadkeys Tainted: G      D  C        5.1.0-rc7 #98
Apr 29 13:12:24 sf kernel: Hardware name: Gigabyte Technology Co., Ltd. To be filled by O.E.M./H77M-D3H, BIOS F12 11/14/2013
Apr 29 13:12:24 sf kernel: Call Trace:
Apr 29 13:12:24 sf kernel:  dump_stack+0x67/0x90
Apr 29 13:12:24 sf kernel:  ? wake_up_klogd+0x10/0x70
Apr 29 13:12:24 sf kernel:  ___might_sleep.cold.18+0xd4/0xe4
Apr 29 13:12:24 sf kernel:  exit_signals+0x1c/0x200
Apr 29 13:12:24 sf kernel:  do_exit+0xa8/0xbb0
Apr 29 13:12:24 sf kernel:  ? ksys_ioctl+0x3a/0x70
Apr 29 13:12:24 sf kernel:  rewind_stack_do_exit+0x17/0x20

These backtraces did not prevent machine from booting and did not seem to cause any ill immediate effect. But they still looked very scary: something failed to copy data somewhere after all, that meant certain corruption.

This trace says that loadkeys program managed to crash the kernel by calling an ioctl syscall(__x64_sys_ioctl) and that crash happens somewhere in memmove() function.

Sounds like a very strange bug to have. What could loadkeys do so complicated to get kernel confused? It’s whole source is 200 lines. Well, actual key loading happens here via ioctl(KDSKBMODE) and ioctl(KDSKBENT).

Searching internet for __memmove+loadkeys showsh that people are occasionally seeing these crashes since at least 2009 (kernel 4.1). I encountered no conclusive investigations and dived in.

The backtrace above suggests crash happened somewhere at vt_do_kdgkb_ioctl():

/* FIXME: This one needs untangling and locking */
int vt_do_kdgkb_ioctl(int cmd, struct kbsentry __user * user_kdgkb, int perm)
{
	struct kbsentry *kbs;
	char *p;
	u_char *q;
	u_char __user *up;
	int sz;
	int delta;
	char *first_free, *fj, *fnw;
	int i, j, k;
	int ret;

	if (!capable(CAP_SYS_TTY_CONFIG))
		perm = 0;

	kbs = kmalloc(sizeof(*kbs), GFP_KERNEL);
	if (!kbs) {
		ret = -ENOMEM;
		goto reterr;
	}

	/* we mostly copy too much here (512bytes), but who cares ;) */
	if (copy_from_user(kbs, user_kdgkb, sizeof(struct kbsentry))) {
		ret = -EFAULT;
		goto reterr;
	}
	kbs->kb_string[sizeof(kbs->kb_string) - 1] = '\0';
	i = kbs->kb_func;

	switch (cmd) {
	case KDGKBSENT:
		sz = sizeof(kbs->kb_string) - 1;	/* sz should have been
							   a struct member */
		up = user_kdgkb->kb_string;
		p = func_table[i];
		if (p)
			for (; *p && sz; p++, sz--)
				if (put_user(*p, up++)) {
					ret = -EFAULT;
					goto reterr;
				}
		if (put_user('\0', up)) {
			ret = -EFAULT;
			goto reterr;
		}
		kfree(kbs);
		return ((p && *p) ? -EOVERFLOW : 0);
	case KDSKBSENT:
		if (!perm) {
			ret = -EPERM;
			goto reterr;
		}

		q = func_table[i];
		first_free = funcbufptr + (funcbufsize - funcbufleft);
		for (j = i + 1; j < MAX_NR_FUNC && !func_table[j]; j++) ;
		if (j < MAX_NR_FUNC)
			fj = func_table[j];
		else
			fj = first_free;

		delta = (q ? -strlen(q) : 1) + strlen(kbs->kb_string);
		if (delta <= funcbufleft) {	/* it fits in current buf */
			if (j < MAX_NR_FUNC) {
				memmove(fj + delta, fj, first_free - fj);
				for (k = j; k < MAX_NR_FUNC; k++)
					if (func_table[k])
						func_table[k] += delta;
			}
			if (!q)
				func_table[i] = fj;
			funcbufleft -= delta;
		} else {	/* allocate a larger buffer */
			sz = 256;
			while (sz < funcbufsize - funcbufleft + delta)
				sz <<= 1;
			fnw = kmalloc(sz, GFP_KERNEL);
			if (!fnw) {
				ret = -ENOMEM;
				goto reterr;
			}

			if (!q)
				func_table[i] = fj;
			if (fj > funcbufptr)
				memmove(fnw, funcbufptr, fj - funcbufptr);
			for (k = 0; k < j; k++)
				if (func_table[k])
					func_table[k] =
					    fnw + (func_table[k] - funcbufptr);

			if (first_free > fj) {
				memmove(fnw + (fj - funcbufptr) + delta, fj,
					first_free - fj);
				for (k = j; k < MAX_NR_FUNC; k++)
					if (func_table[k])
						func_table[k] =
						    fnw + (func_table[k] -
							   funcbufptr) + delta;
			}
			if (funcbufptr != func_buf)
				kfree(funcbufptr);
			funcbufptr = fnw;
			funcbufleft = funcbufleft - delta + sz - funcbufsize;
			funcbufsize = sz;
		}
		strcpy(func_table[i], kbs->kb_string);
		break;
	}
	ret = 0;
 reterr:
	kfree(kbs);
	return ret;
}

It’s a huge function but it’s high-level purpose is simple:

  • handle ioctl(KDGKBSENT) call (Get KeyBoard Entries)
  • handle ioctl(KDSKBSENT) call (Set KeyBoard Entries)

Entries are struct kbsentry:

All it does is to substitute input char kb_func for a sequence of chars as kb_string (they can be scape sequences understood by linux terminal).

KDSKBSENT handler above is full of array handling logic. To understand is we need to look at the actual data structures in drivers/tty/vt/defkeymap.c_shipped:

/* Do not edit this file! It was automatically generated by   */
/*    loadkeys --mktable defkeymap.map > defkeymap.c          */

#include <linux/types.h>
#include <linux/keyboard.h>
#include <linux/kd.h>

...

/*
 * Philosophy: most people do not define more strings, but they who do
 * often want quite a lot of string space. So, we statically allocate
 * the default and allocate dynamically in chunks of 512 bytes.
 */

char func_buf[] = {
 '\033', '[', '[', 'A', 0, 
 '\033', '[', '[', 'B', 0, 
 '\033', '[', '[', 'C', 0, 
 '\033', '[', '[', 'D', 0, 
 '\033', '[', '[', 'E', 0, 
 '\033', '[', '1', '7', '~', 0, 
 '\033', '[', '1', '8', '~', 0, 
 '\033', '[', '1', '9', '~', 0, 
 '\033', '[', '2', '0', '~', 0, 
 '\033', '[', '2', '1', '~', 0, 
 '\033', '[', '2', '3', '~', 0, 
 '\033', '[', '2', '4', '~', 0, 
 '\033', '[', '2', '5', '~', 0, 
 '\033', '[', '2', '6', '~', 0, 
 '\033', '[', '2', '8', '~', 0, 
 '\033', '[', '2', '9', '~', 0, 
 '\033', '[', '3', '1', '~', 0, 
 '\033', '[', '3', '2', '~', 0, 
 '\033', '[', '3', '3', '~', 0, 
 '\033', '[', '3', '4', '~', 0, 
 '\033', '[', '1', '~', 0, 
 '\033', '[', '2', '~', 0, 
 '\033', '[', '3', '~', 0, 
 '\033', '[', '4', '~', 0, 
 '\033', '[', '5', '~', 0, 
 '\033', '[', '6', '~', 0, 
 '\033', '[', 'M', 0, 
 '\033', '[', 'P', 0, 
};

char *funcbufptr = func_buf;
int funcbufsize = sizeof(func_buf);
int funcbufleft = 0;          /* space left */

char *func_table[MAX_NR_FUNC] = {
 func_buf + 0,
 func_buf + 5,
 func_buf + 10,
 func_buf + 15,
 func_buf + 20,
 func_buf + 25,
 func_buf + 31,
 func_buf + 37,
 func_buf + 43,
 func_buf + 49,
 func_buf + 55,
 func_buf + 61,
 func_buf + 67,
 func_buf + 73,
 func_buf + 79,
 func_buf + 85,
 func_buf + 91,
 func_buf + 97,
 func_buf + 103,
 func_buf + 109,
 func_buf + 115,
 func_buf + 120,
 func_buf + 125,
 func_buf + 130,
 func_buf + 135,
 func_buf + 140,
 func_buf + 145,
 NULL,
 NULL,
 func_buf + 149,
 NULL,
};

Here we can see that func_buf is statically allocated flattened array of default keymaps. func_table array of pointers is a fast lookup table into flat func_buf array. If func_buf has not enough space it gets reallocated at funcbufptr.

That’s why vt_do_kdgkb_ioctl() is so complicated: it patches and update all these offsets.

Also note: func_buf and funcbufptr are both global pointers without any locking around these globals (also stressed by a FIXME above).

This is our somewhat smoking gun: if something in my system happens to call ioctl(KDSKBSENT) in parallel on multiple CPUs it will be able to mess up func_table into something that does not make sense. That can lead to strange things when you press these keys!

The only problem was that normally you have only one loadkeys being ran for a short time when your system boots up. Nothing else should be touching keymaps at that time anyway (or after).

Into the rabbit hole

To validate the race theory I added debug statement into vt_do_kdgkb_ioctl() function to see who calls it at boot:

Feb 24 12:06:35 sf systemd-vconsole-setup[343]: Executing "/usr/bin/loadkeys -q -C /dev/tty1 -u ru4"...
Feb 24 12:06:35 sf systemd-vconsole-setup[344]: /usr/bin/setfont succeeded.
Feb 24 12:06:35 sf systemd-vconsole-setup[344]: Executing "/usr/bin/loadkeys -q -C /dev/tty1 -u ru4"...
Feb 24 12:06:35 sf systemd-vconsole-setup[343]: Successfully forked off '(loadkeys)' as PID 423.
Feb 24 12:06:35 sf systemd-vconsole-setup[344]: Successfully forked off '(loadkeys)' as PID 424.
...
Feb 24 12:06:35 sf kernel: In vt_do_kdgkb_ioctl(19273=KDSKBSENT)/cpu=5/comm=loadkeys(424)
Feb 24 12:06:35 sf kernel: In vt_do_kdgkb_ioctl(19273=KDSKBSENT)/cpu=2/comm=loadkeys(423)
...
<more of these with interleaved PIDs>

Bingo: systemd was running exactly two instances of loadkeys at the same time: loadkeys(424) and loadkeys(423). It’s an ideal way to trigger the race: two processes are likely blocked by IO as they are executed for the first time from disk, and once unblocked execute exactly the same code in parallel instruction for instruction.

But why does systemd runs loadkeys twice? Why not once or as many times as I have ttys?

For many systems it’s supposed to happen only once. See 90-vconsole.rules udev rule:

# Each vtcon keeps its own state of fonts.
#
ACTION=="add", SUBSYSTEM=="vtconsole", KERNEL=="vtcon*", RUN+="@rootlibexecdir@/systemd-vconsole-setup"

Normally you have only one /sys/devices/virtual/vtconsole/vtcon0. But my system has two of these:

# cat /sys/devices/virtual/vtconsole/vtcon0/name
(S) dummy device
# cat /sys/devices/virtual/vtconsole/vtcon1/name
(M) frame buffer device

That dummy console comes from intel framebuffer driver:

i915 is an intel VGA video driver. My system has this driver compiled into kernel. That triggers kernel to discover and expose vtcon0/vtcon1 at the same time.

My speculation is that for non-intel-video systems (or for systems with intel driver loaded at a late stage) the condition might not trigger at all because those get only one loadkeys run (or a few runs spanned in time after each module is loaded).

The fix was simple: add some locking at least for write/write race. I did not touch read paths as I was not sure which subsystems use vt subsystem. Maybe some of them require decent throughput and lock for every character would be too much.

After this patch applied I had no bactraces at boot and no more unexpected poweroffs. But who knows, maybe it was a distraction and power button can’t be simulated through any tty escapes. We’ll see.

If you are wondering what you could fix yourself in linux kernel you can finish this work and also add read/write locking!

Parting words

  • The possible cause of spurious reboots was data corruption caused by very old race condiiton in kernel.
  • Silent data corruption is hard to diagnose if you don’t know where to look. I was lucky to get a kernel oops in the same buggy code.
  • tty/vt driver is full of globals. Those should perhaps be changed to be per-vtcon arrays (some non-x86 already have it tht way).
  • tty/vt global tables are actually generated by an old userspace tool loadkeys --mktable tool and stored in kernel as-is.
  • There is still a read/write race in kernel waiting for you to fix it!

Have fun!

Posted on June 26, 2019
<noscript>Please enable JavaScript to view the <a href="http://disqus.com/?ref_noscript">comments powered by Disqus.</a></noscript> comments powered by Disqus

June 16, 2019
Thomas Raschbacher a.k.a. lordvan (homepage, bugs)
DBMail swap from LDAP to SQL auth (June 16, 2019, 19:05 UTC)

Due to my old ldap server (with samba 2.X) dying I wanted to decouple the emails from the rest of the accounts since only 3 users actually use the email anyway the whole setup with LDAP was complete overkill..

Fortunately swapping that was relatively simple! - more to come soon (when I get a chance to finish this blog entry)

June 14, 2019
Thomas Raschbacher a.k.a. lordvan (homepage, bugs)

So I had errors in my nagios for UCS for the backup domain server and fileserver (after an upgrade probably):

these services were CRITICAL on several servers:

  • UNIVENTION_DISK_ROOT
  • UNIVENTION_JOINSTATUS
  • UNIVENTION_LOAD
  • UNIVENTION_NCSD2
  • UNIVENTION_PACKAGE_STATUS
  • UNIVENTION_SMTP2
  • UNIVENTION_SWAP

The errors were like "NRPE: Command 'UNIVENTION_DISK_ROOT' not defined" .. and so on ...

After a bit of searching i found this help forum page (in german): https://help.univention.com/t/nagios-zeigt-nach-update-auf-4-3-0-fur-slave-dc-nrpe-command-not-defined/8261/21

that in the end helped me solve the problem.

the following commands are what I did run (not all may be needed on each system, since for some reason one package was missing on the fileserver but not the backup domain server (also i am not sure if it is 100% needed but it works now so ..)

apt-get install nagios-nrpe-server nagios-nrpe-plugin univention-nagios-s4-connector # make sure all packages are installed
systemctl restart nagios-nrpe-server.service # restart the service
univention-directory-listener-ctrl resync nagios-client # resync - settings i guess?

I had ran the following 2 commands before the above too, but just got errors..

sudo -u nagios /usr/lib/nagios/plugins/check_univention_s4_connector_suidwrapper 
sudo -u nagios /usr/lib/nagios/plugins/check_univention_samba_drs_failures_suidwrapper

now the first of the 2 just gives me this: S4CONNECTOR OK: Connector not activated (connector/s4/autostart =! true)
The second one still gives command not found, but seems to not matter in my case.

Then i re-scheduled the nagios checks because I didn'T feel like waiting 10 minutes..then everything was ok again. looks like the upgrade missed some packages and/or settings?

June 12, 2019
Craig Andrews a.k.a. candrews (homepage, bugs)

To connect to AWS RDS databases using TLS/SSL, the client must trust the certificate provided by RDS; RDS doesn’t use certificates trusted by the CAs (Certificate Authorities) included by operating systems.

Without TLS/SSL, the connection to the database isn’t secure, meaning an attacker on the network between the client (running in EC2) and the database (running RDS) could eavesdrop or modify data.

To trust the AWS RDS certificate authority, on Docker, for a Red Hat / CentOS / Fedora / Amazon Linux (or other Fedora-type system) derived container, add the following to the Dockerfile:

#!/bin/bash
set -e # stop on all errors
RUN curl "https://s3-us-gov-west-1.amazonaws.com/rds-downloads/rds-combined-ca-us-gov-bundle.pem" --output /etc/pki/ca-trust/source/anchors/rds-combined-ca-us-gov-bundle.pem \
&& curl "https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem" --output /etc/pki/ca-trust/source/anchors/rds-combined-ca-bundle.pem \
&& update-ca-trust \
&& update-ca-trust force-enable

On AWS Elastic Beanstalk the .ebextensions mechanism can be used. In the jar/war/etc deployment archive, add this file:

.ebextensions/install_rds_certificates.config

packages:
  yum:
    bash: []
    curl: []
files:
  "/tmp/install_rds_certificates.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/usr/bin/env bash
      set -Eeuo pipefail # stop on all errors
      AVAILABILITY_ZONE=$(curl -s http://169.254.169.254/latest/dynamic/instance-identity/document | grep region | cut -d\" -f4)
      if [[ ${AVAILABILITY_ZONE} == us-gov-* ]]
      then
	curl "https://s3-us-gov-west-1.amazonaws.com/rds-downloads/rds-combined-ca-us-gov-bundle.pem" --output /etc/pki/ca-trust/source/anchors/rds-combined-ca-us-gov-bundle.pem
      else
	curl "https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem" --output /etc/pki/ca-trust/source/anchors/rds-combined-ca-bundle.pem
      fi
      update-ca-trust
      update-ca-trust force-enable
commands:
  01install_rds_certificates:
    command: "/tmp/install_rds_certificates.sh"

Next, modify the client to require a secure connection. For example, with the PostgreSQL JDBC client, add “?ssl=true” to the connection string url.

That it – you can now connect to your RDS database using SSL/TLS with the assurance that no MITM (Man In The Middle) attacks, eavesdropping attacks, etc are possible.

June 03, 2019
Thomas Raschbacher a.k.a. lordvan (homepage, bugs)
Server dying at inconvenient times .. (June 03, 2019, 06:33 UTC)

So our older server - which still had a few network shares on - decided to die .. on Saturday evening around 22:00 .. yay .. -- did I mention that server was also still the internet gateway for .. well everything? -- + DHCP server was on that (and DNS, but that is mirrored anyway so whatever)

Fortunately I had another small server already lined up and (mostly) prepared to take over the routing - and while I was at it i made it the DHCP server too for now.

So apart from the fact that the silly router we got from our ISP likes to not accept a different MAC address with the same IP as before immediately I had to wait a bit for that to work, but then just set up DHCP and the internet was up & running again .

Now about figuring out what is wrong with the server .. switiching it back on the bios gives cirtical warnings about voltages .. wonderful.. I managed to get it to boot to a root shell once more - which allowed me to copy /etc (which included the up2date DHCP config so yay!)

Since it seems to be either the power supply or maybe the - by now at least 15 years old - mainboard I decided *not* to dwell on that much longer that evening, and after internet and a few other basics were back online I called it a night.

Next morning 07:30 I'm back in the server room, opening up the server, but not seeing any easy way to use a normal (non-server) power supply to easily test things out.. after some consideration plan B: swap (some of) the hard disks over to the new server (which fortunately has 3 HDDs in, but has 8 slots as I went for the bigger one -- which was definitely good considering now!.

So i eject the old hdd from the hot-swap and try to remove the HDD .. to find out I have to go into the workshop because some joker decided to use Torx screws for that and not the usual cross/phillips .. after that out of the way the 2 hdds were in the new server and found.. but INACTIVE ARRAY was shown .. took me a while to figure out how to deal with it -- I purposely did only put one of each of the mirrored disks in the new server as a precaution.

So anyway the steps to get them working:

  1. run mdadm -E --scan
  2. add the arrays to /etc/mdadm.conf
  3. then - because the array is incomplete need to stop, then re-assemble (with --run) like this: mdadm --stop /dev/md121 ;  mdadm --assemble --run /dev/md121 /dev/sdd2

After that it started up again and in mdstat the inactive changed to active:

md121 : active raid1 sdd2[1]
505920 blocks [2/1] [_U]

Fortunately the LVM on that server had different names than on the newer server so no conflicts there, so I could just mount the devices.

But instead of mounting the Data shares on that server I directly passed it with virtio to the KVM/QEMU instance with my Univention Coroporate Server fileserver - as described here in my old blog post - just add a 2nd -disk and reboot it: Running UCS (Univention Corporate Server) Core on Gentoo with kvm + using an LVM volume

May 23, 2019
Craig Andrews a.k.a. candrews (homepage, bugs)
End to End Encryption with Beanstalk (May 23, 2019, 14:25 UTC)

Beanstalk is often configured to terminate SSL at the load balancer then make the connection to the web server/application instances using unencrypted HTTP. That’s usually okay as the AWS network is designed to keep such traffic private, but under certain conditions, such as those requiring PCI compliance, DoD/government rules, or simply out of an abundance of caution, there’s a desire to have all traffic encrypted – including that between the Beanstalk load balancer and servers.

There are two approaches for implementing end to end encryption on Beanstalk:

  • Use a layer 4 load balancer (Network or Classic Elastic Load Balancer.
    Using this approach, the load balancer never decrypts the traffic. The downside is that advanced reporting isn’t possible and layer 7 features, such as session affinity, cannot be implemented.
  • Use a layer 7 load balancer (Application or Classic Load Balancer).
    Using this approach, traffic is decrypted at the load balancer. The load balancer would then re-encrypt traffic to the servers. Session affinity and traffic reporting are available.

The preferred solution is to use the layer 7 approach with an Application Load Balancer. This preference is due to the additional features the layer 7 offers, because Network Load Balancers are more expensive, and because AWS is deprecating Classic Load Balancers.

The simplest way to accomplish this goal is to use a self signed certificate on the servers and then use HTTPS from the load balancer to the server. Application Load Balancers do not currently perform validation of certificates which is why the self signed approach works and why there’s no advantage to using a CA issued certificate.

The following approach will work on any Beanstalk supported platform that uses nginx as the proxy server. This configuration is based on AWS’s documentation, but trimmed for only Application Load Balancers and to include the nginx configuration and self-signed certificate generation.

In your Beanstalk application archive, add these files:

.ebextensions/nginx/conf.d/https.conf

# HTTPS server

server {
    listen       443;
    server_name  localhost;
    
    ssl                  on;
    # The certificate is generated in generate-certificate.sh
    ssl_certificate      /etc/pki/tls/certs/server.crt;
    ssl_certificate_key  /etc/pki/tls/certs/server.key;
    
    ssl_session_timeout  5m;
    
    ssl_protocols  TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers   on;
    
    location / {
        proxy_pass  http://localhost:5000;
        proxy_set_header   Connection "";
        proxy_http_version 1.1;
        proxy_set_header        Host            $host;
        proxy_set_header        X-Real-IP       $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header        X-Forwarded-Proto https;
    }
}

.ebextensions/nginx-https.conf

option_settings:
  aws:elasticbeanstalk:application:
    Application Healthcheck URL: HTTPS:443/
  aws:elasticbeanstalk:environment:process:default:
    Port: '443'
    Protocol: HTTPS
  aws:elbv2:listener:443:
    DefaultProcess: https
    ListenerEnabled: 'true'
    Protocol: HTTPS
  aws:elasticbeanstalk:environment:process:https:
    Port: '443'
    Protocol: HTTPS
packages:
  yum:
    bash: []
    openssl: []
files:
  "/tmp/generate_nginx_certificate.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/usr/bin/env bash
      set -Eeuo pipefail # stop on all errors
      # These files are used by nginx, see nginx/conf.d/https.conf
      openssl genrsa 2048 > /etc/pki/tls/certs/server.key
      openssl req -new -x509 -nodes -sha1 -days 3650 -extensions v3_ca -key /etc/pki/tls/certs/server.key -subj "/CN=localhost" > /etc/pki/tls/certs/server.crt
commands:
  01generate_nginx_certificate:
    command: "/tmp/generate_nginx_certificate.sh"

May 22, 2019
Craig Andrews a.k.a. candrews (homepage, bugs)

The US DoD (Department of Defense) uses its own root certificate when signing https certificates for its domains. For example, https://www.my.af.mil/ uses such a certificate. These root certificates are not trusted by any (commercial/public) operating system, browser, or other client. Therefore, in order to access these sites and not get an error, the DoD certificates must be trusted.

On Windows, go to DISA’s PKI and PKE Tools page and under “Trust Store” follow the directions for the “InstallRoot X: NIPR Windows Installer”

On Linux, download the certificates from MilitaryCAC’s Linux Information page (direct link to the certificates). Then follow your distribution’s instructions on how to install certificates to the trust store. For example, on Red Hat / CentOS / Fedora / Amazon Linux, copy the certificates to /etc/pki/ca-trust/source/anchors/ then run update-ca-trust. On Debian / Ubuntu and Gentoo, copy the certificates to /usr/local/share/ca-certificates/ then run update-ca-certificates.

On Docker, for a Red Hat / CentOS / Fedora / Amazon Linux (or other Fedora-type system) derived container, add the following to the Dockerfile:

#!/bin/bash
set -e # stop on all errors
RUN yum -y install openssl \
&& CERT_BUNDLE="Certificates_PKCS7_v5.3_DoD" \
&& curl "https://iasecontent.disa.mil/pki-pke/${CERT_BUNDLE}.zip" --output certs.zip \
&& unzip certs.zip "${CERT_BUNDLE}/${CERT_BUNDLE}.pem.p7b" \
&& openssl pkcs7 -in "${CERT_BUNDLE}/${CERT_BUNDLE}.pem.p7b" -print_certs -out "/etc/pki/ca-trust/source/anchors/${CERT_BUNDLE}.pem" \
&& update-ca-trust \
&& update-ca-trust force-enable \
&& rm -rf certs.zip "${CERT_BUNDLE}" \
&& yum -y remove openssl \
&& rm -rf /var/cache/yum

On AWS Elastic Beanstalk the .ebextensions mechanism can be used. In the jar/war/etc deployment archive, add these files:

.ebextensions/install_dod_certificates.config

packages:
  yum:
    bash: []
    curl: []
    openssl: []
    unzip: []
files:
  "/tmp/install_dod_certificates.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/usr/bin/env bash
      set -Eeuo pipefail # stop on all errors
      cd /tmp
      CERT_BUNDLE="Certificates_PKCS7_v5.3_DoD"
      curl "https://iasecontent.disa.mil/pki-pke/${CERT_BUNDLE}.zip" --output certs.zip
      unzip certs.zip "${CERT_BUNDLE}/${CERT_BUNDLE}.pem.p7b"
      openssl pkcs7 -in "${CERT_BUNDLE}/${CERT_BUNDLE}.pem.p7b" -print_certs -out "/etc/pki/ca-trust/source/anchors/${CERT_BUNDLE}.pem"
      update-ca-trust
      update-ca-trust force-enable
      rm -rf certs.zip "${CERT_BUNDLE}"
commands:
  01install_dod_certificates:
    command: "/tmp/install_dod_certificates.sh"

May 08, 2019
Craig Andrews a.k.a. candrews (homepage, bugs)

Spring Session JDBC is a great way to allow an application to be stateless. By storing the session in the database, a request can be routed to any application server. This approach provides significant advantages such as automatic horizontal scaling, seamless failover, and no need for session affinity. By using JDBC, the database the application is already using provides the storage avoiding the need to setup and maintain other software, such as Memcache or Redis.

When Spring Session JDBC stores the session in the database, it has to serialize (convert from a Java object to a string) the session and also deserialize it (convert from a string back to a Java object). By default, it uses Java’s built in serialization.

There are numerous reasons not to use Java’s built in serialization (ObjectInputSteam / ObjectOutputStream). Oracle calls it a “Horrible Mistake” and plans to remove it in a future Java release. It’s also less performant and produces a larger serialized form than many alternatives.

Since Java serialization is (at least, for now) included in Java, it’s still commonly used, including by Spring Session JDBC. Switching to another serialization method can be a relatively quick and easy way to improve performance.

Any serialization can be used, including Jackson (which uses the JSON or XML format), Protocol Buffers, Avro, and more. However, all require work to define schemas for the data and additional configuration. In the interest of avoiding those efforts (which is especially important for legacy applications), a schemaless serializer (which is what Java’s built in serializer is) can be used such as FST (fast-serializer) or Kryo.

Switching the serializer used by Spring Session JDBC is done by defining a a bean named springSessionConversionService of type ConversionService. The following examples provide the code to use FST or Kryo.

Using FST with Spring Session JDBC

Add FST as dependency to the project. For example, using Maven:

<dependency>
    <groupId>de.ruedigermoeller</groupId>
    <artifactId>fst</artifactId>
    <version>2.56</version>
</dependency>

And these add these classes:

FstSessionConfig.java

import org.springframework.beans.factory.BeanClassLoaderAware;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.convert.ConversionService;
import org.springframework.core.convert.support.GenericConversionService;
import org.springframework.core.serializer.support.DeserializingConverter;
import org.springframework.core.serializer.support.SerializingConverter;

@Configuration
public class FstSessionConfig implements BeanClassLoaderAware {

	private ClassLoader classLoader;

	@Bean
	public ConversionService springSessionConversionService() {
		final FstDeserializerSerializer fstDeserializerSerializer = new FstDeserializerSerializer(classLoader);

		final GenericConversionService conversionService = new GenericConversionService();
		conversionService.addConverter(Object.class, byte[].class,
				new SerializingConverter(fstDeserializerSerializer));
		conversionService.addConverter(byte[].class, Object.class,
				new DeserializingConverter(fstDeserializerSerializer));
		return conversionService;
	}

	@Override
	public void setBeanClassLoader(final ClassLoader classLoader) {
		this.classLoader = classLoader;
	}
}

FstDeserializerSerializer.java

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;

import org.nustaq.serialization.FSTConfiguration;
import org.nustaq.serialization.FSTObjectOutput;
import org.springframework.core.NestedIOException;
import org.springframework.core.serializer.Deserializer;
import org.springframework.core.serializer.Serializer;

public class FstDeserializerSerializer implements Serializer<Object>, Deserializer<Object> {

	private final FSTConfiguration fstConfiguration;
	
	public FstDeserializerSerializer(final ClassLoader classLoader) {
		fstConfiguration = FSTConfiguration.createDefaultConfiguration();
		fstConfiguration.setClassLoader(classLoader);
	}

	@Override
	public Object deserialize(InputStream inputStream) throws IOException {
		try{
			return fstConfiguration.getObjectInput(inputStream).readObject();
		}
		catch (ClassNotFoundException ex) {
			throw new NestedIOException("Failed to deserialize object type", ex);
		}
	}

	@Override
	public void serialize(Object object, OutputStream outputStream) throws IOException {
		// Do not close fstObjectOutput - that would prevent reuse and cause an error
		// see https://github.com/RuedigerMoeller/fast-serialization/wiki/Serialization
		@SuppressWarnings("resource")
		final FSTObjectOutput fstObjectOutput = fstConfiguration.getObjectOutput(outputStream);
		fstObjectOutput.writeObject(object);
		fstObjectOutput.flush();
	}
}

Using Kryo with Spring Session JDBC

Add Kryo as dependency to the project. For example, using Maven:

<dependency>
	<groupId>com.esotericsoftware</groupId>
	<artifactId>kryo</artifactId>
	<version>5.0.0-RC4</version>
</dependency>

And these add these classes:

KryoSessionConfig.java

import org.springframework.beans.factory.BeanClassLoaderAware;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.convert.ConversionService;
import org.springframework.core.convert.support.GenericConversionService;
import org.springframework.core.serializer.support.DeserializingConverter;
import org.springframework.core.serializer.support.SerializingConverter;

@Configuration
public class KryoSessionConfig implements BeanClassLoaderAware {

	private ClassLoader classLoader;

	@Bean
	public ConversionService springSessionConversionService() {
		final KryoDeserializerSerializer kryoDeserializerSerializer = new KryoDeserializerSerializer(classLoader);

		final GenericConversionService conversionService = new GenericConversionService();
		conversionService.addConverter(Object.class, byte[].class,
				new SerializingConverter(kryoDeserializerSerializer));
		conversionService.addConverter(byte[].class, Object.class,
				new DeserializingConverter(kryoDeserializerSerializer));
		return conversionService;
	}

	@Override
	public void setBeanClassLoader(final ClassLoader classLoader) {
		this.classLoader = classLoader;
	}
}

KryoDeserializerSerializer.java

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;

import org.springframework.core.serializer.Deserializer;
import org.springframework.core.serializer.Serializer;

import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.io.Input;
import com.esotericsoftware.kryo.io.Output;
import com.esotericsoftware.kryo.util.Pool;

public class KryoDeserializerSerializer implements Serializer<Object>, Deserializer<Object> {
	private final ClassLoader classLoader;
	
	// Pool constructor arguments: thread safe, soft references, maximum capacity
	private final Pool<Kryo> kryoPool = new Pool<Kryo>(true, true) {
	   protected Kryo create () {
	      final Kryo kryo = new Kryo();
	      kryo.setClassLoader(classLoader);
	      kryo.setRegistrationRequired(false);
	      return kryo;
	   }
	};
	
	public KryoDeserializerSerializer(final ClassLoader classLoader) {
		this.classLoader = classLoader;
	}

	@Override
	public Object deserialize(InputStream inputStream) throws IOException {
		final Kryo kryo = kryoPool.obtain();
		try(final Input input = new Input(inputStream)){
			return kryo.readObjectOrNull(input, null);
		}finally {
			kryoPool.free(kryo);
		}
	}

	@Override
	public void serialize(Object object, OutputStream outputStream) throws IOException {
		final Kryo kryo = kryoPool.obtain();
		try(final Output output = new Output(outputStream)){
			kryo.writeObject(output, object);
		}finally {
			kryoPool.free(kryo);
		}
	}
}

How to Choose which Serializer to Use

The process of selecting which serializer to use for an application should be done only through testing, both for functionality and for performance.

For some applications, a serializer won’t work due to known limitations in the serializer or bugs in it. For example, FST doesn’t currently support the Java 8 time classes, so if your application stores session data using such a class, FST is not for you. With Kryo, I ran into a bug stopping me from using it (which will be fixed in version 5.0.0-RC5 and later).

Performance will also vary between serializers for each application. Factors that impact performance include exactly what is being serialized, how big that data is, how often it’s accessed, the version of Java, and how the system is configured. FST has published some benchmarks, but that information must be taken with a grain of salt as those benchmarks are only measuring very specific, isolated scenarios. That data does provide general guidance though – you can expect better performance when you switch from the Java serializer to FST, for example, but testing of the full application will need to be done to determine if the improvement is 0.1% or 10%.

May 03, 2019
Luca Barbato a.k.a. lu_zero (homepage, bugs)
Using Wireguard (May 03, 2019, 08:51 UTC)

wireguard is a modern, secure and fast vpn tunnel that is extremely simple to setup and works already nearly everywhere.

Since I spent a little bet to play with it because this looked quite interesting, I thought of writing a small tutorial.

I normally use Gentoo (and macos) so this guide is about Gentoo.

General concepts

Wireguard sets up peers identified by an public key and manages a virtual network interface and the routing across them (optionally).

The server is just a peer that knows about loots of peers while a client knows how to directly reach the server and that’s it.

Setting up in Gentoo

Wireguard on Linux is implemented as a kernel module.

So in general you have to build the module and the userspace tools (wg).
If you want to have some advanced feature make sure that your kernel has the following settings:

IP_ADVANCED_ROUTER
IP_MULTIPLE_TABLES
NETFILTER_XT_MARK

After that using emerge will get you all you need:

$ emerge wireguard

Tools

The default distribution of tools come with the wg command and an helper script called wg-quick that makes easier to bring up and down the virtual network interface.

wg help
Usage: wg <cmd> [<args>]

Available subcommands:
  show: Shows the current configuration and device information
  showconf: Shows the current configuration of a given WireGuard interface, for use with `setconf'
  set: Change the current configuration, add peers, remove peers, or change peers
  setconf: Applies a configuration file to a WireGuard interface
  addconf: Appends a configuration file to a WireGuard interface
  genkey: Generates a new private key and writes it to stdout
  genpsk: Generates a new preshared key and writes it to stdout
  pubkey: Reads a private key from stdin and writes a public key to stdout
You may pass `--help' to any of these subcommands to view usage.
Usage: wg-quick [ up | down | save | strip ] [ CONFIG_FILE | INTERFACE ]

  CONFIG_FILE is a configuration file, whose filename is the interface name
  followed by `.conf'. Otherwise, INTERFACE is an interface name, with
  configuration found at /etc/wireguard/INTERFACE.conf. It is to be readable
  by wg(8)'s `setconf' sub-command, with the exception of the following additions
  to the [Interface] section, which are handled by wg-quick:

  - Address: may be specified one or more times and contains one or more
    IP addresses (with an optional CIDR mask) to be set for the interface.
  - DNS: an optional DNS server to use while the device is up.
  - MTU: an optional MTU for the interface; if unspecified, auto-calculated.
  - Table: an optional routing table to which routes will be added; if
    unspecified or `auto', the default table is used. If `off', no routes
    are added.
  - PreUp, PostUp, PreDown, PostDown: script snippets which will be executed
    by bash(1) at the corresponding phases of the link, most commonly used
    to configure DNS. The string `%i' is expanded to INTERFACE.
  - SaveConfig: if set to `true', the configuration is saved from the current
    state of the interface upon shutdown.

See wg-quick(8) for more info and examples.

Creating a configuration

Wireguard is quite straightforward, you can either prepare a configuration with your favourite text editor or generate one by setting by hand the virtual network device and then saving the result wg showconf presents.

A configuration file then can be augmented with wg-quick-specific options (such as Address) or just passed to wg setconf while the other networking details are managed by your usual tools (e.g. ip).

Create your keys

The first step is to create the public-private key pair that identifies your peer.

  • wg genkey generates a private key for you.
  • You feed it to wg pubkey to have your public key.

In a single line:

$ wg genkey | tee privkey | wg pubkey > pubkey

Prepare a configuration file

Both wg-quick and wg setconf use an ini-like configuration file.

If you put it in /etc/wireguard/${ifname}.conf then wg-quick would just need the interface name and would look it up for you.

The minimum configuration needs an [Interface] and a [Peer] set.
You may add additional peers later.
A server would specify its ListenPort and identify the peers by their PublicKey.

[Interface]
Address = 192.168.2.1/24
ListenPort = 51820
PrivateKey = <key>

[Peer]
PublicKey = <key>
AllowedIPs = 192.168.2.2/32

A client would have a peer with an EndPoint defined and optionally not specify the ListenPort in its interface description.

[Interface]
PrivateKey = <key>
Address = 192.168.2.2/24

[Peer]
PublicKey = <key>
AllowedIPs = 192.168.2.0/24
Endpoint = <ip>:<port>

The AllowedIPs mask let you specify how much you want to route over the vpn.
By setting 0.0.0.0/0 you tell you want to route ALL the traffic through it.

NOTE: Address is a wg-quick-specific option.

Using a configuration

wg-quick is really simple to use, assuming you have created /etc/wireguard/wg0.conf:

$ wg-quick up wg0
$ wg-quick down wg0

If you are using netifrc from version 0.6.1 wireguard is supported and you can have a configuration such as:

config_wg0="192.168.2.4/24"
wireguard_wg0="/etc/wireguard/wg0.conf"

With the wg0.conf file like the above but stripped of the wg-quick-specific options.

Summing up

Wireguard is a breeze to set up compared to nearly all the other vpn solutions.

Non-linux systems can currently use a go implementation and in the future a rust implementation (help welcome).

Android and macos have already some pretty front-ends that make the setup easy even on those platforms.

I hope you enjoyed it 🙂

May 02, 2019
Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)

Term has already started, so this announcement is technically a bit late, however... This summer term I'm offering a lecture "High Frequency Engineering for Physicists". If you plan to work with signals in the frequency range 10MHz - 50GHz, this might be interesting for you...

When and where? Wednesdays, 12h - 14h, seminar room PHY 9.1.10. The next lecture is on 8 May 2019
  • Concepts and formalisms for the frequency range 10MHz - 50GHz
  • Handling equipment for this frequency range, designing devices and measurements
  • Using this frequency range in a (millikelvin) cryostat
More information can be found soon on the homepage of the lecture.

See you next wednesday!

April 30, 2019
Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)
Press release (in German) on our recent PRL (April 30, 2019, 13:43 UTC)

Regensburg University has published a press release (in German) on our recent Physical Review Letters "Editor's Suggestion" publication, "Shaping Electron Wave Functions in a Carbon Nanotube with a Parallel Magnetic Field". Read it on the university web page!

(A summary in English can be found in a previous blog post.)

April 29, 2019
Yury German a.k.a. blueknight (homepage, bugs)
Gentoo Blogs Update (April 29, 2019, 03:41 UTC)

This is just a notification that the Blogs and the appropriate plug-ins for the release 5.1.1 have been updated.

With the release of these updated we (The Gentoo Blog Team) have updated the themes that had updates. If you have a blog on this site, and have a theme that is based on one of the following themes please consider updating as these themes are no longer updated and things will break in your blogs.

  • KDE Breathe
  • KDE Graffiti
  • Oxygen
  • The Following WordPress versions might stop working (Simply because of age)
    • Twenty Fourteen
    • Twenty Fifteen
    • Twenty Sixteen

If you are using one of these themes it is recommended that you update to the other themes available. If you think that there is an open source theme that you would like to have available please contact the Blogs team by opening a Bugzilla Bug with pertinent information.

April 25, 2019
Craig Andrews a.k.a. candrews (homepage, bugs)
The Sad Story of TCP Fast Open (April 25, 2019, 19:05 UTC)

If there’s a way to make something fast, you’ve got my attention. Especially when there’s a way to make a lot of things fast with a simple change – and that’s what TCP Fast Open (TFO) promises to do.

TFO (RFC 7413) started out in 2011 as a way to eliminate one of the round trips involved in opening a TCP connection. Since TCP (Transmission Control Protocol) is the underlying fundamental technology used for almost all connections on the Internet, an improvement to TCP connection establishment performance would improve the performance of virtually everything, especially web browsing. In early testing discussed at the 2011 Linux Plumbers Conference, Google found that TFO reduced page load times by 4-40%. Because of the elimination of a round trip, the slowest, highest latency connections would benefit the most – TFO promised to be a great improvement for many users.

Given the immediate success, support for this performance improving technology rapidly grew. In 2012, Linux 3.7 gained support for client and server TFO. In 2013, Android gained support when KitKat (4.4) was released using the Linux 3.10 kernel. In 2015, iOS gained support. In 2016, Windows 10 got support in the Anniversary Update. Even load balancers, such as F5, added support.

And yet, today, not one browser supports it. Chrome, Firefox, and Edge all have use of TFO disabled by default.

So, what happened to this technology that once sounded so promising?

Initial Optimism Meets Hard Reality

I attribute the failure to achieve widespread adoption of TCP Fast Open to four factors:

  1. Imperfect initial planning
  2. Middleboxes
  3. Tracking concerns
  4. Other performance improvements

Factor 1: Imperfect Initial Planning

TCP Fast Open was in trouble from its initial conception. Because it involves change to an operating system, it had to be done perfectly from the very beginning. Operating systems have long lifespans – updates happen slowly, backwards compatibility is paramount, and changes are, rightfully so, difficult to make. So, when the TFO specification wasn’t perfect the first time, it was a major blow to the changes of its ever achieving widespread adoption.

TFO requires the allocation of a new, dedicated TCP Option Kind Number. Option Kind Numbers specify which features are in use during a TCP connection and, as a new feature, TFO requires a new TCP Option Kind Number. Since TFO was experimental when it started out, it used a number (254 with magic 0xF989) from the experimental allocation (as described in RFC 4727). This was quickly ingrained into Windows, iOS, Linux. and more. As the saying goes, “nothing is as permanent as a temporary solution.”

So, when TFO left experimental status with RFC 7413, they released a statement saying that all current versions should migrated over to the new option. Or, in more complex terms,“Existing implementations that are using experimental option 254 per [RFC6994] with magic number 0xF989 (16 bits) as allocated in the IANA “TCP Experimental Option Experiment Identifiers (TCP ExIDs)” registry by this document, should migrate to use this new option (34) by default.”

Did all implementations migrate? If they did, they would lose compatibility with those that didn’t migrate. Therefore, all systems must now support both the experimental TCP Option Kind Number and the permanent one.

This issue isn’t a deal breaker – but it certainly wasn’t a great way to start out.

Factor 2: Middleboxes

Middleboxes are the appliances that sit between the end user and the server they’re trying to reach. They’re firewall, proxies, routers, caches, security devices, and more. They’re rarely updated, very expensive, and run proprietary software. Middleboxes are, in short, why almost everything runs over HTTP today and not other protocols as the original design for the Internet envisioned.

The first sign of trouble appeared in the initial report from Google in 2011 regarding TFO. As reported by LWN, “about 5% of the systems on the net will drop SYN packets containing unknown options or data. There is little to be done in this situation; TCP fast open simply will not work. The client must thus remember cases where the fast-open SYN packet did not get through and just use ordinary opens in the future.”

Over the years, Google and Mozilla did more testing and eventually found that TFO wasn’t beneficial. Clients that initiate TFO connections encounter failures requiring them to re-try without TFO so often that, on average, TFO costs more time than it saves. In some networks, TFO never works – for example, China Mobile’s firewall consistently fails to accept TFO requiring every connection to be retried without it, leading to TFO actually increasing roundtrips.

Middleboxes are probably the fatal blow for TFO: the existing devices won’t be replaced for (many) years, and the new replacement devices may have the same problems.

Factor 3: Tracking Concerns

When a client makes an initial connection to a host, TFO negotiates a unique random number called a cookie; on subsequent connections to the same host, the client uses the cookie to eliminate one round trip. Using this unique cookie allows servers using TFO to track users. For example, if a user browses to a site, then opens an incognito window and goes to the same site, the same TFO cookie would be used in both windows. Furthermore, if a user goes to a site at work, then uses the same browser to visit that site from a coffee shop, the same TFO cookie would be used in both cases, allowing the site to know it’s the same user.

In 2011, tracking by the governments and corporations wasn’t nearly as much of a concern as it is today. It would still be two years before Edward Snowden would release documents describing the US government’s massive surveillance programs.

But, in 2019, tracking concerns are real. TFO potential to be used for user tracking makes it unacceptable for most use cases.

One way to mitigate tracking concerns would be for the TFO cookie cache to be cleared whenever the active network changes. Windows/Linux/MacOS/FreeBSD/etc should consider clearing the OS’s TFO cookie cache when changing networks. See this discussion on curl’s issue tracker for more.

Factor 4: Other Performance Improvements

When TFO started out, HTTP/2 was not yet in use – in fact, HTTP/2’s precursor, SPDY, had a draft until 2012. With HTTP/1, a client would make many connections to the same server to make parallel requests. With HTTP/2, clients can make parallel requests over the same TCP connections. Therefore, since it setups up far fewer TCP connections, HTTP/2 benefits much less than HTTP/1 from TFO.

HTTP/3 plans to use UDP (User Datagram Protocol) to reduce connection setup round trips gaining the same performance advantage of TFO but without its problems. UDP is a fundamental Internet protocol like TCP but originally designed for one way connectionless communication. HTTP/3 will build a highly optimized connection establishment system logically analogous to TCP on top of UDP. The end result will be faster than what TCP can do even with TFO.

TLS (Transport Layer Security) 1.3 offers another improvement that reduces round trips called 0RTT. TLS is the system used for encryption in HTTPS so improvements to TLS potentially improve all users of HTTPS.

In the end, performance has been improving without requiring TFO’s drawbacks/costs.

The Future of TFO

TFO may never be universally used, but it still has its place. The best use case for TFO is with relatively new clients and servers, connected by a network using either no middleboxes or only middleboxes that don’t interfere with TFO, when user tracking isn’t a concern.

Domain Name System (DNS) is such a use case. DNS is how software (such as a browser) resolves human-readable names (such as integralblue.com) to an IP (Internet Protocol) address to which the computer can connect. DNS is very latency sensitive and very frequently used – eliminating the latency from one round trip would give a perceivable improvement to users. The same TCP connections are made from the same clients to the same servers repeatedly, which is TFO’s best case scenario. And, there’s no tracking concern since many DNS clients and servers don’t move around (there’s no “incognito” mode for DNS). Stubby, Unbound, dnsmasq, BIND, and PowerDNS, for example, include or are currently working on support for TFO.

TFO is already supported by all the major operating systems so it is here to stay. The question is, will it ever see widespread real world adoption? In the short term, TFO will be adopted in specialty areas such as DNS. But, eventually, , perhaps in the 5 or 10 year time frame, TFO will see widespread adoption on the Web as troublesome middleboxes are gradually replaced allowing browsers to enable support for it. By that time, however, HTTP/3 will be widely deployed, offering a better performance improvement than TFO could ever offer for the use case of web browsing. In the end, TFO is an interesting example of an idea and its implementation being challenged (due to technical limitations and a changing political landscape), resulting in its inability to live up to expectations. However, like so many other technologies, the original implementation isn’t what matters most – the idea is the significant part. In the case of TFO, the performance improving concept behind it will benefit users for years to it influences future technologies such as HTTP/3.

DNSSEC on OpenWrt 18.06 (April 25, 2019, 15:18 UTC)

DNSSEC ensures that the results of DNS queries (for DNSSEC enabled domains) are authentic. For example, integralblue.com uses DNSSEC, so if an attacker (using a man in the middle or spoofing attack) changes the IP address that www.integralblue.com resolves to, then a DNS resolver supporting DNSSEC will be able to tell and return an error.

DNSSEC provides authentication and integrity; it does not provide for confidentiality. For confidentiality (so your ISP, for example, cannot tell what DNS queries are being made), you can easily add TLS over DNS which I’ve described how to do in OpenWrt in another post.

By setting up DNSSEC on your OpenWrt router, you protect your entire network as all clients will perform DNS requests using your OpenWrt router’s DNS server which in turn will do DNSSEC checking for all queries.

Setting up DNSSEC on OpenWrt 18.06 is remarkably easy. You can use the LuCI web interface to perform these steps or shell command over ssh; I’m providing the commands here.

  1. Refresh the package list: opkg update
  2. Swap dnsmasq for dnsmasq-full (-full includes DNSSEC support) and remove odhcpd-ipv6only: opkg install dnsmasq-full --download-only && opkg remove dnsmasq odhcpd-ipv6only && opkg install dnsmasq-full --cache . && rm *.ipk
  3. Edit /etc/config/dhcp
    In the config dnsmasq section, add (or change the values of, if these settings already exist) these settings:

    • option dnssec '1'
    • option dnsseccheckunsigned '1'
  4. Restart dnsmasq so the changes take effect: /etc/init.d/dnsmasq restart

Enjoy knowing that now no one is tampering with your DNS queries.


April 24, 2019
Matthew Thode a.k.a. prometheanfire (homepage, bugs)
Building Gentoo disk images (April 24, 2019, 05:00 UTC)

Disclaimer

I'm not responsible if you ruin your system, this guide functions as documentation for future me. Remember to back up your data.

Why this is useful / needed

It's useful to have a way of building a disk image for shipping, either for testing or production usage. The image output formats could be qcow2, raw or compressed tarball, it's up to you to make this what you want it to be.

Pre-work

Install diskimage-builder, for Gentoo you just have to 'emerge' the latest version. I personally keep one around in a virtual environment for testing (this allows me to build musl images as well easily).

The actual setup

What diskimage-builder actually does is take elements and run them. Each elements consists of a set of phases where the element takes actions. All you are really doing is defining the elements and they will insert themselves where needed. It also uses environment variables for tunables, or for other various small tweaks.

This is how I build the images at http://distfiles.gentoo.org/experimental/amd64/openstack/

export GENTOO_PORTAGE_CLEANUP=True
export DIB_INSTALLTYPE_pip_and_virtualenv=package
export DIB_INSTALLTYPE_simple_init=repo
export GENTOO_PYTHON_TARGETS="python3_6"
export GENTOO_PYTHON_ACTIVE_VERSION="python3.6"
export ELEMENTS="gentoo simple-init growroot vm openssh-server block-device-mbr"
export COMMAND="disk-image-create -a amd64 -t qcow2 --image-size 3"
export DATE="$(date -u +%Y%m%d)"

GENTOO_PROFILE=default/linux/amd64/17.0/no-multilib/hardened ${COMMAND} -o "gentoo-openstack-amd64-hardened-nomultilib-${DATE}" ${ELEMENTS}
GENTOO_PROFILE=default/linux/amd64/17.0/no-multilib ${COMMAND} -o "gentoo-openstack-amd64-default-nomultilib-${DATE}" ${ELEMENTS}
GENTOO_PROFILE=default/linux/amd64/17.0/hardened ${COMMAND} -o "gentoo-openstack-amd64-hardened-${DATE}" ${ELEMENTS}
GENTOO_PROFILE=default/linux/amd64/17.0/systemd ${COMMAND} -o "gentoo-openstack-amd64-systemd-${DATE}" ${ELEMENTS}
${COMMAND} -o "gentoo-openstack-amd64-default-${DATE}" ${ELEMENTS}

For musl I've had to do some custom work as I have to build the stage4s locally, but it's largely the same (with the additional need to define a musl overlay.

cd ~/diskimage-builder
cp ~/10-gentoo-image.musl diskimage_builder/elements/gentoo/root.d/10-gentoo-image
pip install -U .
cd ~/

export GENTOO_PORTAGE_CLEANUP=False
export DIB_INSTALLTYPE_pip_and_virtualenv=package
export DIB_INSTALLTYPE_simple_init=repo
export GENTOO_PYTHON_TARGETS="python3_6"
export GENTOO_PYTHON_ACTIVE_VERSION="python3.6"
DATE="$(date +%Y%m%d)"
export GENTOO_OVERLAYS="musl"
export GENTOO_PROFILE=default/linux/amd64/17.0/musl/hardened

disk-image-create -a amd64 -t qcow2 --image-size 3 -o gentoo-openstack-amd64-hardened-musl-"${DATE}" gentoo simple-init growroot vm

cd ~/diskimage-builder
git checkout diskimage_builder/elements/gentoo/root.d/10-gentoo-image
pip install -U .
cd ~/

Generic images

The elements I use are for an OpenStack image, meaning there is no default user/pass, those are set by cloud-init / glean. For a generic image you will want the following elements.

'gentoo growroot devuser vm'

The following environment variables are needed as well (changed to match your needs).

DIB_DEV_USER_PASSWORD=supersecrete DIB_DEV_USER_USERNAME=secrete DIB_DEV_USER_PWDLESS_SUDO=yes DIB_DEV_USER_AUTHORIZED_KEYS=/foo/bar/.ssh/authorized_keys

Fin

All this work was done upstream, if you have a question (or feature request) just ask. I'm on irc (Freenode) as prometheanfire or the same nick at gentoo.org for email.

April 17, 2019
Alexys Jacob a.k.a. ultrabug (homepage, bugs)
Meet the py3status logo (April 17, 2019, 08:11 UTC)

I’m proud and very pleased to introduce the py3status logo that Tobaloidee has created for our beloved project!

We’ve been discussing and dreaming about this for a while in the dedicated logo issue. So when Tobaloidee came with his awesome concept and I first saw the logo I was amazed at how he perfectly gave life to the poor brief that I expressed.

Concept

Thanks again Tobaloidee and of course all of the others who participated (with a special mention to @cyrinux’s girlfriend)!

Variants

We have a few other variants that exist, I’m putting some of them here for quick download & use.

April 16, 2019

Nitrokey logo

The Gentoo Foundation has partnered with Nitrokey to equip all Gentoo developers with free Nitrokey Pro 2 devices. Gentoo developers will use the Nitrokey devices to store cryptographic keys for signing of git commits and software packages, GnuPG keys, and SSH accounts.

Thanks to the Gentoo Foundation and Nitrokey’s discount, each Gentoo developer is eligible to receive one free Nitrokey Pro 2. To receive their Nitrokey, developers will need to register with their @gentoo.org email address at the dedicated order form.

A Nitrokey Pro 2 Guide is available on the Gentoo Wiki with FAQ & instructions for integrating Nitrokeys into developer workflow.

ABOUT NITROKEY PRO 2

Nitrokey Pro 2 has strong reliable hardware encryption, thanks to open source. It can help you to: sign Git commits; encrypt emails and files; secure server access; and protect accounts against identity theft via two-factor authentication (one-time passwords).

ABOUT GENTOO

Gentoo Linux is a free, source-based, rolling release meta distribution that features a high degree of flexibility and high performance. It empowers you to make your computer work for you, and offers a variety of choices at all levels of system configuration.

As a community, Gentoo consists of approximately two hundred developers and over fifty thousand users globally.

The Gentoo Foundation supports the development of Gentoo, protects Gentoo’s intellectual property, and oversees adherence to Gentoo’s Social Contract.

ABOUT NITROKEY

Nitrokey is a German IT security startup committed to open source hardware and software. Nitrokey develops and produces USB keys for data encryption, email encryption (PGP/GPG, S/MIME), and secure account logins (SSH, two-factor authentication via OTP and FIDO).

Nitrokey is proud to support the Gentoo Foundation in further securing the Gentoo infrastructure and contributing to a secure open source Linux ecosystem.

April 09, 2019
Luca Barbato a.k.a. lu_zero (homepage, bugs)
Using rav1e – from your code (April 09, 2019, 13:52 UTC)

AV1, Rav1e, Crav1e, an intro

(this article is also available on my dev.to profile, I might use it more often since wordpress is pretty horrible at managing markdown.)

AV1 is a modern video codec brought to you by an alliance of many different bigger and smaller players in the multimedia field.
I’m part of the VideoLan organization and I spent quite a bit of time on this codec lately.


rav1e: The safest and fastest AV1 encoder, built by many volunteers and Mozilla/Xiph developers.
It is written in rust and strives to provide good speed, quality and stay maintainable.


crav1e: A companion crate, written by yours truly, that provides a C-API, so the encoder can be used by C libraries and programs.


This article will just give a quick overview of the API available right now and it is mainly to help people start using it and hopefully report issues and problem.

Rav1e API

The current API is built around the following 4 structs and 1 enum:

  • struct Frame: The raw pixel data
  • struct Packet: The encoded bitstream
  • struct Config: The encoder configuration
  • struct Context: The encoder state

  • enum EncoderStatus: Fatal and non-fatal condition returned by the Contextmethods.

Config

The Config struct currently is simply constructed.

    struct Config {
        enc: EncoderConfig,
        threads: usize,
    }

The EncoderConfig stores all the settings that have an impact to the actual bitstream while settings such as threads are kept outside.

    let mut enc = EncoderConfig::with_speed_preset(speed);
    enc.width = w;
    enc.height = h;
    enc.bit_depth = 8;
    let cfg = Config { enc, threads: 0 };

NOTE: Some of the fields above may be shuffled around until the API is marked as stable.

Methods

new_context
    let cfg = Config { enc, threads: 0 };
    let ctx: Context<u8> = cfg.new_context();

It produces a new encoding context. Where bit_depth is 8, it is possible to use an optimized u8 codepath, otherwise u16 must be used.

Context

It is produced by Config::new_context, its implementation details are hidden.

Methods

The Context can be grouped into essential, optional and convenience.

    // Essential API
    pub fn send_frame<F>(&mut self, frame: F) -> Result<(), EncoderStatus>
      where F: Into<Option<Arc<Frame<T>>>>, T: Pixel;
    pub fn receive_packet(&mut self) -> Result<Packet<T>, EncoderStatus>;

The encoder works by processing each Frame fed through send_frame and producing each Packet that can be retrieved by receive_packet.

    // Optional API
    pub fn container_sequence_header(&mut self) -> Vec<u8>;
    pub fn get_first_pass_data(&self) -> &FirstPassData;

Depending on the container format, the AV1 Sequence Header could be stored in the extradata. container_sequence_header produces the data pre-formatted to be simply stored in mkv or mp4.

rav1e supports multi-pass encoding and the encoding data from the first pass can be retrieved by calling get_first_pass_data.

    // Convenience shortcuts
    pub fn new_frame(&self) -> Arc<Frame<T>>;
    pub fn set_limit(&mut self, limit: u64);
    pub fn flush(&mut self) {
  • new_frame() produces a frame according to the dimension and pixel format information in the Context.
  • flush() is functionally equivalent to call send_frame(None).
  • set_limit()is functionally equivalent to call flush()once limit frames are sent to the encoder.

Workflow

The workflow is the following:

  1. Setup:
    • Prepare a Config
    • Call new_context from the Config to produce a Context
  2. Encode loop:
    • Pull each Packet using receive_packet.
    • If receive_packet returns EncoderStatus::NeedMoreData
      • Feed each Frame to the Context using send_frame
  3. Complete the encoding
    • Issue a flush() to encode each pending Frame in a final Packet.
    • Call receive_packet until EncoderStatus::LimitReached is returned.

Crav1e API

The crav1e API provides the same structures and features beside few key differences:

  • The Frame, Config, and Context structs are opaque.
typedef struct RaConfig RaConfig;
typedef struct RaContext RaContext;
typedef struct RaFrame RaFrame;
  • The Packet struct exposed is much simpler than the rav1e original.
typedef struct {
    const uint8_t *data;
    size_t len;
    uint64_t number;
    RaFrameType frame_type;
} RaPacket;
  • The EncoderStatus includes a Success condition.
typedef enum {
    RA_ENCODER_STATUS_SUCCESS = 0,
    RA_ENCODER_STATUS_NEED_MORE_DATA,
    RA_ENCODER_STATUS_ENOUGH_DATA,
    RA_ENCODER_STATUS_LIMIT_REACHED,
    RA_ENCODER_STATUS_FAILURE = -1,
} RaEncoderStatus;

RaConfig

Since the configuration is opaque there are a number of functions to assemble it:

  • rav1e_config_default allocates a default configuration.
  • rav1e_config_parse and rav1e_config_parse_int set a specific value for a specific field selected by a key string.
  • rav1e_config_set_${field} are specialized setters for complex information such as the color description.
RaConfig *rav1e_config_default(void);

/**
 * Set a configuration parameter using its key and value as string.
 * Available keys and values
 * - "quantizer": 0-255, default 100
 * - "speed": 0-10, default 3
 * - "tune": "psnr"-"psychovisual", default "psnr"
 * Return a negative value on error or 0.
 */
int rav1e_config_parse(RaConfig *cfg, const char *key, const char *value);

/**
 * Set a configuration parameter using its key and value as integer.
 * Available keys and values are the same as rav1e_config_parse()
 * Return a negative value on error or 0.
 */
int rav1e_config_parse_int(RaConfig *cfg, const char *key, int value);

/**
 * Set color properties of the stream.
 * Supported values are defined by the enum types
 * RaMatrixCoefficients, RaColorPrimaries, and RaTransferCharacteristics
 * respectively.
 * Return a negative value on error or 0.
 */
int rav1e_config_set_color_description(RaConfig *cfg,
                                       RaMatrixCoefficients matrix,
                                       RaColorPrimaries primaries,
                                       RaTransferCharacteristics transfer);

/**
 * Set the content light level information for HDR10 streams.
 * Return a negative value on error or 0.
 */
int rav1e_config_set_content_light(RaConfig *cfg,
                                   uint16_t max_content_light_level,
                                   uint16_t max_frame_average_light_level);

/**
 * Set the mastering display information for HDR10 streams.
 * primaries and white_point arguments are RaPoint, containing 0.16 fixed point values.
 * max_luminance is a 24.8 fixed point value.
 * min_luminance is a 18.14 fixed point value.
 * Returns a negative value on error or 0.
 */
int rav1e_config_set_mastering_display(RaConfig *cfg,
                                       RaPoint primaries[3],
                                       RaPoint white_point,
                                       uint32_t max_luminance,
                                       uint32_t min_luminance);

void rav1e_config_unref(RaConfig *cfg);

The bare minimum setup code is the following:

    int ret = -1;
    RaConfig *rac = rav1e_config_default();
    if (!rac) {
        printf("Unable to initialize\n");
        goto clean;
    }

    ret = rav1e_config_parse_int(rac, "width", 64);
    if (ret < 0) {
        printf("Unable to configure width\n");
        goto clean;
    }

    ret = rav1e_config_parse_int(rac, "height", 96);
    if (ret < 0) {
        printf("Unable to configure height\n");
        goto clean;
    }

    ret = rav1e_config_parse_int(rac, "speed", 9);
    if (ret < 0) {
        printf("Unable to configure speed\n");
        goto clean;
    }

RaContext

As per the rav1e API, the context structure is produced from a configuration and the same send-receive model is used.
The convenience methods aren’t exposed and the frame allocation function is actually essential.

// Essential API
RaContext *rav1e_context_new(const RaConfig *cfg);
void rav1e_context_unref(RaContext *ctx);

RaEncoderStatus rav1e_send_frame(RaContext *ctx, const RaFrame *frame);
RaEncoderStatus rav1e_receive_packet(RaContext *ctx, RaPacket **pkt);
// Optional API
uint8_t *rav1e_container_sequence_header(RaContext *ctx, size_t *buf_size);
void rav1e_container_sequence_header_unref(uint8_t *sequence);

RaFrame

Since the frame structure is opaque in C, we have the following functions to create, fill and dispose of the frames.

RaFrame *rav1e_frame_new(const RaContext *ctx);
void rav1e_frame_unref(RaFrame *frame);

/**
 * Fill a frame plane
 * Currently the frame contains 3 planes, the first is luminance followed by
 * chrominance.
 * The data is copied and this function has to be called for each plane.
 * frame: A frame provided by rav1e_frame_new()
 * plane: The index of the plane starting from 0
 * data: The data to be copied
 * data_len: Lenght of the buffer
 * stride: Plane line in bytes, including padding
 * bytewidth: Number of bytes per component, either 1 or 2
 */
void rav1e_frame_fill_plane(RaFrame *frame,
                            int plane,
                            const uint8_t *data,
                            size_t data_len,
                            ptrdiff_t stride,
                            int bytewidth);

RaEncoderStatus

The encoder status enum is returned by the rav1e_send_frame and rav1e_receive_packet and it is possible already to arbitrarily query the context for its status.

RaEncoderStatus rav1e_last_status(const RaContext *ctx);

To simulate the rust Debug functionality a to_str function is provided.

char *rav1e_status_to_str(RaEncoderStatus status);

Workflow

The C API workflow is similar to the Rust one, albeit a little more verbose due to the error checking.

    RaContext *rax = rav1e_context_new(rac);
    if (!rax) {
        printf("Unable to allocate a new context\n");
        goto clean;
    }
    RaFrame *f = rav1e_frame_new(rax);
    if (!f) {
        printf("Unable to allocate a new frame\n");
        goto clean;
    }
while (keep_going(i)){
     RaPacket *p;
     ret = rav1e_receive_packet(rax, &p);
     if (ret < 0) {
         printf("Unable to receive packet %d\n", i);
         goto clean;
     } else if (ret == RA_ENCODER_STATUS_SUCCESS) {
         printf("Packet %"PRIu64"\n", p->number);
         do_something_with(p);
         rav1e_packet_unref(p);
         i++;
     } else if (ret == RA_ENCODER_STATUS_NEED_MORE_DATA) {
         RaFrame *f = get_frame_by_some_mean(rax);
         ret = rav1e_send_frame(rax, f);
         if (ret < 0) {
            printf("Unable to send frame %d\n", i);
            goto clean;
        } else if (ret > 0) {
        // Cannot happen in normal conditions
            printf("Unable to append frame %d to the internal queue\n", i);
            abort();
        }
     } else if (ret == RA_ENCODER_STATUS_LIMIT_REACHED) {
         printf("Limit reached\n");
         break;
     }
}

In closing

This article was mainly a good excuse to try dev.to and see write down some notes and clarify my ideas on what had been done API-wise so far and what I should change and improve.

If you managed to read till here, your feedback is really welcome, please feel free to comment, try the software and open issues for crav1e and rav1e.

Coming next

  • Working crav1e got me to see what’s good and what is lacking in the c-interoperability story of rust, now that this landed I can start crafting and publishing better tools for it and maybe I’ll talk more about it here.
  • Soon rav1e will get more threading-oriented features, some benchmarking experiments will happen soon.

Thanks

  • Special thanks to Derek and Vittorio spent lots of time integrating crav1e in larger software and gave precious feedback in what was missing and broken in the initial iterations.
  • Thanks to David for the review and editorial work.
  • Also thanks to Matteo for introducing me to dev.to.

April 02, 2019
Thomas Raschbacher a.k.a. lordvan (homepage, bugs)

So G+ is closing down today .. I was the "owner" of the "Japanese Learners", "Photowalks Austria" and a few others (barely active) communities there and - after a vote created a MeWe group for it.

I made a MeWe Group for Photowalks Austria The community was Photowalks Austria G+ Community

Just in case someone is searching for it after g+ is gone here the links:

Old G+ community for Japanese Learners

The new Japanese Learners Group on MeWe

The discord server for the Japanse Learners Group

That just leaves to say: Goodbye Google+ it was a fun ride!

March 29, 2019
Alexys Jacob a.k.a. ultrabug (homepage, bugs)

We recently had to face free disk space outages on some of our scylla clusters and we learnt some very interesting things while outlining some improvements that could be made to the ScyllaDB guys.

100% disk space usage?

First of all I wanted to give a bit of a heads up about what happened when some of our scylla nodes reached (almost) 100% disk space usage.

Basically they:

  • stopped listening to client requests
  • complained in the logs
  • wouldn’t flush commitlog (expected)
  • abort their compaction work (which actually gave back a few GB of space)
  • stay in a stuck / unable to stop state (unexpected, this has been reported)

After restarting your scylla server, the first and obvious thing you can try to do to get out of this situation is to run the nodetool clearsnapshot command which will remove any data snapshot that could be lying around. That’s a handy command to reclaim space usually.

Reminder: depending on your compaction strategy, it is usually not advised to allow your data to grow over 50% of disk space...

But that’s only a patch so let’s go down the rabbit hole and look at the optimization options we have.


Optimize your schemas

Schema design and the types your choose for your columns have a huge impact on disk space usage! And in our case we indeed overlooked some of the optimizations that we could have done from the start and that did cost us a lot of wasted disk space. Fortunately it was easy and fast to change.

To illustrate this, I’ll take a sample of 100,000 rows of a simple and naive schema associating readings of 50 integers to a user ID:

Note: all those operations were done using Scylla 3.0.3 on Gentoo Linux.

CREATE TABLE IF NOT EXISTS test.not_optimized
(
uid text,
readings list<int>,
PRIMARY KEY(uid)
) WITH compression = {};

Once inserted on disk, this takes about 250MB of disk space:

250M    not_optimized-00cf1500520b11e9ae38000000000004

Now depending on your use case, if those readings at not meant to be updated for example you could use a frozen list instead, which will allow a huge storage optimization:

CREATE TABLE IF NOT EXISTS test.mid_optimized
(
uid text,
readings frozen<list<int>>,
PRIMARY KEY(uid)
) WITH compression = {};

With this frozen list we now consume 54MB of disk space for the same data!

54M     mid_optimized-011bae60520b11e9ae38000000000004

There’s another optimization that we could do since our user ID are UUIDs. Let’s switch to the uuid type instead of text:

CREATE TABLE IF NOT EXISTS test.optimized
(
uid uuid,
readings frozen<list<int>>,
PRIMARY KEY(uid)
) WITH compression = {};

By switching to uuid, we now consume 50MB of disk space: that’s a 80% reduced disk space consumption compared to the naive schema for the same data!

50M     optimized-01f74150520b11e9ae38000000000004

Enable compression

All those examples were not using compression. If your workload latencies allows it, you should probably enable compression on your sstables.

Let’s see its impact on our tables:

ALTER TABLE test.not_optimized WITH compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'};
ALTER TABLE test.mid_optimized WITH compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'};
ALTER TABLE test.optimized WITH compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'};

Then we run a nodetool compact test to force a (re)compaction of all the sstables and we get:

63M     not_optimized-00cf1500520b11e9ae38000000000004
28M mid_optimized-011bae60520b11e9ae38000000000004
24M optimized-01f74150520b11e9ae38000000000004

Compression is really a great gain here allowing another 50% reduced disk space usage reduction on our optimized table!

Switch to the new “mc” sstable format

Since the Scylla 3.0 release you can use the latest “mc” sstable storage format on your scylla clusters. It promises a greater efficiency for usually a way more reduced disk space consumption!

It is not enabled by default, you have to add the enable_sstables_mc_format: true parameter to your scylla.yaml for it to be taken into account.

Since it’s backward compatible, you have nothing else to do as new compactions will start being made using the “mc” storage format and the scylla server will seamlessly read from old sstables as well.

But in our case of immediate disk space outage, we switched to the new format one node at a time, dropped the data from it and ran a nodetool rebuild to reconstruct the whole node using the new sstable format.

Let’s demonstrate its impact on our test tables: we add the option to the scylla.yaml file, restart scylla-server and run nodetool compact test again:

49M     not_optimized-00cf1500520b11e9ae38000000000004
26M mid_optimized-011bae60520b11e9ae38000000000004
22M optimized-01f74150520b11e9ae38000000000004

That’s a pretty cool gain of disk space, even more for the not optimized version of our schema!

So if you’re in great need of disk space or it is hard for you to change your schemas, switching to the new “mc” sstable format is a simple and efficient way to free up some space without effort.

Consider using secondary indexes

While denormalization is the norm (yep.. legitimate pun) in the NoSQL world this does not mean we have to duplicate everything all the time. A good example lies in the internals of secondary indexes if your workload can compromise with its moderate impact on latency.

Secondary indexes on scylla are built on top of Materialized Views that basically stores an up to date pointer from your indexed column to your main table partition key. That means that secondary indexes MVs are not duplicating all the columns (and thus the data) from your main table as you would have to do when denormalizing a table to query by another column: this saves disk space!

This of course comes with a latency drawback because if your workload is interested in the other columns than the partition key of the main table, the coordinator node will actually issue two queries to get all your data:

  1. query the secondary index MV to get the pointer to the partition key of the main table
  2. query the main table with the partition key to get the rest of the columns you asked for

This has been an effective trick to avoid duplicating a table and save disk space for some of our workloads!

(not a tip) Move the commitlog to another disk / partition?

This should only be considered as a sort of emergency procedure or for cost efficiency (cheap disk tiering) on non critical clusters.

While this is possible even if the disk is not formatted using XFS, it not advised to separate the commitlog from data on modern SSD/NVMe disks but… you technically can do it (as we did) on non production clusters.

Switching is simple, you just need to change the commitlog_directory parameter in your scylla.yaml file.

March 27, 2019
Gentoo GNOME 3.30 for all init systems (March 27, 2019, 00:00 UTC)

GNOME logo

GNOME 3.30 is now available in Gentoo Linux testing branch. Starting with this release, GNOME on Gentoo once again works with OpenRC, in addition to the usual systemd option. This is achieved through the elogind project, a standalone logind implementation based on systemd code, which is currently maintained by a fellow Gentoo user. Gentoo would like to thank Mart Raudsepp (leio), Gavin Ferris, and all others working on this for their contributions. More information can be found in Mart’s blog post.

March 26, 2019
Mart Raudsepp a.k.a. leio (homepage, bugs)
Gentoo GNOME 3.30 for all init systems (March 26, 2019, 16:51 UTC)

GNOME 3.30 is now available in Gentoo Linux testing branch.
Starting with this release, GNOME on Gentoo once again works with OpenRC, in addition to the usual systemd option. This is achieved through the elogind project, a standalone logind implementation based on systemd code, which is currently maintained by a fellow Gentoo user. It provides the missing logind interfaces currently required by GNOME without booting with systemd.

For easier GNOME install, the desktop/gnome profiles now set up default USE flags with elogind for OpenRC systems, while the desktop/gnome/systemd profiles continue to do that for systemd systems. Both have been updated to provide a better initial GNOME install experience. After profile selection, a full install should be simply a matter of `emerge gnome` for testing branch users. Don’t forget to adapt your system to any changed USE flags on previously installed packages too.

GNOME 3.32 is expected to be made available in testing branch soon as well, followed by introducing all this for stable branch users. This is hoped to complete within 6-8 weeks.

If you encounter issues, don’t hesitate to file bug reports or, if necessary, contact me via e-mail or IRC. You can also discuss the elogind aspects on the Gentoo Forums.

Acknowledgements

I’d like to thank Gavin Ferris, for kindly agreeing to sponsor my work on the above (upgrading GNOME on Gentoo from 3.26 to 3.30 and introducing Gentoo GNOME elogind support); and dantrell, for his pioneering overlay work integrating GNOME 3 with OpenRC on Gentoo, and also the GNOME and elogind projects.

March 25, 2019
Alexys Jacob a.k.a. ultrabug (homepage, bugs)
py3status v3.17 (March 25, 2019, 14:12 UTC)

I’m glad to announce a new (awaited) release of py3status featuring support for the sway window manager which allows py3status to enter the wayland environment!

Updated configuration and custom modules paths detection

The configuration section of the documentation explains the updated detection of the py3status configuration file (with respect of XDG_CONFIG environment variables):

  • ~/.config/py3status/config
  • ~/.config/i3status/config
  • ~/.config/i3/i3status.conf
  • ~/.i3status.conf
  • ~/.i3/i3status.conf
  • /etc/xdg/i3status/config
  • /etc/i3status.conf

Regarding custom modules paths detection, py3status does as described in the documentation:

  • ~/.config/py3status/modules
  • ~/.config/i3status/py3status
  • ~/.config/i3/py3status
  • ~/.i3/py3status

Highlights

Lots of modules improvements and clean ups, see changelog.

  • we worked on the documentation sections and content which allowed us to fix a bunch of typos
  • our magic @lasers have worked a lot on harmonizing thresholds on modules along with a lot of code clean ups
  • new module: scroll to scroll modules on your bar (#1748)
  • @lasers has worked a lot on a more granular pango support for modules output (still work to do as it breaks some composites)

Thanks contributors

  • Ajeet D’Souza
  • @boucman
  • Cody Hiar
  • @cyriunx
  • @duffydack
  • @lasers
  • Maxim Baz
  • Thiago Kenji Okada
  • Yaroslav Dronskii

March 20, 2019
Install Gentoo in less than one minute (March 20, 2019, 18:35 UTC)

I’m pretty sure that the title of this post will catch your attention…and/or maybe your curiosity.

Well..this is something I’m doing since years…and since did not cost too much to make it in a public and usable state, I decided to share my work, to help some people to avoid waste of time and to avoid to be angry when your cloud provider does not offer the gentoo image.

So what are the goals of this project?

  1. Install gentoo on cloud providers that do not offer a Gentoo image (e.g Hetzner)
  2. Install gentoo everywhere in few seconds.

To do a fast installation, we need a stage4….but what is exactly a stage4? In this case the stage4 is composed by the official gentoo stage3 plus grub, some more utilities and some file already configured.

So since the stage4 has already everything to complete the installation, we just need to make some replacement (fstab, grub and so on), install grub on the disk………..and…..it’s done (by the auto-installer script)!

At this point I’d expect some people to say….”yeah…it’s so simply and logical…why I didn’t think about that” – Well, I guess that every gentoo user didn’t discover that just after the first installation…so you don’t need to blame yourself 🙂

The technical details are covered by the README in the gentoo-stage4 git repository

As said in the README:

  • If you have any request, feel free to contact me
  • A star on the project will give me the idea of the usage and then the effort to put here.

So what’s more? Just a screenshot of the script in action 🙂

# Gentoo hetzner cloud
# Gentoo stage4
# Gentoo cloud

March 18, 2019
Craig Andrews a.k.a. candrews (homepage, bugs)
Performance Testing WebDAV Clients (March 18, 2019, 13:45 UTC)

Part of migrating applications from on-premises hosting to cloud hosting (AWS, Azure, etc) involves re-evaluating how users access their data. A recent migration involved users running Windows 10 accessing a Windows file share using the SMB protocol. Since SMB isn’t safe to run directly over the Internet (it’s usually not encrypted and it has a long history of security vulnerabilities), potential options included tunneling SMB within a VPN or changing away from the SMB protocol. One such alternative is WebDAV.

Web Distributed Authoring and Versioning (WebDAV) provides the same basic functionality as NFS, SMB, or FTP over the familiar, widely deployed well supported HTTP protocol. It was literally designed for this use case.

Testing Methodology

I evaluated 7 approaches for clients to access a WebDAV share, testing the performance of each and noting pros and cons.

Testing was performed in March 2019. The same client, same server, and same connection was used for each test.

The client is a Dell Latitude 7490 running Fedora 29. Windows 10 (version 1809 build 17763.348) was run as a VM in KVM using user mode networking to connect to the Internet. For the smb+davfs2 testing, smb+davfs2 ran in Docker running in Linux and the Windows 10 VM connected to it.

The WebDAV server is running Apache 2.4.38 and Linux kernel 5.0.0-rc8. The WebDAV site is served over HTTPS.

Ping time between them is 18ms average, httping to the WebDAV server URL is 140ms average.

Windows 10’s Redirector over HTTP/1.1

WebDAV Redirector is the name of the built in WebDAV client in Windows. It’s accessed from Windows Explorer by using the “Map network drive…” option and selecting “Connect to a web site that you can use to store your documents and pictures”

Recent versions of Windows 10 support HTTP/2, so for this test, the “h2” protocol was disabled on the Apache server.

Windows 10’s Redirector over HTTP/2

This test is the same as the previous one except the “h2” protocol is enabled on the Apache server.

WebDrive

WebDrive (version 2019 build 5305) was used with “Basic Cache Settings” set to “Multi-User.” All other settings were left at their defaults.

WebDrive does not currently support TLS 1.3 or HTTP/2; I suspect these would improve performance.

davfs2 on Linux

davfs2 is a Linux file system driver which allows mounting WebDAV shares just like any other file system.

For this testing, the client is Linux, not Windows 10. Therefore, this test isn’t a real solution (the clients do need to run Windows 10). This test is for comparison and testing on component of a possible solution (Samba + davfs).

davfs2 does not support HTTP/2.

davfs2 offer a number of configuration options; here’s what was used for this testing:

/home/candrews/.davfs2/davfs2.conf
[/home/candrews/webdavpoc]
use_locks 1
delay_upload 0 # disable write cache so users know that when the UI tells them the upload is done it’s actually done

And the mount command:

sudo mount -t davfs https://www.integralblue.com/cacdav /home/candrews/webdavpoc/ -ouid=1000,gid=1000,conf=/home/candrews/.davfs2/davfs2.conf

The davfs2 version is 1.5.4.

davfs2 with locks disabled (using a Linux client)

This test is the same as the previous one but this WebDAV locks disabled by setting “use_locks 1” in davfs2.conf.

Samba + davfs2

This test uses Samba to expose the davfs2 mount from the davfs2 test and uses Windows 10 to access the result SMB share.

The davfs2 mount exposed using SMB with the dperson/samba docker image using:

name smb -e USERID=1000 -e GROUPID=1000 --security-opt label:disable -v /home/candrews/webdavpoc:/webdavpoc:Z --rm -it -p 139:139 -p 445:445 -d dperson/samba -s "webdavpoc;/webdavpoc;yes;no;no;testuser" -u "testuser;testpass" -g "vfs objects ="

The biggest challenge with this approach, which I didn’t address, is with regards to security. The davfs2 mount that Samba exposes uses fixed credentials – so all Windows 10 users using the smb+davfs proxy will appear to the WebDAV server to be the same user. There is no way to pass through the credentials from the Windows 10 client through Samba through davfs2 to the WebDAV server. I imagine this limitation may disqualify this solution.

samba + davfs2 with locks disabled

This test is the same as the previous one but WebDAV locks are disabled in davfs2.

Results

Conclusion

Key takeaways from the results:

  • The built in Windows Redirector (which is free and the easiest solution) is by far the slowest.
  • WebDAV benefits greatly from HTTP/2. HTTP/2 was designed to improve the performance of multiple concurrent small requests (as is the case of HTML with references to CSS, Javascript, and images), and that’s exactly how WebDAV works. Each download is a PROPFIND and a GET, and each upload is a PROPFIND, PROPPUT, PUT, and, if locking is enabled, LOCK and UNLOCK.
  • Disabling WebDAV locks improves performance. By eliminating the LOCK/UNLOCK requests for every file, upload performance doubled. Unfortunately, disabling WebDAV locks for the built in Windows Redirector requires a registry change, a non-starter for many organizations. WebDrive has locks disabled by default. davfs2 only uses locks when editing files (when a program actually has the file open for writing), not when uploading a new file.
  • Surprisingly, the overall fastest (by far) approach involves a middle box running Linux using smb+davfs2. Adding a hop and another protocol usually slows things down, but not in this case.
  • davfs2 is fast because it opens more HTTP connections allowing it to do more requests in parallel. WebDrive supports adjusting the concurrency too; in “Connection Settings” there are settings for “Active Connections Limit” (defaults to 4) and “Active Upload Limit” (defaults to 2). Adjusting these settings would impact performance.
  • The client/server connection was high bandwidth (~100mpbs) and low latency (~18ms). Lower bandwidth and high latency would result in even greater benefits from using HTTP/2 and disabling locking.

Considerations for future research / testing include:

  • AWS Storage Gateway may be a solution. Concerns for this approach include:
    • Incompatible with MITM HTTPS-decrypting security devices
    • Uses a local, on premises cache (called an “upload buffer” in the documentation) so changes are not immediately available globally
    • Integrates with Active Directory for authentication/authorization, but setting up for a large number of users is onerous.
    • Requires the use of S3 for file storage, which may not be compatible with the servers in AWS
  • Various other WebDAV clients for Windows, such as DokanCloudFS, NetDrive, DirectNet Drive, ExpanDrive, and Cyberduck / Mountain Duck.
  • Further tweaks to configuration options, such as the concurrency settings in WebDrive and davfs2

March 14, 2019
Thomas Raschbacher a.k.a. lordvan (homepage, bugs)

Update: Still having the issue after some time after all :(

So I have a a (private) nextcloud instance and I kept having errors like "unable to process request,." "Connection to server lost" and it would stop loading at an empty page..

In the logs I had loads of messages like this:

Error PHP mime_content_type(): Failed identify data 0:indirect recursion nesting (0) exceeded at /..../apps/theming/lib/IconBuilder.php#138 2019-03-14T08:08:27+0100
Debug no app in context No cache entry found for /appdata_occ9eea91961/theming/5/favIcon-settings (storage: local::/.../owncloud_data/, internalPath: appdata_occ9eea91961/theming/5/favIcon-settings) 2019-03-14T08:08:27+0100

The first thing I found was reparing the file cache so I ran this:

 php occ maintenance:mimetype:update-db --repair-filecache

That did add about 56 mime types .. ok sounds good .. let's try again .. nope same error (that error btw pops up wherever - in my case also the app update page for example.

so next I check permissions and give my webserver user permissions on both the nextcloud install directory and also the data directory recursively - just to be on the safe side. (chmod -R apache:apache <path>).

Nope still nothing .. getting frustrated here.. - also an update to the latest php 7.1 release (and shared-mime-info package) did nothing either..

Checking for the file it cannot find a cache entry just leads to the discovery, that this file does not exist..

the next thing that i thought of was I could check out themeing so I went to (https://<my-install-webroot>/settings/admin/theming). There I just changed my "Web link" to the actual nextcloud install url (and while i was there also changed the color, name and slogan for fun). after saving this and going back to the app update page it now worked .. so i have no clue what happened .. maybe it was trying to get the favicon from the nextcloud.com page (which is the default "Web link" it seems).

But now it works .. still got those log messages, but they seem to have somehow led me to resolve my issue...

March 07, 2019
Thomas Raschbacher a.k.a. lordvan (homepage, bugs)
egroupware & DB connections (postgresql) (March 07, 2019, 15:14 UTC)

So I recently had this issue with running out of DB connections to my postgresql server.

Now i figured out that egroupware by default has persistent connections enabled (for performance reasons). Which for a small installation with 2 users and a few cal/cardDAV sync devices really is not an issue .. so after checking that egw hat 25+ persistent connections I decided to just disable persistent connections.

The result: less resources consumed on my DB server, free connections for other things that actually use them and absolutely no noticeable issue or slowdown with egroupware.

So my recommendation for small installs of egroupware: do not enable persistent connections.

March 01, 2019
Alexys Jacob a.k.a. ultrabug (homepage, bugs)
Bye bye Google Analytics (March 01, 2019, 12:15 UTC)

A few days ago, I removed Google Analytics from my blog and trashed the associated account.

I’ve been part of the Marketing Tech and Advertising Tech industries for over 15 years. I design and operate data processing platforms (including web navigation trackers) for a living. So I thought that maybe sharing the reasons of why I took this decision might be of interest for some people. I’ll keep it short.

MY convenience is not a enough reason to send YOUR data to Google

The first and obvious question I asked myself is why did I (and so many people) set up this tracker on my web site?

My initial answer was a mix of:

  • convenience : it’s easy to set up, there’s a nice interface, you get a lot of details, you don’t have to ask yourself how it’s done, it just works
  • insight : it sounded somewhat important to know who was visiting what content and somehow know about the interest of people visiting

With also a (hopefully not too much) of:

  • pride: are some blog posts popular? if so which one and let’s try to do more like this!

I don’t think those are good enough reasons to add a tracker that sends YOUR data to Google.

Convenience kills diversity

I’m old enough to have witnessed the rise of internet and its availability to (almost) everyone. The first things I did when I could connect was create and host my own web site, it was great and looked ugly!

But while Internet could have been a catalyst for diversity, it turned out to create an over concentration on services and tools that we think are hard to live without because of their convenience (and a human tendency for mimicry).

When your choices are reduced and the mass adoption defines your standards, it’s easy to let it go and pretend you don’t care that much.

I decided to stop pretending that I don’t care. I don’t want to participate in the concentration of web navigation tracking to Google.

Open Internet is at risk

When diversity is endangered so is Open Internet. This idea that a rich ecosystem can bring their own value and be free to grow by using the data they generate or collect is threatened by the GAFA who are building walled gardens around OUR data.

For instance, Google used the GDPR regulation as an excuse to close down the access to the data collected by their (so convenient) services. If a company (or you) wants to access / query this data (YOUR data) then you can only by using their billed tools.

What should have been only a clear win for us people turned out to also benefit those super blocks and threaten diversity and Open Internet even more.

Adding Google Analytics to your web site helps Google have a larger reach and tracking footprint on the whole web: imagine all those millions of web sites visits added together, that’s where the value is for them. No wonder GA is free.

So in this regard too, I decided to stop contributing to the empowerment of Google.

This blog is Tracking Free

So from now on if you want to share your thoughts of just let me know you enjoyed a post on this blog, take the lead on YOUR data and use the comment box.

The choice is yours!

February 27, 2019
Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)

We're happy to announce that our manuscript "Shaping electron wave functions in a carbon nanotube with a parallel magnetic field" has been published as Editor's Suggestion in Physical Review Letters.

When a physicist thinks of an electron confined to a one-dimensional object such as a carbon nanotube, the first idea that comes to mind is the „particle in a box“ from elementary quantum mechanics. A particle can behave as a wave, and in this model it is essentially a standing wave, reflected at two infinitely high, perfect barrier walls. The mathematical solutions for the wave function describing it are the well-known half-wavelength resonator solutions, with a fundamental mode where exactly half a wavelength fits between the walls, a node of the wave function at each wall and an antinode in the center.

In this publication, we show how wrong this first idea can be, and what impact that has. In a carbon nanotube as quasi one-dimensional system, an electron is not in free space, but confined to the lattice of carbon atoms which forms the nanotube walls. This hexagonal lattice, the same that also forms in planar form graphene, is called bipartite, since every elementary cell of the lattice contains two carbon atoms; one can imagine the nanotube wall as being built out of two sub-lattices, with one atom per cell each, that are shifted relative to each other. Surprisingly, the hexagonal bipartite lattice does not generally support the half-wavelength solutions mentioned above, where the electronic wave function becomes zero at the edges. In each sublattice, we can only force the wave function to zero at one end of the nanotube "box"; its value at the other end remains finite. This means that the wave function shape for each of the two sublattices is more similar to that of a quarter wavelength resonator, where one end displays a node, the other an antinode. The two sublattice wave functions are mirrored in shape to each other, with node and antinode swapping position.

When we now apply a magnetic field along the carbon nanotube, a magnetic flux enters the nanotube, and the boundary conditions for the electron wave function change via the Aharonov-Bohm effect. Astonishingly, its shape along the carbon nanotube can thereby be tuned between half-wavelength and quarter-wavelength behaviour. This means that the probability of the trapped electron to be near the contacts changes, and with it the tunnel current, leading to a very distinct behaviour of the electronic conductance. It turns out that our measurement and the corresponding calculations are agreeing very well. Thus, our work shows the impact of a non-trivial host crystal on the electronic behaviour, important for many novel types of material.

"Shaping electron wave functions in a carbon nanotube with a parallel magnetic field"
M. Marganska, D. R. Schmid, A. Dirnaichner, P. L. Stiller, Ch. Strunk, M. Grifoni, and A. K. Hüttel
Physical Review Letters 122, 086802 (2019), Editor's Suggestion; arXiv:1712.08545 (PDF, supplementary information)

February 25, 2019
Craig Andrews a.k.a. candrews (homepage, bugs)
DNS Over TLS on OpenWrt 18.06 (February 25, 2019, 17:36 UTC)

DNS over TLS encrypts DNS queries so no one between you and the DNS server you’re using (which, by default using these steps, will be Cloudflare’s 1.1.1.1), can tell what DNS queries/responses are being exchanged.

DNS over TLS provides confidentiality but not integrity or authenticity. For those, you need to setup DNSSEC which I’ve described how to do on OpenWrt in another post.

By setting up DNS over TLS on your OpenWrt router, you protect your entire network as all clients will perform DNS requests using your OpenWrt router’s DNS server which in turn will use DNS over TLS to perform the actual resolution.

Setting up DNS over TLS using Stubby on OpenWrt 18.06 is remarkably easy. You can use the LuCI web interface to perform these steps or shell command over ssh; I’m providing the commands here.

  1. Refresh the package list: opkg update
  2. Install the stubby package: opkg install stubby
  3. Start stubby: /etc/init.d/stubby start
  4. Set stubby to start automatically at boot: /etc/init.d/stubby enable
  5. Use stubby as the DNS server by editing /etc/config/dhcp
    In the config dnsmasq section, add (or change the values of, if these settings already exist) these settings:

    • option noresolv '1'
    • list server '127.0.0.1#5453'
  6. Restart dnsmasq so the changes take effect: /etc/init.d/dnsmasq restart

If you’d rather use a different DNS over TLS server than Cloudflare’s 1.1.1.1, edit /etc/stubby/stubby.yml.

Now you can restart assured that your DNS queries can’t be seen by 3rd parties.

February 20, 2019
Michał Górny a.k.a. mgorny (homepage, bugs)

Traditionally, OpenPGP revocation certificates are used as a last resort. You are expected to generate one for your primary key and keep it in a secure location. If you ever lose the secret portion of the key and are unable to revoke it any other way, you import the revocation certificate and submit the updated key to keyservers. However, there is another interesting use for revocation certificates — revoking shared organization keys.

Let’s take Gentoo, for example. We are using a few keys needed to perform automated signatures on servers. For this reason, the key is especially exposed to attacks and we want to be able to revoke it quickly if the need arises. Now, we really do not want to have every single Infra member hold a copy of the secret primary key. However, we can give Infra members revocation certificates instead. This way, they maintain the possibility of revoking the key without unnecessarily increasing its exposure.

The problem with traditional revocation certificates is that they are supported for the purpose of revoking the primary key only. In our security model, the primary key is well protected, compared to subkeys that are totally exposed. Therefore, it is superfluous to revoke the complete key when only a subkey is compromised. To resolve this limitation, gen-revoke tool was created that can create exported revocation signatures for both the primary key and subkeys.

Technical background

The OpenPGP key (v4, as defined by RFC 4880) consists of a primary key, one or more UIDs and zero or more subkeys. Each of those keys and UIDs can include zero or more signature packets. Those packets bind information to the specific key or UID, and their authenticity is confirmed by a signature made using the secret portion of a primary key.

Signatures made by the key’s owner are called self-signatures. The most basic form of them serve as a binding between the primary key and its subkeys and UIDs. Since both those classes of objects are created independently of the primary key, self-signatures are necessary to distinguish authentic subkeys and UIDs created by the key owner from potential fakes. Appropriately, GnuPG will only accept subkeys and UIDs that have valid self-signature.

One specific type of signatures are revocation signatures. Those signatures indicate that the relevant key, subkey or UID has been revoked. If a revocation signature is found, it takes precedence over any other kinds of signatures and prevents the revoked object from being further used.

Key updates are means of distributing new data associated with the key. What’s important is that during an update the key is not replaced by a new one. Instead, GnuPG collects all the new data (subkeys, UIDs, signatures) and adds it to the local copy of the key. The validity of this data is verified against appropriate signatures. Appropriately, anyone can submit a key update to the keyserver, provided that the new data includes valid signatures. Similarly to local GnuPG instance, the keyserver is going to update its copy of the key rather than replacing it.

Revocation certificates specifically make use of this property. Technically, a revocation certificate is simply an exported form of a revocation signature, signed using the owner’s primary key. As long as it’s not on the key (i.e. GnuPG does not see it), it does not do anything. When it’s imported, GnuPG adds it to the key. Further submissions and exports include it, effectively distributing it to all copies of the key.

gen-revoke builds on this idea. It creates and exports revocation signatures for the primary key and subkeys. Due to implementation limitations (and for better compatibility), rather than exporting the signature alone it exports a minimal copy of the relevant key. This copy can be imported just like any other key export, and it causes the revocation signature to be added to the key. Afterwards, it can be exported and distributed just like a revocation done directly on the key.

Usage

To use the script, you need to have the secret portion of the primary key available, and public encryption keys for all the people who are supposed to obtain a copy of the revocation signatures (recipients).

The script takes at least two parameters: an identifier of the key for which revocation signatures should be created, followed by one or more e-mail addresses of signature recipients. It creates revocation signatures both for the primary key and for all valid subkeys, for all the people specified.

The signatures are written into the current directory as key exports and are encrypted to each specified person. They should be distributed afterwards, and kept securely by all the individuals. If a need to revoke either a subkey or the primary key arises, the first person available can decrypt the signature, import it and send the resulting key to keyservers.

Additionally, each signature includes a comment specifying the person it was created for. This comment will afterwards be displayed by GnuPG if one of the revocation signatures is imported. This provides a clear audit trace as to who revoked the key.

Security considerations

Each of the revocation signatures can be used by an attacker to disable the key in question. The signatures are protected through encryption. Therefore, the system is vulnerable to the key of a single signature owner being compromised.

However, this is considerably safer than the equivalent option of distributing the secret portion of the primary key. In the latter case, the attacker would be able to completely compromise the key and use it for malicious purposes; in the former, it is only capable of revoking the key and therefore causing some frustration. Furthermore, the revocation comment helps identifying the compromised user.

The tradeoff between reliability and security can be adjusted by changing the number of revocation signature holders.

January 31, 2019
Michał Górny a.k.a. mgorny (homepage, bugs)

This article describes the UI deficiency of Evolution mail client that extrapolates the trust of one of OpenPGP key UIDs into the key itself, and reports it along with the (potentially untrusted) primary UID. This creates the possibility of tricking the user into trusting a phished mail via adding a forged UID to a key that has a previously trusted UID.

Continue reading

January 29, 2019
Michał Górny a.k.a. mgorny (homepage, bugs)
Identity with OpenPGP trust model (January 29, 2019, 13:50 UTC)

Let’s say you want to send a confidential message to me, and possibly receive a reply. Through employing asymmetric encryption, you can prevent a third party from reading its contents, even if it can intercept the ciphertext. Through signatures, you can verify the authenticity of the message, and therefore detect any possible tampering. But for all this to work, you need to be able to verify the authenticity of the public keys first. In other words, we need to be able to prevent the aforementioned third party — possibly capable of intercepting your communications and publishing a forged key with my credentials on it — from tricking you into using the wrong key.

This renders key authenticity the fundamental problem of asymmetric cryptography. But before we start discussing how key certification is implemented, we need to cover another fundamental issue — identity. After all, who am I — who is the person you are writing to? Are you writing to a person you’ve met? Or to a specific Gentoo developer? Author of some project? Before you can distinguish my authentic key from a forged key, you need to be able to clearly distinguish me from an impostor.

Forms of identity

Identity via e-mail address

If your primary goal is to communicate with the owner of the particular e-mail address, it seems obvious to associate the identity with the owner of the e-mail address. However, how in reality would you distinguish a ‘rightful owner’ of the e-mail address from a cracker who managed to obtain access to it, or to intercept your network communications and inject forged mails?

The truth is, the best you can certify is that the owner of a particular key is able to read and/or send mails from a particular e-mail address, at a particular point in time. Then, if you can certify the same for a long enough period of time, you may reasonably assume the address is continuously used by the same identity (which may qualify as a legitimate owner or a cracker with a lot of patience).

Of course, all this relies on your trust in mail infrastructure not being compromised.

Identity via personal data

A stronger protection against crackers may be provided by associating the identity with personal data, as confirmed by government-issued documents. In case of OpenPGP, this is just the real name; X.509 certificates also provide fields for street address, phone number, etc.

The use of real names seems to be based on two assumptions: that your real name is reasonable well-known (e.g. it can be established with little risk of being replaced by a third party), and that the attacker does not wish to disclose his own name. Besides that, using real names meets with some additional criticism.

Firstly, requiring one to use his real name may be considered an invasion on privacy. Most notably, some people wish not to disclose or use their real names, and this effectively prevents them from ever being certified.

Secondly, real names are not unique. After all, the naming systems developed from the necessity of distinguishing individuals in comparatively small groups, and they simply don’t scale to the size of the Internet. Therefore, name collisions are entirely possible and we are relying on sheer luck that the attacker wouldn’t happen to have the same name as you do.

Thirdly and most importantly, verifying identity documents is non-trivial and untrained individuals are likely to fall victim of mediocre quality fakes. After all, we’re talking about people who hopefully read some article on verifying a particular kind of document but have no experience recognizing forgery, no specialized hardware (I suppose most of you don’t carry a magnifying glass and a UV light on yourself) and who may lack skills in comparing signatures or photographs (not to mention some people have really old photographs in documents). Some countries don’t even issue any official documentation for document verification in English!

Finally, even besides the point of forged documents, this relies on trust in administration.

Identity via photographs

This one I’m mentioning merely for completeness. OpenPGP keys allow adding a photo as one of your UIDs. However, this is rather rarely used (out of the keys my GnuPG fetched so far, less than 10% have photographs). The concerns are similar as for personal data: it assumes that others are reliably able to know how you look like, and that they are capable of reliably comparing faces.

Online identity

An interesting concept is to use your public online activity to prove your identity — such as websites or social media. This is generally based on cross-referencing multiple resources with cryptographically proven publishing access, and assuming that an attacker would not be able to compromise all of them simultaneously.

A form of this concept is utilized by keybase.io. This service builds trust in user profiles via cryptographically cross-linking your profiles on some external sites and/or your websites. Furthermore, it actively encourages other users to verify those external proofs as well.

This identity model entirely relies on trust in network infrastructure and external sites. The likeliness of it being compromised is reduced by (potentially) relying on multiple independent sites.

Web of Trust model

Most of time, you won’t be able to directly verify the identity of everyone you’d like to communicate with. This creates a necessity of obtaining indirect proof of authenticity, and the model normally used for that purpose in OpenPGP is the Web of Trust. I won’t be getting into the fine details — you can find them e.g. in the GNU Privacy Handbook. For our purposes, it suffices to say that in WoT the authenticity of keys you haven’t verified may be assessed by people whose keys you trust already, or people they know, with a limited level of recursion.

The more key holders you can trust, the more keys you can have verified indirectly and the more likely it is that your future recipient will be in that group. Or that you will be able to get someone from across the world into your WoT by meeting someone residing much closer to yourself. Therefore, you’d naturally want the WoT to grow fast and include more individuals. You’d want to preach OpenPGP onto non-crypto-aware people. However, this comes with inherent danger: can you really trust that they will properly verify the identity of the keys they sign?

I believe this is the most fundamental issue with WoT model: for it to work outside of small specialized circles, it has to include more and more individuals across the world. But this growth inevitable makes it easier for a malicious third party to find people that can be tricked into certifying keys with forged identities.

Conclusion

The fundamental problem in OpenPGP usage is finding the correct key and verifying its authenticity. This becomes especially complex given there is no single clear way of determining one’s identity in the Internet. Normally, OpenPGP uses a combination of real name and e-mail address, optionally combined with a photograph. However, all of them have their weaknesses.

Direct identity verification for all recipients is non-practical, and therefore requires indirect certification solutions. While the WoT model used by OpenPGP attempts to avoid centralized trust specific to PKI, it is not clear whether it’s practically manageable. On one hand, it requires trusting more people in order to improve coverage; on the other, it makes it more vulnerable to fraud.

Given all the above, the trust-via-online-presence concept may be of some interest. Most importantly, it establishes a closer relationship between the identity you actually need and the identity you verify — e.g. you want to mail the person being an open source developer, author of some specific projects rather than arbitrary person with a common enough name. However, this concept is not established broadly yet.

January 26, 2019
Michał Górny a.k.a. mgorny (homepage, bugs)

This article shortly explains the historical git weakness regarding handling commits with multiple OpenPGP signatures in git older than v2.20. The method of creating such commits is presented, and the results of using them are described and analyzed.

Continue reading

January 24, 2019
Craig Andrews a.k.a. candrews (homepage, bugs)
WordPress on AWS The Easy Way with VersionPress (January 24, 2019, 18:48 UTC)

Developing and administering WordPress can be painful, especially when requirements include the ability to have a team of developers, the ability for a developer to run the site (including content) on their system (so they can reproduce issues caused by content or configuration), multiple content authors, multiple environments (such as staging and production), and the ability to roll back any problematic changes (content, code, or configuration) with ease.

To solve this problem, I developed a solution to make things easier: VersionPress on AWS. It uses Docker to allow developers to run the site locally and an assortment of AWS services to host the site. Check it out and I think you’ll find that life becomes a little bit easier.

VersionPress on AWS is Free Software under the GPLv3 hosted at gitlab. Contribution in the forms of issue reports and pull request are very much welcome.

What is VersionPress?

VersionPress stores content (posts, media, etc) as well as code (themes, plugins, configuration, etc) in source control (git).

  • By looking at the git log, it’s quick and easy to see who changed what and when.
  • All code (plugins, themes, WordPress core itself) and content (pages, posts, comments, configuration) are stored in git. This approach allows content changes as well as code changes can be reverted if there’s a problem and merged between branches for different environments.
  • Wipe out and recreate the environment at any time without data loss – everything is in git. No need to worry about the AWS RDS server. Migrate between RDS for MariaDB and Aurora at any time.
  • Need a staging site, or a new site to test work in progress? Create a new branch and launch a new stack, be up and running in minutes
  • Run the exact same site with the same content locally so you can reproduce issues in production effortlessly – no more “works on my machine” situations

Hosting with AWS

Need a small, cheap staging site, but also a full fledged scalable production site with a CDN? Use the same stack for both – simply specify different parameter values. Change parameter values whenever you want without downtime or data loss. For example, when starting out, leave the CloudFront CDN off to save money. When the site becomes popular, add the CloudFront CDN to better handle the load and improve performance for end users.

AWS features leveraged include:

Docker

Docker is used to run WordPress in AWS Elastic Beanstalk as well as for developers running the site locally. This consistency reduces the occurrences of “it works on my machine” situations and gets new developers on-boarded quicker.

When not to use VersionPress on AWS

Since VersionPress commits all content changes to git, content changes are a bit slower. Therefore, if the site is very content change heavy, such as if it’s a forum with many frequent comments being made, VersionPress on AWS may not be the right solution.

However, the vast majority of WordPress sites have very infrequent content changes, so the slightly slower writes are rarely an issue.

Get Started

Check out the VersionPress on AWS documentation to get started.

January 20, 2019
Alexys Jacob a.k.a. ultrabug (homepage, bugs)
py3status v3.16 (January 20, 2019, 21:10 UTC)

Two py3status versions in less than a month? That’s the holidays effect but not only!

Our community has been busy discussing our way forward to 4.0 (see below) and organization so it was time I wrote a bit about that.

Community

A new collaborator

First of all we have the great pleasure and honor to welcome Maxim Baz @maximbaz as a new collaborator on the project!

His engagement, numerous contributions and insightful reviews to py3status has made him a well known community member, not to mention his IRC support 🙂

Once again, thank you for being there Maxim!

Zen of py3status

As a result of an interesting discussion, we worked on defining better how to contribute to py3status as well as a set of guidelines we agree on to get the project moving on smoothly.

Here is born the zen of py3status which extends the philosophy from the user point of view to the contributor point of view!

This allowed us to handle the numerous open pull requests and get their number down to 5 at the time of writing this post!

Even our dear @lasers don’t have any open PR anymore 🙂

3.15 + 3.16 versions

Our magic @lasers has worked a lot on general modules options as well as adding support for i3-gaps added features such as border coloring and fine tuning.

Also interesting is the work of Thiago Kenji Okada @m45t3r around NixOS packaging of py3status. Thanks a lot for this work and for sharing Thiago!

I also liked the question of Andreas Lundblad @aioobe asking if we could have a feature allowing to display a custom graphical output, such as a small PNG or anything upon clicking on the i3bar, you might be interested in following up the i3 issue he opened.

Make sure to read the amazing changelog for details, a lot of modules have been enhanced!

Highlights

  • You can now set a background, border colors and their urgent counterparts on a global scale or per module
  • CI now checks for black format on modules, so now all the code base obey the black format style!
  • All HTTP requests based modules now have a standard way to define HTTP timeout as well as the same 10 seconds default timeout
  • py3-cmd now allows sending click events with modifiers
  • The py3status -n / –interval command line argument has been removed as it was obsolete. We will ignore it if you have set it up, but better remove it to be clean
  • You can specify your own i3status binary path using the new -u, –i3status command line argument thanks to @Dettorer and @lasers
  • Since Yahoo! decided to retire its public & free weather API, the weather_yahoo module has been removed

New modules

  • new conky module: display conky system monitoring (#1664), by lasers
  • new module emerge_status: display information about running gentoo emerge (#1275), by AnwariasEu
  • new module hueshift: change your screen color temperature (#1142), by lasers
  • new module mega_sync: to check for MEGA service synchronization (#1458), by Maxim Baz
  • new module speedtest: to check your internet bandwidth (#1435), by cyrinux
  • new module usbguard: control usbguard from your bar (#1376), by cyrinux
  • new module velib_metropole: display velib metropole stations and (e)bikes (#1515), by cyrinux

A word on 4.0

Do you wonder what’s gonna be in the 4.0 release?
Do you have ideas that you’d like to share?
Do you have dreams that you’d love to become true?

Then make sure to read and participate in the open RFC on 4.0 version!

Development has not started yet; we really want to hear from you.

Thank you contributors!

There would be no py3status release without our amazing contributors, so thank you guys!

  • AnwariasEu
  • cyrinux
  • Dettorer
  • ecks
  • flyingapfopenguin
  • girst
  • Jack Doan
  • justin j lin
  • Keith Hughitt
  • L0ric0
  • lasers
  • Maxim Baz
  • oceyral
  • Simon Legner
  • sridhars
  • Thiago Kenji Okada
  • Thomas F. Duellmann
  • Till Backhaus

January 13, 2019
Alice Ferrazzi a.k.a. alicef (homepage, bugs)
Getting control on my data (January 13, 2019, 19:28 UTC)

Since 2009 I started to get always more interested on privacy,
radical decentralization and self-hosting.
But only recently I started to actively work on keeping
my own privacy and making more strict my open source usage
(no dropbox, no google services).
The point of this more radical change is not only privacy.
The point is partially because I don't want that corporation1 use
my data for their business and partially because I think
that open source and decentralization is the way to go.
Not using open source is giving corporations the ability to automate us27.
Continuing to use a central controlled entity is giving away our
freedom and our privacy (yes, also in our life).
Corporation that have many users and that are dominating our communication services
they can control every aspect of our life.
They can remove and censor whatever content is against their view,
adding crazily expensive service features and owning your data.
I prefer to use good open source, taking control back over my data
and be sure that they are contributing back to the ecosystem.
Taking control back over my freedom and having the possibility to
contribute back and helping out.
I prefer to donate to a service that is giving users freedom than
giving money to a service that is removing user rights.

dontspyonme image from: https://www.reddit.com/r/degoogle/comments/8bzumr/dont_spy_on_me/

Unfortunatly, also, my server hosting my irc znc bouncer and my previous
website started to get too full for what I wanted to do,
so I had to get a new VPS for hosting my services and I'm
using the old one for keeping just my email server.
Before moving out I also had a Google account that was already asking money for
keeping my google email account space (I would have to pay google for doing
data analysis on my email...).

So I decided to quit.
Quitting facebook5, google6 and dropbox18.
Probably also quitting twitter in the near future.

I started setting my servers but I wanted something simple to setup
and that I could easily move away, if I have any kind of problem
(like moving to a different server or just keeping simple data backup).

As now I'm heavily relying on docker.
I changed my google email to mailcow17, and having control on the own mail service is
a really good experience.
Mailcow is using only open source, like SoGo that is also really easy to use,
and offer the possibility to make mail filters similar to google mail.
The move to mailcow was straightforward but I still need to finish to move
all my mail to the new accout.
Moved away from Google drive and dropbox for nextcloud + collabora online (stable libreoffice online)7.
Installed back znc and quassel core for my irc bouncer.
I used grammar for sometime in the browser and now I'm using language-tool 9
with own docker server.
I stopped searching videos on youtube and just using peertube10.
I'm still unfortunatly using twitter but I opened a account on mastodon11 (fosstodon),
I could talk with the owner and looks a reasonable setup.
Google search became searx12 + yacy13.
Android became lineage os19 + fdroid20 + aurora store28 (unfortuantly not all the application that I need are Open Source). Also my password as been moved away from lastpass to bitwarden21
keepassxc22 and pass23.
The feeling got by selfhosting most service that I use,
is definetly, as Douglas Rushkoff (team human24) would say, more human.
Is less internet of the corporations and feels more like what internet need to be,
something that is managed and owned by human not by algorithms trying to
use your data for making growth.

privacytools Nice inspiration for quitting was privacytools.io 1

Read also:
Nothing to hide argument (Wikipedia)2
How do you counter the "I have nothing to hide?" argument? (reddit.com)3
'I've Got Nothing to Hide' and Other Misunderstandings of Privacy (Daniel J. Solove - San Diego Law Review)4
Richard Stallman on privacy16

I also moved from ikiwiki to pelican, but this was more a personal preference, ikiwiki is really good but pelican25 is more simple for me to customize as is made in python.
I also went back to Qtile26 from i3.

so enjoy my new blogs and my new rants :)

January 12, 2019
Alice Ferrazzi a.k.a. alicef (homepage, bugs)
My First Review (January 12, 2019, 18:06 UTC)

This was a test post but I think it can became a real post.
The test site post was this:
"Following is a review of my favorite mechanical keyboard."

I actually recently bought a new keyboard.
Usually I use a Happy Hacking Keyboard professional 2 english layout, I have two of them.
HHB_work HHKB_home

I really like my HHKB and I have no drow back on using it.
Both keyboard are modded with the hasu TMK controller.
The firmware is formatted with a colemak layout.

But recently I see the advertisment of the ulitmate hacking
keyboard.
Is interesting that is made by a crowd founding and that looks
heavily customizable.
Was looking pretty solid, so I bought one.
Here it is:
HHKB_work

Having the key mark on the key, as a colemak user, was
enough useless.
But I had no problem remapping the firmware for following
the colemak layout.