Gentoo Logo
Gentoo Logo Side
Gentoo Spaceship

Contributors:
. Aaron W. Swenson
. Agostino Sarubbo
. Alexey Shvetsov
. Alexis Ballier
. Alexys Jacob
. Alice Ferrazzi
. Alice Ferrazzi
. Andreas K. Hüttel
. Anthony Basile
. Arun Raghavan
. Bernard Cafarelli
. Brian Harring
. Christian Ruppert
. Chí-Thanh Christopher Nguyễn
. Denis Dupeyron
. Detlev Casanova
. Diego E. Pettenò
. Domen Kožar
. Doug Goldstein
. Eray Aslan
. Fabio Erculiani
. Gentoo Haskell Herd
. Gentoo Miniconf 2016
. Gentoo Monthly Newsletter
. Gentoo News
. Gilles Dartiguelongue
. Greg KH
. Göktürk Yüksek
. Hanno Böck
. Hans de Graaff
. Ian Whyman
. Jan Kundrát
. Jason A. Donenfeld
. Jeffrey Gardner
. Joachim Bartosik
. Johannes Huber
. Jonathan Callen
. Jorge Manuel B. S. Vicetto
. Kristian Fiskerstrand
. Lance Albertson
. Liam McLoughlin
. Luca Barbato
. Marek Szuba
. Mart Raudsepp
. Matt Turner
. Matthew Thode
. Michael Palimaka
. Michal Hrusecky
. Michał Górny
. Mike Doty
. Mike Gilbert
. Mike Pagano
. Nathan Zachary
. Pacho Ramos
. Patrick Kursawe
. Patrick Lauer
. Patrick McLean
. Paweł Hajdan, Jr.
. Petteri Räty
. Piotr Jaroszyński
. Rafael G. Martins
. Remi Cardona
. Richard Freeman
. Robin Johnson
. Ryan Hill
. Sean Amoss
. Sebastian Pipping
. Steev Klimaszewski
. Stratos Psomadakis
. Sven Vermeulen
. Sven Wegener
. Tom Wijsman
. Tomáš Chvátal
. Yury German
. Zack Medico

Last updated:
October 17, 2017, 03:05 UTC

Disclaimer:
Views expressed in the content published here do not necessarily represent the views of Gentoo Linux or the Gentoo Foundation.


Bugs? Comments? Suggestions? Contact us!

Powered by:
Planet Venus

Welcome to Gentoo Universe, an aggregation of weblog articles on all topics written by Gentoo developers. For a more refined aggregation of Gentoo-related topics only, you might be interested in Planet Gentoo.

October 16, 2017
Greg KH a.k.a. gregkh (homepage, bugs)
Linux Kernel Community Enforcement Statement FAQ (October 16, 2017, 09:05 UTC)

Based on the recent Linux Kernel Community Enforcement Statement and the article describing the background and what it means , here are some Questions/Answers to help clear things up. These are based on questions that came up when the statement was discussed among the initial round of over 200 different kernel developers.

Q: Is this changing the license of the kernel?

A: No.

Q: Seriously? It really looks like a change to the license.

A: No, the license of the kernel is still GPLv2, as before. The kernel developers are providing certain additional promises that they encourage users and adopters to rely on. And by having a specific acking process it is clear that those who ack are making commitments personally (and perhaps, if authorized, on behalf of the companies that employ them). There is nothing that says those commitments are somehow binding on anyone else. This is exactly what we have done in the past when some but not all kernel developers signed off on the driver statement.

Q: Ok, but why have this “additional permissions” document?

A: In order to help address problems caused by current and potential future copyright “trolls” aka monetizers.

Q: Ok, but how will this help address the “troll” problem?

A: “Copyright trolls” use the GPL-2.0’s immediate termination and the threat of an immediate injunction to turn an alleged compliance concern into a contract claim that gives the troll an automatic claim for money damages. The article by Heather Meeker describes this quite well, please refer to that for more details. If even a short delay is inserted for coming into compliance, that delay disrupts this expedited legal process.

By simply saying, “We think you should have 30 days to come into compliance”, we undermine that “immediacy” which supports the request to the court for an immediate injunction. The threat of an immediate junction was used to get the companies to sign contracts. Then the troll goes back after the same company for another known violation shortly after and claims they’re owed the financial penalty for breaking the contract. Signing contracts to pay damages to financially enrich one individual is completely at odds with our community’s enforcement goals.

We are showing that the community is not out for financial gain when it comes to license issues – though we do care about the company coming into compliance.  All we want is the modifications to our code to be released back to the public, and for the developers who created that code to become part of our community so that we can continue to create the best software that works well for everyone.

This is all still entirely focused on bringing the users into compliance. The 30 days can be used productively to determine exactly what is wrong, and how to resolve it.

Q: Ok, but why are we referencing GPL-3.0?

A: By using the terms from the GPLv3 for this, we use a very well-vetted and understood procedure for granting the opportunity to come fix the failure and come into compliance. We benefit from many months of work to reach agreement on a termination provision that worked in legal systems all around the world and was entirely consistent with Free Software principles.

Q: But what is the point of the “non-defensive assertion of rights” disclaimer?

A: If a copyright holder is attacked, we don’t want or need to require that copyright holder to give the party suing them an opportunity to cure. The “non-defensive assertion of rights” is just a way to leave everything unchanged for a copyright holder that gets sued.  This is no different a position than what they had before this statement.

Q: So you are ok with using Linux as a defensive copyright method?

A: There is a current copyright troll problem that is undermining confidence in our community – where a “bad actor” is attacking companies in a way to achieve personal gain. We are addressing that issue. No one has asked us to make changes to address other litigation.

Q: Ok, this document sounds like it was written by a bunch of big companies, who is behind the drafting of it and how did it all happen?

A: Grant Likely, the chairman at the time of the Linux Foundation’s Technical Advisory Board (TAB), wrote the first draft of this document when the first copyright troll issue happened a few years ago. He did this as numerous companies and developers approached the TAB asking that the Linux kernel community do something about this new attack on our community. He showed the document to a lot of kernel developers and a few company representatives in order to get feedback on how it should be worded. After the troll seemed to go away, this work got put on the back-burner. When the copyright troll showed back up, along with a few other “copycat” like individuals, the work on the document was started back up by Chris Mason, the current chairman of the TAB. He worked with the TAB members, other kernel developers, lawyers who have been trying to defend these claims in Germany, and the TAB members’ Linux Foundation’s lawyers, in order to rework the document so that it would actually achieve the intended benefits and be useful in stopping these new attacks. The document was then reviewed and revised with input from Linus Torvalds and finally a document that the TAB agreed would be sufficient was finished. That document was then sent to over 200 of the most active kernel developers for the past year by Greg Kroah-Hartman to see if they, or their company, wished to support the document. That produced the initial “signatures” on the document, and the acks of the patch that added it to the Linux kernel source tree.

Q: How do I add my name to the document?

A: If you are a developer of the Linux kernel, simply send Greg a patch adding your name to the proper location in the document (sorting the names by last name), and he will be glad to accept it.

Q: How can my company show its support of this document?

A: If you are a developer working for a company that wishes to show that they also agree with this document, have the developer put the company name in ‘(’ ‘)’ after the developer’s name. This shows that both the developer, and the company behind the developer are in agreement with this statement.

Q: How can a company or individual that is not part of the Linux kernel community show its support of the document?

A: Become part of our community! Send us patches, surely there is something that you want to see changed in the kernel. If not, wonderful, post something on your company web site, or personal blog in support of this statement, we don’t mind that at all.

Q: I’ve been approached by a copyright troll for Netfilter. What should I do?

A: Please see the Netfilter FAQ here for how to handle this

Q: I have another question, how do I ask it?

A: Email Greg or the TAB, and they will be glad to help answer them.

Linux Kernel Community Enforcement Statement (October 16, 2017, 09:00 UTC)

By Greg Kroah-Hartman, Chris Mason, Rik van Riel, and Shuah Khan

The Linux kernel ecosystem of developers, companies and users has been wildly successful by any measure over the last couple decades. Even today, 26 years after the initial creation of the Linux kernel, the kernel developer community continues to grow, with more than 500 different companies and over 4,000 different developers getting changes merged into the tree during the past year. As Greg always says every year, the kernel continues to change faster this year than the last, this year we were running around 8.5 changes an hour, with 10,000 lines of code added, 2,000 modified, and 2,500 lines removed every hour of every day.

The stunning growth and widespread adoption of Linux, however, also requires ever evolving methods of achieving compliance with the terms of our community’s chosen license, the GPL-2.0. At this point, there is no lack of clarity on the base compliance expectations of our community. Our goals as an ecosystem are to make sure new participants are made aware of those expectations and the materials available to assist them, and to help them grow into our community.  Some of us spend a lot of time traveling to different companies all around the world doing this, and lots of other people and groups have been working tirelessly to create practical guides for everyone to learn how to use Linux in a way that is compliant with the license. Some of these activities include:

Unfortunately the same processes that we use to assure fulfillment of license obligations and availability of source code can also be used unjustly in trolling activities to extract personal monetary rewards. In particular, issues have arisen as a developer from the Netfilter community, Patrick McHardy, has sought to enforce his copyright claims in secret and for large sums of money by threatening or engaging in litigation. Some of his compliance claims are issues that should and could easily be resolved. However, he has also made claims based on ambiguities in the GPL-2.0 that no one in our community has ever considered part of compliance.  

Examples of these claims have been distributing over-the-air firmware, requiring a cell phone maker to deliver a paper copy of source code offer letter; claiming the source code server must be setup with a download speed as fast as the binary server based on the “equivalent access” language of Section 3; requiring the GPL-2.0 to be delivered in a local language; and many others.

How he goes about this activity was recently documented very well by Heather Meeker.

Numerous active contributors to the kernel community have tried to reach out to Patrick to have a discussion about his activities, to no response. Further, the Netfilter community suspended Patrick from contributing for violations of their principles of enforcement. The Netfilter community also published their own FAQ on this matter.

While the kernel community has always supported enforcement efforts to bring companies into compliance, we have never even considered enforcement for the purpose of extracting monetary gain.  It is not possible to know an exact figure due to the secrecy of Patrick’s actions, but we are aware of activity that has resulted in payments of at least a few million Euros.  We are also aware that these actions, which have continued for at least four years, have threatened the confidence in our ecosystem.

Because of this, and to help clarify what the majority of Linux kernel community members feel is the correct way to enforce our license, the Technical Advisory Board of the Linux Foundation has worked together with lawyers in our community, individual developers, and many companies that participate in the development of, and rely on Linux, to draft a Kernel Enforcement Statement to help address both this specific issue we are facing today, and to help prevent any future issues like this from happening again.

A key goal of all enforcement of the GPL-2.0 license has and continues to be bringing companies into compliance with the terms of the license. The Kernel Enforcement Statement is designed to do just that.  It adopts the same termination provisions we are all familiar with from GPL-3.0 as an Additional Permission giving companies confidence that they will have time to come into compliance if a failure is identified. Their ability to rely on this Additional Permission will hopefully re-establish user confidence and help direct enforcement activity back to the original purpose we have all sought over the years – actual compliance.  

Kernel developers in our ecosystem may put their own acknowledgement to the Statement by sending a patch to Greg adding their name to the Statement, like any other kernel patch submission, and it will be gladly merged. Those authorized to ‘ack’ on behalf of their company may add their company name in (parenthesis) after their name as well.

Note, a number of questions did come up when this was discussed with the kernel developer community. Please see Greg’s FAQ post answering the most common ones if you have further questions about this topic.

October 13, 2017
Alexys Jacob a.k.a. ultrabug (homepage, bugs)
Gentoo Linux listed RethinkDB’s website (October 13, 2017, 08:22 UTC)

 

The rethinkdb‘s website has (finally) been updated and Gentoo Linux is now listed on the installation page!

Meanwhile, we have bumped the ebuild to version 2.3.6 with fixes for building on gcc-6 thanks to Peter Levine who kindly proposed a nice PR on github.

October 03, 2017

Description:
binutils is a set of tools necessary to build programs.

The complete ASan output of the issue:

# nm -A -a -l -S -s --special-syms --synthetic --with-symbol-versions -D $FILE
==26890==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6130000006d3 at pc 0x000000472115 bp 0x7ffdb7d8a0d0 sp 0x7ffdb7d89880                                                                         
READ of size 298 at 0x6130000006d3 thread T0                                                                                                                                                                      
    #0 0x472114 in __interceptor_strlen /var/tmp/portage/sys-libs/compiler-rt-sanitizers-5.0.0/work/compiler-rt-5.0.0.src/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:302                      
    #1 0x68fea5 in parse_die /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf1.c:254:12                                                                                                           
    #2 0x68ddda in _bfd_dwarf1_find_nearest_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf1.c:521:13                                                                                       
    #3 0x5f2f00 in _bfd_elf_find_nearest_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/elf.c:8659:10                                                                                            
    #4 0x517755 in print_symbol /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1004:12                                                                                                      
    #5 0x514e4d in print_symbols /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1084:7                                                                                                      
    #6 0x514e4d in display_rel_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1200                                                                                                     
    #7 0x510976 in display_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1318:7                                                                                                       
    #8 0x50f4ce in main /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1792:12                                                                                                              
    #9 0x7f3dea34e680 in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.23-r4/work/glibc-2.23/csu/../csu/libc-start.c:289                                                                                    
    #10 0x41a638 in chmod (/usr/x86_64-pc-linux-gnu/binutils-bin/git/nm+0x41a638)                                                                                                                                 
                                                                                                                                                                                                                  
0x6130000006d3 is located 0 bytes to the right of 339-byte region [0x613000000580,0x6130000006d3)                                                                                                                 
allocated by thread T0 here:                                                                                                                                                                                      
    #0 0x4d8828 in malloc /var/tmp/portage/sys-libs/compiler-rt-sanitizers-5.0.0/work/compiler-rt-5.0.0.src/lib/asan/asan_malloc_linux.cc:67                                                                      
    #1 0x53f138 in bfd_malloc /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/libbfd.c:193:9
    #2 0x799bc8 in bfd_get_full_section_contents /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/compress.c:248:21
    #3 0x7b8797 in bfd_simple_get_relocated_section_contents /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/simple.c:193:12
    #4 0x68e3b1 in _bfd_dwarf1_find_nearest_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf1.c:490:4
    #5 0x5f2f00 in _bfd_elf_find_nearest_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/elf.c:8659:10
    #6 0x517755 in print_symbol /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1004:12
    #7 0x514e4d in print_symbols /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1084:7
    #8 0x514e4d in display_rel_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1200
    #9 0x510976 in display_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1318:7
    #10 0x50f4ce in main /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1792:12
    #11 0x7f3dea34e680 in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.23-r4/work/glibc-2.23/csu/../csu/libc-start.c:289

SUMMARY: AddressSanitizer: heap-buffer-overflow /var/tmp/portage/sys-libs/compiler-rt-sanitizers-5.0.0/work/compiler-rt-5.0.0.src/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:302 in __interceptor_strlen
Shadow bytes around the buggy address:
  0x0c267fff8080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c267fff8090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c267fff80a0: 00 00 00 04 fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c267fff80b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c267fff80c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c267fff80d0: 00 00 00 00 00 00 00 00 00 00[03]fa fa fa fa fa
  0x0c267fff80e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c267fff80f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c267fff8100: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c267fff8110: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c267fff8120: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==26890==ABORTING

Affected version:
2.29.51.20170924 and maybe past releases

Fixed version:
N/A

Commit fix:
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=1da5c9a485f3dcac4c45e96ef4b7dae5948314b5

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
CVE-2017-15020

Reproducer:
https://github.com/asarubbo/poc/blob/master/00376-binutils-heapoverflow-parse_die

Timeline:
2017-09-25: bug discovered and reported to upstream
2017-09-25: upstream released a patch
2017-10-03: blog post about the issue
2017-10-04: CVE assigned

Note:
This bug was found with American Fuzzy Lop.
This bug was identified with bare metal servers donated by Packet. This work is also supported by the Core
Infrastructure Initiative
.

Permalink:

binutils: heap-based buffer overflow in parse_die (dwarf1.c)

Description:
binutils is a set of tools necessary to build programs.

The stacktrace of this issue appears to be a NULL pointer access. However the upstream maintainer changed the summary of the bugreport to “DW_AT_name with out of bounds reference”. The commit also reference to “DW_AT_name with out of bounds reference”

The complete ASan output of the issue:

# nm -A -a -l -S -s --special-syms --synthetic --with-symbol-versions -D $FILE
==8739==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x00000053bf16 bp 0x7ffcab59ee60 sp 0x7ffcab59ee20 T0)
==8739==The signal is caused by a READ memory access.
==8739==Hint: address points to the zero page.
    #0 0x53bf15 in bfd_hash_hash /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/hash.c:441:15
    #1 0x53bf15 in bfd_hash_lookup /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/hash.c:467
    #2 0x6a2049 in insert_info_hash_table /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:487:37
    #3 0x6a2049 in comp_unit_hash_info /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:3776
    #4 0x6a2049 in stash_maybe_update_info_hash_tables /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:4120
    #5 0x69cbbc in stash_maybe_enable_info_hash_tables /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:4214:3
    #6 0x69cbbc in _bfd_dwarf2_find_nearest_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:4613
    #7 0x5f330e in _bfd_elf_find_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/elf.c:8695:10
    #8 0x5176a3 in print_symbol /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1003:9
    #9 0x514e4d in print_symbols /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1084:7
    #10 0x514e4d in display_rel_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1200
    #11 0x510976 in display_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1318:7
    #12 0x50f4ce in main /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1792:12
    #13 0x7fd148c7b680 in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.23-r4/work/glibc-2.23/csu/../csu/libc-start.c:289
    #14 0x41a638 in chmod (/usr/x86_64-pc-linux-gnu/binutils-bin/git/nm+0x41a638)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/hash.c:441:15 in bfd_hash_hash
==8739==ABORTING

Affected version:
2.29.51.20170924 and maybe past releases

Fixed version:
N/A

Commit fix:
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=11855d8a1f11b102a702ab76e95b22082cccf2f8

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
CVE-2017-15022

Reproducer:
https://github.com/asarubbo/poc/blob/master/00375-binutils-NULLptr-bfd_hash_hash

Timeline:
2017-09-25: bug discovered and reported to upstream
2017-09-25: upstream released a patch
2017-10-03: blog post about the issue
2017-10-04: CVE assigned

Note:
This bug was found with American Fuzzy Lop.
This bug was identified with bare metal servers donated by Packet. This work is also supported by the Core
Infrastructure Initiative
.

Permalink:

binutils: NULL pointer dereference in bfd_hash_hash (hash.c)

Description:
binutils is a set of tools necessary to build programs.

The complete ASan output of the issue:

# nm -A -a -l -S -s --special-syms --synthetic --with-symbol-versions -D $FILE
==3765==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x0000006a7376 bp 0x7ffd5f9a3d50 sp 0x7ffd5f9a3d20 T0)
==3765==The signal is caused by a READ memory access.
==3765==Hint: address points to the zero page.
    #0 0x6a7375 in concat_filename /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:1601:8
    #1 0x696e83 in decode_line_info /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:2258:44
    #2 0x6a2ab8 in comp_unit_maybe_decode_line_info /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:3642:26
    #3 0x6a2ab8 in comp_unit_find_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:3677
    #4 0x6a0104 in _bfd_dwarf2_find_nearest_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:4789:11
    #5 0x5f330e in _bfd_elf_find_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/elf.c:8695:10
    #6 0x5176a3 in print_symbol /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1003:9
    #7 0x514e4d in print_symbols /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1084:7
    #8 0x514e4d in display_rel_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1200
    #9 0x510976 in display_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1318:7
    #10 0x50f4ce in main /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1792:12
    #11 0x7f0f4a74b680 in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.23-r4/work/glibc-2.23/csu/../csu/libc-start.c:289
    #12 0x41a638 in chmod (/usr/x86_64-pc-linux-gnu/binutils-bin/git/nm+0x41a638)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:1601:8 in concat_filename
==3765==ABORTING

Affected version:
2.29.51.20170924 and maybe past releases

Fixed version:
N/A

Commit fix:
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=c361faae8d964db951b7100cada4dcdc983df1bf

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
CVE-2017-15023

Reproducer:
https://github.com/asarubbo/poc/blob/master/00374-binutils-NULLptr-concat_filename

Timeline:
2017-09-25: bug discovered and reported to upstream
2017-09-25: upstream released a patch
2017-10-03: blog post about the issue
2017-10-04: CVE assigned

Note:
This bug was found with American Fuzzy Lop.
This bug was identified with bare metal servers donated by Packet. This work is also supported by the Core
Infrastructure Initiative
.

Permalink:

binutils: NULL pointer dereference in concat_filename (dwarf2.c)

Description:
binutils is a set of tools necessary to build programs.

The complete ASan output of the issue:

# nm -A -a -l -S -s --special-syms --synthetic --with-symbol-versions -D $FILE
==11994==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60200000029e at pc 0x7f800af7095d bp 0x7ffeab0e5c90 sp 0x7ffeab0e5c88            
READ of size 1 at 0x60200000029e thread T0                                                                                                           
    #0 0x7f800af7095c in bfd_getl32 /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/libbfd.c:559:24                                       
    #1 0x7f800af91323 in bfd_get_debug_link_info_1 /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/opncls.c:1206:12                       
    #2 0x7f800af91b8a in find_separate_debug_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/opncls.c:1423:10                        
    #3 0x7f800af91a0f in bfd_follow_gnu_debuglink /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/opncls.c:1582:10                        
    #4 0x7f800b110614 in _bfd_dwarf2_slurp_debug_info /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:4345:19                    
    #5 0x7f800b11bc67 in _bfd_dwarf2_find_nearest_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:4538:9                    
    #6 0x7f800b05e38b in _bfd_elf_find_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/elf.c:8695:10                                 
    #7 0x517c83 in print_symbol /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1003:9                                          
    #8 0x51542d in print_symbols /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1084:7                                         
    #9 0x51542d in display_rel_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1200                                        
    #10 0x510f56 in display_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1318:7                                         
    #11 0x50faae in main /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1792:12                                                
    #12 0x7f8009fa3680 in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.23-r4/work/glibc-2.23/csu/../csu/libc-start.c:289                      
    #13 0x41ac18 in _init (/usr/x86_64-pc-linux-gnu/binutils-bin/git/nm+0x41ac18)                                                                    

0x60200000029e is located 0 bytes to the right of 14-byte region [0x602000000290,0x60200000029e)
allocated by thread T0 here:
    #0 0x4d8e08 in malloc /var/tmp/portage/sys-libs/compiler-rt-sanitizers-5.0.0/work/compiler-rt-5.0.0.src/lib/asan/asan_malloc_linux.cc:67
    #1 0x7f800af6f3fc in bfd_malloc /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/libbfd.c:193:9
    #2 0x7f800af64b9f in bfd_get_full_section_contents /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/compress.c:248:21
    #3 0x7f800af91230 in bfd_get_debug_link_info_1 /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/opncls.c:1191:8
    #4 0x7f800af91b8a in find_separate_debug_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/opncls.c:1423:10
    #5 0x7f800af91a0f in bfd_follow_gnu_debuglink /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/opncls.c:1582:10
    #6 0x7f800b110614 in _bfd_dwarf2_slurp_debug_info /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:4345:19
    #7 0x7f800b11bc67 in _bfd_dwarf2_find_nearest_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:4538:9
    #8 0x7f800b05e38b in _bfd_elf_find_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/elf.c:8695:10
    #9 0x517c83 in print_symbol /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1003:9
    #10 0x51542d in print_symbols /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1084:7
    #11 0x51542d in display_rel_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1200
    #12 0x510f56 in display_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1318:7
    #13 0x50faae in main /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1792:12
    #14 0x7f8009fa3680 in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.23-r4/work/glibc-2.23/csu/../csu/libc-start.c:289

SUMMARY: AddressSanitizer: heap-buffer-overflow /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/libbfd.c:559:24 in bfd_getl32
Shadow bytes around the buggy address:
  0x0c047fff8000: fa fa 00 01 fa fa 00 06 fa fa fd fa fa fa fd fa
  0x0c047fff8010: fa fa fd fd fa fa fd fa fa fa fd fa fa fa fd fa
  0x0c047fff8020: fa fa fd fa fa fa fd fd fa fa fd fa fa fa fd fa
  0x0c047fff8030: fa fa fd fa fa fa fd fd fa fa fd fa fa fa fd fa
  0x0c047fff8040: fa fa fd fa fa fa fd fd fa fa fd fa fa fa 00 fa
=>0x0c047fff8050: fa fa 00[06]fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff8060: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff8070: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff8080: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff8090: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff80a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==11994==ABORTING

Affected version:
2.29.51.20170924 and maybe past releases

Fixed version:
N/A

Commit fix:
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=52b36c51e5bf6d7600fdc6ba115b170b0e78e31d

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
CVE-2017-15021

Reproducer:
https://github.com/asarubbo/poc/blob/master/00373-binutils-heapoverflow-bfd_getl32

Timeline:
2017-09-24: bug discovered and reported to upstream
2017-09-24: upstream released a patch
2017-10-03: blog post about the issue
2017-10-04: CVE assigned

Note:
This bug was found with American Fuzzy Lop.
This bug was identified with bare metal servers donated by Packet. This work is also supported by the Core
Infrastructure Initiative
.

Permalink:

binutils: heap-based buffer overflow in bfd_getl32 (opncls.c)

Description:
binutils is a set of tools necessary to build programs.

The complete ASan output of the issue:

 # nm -A -a -l -S -s --special-syms --synthetic --with-symbol-versions -D $FILE
==11125==ERROR: AddressSanitizer: FPE on unknown address 0x7f5e01fd42e5 (pc 0x7f5e01fd42e5 bp 0x7ffdaa5de290 sp 0x7ffdaa5de0e0 T0)
    #0 0x7f5e01fd42e4 in decode_line_info /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c
    #1 0x7f5e01fe192b in comp_unit_maybe_decode_line_info /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:3608:26
    #2 0x7f5e01fe192b in comp_unit_find_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:3643
    #3 0x7f5e01fde94f in _bfd_dwarf2_find_nearest_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:4755:11
    #4 0x7f5e01f1c20b in _bfd_elf_find_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/elf.c:8694:10
    #5 0x517c83 in print_symbol /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1003:9
    #6 0x51542d in print_symbols /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1084:7
    #7 0x51542d in display_rel_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1200
    #8 0x510f56 in display_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1318:7
    #9 0x50faae in main /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1792:12
    #10 0x7f5e00e61680 in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.23-r4/work/glibc-2.23/csu/../csu/libc-start.c:289
    #11 0x41ac18 in _init (/usr/x86_64-pc-linux-gnu/binutils-bin/git/nm+0x41ac18)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: FPE /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c in decode_line_info
==11125==ABORTING

Affected version:
2.29.51.20170921 and maybe past releases

Fixed version:
N/A

Commit fix:
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=d8010d3e75ec7194a4703774090b27486b742d48

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
CVE-2017-15025

Reproducer:
https://github.com/asarubbo/poc/blob/master/00372-binutils-FPE-decode_line_info

Timeline:
2017-09-22: bug discovered and reported to upstream
2017-09-24: upstream released a patch
2017-10-03: blog post about the issue
2017-10-04: CVE assigned

Note:
This bug was found with American Fuzzy Lop.
This bug was identified with bare metal servers donated by Packet. This work is also supported by the Core
Infrastructure Initiative
.

Permalink:

binutils: divide-by-zero in decode_line_info (dwarf2.c)

Description:
binutils is a set of tools necessary to build programs.

The relevant ASan output of the issue:

# nm -A -a -l -S -s --special-syms --synthetic --with-symbol-versions -D $FILE
==22616==ERROR: AddressSanitizer: stack-overflow on address 0x7ffc2948efe8 (pc 0x0000004248eb bp 0x7ffc2948f8e0 sp 0x7ffc2948efe0 T0)
    #0 0x4248ea in __asan::Allocator::Allocate(unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType, bool) /var/tmp/portage/sys-libs/compiler-rt-sanitizers-5.0.0/work/compiler-rt-5.0.0.src/lib/asan/asan_allocator.cc:381
    #1 0x41f8f3 in __asan::asan_malloc(unsigned long, __sanitizer::BufferedStackTrace*) /var/tmp/portage/sys-libs/compiler-rt-sanitizers-5.0.0/work/compiler-rt-5.0.0.src/lib/asan/asan_allocator.cc:814
    #2 0x4d8de4 in malloc /var/tmp/portage/sys-libs/compiler-rt-sanitizers-5.0.0/work/compiler-rt-5.0.0.src/lib/asan/asan_malloc_linux.cc:68
    #3 0x7ff17b5b237c in bfd_malloc /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/libbfd.c:193:9                                                                                                     
    #4 0x7ff17b5a7b2f in bfd_get_full_section_contents /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/compress.c:248:21                                                                               
    #5 0x7ff17b5e16d3 in bfd_simple_get_relocated_section_contents /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/simple.c:193:12                                                                     
    #6 0x7ff17b75626e in read_section /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:556:8                                                                                                   
    #7 0x7ff17b772053 in read_indirect_string /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:730:9                                                                                           
    #8 0x7ff17b772053 in read_attribute_value /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:1189                                                                                            
    #9 0x7ff17b76ebf4 in read_attribute /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:1306:14                                                                                               
    #10 0x7ff17b76ebf4 in find_abstract_instance_name /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:2913                                                                                    
    #11 0x7ff17b76ec98 in find_abstract_instance_name /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:2930:12                                                                                 
    #12 0x7ff17b76ec98 in find_abstract_instance_name /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:2930:12                                                                                 
    [..cut..]
    #252 0x7ff17b76ec98 in find_abstract_instance_name /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:2930:12

Affected version:
2.29.51.20170921 and maybe past releases

Fixed version:
N/A

Commit fix:
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=52a93b95ec0771c97e26f0bb28630a271a667bd2

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
CVE-2017-15024

Reproducer:
https://github.com/asarubbo/poc/blob/master/00371-binutils-infiniteloop-find_abstract_instance_name

Timeline:
2017-09-22: bug discovered and reported to upstream
2017-09-24: upstream released a patch
2017-10-03: blog post about the issue
2017-10-04: CVE assigned

Note:
This bug was found with American Fuzzy Lop.
This bug was identified with bare metal servers donated by Packet. This work is also supported by the Core
Infrastructure Initiative
.

Permalink:

binutils: infinite loop in find_abstract_instance_name (dwarf2.c)

Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
Taking over a postal address, An Post edition (October 03, 2017, 11:03 UTC)

As I announced a few months ago, I’m moving to London. One of the tasks before the move is setting up postal address redirection, so that the services unable to mail me across the Irish Sea can still reach me. Luckily I know for a fact that An Post (the Irish postal service) has a redirection service, if not a cheap one.

A couple of weeks ago, I went on to sign up for the services, and I found that I had two choices: I could go to the post office (which is inside the EuroSpar next door), show a photo ID and a proof of address, and pay cash or with a debit card1; or I could fill in the form online, pay with a credit card, and then post out a physical signed piece of paper. I chose the latter.

There are many laughable things that I could complain about, in the process of setting up the redirection, but I want to focus on what I think is the big, and most important problem. After you choose the addresses (original and new destination), it will ask you where you want your confirmation PIN sent.

There is a reason why they do that. I set up the redirect well before I moved, and in particular I chose to redirect mail from my apartment to my local office — this way I can either batch together the mail, or simply ask for an inter-office forwarding. This meant I had access to both the original and the new address at the same time — but for many people, particularly moving out of the country, by the time they know where to forward the mail, they might only have access to the new address.

The issue is that if you decide to get the PIN at the new address, the only notification sent to the old address is one letter, confirming the activation of the redirection, sent to the old address. This is likely meant so you can call An Post and have them cancel the redirection if that was done against your will.

While this stops a possible long-term takeover of a mail address, it still allows a wide window of opportunity for a takeover. Also, it has one significant drawback: the letter does not tell you where the mail will be redirected!

Let’s say you want to take over someone’s address (let’s look later what for). First you need to know their address; this is the simplest part of course. Now you can fill in the request on An Post’s website for the redirection — the original address is not given any indication that a request was filled – and get the PIN at the new address. Once the PIN is received, there is some time to enable the redirection.

Until activation is completed, and the redirection time is selected, no communication is given to the original address.

If your target happens to be travelling or otherwise unable to get to their mail for a few weeks, then you have an opportunity. You can take over the address, get some documents at the given address, and get your hands on them. Of course the target will become suspicious when coming back, finding a note about redirection and no mail. But finding a way to recover the mail without being tied to an identity is left as an exercise to the reader.

So what would you accomplish, beside annoying your target, and possibly get some of their unsolicited mail? Well, there are a significant amount of interesting targets in the postal mail you receive in Ireland.

For instance, take credit card statements. Tesco Bank does not allow you to receive them electronic, and Ulster Bank will send you the paper copy even though you opt-in to all the possible electronic communications. And a credit card statement in Ireland include a lot more information than other countries, including just enough to take over the credit card. Tesco Bank for instance will authenticate you with the 16 digits PAN (on the statement), your full address (on the statement), the credit limit (you guessed it, on the statement), and your date of birth (okay, this one is not on the statement, but you can probably find my date of birth pretty easily).

And even if you don’t want to take over the full credit card, having the PAN is extremely useful in and by itself, to take over other accounts. And since you have the statement, it wouldn’t be difficult to figure out what the card is used for — take over an Amazon account, you can take over a lot more things.

But there are more concrete problems too — for instance I do receive a significant amount of pseudo-cash2 in form of Tesco vouchers — having physical control of the vouchers effectively means having the cash in your hand. Or say you want to get a frequent guest or frequent flyer card, because a card is often just enough to get the benefits, and have access to the information on the account. Or just get enough of a proof of address to register on any other service that will require one.

Because let’s remember: an authentication system is just as weak as its weakest link. So all those systems requiring a proof of address? You can skip over all of them by just having one recent enough proof of address, by hijacking someone’s physical mail. And that’s just a matter of paying for it.


  1. An Post is well known for only accepting VISA Debit cards, and refuses both MasterCard Debit and VISA Credit cards. Funnily enough, they issue MasterCard cards, but that’s a story for another time. [return]
  2. I should at some point write a post about pseudo-cash and the value of a euro when it’s not a coin. [return]

October 01, 2017
Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
How blogging changed in the past ten years (October 01, 2017, 16:04 UTC)

One of the problems that keeps poking back at me every time I look for an alternative software for this blog, is that it somehow became not your average blog, particularly not in 2017.

The first issue is that there is a lot of history. While the current “incarnation” of the blog, with the Hugo install, is fairly recent, I have been porting over a long history of my pseudo-writing, merging back into this one big collection the blog posts coming from my original Gentoo Developer blog, as well as the few posts I wrote on the KDE Developers blog and a very minimal amount of content from my (mostly Italian) blog when I was in high school.

Why did I do it that way? Well the main thing is that I don’t want to lose the memories. As some of you might know already, I faced my mortality before, and I came to realize that this blog is probably the only thing of substance that I had a hand on, that will outlive me. And so I don’t want to just let migration, service turndowns, and other similar changes take away what I did. This is also why I did publish to this blog the articles I wrote for other websites, namely NewsForge and Linux.com (back when they were part of Geeknet).

Some of the recovery work actually required effort. As I said above there’s a minimal amount of content that comes from my high school days blog. And it’s in Italian that does not make it particularly interesting or useful. I had deleted that blog altogether years and years ago, so I had to use the Wayback Machine to recover at least some of the posts. I will be going through all my old backups in the hope of finding that one last backup that I remember making before tearing the thing down.

Why did I tear it down in the first place? It’s clearly a teenager’s blog and I am seriously embarrassed by the way I thought and wrote. It was 1314 years ago, and I have admitted last year that I can tell so many times I’ve been wrong. But this is not the change I want to talk about.

The change I want to talk about is the second issue with finding a good software to run my blog: blogging is not what it used to be ten years ago. Or fifteen years ago. It’s not just that a lot of money got involved in the mean time, so now there are a significant amount of “corporate blogs”, that end up being either product announcements in a different form, or the another outlet for not-quite-magazine content. I know of at least a couple of Italian newspapers that provide “blogs” for their writers, which look almost exactly like the paper’s website, but do not have to be reviewed by the editorial board.

In addition to this, a lot of people’s blogs stopped providing as much details of their personal life as they used to. Likely, this is related to the fact that we now know just how nasty people on the Internet can be (read: just as nasty as people off the Internet), and a lot of the people who used to write lightheartedly don’t feel as safe, correctly. But there is probably another reason: “Social Media”.

The advent of Twitter and Facebook made it so that there is less need to post short personal entries, too. And Facebook in particular appears to have swallowed most of the “cutesy memes” such as quizzes and lists of things people have or have not done. I know there are still a few people who insist on not using these big names social networks, and still post for their friends and family on blogs, but I have a feeling they are quite the minority. And I can tell you for sure that since I signed up for Facebook, a lot of my smaller “so here’s that” posts went away.

Distribution chart of blog post sizes over time

This is a bit of a rough plot of blog sizes. In particular I have used the raw file size of the markdown sources used by Hugo, in bytes, which make it not perfect for Unicode symbols, and it includes the “front matter”, which means that particularly all the non-Hugo-native posts have their title effectively doubled by the slug. But it shows trends particularly well.

You can see from that graph that some time around 2009 I almost entirely stopped writing short blog posts. That is around the time Facebook took off in Italy, and a lot of my interaction with friends started happening there. If you’re curious of that visible lack of posts just around half of 2007, that was the pancreatitis that had me disappear for nearly two months.

With this reduction in scope of what people actually write on blogs, I also have a feeling that lots of people were left without anything to say. A number of blogs I still follow (via NewsBlur since Google Reader was shut down), post once or twice a year. Planets are still a thing, and I still subscribe to a number of them, but I realize I don’t even recognize half the names nowadays. Lots of the “old guard” stopped blogging almost entirely, possibly because of a lack of engagement, or simply because, like me, many found a full time job (or a full time family), that takes most of their time.

You can definitely see from the plot that even my own blogging has significantly slowed down over the past few years. Part of it was the tooling giving up on me a few times, but it also involves the lack of energy to write all the time as I used to. Plus there is another problem: I now feel I need to be more accurate in what I’m saying and in the words I’m using. This is in part because I grew up, and know how much words can hurt people even when meant the right way, but also because it turns out when you put yourself in certain positions it’s too easy to attack you (been there, done that).

A number of people that think argue that it was the demise of Google Reader1 that caused blogs to die, but as I said above, I think it’s just the evolution of the concept veering towards other systems, that turned out to be more easily reachable by users.

So are blogs dead? I don’t think so. But they are getting harder to discover, because people use other platforms and it gets difficult to follow all of them. Hacker News and Reddit are becoming many geeks’ default way to discover content, and that has the unfortunate side effect of not having as much of the conversation to happen in shared media. I am indeed bothered about those people who prefer discussing the merit of my posts on those external websites than actually engaging on the comments, if nothing else because I do not track those platforms, and so the feeling I got is of talking behind one’s back — I would prefer if people actually told me if they shared my post on those platforms; for Reddit I can at least IFTTT to self-stalk the blog, but that’s a different problem.

Will we still have blogs in 10 years? Probably yes, but they will not look like the ones we’re used to most likely. The same way as nowadays there still are personal homepages, but they clearly don’t look like Geocities, and there are social media pages that do not look like MySpace.


  1. Usual disclaimer: I do work for Google at the time of writing this, but these are all personal opinions that have no involvement from the company. For reference, I signed the contract before the Google Reader shutdown announcement, but started after it. I was also sad, but I found NewsBlur a better replacement anyway. [return]

September 27, 2017
Nathan Zachary a.k.a. nathanzachary (homepage, bugs)

Important!

This is a workaround for a FreeType/fontconfig problem, but my be applicable in other cases as well. For Gentoo users, the related bug is 631502.

Recently, after updating to Mozilla Firefox to version 52 or later (55.0.2, in my case), and Mozilla Thunderbird to version 52 or later (52.3.0, in my case), I found that fonts were rendering horribly under Linux. It looked essentially like there was no anti-aliasing or hinting at all.

Come to find out, this was due to a change in the content rendering engine, which is briefly mentioned in the release notes for Firefox 52 (but it also applies to Thunderbird). Basically, in Linux, the default engine changed from cairo to Google’s Skia.

Ugly fonts in Firefox and Thunderbird under Linux - skia and cairo

For each application, the easiest method for getting the fonts to render nicely again is to make two changes directly in the configuration editor. To do so in Firefox, simply go to the address bar and type about:config. Within Thunderbird, it can be launched by going to Menu > Preferences > Advanced > Config Editor. Once there, the two keys that need to change are:

gfx.canvas.azure.backends
gfx.content.azure.backends

They likely have values of “skia” or a comma-separated list with “skia” being the first value. On my Linux hosts, I changed the value from skia back to cairo, restarted the applications, and all was again right in the world (or at least in the Mozilla font world 😛 ).

Hope that helps.

Cheers,
Zach

Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)

This is the story of how I ended up calling my bank at 11pm on a Sunday night to ask them to cancel my credit card. But it started with a complete different problem: I thought I found a bug in some PDF library.

I asked Hanno and Ange since they both have lots more experience with PDF as a format than me (I have nearly zero), as I expected this to be complete garbage either coming from random parts of the file or memory within the process that was generating or reading it, and thought it would be completely inconsequential. As you probably have guessed by the spoiler in both the title of the post and the first paragraph, it was not the case. Instead that string is a representation of my credit card number.

After a few hours, having worked on other tasks, and having just gone back and forth with various PDFs, including finding a possibly misconfigured AGPL library in my bank’s backend (worth of another blog post), I realized that Okular does not actually show a title for this PDF, which suggested a bug in Dolphin (the Plasma file manager). In particular Poppler’s pdfinfo also didn’t show any title at all, which suggested there’s a problem with a different part of the code. Since the problem was happening with my credit card statements, and the credit card statements include the full 16-digits PAN, I didn’t want to just file a bug attaching a sample, so instead I started asking around for help to figure out which part of the code is involved.

Albert Astals Cid sent me the right direction by telling me the low-level implementation was coming from KFileMetadata, and that quickly pointed me at this interesting piece of heuristics which is designed to guess the title of a document by looking at the first page. The code is quite a bit convoluted, so I couldn’t at first just exclude uninitialized memory access, but I couldn’t figure out where it could be coming from, so I decided to copy the code into a single executable to play around with it. The good news was that it would give me the exact same answer, so it was not uninitialized memory. Instead, the parser was mis-reading something in the file, which by being stable meant it wasn’t likely a security issue, just sub-optimal code.

As there is no current, updated tool for PDF that behaves like mkvinfo, that is print an element-by-element description of the content of a PDF file, I decided to just play with the code to figure out how it decided what to use as the title. Printing out each of the possible titles being evaluated showed it was considering first my address, then part of the summary information, then this strange string. What is going on there?

The code is a bit difficult to follow, particularly for me at first since I had no idea how PDF works to begin with. But the summary of it is that it goes through the textboxes (I knew already that PDF text is laid out in boxes) of the first page, joining together the text if the box has markers to follow up. Each of these entries is stored into a map of text heights, together with a “watermark” of the biggest text size encountered during this loop. If, when looking at a textbox, the height is lower than the previous maximum height, it gets discarded. At the end, the first biggest textbox content is reported as the title.

Once I disabled the height check and always reported all the considered title textboxes, I noticed something interesting: the string that kept being reported was found together with a number of textboxes that are drawn on top of the bank giro1 credit system. The cheque includes a very big barcode… and that’s where I started sweating a bit.

The reason of the sweat is that by then I already guessed I made a huge mistake sharing the string that Dolphin was showing me. The reference to pay up a credit card is universally the full 16-digits number (PAN). Indeed the full number is printed on the cheque, and as the “An Post Ref” (An Post being the Irish postal system), and the account information (10-digits, excluding the 6-digits IIN) is printed on the bottom of the same. All of this is why I didn’t want to share the sample file, and why I always destroy the statements that arrive, in paper form, from the banks. At this point, the likeliness of the barcode containing the same information was seriously high.

My usual Barcode Scanner for Android didn’t manage to understand the barcode though, which made it awkward. Instead I decided to confirm I was actually looking at the content of the barcode in an encoded form with a very advanced PDF inspection tool: strings $file | grep Font. This did bring up a reference to /BaseFont /Code128ARedA. And that was the confirmation I needed. Indeed a quick search for that name brings you to a public domain font that implements Code 128 barcodes as a TrueType font. This is not uncommon, particularly as it’s the same method used by most label printers, including the Dymo I used to use for labelling computers.

At that point a quick comparison of the barcode I had in front of me with one generated through an online generator (but only for the IIN because I don’t want to leak it all), confirmed I was looking at my credit card number, and that my tweet just leaked it — in a bit of a strange encoding that may take some work to decode, but still leaked it. I called Ulster Bank and got the card cancelled and replaced.

Which lessons I can learn from this experience? First of all to consider credit card statements even more of a security risk than I ever imagine. It also gave me a practical instance of what Brian Krebs advocates for years regarding barcodes of boarding passes and similar. In particular it looks like both Ulster Bank and Tesco Bank use the same software to generate the credit card statements (which is easily told not to be the same system that generates the normal bank statements), which is developed by Fiserv (their name is in the Author field of the PDF), and they all rely on using the normal full card number for payment.

This is something I don’t really understand. In Italy, you only use the 16-digits number to pay the bank one-off by wire, and instead the statements never had more than the last five digits of the card. Except for the Italian American Express — but that does not surprise me too much as they manage it from London as well.

I’m now looking to see how I can improve on the guessing of the title for the PDFs in the KFileMetadata library — although I’m warming up to the idea of just sending a patch that delete that part of the code altogether, and if the file has no title, no title is displayed. The simplest solutions are, usually, the better.


  1. The Wikipedia page appears to talk only of the UK system. Ireland, as usual, appears to have kept their own version of the same system, and all the credit card statements, and most bills, will have a similar pre-printed “credit cheque” at the bottom. Even when they are direct-debited. [return]

September 26, 2017
Sven Vermeulen a.k.a. swift (homepage, bugs)
SELinux Userspace 2.7 (September 26, 2017, 12:50 UTC)

A few days ago, Jason "perfinion" Zaman stabilized the 2.7 SELinux userspace on Gentoo. This release has quite a few new features, which I'll cover in later posts, but for distribution packagers the main change is that the userspace now has many more components to package. The project has split up the policycoreutils package in separate packages so that deployments can be made more specific.

Let's take a look at all the various userspace packages again, learn what their purpose is, so that you can decide if they're needed or not on a system. Also, when I cover the contents of a package, be aware that it is based on the deployment on my system, which might or might not be a complete installation (as with Gentoo, different USE flags can trigger different package deployments).

libsepol - manipulating SELinux binary policies

The first package, known in Gentoo as sys-libs/libsepol, is the library that enables manipulating the SELinux binary policies. This is a core library, and is the first SELinux userspace package that is installed on a system.

It contains one command, chkcon, which allows users to validate if a specific security context exists within a binary policy file:

~$ chkcon policy.29 user_u:user_r:mozilla_t:s0
user_u:user_r:mozilla_t:s0 is valid

The package does contain two manpages of old commands which are no longer available (or I'm blind, either way, they're not installed and not found in the SELinux userspace repository either) such as genpolusers and genpolbools.

libselinux - the main SELinux handling library

The libselinux library, known in Gentoo as sys-libs/libselinux, is the main SELinux library. Almost all applications that are SELinux-aware (meaning they not only know SELinux is a thing, but are actively modifying their behavior with SELinux-specific code) will link to libselinux.

Because it is so core, the package also provides the necessary bindings for different scripting languages besides the standard shared objects approach, namely Python (as many SELinux related tooling is written in Python) and Ruby.

Next to the bindings and libraries, libselinux also offers quite a few executables to query and manipulate SELinux settings on the system, which are shortly described on the SELinux userspace wiki but repeated here for convenience. Most of these are meant for debugging purposes, as they are simple wrappers toward the libselinux provided functions, but some of them are often used by administrations.

  • avcstat gives statistics about the in-kernel access vector cache, such as number of lookups, hits and misses
  • compute_create queries the kernel security server for a transition decision
  • compute_av queries the kernel security server for an access vector decision
  • compute_relabel queries the kernel security server for a relabel decision
  • compute_member queries the kernel security server for a labeling decision on a polyinstantiated object
  • getconlist uses the security\_compute\_user() function, and orders the resulting list based on the default\_contexts file and per-user context files
  • getdefaultcon is like getconlist but only returns the first context
  • compute_user queries the kernel security server fo a set of reachable user contexts from a source context
  • getfilecon gets the context of a file by path
  • getpidcon gets the context of a process by PID
  • getseuser queries the seuser file for the resulting SELinux user and contxt for a particular linux login and login context
  • getsebool gets the current state of a SELinux boolean in the SELinux security server
  • matchpathcon queries the active filecontext file for how a particular path should be labeled
  • policyvers queries the kernel security server for the maximum policy version supported
  • getenforce gets the enforcing state of the kernel access vector cache
  • sefcontext_compile generates binary filecontext files, optimized for fast querying
  • selabel_lookup looks up what the target default context is for various classes (supporting the X related SELinux types, database types, etc.)
  • selabel_digest calculates the SHA1 digest of spec files, and returns a list of the specfiles used to calculate the digest. This is used by Android.
  • selabel_partial_match determines if a direct or partial match is possible on a file path
  • selabel_lookup_best_match obtains the best matching SELinux security context for file-based operations
  • selinux_check_securetty_context checks whether a SELinux tty security context is defined as a securetty context
  • selinux_check_access checks if the source context has the access permission for the specified class on the target context
  • selinuxexeccon reports the SELinux context for an executable
  • selinuxenabled returns if SELinux is enabled or not
  • setfilecon sets the context of a path
  • setenforce sets the enforcing state of the kernel access vector cache
  • togglesebool toggles a SELinux boolean, but only runtime (so it does not persist across reboots)

checkpolicy - policy compiler

The checkpolicy package, known in Gentoo as sys-apps/checkpolicy, provides two main applications, checkpolicy and checkmodule. Both applications are compilers (unlike what the name implies) which build a binary SELinux policy. The main difference between these two is that one builds a policy binary, whereas the other one builds a SELinux module binary.

Developers don't often call these applications themselves, but use the build scripts. For instance, the semodule_package binary would be used to combine the binary policy with additional files such as file contexts.

libsemanage - facilitating use of SELinux overall

The libsemanage library, known in Gentoo as sys-libs/libsemanage, contains SELinux supporting functions that are needed for any regular SELinux use. Whereas libselinux would be used everywhere, even for embedded systems, libsemanage is generally not for embedded systems but is very important for Linux systems in overall.

Most SELinux management applications that administrators come in contact with will be linked with the libsemanage library. As can be expected, the semanage application as offered by the selinux-python package is one of them.

The only application that is provided by libsemanage is the semanage_migrate_store, used to migrate the policy store from the /etc/selinux to the /var/lib/selinux location. This was done with the introduction of the 2.4 userspace.

selinux-python - Python-based command-line management utilities

The selinux-python package, known in Gentoo as sys-apps/selinux-python, is one of the split packages that originally where part of the policycoreutils package. It contains the majority of management utilities that administrators use for handling SELinux on their systems.

The most known application here is semanage, but it contains quite a few others as well:

  • sepolgen generates an initial SELinux policy module template, and is short for the sepolicy generate command
  • audit2why translates SELinux audit messages into a description of why the access was denied. It is short for the audit2allow -w command.
  • audit2allow generates SELinux policy allow/dontaudit rules from logs of denied operations
  • sepolgen-ifgen generates an overview of available interfaces. This overview is used by audit2allow to guess the right interface to use when allowing or dontauditing certain operations.
  • sepolicy is the SELinux policy inspection tool, allowing to query various aspects of a SELinux configuration (namely booleans, communication flows, interfaces, network information and transition information). It also provides the ability to generate skeleton policies (as described with sepolgen) and manual pages.
  • chcat changes a file's SELinux security category
  • sepolgen-ifgen-attr-helper generates an overview of attributes and attribute mappings. This overview is used by audit2allow to guess the right attribute to use when allowing or dontauditing certain operations.
  • semanage is a SELinux policy management tool, allowing a multitude of operations against the SELinux policy and the configuration. This includes definition import/export, login mappings, user definitions, ports and interface management, module handling, file contexts, booleans and more.

semodule-utils - Developing SELinux modules

The semodule-utils package, known in Gentoo as sys-apps/semodule-utils, is another split package that originally was part of the policycoreutils package. In it, SELinux policy module development utilities are provided. The package is not needed for basic operations such as loading and unloading modules though.

  • semodule_expand expands a SELinux base module package into a kernel binary policy file
  • semodule_deps shows the dependencies between SELinux policy packages
  • semodule_link links SELinux policy module packages together into a single SELinux policy module
  • semodule_unpackage extracts a SELinux module into the binary policy and its associated files (such as file context definitions)
  • semodule_package combines a modular binary policy file with its associated files (such as file context definitions) into a module package

mcstrans - Translate context info in human readable names

The mcstrans package, known in Gentoo as sys-apps/mcstrans, is another split package that originally was part of the policycoreutils package. In it, the MCS translation daemon is hosted. This daemon translates the SELinux-specific context ranges, like s0-s0:c0.c1024 to a human-readable set, like SystemLow-SystemHigh.

This is a purely cosmetic approach (as SELinux internally always uses the sensitivity and category numbers) but helps when dealing with a large number of separate categories.

restorecond - Automatically resetting file contexts

The restorecond package, known in Gentoo as sys-apps/restorecond, is another split package that originally was part of the policycoreutils package. It contains the restorecond daemon, which watches over files and directories and forces the right SELinux label on it.

This daemon was originally intended to resolve a missing feature in SELinux (having more fine-grained rules for label naming) but with the named file transition support, the need for this daemon has diminished a lot.

secilc - SELinux common intermediate language compiler

The secilc package, known in Gentoo as sys-apps/secilc, is the CIL compiler which builds kernel binary policies based on the passed on CIL code. Although the majority of policy development still uses the more traditional SELinux language (and supporting macro's from the reference policy), developers can already use CIL code for policy generation.

With secilc, a final policy file can be generated through the CIL code.

selinux-dbus - SELinux DBus server

The selinux-dbus package (not packaged in Gentoo at this moment) provides a SELinux DBus service which systems can use to query and interact with SELinux management utilities on the system. If installed, the org.selinux domain is used for various supported operations (such as listing SELinux modules, through org.selinux.semodule_list).

selinux-gui - Graphical SELinux settings manager

The selinux-gui package (not packaged in Gentoo at this moment) provides the system-config-selinux application which offers basic SELinux management support in a graphical application. It supports boolean handling, file labeling, user mapping, SELinux user management, network port definitions and module handling. As such, it can be seen as the graphical helper utility for the semanage command.

selinux-sandbox - Sandbox utility utilizing SELinux sandbox domains

The selinux-sandbox package (not packaged in Gentoo at this moment) is a set of scripts to facilitate the creation of SELinux sandboxes. With these utilities, which not only use SELinux sandbox domains like sandbox_t but also Linux namespaces, end users can launch applications in a restricted environment.

policycoreutils - Core SELinux management utilities

The policycoreutils package, known in Gentoo as sys-apps/policycoreutils, contains basic SELinux tooling which is necessary to handle SELinux in a regular environment. Supported utilities are:

  • newrole to switch a user session from one role to another
  • secon to query the SELinux context of a file, program or user input
  • genhomedircon to regenerate home directory context files, necessary when new users are defined on the system
  • setfiles to set SELinux file security contexts on resources
  • semodule to list, load and unload SELinux modules
  • run_init to launch an init script in the right domain
  • open_init_pty to run a program under a pseudo terminal with the right context set
  • sestatus to query current policy status
  • setsebool to set and, if wanted, persist a SELinux boolean value
  • selinuxconfig to display the current active configuration paths
  • restorecon to set SELinux file security contexts on resources
  • load_policy to load the SELinux policy, generally called from initramfs systems if the init system is not SELinux-aware
  • restorecon_xattr manages the security.restorecon_last extended attribute which is set by setfiles or restorecon

Gentoo also adds in two additional scripts: rlpkg to reset file contexts on files provided by a Gentoo package selocal to easily handle small SELinux rule additions to the active policy

There are even more

Attentive readers will notice that the setools package is not discussed here. This package is not provided by the SELinux userspace project, but is an important package for SELinux policy developers as it contains the sesearch command - an often used command to query the active policy.

The above list is thus a picture of the SELinux userspace utilities, which is becoming quite a big application set now that some functionality is split off from the policycoreutils package.

Description:
binutils is a set of tools necessary to build programs.

The complete ASan output of the issue:

# nm -A -a -l -S -s --special-syms --synthetic --with-symbol-versions -D $FILE
==3235==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x613000000512 at pc 0x7f7c93ae3c88 bp 0x7ffe38d7a970 sp 0x7ffe38d7a968
READ of size 1 at 0x613000000512 thread T0
    #0 0x7f7c93ae3c87 in read_1_byte /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:616:10
    #1 0x7f7c93ae3c87 in decode_line_info /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:2311
    #2 0x7f7c93aee92b in comp_unit_maybe_decode_line_info /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:3608:26
    #3 0x7f7c93aee92b in comp_unit_find_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:3643
    #4 0x7f7c93aeb94f in _bfd_dwarf2_find_nearest_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:4755:11
    #5 0x7f7c93a2920b in _bfd_elf_find_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/elf.c:8694:10
    #6 0x517c83 in print_symbol /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1003:9
    #7 0x51542d in print_symbols /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1084:7
    #8 0x51542d in display_rel_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1200
    #9 0x510f56 in display_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1318:7
    #10 0x50faae in main /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1792:12
    #11 0x7f7c9296e680 in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.23-r4/work/glibc-2.23/csu/../csu/libc-start.c:289
    #12 0x41ac18 in _init (/usr/x86_64-pc-linux-gnu/binutils-bin/git/nm+0x41ac18)

0x613000000512 is located 0 bytes to the right of 338-byte region [0x6130000003c0,0x613000000512)
allocated by thread T0 here:
    #0 0x4d8e08 in malloc /var/tmp/portage/sys-libs/compiler-rt-sanitizers-5.0.0/work/compiler-rt-5.0.0.src/lib/asan/asan_malloc_linux.cc:67
    #1 0x7f7c9393a37c in bfd_malloc /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/libbfd.c:193:9
    #2 0x7f7c9392fb2f in bfd_get_full_section_contents /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/compress.c:248:21
    #3 0x7f7c939696d3 in bfd_simple_get_relocated_section_contents /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/simple.c:193:12
    #4 0x7f7c93ade26e in read_section /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:556:8
    #5 0x7f7c93adef3c in decode_line_info /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:2047:9
    #6 0x7f7c93aee92b in comp_unit_maybe_decode_line_info /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:3608:26
    #7 0x7f7c93aee92b in comp_unit_find_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:3643
    #8 0x7f7c93aeb94f in _bfd_dwarf2_find_nearest_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:4755:11
    #9 0x7f7c93a2920b in _bfd_elf_find_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/elf.c:8694:10
    #10 0x517c83 in print_symbol /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1003:9
    #11 0x51542d in print_symbols /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1084:7
    #12 0x51542d in display_rel_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1200
    #13 0x510f56 in display_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1318:7
    #14 0x50faae in main /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1792:12
    #15 0x7f7c9296e680 in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.23-r4/work/glibc-2.23/csu/../csu/libc-start.c:289

SUMMARY: AddressSanitizer: heap-buffer-overflow /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:616:10 in read_1_byte
Shadow bytes around the buggy address:
  0x0c267fff8050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c267fff8060: 00 00 00 00 00 00 00 00 00 00 00 04 fa fa fa fa
  0x0c267fff8070: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x0c267fff8080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c267fff8090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c267fff80a0: 00 00[02]fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c267fff80b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c267fff80c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c267fff80d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c267fff80e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c267fff80f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==3235==ABORTING

Affected version:
2.29.51.20170921 and maybe past releases

Fixed version:
N/A

Commit fix:
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=515f23e63c0074ab531bc954f84ca40c6281a724

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
CVE-2017-14939

Reproducer:
https://github.com/asarubbo/poc/blob/master/00370-binutils-heapoverflow-read_1_byte

Timeline:
2017-09-21: bug discovered and reported to upstream
2017-09-24: upstream released a patch
2017-09-26: blog post about the issue
2017-09-29: CVE assigned

Note:
This bug was found with American Fuzzy Lop.
This bug was identified with bare metal servers donated by Packet. This work is also supported by the Core Infrastructure Initiative.

Permalink:

binutils: heap-based buffer overflow in read_1_byte (dwarf2.c)

Description:
binutils is a set of tools necessary to build programs.

The complete ASan output of the issue:

# nm -A -a -l -S -s --special-syms --synthetic --with-symbol-versions -D $FILE
==491==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7f6e3316d573 bp 0x7ffda2ee9e50 sp 0x7ffda2ee9c60 T0)
==491==The signal is caused by a READ memory access.
==491==Hint: address points to the zero page.
    #0 0x7f6e3316d572 in scan_unit_for_symbols /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:3213:13
    #1 0x7f6e331769e4 in comp_unit_maybe_decode_line_info /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:3617:9
    #2 0x7f6e331769e4 in comp_unit_find_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:3643
    #3 0x7f6e331707c8 in _bfd_dwarf2_find_nearest_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:4601:11
    #4 0x7f6e330b120b in _bfd_elf_find_line /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/elf.c:8694:10
    #5 0x517c83 in print_symbol /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1003:9
    #6 0x51542d in print_symbols /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1084:7
    #7 0x51542d in display_rel_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1200
    #8 0x510f56 in display_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1318:7
    #9 0x50faae in main /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1792:12
    #10 0x7f6e31ff6680 in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.23-r4/work/glibc-2.23/csu/../csu/libc-start.c:289
    #11 0x41ac18 in _init (/usr/x86_64-pc-linux-gnu/binutils-bin/git/nm+0x41ac18)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/dwarf2.c:3213:13 in scan_unit_for_symbols
==491==ABORTING

Affected version:
2.29.51.20170921 and maybe past releases

Fixed version:
N/A

Commit fix:
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=0d76029f92182c3682d8be2c833d45bc9a2068fe

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
CVE-2017-14940

Reproducer:
https://github.com/asarubbo/poc/blob/master/00369-binutils-NULLptr-scan_unit_for_symbols

Timeline:
2017-09-21: bug discovered and reported to upstream
2017-09-24: upstream released a patch
2017-09-26: blog post about the issue
2017-09-29: CVE assigned

Note:
This bug was found with American Fuzzy Lop.
This bug was identified with bare metal servers donated by Packet. This work is also supported by the Core Infrastructure Initiative.

Permalink:

binutils: NULL pointer dereference in scan_unit_for_symbols (dwarf2.c)

Description:
binutils is a set of tools necessary to build programs.

This issue was initially discovered because the nm process eat ~230GB of ram. Later on, another testcase hits the issue and now I have a stracktrace.
The relevant ASan output of the issue:

# nm -A -a -l -S -s --special-syms --synthetic --with-symbol-versions -D $FILE
    #8 0x4d8de4 in malloc /var/tmp/portage/sys-libs/compiler-rt-sanitizers-5.0.0/work/compiler-rt-5.0.0.src/lib/asan/asan_malloc_linux.cc:68
    #9 0x7fd0deccb41d in _objalloc_alloc /var/tmp/portage/sys-devel/binutils-9999/work/binutils/libiberty/objalloc.c:143:22
    #10 0x7fd0de921c24 in bfd_alloc /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/opncls.c:949:9
    #11 0x7fd0de921c24 in bfd_zalloc2 /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/opncls.c:1031
    #12 0x7fd0de9b2db8 in _bfd_elf_slurp_version_tables /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/elf.c:8222:2
    #13 0x7fd0de999da7 in bfd_elf64_slurp_symbol_table /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/elfcode.h:1165:9
    #14 0x7fd0de9ed876 in _bfd_elf_canonicalize_dynamic_symtab /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/elf.c:8099:19
    #15 0x7fd0de935fc7 in _bfd_generic_read_minisymbols /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/syms.c
    #16 0x513a53 in display_rel_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1107:14
    #17 0x510f56 in display_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1318:7
    #18 0x50faae in main /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1792:12
    #19 0x7fd0dd934680 in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.23-r4/work/glibc-2.23/csu/../csu/libc-start.c:289
    #20 0x41ac18 in _init (/usr/x86_64-pc-linux-gnu/binutils-bin/git/nm+0x41ac18)

Affected version:
2.29.51.20170921 and maybe past releases

Fixed version:
N/A

Commit fix:
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=bd61e135492ecf624880e6b78e5fcde3c9716df6

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
CVE-2017-14938

Reproducer:
https://github.com/asarubbo/poc/blob/master/00368-binutils-memallocfailure-_bfd_elf_slurp_version_tables

Timeline:
2017-09-21: bug discovered and reported to upstream
2017-09-24: upstream released a patch
2017-09-26: blog post about the issue
2017-09-29: CVE assigned

Note:
This bug was found with American Fuzzy Lop.
This bug was identified with bare metal servers donated by Packet. This work is also supported by the Core Infrastructure Initiative.

Permalink:

binutils: memory allocation failure in _bfd_elf_slurp_version_tables (elf.c)

September 25, 2017

Description:
binutils is a set of tools necessary to build programs.

The complete ASan output of the issue:

# nm -A -a -l -S -s --special-syms --synthetic --with-symbol-versions -D $FILE
==40547==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61a000000574 at pc 0x0000004c1ca8 bp 0x7ffc34f58d10 sp 0x7ffc34f584c0
WRITE of size 6 at 0x61a000000574 thread T0
    #0 0x4c1ca7 in __asan_memcpy /var/tmp/portage/sys-libs/compiler-rt-sanitizers-5.0.0/work/compiler-rt-5.0.0.src/lib/asan/asan_interceptors.cc:466
    #1 0x7f6df2a247e5 in _bfd_x86_elf_get_synthetic_symtab /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/elfxx-x86.c:1946:3
    #2 0x7f6df29f7b7a in elf_x86_64_get_synthetic_symtab /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/elf64-x86-64.c:4963:10
    #3 0x513df5 in display_rel_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1155:21
    #4 0x510f56 in display_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1318:7
    #5 0x50faae in main /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1792:12
    #6 0x7f6df19d1680 in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.23-r4/work/glibc-2.23/csu/../csu/libc-start.c:289
    #7 0x41ac18 in _init (/usr/x86_64-pc-linux-gnu/binutils-bin/git/nm+0x41ac18)

0x61a000000574 is located 0 bytes to the right of 1268-byte region [0x61a000000080,0x61a000000574)
allocated by thread T0 here:
    #0 0x4d8e08 in malloc /var/tmp/portage/sys-libs/compiler-rt-sanitizers-5.0.0/work/compiler-rt-5.0.0.src/lib/asan/asan_malloc_linux.cc:67
    #1 0x7f6df299dd5c in bfd_malloc /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/libbfd.c:193:9
    #2 0x7f6df299dd5c in bfd_zmalloc /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/libbfd.c:278
    #3 0x7f6df2a23e29 in _bfd_x86_elf_get_synthetic_symtab /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/elfxx-x86.c:1829:26
    #4 0x7f6df29f7b7a in elf_x86_64_get_synthetic_symtab /var/tmp/portage/sys-devel/binutils-9999/work/binutils/bfd/elf64-x86-64.c:4963:10
    #5 0x513df5 in display_rel_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1155:21
    #6 0x510f56 in display_file /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1318:7
    #7 0x50faae in main /var/tmp/portage/sys-devel/binutils-9999/work/binutils/binutils/nm.c:1792:12
    #8 0x7f6df19d1680 in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.23-r4/work/glibc-2.23/csu/../csu/libc-start.c:289

SUMMARY: AddressSanitizer: heap-buffer-overflow /var/tmp/portage/sys-libs/compiler-rt-sanitizers-5.0.0/work/compiler-rt-5.0.0.src/lib/asan/asan_interceptors.cc:466 in __asan_memcpy
Shadow bytes around the buggy address:
  0x0c347fff8050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c347fff8060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c347fff8070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c347fff8080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c347fff8090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c347fff80a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00[04]fa
  0x0c347fff80b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c347fff80c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c347fff80d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c347fff80e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c347fff80f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==40547==ABORTING

Affected version:
2.29.51.20170921 and maybe past releases

Fixed version:
N/A

Commit fix:
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=61e3bf5f83f7e505b6bc51ef65426e5b31e6e360
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=56933f9e3e90eebf1018ed7417d6c1184b91db6b

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
CVE-2017-14729

Reproducer:
https://github.com/asarubbo/poc/blob/master/00367-binutils-heapoverflow-_bfd_x86_elf_get_synthetic_symtab

Timeline:
2017-09-21: bug discovered and reported to upstream
2017-09-22: upstream released a patch
2017-09-25: blog post about the issue
2017-09-25: CVE assigned

Note:
This bug was found with American Fuzzy Lop.
This bug was identified with bare metal servers donated by Packet. This work is also supported by the Core Infrastructure Initiative.

Permalink:

binutils: heap-based buffer overflow in _bfd_x86_elf_get_synthetic_symtab (elfxx-x86.c)

September 24, 2017
Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
(Audio)book review: We Are Legion (We Are Bob). (September 24, 2017, 11:04 UTC)

I have not posted a book review in almost a year, and I have not even written one for Goodreads, which I probably should do as well. I feel kind of awful for it because I do have a long list of good titles I appreciated in the meantime. So let me spend a few words on this one.

We Are Legion (We Are Bob) tickled me in the list of Audible books for a while because the name sounded so ludicrous I was expecting something almost along the lines of the Hitchikers’ Guide To The Galaxy. It was not that level of humour, but the book didn’t really disappoint either.

The book starts in the first scene in present time, with the protagonist going to a cryogenic facility… and you can tell from the cover that’s just a setup of course. I found it funny from the first scenes that the author clearly is talking of something he knows directly, so I wasn’t entirely too surprised when I found that he’s a computer programmer. I’m not sure what it is with people in my line of work deciding to write books, but the results are quite often greatly enjoyable, even if it takes a while to get into them. On this note, Tobias Klausmann of Gentoo fame wrote a two-part series1, which I definitely recommend.

Once you get on with the main stage for the book, it starts off in the direction you expect with spaceships and planets as the covers lets you to imagine. Some of the reviews I read before buying the book found it very lightweight and no-brainer, but I don’t see myself agreeing. While taking it with a lot of spirit and humour, and a metric ton of pop-culture references2, the topics that are brought up include self-determination, the concept of soul as seen by an atheist point of view, global politics as seen from lightyears away3, and the vast multitudes of “oneselves”.

Spoilers in this paragraph, yes definitely spoilers, and a bit of text so you may not read them out of line of sight. Go back to the following paragraph if you don’t want any. Indeed, it’s very hard to tell, and a question that the book spends quite a bit of time pondering over without an answer, whether the character we see in the first scene is actually the protagonist of the book. Because what we have later is a computer “replicant” of the memories and consciousness of him… and a multitudes of copies of that, each acting more or less differently from the original, leaving open the question whether the copies are losing something in the process, or whether it is the knowledge of not being the “original” that make them change. I found this maybe even more profound than the author intended.

Spoilers aside, I found the book enjoyable. It’s not an all-out bright and shiny future, but it’s also not the kind of grim and dark dystopia that appears to be a dime a dozen nowadays. The one thing that still bothers me a little bit, and that probably is because I would have fallen into the same trap, is that the vast majority of the book focuses on technical problems and solutions, though to be fair it pulls it off (in my opinion) quite healthily, rather than by hiding all the human factors away into “someone else’s problem” territory. It reminded me of an essay I had to write in middle school about the “school of the future”, and I ended up not spending a single word on people, even after the teacher pointed out I should have done so and got me to rewrite it. I’m glad there are people (who are not me) studying humanities.

I found it funny that the Wikipedia page about the book insisted on pointing out that reviewers noted the lack of female characters. That’s true, there are a handful of throwaway women throughout the book, but no major character. I don’t know if there was any way around it given the plot as it stands now though, so I wouldn’t read it too much into it, as the book itself feels a lot like a trip into one’s own essence, and I’m not sure I’d expect an author to be able to analyse this way someone else but themselves. I have not read/listened to the other books in the series (though I did add them to my list now), so maybe that changes with the change of focus, not sure.

As for the audiobook itself, which I got through Audible where it was at “special price” $1.99, I just loved the production. Ray Porter does a fantastic job, and since the book is all written in the first person (from somewhat different points of view), his voicework to make you know which point of view is speaking is extremely helpful not to get lost.

All in all, I’ve really enjoyed the book, and look forward to compare with the rest of the series. If you’re looking for something that distracts you from all the dread that is happening right now in the world, and can give you a message of “If we get together, we can do it!”, then this is a worthy book.


  1. I hadn’t realized book two was out until I looked Tobias up on Amazon. I’ll have stern words with him next time I see him for not warning me! [return]
  2. This happens most of the time with geeks writing books, although not all the time thankfully. From one side it does build a nice sense of camaraderie with the protagonists because they feel like “one of us” but on the other hand sometimes it feels too much. Unless it’s part of the story, like here or in Magic 2.0. [return]
  3. Pun totally intended. [return]

September 20, 2017
Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
Free Idea: structured access logs for Apache HTTPD (September 20, 2017, 14:04 UTC)

This post is part of a series of free ideas that I’m posting on my blog in the hope that someone with more time can implement. It’s effectively a very sketched proposal that comes with no design attached, but if you have time you would like to spend learning something new, but no idea what to do, it may be a good fit for you.

I have been commenting on Twitter a bit about the lack of decent tooling to deal with Apache HTTPD’s Combined Logging Format (inherited from NCSA). For those who do not know about it, this is hte format used by standard access_log files, which include information about requests, including the source IP, the time, the requested path, the status code and the User-Agent used.

These logs are useful for debugging but are also consumed by tools such as AWStats to produce useful statistics about the request patterns of a website. I used these extensively when writing my ModSecurity rulesets, and I still keep an eye out on them for instance to report wasteful feed readers.

The files are simple text files, and that makes it easy to act on them: you can use tail and grep, and logrotate needs no special code beside moving the file and reloading Apache to have it re-open the paths. This makes it hard to query for particular entries in fields, such as to get the list of User-Agent strings present in a log. Some of the suggestions I got over Twitter to solve this were to use awk, but as it happens, these logs are not actually parseable with a straightforward field separation.

Lacking finding a good set of tools to handle these formats directly, I have been complaining that we should probably start moving away from simple text files into more structured log formats. Indeed, I know that there used to be at least some support for logging directly to MySQL and other relational databases, and that there are more complicated machinery often used by companies and startups that process these access logs into analysis software and so on. But all of these tend to be high overhead, much more than what I or someone else with a small personal blog would care about implementing.

Instead I think it’s time to start using structured file logs. A few people including thresh from VideoLAN suggested using JSON to write the log files. This is not a terrible idea, as the format is at least well understood and easy to interface with most other software, but honestly I would prefer something with an actual structure, a schema that can be followed. Of course I’m not meaning XML, and I would rather suggest having a standardized schema for proto3. Part of that I guess is because I’m used to use this at work, but also because I like the idea of being able to just define my schema and have it generate the code to parse the messages.

Unfortunately currently there is no support or library to access a sequence of protocol buffer messages. Using a single message with repeated sub-messages would work, but it is not append-friendly so there is no way to just keep writing this to a file, and being able to truncate and resume writing to it, which is a property needed for a proper structured log format to actually fit in the space previously occupied by text formats. This is something I don’t usually have to deal with at work, but I would assume that a simple LV (Length-Value) or LVC (Length-Value-Checksum) encoding would be okay to solve this problem.

But what about other properties of the current format? Well, the obvious answer is that, assuming your structured log contains at least as much information (but possibly more) as the current log, you can always have tools that convert on the fly to the old format. This would for instance allow to have a special tail-like command and a grep-like command that provides compatibility with the way the files are currently looked at manually by your friendly sysadmin.

Having more structured information would also allow easier, or deeper analysis of the logs. For instance you could log the full set of headers (like ModSecurity does) instead of just the referrer and User-Agent. And allow for customizing the output on the conversion side rather than lose the details when writing.

Of course this is just one possible way to solve this problem, and just because I would prefer working with technologies that I’m already friendly with it does not mean I wouldn’t take another format that is similarly low-dependency and easy to deal with. I’m just thinking that the change-averse solution of not changing anything and keeping logs in text format may be counterproductive in this situation.

September 17, 2017
Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
Anyone working on motherboard RGB controllers? (September 17, 2017, 11:04 UTC)

I have been contacted by email last week by a Linux user, probably noticing my recent patch for the gpio_it87 driver in the kernel. They have been hoping my driver could extend to IT7236 chips that are used in a number of gaming motherboards for controlling RGB LEDs.

Having left the case modding world after my first and only ThermalTake chassis – my mother gave me hell for the fans noise, mostly due to the plexiglass window on the side of the case – I still don’t have any context whatsoever on what the current state of these boards is, whether someone has written generic tools to set the LEDs, or even UIs for them. But it was an interesting back and forth of looking for leads into figuring out what is needed.

The first problem is, like most of you who already know a bit about electrical engineering and electronics, that clearly the IT7236 chip is clearly not the same series as the IT87xx chips that my driver supports. And since they are not the same series, they are unlikely to share the same functionality.

The IT87xx series chips are Super I/O controllers, which mean they implement functionality such as floppy-disk controllers, serial and parallel ports and similar interfaces, generally via the LPC bus. You usually know these chip names because these need to be supported by the kernel for them to show up in sensors output. In addition to these standard devices, many controllers include at least a set of general purpose I/O (GPIO) lines. On most consumer motherboards these are not exposed in any way, but boards designed for industrial applications, or customized boards tend to expose those lines easily.

Indeed, I wrote the gpio_it87 driver (well, actually adapted and extended it from a previous driver), because the board I was working on in Los Angeles had one of those chips, and we were interested in having access to the GPIO lines to drive some extra LEDs (and possibly in future versions more external interfaces, although I don’t know if anything was made of those). At the time, I did not manage to get the driver merged; a couple of years back, LaCie manufactured a NAS using a compatible chip, and two of their engineers got my original driver (further extended) merged into the Linux kernel. Since then I only submitted one other patch to add another ID for a compatible chip, because someone managed to send me a datasheet, and I could match it to the one I originally used to implement the driver as having the same behaviour.

Back to the original topic, the IT7236 chip is clearly not a Super I/O controller. It’s also not an Environmental Control (EC) chip, as I know that series is actually IT85xx, which is what my old laptop had. Somewhat luckily though, a “Preliminary Specifications” datasheet for that exact chip is available online from a company that appears to distribute electronics component in the general sense. I’m not sure if that was intentional or not, but having the datasheet is always handy of course.

According to these specifications, the IT7236xFN chips are “Touch ASIC Cap Button Controllers”. And indeed, ITE lists them as such. Comparing this with a different model in the same series shows that probably LED driving was not their original target, but they came to be useful for that. These chips also include an MCU based on a 8051 core, similarly to their EC solution — this makes them, and in particular the datasheet I found earlier, a bit more interesting to me. Unfortunately the datasheet is clearly amended to be the shorter version, and does not include a programming interface description.

Up to this point this tells us exactly one thing only: my driver is completely useless for this chip, as it implements specifically the Super I/O bus access, and it’s unlikely to be extensible to this series of chips. So a new driver is needed and some reverse engineering is likely to be required. The user who wrote me also gave me two other ITE chip names found on the board they have: IT87920 and IT8686 (which appears to be a PWN fan controller — I couldn’t find it on the ITE website at all). Since the it87 (hwmon) driver is still developed out-of-kernel on GitHub, I checked and found an issue that appears to describe a common situation for gaming motherboards: the fans are not controlled with the usual Super I/O chip, but with a separate one (more accurate?) one, and that suggests that the LEDs are indeed controlled by another separate chip, which makes sense. The user ran strings on the UEFI/BIOS image and did indeed find modules named after IT8790 and IT7236 (and IT8728 for whatever reason), to confirm this.

None of this brings us any closer to supporting it though, so let’s take a loop at the datasheet, and we can see that the device has an I²C bus, instead of the LPC (or ISA) bus used by Super I/O and the fan controller. Which meant looking at i2cdev and lsi2c. Unfortunately the output can only see that there are things connected to the bus, but not what they are.

This leaves us pretty much dry. Particularly me since I don’t have hardware access. So my suggestion has been to consider looking into the Windows driver and software (that I’m sure the motherboard manufacturer provides), and possibly figure out if they can run in a virtualized environment (qemu?) where I²C traffic can be inspected. But there may be simpler, more useful or more advanced tools to do most of this already, since I have not spent any time on this particular topic before. So if you know of any of them, feel free to leave a comment on the blog, and I’ll make sure to forward them to the concerned user (since I have not asked them if I can publish their name I’m not going to out them — they can, if they want, leave a comment with their name to be reached directly!).

September 14, 2017
Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
Public Money, Public Code (September 14, 2017, 20:04 UTC)

Imagine that all publicly funded software were under a free license: Everybody would be able to use, study, share and improve it.

I have been waiting for Free Software Foundation Europe to launch the Public Money, Public Code campaign for almost a year now, when first Matthias told me about this being in the works. I have been arguing the same point, although not quite as organized, since back in 2009 when I complained about how the administration of Venice commissioned a GIS application to a company they directly own.

For those who have not seen the campaign yet, the idea is simple: software built with public money (that is, commissioned and paid for by public agencies), should be licensed using a FLOSS license, to make it public code. I like this idea and will support it fully. I even rejoined the Fellowship!

The timing of this campaign ended up resonating with a post on infrastructure projects and their costs, which I find particularly interesting and useful to point out. Unlike the article that is deep-linked there, which lamented of the costs associated with this project, this article focuses on pointing out how that money actually needs to be spent, because for the most part off the shelf Free Software is not really up to the task of complex infrastructure projects.

You may think the post I linked is overly critical of Free Software, and that it’s just a little rough around the edges and everything is okay once you spend some time on it. But that’s exactly what the article is saying! Free Software is a great baseline to build complex infrastructure on top of. This is what all the Cloud companies do, this is what even Microsoft has been doing in the past few years, and it is reasonable to expect most for-profit projects would do that, for a simple reason: you don’t want to spend money working on reinventing the wheel when you can charge for designing an innovative engine — which is a quite simplistic view of course, as sometimes you can invent a more efficient wheel indeed, but that’s a different topic.

Why am I bringing this topic up together with the FSFE campaign? Because I think this is exacly what we should be asking from our governments and public agencies, and the article I linked shows exactly why!

You can’t take off the shelf FLOSS packages and have them run a whole infrastructure, because they usually they are unpolished, and might not scale or require significant work to bring them up to the project required. You will have to spend money to do that, and maybe in some cases it will be cheaper to just not use already existing FLOSS projects at all, and build your own new, innovative wheel. So publicly funded projects need money to produce results, we should not complain about the cost1, but rather demand that the money spent actually produces something that will serve the public in all possible ways, not only with the objective of the project, but also with any byproduct of it, which include the source code.

Most of the products funded with public money are not particularly useful for individuals, or for most for-profit enterprises, but byproducts and improvements may very well be. For example, in the (Italian) post I wrote in 2009 I was complaining about a GIS application that was designed to report potholes and other roadwork problems. In abstract, this is a way to collect and query points of interests (POI), which is the base of many other current services, from review sites, to applications such as Field Trip.

But do we actually care? Sure, by making the code available of public projects, you may now actually be indirectly funding private companies that can reuse that code, and thus be jumpstarted into having applications that would otherwise cost time or money to build from scratch. On the other hand, this is what Free Software has been already about before: indeed, Linux, the GNU libraries and tools, Python, Ruby, and all those tools out there are nothing less than a full kit to quickly start projects that a long time ago would have taken a lot of money or a lot of time to start.

You could actually consider the software byproducts of these project similarly to the public infrastructure that we probably all take from granted: roads, power distribution, communication, and so on. Businesses couldn’t exist without all of this infrastructure, and while it is possible for a private enterprise to set out and build all the infrastructure themselves (road, power lines, fiber), we don’t expect them to do so. Instead we accept that we want more enterprises, because they bring more jobs, more value, and the public investment is part of it.

I actually fear the reason a number of people may disagree with this campaign is rooted in localism — as I said before, I’m a globalist. Having met many people with such ideas, I can hear them in my mind complaining that, to take again the example of the IRIS system in Venice, the Venetian shouldn’t have to pay for something and then give it away for free to Palermo. It’s a strawman, but just because I replaced the city that they complained about when I talked about my idea those eight years ago.

This argument may make sense if you really care about local money being spent locally and not counting on any higher-order funding. But myself I think that public money is public, and I don’t really care if the money from Venice is spent to help reporting potholes in Civitella del Tronto. Actually, I think that cities where the median disposable income is higher have a duty to help providing infrastructure for the smaller, poorer cities at the very least in their immediate vicinity, but overall too.

Unfortunately “public money” may not always be so, even if it appears like that. So I’m not sure if, even if a regulation was passed for publicly funded software development to be released as FLOSS, we’d get a lot in form of public transport infrastructure being open sourced. I would love for it to be though: we’d more easily get federated infrastructure, if they would share the same backend, and if you knew how the system worked you could actually build tools around it, for instance integrating Open Street Map directly with the transport system itself. But I fear this is all wishful thinking and it won’t happen in my lifetime.

There is also another interesting point to make here, which I think I may expand upon, for other contexts, later on. As I said above, I’m all for requiring the software developed with public money to be released to the public with a FLOSS-compatible license. Particularly one that allows using other FLOSS components, and the re-use of even part of the released code into bigger projects. This does not mean that everybody should have a say in what’s going on with that code.

While it makes perfect sense to be able to fix bugs and incompatibilities with websites you need to use as part of your citizen life (in the case of the Venetian GIS I would probably have liked to fix the way they identified the IP address they received the request for), adding new features may actually not be in line with the roadmap of the project itself. Particularly if the public money is already tight rather than lavish, I would surely prefer that they focused on delivering what the project needs and just drop the sources out in compatible licenses, without trying to create a community around them. While the latter would be nice to have, it should not steal the focus on the important part: a lot of this code is currently one-off and is not engineered to be re-used or extensible.

Of course on the long run, if you do have public software available already as open-source, there would be more and more situations where solving the same problem again may become easier, particularly if an option is added there, or a constant string can become a configured value, or translations were possible at all. And in that case, why not have them as features of a single repository, rather than have a lot of separate forks?

But all of this should really be secondary, in my opinion. Let’s focus on getting those sources, they are important, they matter and they can make a difference. Building communities around this will take time. And to be honest, even making these secure will take time. I’m fairly sure that in many cases right now if you do take a look at the software that is running for public services, you can find backdoors, voluntary or not, and even very simple security issues. While the “many eyes” idea is easily disproved, it’s also true that for the most part those projects cut corners, and are very difficult to make sure to begin with.

I want to believe we can do at least this bit.


  1. Okay, so there are case of artificially inflated costs due to friends-of-friends. Those are complicated issues, and I’ll leave them to experts. We should still not be complaining that these projects don’t appear for free. [return]

September 13, 2017
Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
The breadwinner product (September 13, 2017, 12:04 UTC)

This may feel a bit random of a post, as Business and Economics are not my areas of expertise and I usually do my best not talk about stuff I don’t know, but I have seen the complete disregard for this concept lately and I thought it would be a good starting point to define here, before I talk about it, what a “breadwinner product” is, from my point of view.

The term breadwinner is used generally to refer to the primary income-earner in a household. While I have not seen this very often extended to products and services in companies, I think it should be fairly obvious how the extension would work.

In a world of startups there are still plenty of companies that have a real “breadwinner product”, even when acting as startups. This is the case for instance of the company I used to contract out for, in Los Angeles: they have been in activity for a number of years with a different, barely related product, and I was contracting out for their new project.

I think it’s important to think of this term, because without having this concept in mind, it’s hard to understand a lot of business decisions of many companies, why startups such as Revolut are “sweeping up the market” and so on.

This is something that came up on Twitter a time or two: a significant amount of geeks appear to wilfully ignore the needs of a business, and the marketing concepts as words of the devil, and will refuse to try considering whether decisions made business sense, and instead they will either try to judge decisions based purely on technical merits, or even just on their own direct interests. Now it is true that technical merits can make good business sense, but sometimes there are very good long-term vision reasons that people don’t appreciate on the pure technical point of view.

In particular, sometimes it’s hard to understand why a service by a company that may appear as a startup is based on “old” technology, but it may just be the case that it is actually a “traditional” company trying to pivot into a different market or a different brand or level of service. And when that happens, there’s at least some gravity pull to keep the stack in line with the previous technology. Particularly if the new service can piggyback on the old one for a while, both in term of revenue, technology and staff.

So in the case of the company I referred to above, when I started contracting out they were already providing a separate service that was built on a real legacy technology, ran on a stack based on bare metal servers with RedHat 5. Since the new service had two components, one of them ended up being based on the same stack and the other one (which I was setting up) ended up based on Gentoo Linux with containers instead. The same way as the Tinderbox used to be run. If you wonder why one would run two stacks this separate, the answer is that messing with the breadwinner product, most of the time, is a risky endeavour and unless you have a very good reason to do so, you don’t.

So even though I was effectively building a new architecture from scratch, and was setting up new servers, with more proper monitoring (based on Munin and Icinga), and Puppet for configuration management, I was not allowed to touch the old service. And rightly so, as it was definitely brittle and it would have lead to actually losing money, as that service was running in production, while the new one was not ready yet, and the few users of it would be able to be told about maintenance windows in advance.

There is often a tipping point though, when the cost of running a legacy service is higher than the revenue the service is bringing in. For that company that happened right as I was leaving it to start working at my current place of work. The owner though was more business savvy than many other people I met before and since, and was actually already planning how to cut some expenses. Indeed the last thing I helped that company with was setting up a single1 baremetal server with multiple containers to virtualise their former fully bare metal hardware, and bring it physically to a new location (Fremont, CA) to cut on the hosting costs.

The more the breadwinner service is making money, and the less the company is experimenting with alternative approaches to cut the costs in the future or to build up new services or open new market opportunities, the more working for those companies become hard. Of all the possible things I can complain about my boss at the time, ability to deal with business details was not one of those. Actually, I think that despite leaving me in quite the bad predicament afterwards, he did end up teaching me quite a bit of the nitty-gritty details of doing business, particularly US-style — and I may not entirely like it either.

But all in all, I think this is something lots more people in tech should learn about. Because I still maintain that Free Software can only be marketed by businesses and to be able to have your project cater to business users without selling its soul, you need to be able to tell what they need and how they need it provided.


  1. Okay, actually a bit more than one: a single machine ran the production environment for the legacy servers, and acted as warm-backup for the new service; another machine ran the production environment for the new service, and acted as warm-backup for the new service. A pair of the older baremetal servers acted as database backends for both systems. [return]

September 11, 2017
Sven Vermeulen a.k.a. swift (homepage, bugs)
Authenticating with U2F (September 11, 2017, 16:25 UTC)

In order to further secure access to my workstation, after the switch to Gentoo sources, I now enabled two-factor authentication through my Yubico U2F USB device. Well, at least for local access - remote access through SSH requires both userid/password as well as the correct SSH key, by chaining authentication methods in OpenSSH.

Enabling U2F on (Gentoo) Linux is fairly easy. The various guides online which talk about the pam_u2f setup are indeed correct that it is fairly simple. For completeness sake, I've documented what I know on the Gentoo Wiki, as the pam_u2f article.

September 10, 2017
Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)


If you're reading this, the last act in this drama (see the previous blog post) was that in Patras a friendly employee from Hertz picked up the rental car to bring it to a repair workshop. A bit later than planned, but nevertheless. Now the story continues.

  • About 20:00 the same day I get a phone call from the same lady that my car was ready, and we could meet in about 20min next to my hotel so I can pick it up again. That sounded great to me. Some minutes later I saw the car coming.
  • Of course I wanted to try out the repaired roof / window immediately, so we did that. Opened the roof, closed the roof. The passenger side window did not close; precisely the same phenomenon. Oops.
  • I tried a few more times on instruction by the Hertz employee, with the result that the window got stuck at half height and did not move anymore even after shutting down and restarting the ignition. Since it was stuck on the wrong side of its rubber seal, also the passenger door did not open anymore.
  • The visibly nervous Hertz employee calls her manager on the mobile, who arrives after a few minutes. The manager opens the passenger door with application of force. Afterwards, and after restarting the engine, the window slides up again.
  • We have some discussion about a replacement car, where I point out that I paid a lot of money for having a convertible, and really want one. I agree to come to the office thursday morning to sort things out.
  • Next morning, Thursday, at the Hertz office, I'm glad to learn that a replacement car will be sent. Of course, I'm now leaving Patras, so the car will have to be sent to a station near my next stops.
  • We discuss this and agree that I will pick the car up tomorrow (Friday) afternoon in Kalamata (which is only about 80km from my Friday evening hotel in Kyparissia).
Oh well. Glad that things are somehow sorted out. I spend the rest of the day visiting a Mycenaean castle (1200 BC), a Frankish castle (1200 AD), and re-visiting Olympia, spend the night near Olympia, and then start towards Kalamata through the mountains via, e.g. the excavations of ancient Messene. Sometime on the way I realize that the Kalamata Hertz offices (according to the website) are closed 14:00 to 17:00, so I plan with arriving there around 18:00. That's ample time since they should be open 17:00 - 21:00 (search for Kalamata here).
  • Arrive 18:10 at the Kalamata city office. Nobody there, and there's a sign on the door saying "We are at Kalamata Airport."
  • Drive back the ~10km to the airport (which I passed on the way before). Arrive there around 18:30. The entire airport is already closed for the day. No Hertz employees in sight.
  • Call the Kalamata office. First response, "We closed half an hour ago." When I start explaining my problem, the lady on the phone says "But your car has not arrived from Athens yet!" I point out that I have to go back to Kyparissia, quite some way, today. She doesnt know when it will arrive, but says something about late evening. 
  • I tell her I will now get dinner here in Kalamata, and afterwards call her again.
That's where we are now. Just as a reminder, it's now Friday evening, and the problem has essentially been known to Hertz since last Sunday.

Update:
  • Tried calling the Hertz Kalamata office again around 20:45. No response, after a while some mailbox text in Greek. 
  • Drove back the 60km to Kyparissia, arrived at the hotel 22:00. Will call Hertz again tomorrow.
Update 2: Yes, Hertz knows my mobile phone number. It's big and fat on my contract, and I also gave it again and reconfirmed it to the employee at Patras. So, one could assume if something goes wrong they phone me...

Update 3: It ends well. See the next post.

    Fun with Hertz car rentals, part 3 (it ends well) (September 10, 2017, 19:37 UTC)

    If you've read the last part, I had just arrived at my hotel in Kyparissia late in the night, slightly fuming. Well...

    Next morning, saturday, around 10:15 somebody called my mobile phone. For some reason I didn't notice, but only got a text notification of a missed call an hour later. I called back; turns out this was the Kalamata airport Hertz office. "Your replacement car has arrived; you can pick it up anytime."

    I arranged to come by around 16:00 in the afternoon, and from here on everything went smoothly. Now I'm driving a white BMW Mini convertible, and the roof and windows work just fine.

    In the end, obviously I'm quite happy that a replacement car was driven from Athens to Kalamata and that I can now continue with my vacation as planned. The path that lead to that outcome, however, was not so great...

    September 09, 2017
    Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
    Dell XPS 13, problems with WiFi (September 09, 2017, 18:04 UTC)

    A couple of months ago I bought a Dell XPS 13. I’m still very happy with the laptop, particularly given the target use that I have for it, but I have started noticing a list of problems that do bother me more than a little bit.

    The first problem is something that I have spoken of in the original post and updated a couple of times: the firmware (“BIOS”) update. While the firmware is actually published through LVFS by Dell, either Antergos or Arch Linux have some configuration issue with EFI and the System Partition, that cause the EFI shim not to be able to find the right capsule. I ended up just running the update manually twice now, since I didn’t want to spare time to fix the packaging of the firmware updater, and trying with different firmware updates is not easy.

    Also, while the new firmware updates made the electrical whining noise effectively disappear, making the laptop very nice to use in quiet hotel rooms (not all hotel rooms are quiet), it seems to have triggered more WiFi problems. Indeed, it got to the point that I could not use the laptop at home at all. I’m not sure what exactly was the problem, but my Linksys WRT1900ACv2 seems to trigger known problems with the WiFi card on this model.

    At first I thought it would be a problem with using Arch Linux rather than Dell’s own Ubuntu image, that appeared to have separate Qualcomm drivers for the ath10k card. But it turns out the same error pops up repeated in Dell forums and LaunchPad too. A colleague with the same laptop suggested to just replace the card, getting rid of the whole set of problems introduced by the ath10k driver. Indeed, even looking around the Windows users websites, the recommendation appear to be the same: just replace your card.

    The funny bit is that I only really noticed this when I came back from my long August trips, because since I bought the laptop, I hadn’t spent more than a few days at home at that point. I have been in Helsinki, Vancouver and Seattle, used the laptop in airports, lounges, hotels and cafes, as well as my office. And none of those places had any issue with my laptop. I used the laptop extensively to livetweet SREcon Europe from the USENIX wireless at the hotel, and it had no problem whatsoever.

    My current theory for this is that there is some mostly-unused feature that is triggered by high-performance access point like the one I have at home, that runs LEDE, and as such is not something you’ll encounter in the wild. This also would explain why the Windows sites that I found referencing the problem are suggesting the card replacement — your average Windows user is unlikely to know how to do so or interested in a solution that does not involve shipping the device back to Dell, and to be fair they probably have a point, why on earth are they selling laptops with crappy WiFi cards?

    So anyway my solution to this was to order an Intel 8265 wireless card which includes the same 802.11ac dual-band support and Bluetooth 4.2, and is the same format as the ath10k that the laptop comes with. It feels a bit strange having to open up a new laptop to replace a component, but since this is the serviceable version of Dell, it was not a horrible experience (my Vostro laptop still has a terrible 802.11g 2.4GHz-only card on it, but I can’t replace it easily).

    Moving onto something else, the USB-C dock is working great, although I found out the hard way that if you ask Plasma, or whatever else it is that I ended up asking it to, not to put the laptop to sleep the moment the lid is closed, if the power is connected (which I need to make sure I can use the laptop “docked” onto my usual work-from-home setup), it also does not go to sleep if the power is subsequently disconnected. So the short version is that I now usually run the laptop without the power connected unless it’s already running low, and I can easily stay a whole day at a conference without charging, which is great!

    Speaking of charging, turns out that the Apple 65W USB-C charger also works great with the XPS 13. Unfortunately it comes without a cable, and particularly with Apple USB-C cable your mileage may vary. It seems to be fine with the Google Pixel phone cable though. I have not tried measuring how much power and which power mode it uses, among other things because I wouldn’t know how to query the USB-C controller to get that information. If you have suggestions I’m all ears.

    Otherwise the laptop appears to be working great for me. I only wish I could wake it up from sleep without opening it, when using it docked, but that’s also a minor feature.

    The remaining problems are software. For instance Plasma sometimes crashes when I dock the laptop, and the new monitor comes online. And I can’t reboot while docked because the external keyboard (connected on the USB-C dock) is not able to type in the password for the full-disk encryption. Again this is a bother but not a big deal.

    September 08, 2017
    Andreas K. Hüttel a.k.a. dilfridge (homepage, bugs)


    So, I decided to get myself a rather expensive treat this summer. For travelling the Peloponnes I rented a Mini Cooper convertible. These are really cute, and driving around in the sun with the roof open felt like a very nice idea. I'm a Hertz Gold Club customer, so why not go for Hertz again.

    I picked up the car in Athens, all looked fine. The first day I had some longer driving to do, and also the manual was only in Greek, so I decided to drive to my first stop and check out the convertible roof there. OK, with some fiddling I found and read a German manual on the BMW website (now I know where to find the VIN number, if anyone asks :), opened the roof, enjoyed half a day in the mountains near Kalavrita.

    Afterwards the passenger side window didn't close anymore.

    It turns out something was already bent or damaged inside the door, so the window was sliding up on the wrong side of its rubber seal. At some point it can't move any further, so the electronics stops and disables the window. The effect is perfectly reproducible, and scratch marks on the rubber seal and door frame indicate it's been doing that already for a while. Oh well.

    • Phoned the nearest Hertz office in Patras. After some complicated discussion in English they advised me to contact the office in Athens.
    • Phoned the Hertz office in Athens. I managed to explain the problem there. They said I should contact their central technical service office, since maybe they know something easy to do. 
    • Phoned the central technical service office. There the problem was quickly understood; a very helpful lady explained to me that most likely the car would have to be exchanged. Since it was Sunday afternoon, they couldn't do it now, but somebody would call me back on Monday morning 9-10.
    • Waited Monday morning for the call. Nothing happened. 
    • Phoned the central technical service office, Monday around 13:00. They asked me where I was. After telling them I'm going to Patras the next day, they told me I should come by their office there.
    • Arrived at the Patras office tuesday around 17:30. I demonstrated the problem to the lady there. She acknowledged that something's broken, and told me she'd come to my hotel the next day between 11:00 and 12:00 to pick up the car and bring it to the BMW service for repair. 
    • Now I'm sitting in the bar of the hotel, it's 12:30, no one has called or come by, and slowly I'm getting seriously annoyed.
    Let's see how the story continues...
    • Update: 13:00, friendly lady from Hertz picked up the car. Fingers crossed. Made clear it's a long rental, so delaying makes no sense. Wants to phone me either in the afternoon or tomorrow morning.
    • Update 2: The drama continues in the next blog post.

    September 07, 2017
    Hanno Böck a.k.a. hanno (homepage, bugs)
    In Search of a Secure Time Source (September 07, 2017, 15:07 UTC)

    ClockAll our computers and smartphones have an internal clock and need to know the current time. As configuring the time manually is annoying it's common to set the time via Internet services. What tends to get forgotten is that a reasonably accurate clock is often a crucial part of security features like certificate lifetimes or features with expiration times like HSTS. Thus the timesetting should be secure - but usually it isn't.

    I'd like my systems to have a secure time. So I'm looking for a timesetting tool that fullfils two requirements:

    1. It provides authenticity of the time and is not vulnerable to man in the middle attacks.
    2. It is widely available on common Linux systems.

    Although these seem like trivial requirements to my knowledge such a tool doesn't exist. These are relatively loose requirements. One might want to add:
    1. The timesetting needs to provide a good accuracy.
    2. The timesetting needs to be protected against malicious time servers.

    Some people need a very accurate time source, for example for certain scientific use cases. But that's outside of my scope. For the vast majority of use cases a clock that is off by a few seconds doesn't matter. While it's certainly a good idea to consider rogue servers given the current state of things I'd be happy to have a solution where I simply trust a server from Google or any other major Internet entity.

    So let's look at what we have:

    NTP

    The common way of setting the clock is the NTP protocol. NTP itself has no transport security built in. It's a plaintext protocol open to manipulation and man in the middle attacks.

    There are two variants of "secure" NTP. "Autokey", an authenticated variant of NTP, is broken. There's also a symmetric authentication, but that is impractical for widespread use, as it would require to negotiate a pre-shared key with the time server in advance.

    NTPsec and Ntimed

    In response to some vulnerabilities in the reference implementation of NTP two projects started developing "more secure" variants of NTP. Ntimed - a rewrite by Poul-Henning Kamp - and NTPsec, a fork of the original NTP software. Ntimed hasn't seen any development for several years, NTPsec seems active. NTPsec had some controversies with the developers of the original NTP reference implementation and its main developer is - to put it mildly - a controversial character.

    But none of that matters. Both projects don't implement a "secure" NTP. The "sec" in NTPsec refers to the security of the code, not to the security of the protocol itself. It's still just an implementation of the old, insecure NTP.

    Network Time Security

    There's a draft for a new secure variant of NTP - called Network Time Security. It adds authentication to NTP.

    However it's just a draft and it seems stalled. It hasn't been updated for over a year. In any case: It's not widely implemented and thus it's currently not usable. If that changes it may be an option.

    tlsdate

    tlsdate is a hack abusing the timestamp of the TLS protocol. The TLS timestamp of a server can be used to set the system time. This doesn't provide high accuracy, as the timestamp is only given in seconds, but it's good enough.

    I've used and advocated tlsdate for a while, but it has some problems. The timestamp in the TLS handshake doesn't really have any meaning within the protocol, so several implementers decided to replace it with a random value. Unfortunately that is also true for the default server hardcoded into tlsdate.

    Some Linux distributions still ship a package with a default server that will send random timestamps. The result is that your system time is set to a random value. I reported this to Ubuntu a while ago. It never got fixed, however the latest Ubuntu version Zesty Zapis (17.04) doesn't ship tlsdate any more.

    Given that Google has shipped tlsdate for some in ChromeOS time it seems unlikely that Google will send randomized timestamps any time soon. Thus if you use tlsdate with www.google.com it should work for now. But it's no future-proof solution.

    TLS 1.3 removes the TLS timestamp, so this whole concept isn't future-proof. Alternatively it supports using an HTTPS timestamp. The development of tlsdate has stalled, it hasn't seen any updates lately. It doesn't build with the latest version of OpenSSL (1.1) So it likely will become unusable soon.

    OpenNTPDOpenNTPD

    The developers of OpenNTPD, the NTP daemon from OpenBSD, came up with a nice idea. NTP provides high accuracy, yet no security. Via HTTPS you can get a timestamp with low accuracy. So they combined the two: They use NTP to set the time, but they check whether the given time deviates significantly from an HTTPS host. So the HTTPS host provides safety boundaries for the NTP time.

    This would be really nice, if there wasn't a catch: This feature depends on an API only provided by LibreSSL, the OpenBSD fork of OpenSSL. So it's not available on most common Linux systems. (Also why doesn't the OpenNTPD web page support HTTPS?)

    Roughtime

    Roughtime is a Google project. It fetches the time from multiple servers and uses some fancy cryptography to make sure that malicious servers get detected. If a roughtime server sends a bad time then the client gets a cryptographic proof of the malicious behavior, making it possible to blame and shame rogue servers. Roughtime doesn't provide the high accuracy that NTP provides.

    From a security perspective it's the nicest of all solutions. However it fails the availability test. Google provides two reference implementations in C++ and in Go, but it's not packaged for any major Linux distribution. Google has an unfortunate tendency to use unusual dependencies and arcane build systems nobody else uses, so packaging it comes with some challenges.

    One line bash script beats all existing solutions

    As you can see none of the currently available solutions is really feasible and none fulfils the two mild requirements of authenticity and availability.

    This is frustrating given that it's a really simple problem. In fact, it's so simple that you can solve it with a single line bash script:

    date -s "$(curl -sI https://www.google.com/|grep -i 'date:'|sed -e 's/^.ate: //g')"

    This line sends an HTTPS request to Google, fetches the date header from the response and passes that to the date command line utility.

    It provides authenticity via TLS. If the current system time is far off then this fails, as the TLS connection relies on the validity period of the current certificate. Google currently uses certificates with a validity of around three months. The accuracy is only in seconds, so it doesn't qualify for high accuracy requirements. There's no protection against a rogue Google server providing a wrong time.

    Another potential security concern may be that Google might attack the parser of the date setting tool by serving a malformed date string. However I ran american fuzzy lop against it and it looks robust.

    While this certainly isn't as accurate as NTP or as secure as roughtime, it's better than everything else that's available. I put this together in a slightly more advanced bash script called httpstime.

    September 06, 2017
    Greg KH a.k.a. gregkh (homepage, bugs)
    4.14 == This years LTS kernel (September 06, 2017, 14:41 UTC)

    As the 4.13 release has now happened, the merge window for the 4.14 kernel release is now open. I mentioned this many weeks ago, but as the word doesn’t seem to have gotten very far based on various emails I’ve had recently, I figured I need to say it here as well.

    So, here it is officially, 4.14 should be the next LTS kernel that I’ll be supporting with stable kernel patch backports for at least two years, unless it really is a horrid release and has major problems. If so, I reserve the right to pick a different kernel, but odds are, given just how well our development cycle has been going, that shouldn’t be a problem (although I guess I just doomed it now…)

    As always, if people have questions about this, email me and I will be glad to discuss it, or talk to me in person next week at the LinuxCon^WOpenSourceSummit or Plumbers conference in Los Angeles, or at any of the other conferences I’ll be at this year (ELCE, Kernel Recipes, etc.)

    Diego E. Pettenò a.k.a. flameeyes (homepage, bugs)
    A selection of good papers from USENIX Security '17 (September 06, 2017, 10:04 UTC)

    I have briefly talked about Adrienne’s and April’s talk at USENIX Security 2017, but I have not given much light to other papers and presentations that got my attention at the conference. I thought I should do a round up of good content for this conference, and if I can manage, go back to it later.

    First of all, the full proceedings are available on the Program page of the conference. As usual, USENIX open access policy means that everybody has access to these proceedings, and since we’re talking academic papers, effectively everything I’m talking about is available to the public. I know that some videos were recorded, but I’m not sure when they will be published1.

    Before I go into link you to interesting content and give brief comments on them, I would like to start with a complaint about academic papers. The proper name of the conference would be 26th USENIX Security Symposium, and it’s effectively an academic conference. This means that the content is all available in form of papers. These papers are written, as usual, in LaTeX, and available in 2-columns PDFs, as it is usual. Usual, but not practical. This is a perfect format to read the paper when doing so on actual paper. But the truth is that nowadays this content is almost exclusively read in digital form.

    I would love to be able to have an ePub version of the various papersto just load on an ebook reader, for instance2. But even just providing a clear HTML file would be an improvement! When reading these PDFs on a screen, you end up having to zoom in and move around a freaking lot because of the column format, and more than once that would be enough for me to stop caring and not read the paper unless I really have interest in it, and I think this is counterproductive.

    Since I already wrote about Measuring HTTPS Adoption on the Web, I should not go back to that particular presentation. Right after that one, though, Katharina Krombholz presented “I Have No Idea What I’m Doing” - On the Usability of Deploying HTTPS which was definitely interesting to show how complicated still is setting up HTTPS properly, without even going into further advanced features such as HPKP, CSP and similar.

    And speaking of these, an old acquaintance of mine from university time3, Stefano Calzavara, presented CCSP: Controlled Relaxation of Content Security Policies by Runtime Policy Composition (my, what a mouthful!) and I really liked the idea. Effectively the idea behind this is that CSP is too complicated to use and is turning down a significant amount of people from implementing at least the basic parts of security policies. This fits very well with the previous talk, and with my experience. This blog currently depends on a few external resources and scripts, namely Google Analytics, Amazon OneLink, and Font Awesome, and I can’t really spend the time figuring out whether I can make all the changes all the time.

    In the same session as Stefano, Iskander Sanchez-Rola presented Extension Breakdown: Security Analysis of Browsers Extension Resources Control Policies, which easily sounded familiar to me, as it overlaps and extends my own complaint back in 2013 that browser extensions were becoming the next source of entropy for fingerprinting, replacing plugins. Since we had dinner with Stefano, Iskander and Igor (co-author of the paper above), we managed to have quite a chat on the topic. I’m glad to see that my hunches back in the days was not completely off and that there is more interest in fixing this kind of problems nowadays.

    Another interesting area to hear from was the Understanding the Mirai Botnet that revealed one very interesting bit of information: the attack on Dyn that caused a number of outages just last year appears to have as its target not the Dyn service itself but rather Sony PlayStation Network, and should thus be looked at in the light of the previous attacks to that. This should remind to everyone that just because you get something out personally from a certain attack, you should definitely not cheer on them; you may be the next target, even just as a bystander.

    Now, not all the talks were exceptional. In particular, I found See No Evil, Hear No Evil, Feel No Evil, Print No Evil? Malicious Fill Patterns Detection in Additive Manufacturing a bit… hypy. In the sense that the whole premise of considering 3D-printed sourcing as trusted by default, and then figure out a minimal amount of validation seemed to be stemming from the crowd that has been insisting that 3D printing is the future, for the past ten years or so. While it clearly is interesting, and it has a huge amount of use for prototyping, one-off designs and even cosplay, it does not seem like it got as far as people kept thinking it would. And at least from the talk and skimming the paper I couldn’t find a good explanation of how it compares against “classic” manufacturing trust.

    On a similar note I found not particularly enticing the out-of-band call verification system proposed by AuthentiCall: Efficient Identitiy and Content Authentication for Phone Calls which appears to leave out all the details of identity verification and trust system. And assumes a fairly North American point of view on the communication space.

    Of course I was interested in the talk about mobile payments, Picking Up My Tab: Understanding and Mitigating Synchronized Token Lifting and Spending in Mobile Payment, given my previous foray into related topics. It was indeed good, although the final answer of adding a QR-code to do a two-way verification of who it is you’re going to pay sounds like a NIH implementation of the EMV protocol. It is worth it to read to figure out the absurd implementation of Magnetic Secure Transmission that is used in Samsung Pay implementation: spoilers, it implements magnetic stripe payments through a mobile phone.

    For the less academic of you, TrustBase: An Architecture to Repair and Strengthen Certificate-based Authentication appears fairly interesting, particularly as the source code is available. The idea is to move the implementation of SSL clients into an operating system service, rather than into libraries, so that it can be configured once and for all at the system level, including selecting the available cipher to use and the Authorities to trust. It sounds good, but at the same time it sounds a lot like what NSS (the Mozilla one, not the glibc one) tried to implement. Except that didn’t go anywhere, not just because of API differences.

    But it can’t be an interesting post (or conference) without a bit of controversy. A Longitudinal, End-to-End View of the DNSSEC Ecosystem has been an interesting talk, and one that once again confirmed the fears around the lack of proper DNSSEC support in the wild right now. But in that very same talk, the presenter pointed out how they used a service Luminati to get access to endpoints within major ISPs networks to test their DNSSEC resolution. While I understand why a similar service would be useful in these circumstances, I need to remind people that the Luminati service is not one of the good guys!

    Indeed, Luminati is described as allowing you to request access to connections following certain characteristics. What it omits to say, is that it does so by targeting connections of users who installed the Hola “VPN” tool. If you haven’t come across this, Hola is one of the many extensions that allowed users to appear as if connecting from a different country to fool Netflix and other streaming services. Beside being against terms of services (but who cares, right?), in 2015 Hola was found to be compromising its users. In particular, the users running Hola are running the equivalent of a Tor exit node, without any of the security measures to protect its users, and – because its target is non-expert users who are trying to watch content not legally available in their country – without a good understanding of what such an exit node allows.

    I cannot confirm whether currently they still allow access to the full local network to the users of the “commercial” service, which include router configuration pages (cough DNS hijacking cough), and local office LANs that are usually trusted more than they should be. But it gives you quite an idea, as that was clearly the case before.

    So here is my personal set of opinions and a number of pointers to good and interesting talks and papers. I just wish they would be more usable by the non-academics by not being forced only in LaTeX format, but I’m afraid the two worlds shall never meet enough.


    1. As it turns out you can blame me a little bit for this part, I promised to help out. [return]
    2. Thankfully, for USENIX conferences, the full proceedings are available as ePub and Mobi. Although the size is big enough that you can’t use the mail-to-Kindle feature. [return]
    3. All the two weeks I managed to stay in it. [return]

    September 05, 2017
    Hanno Böck a.k.a. hanno (homepage, bugs)
    Abandoned Domain Takeover as a Web Security Risk (September 05, 2017, 17:11 UTC)

    In the modern web it's extremely common to include thirdparty content on web pages. Youtube videos, social media buttons, ads, statistic tools, CDNs for fonts and common javascript files - there are plenty of good and many not so good reasons for this. What is often forgotten is that including other peoples content means giving other people control over your webpage. This is obviously particularly risky if it involves javascript, as this gives a third party full code execution rights in the context of your webpage.

    I recently helped a person whose Wordpress blog had a problem: The layout looked broken. The cause was that the theme used a font from a web host - and that host was down. This was easy to fix. I was able to extract the font file from the Internet Archive and store a copy locally. But it made me thinking: What happens if you include third party content on your webpage and the service from which you're including it disappears?

    I put together a simple script that would check webpages for HTML tags with the src attribute. If the src attribute points to an external host it checks if the host name actually can be resolved to an IP address. I ran that check on the Alexa Top 1 Million list. It gave me some interesting results. (This methodology has some limits, as it won't discover indirect src references or includes within javascript code, but it should be good enough to get a rough picture.)

    Yahoo! Web Analytics was shut down in 2012, yet in 2017 Flickr still tried to use it

    The webpage of Flickr included a script from Yahoo! Web Analytics. If you don't know Yahoo Analytics - that may be because it's been shut down in 2012. Although Flickr is a Yahoo! company it seems they haven't noted for quite a while. (The code is gone now, likely because I mentioned it on Twitter.) This example has no security impact as the domain still belongs to Yahoo. But it likely caused an unnecessary slowdown of page loads over many years.

    Going through the list of domains I saw plenty of the things you'd expect: Typos, broken URLs, references to localhost and subdomains no longer in use. Sometimes I saw weird stuff, like references to javascript from browser extensions. My best explanation is that someone had a plugin installed that would inject those into pages and then created a copy of the page with the browser which later endet up being used as the real webpage.

    I looked for abandoned domain names that might be worth registering. There weren't many. In most cases the invalid domains were hosts that didn't resolve, but that still belonged to someone. I found a few, but they were only used by one or two hosts.

    Takeover of unregistered Azure subdomain

    But then I saw a couple of domains referencing a javascript from a non-resolving host called piwiklionshare.azurewebsites.net. This is a subdomain from Microsoft's cloud service Azure. Conveniently Azure allows creating test accounts for free, so I was able to grab this subdomain without any costs.

    Doing so allowed me to look at the HTTP log files and see what web pages included code from that subdomain. All of them were local newspapers from the US. 20 of them belonged to two adjacent IP addresses, indicating that they were all managed by the same company. I was able to contact them. While I never received any answer, shortly afterwards the code was gone from all those pages.

    Saline Courier defacement
    "Friendly defacement" of the Saline Courier.
    However the page with most hits was not so easy to contact. It was also a newspaper, the Saline Courier. I tried contacting them directly, their chief editor and their second chief editor. No answer.

    After a while I wondered what I could do. Ultimately at some point Microsoft wouldn't let me host that subdomain any longer for free. I didn't want to risk that others could grab that subdomain, but at the same time I obviously also didn't want to pay in order to keep some web page safe whose owners didn't even bother to read my e-mails.

    But of course I had another way of contacting them: I could execute Javascript on their web page and use that for some friendly defacement. After some contemplating whether that would be a legitimate thing to do I decided to go for it. I changed the background color to some flashy pink and send them a message. The page remained usable, but it was a message hard to ignore.

    With some trouble on the way - first they broke their CSS, then they showed a PHP error message, then they reverted to the page with the defacement. But in the end they managed to remove the code.

    There are still a couple of other pages that include that Javascript. Most of them however look like broken test webpages. The only legitimately looking webpage that still embeds that code is the Columbia Missourian. However they don't embed it on the start page, only on the error reporting form they have for every article. It's been several weeks now, they don't seem to care.

    What happens to abandoned domains?

    There are reasons to believe that what I showed here is only the tip of the iceberg. In many cases when services discontinue their domains don't simply disappear. If the domain name is valuable then almost certainly someone will try to register it immediately after it becomes available.

    Someone trying to abuse abandoned domains could watch out for services going ot of business or widely referenced domains becoming available. Just to name an example: I found a couple of hosts referencing subdomains of compete.com. If you go to their web page you can learn that the company Compete has discontinued its service in 2016. How long will they keep their domain? And what will happen with it afterwards? Whoever gets the domain can hijack all the web pages that still include javascript from it.

    Be sure to know what you include

    There are some obvious takeaways from this. If you include other peoples code on your web page then you should know what that means: You give them permission to execute whatever they want on your web page. This means you need to wonder how much you can trust them.

    At the very least you should be aware who is allowed to execute code on your web page. If they shut down their business or discontinue the service you have been using then you obviously should remove that code immediately. And if you include code from a web statistics service that you never look at anyway you may simply want to remove that as well.

    September 04, 2017
    Arun Raghavan a.k.a. ford_prefect (homepage, bugs)
    A Late GUADEC 2017 Post (September 04, 2017, 03:20 UTC)

    It’s been a little over a month since I got back from Manchester, and this post should’ve come out earlier but I’ve been swamped.

    The conference was absolutely lovely, the organisation was a 110% on point (serious kudos, I know first hand how hard that is). Others on Planet GNOME have written extensively about the talks, the social events, and everything in between that made it a great experience. What I would like to write about is about why this year’s GUADEC was special to me.

    GNOME turning 20 years old is obviously a large milestone, and one of the main reasons I wanted to make sure I was at Manchester this year. There were many occasions to take stock of how far we had come, where we are, and most importantly, to reaffirm who we are, and why we do what we do.

    And all of this made me think of my own history with GNOME. In 2002/2003, Nat and Miguel came down to Bangalore to talk about some of the work they were doing. I know I wasn’t the only one who found their energy infectious, and at Linux Bangalore 2003, they got on stage, just sat down, and started hacking up a GtkMozEmbed-based browser. The idea itself was fun, but what I took away — and I know I wasn’t the only one — is the sheer inclusive joy they shared in creating something and sharing that with their audience.

    For all of us working on GNOME in whatever way we choose to contribute, there is the immediate gratification of shaping this project, as well as the larger ideological underpinning of making everyone’s experience talking to their computers better and free-er.

    But I think it is also important to remember that all our efforts to make our community an inviting and inclusive space have a deep impact across the world. So much so that complete strangers from around the world are able to feel a sense of belonging to something much larger than themselves.

    I am excited about everything we will achieve in the next 20 years.

    (thanks go out to the GNOME Foundation for helping me attend GUADEC this year)

    Sponsored by GNOME!

    August 26, 2017
    Sebastian Pipping a.k.a. sping (homepage, bugs)
    GIMP 2.9.6 now in Gentoo (August 26, 2017, 19:53 UTC)

    Here’s what upstream has to say about the new release 2.9.6. Enjoy 🙂

    August 23, 2017
    Sven Vermeulen a.k.a. swift (homepage, bugs)
    Using nVidia with SELinux (August 23, 2017, 17:04 UTC)

    Yesterday I've switched to the gentoo-sources kernel package on Gentoo Linux. And with that, I also attempted (succesfully) to use the propriatary nvidia drivers so that I can enjoy both a smoother 3D experience while playing minecraft, as well as use the CUDA support so I don't need to use cloud-based services for small exercises.

    The move to nvidia was quite simple, as the nvidia-drivers wiki article on the Gentoo wiki was quite easy to follow.

    Sebastian Pipping a.k.a. sping (homepage, bugs)
    Expat 2.2.4 released (August 23, 2017, 16:52 UTC)

    Expat 2.2.4 has recently been released. It features one major bugfix regarding files encoded as UTF-8, and improvements to the build system.

    If you are using a more ancient version of Visual Studio like 2012, please check the post-2.2.4 commits in Git for related fixes to compilation.

    Also, founding of Rhodri’s work on Expat by the Core Infrastructure Initiative is coming to an end. If you can fund additional developers for work on Expat — including smooth integration of by-default protection against billion laughs denial-of-service attacks — please get in touch.

    Sebastian Pipping

    August 22, 2017
    Sven Vermeulen a.k.a. swift (homepage, bugs)
    Switch to Gentoo sources (August 22, 2017, 17:04 UTC)

    You've might already read it on the Gentoo news site, the Hardened Linux kernel sources are removed from the tree due to the grsecurity change where the grsecurity Linux kernel patches are no longer provided for free. The decision was made due to supportability and maintainability reasons.

    That doesn't mean that users who want to stick with the grsecurity related hardening features are left alone. Agostino Sarubbo has started providing sys-kernel/grsecurity-sources for the users who want to stick with it, as it is based on minipli's unofficial patchset. I seriously hope that the patchset will continue to be maintained and, who knows, even evolve further.

    Personally though, I'm switching to the Gentoo sources, and stick with SELinux as one of the protection measures. And with that, I might even start using my NVidia graphics card a bit more, as that one hasn't been touched in several years (I have an Optimus-capable setup with both an Intel integrated graphics card and an NVidia one, but all attempts to use nouveau for the one game I like to play - minecraft - didn't work out that well).

    Alexys Jacob a.k.a. ultrabug (homepage, bugs)
    py3status v3.6 (August 22, 2017, 06:00 UTC)

    After four months of cool contributions and hard work on normalization and modules’ clean up, I’m glad to announce the release of py3status v3.6!

    Milestone 3.6 was mainly focused about existing modules, from their documentation to their usage of the py3 helper to streamline their code base.

    Other improvements were made about error reporting while some sneaky bugs got fixed along the way.

    Highlights

    Not an extensive list, check the changelog.

    • LOTS of modules streamlining (mainly the hard work of @lasers)
    • error reporting improvements
    • py3-cmd performance improvements

    New modules

    • i3blocks support (yes, py3status can now wrap i3blocks thanks to @tobes)
    • cmus module: to control your cmus music player, by @lasers
    • coin_market module: to display custom cryptocurrency data, by @lasers
    • moc module: to control your moc music player, by @lasers

    Milestone 3.7

    This milestone will give a serious kick into py3status performance. We’ll do lots of profiling and drastic work to reduce py3status CPU and memory footprints!

    For now we’ve been relying a lot on threads, which is simple to operate but not that CPU/memory friendly. Since i3wm users rightly care for their efficiency we think it’s about time we address this kind of points in py3status.

    Stay tuned, we have some nice ideas in stock 🙂

    Thanks contributors!

    This release is their work, thanks a lot guys!

    • aethelz
    • alexoneill
    • armandg
    • Cypher1
    • docwalter
    • enguerrand
    • fmorgner
    • guiniol
    • lasers
    • markrileybot
    • maximbaz
    • tablet-mode
    • paradoxisme
    • ritze
    • rixx
    • tobes
    • valdur55
    • vvoland
    • yabbes

    August 19, 2017
    Hardened Linux kernel sources removal (August 19, 2017, 00:00 UTC)

    As you may know the core of sys-kernel/hardened-sources has been the grsecurity patches. Recently the grsecurity developers have decided to limit access to these patches. As a result, the Gentoo Hardened team is unable to ensure a regular patching schedule and therefore the security of the users of these kernel sources. Thus, we will be masking hardened-sources on the 27th of August and will proceed to remove them from the main ebuild repository by the end of September. We recommend to use sys-kernel/gentoo-sources instead. Userspace hardening and support for SELinux will of course remain in the Gentoo ebuild repository. Please see the full news item for additional information and links.

    August 18, 2017

    FroSCon logo

    Upcoming weekend, 19-20th August 2017, there will be a Gentoo booth again at the FrOSCon “Free and Open Source Conference” 12, in St. Augustin near Bonn! Visitors can see Gentoo live in action, get Gentoo swag, and prepare, configure, and compile their own Gentoo buttons. See you there!

    August 12, 2017
    Luca Barbato a.k.a. lu_zero (homepage, bugs)
    Optimizing rust (August 12, 2017, 19:16 UTC)

    After the post about optimization, Kostya and many commenters (me included) discussed a bit about if there are better ways to optimize that loop without using unsafe code.

    Kostya provided me with a test function and multiple implementations from him and I polished and benchmarked the whole thing.

    The code

    I put the code in a simple project, initially it was a simple main.rs and then it grew a little.

    All it started with this function:

    pub fn recombine_plane_reference(
        src: &[i16],
        sstride: usize,
        dst: &mut [u8],
        dstride: usize,
        w: usize,
        h: usize,
    ) {
        let mut idx0 = 0;
        let mut idx1 = w / 2;
        let mut idx2 = (h / 2) * sstride;
        let mut idx3 = idx2 + idx1;
        let mut oidx0 = 0;
        let mut oidx1 = dstride;
    
        for _ in 0..(h / 2) {
            for x in 0..(w / 2) {
                let p0 = src[idx0 + x];
                let p1 = src[idx1 + x];
                let p2 = src[idx2 + x];
                let p3 = src[idx3 + x];
                let s0 = p0.wrapping_add(p2);
                let d0 = p0.wrapping_sub(p2);
                let s1 = p1.wrapping_add(p3);
                let d1 = p1.wrapping_sub(p3);
                let o0 = s0.wrapping_add(s1).wrapping_add(2);
                let o1 = d0.wrapping_add(d1).wrapping_add(2);
                let o2 = s0.wrapping_sub(s1).wrapping_add(2);
                let o3 = d0.wrapping_sub(d1).wrapping_add(2);
                dst[oidx0 + x * 2 + 0] = clip8(o0.wrapping_shr(2).wrapping_add(128));
                dst[oidx0 + x * 2 + 1] = clip8(o1.wrapping_shr(2).wrapping_add(128));
                dst[oidx1 + x * 2 + 0] = clip8(o2.wrapping_shr(2).wrapping_add(128));
                dst[oidx1 + x * 2 + 1] = clip8(o3.wrapping_shr(2).wrapping_add(128));
            }
            idx0 += sstride;
            idx1 += sstride;
            idx2 += sstride;
            idx3 += sstride;
            oidx0 += dstride * 2;
            oidx1 += dstride * 2;
        }
    }
    

    Benchmark

    Kostya used perf to measure the number of samples it takes over a large number of iterations, I wanted to make the benchmark a little more portable so I used the time::PreciseTime Rust provides to measure something a little more coarse, but good enough for our purposes.

    We want to see if rewriting the loop using unsafe pointers or using high level iterators provides a decent speedup, no need to be overly precise.

    NB: I decided to not use the bencher utility provided with nightly rust to make the code even easier to use.

    +fn benchme<F>(name: &str, n: usize, mut f: F)
    +    where F : FnMut() {
    +    let start = PreciseTime::now();
    +    for _ in 0..n {
    +        f();
    +    }
    +    let end = PreciseTime::now();
    +    println!("Runtime {} {}", name, start.to(end));
    +}
    
    # cargo run --release
    

    Unsafe code

    Both me and Kostya have a C background so for him (and for me), was sort of natural embracing unsafe {} and use the raw pointers like we are used to.

    pub fn recombine_plane_unsafe(
        src: &[i16],
        sstride: usize,
        dst: &mut [u8],
        dstride: usize,
        w: usize,
        h: usize,
    ) {
        unsafe {
            let hw = (w / 2) as isize;
            let mut band0 = src.as_ptr();
            let mut band1 = band0.offset(hw);
            let mut band2 = band0.offset(((h / 2) * sstride) as isize);
            let mut band3 = band2.offset(hw);
            let mut dst0 = dst.as_mut_ptr();
            let mut dst1 = dst0.offset(dstride as isize);
            let hh = (h / 2) as isize;
            for _ in 0..hh {
                let mut b0_ptr = band0;
                let mut b1_ptr = band1;
                let mut b2_ptr = band2;
                let mut b3_ptr = band3;
                let mut d0_ptr = dst0;
                let mut d1_ptr = dst1;
                for _ in 0..hw {
                    let p0 = *b0_ptr;
                    let p1 = *b1_ptr;
                    let p2 = *b2_ptr;
                    let p3 = *b3_ptr;
                    let s0 = p0.wrapping_add(p2);
                    let s1 = p1.wrapping_add(p3);
                    let d0 = p0.wrapping_sub(p2);
                    let d1 = p1.wrapping_sub(p3);
                    let o0 = s0.wrapping_add(s1).wrapping_add(2);
                    let o1 = d0.wrapping_add(d1).wrapping_add(2);
                    let o2 = s0.wrapping_sub(s1).wrapping_add(2);
                    let o3 = d0.wrapping_sub(d1).wrapping_add(2);
                    *d0_ptr.offset(0) = clip8((o0 >> 2).wrapping_add(128));
                    *d0_ptr.offset(1) = clip8((o1 >> 2).wrapping_add(128));
                    *d1_ptr.offset(0) = clip8((o2 >> 2).wrapping_add(128));
                    *d1_ptr.offset(1) = clip8((o3 >> 2).wrapping_add(128));
                    b0_ptr = b0_ptr.offset(1);
                    b1_ptr = b1_ptr.offset(1);
                    b2_ptr = b2_ptr.offset(1);
                    b3_ptr = b3_ptr.offset(1);
                    d0_ptr = d0_ptr.offset(2);
                    d1_ptr = d1_ptr.offset(2);
                }
                band0 = band0.offset(sstride as isize);
                band1 = band1.offset(sstride as isize);
                band2 = band2.offset(sstride as isize);
                band3 = band3.offset(sstride as isize);
                dst0 = dst0.offset((dstride * 2) as isize);
                dst1 = dst1.offset((dstride * 2) as isize);
            }
        }
    }
    

    The function is faster than baseline:

        Runtime reference   PT1.598052169S
        Runtime unsafe      PT1.222646190S
    

    Explicit upcasts

    Kostya noticed that telling rust to use i32 instead of i16 gave some performance boost.

        Runtime reference       PT1.601846926S
        Runtime reference 32bit PT1.371876242S
        Runtime unsafe          PT1.223115917S
        Runtime unsafe 32bit    PT1.124667021S
    

    I’ll keep variants between i16 and i32 to see when it is important and when it is not.

    Note: Making code generic over primitive types is currently pretty painful and hopefully will be fixed in the future.

    High level abstractions

    Most of the comments to Kostya’s original post were about leveraging the high level abstractions to make the compiler understand the code better.

    Use Iterators

    Rust is able to omit the bound checks if there is a warranty that the code cannot go out of the array boundaries. Using Iterators instead of for loops over an external variables should do the trick.

    Use Chunks

    chunks and chunks_mut take a slice and provides a nice iterator that gets you at-most-N-sized pieces of the input slice.

    Since that the code works by line it is sort of natural to use it.

    Use split_at

    split_at and split_at_mut get you independent slices, even mutable. The code is writing two lines at time so having the ability to access mutably two regions of the frame is a boon.

    The (read-only) input is divided in bands and the output produced is 2 lines at time. split_at is much better than using hand-made slicing and
    split_at_mut is perfect to write at the same time the even and the odd line.

    All together

    pub fn recombine_plane_chunks_32(
        src: &[i16],
        sstride: usize,
        dst: &mut [u8],
        dstride: usize,
        w: usize,
        h: usize,
    ) {
        let hw = w / 2;
        let hh = h / 2;
        let (src1, src2) = src.split_at(sstride * hh);
        let mut src1i = src1.chunks(sstride);
        let mut src2i = src2.chunks(sstride);
        let mut dstch = dst.chunks_mut(dstride * 2);
        for _ in 0..hh {
            let s1 = src1i.next().unwrap();
            let s2 = src2i.next().unwrap();
            let mut d = dstch.next().unwrap();
            let (mut d0, mut d1) = d.split_at_mut(dstride);
            let (b0, b1) = s1.split_at(hw);
            let (b2, b3) = s2.split_at(hw);
            let mut di0 = d0.iter_mut();
            let mut di1 = d1.iter_mut();
            let mut bi0 = b0.iter();
            let mut bi1 = b1.iter();
            let mut bi2 = b2.iter();
            let mut bi3 = b3.iter();
            for _ in 0..hw {
                let p0 = bi0.next().unwrap();
                let p1 = bi1.next().unwrap();
                let p2 = bi2.next().unwrap();
                let p3 = bi3.next().unwrap();
                recombine_core_32(*p0, *p1, *p2, *p3, &mut di0, &mut di1);
            }
        }
    }
    

    It is a good improvement over the reference baseline, but still not as fast as unsafe.

        Runtime reference       PT1.621158410S
        Runtime reference 32bit PT1.467441931S
        Runtime unsafe          PT1.226046003S
        Runtime unsafe 32bit    PT1.126615305S
        Runtime chunks          PT1.349947181S
        Runtime chunks 32bit    PT1.350027322S
    

    Use of zip or izip

    Using next().unwrap() feels clumsy and force the iterator to be explicitly mutable. The loop can be written in a nicer way using the system provided zip and the itertools-provided izip.

    zip works fine for 2 iterators, then you start piling up (so, (many, (tuples, (that, (feels, lisp))))) (or (feels (lisp, '(so, many, tuples))) according to a reader). izip flattens the result so it is sort of nicers.

    pub fn recombine_plane_zip_16(
        src: &[i16],
        sstride: usize,
        dst: &mut [u8],
        dstride: usize,
        w: usize,
        h: usize,
    ) {
        let hw = w / 2;
        let hh = h / 2;
        let (src1, src2) = src.split_at(sstride * hh);
        let src1i = src1.chunks(sstride);
        let src2i = src2.chunks(sstride);
        let mut dstch = dst.chunks_mut(dstride * 2);
        for (s1, s2) in src1i.zip(src2i) {
            let mut d = dstch.next().unwrap();
            let (mut d0, mut d1) = d.split_at_mut(dstride);
            let (b0, b1) = s1.split_at(hw);
            let (b2, b3) = s2.split_at(hw);
            let mut di0 = d0.iter_mut();
            let mut di1 = d1.iter_mut();
            let iterband = b0.iter().zip(b1.iter().zip(b2.iter().zip(b3.iter())));
            for (p0, (p1, (p2, p3))) in iterband {
                recombine_core_16(*p0, *p1, *p2, *p3, &mut di0, &mut di1);
            }
        }
    }
    

    How they would fare?

        Runtime reference        PT1.614962959S
        Runtime reference 32bit  PT1.369636641S
        Runtime unsafe           PT1.223157417S
        Runtime unsafe 32bit     PT1.125534521S
        Runtime chunks           PT1.350069795S
        Runtime chunks 32bit     PT1.381841742S
        Runtime zip              PT1.249227707S
        Runtime zip 32bit        PT1.094282423S
        Runtime izip             PT1.366320546S
        Runtime izip 32bit       PT1.208708213S
    

    Pretty well.

    Looks like izip is a little more wasteful than zip currently, so looks like we have a winner 🙂

    Conclusions

    • Compared to common imperative programming patterns, using the high level abstractions does lead to a nice speedup: use iterators when you can!
    • Not all the abstractions cost zero, zip made the overall code faster while izip lead to a speed regression.
    • Do benchmark your time critical code. nightly has some facility for it BUT it is not great for micro-benchmarks.

    Overall I’m enjoying a lot writing code in Rust.

    August 08, 2017
    Alexys Jacob a.k.a. ultrabug (homepage, bugs)
    ScyllaDB meets Gentoo Linux (August 08, 2017, 14:19 UTC)

    I am happy to announce that my work on packaging ScyllaDB for Gentoo Linux is complete!

    Happy or curious users are very welcome to share their thoughts and ping me to get it into portage (which will very likely happen).

    Why Scylla?

    Ever heard of the Cassandra NoSQL database and Java GC/Heap space problems?… if you do, you already get it 😉

    I will not go into the details as their website does this way better than me but I got interested into Scylla because it fits the Gentoo Linux philosophy very well. If you remember my writing about packaging Rethinkdb for Gentoo Linux, I think that we have a great match with Scylla as well!

    • it is written in C++ so it plays very well with emerge
    • the code quality is so great that building it does not require heavy patching on the ebuild (feels good to be a packager)
    • the code relies on system libs instead of bundling them in the sources (hurrah!)
    • performance tuning is handled by smart scripting and automation, allowing the relationship between the project and the hardware is strong

    I believe that these are good enough points to go further and that such a project can benefit from a source based distribution like Gentoo Linux. Of course compiling on multiple systems is a challenge for such a database but one does not improve by staying in their comfort zone.

    Upstream & contributions

    Packaging is a great excuse to get to know the source code of a project but more importantly the people behind it.

    So here I got to my first contributions to Scylla to get Gentoo Linux as a detected and supported Linux distribution in the different scripts and tools used to automatically setup the machine it will run upon (fear not, I contributed bash & python, not C++)…

    Even if I expected to contribute using Github PRs and got to change my habits to a git-patch+mailing list combo, I got warmly welcomed and received positive and genuine interest in the contributions. They got merged quickly and thanks to them you can install and experience Scylla in Gentoo Linux without heavy patching on our side.

    Special shout out to Pekka, Avi and Vlad for their welcoming and insightful code reviews!

    I’ve some open contributions about pushing further on the python code QA side to get the tools to a higher level of coding standards. Seeing how upstream is serious about this I have faith that it will get merged and a good base for other contributions.

    Last note about reaching them is that I am a bit sad that they’re not using IRC freenode to communicate (I instinctively joined #scylla and found myself alone) but they’re on Slack (those “modern folks”) and pretty responsive to the mailing lists 😉

    Java & Scylla

    Even if scylla is a rewrite of Cassandra in C++, the project still relies on some external tools used by the Cassandra community which are written in Java.

    When you install the scylla package on Gentoo, you will see that those two packages are Java based dependencies:

    • app-admin/scylla-tools
    • app-admin/scylla-jmx

    It pained me a lot to package those (thanks to help of @monsieurp) but they are building and working as expected so this gets the packaging of the whole Scylla project pretty solid.

    emerge dev-db/scylla

    The scylla packages are located in the ultrabug overlay for now until I test them even more and ultimately put them in production. Then they’ll surely reach the portage tree with the approval of the Gentoo java team for the app-admin/ packages listed above.

    I provide a live ebuild (scylla-9999 with no keywords) and ebuilds for the latest major version (2.0_rc1 at time of writing).

    It’s as simple as:

    $ sudo layman -a ultrabug
    $ sudo emerge -a dev-db/scylla
    $ sudo emerge --config dev-db/scylla

    Try it out and tell me what you think, I hope you’ll start considering and using this awesome database!

    August 06, 2017
    Sebastian Pipping a.k.a. sping (homepage, bugs)

    Update: I moved to disroot.org now.

    August 02, 2017
    Sebastian Pipping a.k.a. sping (homepage, bugs)

    Just a quick note that Expat 2.2.3 has been released. For Windows users, it fixes DLL hijacking (CVE-2017-11742). On Linux, extracting entropy for Hash DoS protection no longer blocks, which affected D-Bus and systems that are low on entropy early in the boot process. For more details, please check the change log.

    July 27, 2017
    Nathan Zachary a.k.a. nathanzachary (homepage, bugs)

    Last evening, I ran some updates on one of my servers. One of the updates was from MariaDB 10.1 to 10.2 (some minor release as well). After compiling, I went to restart, but it failed with:

    # /etc/init.d/mysql start
    * Checking mysqld configuration for mysql ...
    [ERROR] Can't find messagefile '/usr/share/mysql/errmsg.sys'
    [ERROR] Aborting

    * mysql config check failed [ !! ]
    * ERROR: mysql failed to start

    I’m not sure why this just hit me now, but it looks like it is a function within the init script that’s causing it to look for files in the nonexistent directory of /usr/share/mysql/ instead of the appropriate /usr/share/mariadb/. The fast fix here (so that I could get everything back up and running as quickly as possible) was to simply symlink the directory:

    cd /usr/share
    ln -s mariadb/ mysql

    Thereafter, MariaDB came up without any problem:

    # /etc/init.d/mysql start
    * Caching service dependencies ... [ ok ]
    * Checking mysqld configuration for mysql ... [ ok ]
    * Starting mysql ... [ ok ]
    # /etc/init.d/mysql status
    * status: started

    I hope that information helps if you’re in a pinch and run into the same error message.

    Cheers,
    Zach

    UPDATE: It seems as if the default locations for MySQL/MariaDB configurations have changed (in Gentoo). Please see this comment for more information about a supportable fix for this problem moving forward. Thanks to Brian Evans for the information. 🙂

    July 23, 2017
    Michał Górny a.k.a. mgorny (homepage, bugs)
    Optimizing ccache using per-package caches (July 23, 2017, 18:03 UTC)

    ccache can be of great assistance to Gentoo developers and users who frequently end up rebuilding similar versions of packages. By providing a caching compiler frontend, it can speed up builds by removing the need to build files that have not changed again. However, it uses a single common cache directory by default which can be suboptimal even if you are explicitly enabling ccache only for a subset of packages needing that.

    The likeliness of cross-package ccache hits is pretty low — majority of the hits occurs within a single package. If you use a single cache directory for all affected packages, it grows pretty quick. Besides a possible performance hit from having a lot of files in every directory, this means that packages built later can shift earlier packages out of the cache, resulting in meaninglessly lost cache hits. A simple way to avoid both of the problems is to use separate ccache directories.

    In my solution, a separate subdirectory of /var/cache/ccache is used for every package, named after the category, package name and slot. While the last one is not strictly necessary, it can be useful for slotted packages such as LLVM where I do not want frequently changing live package sources to shift the release versions out of the cache.

    To use it, put a code similar to the following in your /etc/portage/bashrc:

    if [[ ${FEATURES} == *ccache* && ${EBUILD_PHASE_FUNC} == src_* ]]; then
    	if [[ ${CCACHE_DIR} == /var/cache/ccache ]]; then
    		export CCACHE_DIR=/var/cache/ccache/${CATEGORY}/${PN}:${SLOT}
    		mkdir -p "${CCACHE_DIR}" || die
    	fi
    fi

    The first condition makes sure the code is only run when ccache is enabled, and only for src_* phases where we can rely on userpriv being used consistently. The second one makes sure the code only applies to a specific (my initial) value of CCACHE_DIR and therefore avoids both nesting the cache indefinitely when Portage calls subsequent phase functions, and applying the replacement if user overrides CCACHE_DIR.

    You need to either adjust the value used here to the directory used on your system, or change it in your /etc/portage/make.conf:

    CCACHE_DIR="/var/cache/ccache"

    Once this is done, Portage should start creating separate cache directories for every package where you enable ccache. This should improve the cache hit ratio, especially if you are using ccache for large packages (why else would you need it?). However, note that you will no longer have a single cache size limit — every package will have its own limit. Therefore, you may want to reduce the limits per-package, or manually look after the cache periodically.

    July 20, 2017
    Hanno Böck a.k.a. hanno (homepage, bugs)

    KeyLately, some attention was drawn to a widespread problem with TLS certificates. Many people are accidentally publishing their private keys. Sometimes they are released as part of applications, in Github repositories or with common filenames on web servers.

    If a private key is compromised, a certificate authority is obliged to revoke it. The Baseline Requirements – a set of rules that browsers and certificate authorities agreed upon – regulate this and say that in such a case a certificate authority shall revoke the key within 24 hours (Section 4.9.1.1 in the current Baseline Requirements 1.4.8). These rules exist despite the fact that revocation has various problems and doesn’t work very well, but that’s another topic.

    I reported various key compromises to certificate authorities recently and while not all of them reacted in time, they eventually revoked all certificates belonging to the private keys. I wondered however how thorough they actually check the key compromises. Obviously one would expect that they cryptographically verify that an exposed private key really is the private key belonging to a certificate.

    I registered two test domains at a provider that would allow me to hide my identity and not show up in the whois information. I then ordered test certificates from Symantec (via their brand RapidSSL) and Comodo. These are the biggest certificate authorities and they both offer short term test certificates for free. I then tried to trick them into revoking those certificates with a fake private key.

    Forging a private key

    To understand this we need to get a bit into the details of RSA keys. In essence a cryptographic key is just a set of numbers. For RSA a public key consists of a modulus (usually named N) and a public exponent (usually called e). You don’t have to understand their mathematical meaning, just keep in mind: They’re nothing more than numbers.

    An RSA private key is also just numbers, but more of them. If you have heard any introductory RSA descriptions you may know that a private key consists of a private exponent (called d), but in practice it’s a bit more. Private keys usually contain the full public key (N, e), the private exponent (d) and several other values that are redundant, but they are useful to speed up certain things. But just keep in mind that a public key consists of two numbers and a private key is a public key plus some additional numbers. A certificate ultimately is just a public key with some additional information (like the host name that says for which web page it’s valid) signed by a certificate authority.

    A naive check whether a private key belongs to a certificate could be done by extracting the public key parts of both the certificate and the private key for comparison. However it is quite obvious that this isn’t secure. An attacker could construct a private key that contains the public key of an existing certificate and the private key parts of some other, bogus key. Obviously such a fake key couldn’t be used and would only produce errors, but it would survive such a naive check.

    I created such fake keys for both domains and uploaded them to Pastebin. If you want to create such fake keys on your own here’s a script. To make my report less suspicious I searched Pastebin for real, compromised private keys belonging to certificates. This again shows how problematic the leakage of private keys is: I easily found seven private keys for Comodo certificates and three for Symantec certificates, plus several more for other certificate authorities, which I also reported. These additional keys allowed me to make my report to Symantec and Comodo less suspicious: I could hide my fake key report within other legitimate reports about a key compromise.

    Symantec revoked a certificate based on a forged private key

    SymantecComodo didn’t fall for it. They answered me that there is something wrong with this key. Symantec however answered me that they revoked all certificates – including the one with the fake private key.

    No harm was done here, because the certificate was only issued for my own test domain. But I could’ve also fake private keys of other peoples' certificates. Very likely Symantec would have revoked them as well, causing downtimes for those sites. I even could’ve easily created a fake key belonging to Symantec’s own certificate.

    The communication by Symantec with the domain owner was far from ideal. I first got a mail that they were unable to process my order. Then I got another mail about a “cancellation request”. They didn’t explain what really happened and that the revocation happened due to a key uploaded on Pastebin.

    I then informed Symantec about the invalid key (from my “real” identity), claiming that I just noted there’s something wrong with it. At that point they should’ve been aware that they revoked the certificate in error. Then I contacted the support with my “domain owner” identity and asked why the certificate was revoked. The answer: “I wanted to inform you that your FreeSSL certificate was cancelled as during a log check it was determined that the private key was compromised.”

    To summarize: Symantec never told the domain owner that the certificate was revoked due to a key leaked on Pastebin. I assume in all the other cases they also didn’t inform their customers. Thus they may have experienced a certificate revocation, but don’t know why. So they can’t learn and can’t improve their processes to make sure this doesn’t happen again. Also, Symantec still insisted to the domain owner that the key was compromised even after I already had informed them that the key was faulty.

    How to check if a private key belongs to a certificate?

    SSLShopper checkIn case you wonder how you properly check whether a private key belongs to a certificate you may of course resort to a Google search. And this was fascinating – and scary – to me: I searched Google for “check if private key matches certificate”. I got plenty of instructions. Almost all of them were wrong. The first result is a page from SSLShopper. They recommend to compare the MD5 hash of the modulus. That they use MD5 is not the problem here, the problem is that this is a naive check only comparing parts of the public key. They even provide a form to check this. (That they ask you to put your private key into a form is a different issue on its own, but at least they have a warning about this and recommend to check locally.)

    Furthermore we get the same wrong instructions from the University of Wisconsin, Comodo (good that their engineers were smart enough not to rely on their own documentation), tbs internet (“SSL expert since 1996”), ShellHacks, IBM and RapidSSL (aka Symantec). A post on Stackexchange is the only result that actually mentions a proper check for RSA keys. Two more Stackexchange posts are not related to RSA, I haven’t checked their solutions in detail.

    Going to Google results page two among some unrelated links we find more wrong instructions and tools from Symantec, SSL247 (“Symantec Specialist Partner Website Security” - they learned from the best) and some private blog. A documentation by Aspera (belonging to IBM) at least mentions that you can check the private key, but in an unrelated section of the document. Also we get more tools that ask you to upload your private key and then not properly check it from SSLChecker.com, the SSL Store (Symantec “Website Security Platinum Partner”), GlobeSSL (“in SSL we trust”) and - well - RapidSSL.

    Documented Security Vulnerability in OpenSSL

    So if people google for instructions they’ll almost inevitably end up with non-working instructions or tools. But what about other options? Let’s say we want to automate this and have a tool that verifies whether a certificate matches a private key using OpenSSL. We may end up finding that OpenSSL has a function x509_check_private_key() that can be used to “check the consistency of a private key with the public key in an X509 certificate or certificate request”. Sounds like exactly what we need, right?

    Well, until you read the full docs and find out that it has a BUGS section: “The check_private_key functions don't check if k itself is indeed a private key or not. It merely compares the public materials (e.g. exponent and modulus of an RSA key) and/or key parameters (e.g. EC params of an EC key) of a key pair.”

    I think this is a security vulnerability in OpenSSL (discussion with OpenSSL here). And that doesn’t change just because it’s a documented security vulnerability. Notably there are downstream consumers of this function that failed to copy that part of the documentation, see for example the corresponding PHP function (the limitation is however mentioned in a comment by a user).

    So how do you really check whether a private key matches a certificate?

    Ultimately there are two reliable ways to check whether a private key belongs to a certificate. One way is to check whether the various values of the private key are consistent and then check whether the public key matches. For example a private key contains values p and q that are the prime factors of the public modulus N. If you multiply them and compare them to N you can be sure that you have a legitimate private key. It’s one of the core properties of RSA that it’s secure based on the assumption that it’s not feasible to calculate p and q from N.

    You can use OpenSSL to check the consistency of a private key:
    openssl rsa -in [privatekey] -check

    For my forged keys it will tell you:
    RSA key error: n does not equal p q

    You can then compare the public key, for example by calculating the so-called SPKI SHA256 hash:
    openssl pkey -in [privatekey] -pubout -outform der | sha256sum
    openssl x509 -in [certificate] -pubkey |openssl pkey -pubin -pubout -outform der | sha256sum

    Another way is to sign a message with the private key and then verify it with the public key. You could do it like this:
    openssl x509 -in [certificate] -noout -pubkey > pubkey.pem
    dd if=/dev/urandom of=rnd bs=32 count=1
    openssl rsautl -sign -pkcs -inkey [privatekey] -in rnd -out sig
    openssl rsautl -verify -pkcs -pubin -inkey pubkey.pem -in sig -out check
    cmp rnd check
    rm rnd check sig pubkey.pem

    If cmp produces no output then the signature matches.

    As this is all quite complex due to OpenSSLs arcane command line interface I have put this all together in a script. You can pass a certificate and a private key, both in ASCII/PEM format, and it will do both checks.

    Summary

    Symantec did a major blunder by revoking a certificate based on completely forged evidence. There’s hardly any excuse for this and it indicates that they operate a certificate authority without a proper understanding of the cryptographic background.

    Apart from that the problem of checking whether a private key and certificate match seems to be largely documented wrong. Plenty of erroneous guides and tools may cause others to fall for the same trap.

    Update: Symantec answered with a blog post.

    July 18, 2017
    Sven Vermeulen a.k.a. swift (homepage, bugs)
    Project prioritization (July 18, 2017, 18:40 UTC)

    This is a long read, skip to “Prioritizing the projects and changes” for the approach details...

    Organizations and companies generally have an IT workload (dare I say, backlog?) which needs to be properly assessed, prioritized and taken up. Sometimes, the IT team(s) get an amount of budget and HR resources to "do their thing", while others need to continuously ask for approval to launch a new project or instantiate a change.

    Sizeable organizations even require engineering and development effort on IT projects which are not readily available: specialized teams exist, but they are governance-wise assigned to projects. And as everyone thinks their project is the top-most priority one, many will be disappointed when they hear there are no resources available for their pet project.

    So... how should organizations prioritize such projects?

    July 16, 2017
    Michał Górny a.k.a. mgorny (homepage, bugs)
    GLEP 73 check results explained (July 16, 2017, 08:40 UTC)

    The pkgcheck instance run for the Repo mirror&CI project has finished gaining a full support for GLEP 73 REQUIRED_USE validation and verification today. As a result, it can report 5 new issues defined by that GLEP. In this article, I’d like to shortly summarize them and explain how to interpret and solve the reports.

    Technical note: the GLEP number has not been formally assigned yet. However, since there is no other GLEP request open at the moment, I have taken the liberty of using the next free number in the implementation.

    GLEP73Syntax: syntax violates GLEP 73

    GLEP 73 specifies a few syntax restrictions as compared to the pretty much free-form syntax allowed by the PMS. The restrictions could be shortly summarized as:

    • ||, ^^ and ?? can not not be empty,
    • ||, ^^ and ?? can not not be nested,
    • USE-conditional groups can not be used inside ||, ^^ and ??,
    • All-of groups (expressed using parentheses without a prefix) are banned completely.

    The full rationale for the restrictions, along with examples and proposed fixes is provided in the GLEP. For the purpose of this article, it is enough to say that in all the cases found, there was a simpler (more obvious) way of expressing the same constraint.

    Violation of this syntax prevents pkgcheck from performing any of the remaining checks. But more importantly, the report indicates that the constraint is unnecessarily complex and could result in REQUIRED_USE mismatch messages that are unnecessarily confusing to the user. Taking a real example, compare:

      The following REQUIRED_USE flag constraints are unsatisfied:
        exactly-one-of ( ( !32bit 64bit ) ( 32bit !64bit ) ( 32bit 64bit ) )

    and the effect of a valid replacement:

      The following REQUIRED_USE flag constraints are unsatisfied:
    	any-of ( 64bit 32bit )

    While we could debate about usefulness of the Portage output, I think it is clear that the second output is simpler to comprehend. And the best proof is that you actually need to think a bit before confirming that they’re equivalent.

    GLEP73Immutability: REQUIRED_USE violates immutability rules

    This one is rather simple: it means this constraint may tell user to enable (disable) a flag that is use.masked/forced. Taking a trivial example:

    a? ( b )

    GLEP73Immutability report will trigger if a profile masks the b flag. This means that if the user has a enabled, the PM would normally tell him to enable b as well. However, since b is masked, it can not be enabled using normal methods (we assume that altering use.mask is not normally expected).

    The alternative is to disable a then. But what’s the point of letting user enable it if we afterwards tell him to disable it anyway? It is more friendly to disable both flags together, and this is pretty much what the check is about. So in this case, the solution is to mask a as well.

    How to read it? Given the generic message of:

    REQUIRED_USE violates immutability rules: [C] requires [E] while the opposite value is enforced by use.force/mask (in profiles: [P])

    It indicates that in profiles P (a lot of profiles usually indicates you’re looking for base or top-level arch profile), E is forced or masked, and that you probably need to force/mask C appropriately as well.

    GLEP73SelfConflicting: impossible self-conflicting condition

    This one is going to be extremely rare. It indicates that somehow the REQUIRED_USE nested a condition and its negation, causing it to never evaluate to true. It is best explained using the following trivial example:

    a? ( !a? ( b ) )

    This constraint will never be enforced since a and !a can not be true simultaneously.

    Is there a point in having such a report at all? Well, such a thing is extremely unlikely to happen. However, it would break the verification algorithms and so we need to account for it explicitly. Since we account for it anyway and it is a clear mistake, why not report it?

    GLEP73Conflict: request for conflicting states

    This warning indicates that there are at least two constraints that can apply simultaneously and request the opposite states for the same USE flag. Again, best explained on a generic example:

    a? ( c ) b? ( !c )

    In this example, any USE flag set with both a and b enabled could not satisfy the constraint. However, Portage will happily led us astray:

      The following REQUIRED_USE flag constraints are unsatisfied:
    	a? ( c )

    If we follow the advice and enable c, we get:

      The following REQUIRED_USE flag constraints are unsatisfied:
    	b? ( !c )

    The goal of this check is to avoid such a bad advices, and to require constraints to clearly indicate a suggested way forward. For example, the above case could be modified to:

    a? ( !b c ) b? ( !c )

    to indicate that a takes precedence over b, and that b should be disabled to avoid the impossible constraint. The opposite can be stated similarly — however, note that you need to reorder the constraints to make sure that the PM will get it right:

    b? ( !a !c ) a? ( c )

    How to read it? Given the generic message of:

    REQUIRED_USE can request conflicting states: [Ci] requires [Ei] while [Cj] requires [Ej]

    It means that if the user enables Ci and Cj simultaneously, the PM will request conflicting Ei and Ej. Depending on the intent, the solution might involve negating one of the conditions in the other constraint, or reworking the REQUIRED_USE towards another solution.

    GLEP73BackAlteration: previous condition starts applying

    This warning is the most specific and the least important from all the additions at the moment. It indicates that the specific constraint may cause a preceding condition to start to apply, enforcing additional requirements. Consider the following example:

    b? ( c ) a? ( b )

    If the user has only a enabled, the second rule will enforce b. Then the condition for the first rule will start matching, and additionally enforce c. Is this a problem? Usually not. However, for the purpose of GLEP 73 we prefer that the REQUIRED_USE can be enforced while processing left-to-right, in a single iteration. If a previous rule starts applying, we may need to do another iteration.

    The solution is usually trivial: to reorder (swap) the constraints. However, in some cases developers seem to prefer copying the enforcements into the subsequent rule, e.g.:

    b? ( c ) a? ( b c )

    Either way works for the purposes of GLEP 73, though the latter increases complexity.

    How to read it? Given the generic message of:

    REQUIRED_USE causes a preceding condition to start applying: [Cj] enforces [Ej] which may cause preceding [Ci] enforcing [Ei] to evaluate to true

    This indicates that if Cj is true, Ej needs to be true as well. Once it is true, a preceding condition of Ci may also become true, adding another requirement for Ei. To fix the issue, you need to either move the latter constraint before the former, or include the enforcement of Ei in the rule for Cj, rendering the application of the first rule unnecessary.

    Constructs using ||, ^^ and ?? operators

    GLEP 73 specifies a leftmost-preferred behavior for the ||, ^^ and ?? operators. It is expressed in a simple transformation into implications (USE-conditional groups). Long story short:

    • || and ^^ groups force the leftmost unmasked flag if none of the flags are enabled already, and
    • ?? and ^^ groups disable all but the leftmost enabled flag if more than one flag is enabled.

    All the verification algorithms work on the transformed form, and so their output may list conditions resulting from it. For example, the following construct:

    || ( a b c ) static? ( !a )

    will report a conflict between !b !c ⇒ a and static ⇒ !a. This indicates the fact that per the forementioned rule, || group is transformed into !b? ( !c? ( a ) ) which explains that if none of the flags are enabled, the first one is preferred, causing a conflict with the static flag.

    In this particular case you could debate that the algorithm could choose b or c instead in order to avoid the problem. However, we determined that this kind of heuristic is not a goal for GLEP 73, and instead we always obide the developer’s preference expressed in the ordering. The only exception to this rule is when the leftmost flag can not match due to a mask, in which case the first unmasked flag is used.

    For completeness, I should add that ?? and ^^ blocks create implications in the form of: a ⇒ !b !c…, b ⇒ !c… and so on.

    At some point I might work on making the reports include the original form to avoid ambiguity.

    The future

    The most important goal for GLEP 73 is to make it possible for users to install packages out-of-the-box without having to fight through mazes of REQUIRED_USE, and for developers to use REQUIRED_USE not only sparingly but whenever possible to improve the visibility of resulting package configuration. However, there is still a lot of testing, some fixing and many bikesheds before that could happen.

    Nevertheless, I think we can all agree that most of the reports produced so far (with the exception of the back-alteration case) are meaningful even without automatic enforcing of REQUIRED_USE, and fixing them would benefit our users already. I would like to ask you to look for the reports on your packages and fix them whenever possible. Feel free to ping me if you need any help with that.

    Once the number of non-conforming packages goes down, I will convert the reports successively into warning levels, making the CI report new issues and the pull request scans proactively complain about them.

    July 14, 2017
    Sebastian Pipping a.k.a. sping (homepage, bugs)
    Expat 2.2.2 released (July 14, 2017, 17:32 UTC)

    (This article first appeared on XML.com.)

    A few weeks after release 2.2.1 of the free software XML parsing library Expat, version 2.2.2 now improves on few rough edges (mostly related to compilation) but also fixes security issues.

    Windows binaries compiled with _UNICODE now use proper entropy for seeding the SipHash algorithm. On Unix-like platforms, accidentally missing out on high quality entropy sources is now prevented from going unnoticed: It would happen when some other build system than the configure script was used, e.g. the shipped CMake one or when the source code was copied into some parent project’s build system without paying attention to the new compile flags (that the configure script would auto-detect for you). After some struggle with a decision about C99, Expat requires a C99 compiler now; 18 years after its definition, that’s a defendable move. The uint64_t type and ULL integer literals (unsigned long long) for SipHash made us move.

    Expat would like to thank the community for the bug reports and patches that went into Expat 2.2.2. If you maintain a bundled copy of Expat somewhere, please make sure it gets updated.

    Sebastian Pipping
    for the Expat development team