Post details: Python threading sucks

18 March, 2005

Permalink 23:48 UTC, by Andrew Gaffney Email , 202 words, 2709 views   English (US)
Categories: Gentoo

Python threading sucks

Recently, when testing the installer, we noticed a weird issue. When 'emerge sync' is run, rsync would complete the sync but not exit, which caused the install to hang. It took 2 days and a lot of frustration to finally track this one down. We owe the answer to d_m (on freenode).

The installer runs with 2 separate threads. The frontend is the main thread and the backend starts up a second thread to run the commands that perform the install. It turns out that when python creates a new thread, it blocks all signals for that thread. I'm not quite sure why. This has the side effect of blocking all signals for any processes that are fork()ed and/or exec()ed from that child thread.

Rsync fork()s when it's doing it's thing. One process waits for a USR2 signal from the other before they mutually exit. Since python had blocked all signals, the signal wasn't getting from one process to another so they could exit. d_m's solution was to write a small C program that resets the signal mask when running external programs. While this works, it is a huge hack. Although, this bug should be fixed in python 2.4 :-/

Comments:

Comment from: pH0BiAn [Visitor] · http://www.geocities.com/schwiezer17
I have a similar program in a chat program I created in python. I am using the medium level "threading" module to create server connection accepting threads but I can't exit them when they finish their thing, they stack up and the receiving chat program hangs. Sometimes after just one message! (I know, not the cleanest code ever).

Do you think you could email that C program/source to me? Or at least some other info on exiting threads? My address is:
whitehatlad@hotmail.com

Cheers,
phobian
PermalinkPermalink 13 May , 2005 @ 08:32
Comment from: Deepak [Visitor] · http://www.niecdelhi.com/~deepak/
Not really? Python threads are easy and light! i just wanna add Python thread sucks on windows. ;)
PermalinkPermalink 13 January, 2006 @ 15:52
Comment from: Andrew Gaffney [Member] Email · http://dev.gentoo.org/~agaffney/
They might be light and easy, but when you mix in signal handling, 'sane' seems to go right out the window.
PermalinkPermalink 13 January, 2006 @ 16:41
Comment from: Deepak [Visitor] · http://www.niecdelhi.com/~deepak/
Signal handling on windows .. its a big joke .. SIGALRM ..huh! i still remember my experience with it. Somewhere singals are not platform independent .. on windows use thread + win32api and for Posix use thread + std. signals .. thats how i resolved my problem.
PermalinkPermalink 20 January, 2006 @ 21:45

Comments are closed for this post.

Andrew Gaffney

August 2008
Mon Tue Wed Thu Fri Sat Sun
<< <     
        1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31

Search

Categories

Misc

XML Feeds

What is RSS?

Who's Online?

  • Guest Users: 156

powered by
b2evolution