加载中…
个人资料
麦兜搞IT
麦兜搞IT
  • 博客等级:
  • 博客积分:0
  • 博客访问:1,533,647
  • 关注人气:531
  • 获赠金笔:0支
  • 赠出金笔:0支
  • 荣誉徽章:
相关博文
推荐博文
谁看过这篇博文
加载中…
正文 字体大小:

python多线程ctrl-c退出问题(转)

(2010-06-03 20:41:37)
标签:

python

多线程

it

分类: Python

http://yiminghe.javaeye.com/blog/673524

场景:

经常会遇到下述问题:很多io busy的应用采取多线程的方式来解决,但这时候会发现python命令行不响应ctrl-c 了,而对应的java代码则没有问题:

 

Java代码 复制代码
  1. public class Test {   
  2.     public static void main(String[] args) throws Exception {   
  3.   
  4.         new Thread(new Runnable() {   
  5.   
  6.             public void run() {   
  7.                 long start System.currentTimeMillis();   
  8.                 while (true{   
  9.                     try {   
  10.                         Thread.sleep(1000);   
  11.                     catch (Exception e) {   
  12.                     }   
  13.                     System.out.println(System.currentTimeMillis());   
  14.                     if (System.currentTimeMillis() start 1000 100break;   
  15.                 }   
  16.             }   
  17.         }).start();   
  18.   
  19.     }   
  20.  
public class Test {
    public static void main(String[] args) throws Exception {

        new Thread(new Runnable() {

            public void run() {
                long start = System.currentTimeMillis();
                while (true) {
                    try {
                        Thread.sleep(1000);
                    } catch (Exception e) {
                    }
                    System.out.println(System.currentTimeMillis());
                    if (System.currentTimeMillis() - start > 1000 * 100) break;
                }
            }
        }).start();

    }
}

java Test

ctrl-c则会结束程序


而对应的python代码:

 

Python代码 复制代码
  1. -*- coding: utf-8 -*-   
  2. import time   
  3. import threading   
  4. start=time.time()   
  5. def foreverLoop():   
  6.     start=time.time()   
  7.     while 1:   
  8.         time.sleep(1)   
  9.         print time.time()   
  10.         if time.time()-start>100:   
  11.             break  
  12.                
  13.   
  14. thread_=threading.Thread(target=foreverLoop)   
  15. #thread_.setDaemon(True)   
  16. thread_.start()  
# -*- coding: utf-8 -*-
import time
import threading
start=time.time()
def foreverLoop():
    start=time.time()
    while 1:
        time.sleep(1)
        print time.time()
        if time.time()-start>100:
            break
            

thread_=threading.Thread(target=foreverLoop)
#thread_.setDaemon(True)
thread_.start()

 

python p.py


后ctrl-c则完全不起作用了。

 

 

不成熟的分析:

 

首先单单设置 daemon 为 true 肯定不行,就不解释了。当daemon为 false 时,导入python线程库后实际上,threading会在主线程执行完毕后,检查是否有不是 daemon 的线程,有的化就wait,等待线程结束了,在主线程等待期间,所有发送到主线程的信号也会被阻测,可以在上述代码加入signal模块验证一下:

 

Python代码 复制代码
  1. def sigint_handler(signum,frame):     
  2.     print "main-thread exit"  
  3.     sys.exit()     
  4. signal.signal(signal.SIGINT,sigint_handler)  
def sigint_handler(signum,frame):  
    print "main-thread exit"
    sys.exit()  
signal.signal(signal.SIGINT,sigint_handler)

 

在100秒内按下ctrl-c没有反应,只有当子线程结束后才会出现打印 "main-thread exit",可见 ctrl-c被阻测了


threading 中在主线程结束时进行的操作:

Python代码 复制代码
  1. _shutdown _MainThread()._exitfunc   
  2. def _exitfunc(self):   
  3.         self._Thread__stop()   
  4.         _pickSomeNonDaemonThread()   
  5.         if t:   
  6.             if __debug__:   
  7.                 self._note("%s: waiting for other threads"self)   
  8.         while t:   
  9.             t.join()   
  10.             _pickSomeNonDaemonThread()   
  11.         if __debug__:   
  12.             self._note("%s: exiting"self)   
  13.         self._Thread__delete()  
_shutdown = _MainThread()._exitfunc
def _exitfunc(self):
        self._Thread__stop()
        t = _pickSomeNonDaemonThread()
        if t:
            if __debug__:
                self._note("%s: waiting for other threads", self)
        while t:
            t.join()
            t = _pickSomeNonDaemonThread()
        if __debug__:
            self._note("%s: exiting", self)
        self._Thread__delete()

 

 对所有的非daemon线程进行join等待,其中join中可自行察看源码,又调用了wait,同上文分析 ,主线程等待到了一把锁上。

 

不成熟的解决:

 

只能把线程设成daemon才能让主线程不等待,能够接受ctrl-c信号,但是又不能让子线程立即结束,那么只能采用传统的轮询方法了,采用sleep间歇省点cpu吧:

 

Python代码 复制代码
  1. -*- coding: utf-8 -*-   
  2. import time,signal,traceback   
  3. import sys   
  4. import threading   
  5. start=time.time()   
  6. def foreverLoop():   
  7.     start=time.time()   
  8.     while 1:   
  9.         time.sleep(1)   
  10.         print time.time()   
  11.         if time.time()-start>5:   
  12.             break  
  13.                
  14. thread_=threading.Thread(target=foreverLoop)   
  15. thread_.setDaemon(True)   
  16. thread_.start()   
  17.   
  18. #主线程wait住了,不能接受信号了   
  19. #thread_.join()   
  20.   
  21. def _exitCheckfunc():   
  22.     print "ok"  
  23.     try:   
  24.         while 1:   
  25.             alive=False  
  26.             if thread_.isAlive():   
  27.                 alive=True  
  28.             if not alive:   
  29.                 break  
  30.             time.sleep(1    
  31.     #为了使得统计时间能够运行,要捕捉  KeyboardInterrupt :ctrl-c         
  32.     except KeyboardInterrupt, e:   
  33.         traceback.print_exc()   
  34.     print "consume time :",time.time()-start   
  35.            
  36. threading._shutdown=_exitCheckfunc  
# -*- coding: utf-8 -*-
import time,signal,traceback
import sys
import threading
start=time.time()
def foreverLoop():
    start=time.time()
    while 1:
        time.sleep(1)
        print time.time()
        if time.time()-start>5:
            break
            
thread_=threading.Thread(target=foreverLoop)
thread_.setDaemon(True)
thread_.start()

#主线程wait住了,不能接受信号了
#thread_.join()

def _exitCheckfunc():
    print "ok"
    try:
        while 1:
            alive=False
            if thread_.isAlive():
                alive=True
            if not alive:
                break
            time.sleep(1)  
    #为了使得统计时间能够运行,要捕捉  KeyboardInterrupt :ctrl-c      
    except KeyboardInterrupt, e:
        traceback.print_exc()
    print "consume time :",time.time()-start
        
threading._shutdown=_exitCheckfunc

   缺点:轮询总会浪费点cpu资源,以及battery.


有更好的解决方案敬请提出。

 

ps1: 进程监控解决方案

 

用另外一个进程来接受信号后杀掉执行任务进程,牛

Python代码 复制代码
  1. -*- coding: utf-8 -*-   
  2. import time,signal,traceback,os   
  3. import sys   
  4. import threading   
  5. start=time.time()   
  6. def foreverLoop():   
  7.     start=time.time()   
  8.     while 1:   
  9.         time.sleep(1)   
  10.         print time.time()   
  11.         if time.time()-start>5:   
  12.             break  
  13.   
  14. class Watcher:   
  15.     """this class solves two problems with multithreaded  
  16.     programs in Python, (1) signal might be delivered  
  17.     to any thread (which is just malfeature) and (2) if  
  18.     the thread that gets the signal is waiting, the signal  
  19.     is ignored (which is bug).  
  20.  
  21.     The watcher is concurrent process (not thread) that  
  22.     waits for signal and the process that contains the  
  23.     threads.  See Appendix of The Little Book of Semaphores.  
  24.     http://greenteapress.com/semaphores/  
  25.  
  26.     have only tested this on Linux.  would expect it to  
  27.     work on the Macintosh and not work on Windows.  
  28.     """  
  29.   
  30.     def __init__(self):   
  31.         """ Creates child thread, which returns.  The parent  
  32.             thread waits for KeyboardInterrupt and then kills  
  33.             the child thread.  
  34.         """  
  35.         self.child os.fork()   
  36.         if self.child == 0:   
  37.             return  
  38.         else:   
  39.             self.watch()   
  40.   
  41.     def watch(self):   
  42.         try:   
  43.             os.wait()   
  44.         except KeyboardInterrupt:   
  45.             put the capital in KeyBoardInterrupt so can   
  46.             tell when the Watcher gets the SIGINT   
  47.             print 'KeyBoardInterrupt'  
  48.             self.kill()   
  49.         sys.exit()   
  50.   
  51.     def kill(self):   
  52.         try:   
  53.             os.kill(self.child, signal.SIGKILL)   
  54.         except OSError: pass  
  55.   
  56. Watcher()               
  57. thread_=threading.Thread(target=foreverLoop)   
  58. thread_.start()  
# -*- coding: utf-8 -*-
import time,signal,traceback,os
import sys
import threading
start=time.time()
def foreverLoop():
    start=time.time()
    while 1:
        time.sleep(1)
        print time.time()
        if time.time()-start>5:
            break

class Watcher:
    """this class solves two problems with multithreaded
    programs in Python, (1) a signal might be delivered
    to any thread (which is just a malfeature) and (2) if
    the thread that gets the signal is waiting, the signal
    is ignored (which is a bug).

    The watcher is a concurrent process (not thread) that
    waits for a signal and the process that contains the
    threads.  See Appendix A of The Little Book of Semaphores.
    http://greenteapress.com/semaphores/

    I have only tested this on Linux.  I would expect it to
    work on the Macintosh and not work on Windows.
    """

    def __init__(self):
        """ Creates a child thread, which returns.  The parent
            thread waits for a KeyboardInterrupt and then kills
            the child thread.
        """
        self.child = os.fork()
        if self.child == 0:
            return
        else:
            self.watch()

    def watch(self):
        try:
            os.wait()
        except KeyboardInterrupt:
            # I put the capital B in KeyBoardInterrupt so I can
            # tell when the Watcher gets the SIGINT
            print 'KeyBoardInterrupt'
            self.kill()
        sys.exit()

    def kill(self):
        try:
            os.kill(self.child, signal.SIGKILL)
        except OSError: pass

Watcher()            
thread_=threading.Thread(target=foreverLoop)
thread_.start()

 注意 watch()一定要放在线程创建前,原因未知。。。。,否则立刻就结束

0

阅读 评论 收藏 转载 喜欢 打印举报/Report
  • 评论加载中,请稍候...
发评论

    发评论

    以上网友发言只代表其个人观点,不代表新浪网的观点或立场。

      

    新浪BLOG意见反馈留言板 电话:4000520066 提示音后按1键(按当地市话标准计费) 欢迎批评指正

    新浪简介 | About Sina | 广告服务 | 联系我们 | 招聘信息 | 网站律师 | SINA English | 会员注册 | 产品答疑

    新浪公司 版权所有