Going to FOSDEM 2010 – Thomas Koch
I’m going to fosdem 2010.
These are things I’d like to attend:
| time | room | topic | speaker |
|---|---|---|---|
| 10:45-11:30 | Janson | Promoting Open Source Methods at a Large Company | Brooks Davis |
| 13:15-13:45 | AY | Packaging Java Software for Debian | Thomas Koch |
| 13:40-13:55 | Ferrer | CiviCRM: Common goals of FOSS and Not For Profit Organisations | Xavier DUTOIT |
| 14:00-14:45 | H.1308 | The Maemo Community Council: a case-study in governance | Dave Neary |
| 14:00-15:30 | Guillissen | LPI exam session 1 | Klaus Behrla |
| 14:00-14:45 | Janson | What is my system doing – Full System Observability with SystemTap | Mark Wielaard |
| 14:30-15:00 | H.2213 | Building Federated Social Networks on XMPP | Tuomas Koski, Simon Tennant |
| 15:00-15:45 | Janson | Ganglia: 10 years of monitoring clusters and grids | Bernard Li |
| 15:00-15:30 | H.2213 | XMPP and the Social Web | Alard Weisscher, Laurent Eschenauer |
| 15:30-16:00 | H.2213 | PubSub Gone Wild: Info Sharing at Mediamatic | Ralph Meijer |
| 16:00-17:30 | Guillissen | LPI exam session 2 | Klaus Behrla |
| 16:00-16:15 | Ferrer | SIP Communicator: Skype-like conf calls with SIP Communicator | Emil Ivov |
| 16:20-16:35 | Ferrer | Kamailio (OpenSER) 3.0.0: redefinition of SIP server | Daniel-Constantin Mierla |
| 16:40-16:55 | Ferrer | asterisk: An introduction to Asterisk Development | Mark Michelson |
| 17:00-17:15 | Ferrer | csync: Roaming Home Directories | Andreas Schneider |
| 17:30-18:00 | H.2213 | Jingle Nodes: An Open Alternative to Skype | Tiago Camargo |
| 18:00-18:30 | H.2213 | Multi-User Jingle: Voice and Video Conferencing with XMPP | Dafydd Harries, Sjoerd Simons |
| 18:15-19:00 | H.1302 | Spacewalk: Linux Systems Lifecycle Management | Marcus Moeller, Sandro Mathys |
| 09:15-10:00 | H.1302 | Linux distribution for the cloud | Peter Eisentraut |
| 10:00-10:45 | Janson | RepRap – Manufacturing for the Masses | Adrian Bowyer |
| 10:30-12:00 | Guillissen | LPI exam session 3 | Klaus Behrla |
| 11:00-11:45 | AW1.120 | My life with HBase | Lars George |
| 11:00-11:45 | Janson | Tor: Building, Growing, and Extending Online Anonymity | Andrew Lewman |
| 12:15-13:00 | H.1302 | Cross distro packaging with (top)git | Thomas Koch |
| 12:15-13:00 | H.1308 | Shared libraries in Debian | Sune Vuorela |
| 13:00-14:30 | Guillissen | LPI exam session 4 | Klaus Behrla |
| 13:15-14:00 | AW1.120 | CouchDB, a database designed for the web and more | Benoit Chesneau |
| 14:00-14:45 | Janson | Large scale data analysis made easy – Apache Hadoop | Isabel Drost |
| 15:00-16:30 | Guillissen | LPI exam session 5 | Klaus Behrla |
| 15:30-16:15 | H.1308 | Continuous Packaging with Project-Builder.org | Bruno Cornec |
| 15:30-16:15 | AW1.120 | Comparing the MapReduce way in CouchDB with the SQL way in a RDBMS | Stéphane Combaudon |
| 16:15-16:45 | AW1.120 | Designing a scalable content management system on NoSQL technologies | Evert Arckens |
| 16:15-17:00 | H.1308 | Debian Secrets: power tools for power users | Wouter Verhelst |
Menno’s Musings: TypeError: object.__init__() takes no parameters
At my employer we are in the process of migrating from Python 2.4 to
2.6. When running some existing code under Python 2.6 we started
getting DeprecationWarnings about "object.__new__() takes no
parameters" and "object.__init__() takes no parameters".
A simple example that triggers the warning:
class MyClass(object):
def __new__(cls, a, b):
print 'MyClass.__new__', a, b
return super(MyClass, cls).__new__(cls, a, b)
def __init__(self, a, b):
print 'MyClass.__init__', a, b
super(MyClass, self).__init__(a, b)
obj = MyClass(6, 7)
This gives:
$ python2.4 simple-warning.py MyClass.__new__ 6 7 MyClass.__init__ 6 7 $ python2.6 simple-warning.py MyClass.__new__ 6 7 simple-warning.py:5: DeprecationWarning: object.__new__() takes no parameters return super(MyClass, cls).__new__(cls, a, b) MyClass.__init__ 6 7 simple-warning.py:9: DeprecationWarning: object.__init__() takes no parameters super(MyClass, self).__init__(a, b)
It turns out that a change to Python for 2.6 (and 3) means that
object.__new__ and object.__init__ no longer take arguments – a
TypeError is raised when arguments are passed. To avoid breaking too
much pre-existing code, there is a special case that will cause a
DeprecationWarning instead of TypeError if both __init__ and __new__
are overridden. This is the case we were running into with our code at
work.
The reason for this change seems to make enough sense: object doesn’t
do anything with arguments to __init__ and __new__ so it shouldn’t
accept them. Raising an error when arguments are passed highlights
code where the code might be doing the wrong thing.
Unfortunately this change also breaks Python’s multiple inheritance in
a fairly serious way when cooperative super calls are used. Looking
at the ticket for this change, this issue was thought about but
perhaps the implications were not fully understood. Given that using
super with multiple inheritance is common and "correct" practice, it
would seem that this change to Python is a step backwards.
Doug Hellmann: PyMOTW: cgitb – Detailed traceback reports
cgitb – Detailed traceback reports
| Purpose: | cgitb provides more detailed traceback information than traceback. |
|---|---|
| Python Version: | 2.2 and later |
cgitb was originally designed for showing errors and debugging
information in web applications. It was later updated to include
plain-text output as well, but unfortunately wasn’t renamed. This has
led to obscurity and the module is not used as often as it should be.
Nonetheless, cgitb is a valuable debugging tool in the standard
library.
Standard Traceback Dumps
Python’s default exception handling behavior is to print a tracebackto standard error with the call stack leading up to the error
position. This basic output is frequently enough information to
understand the cause of the exception and permit a fix.
def func1(arg1): local_var = arg1 * 2 return func2(local_var)
def func2(arg2): local_var = arg2 + 2 return func3(local_var)
def func3(arg3): local_var = arg2 / 2 return local_var
func1(1)
The above sample program has a subtle error in func3().
$ python cgitb_basic_traceback.pyTraceback (most recent call last): File "cgitb_basic_traceback.py", line 22, in <module> func1(1) File "cgitb_basic_traceback.py", line 12, in func1 return func2(local_var) File "cgitb_basic_traceback.py", line 16, in func2 return func3(local_var) File "cgitb_basic_traceback.py", line 19, in func3 local_var = arg2 / 2NameError: global name 'arg2' is not defined
Enabling Detailed Tracebacks
While the basic traceback includes enough information for us to spot
the error, enabling cgitb replaces sys.excepthook with a function
that gives extended tracebacks with even more detail.
import cgitbcgitb.enable(format='text')
def func1(arg1): local_var = arg1 * 2 return func2(local_var)
def func2(arg2): local_var = arg2 + 2 return func3(local_var)
def func3(arg3): local_var = arg2 / 2 return local_var
func1(1)
As you can see below, the error report is much more extensive. Each
frame of the stack is listed, along with:
- the full path to the source file, instead of just the base name
- the values of the arguments to each function in the stack
- a few lines of source context from around the line in the error path
- the values of variables in the expression causing the error
$ python cgitb_extended_traceback.py/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/cgitb.py:245: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 value = pydoc.text.repr(getattr(evalue, name))<type 'exceptions.NameError'>Python 2.6.2: /Users/dhellmann/.virtualenvs/pymotw/bin/pythonSat Jan 30 12:50:27 2010
A problem occurred in a Python script. Here is the sequence offunction calls leading up to the error, in the order they occurred.
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_extended_traceback.py in <module>() 21 def func3(arg3): 22 local_var = arg2 / 2 23 return local_var 24 25 func1(1)func1 = <function func1 at 0x7bbb0>
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_extended_traceback.py in func1(arg1=1) 13 def func1(arg1): 14 local_var = arg1 * 2 15 return func2(local_var) 16 17 def func2(arg2):global func2 = <function func2 at 0x7bbf0>local_var = 2
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_extended_traceback.py in func2(arg2=2) 17 def func2(arg2): 18 local_var = arg2 + 2 19 return func3(local_var) 20 21 def func3(arg3):global func3 = <function func3 at 0x7bc30>local_var = 4
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_extended_traceback.py in func3(arg3=4) 20 21 def func3(arg3): 22 local_var = arg2 / 2 23 return local_var 24local_var undefinedarg2 undefined<type 'exceptions.NameError'>: global name 'arg2' is not defined __class__ = <type 'exceptions.NameError'> __delattr__ = <method-wrapper '__delattr__' of exceptions.NameError object at 0x81710> __dict__ = {} __doc__ = 'Name not found globally.' __format__ = <built-in method __format__ of exceptions.NameError object at 0x81710> __getattribute__ = <method-wrapper '__getattribute__' of exceptions.NameError object at 0x81710> __getitem__ = <method-wrapper '__getitem__' of exceptions.NameError object at 0x81710> __getslice__ = <method-wrapper '__getslice__' of exceptions.NameError object at 0x81710> __hash__ = <method-wrapper '__hash__' of exceptions.NameError object at 0x81710> __init__ = <method-wrapper '__init__' of exceptions.NameError object at 0x81710> __new__ = <built-in method __new__ of type object at 0x5b0aa0> __reduce__ = <built-in method __reduce__ of exceptions.NameError object at 0x81710> __reduce_ex__ = <built-in method __reduce_ex__ of exceptions.NameError object at 0x81710> __repr__ = <method-wrapper '__repr__' of exceptions.NameError object at 0x81710> __setattr__ = <method-wrapper '__setattr__' of exceptions.NameError object at 0x81710> __setstate__ = <built-in method __setstate__ of exceptions.NameError object at 0x81710> __sizeof__ = <built-in method __sizeof__ of exceptions.NameError object at 0x81710> __str__ = <method-wrapper '__str__' of exceptions.NameError object at 0x81710> __subclasshook__ = <built-in method __subclasshook__ of type object at 0x5b0aa0> __unicode__ = <built-in method __unicode__ of exceptions.NameError object at 0x81710> args = ("global name 'arg2' is not defined",) message = "global name 'arg2' is not defined"
The above is a description of an error in a Python program. Here isthe original traceback:
Traceback (most recent call last): File "cgitb_extended_traceback.py", line 25, in <module> func1(1) File "cgitb_extended_traceback.py", line 15, in func1 return func2(local_var) File "cgitb_extended_traceback.py", line 19, in func2 return func3(local_var) File "cgitb_extended_traceback.py", line 22, in func3 local_var = arg2 / 2NameError: global name 'arg2' is not defined
The end of the output also includes the full details of the exception
object (in case it has attributes other than message that would be
useful for debugging) and the original form of a traceback dump.
Local Variables in Tracebacks
Having access to the variables involved in the error stack can help
find a logical error that occurs somewhere higher in the stack than
the line where the actual exception is generated.
import cgitbcgitb.enable(format='text')
def func2(a, divisor): return a / divisor
def func1(a, b): c = b - 5 return func2(a, c)
func1(1, 5)
In the case of this code with a ZeroDivisionError, we can see that the
problem is introduced in the computation of the value of c in
func1(), rather than where the value is used in func2().
$ python cgitb_local_vars.py/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/cgitb.py:245: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 value = pydoc.text.repr(getattr(evalue, name))<type 'exceptions.ZeroDivisionError'>Python 2.6.2: /Users/dhellmann/.virtualenvs/pymotw/bin/pythonSat Jan 30 12:50:27 2010
A problem occurred in a Python script. Here is the sequence offunction calls leading up to the error, in the order they occurred.
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_local_vars.py in <module>() 16 def func1(a, b): 17 c = b - 5 18 return func2(a, c) 19 20 func1(1, 5)func1 = <function func1 at 0x7bbf0>
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_local_vars.py in func1(a=1, b=5) 16 def func1(a, b): 17 c = b - 5 18 return func2(a, c) 19 20 func1(1, 5)global func2 = <function func2 at 0x7bbb0>a = 1c = 0
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_local_vars.py in func2(a=1, divisor=0) 12 13 def func2(a, divisor): 14 return a / divisor 15 16 def func1(a, b):a = 1divisor = 0<type 'exceptions.ZeroDivisionError'>: integer division or modulo by zero __class__ = <type 'exceptions.ZeroDivisionError'> __delattr__ = <method-wrapper '__delattr__' of exceptions.ZeroDivisionError object at 0x825a8> __dict__ = {} __doc__ = 'Second argument to a division or modulo operation was zero.' __format__ = <built-in method __format__ of exceptions.ZeroDivisionError object at 0x825a8> __getattribute__ = <method-wrapper '__getattribute__' of exceptions.ZeroDivisionError object at 0x825a8> __getitem__ = <method-wrapper '__getitem__' of exceptions.ZeroDivisionError object at 0x825a8> __getslice__ = <method-wrapper '__getslice__' of exceptions.ZeroDivisionError object at 0x825a8> __hash__ = <method-wrapper '__hash__' of exceptions.ZeroDivisionError object at 0x825a8> __init__ = <method-wrapper '__init__' of exceptions.ZeroDivisionError object at 0x825a8> __new__ = <built-in method __new__ of type object at 0x5af9c0> __reduce__ = <built-in method __reduce__ of exceptions.ZeroDivisionError object at 0x825a8> __reduce_ex__ = <built-in method __reduce_ex__ of exceptions.ZeroDivisionError object at 0x825a8> __repr__ = <method-wrapper '__repr__' of exceptions.ZeroDivisionError object at 0x825a8> __setattr__ = <method-wrapper '__setattr__' of exceptions.ZeroDivisionError object at 0x825a8> __setstate__ = <built-in method __setstate__ of exceptions.ZeroDivisionError object at 0x825a8> __sizeof__ = <built-in method __sizeof__ of exceptions.ZeroDivisionError object at 0x825a8> __str__ = <method-wrapper '__str__' of exceptions.ZeroDivisionError object at 0x825a8> __subclasshook__ = <built-in method __subclasshook__ of type object at 0x5af9c0> __unicode__ = <built-in method __unicode__ of exceptions.ZeroDivisionError object at 0x825a8> args = ('integer division or modulo by zero',) message = 'integer division or modulo by zero'
The above is a description of an error in a Python program. Here isthe original traceback:
Traceback (most recent call last): File "cgitb_local_vars.py", line 20, in <module> func1(1, 5) File "cgitb_local_vars.py", line 18, in func1 return func2(a, c) File "cgitb_local_vars.py", line 14, in func2 return a / divisorZeroDivisionError: integer division or modulo by zero
The code in cgitb that examines the variables used in the stack frame
leading to the error is smart enough to evaluate object attributes to
display them, too.
import cgitbcgitb.enable(format='text')
class BrokenClass(object): """This class has an error. """
def __init__(self, a, b): """Be careful passing arguments in here. """ self.a = a self.b = b self.c = self.a * self.b self.d = self.a / self.b return
o = BrokenClass(1, 0)
Here we see that self.a and self.b are involved in the
error-prone code.
$ python cgitb_with_classes.py/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/cgitb.py:245: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 value = pydoc.text.repr(getattr(evalue, name))<type 'exceptions.ZeroDivisionError'>Python 2.6.2: /Users/dhellmann/.virtualenvs/pymotw/bin/pythonSat Jan 30 12:50:27 2010
A problem occurred in a Python script. Here is the sequence offunction calls leading up to the error, in the order they occurred.
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_with_classes.py in <module>() 24 return 25 26 o = BrokenClass(1, 0) 27 28o undefinedBrokenClass = <class '__main__.BrokenClass'>
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_with_classes.py in __init__(self=<__main__.BrokenClass object at 0x81e30>, a=1, b=0) 21 self.b = b 22 self.c = self.a * self.b 23 self.d = self.a / self.b 24 return 25self = <__main__.BrokenClass object at 0x81e30>self.d undefinedself.a = 1self.b = 0<type 'exceptions.ZeroDivisionError'>: integer division or modulo by zero __class__ = <type 'exceptions.ZeroDivisionError'> __delattr__ = <method-wrapper '__delattr__' of exceptions.ZeroDivisionError object at 0x7f710> __dict__ = {} __doc__ = 'Second argument to a division or modulo operation was zero.' __format__ = <built-in method __format__ of exceptions.ZeroDivisionError object at 0x7f710> __getattribute__ = <method-wrapper '__getattribute__' of exceptions.ZeroDivisionError object at 0x7f710> __getitem__ = <method-wrapper '__getitem__' of exceptions.ZeroDivisionError object at 0x7f710> __getslice__ = <method-wrapper '__getslice__' of exceptions.ZeroDivisionError object at 0x7f710> __hash__ = <method-wrapper '__hash__' of exceptions.ZeroDivisionError object at 0x7f710> __init__ = <method-wrapper '__init__' of exceptions.ZeroDivisionError object at 0x7f710> __new__ = <built-in method __new__ of type object at 0x5af9c0> __reduce__ = <built-in method __reduce__ of exceptions.ZeroDivisionError object at 0x7f710> __reduce_ex__ = <built-in method __reduce_ex__ of exceptions.ZeroDivisionError object at 0x7f710> __repr__ = <method-wrapper '__repr__' of exceptions.ZeroDivisionError object at 0x7f710> __setattr__ = <method-wrapper '__setattr__' of exceptions.ZeroDivisionError object at 0x7f710> __setstate__ = <built-in method __setstate__ of exceptions.ZeroDivisionError object at 0x7f710> __sizeof__ = <built-in method __sizeof__ of exceptions.ZeroDivisionError object at 0x7f710> __str__ = <method-wrapper '__str__' of exceptions.ZeroDivisionError object at 0x7f710> __subclasshook__ = <built-in method __subclasshook__ of type object at 0x5af9c0> __unicode__ = <built-in method __unicode__ of exceptions.ZeroDivisionError object at 0x7f710> args = ('integer division or modulo by zero',) message = 'integer division or modulo by zero'
The above is a description of an error in a Python program. Here isthe original traceback:
Traceback (most recent call last): File "cgitb_with_classes.py", line 26, in <module> o = BrokenClass(1, 0) File "cgitb_with_classes.py", line 23, in __init__ self.d = self.a / self.bZeroDivisionError: integer division or modulo by zero
Adding More Context
Suppose your function includes a lot of inline comments, whitespace,
or other code that makes it very long. Having the default of 5 lines
of context may not be enough help in that case, if the body of the
function is pushed out of the code window displayed. Using a larger
context value when enabling cgitb gets around this.
import cgitbimport sys
context_length = int(sys.argv[1])cgitb.enable(format='text', context=context_length)
def func2(a, divisor): return a / divisor
def func1(a, b): c = b - 5 # Really # long # comment # goes # here. return func2(a, c)
func1(1, 5)
You can pass context to cgitb.enable() to control the amount
of code displayed for each line of the traceback.
$ python cgitb_more_context.py 5/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/cgitb.py:245: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 value = pydoc.text.repr(getattr(evalue, name))<type 'exceptions.ZeroDivisionError'>Python 2.6.2: /Users/dhellmann/.virtualenvs/pymotw/bin/pythonSat Jan 30 12:50:27 2010
A problem occurred in a Python script. Here is the sequence offunction calls leading up to the error, in the order they occurred.
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_more_context.py in <module>() 24 # goes 25 # here. 26 return func2(a, c) 27 28 func1(1, 5)func1 = <function func1 at 0x7bbb0>
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_more_context.py in func1(a=1, b=5) 24 # goes 25 # here. 26 return func2(a, c) 27 28 func1(1, 5)global func2 = <function func2 at 0x7bb70>a = 1c = 0
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_more_context.py in func2(a=1, divisor=0) 15 16 def func2(a, divisor): 17 return a / divisor 18 19 def func1(a, b):a = 1divisor = 0<type 'exceptions.ZeroDivisionError'>: integer division or modulo by zero __class__ = <type 'exceptions.ZeroDivisionError'> __delattr__ = <method-wrapper '__delattr__' of exceptions.ZeroDivisionError object at 0x816c0> __dict__ = {} __doc__ = 'Second argument to a division or modulo operation was zero.' __format__ = <built-in method __format__ of exceptions.ZeroDivisionError object at 0x816c0> __getattribute__ = <method-wrapper '__getattribute__' of exceptions.ZeroDivisionError object at 0x816c0> __getitem__ = <method-wrapper '__getitem__' of exceptions.ZeroDivisionError object at 0x816c0> __getslice__ = <method-wrapper '__getslice__' of exceptions.ZeroDivisionError object at 0x816c0> __hash__ = <method-wrapper '__hash__' of exceptions.ZeroDivisionError object at 0x816c0> __init__ = <method-wrapper '__init__' of exceptions.ZeroDivisionError object at 0x816c0> __new__ = <built-in method __new__ of type object at 0x5af9c0> __reduce__ = <built-in method __reduce__ of exceptions.ZeroDivisionError object at 0x816c0> __reduce_ex__ = <built-in method __reduce_ex__ of exceptions.ZeroDivisionError object at 0x816c0> __repr__ = <method-wrapper '__repr__' of exceptions.ZeroDivisionError object at 0x816c0> __setattr__ = <method-wrapper '__setattr__' of exceptions.ZeroDivisionError object at 0x816c0> __setstate__ = <built-in method __setstate__ of exceptions.ZeroDivisionError object at 0x816c0> __sizeof__ = <built-in method __sizeof__ of exceptions.ZeroDivisionError object at 0x816c0> __str__ = <method-wrapper '__str__' of exceptions.ZeroDivisionError object at 0x816c0> __subclasshook__ = <built-in method __subclasshook__ of type object at 0x5af9c0> __unicode__ = <built-in method __unicode__ of exceptions.ZeroDivisionError object at 0x816c0> args = ('integer division or modulo by zero',) message = 'integer division or modulo by zero'
The above is a description of an error in a Python program. Here isthe original traceback:
Traceback (most recent call last): File "cgitb_more_context.py", line 28, in <module> func1(1, 5) File "cgitb_more_context.py", line 26, in func1 return func2(a, c) File "cgitb_more_context.py", line 17, in func2 return a / divisorZeroDivisionError: integer division or modulo by zero
Increasing the value gets us enough of the function that we can spot
the problem in the code, again.
$ python cgitb_more_context.py 10/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/cgitb.py:245: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 value = pydoc.text.repr(getattr(evalue, name))<type 'exceptions.ZeroDivisionError'>Python 2.6.2: /Users/dhellmann/.virtualenvs/pymotw/bin/pythonSat Jan 30 12:50:28 2010
A problem occurred in a Python script. Here is the sequence offunction calls leading up to the error, in the order they occurred.
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_more_context.py in <module>() 19 def func1(a, b): 20 c = b - 5 21 # Really 22 # long 23 # comment 24 # goes 25 # here. 26 return func2(a, c) 27 28 func1(1, 5)func1 = <function func1 at 0x7bbb0>
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_more_context.py in func1(a=1, b=5) 19 def func1(a, b): 20 c = b - 5 21 # Really 22 # long 23 # comment 24 # goes 25 # here. 26 return func2(a, c) 27 28 func1(1, 5)global func2 = <function func2 at 0x7bb70>a = 1c = 0
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_more_context.py in func2(a=1, divisor=0) 12 13 context_length = int(sys.argv[1]) 14 cgitb.enable(format='text', context=context_length) 15 16 def func2(a, divisor): 17 return a / divisor 18 19 def func1(a, b): 20 c = b - 5 21 # Reallya = 1divisor = 0<type 'exceptions.ZeroDivisionError'>: integer division or modulo by zero __class__ = <type 'exceptions.ZeroDivisionError'> __delattr__ = <method-wrapper '__delattr__' of exceptions.ZeroDivisionError object at 0x816c0> __dict__ = {} __doc__ = 'Second argument to a division or modulo operation was zero.' __format__ = <built-in method __format__ of exceptions.ZeroDivisionError object at 0x816c0> __getattribute__ = <method-wrapper '__getattribute__' of exceptions.ZeroDivisionError object at 0x816c0> __getitem__ = <method-wrapper '__getitem__' of exceptions.ZeroDivisionError object at 0x816c0> __getslice__ = <method-wrapper '__getslice__' of exceptions.ZeroDivisionError object at 0x816c0> __hash__ = <method-wrapper '__hash__' of exceptions.ZeroDivisionError object at 0x816c0> __init__ = <method-wrapper '__init__' of exceptions.ZeroDivisionError object at 0x816c0> __new__ = <built-in method __new__ of type object at 0x5af9c0> __reduce__ = <built-in method __reduce__ of exceptions.ZeroDivisionError object at 0x816c0> __reduce_ex__ = <built-in method __reduce_ex__ of exceptions.ZeroDivisionError object at 0x816c0> __repr__ = <method-wrapper '__repr__' of exceptions.ZeroDivisionError object at 0x816c0> __setattr__ = <method-wrapper '__setattr__' of exceptions.ZeroDivisionError object at 0x816c0> __setstate__ = <built-in method __setstate__ of exceptions.ZeroDivisionError object at 0x816c0> __sizeof__ = <built-in method __sizeof__ of exceptions.ZeroDivisionError object at 0x816c0> __str__ = <method-wrapper '__str__' of exceptions.ZeroDivisionError object at 0x816c0> __subclasshook__ = <built-in method __subclasshook__ of type object at 0x5af9c0> __unicode__ = <built-in method __unicode__ of exceptions.ZeroDivisionError object at 0x816c0> args = ('integer division or modulo by zero',) message = 'integer division or modulo by zero'
The above is a description of an error in a Python program. Here isthe original traceback:
Traceback (most recent call last): File "cgitb_more_context.py", line 28, in <module> func1(1, 5) File "cgitb_more_context.py", line 26, in func1 return func2(a, c) File "cgitb_more_context.py", line 17, in func2 return a / divisorZeroDivisionError: integer division or modulo by zero
Exception Properties
In addition to the local variables from each stack frame, cgitb shows
all properties of the exception object. If you have a custom
exception type with extra properties, they are printed as part of the
error report.
import cgitbcgitb.enable(format='text')
class MyException(Exception): """Add extra properties to a special exception """
def __init__(self, message, bad_value): self.bad_value = bad_value Exception.__init__(self, message) return
raise MyException('Normal message', bad_value=99)
In this example, the bad_value property is included along with the
standard message and args values.
$ python cgitb_exception_properties.py/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/cgitb.py:245: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 value = pydoc.text.repr(getattr(evalue, name))<class '__main__.MyException'>Python 2.6.2: /Users/dhellmann/.virtualenvs/pymotw/bin/pythonSat Jan 30 12:50:28 2010
A problem occurred in a Python script. Here is the sequence offunction calls leading up to the error, in the order they occurred.
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_exception_properties.py in <module>() 18 self.bad_value = bad_value 19 Exception.__init__(self, message) 20 return 21 22 raise MyException('Normal message', bad_value=99)MyException = <class '__main__.MyException'>bad_value undefined<class '__main__.MyException'>: Normal message __class__ = <class '__main__.MyException'> __delattr__ = <method-wrapper '__delattr__' of MyException object at 0x7f698> __dict__ = {'bad_value': 99} __doc__ = 'Add extra properties to a special exception\n ' __format__ = <built-in method __format__ of MyException object at 0x7f698> __getattribute__ = <method-wrapper '__getattribute__' of MyException object at 0x7f698> __getitem__ = <method-wrapper '__getitem__' of MyException object at 0x7f698> __getslice__ = <method-wrapper '__getslice__' of MyException object at 0x7f698> __hash__ = <method-wrapper '__hash__' of MyException object at 0x7f698> __init__ = <bound method MyException.__init__ of MyException('Normal message',)> __module__ = '__main__' __new__ = <built-in method __new__ of type object at 0x5aebc0> __reduce__ = <built-in method __reduce__ of MyException object at 0x7f698> __reduce_ex__ = <built-in method __reduce_ex__ of MyException object at 0x7f698> __repr__ = <method-wrapper '__repr__' of MyException object at 0x7f698> __setattr__ = <method-wrapper '__setattr__' of MyException object at 0x7f698> __setstate__ = <built-in method __setstate__ of MyException object at 0x7f698> __sizeof__ = <built-in method __sizeof__ of MyException object at 0x7f698> __str__ = <method-wrapper '__str__' of MyException object at 0x7f698> __subclasshook__ = <built-in method __subclasshook__ of type object at 0xa16890> __unicode__ = <built-in method __unicode__ of MyException object at 0x7f698> __weakref__ = None args = ('Normal message',) bad_value = 99 message = 'Normal message'
The above is a description of an error in a Python program. Here isthe original traceback:
Traceback (most recent call last): File "cgitb_exception_properties.py", line 22, in <module> raise MyException('Normal message', bad_value=99)MyException: Normal message
Logging Tracebacks
For many situations, printing the traceback details to standard error
is the best resolution. In a production system, however, logging the
errors is even better. cgitb.enable() includes an optional
argument, logdir, to enable error logging. When a directory name
is provided, each exception is logged to its own file in the given
directory.
import cgitbimport os
cgitb.enable(logdir=os.path.join(os.path.dirname(__file__), 'LOGS'), display=False, format='text', )
def func2(a, divisor): return a / divisor
def func1(a, b): c = b - 5 return func2(a, c)
func1(1, 5)
Event with the error display suppressed, a message is printed
describing where to go to find the error log.
$ python cgitb_log_exception.py/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/cgitb.py:245: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 value = pydoc.text.repr(getattr(evalue, name))<p>A problem occurred in a Python script.<p> /Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/LOGS/tmpohlTP8.txt contains the description of this error.
$ ls LOGStmpohlTP8.txt
$ cat LOGS/*.txt<type 'exceptions.ZeroDivisionError'>Python 2.6.2: /Users/dhellmann/.virtualenvs/pymotw/bin/pythonSat Jan 30 12:50:28 2010
A problem occurred in a Python script. Here is the sequence offunction calls leading up to the error, in the order they occurred.
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_log_exception.py in <module>() 21 def func1(a, b): 22 c = b - 5 23 return func2(a, c) 24 25 func1(1, 5)func1 = <function func1 at 0x7bbb0>
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_log_exception.py in func1(a=1, b=5) 21 def func1(a, b): 22 c = b - 5 23 return func2(a, c) 24 25 func1(1, 5)global func2 = <function func2 at 0x7bb70>a = 1c = 0
/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/cgitb/cgitb_log_exception.py in func2(a=1, divisor=0) 17 18 def func2(a, divisor): 19 return a / divisor 20 21 def func1(a, b):a = 1divisor = 0<type 'exceptions.ZeroDivisionError'>: integer division or modulo by zero __class__ = <type 'exceptions.ZeroDivisionError'> __delattr__ = <method-wrapper '__delattr__' of exceptions.ZeroDivisionError object at 0x81710> __dict__ = {} __doc__ = 'Second argument to a division or modulo operation was zero.' __format__ = <built-in method __format__ of exceptions.ZeroDivisionError object at 0x81710> __getattribute__ = <method-wrapper '__getattribute__' of exceptions.ZeroDivisionError object at 0x81710> __getitem__ = <method-wrapper '__getitem__' of exceptions.ZeroDivisionError object at 0x81710> __getslice__ = <method-wrapper '__getslice__' of exceptions.ZeroDivisionError object at 0x81710> __hash__ = <method-wrapper '__hash__' of exceptions.ZeroDivisionError object at 0x81710> __init__ = <method-wrapper '__init__' of exceptions.ZeroDivisionError object at 0x81710> __new__ = <built-in method __new__ of type object at 0x5af9c0> __reduce__ = <built-in method __reduce__ of exceptions.ZeroDivisionError object at 0x81710> __reduce_ex__ = <built-in method __reduce_ex__ of exceptions.ZeroDivisionError object at 0x81710> __repr__ = <method-wrapper '__repr__' of exceptions.ZeroDivisionError object at 0x81710> __setattr__ = <method-wrapper '__setattr__' of exceptions.ZeroDivisionError object at 0x81710> __setstate__ = <built-in method __setstate__ of exceptions.ZeroDivisionError object at 0x81710> __sizeof__ = <built-in method __sizeof__ of exceptions.ZeroDivisionError object at 0x81710> __str__ = <method-wrapper '__str__' of exceptions.ZeroDivisionError object at 0x81710> __subclasshook__ = <built-in method __subclasshook__ of type object at 0x5af9c0> __unicode__ = <built-in method __unicode__ of exceptions.ZeroDivisionError object at 0x81710> args = ('integer division or modulo by zero',) message = 'integer division or modulo by zero'
The above is a description of an error in a Python program. Here isthe original traceback:
Traceback (most recent call last): File "cgitb_log_exception.py", line 25, in <module> func1(1, 5) File "cgitb_log_exception.py", line 23, in func1 return func2(a, c) File "cgitb_log_exception.py", line 19, in func2 return a / divisorZeroDivisionError: integer division or modulo by zero
HTML Output
Of course, since cgitb was originally developed for handling
exceptions in web apps, no discussion would be complete without an
example of the HTML output it produces.
import cgitbcgitb.enable()
def func1(arg1): local_var = arg1 * 2 return func2(local_var)
def func2(arg2): local_var = arg2 + 2 return func3(local_var)
def func3(arg3): local_var = arg2 / 2 return local_var
func1(1)
By leaving out the format argument (or specifying html), the
traceback format changes to HTML output.
See also
- cgitb
- The standard library documentation for this module.
- traceback
- Standard library module for working with tracebacks.
- inspect
- The inspect module includes more functions for examining the
stack. - sys
- The sys module provides access to the current exception value
and the excepthook handler invoked when an exception
occurs. - Improved traceback module
- Python-dev discussion of improvements to the traceback module
and related enhancements other developers use locally.
The canonical version of this article
Testing with PostgreSQL
My new personal project has a PostgreSQL database. Here’s how I’m handling testing.
Menno’s Musings: rst_break plugin for PyBlosxom
I just scratched an itch by writing a small plugin for PyBlosxom that
allows the rst (reStructured Text) and readmore plugins to work
together [1]. It defines a reST "break" directive which gets transformed
into the breakpoint string the readmore plugin looks out for. This
allows for "Read more…" breaks to be inserted in for reST based
articles.
For further information see the Code page here and at the top of the
plugin itself.
| [1] | Yes, the audience for this plugin is probably tiny! |
Mario Boikov: Named tuple
I recently read a post on planet python and the author mentioned something about named tuple which made my curious.
So what’s a named tuple?
The namedtuple was introduced in Python 2.6. A named tuple is created using a factory function from the collections module and it extends the basic tuple by assigning a name to each position in a tuple but can still be used as a regular tuple. This makes it possible to access fields by name instead of an index. The named tuple should not require more memory, according to the documentation, than regular tuples since they don’t have a per instance dictionary.
The factory function signature is:
collections.namedtuple(typename, field_names[, verbose])
The first argument specifies the name of the new type, the second argument is a string (space or comma separated) containing the field names and finally if verbose is true the factory function will also print the class generated.
Enough theory, I’ll show you an example.
Example
Say you have a tuple containing username and password. To access the username you get the item at position zero and the password is accessed at position one:
credential = ('mario', 'secret')print 'Username:', credential[0]print 'Password:', credential[1]
There’s nothing wrong with this code but the tuple isn’t self-documented. You have to find and read the documentation about the positioning of the fields in the tuple. This is where named tuple can enter the scene. We can rewrite the previous example as following:
import collections# Create a new sub-tuple named CredentialCredential = collections.namedtuple('Credential', 'username, password')
credential = Credential(username='mario', password='secret')
print 'Username:', credential.usernameprint 'Password:', credential.password
Nice, don’t you agree?
If you are interested of what the code looks like for the newly created Credential-type you can add verbose=True to the argument list when creating the type, in this particular case we get the following output:
import collectionsCredential = collections.namedtuple('Credential', 'username, password', verbose=True)
class Credential(tuple): 'Credential(username, password)'
__slots__ = ()
_fields = ('username', 'password')
def __new__(_cls, username, password): return _tuple.__new__(_cls, (username, password))
@classmethod def _make(cls, iterable, new=tuple.__new__, len=len): 'Make a new Credential object from a sequence or iterable' result = new(cls, iterable) if len(result) != 2: raise TypeError('Expected 2 arguments, got %d' % len(result)) return result
def __repr__(self): return 'Credential(username=%r, password=%r)' % self
def _asdict(t): 'Return a new dict which maps field names to their values' return {'username': t[0], 'password': t[1]}
def _replace(_self, **kwds): 'Return a new Credential object replacing specified fields with new values' result = _self._make(map(kwds.pop, ('username', 'password'), _self)) if kwds: raise ValueError('Got unexpected field names: %r' % kwds.keys()) return result
def __getnewargs__(self): return tuple(self)
username = _property(_itemgetter(0)) password = _property(_itemgetter(1))
The named tuple doesn’t only provide access to fields by name but also contains helper functions such as the _make() function which helps creating an Credential instance from a sequence or iterable. For example:
cred_tuple = ('mario', 'secret')credential = Credential._make(cred_tuple)
There are more interesting use-cases and examples in the documentation, so I suggest that you take a peek.
Comments
I think the named tuple is useful. They remove the error-prone indexing in tuples by providingaccess to fields by name without adding any memory overhead. They are also regular Python classes which means you can do anything you can do with classes.
Richard Tew: Tracking class instantiations
One of the experiments I have been doing based on code reloading, is to among other things, add the ability to react to instantiations of arbitrary classes. The way I go about doing this is to react to the definition of a class by giving it a new __init__ method.
If the class came with a custom __init__ method, then this is the easiest case to handle. When my replacement method is called, I can just call the original custom method before I notify whatever is interested.
def init_wrapper(self, *args, **kwargs): class_.__real_init__(self, *args, **kwargs) events.Register(self)
What gets a little more complex, is the case when the class does not have a custom __init__ method defined. In this situation, I still need to notify whatever is interested, but in order to preserve existing behaviour I need to pass the call down to the base classes.
def init_standin(self, *args, **kwargs): if type(self) is types.InstanceType: # Old-style.. for baseClass in class_.__bases__: if hasattr(baseClass, "__init__"): baseClass.__init__(self, *args, **kwargs) else: # New-style.. super(class_, self).__init__(*args, **kwargs) events.Register(self)
The thing that really complicates this, is the class_ variable used in the both snippets included above. This needs to refer to the class that the given method being executed is defined on. I cannot infer the class from any of the existing variables, so it needs to be provided in some other way. Because I know it when I install the __init__ method, I define these functions inline so their use of it includes it in their closures.
def MonitorClassInstantiation(self, class_): def init_wrapper(self, *args, **kwargs): class_.__real_init__(self, *args, **kwargs) events.Register(self)
def init_standin(self, *args, **kwargs): if type(self) is types.InstanceType: for baseClass in class_.__bases__: if hasattr(baseClass, "__init__"): baseClass.__init__(self, *args, **kwargs) else: super(class_, self).__init__(*args, **kwargs) events.Register(self)
init_real = class_.__dict__.get("__init__", None) if init_real is None: class_.__init__ = init_standin else: class_.__init__ = init_wrapper class_.__real_init__ = init_real
With this working, the remaining concern is the effect of a hierarchy of classes modified in this way. Each has one of these two new __init__ methods, and the subclasses call down through them to the base classes. For now, I have a WeakKeyDictionary of instances in the events handler that ignores calls from instances that have already called.
Source code: events.py
Andrew Dalke: Project hosting options?
I started a new project for cheminformatics fingerprints and want to
make it available for general use. It contains software under the MIT
license and specifications under a license as lenient as I can make
it. (Likely CC-BY.)
I looked around for project hosting. My requirements are:
- Mailing list with only Mailman-style double opt-in. Specifically,
I expect most people who subscribe will not want to have to set up an
account on the project hosting provider before signing up for the
mailing list. - Mercurial support
- Simple web hosting, preferable a wiki where I can put up
specifications and a few related documents and where others can do
some edits - Bug and issue tracking would be nice, but not essential since
this is a small project and a TODO in version control should be
fine.
That’s it. Very simple, yes?
The options
I know there’s a bunch of resources these days, and in my searches I
found Wikipedia’s
Comparison
of open source software hosting facilities. As you can see, there
are quite a few. Sort on version control systems and it’s Alioth,
Assembla, BerliOS, Bitbucket, CodePlex, GNU Savannah, Google Code,
JavaForge, KnowledgeForge, Project Kenai, and SourceForge.
Must have mailing list and web page or wiki hosting
Next, filter out those which don’t have mailing lists, which removes
Assembla, Bitbucket, and JavaForge. It’s a shame about losing
BitBucket since that’s what I would have liked. With reluctance I also
dropped Google Code since its mailing lists require a Google account.
I think that’s too high of a barrier of entry. I also dropped GNU
Savannah since it doesn’t have web or wiki hosting.
What’s left are: Alioth, BerliOS, CodePlex, KnowledgeForge, Project
Kenai, and SourceForge.
I want to try something other than SourceForge or a clone
Of those I have only used SourceForge, and done that for over 10
years. It feels very clunky and cluttered compared to Google Code and
downloading packages is a nuisance for people like me who would rather
curl the files than use a browser. Perhaps it’s time to try something
different? That puts BerliOS out, since it’s derived from the
SourceForge code base, as is GNU Savannah, and so is Alioth through
GForge.
What’s left? CodePlex, KnowledgeForge, and Project Kenai.
Must support non-member access to a mailing list
I looked at CodePlex. I think you have to be a CodePlex member in
order to leave dicussions, and it uses web-based forum software
instead of email. That is, I selected some of the project which have
been downloaded the most often but never could find a “subscribe to
the mailing list” option. Perhaps most people in the Microsoft Windows
and .Net space don’t do email?
In any case, it doesn’t seem to fit my requirements.
Remaining options: KnowledgeForget and Project Kenai
I looked at KnowledgeForge and while it seems to fit my requirements,
there aren’t many people using it, although others may be using the
underlying KForge application to host their own system. My concern is
that the rough edges wouldn’t have been worn down by other users.
That left me with Project Kenai, which also seemed to do what I
wanted, and it has more and larger development projects, including JRuby. Okay, I’ll try it out.
Project Kenai
(Update based on feedback.
As of 27 Jan 2010 (or about two days after I registered on Kenai, and
two days before I posted this essay), Oracle, who owns Sun, said they
would be “phasing
out of the public-facing domain used for the Project Kenai Beta
site.” Therefore, you shouldn’t use it.)
I requested a new project hosting and got it. I set up the project,
working on code, and updated the wiki. Seems to be nice enough, with
really no problems to speak about. I was happy enough.
I liked some of the tweaks, like how it uses AJAX to update the
displayed content rather than doing a full page submission like when
editing Wikipedia. Though now that I think of it, I adore how
StackOverflow shows the formatted content while you type.
Show stopper – non-member access to the mailing list
Until I got to the email part. Turns out Project Kenai does
allow non-members to join a list, but they have to email
subscribe request to the Sympa email system. Very much like the
old majordomo list manager, and with no web-based front-end to help
out.
I found that be searching the help files. There’s no clue that that’s
even possible from the normal “mailing lists” page for a project. But
perhaps I could remedy that with instructions on how to sign up
without being a member.
The only way I could do that was on the wiki home page. I did that
then asked a friend of mine to try it out. He followed the main
mailing-list link from Kenai and never saw my note. Once on that page
he couldn’t figure out how to join without being a member, and he
doesn’t want to do all that just to join a mailing list.
Once I pointed out the manual instructions, he tried that out. I got
an email which said I need to manualy confirm him as a member. On his
side he only saw that he was now a member, and didn’t like the lack of
the Mailman-style double opt-in. As far as he could tell, anyone could
register anyone else through a forged email.
That’s a serious down-check, since while technically it meets my
requirements, it doesn’t meet the spirit.
No response to a feature requst
I posted this request to the features list a couple of days ago and got no
response.
I do realize this is a free project, so I can make no demands nor
should I expect fast response. That’s why i waited a couple of days
before writing this posting. But a reason for trying Project Kenai was
because its size should mean it has more of these kinks worked out,
and its support by Sun should imply there’s someone to answer mail.
Just choose SourceForge?
As for the project, my conculsion is to just go ahead and use
SourceForge. It’s clumsy but I know it handles my needs.
Unless you have a better
suggestion? Perhaps you think I should try BerliOS?
Published in PHP|Architect – Evert Pot
The January issue of PHP|Architect just got published, and I’m in it!
I’ve written an apparent 5-page article about SabreDAV. The process was a lot of fun. It’s definitely quite different from blogging, where quick and dirty is the rule rather than the exception. I hope I get the chance to do this again (all provided if there’s another subject I feel knowledgeable enough in).
So if you get a hold of a copy, I would be very interested in your feedback! The magazine is in PDF form these days, so subscribing and downloading is quick and easy =).
Rainy Day Outage (Update Three)
Not a lot of new news.We got a new motherboard for the system, and have installed it…. but it’s not working. Which leads us to believe that its actually the powersupply thats shot. We’re on the hunt for a compatible powersupply locally,…
« go back — keep looking »Warning: include(/home/remarkwit/enterpriselamp.org/wp-content/themes/Enterprise_LAMP/r_sidebar.php) [function.include]: failed to open stream: No such file or directory in /home/remarkwit/enterpriselamp.org/wp-content/themes/Enterprise_LAMP/archive.php on line 23
Warning: include() [function.include]: Failed opening '/home/remarkwit/enterpriselamp.org/wp-content/themes/Enterprise_LAMP/r_sidebar.php' for inclusion (include_path='.:/usr/local/lib/php:/usr/local/php5/lib/pear') in /home/remarkwit/enterpriselamp.org/wp-content/themes/Enterprise_LAMP/archive.php on line 23