Codexon

Have you ever wanted to rewrite your program while it was running? According to wikipedia, this feature is called live coding.

This is a big deal for applications such as internet servers and the telecom industry. Restarting the program every time you make a change is unnecessary downtime when you would like to have +99.9% uptime.

Some languages were made with this feature in mind, such as Erlang and ColdC. However for Python, it is not as easy.

Most people just look at the reload() function for Python and are satisfied with that. Unfortunately, the deeper you dig into it, the more deficiencies you will see.

Example

You make a module file.

# mod.py
class foo: value=5

In your prompt you type:
>>> import mod
>>> a = mod.foo()
>>> a.value
5

Then you edit the module file again.

# mod.py
class foo: value=1

Its still 5! Why is it not 1?

It is because the variable “a” is still linked to the old module. The new module merely replaced the name “mod”. All calls to mod now point to the new code.

But the old code still exists, and will live as long as any variable uses it, such as the object at ‘a’.

>>> b = mod.foo()
>>> b.value
1

The class of b and a are different, even though they come from the same file.
>>> a.__class__ is b.__class__
False

So how do we go about solving this problem without using another language? We have two options.

1. Make a special class that records all it’s instances. Then when you reload, you tell all the instances to change their __bases__ variable to the new version.

2. Copy all the member variables from the new object to the old object. This seems to be the best choice.

Here is a replacement for the reload function. However, note that there is a strange piece of code that looks like the following:

class object(object):pass
__builtin__.object = object

This piece of code replaces the default “object” class in order to bypass an unfixed 6 year old problem in the python interpreter. http://bugs.python.org/issue672115

import sys, types
import __builtin__
 
# override the regular object type
# http://bugs.python.org/issue672115
class object(object):pass
__builtin__.object = object
 
# save the old reload
oldreload = reload
 
# Here is a list of datatypes we want to update.
# We want to update class definitions and functions
types_to_update = (type, types.FunctionType)
 
# the replacement function
def reload(mod):
 
    # record all the variables we want at the lowest level only.
    oldstuff = []
    for key, obj in mod.__dict__.iteritems():
        if type(obj) in types_to_update:
            oldstuff.append( (key,obj) )
 
    # perform the regular reload
    oldreload(mod)
 
    # if the old classes are in the new one, then update them, and copy them over.
    for key, obj in oldstuff:
        if key in mod.__dict__:
            update2(obj, mod.__dict__[key])
            mod.__dict__[key] = obj
 
# the object updating function.
# all of the data from "b" gets ripped and placed into "a"
def update2(a,b):
 
    for varname in dir(a):
        try: delattr(a, varname)
        except: pass
 
    for varname in dir(b):
        newvalue = getattr(b, varname)
        try: setattr(a, varname, newvalue)
        except: pass

Caveats:

1. The entire program should be paused while the update takes place. Bugs could result if part of your program accesses old and new code in one go.
2. Some objects can never be updated in place because their member variables are uncopyable or unchangable. Forunately, regular classes and functions seem to work.
3. Only things defined at the lowest level of the module will be able to be updated reliably. This means you can’t update things defined inside methods and functions.

You can’t update ‘b’ in a module like this. You can only update a.

class a:
    def make(self):
        return class b: pass

Related posts:

1. Memory Size of Python Objects
2. Clearing Passwords in Memory with Python
3. How to customize Eclipse to be the best Python IDE with PyDev
4. PHP Sucks: No stable sort
5. Finding the Current Address in a C Program