Grig Gheorghiu: Use HAProxy 1.4 if you need MySQL health checks

I was using the latest version of HAProxy 1.3 and was load balancing backend MySQL servers while also checking their ports, so if one server went down it would be taken out of the load balancing pool. However, since the port checks in HAProxy happen at the TCP level, the MySQL instance which was being hit by the port checks wasn’t happy, because it wasn’t a proper MySQL connection. As a result, after a number of some checks, MySQL refused to allow clients to connect, with a message like this:

OperationalError: (1129, “Host ‘myhost’ is blocked because of many connection errors; unblock with ‘mysqladmin flush-hosts’”)

Solution: upgrade HAProxy to the newly released version 1.4 (at the date of this writing, the exact version is 1.4.0). Documentation is here.

For MySQL specific checks, you can specify ‘option mysql-check’ in a backend or ‘listen’ section of the configuration file. For example, I have something similar to this in my HAProxy configuration file:

listen mysql 0.0.0.0:3306
mode tcp
option mysql-check
balance roundrobin
server mysql1 10.0.1.1:3306 check port 3306
server mysql2 10.0.1.2:3306 check port 3306 backup

Month of PHP Security 2010 – CALL FOR PAPERS – Stefan Esser

I previously blogged a sneak preview of the Month of PHP Security which is a new initiative to improve security in the PHP ecosystem. Today the call for papers was released. Everyone from the PHP and security community is invited to produce quality articles/advisories about PHP security topics/bugs and submit them to the CFP committee.

The event is generously sponsored by Syscan, SektionEins and CodeScan. And the best submissions can win a number of attractive prizes. The first prize consists of 1000 EUR, a free Syscan ticket and a free CodeScan PHP License. For a full list of the submissions accepted and prizes available check out the website.

In case you are not a PHP security expert you can still help to improve the event. Spread the word about the Month of PHP Security in your blog, link to it and announce your blog posting at drawing@php-security.org and win one of ten 25 EUR amazon coupons.

Mark Mruss: An Introduction to Google Calendars

Note: This article was first published the March 2008 issue of Python Magazine

Mark Mruss

Over the past few years Google has expanded it’s services beyond those of a normal search engine. One of those new services is the Google Calendar. This article will provide an introduction to working with the Google Calendar using Python.

Introduction

As many of you know, Google has branched out and started offering more services besides their ubiquitous search engine. You have email, calendars, documents, spreadsheets, photos, maps, videos, source code hosting, and the list goes on. Fortunately for us Python programmers, Google released the Google data Python Client Library on March 26th, 2007, giving Python programmers easy access to some of these services.

What the Google data Python Client Library, or “gdata-python-client”, does is provide “a library and source code that makes it easy to access data through Google Data APIs.” [1] This leads to the question: “what are the Google Data APIs?” In the words of Google: “The Google data APIs provide a simple standard protocol for reading and writing data on the web. These APIs use either of two standard XML-based syndication formats: Atom or RSS.”[2]

The Google services that use the Google data APIs include many of the services that we have grown to know and love:

Google Apps
Google Base
Blogger
Google Calendar
Google Code Search
Google Documents
Google Notebook
Picasa Web Albums
Spreadsheets
YouTube

This tutorial will only deal with the Google Calendar service specifically, but it’s important to know that many of the techniques used here can easily be applied to other Google services.

The Google data Python Client Library requires Python 2.2 or greater and the ElementTree module to be installed. I recommend using Python 2.5 since the ElementTree module is included in that release of Python. This column assumes that you are using Python 2.5 and version 1.0.10 of the “gdata-python-client”.

Getting and Installing the “gdata-python-client”

We must first download and install the “gdata-python-client” files from the “gdata-python-client” website.[3] Once you have downloaded the compressed file, extract it’s contents to a folder. You will then need to browse to that folder and run the extracted setup.py file with the install command with root access. For me, the command looked like this:

# python2.5 setup.py install

Notice that I ran python2.5 instead of simply python. I did this to ensure that Python 2.5 was used instead of a previous version of Python.

Once you have done this, you can test to make sure that the gdata-python-client module has been installed properly by trying to import the gdata module. You can test this easily in the interactive Python shell:

$ python2.5
Python 2.5.2a0 (r251:54863, Jan  3 2008, 17:59:56)
[GCC 4.2.3 20071123 (prerelease) (Debian 4.2.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import gdata
>>>

If everything works properly you should see no errors when importing gdata. If you do run into problems, you can consult the install.txt file that was extracted with the rest of the files in the “gdata-python-client” archive that you downloaded.

Getting Started

There are some great Google Calendar examples that come with the “gdata-python-client” module. I found them to be very useful learning tools. The “Google Calendar Developers Guide”[4] is also very helpful. If you get stuck it is essential reading. The “Google Calendar API Reference Guide”[5] is also an indispensable help.

It is important to remember that when you are working with the Google Calendar service, you are actually working with XML data. You are sending and receiving XML data to and from the Google Calendar service – Atom or RSS feeds to be precise. The Python classes that wrap these feeds, or XML blocks, are dynamically “formed” around the XML. When you access something like the following:

e_link.href

You are actually accessing the “href” attribute of an XML link block that may look something like this:

<ns0 :link href="http://www.google.com/calendar/feeds/<username>/private/full" rel="alternate" type="application/atom+xml" xmlns:ns0="http:www.w3.org/2005/Atom" />

In the above example, if we had the link as an instance (perhaps an atom.Link object), e_link.rel would be equal to “alternate”.

Logging In

There are two types of Google calendars – public calendars and private calendars. Public calendars do not require any authentication while private calendars do. There are three forms of authentication for private calendars: “AuthSub proxy” authentication, “ClientLogin” authentication, and “Magic cookie” authentication.

“AuthSub proxy” authentication is meant for web applications so it will not be covered in this tutorial. “Magic cookie” authentication requires a string (the magic cookie) obtained from your Google Calendar settings page. This type of authentication gives you read only access to a private calendar. Its usage is quite specific and will not be covered in this column either.

For this column we will focus on good old-fashioned “ClientLogin” authentication. In order to perform “ClientLogin” authentication we need two things: a Google Calendar username and a password. If you do not have these, head over to the Google Calendar website [6] and get yourself signed up. If you have gmail or have signed up for other Google services, you can probably sign in using that username and password.

Note: I say username and so does the documentation, but this will actually be an email address.

We are going to read the username and password in from the command line using the raw_input function and the getpass module. I must thank the people who wrote the Google code examples for introducing me to the getpass module. Before looking through the examples, I had not heard of it.

The raw_input function simply reads a line of input from the command line and returns it with the newline stripped. The getpass.getpass function does the exact same thing as raw_input except it does not echo the input, which is handy when working with passwords.

We need to import the getpass module:

import getpass

Then, in our main function we will ask the user to input their username and password:

def main():
    username = raw_input("Enter your username: ")
    password = getpass.getpass("Enter your password: ")

if __name__ == "__main__":
    main()

Now that we have our username and password, we need to log the user in. To do this we are going to use the CalendarService class. The CalendarService “extends the GDataService to streamline Google Calendar operations.” [7] The GDataService class “provides CRUD ops. and programmatic login for GData services.” [8]. CRUD is an acronym that stands for Create, Retrieve, Update, and Delete. What the CalendarService class will allow us to do is create, retrieve, update, and delete things from a Google Calendar.

The first step in logging in is to create an instance of the CalendarService class. To do this we will pass in the username and password that we have collected:

calendar_service = gdata.calendar.service.CalendarService(username
    , password
    , "PythonMagazine-Calendar_Example-1.0")

The third parameter that we pass to the CalendarService constructor is the “source” string. This is a short “string identifying your application, for logging purposes. This string should take the form: “companyName-applicationName-versionID””[9]

Now that we have a CalendarService instance, we need to login. To do this we will use the ProgrammaticLogin function. This will log into the Google Calendar using the CalendarService classes current username and password. The ProgrammaticLogin function can raise three possible exceptions that we want to be aware of:

CaptchaRequired – Raised if the login requires a “captcha” response in order to login.
BadAuthentication – Raised if the username and/or password were not accepted by the Google Calendar.
Error – Raised if a 403 error occurs that is not a “CaptchaRequired” or “BadAuthentication” error.

For this example, shown in Listing 1, we will only worry about the “BadAuthentication” exception.

Listing 1

def main():
    username = raw_input("Enter your username: ")
    password = getpass.getpass("Enter your password: ")

    calendar_service = gdata.calendar.service.CalendarService(username
        , password
        , "PythonMagazine-Calendar_Example-1.0")
    try:
        calendar_service.ProgrammaticLogin()
    except gdata.service.BadAuthentication, e:
        print "Authentication error logging in: %s" % e
        return
    except Exception, e:
        print "Error Logging in: %s" % e
        return

Working With Calendars

Now that we’ve logged into the calendar service, we can start working with the user’s available calendars. To get a list of all of the available calendars you can call the GetAllCalendarsFeed function. This will return a CalendarListFeed instance representing all of the user’s calendars. Since I want to show how to add and delete Calendars, I’m going to use the GetOwnCalendarsFeed function to get a list of all of the calendars that the “authenticated user has owner access to.”[10]

The GetOwnCalendarsFeed function returns a CalendarListFeed class instance. This contains some information (like a title) along with a list of CalendarListEntry classes. Each CalendarListEntry in this list represents a calendar. An example of a function that uses GetOwnCalendarsFeed and then prints out information about each calendar can be found in Listing 2. Sample output from this function can be found in Listing 3. The function was passed a CalendarService after logging in:

list_own_calendars(calendar_service)

Listing 2

def list_own_calendars(calendar_service):

    try:
        #Get the CalendarListFeed
        all_calendars_feed = calendar_service.GetOwnCalendarsFeed()
    except Exception, e:
        print "Error getting all calendar feed: %s" % (e)
        return
    #Print the feed's title
    print all_calendars_feed.title.text
    #Now loop through all of the CalendarListEntry items.
    for (index, cal) in enumerate(all_calendars_feed.entry):
        #Print out the title and the summary if ther is one
        if (cal.summary is not None):
            print "%d) %s - Summary: %s" % (
                index, cal.title.text, cal.summary.text)
        else:
            print "%d) %s" % (index, cal.title.text)
        #Print out the authors
        print "\tAuthors:"
        for author in cal.author:
            print "\t\t%s" % (author.name.text)
        #Print out other information
        print "\tPublished: %s \n\tUpdated: %s \n\ttimezone: %s" % (
            cal.published.text, cal.updated.text, cal.timezone.value)
        print "\tColour: %s \n\tHidden: %s \n\tSelected: %s" % (
            cal.color.value, cal.hidden.value, cal.selected.value)
        print "\tAccess Level: %s" % (cal.access_level.value)

Listing 3

Mark Mruss's Calendar List
0) Mark Mruss - Summary: Main Calendar
	Authors:
		Mark Mruss
	Published: 2008-02-07T04:15:15.701Z
	Updated: 2008-02-07T03:43:26.000Z
	timezone: America/Toronto
	Colour: #5229A3
	Hidden: false
	Selected: true
	Access Level: owner
1) PythonMagazine - Summary: Calendar for Articles
	Authors:
		PythonMagazine
	Published: 2008-02-07T04:15:15.702Z
	Updated: 2008-02-07T03:37:15.000Z
	timezone: America/Toronto
	Colour: #0D7813
	Hidden: false
	Selected: true
	Access Level: owner

Adding a Calendar

If we want to add a calendar, we need to create a CalendarListEntry class instance and then call the CalendarService classes InsertCalendar function. Let’s say I wanted to add a Banking calendar to my account, I could do the following to create the CalendarListEntry:

new_calendar = gdata.calendar.CalendarListEntry()
new_calendar.title = gdata.atom.Title(text="Banking")
new_calendar.summary = gdata.atom.Summary(text="Bills and payments")
new_calendar.timezone = gdata.calendar.Timezone(value="America/Toronto")
new_calendar.hidden = gdata.calendar.Hidden(value="false")
new_calendar.selected = gdata.calendar.Selected(value="true")

The calendar now needs to be added to the account:

try:
    created_calendar = calendar_service.InsertCalendar(new_calendar)
except gdata.service.RequestError, e:
    print "Error adding calendar: %s" % (e[0]["reason"])

This will add the new calendar to the authenticated users list of calendars, make it visible, and select it. Notice that InsertCalendar also returns a CalendarListEntry instance. This is the calendar that was actually added to the authenticated user’s list of calendars. It is wrapping the XML that represents the actual calendar, as opposed to the smaller version that we created for insertion.

Deleting a Calendar

Deleting a Calendar is very easy. You simply need to call the CalendarService classes DeleteCalendarEntry function. The DeleteCalendarEntry function takes three parameters which are documented in the source code:

edit_uri – The edit uri (Uniform Resource Identifier) of the Calendar that you want to delete.
url_params – Defaults to None. A dictionary containing URL parameters that will be included in the delete.
escape_params – Defaults to True. A boolean that controls whether or not the url_params will be escaped.

Note: There is another optional parameter: extra_headers, which is the second parameter, but at this point it does not seem to be used at all in the source code.

The simplest case is to ignore the optional parameters and simply pass in the edit uri of the calendar that you wish to delete. You can get the edit uri of a calendar by calling the GetEditLink function of the CalendarListEntry instance that represents the calendar that you are going to delete. An example of a function that will delete a calendar can be seen in Listing 4. This function takes a CalendarService instance and a CalendarListEntry instance as parameters.

Listing 4

def delete_calendar(calendar_service, calendar):
    e_link = calendar.GetEditLink()
    if (e_link is not None):
        try:
            calendar_service.DeleteCalendarEntry(e_link.href)
        except gdata.service.RequestError, e:
            print "Error deleting calendar: %s" % (e[0]["reason"])

Listing Events

Now let’s take a look at events. Events are items that are added to a specific calendar. If you want to remind yourself to pay your bills at the end of the month you might add that to your banking calendar. That “note” on your calendar is an event.

You can get a list of events for the primary calendar using the CalendarService classes GetCalendarEventFeed function. Since you may be working with more than one calendar, it’s probably more useful to be able to list the events for a specific calendar. You can do this in one of two ways:

You can pass in the optional uri parameter to the GetCalendarEventFeed function. From testing I found that a Calendar’s “alternate” link works.
You can use the CalendarService classes CalendarQuery function to query for a specific calendars event feed.

If you want to use the first option you can use the CalendarListEntry classes GetAlternateLink member function to get the uri, and then pass it to GetCalendarEventFeed:

a_link = calendar.GetAlternateLink()
#Make sure the link is valid
if (a_link is not None):
    event_feed = calendar_service.GetCalendarEventFeed(a_link.href)

If you use the CalendarQuery method, you need to get the calendar username (or id) of the calendar whose events you want to query. It seems strange to me that there appears to be no way to get a calendar’s username besides parsing one of the calendar’s links. The username of a calendar can be found in the alternate link after “feeds” and before the visibility and projection:

http://www.google.com/calendar/feeds/< <username>>/private/full

Note: You can use the username “default” to query the default calendar.

For the sake of simplicity I will use the alternate link method for my examples. A full example that prints out calendar data and a calendar’s events can be found in Listing 5. The method that prints out the event data is called print_event_feed. It is called near the end of the list_own_calendars method:

# Now Print out the events
print "Events:"
a_link = cal.GetAlternateLink()
if (a_link is not None):
    event_feed = calendar_service.GetCalendarEventFeed(a_link.href)
    print_event_feed(event_feed)

Listing 5

#! /usr/bin/env python

import gdata.calendar.service
import getpass

def print_event_feed(event_feed):

    for index, event in enumerate(event_feed.entry):
        print "\t%d) %s\r\n\tContent: %s" % (
                index, event.title.text, event.content.text)
        print "\t\tWho:"
        for person in event.who:
            print "\t\t\tName: %s\n\t\t\temail: %s" % (person.name
                , person.email)
        print "\t\tAuthors:"
        for author in event.author:
            print "\t\t\t%s" % (author.name.text)
        print "\t\tWhen:"
        for e_index, e_time in enumerate(event.when):
            print "\t\t\t%d) Start time: %s\n\t\t\tEnd time: %s" % (
                e_index
                , e_time.start_time
                , e_time.end_time)

def list_own_calendars(calendar_service):

    try:
        #Get the CalendarListFeed
        all_calendars_feed = calendar_service.GetOwnCalendarsFeed()
    except Exception, e:
        print "Error getting all calendar feed: %s" % (e)
        return
    #Print the feed's title
    print all_calendars_feed.title.text
    #Now loop through all of the CalendarListEntry items.
    for (index, cal) in enumerate(all_calendars_feed.entry):
        #Print out the title and the summary if there is one
        if (cal.summary is not None):
            print "%d) %s - Summary: %s" % (
                index, cal.title.text, cal.summary.text)
        else:
            print "%d) %s" % (index, cal.title.text)
        #Print out the authors
        print "\tAuthors:"
        for author in cal.author:
            print "\t\t%s" % (author.name.text)
        #Print out other information
        print "\tPublished: %s \n\tUpdated: %s \n\ttimezone: %s" % (
            cal.published.text, cal.updated.text, cal.timezone.value)
        print "\tColour: %s \n\tHidden: %s \n\tSelected: %s" % (
            cal.color.value, cal.hidden.value, cal.selected.value)
        print "\tAccess Level: %s" % (cal.access_level.value)
        # Now Print out the events
        print "\tEvents:"
        a_link = cal.GetAlternateLink()
        if (a_link is not None):
            event_feed = calendar_service.GetCalendarEventFeed(a_link.href)
            print_event_feed(event_feed)

def main():
    username = raw_input("Enter your username: ")
    password = getpass.getpass("Enter your password: ")

    calendar_service = gdata.calendar.service.CalendarService(username
        , password
        , "PythonMagazine-Calendar_Example-1.0")
    try:
        calendar_service.ProgrammaticLogin()
    except gdata.service.BadAuthentication, e:
        print "Authentication error logging in: %s" % e
        return
    except Exception, e:
        print "Error Logging in: %s" % e
        return

    list_own_calendars(calendar_service)

if __name__ == "__main__":
    main()

An example of the output produced by Listing 5 can be seen in Listing 6. Don’t mind the formatting – it’s not pretty. It’s merely meant to give you an example of some of the data that you can mine from an event feed.

Listing 6

Mark Mruss's Calendar List
0) Mark Mruss - Summary: Main Calendar
	Authors:
		Mark Mruss
	Published: 2008-02-12T02:30:18.731Z
	Updated: 2008-02-08T04:03:10.000Z
	timezone: America/Toronto
	Colour: #5229A3
	Hidden: false
	Selected: true
	Access Level: owner
	Events:
	0) Dinner at the Drake
	Content: Dinner
		Who:
			Name: Mark Mruss
			email: mark.mruss@gmail.com
		Authors:
			Mark Mruss
		When:
			0) Start time: 2008-02-15T21:00:00.000-05:00
			End time: 2008-02-15T22:30:00.000-05:00
	1) Cezanne's Closet
	Content: Cezanne's Closet
		Who:
			Name: Mark Mruss
			email: mark.mruss@gmail.com
		Authors:
			Mark Mruss
		When:
			0) Start time: 2008-02-09
			End time: 2008-02-11

Adding an Event

Adding a new event to a calendar is very similar to creating a new calendar. You need to create a new CalendarEventEntry instance. You then populate it with options, and pass it to the CalendarService classes InsertEvent function, along with the alternate link of the calendar to which you would like to add the event.

If you take a look at Listing 7 you can see a simple example of how to add an all day event. If you want to specify an event that lasts for less then a day, or include a specific start and end time, you need to use the “RFC 3339 timestamp” format for your start_time and end_time values. Also notice that just like adding a Calendar, the newly created event is returned by the InsertEvent function.

Listing 7

a_link = calendar.GetAlternateLink()

if (a_link is not None):
    new_event = gdata.calendar.CalendarEventEntry()

    new_event.title = gdata.atom.Title(text="New Article")

    new_event.content = gdata.atom.Content(text="Write Article")

    new_event.when.append(gdata.calendar.When(

        start_time="2008-02-20"

        , end_time="2008-02-21"))

    try:

        created_event = calendar_service.InsertEvent(new_event

            , a_link.href)

    except gdata.service.RequestError, e:

        print "Error adding event: %s" % (e[0]["re

 			
	
	
		Mitch Chapman: PyCon 2010 Poster Session
	I finally presented something at a Python conference.  Many thanks to Vern Ceder for organizing the PyCon 2010 poster session!

Here’s a copy of the poster, which describes a way to run QUnit-based JavaScript unit tests using Python’s unittest facilities:

PyCon Poster.pdf

The code described by the poster is available from bitbucket.org, under the BSD license.

 			
	
	
		The human face of Perl projects
	Some Perl projects show you the faces of their community:

you can see the people behind WebGUI

or profiles of the current Bugzilla developers.

Foswiki also has a page for currently active developers

or you can see the Bricolage team or

the Dreamwidt…

 			
	
	
		Jack Diederich: High Trust, High Responsibility Communities
	[This is a blog version of a lightning talk I didn't give at PyCon]
People Hacking is as old as time, and every long lasting organization engages in it.  One example of hacking people (and the title of this post) is to tell people you trust them.  This “trust cue” invites reciprocity — people behave well because you’ve signaled that you expect them to.  Google’s “don’t be evil” motto works this way; the company has promised the outside world that it should be trusted so internal employees feel pressure to live up to the promise.
You can do the opposite and tell people that you don’t trust them ["negative trust cues"] and people being people their behavior is predictably cautious and non-cooperative.  I once worked for an ex-DEA prosecutor and to put it mildly the employer-employee relations weren’t stellar.
The Python community has many positive trust cues going for it.  The coding end is open source so everyone is a volunteer by definition, and volunteering for anything is a positive trust cue.  Likewise PyCon introduces people face-to-face and having met someone as a real-live-person (even once!) invites an obligation to be more forgiving in the sterile world of email and bug trackers (was he being a dick or just having a bad day?  I’ve met him, must have been a bad day).  I got my commit bit after never having one of my many patches to CPython accepted*, but curiously not long after I met the guy who had rejected them (Raymond Hettinger, who was working the registration desk when I checked in). 
I am/was a member of a large number of long standing volunteer orgs (most are fraternal**) and have seen many variations on trust cues: the big ones are the Boy Scouts, Habitat for Humanity, the Kappa Alpha Society, and the Free Masons.  None are as old or put as high an emphasis on trust cues as the Masons – everything is setup to do charity and avoid conflict.  It is prohibited to drink before/during meetings***, discuss politics, or discuss religion.  The taboos are so strong that there aren’t legal punishments for violating them – you are trusted not to violate them so (by reciprocity) no one does.  Very few of the members are half as smart as the average lunch table at PyCon, but it is impossible not to like the members of my lodge because every time I see them they are doing charity: giving blood, contributing to food banks, etc.  The trust cues are through the roof (one side effect of that is a very high attrition rate — people quit because the activities are quite stolid and boring).
So the Python community in general and PyCon in particular inculcate positive trust cues (when was the last time your saw a post titled “PyCon sucked?” never).  That said, I ran away from the python-d7y list because it was so loaded with negative trust cues.  See Anna Ravenscroft’s PyCon2010 talk for very healthy ways to promote positive trust cues and eschew negative ones.  The topic mandated the  opposite of the “no politics, no religion” of my other, more boring, org and the results were predictable.  The d7y list was mostly harmless but 5% of the talk was by people that had no skin in the game (and hence no expectation of kindness or reciprocity).  To be vague and ablative the demands of the 5% was a laundry list of negative trust cues: censorship, name-and-shame, and other minor atrocities.  As a result most discussion of the list actually took place off the list because plain discussion on the list was impossible (par exemplar was one of the moderators emailing me off-book “I wish I could +1 but I can’t.  We need a new list.”).
In closing the Python community exploits human nature in the form of trust == responsibility pretty well.  I am happy to be exploited thusly because I’m human and it tickles my humanity.  Assign bugs to me and – if I’ve met you or have met someone who has met you – I’ll be quick to close it with a more delicate comment than otherwise.
* My patches weren’t bad or wrong, they just weren’t enough of a speed boon considering the maintenance overhead to be added to the stdlib.
** I have three brothers and no sisters so that’s where I’m at ease.  Though until recently (last century) “fraternal” meant “brotherhood of man” instead of the narrower “male only” so there still exist some mixed sex and female-only orgs that have “fraternity” in the name.
*** The “no booze” is very unusual for voluntary associations in the US.  Most clubs have a members only bar that is the major fundraiser for the org. For instance, when I lived in Pennsylvania I was a social member of the King of Prussia Volunteer Fire Company – I was one of the thousand members who didn’t fight fires but paid $30/year for my membership card (you needed to be nominated by an actual fire-fighter to join but like most orgs this was a technicality — they were mainly interested in keeping out people who couldn’t find one related person to say something nice about them)


 			
	
	
		Simon Willison: PiCloud
	
PiCloud. An interesting twist on cloud computing for Python. “import cloud; cloud.call(my_function, arguments)” serialises my_function and its arguments, pushes it up to one of their EC2 servers and hands you back a job ID which you can poll (or block on) for a response. They suggest using it for long running tasks such as web crawling or image processing.


 			
	
	
		New PHP Releases: 5.2.13 and 5.3.2 RC3 – Christopher Jones
	         PHP 5.2.13 has been released. The announcement is here and software is on the downloads page. At the same time, PHP 5.3.2 RC3 is available for testing, see http://news.php.net/php.qa/65489. There are only a few days before the final PHP…

        

 			
	
	
		Wayne Witzel: Boost Python, Threads, and Releasing the GIL
	Note: Wordpress likes to convert my  > inside [ code ] blocks to to &amp;gt; and &amp;lt; so sorry about the formatting
After Beazley’s talk at PyCon "Understanding the Python GIL" I released I had never done any work that released the GIL, spawned threads, did some work, and then restored the GIL. So I wanted to see if I could so something like that with Boost::Python and Boost::Thread and the type of performance I’d get from it with an empty while loop as the baseline. So I hacked up some quick and dirty C++ code and quick bit of runable Python to test out the resulting module and away I went. Below are the code snippets, links to bitbucket, and the results of the Python runable.
#include <iostream>
#include <vector>
#include <boost/shared_ptr.hpp>
#include <boost/thread.hpp>
#include <boost/python.hpp>

class ScopedGILRelease {
public:
	inline ScopedGILRelease() { m_thread_state = PyEval_SaveThread(); }
	inline ~ScopedGILRelease() { PyEval_RestoreThread(m_thread_state); m_thread_state = NULL; }
private:
	PyThreadState* m_thread_state;
};

void loop(long count)
{
	while (count != 0) {
		count -= 1;
	}
	return;
}

void nogil(int threads, long count)
{
	if (threads <= 0 || count <= 0)
		return;

	ScopedGILRelease release_gil = ScopedGILRelease();
	long thread_count = (long)ceil(count / threads);

	std::vector<boost::shared_ptr<boost::thread> > v_threads;
	for (int i=0; i != threads; i++) {
		boost::shared_ptr<boost::thread> m_thread = boost::shared_ptr<boost::thread>(new boost::thread(boost::bind(loop,thread_count)));
		v_threads.push_back(m_thread);
	}

	for (int i=0; i != v_threads.size(); i++)
		v_threads[i]->join();

	return;
}

BOOST_PYTHON_MODULE(nogil)
{
	using namespace boost::python;
	def("nogil", nogil);
}

Then I used the following Python script to run some quick tests.
import time
import nogil

def timer(func):
	def wrapper(*arg):
		t1 = time.time()
		func(*arg)
		t2 = time.time()
		print "%s took %0.3f ms" % (func.func_name, (t2-t1)*1000.0)
	return wrapper

@timer
def loopone():
	count = 5000000
	while count != 0:
		count -= 1

@timer
def looptwo():
	count = 5000000
	nogil.nogil(1,count)

@timer
def loopthree():
	count = 5000000
	nogil.nogil(2,count)

@timer
def loopfour():
	count = 5000000
	nogil.nogil(4,count)

@timer
def loopfive():
	count = 5000000
	nogil.nogil(6,count)

def main():
	loopone()
	looptwo()
	loopthree()
	loopfour()
	loopfive()

if __name__ == '__main__':
	main()

The results I got were quite interesting and very consistent on my MacBook Pro. I ran the script about 1,000 times and got roughly the same results every time.
loopone took 364.159 ms (pure python)
looptwo took 15.295 ms (c++, no GIL, single thread)
loopthree took 7.763 ms (c++, no GIL, two threads)
loopfour took 8.119 ms (c++, no GIL, four threads)
loopfive took 11.102 ms (c++, no GIL, six threads)

Anyway, that’s all really. Nothing profound here, no super insightful ending. Just hey look and stuff is faster and I might use this. All the code for this is available in my bitbucket repo. http://bitbucket.org/wwitzel3/code/src/tip/nogil/
You will require Boost Library including Boost Python and Boost Thread as well as Python libraries and includes to build this. For boost, bjam –with-python –with-thread variant=release toolset=gcc is all I did on my Mac. Then I added the resulting lib’s as Framework dependencies in Xcode along with the Python.framework.

 			
	
	
		PyCharm: PyCharm 1.0 Roadmap Uncovered
	No doubt you are interested what will be in PyCharm 1.0. Many are interested. So, here is our current vision.
The focus of PyCharm development leading to the 1.0 release will be less on major new features and more on ensuring that the existing functionality works great in different kinds of projects. However, there are a number of features that will definitely be added before the 1.0 release. They are:

Integrated REPL with syntax highlighting and code completion
Improved type system (tracking of method return types, tracking of collection content types)
IronPython support
Google App Engine support
Highlighting of unused import statements and Optimize Imports
Whole-project completion (completion of not imported classes and functions, with automatic addition of import statements)
More Python code inspections (e.g. pylint/pyflakes)

There are also some things we are considering, such as

Bundling Mercurial VCS support. It is now available as a  third-party plugin
Support for additional template languages (Genshi, Mako)
Support for completion and resolve of .NET types in IronPython projects
Syntax highlighting, code completion and test runner for doctests

If you think some of these are really important for your work, add requests to PyCharm issue tracker. Of find an existing one and vote for it.
This is our current vision but it may change as time goes. You can influence changes as well — try PyCharm Early Access version, share your thoughts on PyCharm forum and vote for issues in PyCharm bug tracker.
The PyCharm roadmap is available at http://confluence.jetbrains.net/display/PYH/PyCharm+1.0+Roadmap and you can watch this page for changes.
Develop with pleasure!

JetBrains Team


 			
	
	
		« go back — keep looking »

Recently Written

Blogroll

Development Blog
Documentation
Planet Perl
Planet PHP - People blogging about PHP
Planet Python - Planet Python – http://planet.python.org/
Plugins
Suggest Ideas
Support Forum
Themes
WordPress Planet

Warning: include(/home/remarkwit/enterpriselamp.org/wp-content/themes/Enterprise_LAMP/r_sidebar.php) [function.include]: failed to open stream: No such file or directory in /home/remarkwit/enterpriselamp.org/wp-content/themes/Enterprise_LAMP/archive.php on line 23

Warning: include() [function.include]: Failed opening '/home/remarkwit/enterpriselamp.org/wp-content/themes/Enterprise_LAMP/r_sidebar.php' for inclusion (include_path='.:/usr/local/lib/php:/usr/local/php5/lib/pear') in /home/remarkwit/enterpriselamp.org/wp-content/themes/Enterprise_LAMP/archive.php on line 23