A Pythonic Way to Measure and Improve Your Programming Skills?

Those who know me know I am no fan of the programmer certification industry and I agree that your typical certificate arrangement, e.g. to show HR to get that box checked in your candidate file, so a company can skip responsibility in vetting a candidate, is un-Pythonic. However, I do think we need a good answer to the engineering manager or team leader who asks, "how do I know this person knows Python?", and to the candidate who thinks, "I believe I'm good at Python but am I as good as I think?".

The first case primarily cares about the reputation of the testing facility within the industry, so that if they choose wrong, they are absolved of blame. The latter two are often honestly looking for some guidance on bringing good people in-house or to step-up their programming skills.

As Python use spreads, it becomes more important for the community to provide benchmarks for measuring skill levels, to avoid Python being relegated to the, respectfully, lightweight programmer category e.g. PHP or perhaps Ruby on Rails (not Ruby itself).

Prior to PyCon I'd been thinking about some kind of campaign, service or documents, that I call "So you think you know Python...". My initial idea was for use by Python programmers, who are honest with themselves, to have a way to measure their knowledge. We all run into or are programmers who think they are hot-stuff, especially those of us who work solo as many in our community do, but when put side-by-side against others, fall a bit short. PyCon can have that effect, with so many smart people in a small area. ;-) But the objective isn't to denegrate these programmers but to provide a measuring stick, along with an educational roadmap, so they can make wise choices.

The rapid growth in use of Python is going to lead to a shortage in Python programmers, unless we manage the supply side as well. The manager wanting to find and hire good Python programmers is not an adversary seeking to use certificates to commoditize our community, but an ally in helping us get industry respect and career opportunities using our favorite language. We need to behind what he asks for to see what he needs.

I've been carefully watching Crunchy, about which a talk was given at PyCon, for writing tutorials that, with its "doctests" feature, could be used to propose tests that pass and require a candidate to write an acceptable program. This week I was prototyping a "rst2crunchy" tool, that let's you write Crunchy scripts in reStructured Text, rather than HTML. Crunchy needs more features, such as a user login facility and measurement facilities to store test submissions for review by experienced Python programmers after they pass. From what I hear on the edu-sig list, André Roberge and Johannes Woolard are tackling these but could use more help. Johannes in particular was a student for Google's Summer of Code 2006 and I believe is looking for sponsorship for 2007.

But perhaps Crunchy is not the ready answer in this case, so I'm looking for suggestions.

With regard to how a manager can know a person is good at Python, besides looking at their portfolio of work, we could stake out conceptual areas of the language or libraries about which a programmer should be conversant. This might be a document, with example questions but carefully focused on what lies behind those questions, so an interviewer can come up with his own questions and avoid rote memorization. This conceptual map could also eventually form the basis of advanced-placement in schools that goes beyond those terrible Visual BASIC test questions. And for the Python programmer who wonders if his knowledge is broad enough and wants to improve him or herself, the conceptual map could have links/recommendations to training materials on specific topics.

Join me on the advocacy list and let's discuss this further.

Jeff Rush
Python Advocacy Coordinator