Wednesday, October 18, 2006

Tech Walkthru: D-BUS, HAL, Notify and Avahi LAN Services

As a result of researching topics for PyCon keynotes, I've been digging a bit into the One Laptop Per Child project. Being based on Python, essentially seeking to provide a 'Python operating environment', it looked quite interesting. I noticed that they plan to make heavy use of localized wireless grids for children to interact, essentially multicast service-locating DNS the software for which is called Avahi. The Avahi software implements the zeroconf or Bonjour/Rendevous functionality and sits on top of the D-BUS, a Unix desktop bus for interprocess communications within the same machine. There is a Python module for interfacing to D-BUS, both as a client and a server.

And so this past Saturday at the DFW Pythoneers hands-on session, I pulled together some demo code and we walked through this technology, D-BUS, HAL, Avahi, notification-daemon, in Python.

About D-BUS

D-BUS is both a library and a daemon. The library implements point-to-point message-based communication between processes on the same machine. The daemon adds a service registration ability and can start dormant services upon the arrival of requests. Now it turns out that you can start as many daemons and therefore buses as you wish. Usually there are two: a system bus started at boot time for exchanging messages about hardware availability and such, and a session bus started at user login time for desktop objects to talk among themselves.

Python programs can participate on either of these buses, with quite seamless handling of datatypes and asynchronous I/O using glib (wishing for a Twisted reactor myself). Buses aren't of much use though unless there is something to which to talk which leads us to HAL.

About HAL

One of the services that resides on the system bus is HAL, the Hardware Abstraction Layer. Using Python, it appears as a Device Manager mapping (keys, values, items) of device names to device instances, and each device instance has a mapping of properties (name -> value) and capabilities (set of well-known strings). Both the manager and device objects support callbacks upon the addition/removal/change of devices and properties.

In my demo code, I show how to monitor and manipulate the brightness of my laptop LCD display, as well as to detect the insertion of a USB flash drive. But if you're going to be responding to hardware events, you often want to notify and perhaps interact with the user, which means we need something like the notification daemon.

About the Notification Daemon

Another service that resides on the bus, although on the session but not the system bus, is the notification daemon. Not a catchy name it comes associated with the 'libnotify' component. Using it your Python-based services can quietly display one of those non-focus-stealing balloons in the lower-right corner of the screen. The balloons can have lifetimes or require user acknowledgement. Also supported are icons/graphics and sound although I didn't play with those in my demo code.

About Avahi

But getting back to my original goal, the Avahi multicast DNS service. Our group began to run out of time but Avahi is also accessible on the system bus. This means a Python program could stay aware of what machines are in the neighborhood and, combined with the notification daemon alert you.

I hope to further investigate tying these services together, along with the Festival/Cepstral speech synthesis interface I'm developing for Python and it would be really cool if we could have something working in time for PyCon. Something like the groupware programs the Mac users have to share live notes on talks but this time for us Linux people as well. ;-)
The source for the demo, rough code sketched out in an evening, is available in the DFW Pythoneers subversion repository.


Group Discussion: About Dynamic Languages and Python

This month at the DFW Unix User Group our main presentation was given by Dave Thomas about Ruby and Rails. Seeing as I run the Linux SIG afterward and am often short on topic ideas, I figured we could make an evening of dynamic languages, discussing what is a dynamic language, why you would want one and how Python fits into the picture. It was a bit of a walk-through of loosely related programming concepts, attempting to explain various concepts that are commonly tossed about but loosely defined.

The group of about 15-20 people was quite interactive as we discussed (and debated) the meanings of data typing, covering the dimension of dynamic versus static (whether variables can change their type, once set), and the independent dimension of weak versus strong (whether type mismatches are implicitly fixed).

strongPython, RubyJava, FORTRAN
weakPerl, PHPCOBOL, C

Also introduced was the concept of duck typing, yet another aspect of dynamic languages, and brief overviews of continuations and closures, two very different ideas that people often confuse. We also went through the meanings of introspection and reflection, finding it hard to separate the two. We covered the distinction between first-class, second-class and other-class language concepts, showing how functions in C/C++ and classes in C++ are second-class constructs but first-class in Python.

This led to definitions of paradigms (algorithms for computers) versus methodologies (algorithms for people), also frequently confused in conversation. We covered the common methodologies of flowcharting, structured and top-down programming, object-oriented and extreme programming. And the well-known procedural, functional, object-oriented and declarative paradigms. The audience noticed that object-oriented was on both lists, being both a paradigm and a methodology. And we wondered why we call it 'functional' programming instead of function-oriented' programming, similar to OO. It was also pointed out that 'object-oriented' and 'dynamically typed' are independent traits of a language and that you can have one without the other.

At this point we went a bit into the history of object-oriented programming, beginning with Simula-67, which had only non-instanced data and lacked inheritance until those ideas where introduced by Smalltalk. And it was noted that there are (at least) three object-oriented flavors; the usual split between C++ and Smalltalk but also yet another approach with Oberon.

As a group we tackled the definition of a scripting language, concluding it was the use to which the language was put more than anything about the syntax itself.

Moving into Python, we began defining the criterial for choosing a dynamic language; ease of expression, ease of handing maintenance off to others, richness of libraries, quality of documentation, license, and the size, friendliness and skill level of the supporting community. Python excels in all these areas and the history and feature set of of Python was introduced.

It was also pointed out that Python is not an either-or language, in that you can combine it with various environments such as Java/JVM, C#/CLR and C++/CPython.

The talk was not our usual fare of Linux applications but seemed to be well received. Our membership consists of programmers and sysadmins. This talk was also my first use of reStructured text combined with the S5 slide format as I usually produce slides using, exported to PDF. The slideshow is available for online viewing as well as the raw reStructured text making it up.


Thursday, October 12, 2006

A Call for Members of Panels on Web Technologies

Talks at previous PyCons have tended to be of the lecture format. I'd like to see if we can get some panel discussions going and am looking for participants.

At PyCon 2005 in D.C. Michelle Levesque gave a wonderful talk, PyWebOff, contrasting a few of the web frameworks. However, it is a lot of work to research many of them, so how about instead we just invite some experts for the various frameworks to debate them instead? And answer tough questions from the audience?

For updating, there are links to the following rough outlines on the PyCon TalkIdeas wiki page.

We need some moderators to work up more questions, and to coordinate with the panelists. Hey, it may be easier than preparing your own presentation and slides.

I'd like to see three talk panels:

"Panel - Web Frameworks"

Questions for the Panel:
  • When did the framework come into existence and what problem was it created to solve?
  • Why did the frameworks that existed at the time not meet your particular needs?
  • What are its strengths and maturity level? What is it ideal to use for?
  • What are its weaknesses? What would you NOT use it for?
  • Which frameworks let me plug in my choice of template language?
"Panel - Web Templating Languages"

Template Languages:
  • Twisted STAN
  • Cheetah
  • Myghty
  • Django's Template Language
  • TurboGears Kid
  • Quixote Python Template Language (PTL)
Questions for the Panel:
  • Python code in your HTML, or HTML in your code?
  • Which can and cannot be handed off to a graphics designer?
  • Where do they stand in performance? in Caching?
  • Which can work with less than a page i.e. Zope viewlets, widgets?
(initially show the audience a representative sample of each)

"Panel - Object Relational Mappers (ORMs)"

Mappers Covered:
  • SQLObject
  • SQLAlchemy
  • Django ORM
Questions for the Panel:
  • Natural or artificial keys?
  • Support for existing database schemas or must I do it your way?
  • How is the performance?
  • How smart is it in handling complex schema?
  • Any support for access control security at the object level?
  • What flavor and degree of transaction support is offered? Per thread, per context, autocommit?
  • Can a single app easily access multiple databases at the same time (not just different tables within the same db)?
  • Can a single app easily access multiple databases at the same time (not just different tables within the same db)?

A Call for a Presenter on Python Eggs

Eggs are important to Python and will become more so over the next few years, if understood and embraced by the community. They are a key competitive feature - I've been asked so many times what is the Python equivalent to CPAN and finally we're developing an answer.

At PyCon 2007 we need a solid set of talks about eggs, broken down into just using them, how to build and distribute them, and then an advanced talk on some of the more exotic aspects of eggs. At PyCon 2006, essentially the 3rd talk I mention above was given to a standing-room-only audience, and then repeated against an hour later as people in the hall could not get in. But missing were the first two talks - many people were not ready for advanced material but sought an introduction.

Who in our community is building eggs on a frequent basis, and is good at explaining how they work?

Below are three talk outlines I've sketched out, along with the slides from a talk I tossed together for the Dallas Pythoneers group. They need polish and I'm donating them to the cause.

Will someone please step forward and address this critical need?

For revision, the following outlines appear on the PyCon TalkIdeas wiki page.

"Python Eggs - Using and Installing Them"
  • What is so cool about eggs? Wow me.
  • What problem were they intended to solve?
  • What is their relation to distutils?
  • So how does distutils work then?
  • How do they compare to CPAN and RPM approaches?
  • Sound good, how do I enable eggs on my system?
  • How do I accomplish common operations, such as installing/uninstalling an egg, list what eggs are installed at the moment, switch between the versions of an egg and test install an egg to see if it would work?
  • I don't trust eggs - how can I partially install one, examine it and then finish the installation?
  • How do I find out what eggs exist, and their names?
  • What about py2exe and droplets? Competing approaches?
  • Show me a walkthru of the cheeseshop - how to find a package.
  • How do I figure out the author of an egg I'm having trouble with?
"Python Eggs - Creating and Distributing Them"
  • Where does distutils fit in?
  • What do I need to add to to package my software as an egg?
  • How does it work re path configuration files?
  • What are the issues with zipped eggs; how can my code access my data?
  • How do I produce cross-platform eggs?
  • How do I produce platform-specific eggs?
  • How do I get my eggs registered on the cheeseshop?
  • How do I upload my eggs, source and binaries?
  • How can I volunteer as a packager for non-egg authors?
  • Some eggs are broken in the cheeseshop - how can I help?
(Keep this talk simple and focused on just getting your vanilla egg packaged and uploaded. Defer the advanced egg issues to the third talk).

"Python Eggs - Advanced Packaging"
  • What are entry points?
  • How does dynamic discovery/plugins work?
  • I hear Trac uses them - how?
  • What are extras? They sound nifty!
  • What is a namespace package? When and how do I use them?
  • How can I have multiple distribution versions?
  • I hear eggs can detect files under CVS/subversion control - how does that work?
  • Also eggs have support for units tests - how?
  • Where do the egg experts hang out?
An Old Talk of mine about Eggs (from which to borrow slides/ideas):

Seeking Publicity Coordinator for PyCon 2007

The PyCon organizers are looking for someone to volunteer as publicity coordinator for PyCon 2007. This year we have started planning sufficiently in advance that we'd like to get some press coverage in print magazines.

What is involved, you ask?
  1. Brainstorm/collect a list of candidate publications for press releases.
  2. Post the lead time and submission requirements for each to the wiki.
  3. Develop or work with others to develop a (textual) press release layout.
  4. At the appropriate time (e.g. when certain details are settled such as keynote speakers), actually submit the press release.
There is a wiki page at the following location on which to collect ideas on where to submit a press release.

That's it! It really shouldn't take a lot of time; a bit of surfing, perhaps a few email queries.

Wednesday, October 11, 2006

A Word from the Python Advocacy Coordinator

Welcome to my first blog, where I hope to keep the Python community informed about advocacy efforts, my own and those of others. Entrusted by the Python Software Foundation for the next few months on a full-time basis to coordinate the advocacy efforts, a few of the resources I plan to produce are:
  1. two portals (outsiders + advocacy helpers), populated with some solid writing,
  2. get some magazine articles written and in the pipeline,
  3. engage various audiences for Python (corporate, college, K-12, hobbyists) and begin supplying them with what they need to make the choice of Python.
For more details you can read my original proposal to the PSF. To get involved (and I need a lot of help), join our new advocacy mailing list.

More to come...