I wanted to post an entry documenting where Pennyworth is now and some of the next steps I am taking with respect to its development and my own research.

Pennyworth

The C4.5 decision tree implemented in the last release has proven itself to be a much better “brain” than the previous machine learner. I’ve been running Pennyworth fairly constantly for the past week and it’s picking up the context as it should and I find myself training it only on the rare occasion to correct the model when it’s seeing something new. I’m quite pleased with how this has worked out.

That said, I’m contemplating at the moment removing the “interrupt me every X seconds” training mode. This is a fairly standard practice to use in evaluating machine learners (collect a representative sample of features to label and evaluate). However, in the context of this particular machine learner and the user’s interaction with it, I think that this method is unnecessarily burdensome and doesn’t really help the learner. In any case, if a time-based training mode remains in Pennyworth, I’ll be advocating the “train it when it’s wrong” approach in the documentation (that I need to write).

Growl

Pennyworth is designed to work with the Growl notification system, and I’ve been refining both programs so that they can operate in a more harmonious manner. Pennyworth now has a “Suppress Growl notifications” preference in the development build. This is intended for those of you who become annoyed by all of the notifications Pennyworth emits on context change.

I’ve also contributed a patch to the Growl developers that adds support for choosing an appropriate notification style or sound effect through AppleScript. Combined with Pennyworth’s AppleScript-based action-taking component (I need a better name for this), Pennyworth can control Growl and its notifications. This leads us to…

Context-aware notification systems

Last summer when I began working on a context-sensing platform for MacOS X, my goal was to build a context-aware notification system that changed its behavior in order to present the best style of notification, given the user’s state. The fundamental premise underlying this idea is that not all notification styles are appropriate for all situations. For example, a low-interruption notification style is appropriate for “unrelated” information when the user is heavily focused on a specific task. (See “Attuning Notification Design to User Goals and Attention Costs” by McCrickard & Chewar for more details.)

With the modifications to Growl, I can now use Pennyworth+Growl to begin exploring this space. I implemented an example of this over the weekend that worked quite well in my own tests. My specific problem was that I don’t like getting frequent notifications when I am gaming. Due to the MacOS X display system, overlaying a notification on an OpenGL canvas flickers the display and reduces the frames drawn per second (FPS). So, I wanted to change the notification style of two applications when I’m playing Warcraft: Pennyworth and Cidney.

I wanted to eliminate Pennyworth notifications because they are a frequent offender when I temporarily switch out of WoW to check mail or the web while I’m playing. When I do one or the other, I get this rapid sequence of “Activity: E-Mail”, “Activity: Web Browsing”, “Activity: World of Warcraft” messages when I’m multitasking this way. They don’t do me any good and just end up annoying me.

Likewise, when I’m playing WoW, I’m often not interruptible because I’m playing the Battlegrounds. So, when I receive an incoming phone call and notifications from Cidney (a desktop-based virtual Caller ID program), I’d rather listen for the custom ringtones that I’ve assigned to important people rather than be interrupted with a visual notification.

Given these constraints, I put together a script that implements the following pseudocode to deal with this contextual situation:

if Activity is “Playing World of Warcraft”:
tell Growl:
Use the Null notification style for World of Warcraft
Use the Null notification style for Cidney

else if World of Warcraft is not running:
tell Growl:
Use the Smoke notification style for World of Warcraft
Use the Smoke notification style for Cidney

Basically, this script tells Growl to use an invisible notification style the first time it detects me playing World of Warcraft. When I switch activities, it checks if WoW is still running before re-enabling the notifications for the two applications. Since I will probably be coming back to Warcraft if it’s still running, it doesn’t re-enable the notifications until I quit the program.

In an ideal world, I’d be able to refine this script a bit more by having it detect when I’m within an attention/focus-heavy context within World of Warcraft. When I’m playing the battlegrounds, interruptions are not okay. However, when I’m scanning the auction house or traveling from one zone to another, interruptions are acceptable. Unfortunately, until I am able to poke within the WoW process to determine what I’m doing in-game, this fine-grained, within-application context sensing probably isn’t possible. (Though, as I type this, a birdie in my head is whispering “make an add-on”.)

The example above is just one instance of how we can use context-aware notifications within our computing environments. “World of Warcraft” may be substituted with “Writing Novel” or other attention-heavy tasks. As long as users are okay writing AppleScript to define the relationships between context and notifications, the current system will be adequate and useful. I hope that it’s clear how a user could add other specialized configurations by simply adding conditions to the if/else statement above (e.g. “else if Activity is ‘Watching Film’ then turn off all notifications”).

I’ll be packaging up the various parts of this system with the next release of Pennyworth for those interested in defining their own context-aware notification configurations and replicating the above.

In a future post, I may think about the potential of adding a machine learner that determines the right combinations of styles and contexts for the user. The script above could easily be intuited by a decision tree, given access to information such as the current notification styles. I also haven’t touched on the idea of taking action on context changes (“play some opera when I’m reading, play some trashy Euro-techno when I’m programing”). I’ll be bringing this idea up in a later post when my thoughts become a bit more organized on the topic of the types of context-aware applications that may be built in addition to notification and home automation systems.