Change of Theme

I like the basic design of TwentyTen (the WordPress default), but I didn’t like the stock header choices. After some time thinking about it, I decided that there should be something dynamic up there. Dynamic is more invasive than just an image swap, and thus some theme customization is required.

I didn’t want to deal with a completely custom theme, nor with being able to update WordPress if I hacked on TwentyTen directly. A bit of Google querying turned up the idea of Child Themes. WordPress allows you to make a new theme that starts by inheriting from an existing one (see http://codex.wordpress.org/Child_Themes).

So, I made the directory for my new theme, set it to inherit from TwentyTen, then started digging around for how to control the header. The header is pretty tied to just supplying an image referred to by the database (with all sorts of logic for changing the image based on the post/page being viewed), with no hooks for writing your own header HTML code, so I just copied the entire header.php, and whacked out the image.

Now, the question is what to use for the new banner. I ended up deciding to use Processing.js for the banner. This has the advantage of being fairly easy to program. The downside is that it uses the canvas element, and so some web browsers won’t be supported now, and it probably won’t run as fast as limiting myself to div/img sprites would. However, on the upside, I don’t have to reconsider the choice every time or limit myself only to simple sprites.

My goal is to periodically change the banner now. The first program running up there is taken from the header of the Processing.js web site. I changed the canvas size to match WordPress header sizes, and I made some minor graphical changes. The original source code is here. My modified source code is here. At some point, I’m going to move the header above the Processing.js app into the Processing.js app to save some space, but I haven’t gotten there yet.

At this point, I hope to periodically (maybe monthly) write new banners to go up there.

Progress

I’ve been working on two projects. Well, one project with a sub project. And I switched jobs (more soon).

I have a web app that I’ve been working on updating to be ajax’y. It is a feed reader (think Google Reader). I originally forked feedonfeeds when it was going through a period of being abandoned. It was originally written in PHP. First I made some small work flow tweaks. Then, I got sick of the feed downloader code getting stuck, so I rewrote the backend cron job in Python. I think that feedparser is more reliable than whatever the original used. I know forget what it was since I’ve since deleted it. Then I ran into issues with badly formatted HTML in the feeds causing the HTML display to go crazy, so I added in BeautifulSoup to cleanup the HTML. That fixed it for formatting, but I’m still having some issues with forms (yes, people embed those in feeds, irritatingly) interfering with each other, so there is obviously more work to be done there.

My more recent focus was replacing the frames based front-end (still PHP on the server) with more AJAX style code. That is going well, and picking up speed as I learn jQuery more, but there are plenty of rough spots, and some functionality isn’t quite on par with the frames. I’m forcing myself to use the ajax version though.

I don’t know if I will ever share the project. Currently, it is written to be single user and still has a lot of rough edges, as mentioned, so I can’t easily just make it multi-user with me hosting. I could post it for download, but it depends on postgresql, php, python, and quite a few other libraries for both so and I doubt that I’d attract anyone to try to install it. So, for the near future, it will continue to be a private project.

Perhaps once I get the UI more polished, I will replace all the PHP with Python and the CherryPy framework. Then it might start to become distributable at least.

The sub project is that I’m working on a jQuery sliding divider plugin. You can follow that here:
https://github.com/jd-boyd/jsplitdiv

The relationship to the feed reader is that frames are resizable, but divs aren’t, at least not without help. There is no reason for this not to be used plenty of places, so I put it on github.

I’m happy that when I spot check it, it pretty much works on IE 8, even though I’m not doing anything to specifically target it. I am using jQuery though, which I’m sure hides some of the possibly troublesome parts.

Anyway, since it is a project that can’t easily be shared, it is hard to write about.

PodTrack

This is a little command line utility for downloading podcasts into a directory. This can make for easy rsyncing to a flash card, or listening with a command line audio player.

Find it at:
https://github.com/jd-boyd/Podtrack

I use it.  I don’t think anyone else does.  It prints a large amount of crap when running, which I really should clean up.  Otherwise, it is fairly complete, allowing you to do full management from the command line, and offering opml import and export.

Also, it does feature a thread worker pool class.  The implementation is actually a bad idea for any computation task (because of the Python GIL), but it works well enough for code that blocks in C extensions a lot.  Still, another TODO task for me is to update PodTrack to use multiprocessing, which already has worker pools.

C++ Message Passing (overview, between threads)

Having installed Google Analytics two months ago, I now know that the two most popular search phrases that people use to get here from Google are “elisp hex” and “c++ message passing”.  There isn’t much more to say about using hex in elisp, but I’m sure that there is more to say about C++ message passing.

The first thing that comes to mind is what does that phrase mean?  The obvious interpretations are passing messages between threads, passing messages between processes, passing messages on a network (local or Internet).

I previously mentioned that phrase in C/C++ Message Passing and Small Revisit of C++ Message Passing (in a Threaded Program) .  Both relate to passing messages between pthreads in a C++ program.  I rely heavily on that pattern because I prefer to try to treat my threads as shared nothing.  When possible, pass messages between threads instead of relying on shared data structures, and you will eliminate a lot of potential bugs.  If done correctly, it will also help eliminate bottle necks coming from lock contention.

In theory, I don’t like shared everything threading, but there are times when sharing things is required for performance, and there are times when parallization tools like OpenMP (or auto-parallelizers) are very valuable, and they required a shared every thing environment.

One of these days, I should try adjusting the implementation to use shared memory to enforce the seperation even more.  That, of course, would take us into messages between processes, which is a post still in progress.


ReferUrl.net, Updated

I’ve been continuing to poke at the ReferURL.net URL shortening service I run. Recently I’ve added the following new features:

  1. The biggest change is the new bookmarklet.  Instead of redirecting you away from the page you are viewing, it overlays a modal dialog giving you your results. Please delete your old bookmarklet and bookmark the new one to gain this feature.
  2. Accessing a ReferURL in an email or tweet is now much faster. The 1-2 seconds saved will improve the impression your ReferURL leaves on your readers.
  3. Enter an Amazon URL and ReferURL will return a shorter Amaon URL in addition to a ReferURL.  Thus, http://www.amazon.com/jQuery-Cookbook-Solutions-Examples-Developers/dp/0596159773/ref=sr_1_1?ie=UTF8&qid=1292044752&sr=8-1 is now offered to you as http://amzn.com/0596159773 as well as http://referurl.net/3568 .
  4. Also, ReferURL does the same for eBay auction links.
  5. Email from the ReferURL result page.  Just click under Mail It!, and the ReferURL is placed in a new email message for you (this opens the new message in your local Outlook, Mail.app, Thunderbird, or other configured default).

I like using ReferURL.net a lot better now, and I hope you do as well.

RESTq, RESTful queuing, update

First, I renamed it from msgQ to RESTq (or more likely just restq).  Also, I’ve joined the modern age and pushed it up to github:

https://github.com/jd-boyd/restq

Up next, I shall put it into real use, and I do still want to add clients for more languages. Also, just because I can, I might rewrite the server in more languages (Python will probably remain the main one).  I’m thinking of Go, perhaps C, and of course from the beginning I meant to also do so in erlang.

πp-v2, the new Plan 9 file protocol

The symbol is Pi. I point that out since it doesn’t render very nicely in my web browser. The URL for the paper is:
http://proness.kix.in/misc/%CF%80p-v2.pdf
Yes, they stuck the symbol into the file name as well. Plan9 people seem to like to do such things just because they invented UTF-8 and were the first to heavily use unicode.

πp intends to replace NFS, SMB/CIFS, HTTP, FTP, 9p2000/Styx (the former plan9 file system), as well as more obscure options like Coda. To that goal, it tries to contain all of the features of those different choices that the authors deemed still useful into one new protocol.  Ideas like extended metadata, non-file files (meaning the target on the other end is a file representing a printer, raw disk, or any other sort of device), offline operation, caching, and tolerance for proxies (possibly also caching) in the middle.

So, I think there are some features and decisions to like here. Here are a few.

File versioning can serve two purposes. It can be used to preserve history and it can be used for cache invalidation. πp intends to support both. File versioning to support history is potentially the most handy feature, but it will only work if it is matched with an appropriate on disk file system that supports file versioning. ZFS, one file system that can support versioning (by snapshots, but it doesn’t keep every revision automatically), doesn’t need the file sharing protocol’s help to support looking at the versions from remote systems. On ZFS, the snapshot history can be accessed in a directories .zfs sub-directory.

Without such a file system that supports versioning, πp’s versioning feature can still work for cache invalidation by setting an extended attribute on file systems that have some mechanism for doing so (xattr on some, resource forks on others).  Personally, I think that supporting caching is largely only of use when πp is used as a replacement for http, which is further discussed later.

Pipelining would help resolve a major NFS complaint of mine, but it also seems like a large pit of danger. I’d be happy to consider using this protocol, but only for applications that don’t depend on heavy synchronous behavior out of concern about this feature, until it is better proven.

Extended support for special files is great for remotely sharing devices.  This was always one of the nice features of plan9 (at least so it sounded, since I never gotten around to using plan9).

OTOH, here are some “features” that I require convincing of.

To replace http, the author’s get rid of the plain text parts. In web development, having those parts be human readable are tremendously useful and I doubt they waste much time in any given session. They also make web connections stateful using this protocol, while being stateless has always been one of http’s strong points (albeit one with some pain). Also extended file attributes seem like a poor alternative to http request and response headers.

Supporting lossy file transmission is rather a head scratcher. Their idea is that it could make for efficient streaming video transport, but still I’m not convinced. Does the client or server decide that the response is to be lossy? It looks like this is a client decision (client choosing by opening the πp connection over TCP versus UDP), however how is the client to know in advance which type of file this is? Is it supposed to query first? That seems ludicrous.

My conclusion though is that if you drop the usage ideas that I don’t like, I don’t see any changes being required in the protocol itself. This idea is obviously in the early states, but it looks like it could be worth exploring when there is more code to run.

On a side note, one thing that did interest me in the paper is that they wrote the initial implementation in Go, and then translate that to C. I didn’t know that Go existed on Plan9, but in looking into that it seems that it still doesn’t quite exist on plan9 since the runtime has yet to be ported. Oh well, one can hope for the future.

It also made me wonder how many plan9 from user space users make use of 9p still?

Small revisit of C++ message passing (in a threaded program)

The post C/C++ Message Passing from June ’08 is one or the more popular ones, if Google Analytics is to be believed. Not only that, but every one who arrived there by Google search was looking for C++, not C.

Upon re-reading it, it occurs to me that it would be a good candidate for being a lock free structure.  With C++0x, this should be pretty simple to do, using the new atomic template.  For straight C though, I would have to use compiler specific extensions (or inline assembly), so there may be little reason not to make a C++ specific version of the code that is lock free.

I’ve also thought of trying to use C++0x to construct a purely functional data structure library.  In that case, C++0x would be chosen because of the new shared_ptr addition.  I might have to have a post soon exploring that idea a bit further.

So far, I’ve converted this to be a C++ class and to use templates instead of void*s. Otherwise, it still behaves the same and still uses pthread locking directly. Additional versions will be forthcoming. Here is the recent work.

Hopefully it will be of use to some of the searchers. Keep an eye out for future revisions.