Minx Update July 14

Has it been a month already? (Checks.) Yes.

I've just finished converting Minx from the proof-of-concept CGI version to the production-ready CherryPy version. I was kind of ducking this issue, because it required me to make a couple of hundred changes to the code (mostly small changes, but lots of them), and I had to make them all at once.

I like to work on an incremental change/test cycle, and this was the opposite of that. So I ended up spending two days messing around and not achieving anything useful, one day working out how to make it work, one day making the changes, and then a solid day debugging.

But it works! Yay! (Uh, mostly. I'm having a little difficulty getting it to co-operate with out main Apache installation, which is why Minx is currently sitting on port 81.)

Other things I've been working on are comment registration, the caching system, and double-dynamic templates.

Comment Registration

With a community like munu, and even more so with the new site, this is the only way to go. A single login will work across all the munu blogs. We'll have to work out how banning will work; the new site will have a sophisticated dickhead-filtering system, but I haven't even started on that yet.

There will be cool toys for registered users; I'll start working on those next week.

Caching

Minx is already fast enough for munu. We serve something like 100 to 200 pages per minute; Minx can do up to 1500 (if they're complicated pages with inline comments, more for simpler pages). But if you get Instalanched/Slashdotted/Digged/whatever, you don't want the server grinding to a halt.

The caching system keeps a copy of pages generated. The first version of caching just saved the copy and served it up again if the page was requested again in the next five minutes. The new improved version actually tracks if new posts or comments are made, and rebuilds the page as needed.

With caching, Minx could potentially serve 15,000 pages per minute on the munu servers. Though at that rate, we'd run out of bandwidth pretty fast.

Double-Dynamic Templates!

The problem with caching is that there are some things that you want to change depending on who is looking at the page. If you have a welcome message, like "Hello Pixy Misa, welcome to Munuviana!", you really don't want that to show for everyone who looks at the page for the next five minutes.

The solution to that is Minx's Double-Dynamic Templates.

Actually, they're sort of triple-dynamic...

Here's how Minx generates a page:

First of all, it finds out what blog you are looking at. That ?blog=14 bit is what tells it right now; I'll be fixing that up so that it works it out from the domain name.

Then it goes to the database and finds the Page template for that blog. If there isn't one, it fetches the default Page template, which lives on blog number 1.

The Page template has instructions like [posts:here] which says, well, put the default number of posts (currently 20) at this place in the page. For the posts it looks up the Post template; you can specify an alternate template, or you can use the generic [posts] ... [/posts] structure and put all the details in the main Page template.

For each post, and similarly for each comment, Minx can apply a series of filters to the text. This includes things like smilies, macros, bbcode, HTML sanitising, Textile, WikiText, smartquotes, and so on. Once the text from the post or comment is mangled to your satisfaction, it gets slotted into its place in the template, specified by [post.text] or [comment.text].

Finally, after pulling up all the posts and comments and other details, Minx puts together the complete page. It saves that page in the cache for posterity.

And then it processes it again, for a second set of tags. Right now the only one that works is the performance stats block at the bottom of the page. You can tell Minx where (or if) to place the stats blog using the [magic.stats] tag, but that really only replaces that tag with a [final.stats] tag. And that tag gets processed even if the page is read from the cache, in a second template pass.

That's how it can say:

Processing 0.08 seconds.
45 queries taking 0.06 seconds, 141 records returned.
Page size 86 kb.
Powered by Minx/MT 0.6.1.0a.

and then, when the exact same page is reloaded, come back with:

Retrieved from cache, processing time 0.0 seconds.
Page size 86 kb.
Powered by Minx/MT 0.6.1.0a.

This gives Minx the flexibility to do, well, pretty much anything.

The reason I say that it's sort of triple-dynamic is that bbcode and macros both hvae the ability to process template tags right into your posts. I still haven't decided whether that's a good thing... But it does mean that I can do this:

[final.stats]

Next up: The interactive stuff. Comment editing (one of the benefits of user registration). Um, the user registration itself. Posting. Private messages. Searches. And then, the retromingent spam filter!

Update: Tee hee hee!

Posted by Pixy Misa at July 14, 2006 01:57 AM | TrackBack