Announcing... pyfw

February 24th, 2013 about python and soofw

After about a week's worth of effort, I've finally got a functioning version of soofw based on a Python and mod_wsgi. I wrote my own framework system to get a better understanding of the web environment for Python, but I'm not entirely confident it's the best solution for my needs. Earlier today I finished importing all of the legacy posts to the new system and flipped the switch to transition from PHP to Python.

Overall, it was an enjoyable experience. It felt pretty satisfying every time I got a piece to click into place. Python made it easy and quick to implement all of the features of the previous version plus some new ones. I wrote the new system with the eventual goal of open-sourcing it on GitHub, but at the moment it's a little... messy. I'd like to spend some time with it clean it up and get a chance to see where it misses the mark. I can already tell that it's a much more powerful and flexible setup than my old PHP, so I'm pretty hopeful.

Database

When I first started writing the code, I decided I didn't want to use a(n) SQL database anymore. They're too clunky for such a simple project. I didn't want to use an awkward (and probably insecure) admin system every time I wanted to publish a new post, and I definitely didn't want to write that admin system. I also didn't want to use a standard MySQL admin system like phpMyAdmin and its ilk, because they're ugly and hard to use. Using a database system also comes with some other barriers, like migrating between a production and testing server. It annoyed me.

As a result, I decided to use a filesystem based "database". Within the application directory I have a posts directory, and inside of that posts directory I have a list of plaintext files. A script reads the directory and generates a post listing. Each post file contains some Markdown-formatted source text and some meta properties like publish date and title. From there, the view decides which posts to display and fills a buffer with rendered templates, which the WSGI app returns as the page content.

First, yes, I know that most Python web frameworks come with their own database management system. Yes, I know that databases are much faster than a filesystem set up. Yes, I know that I can't query the filesystem and find posts matching a certain tag, date range, etc. But I also don't need any of that. I did away with tags a while ago; Google does it better anyway. I only have 40-something posts, so the speed a non-issue. I even tested it with 1000 posts and the read times were negligible. As for the management system, I'm much more comfortable using a terminal to manage my posts. I can clone the site's Git repository to my computer, write a new post and preview it locally, and then commit and push back to the site to publish. I'm not missing out on any features, either, because I have the full power of the Linux shell to help me out. I can search my posts with grep and write them with Vim. I don't have to deal with a bad web editor or funky search syntax. At the moment, I'm much more content with the flexibility of a filesystem database than a standard one. I also don't have to store my database credentials in my source files anymore, either.

No more PHP

Where to start... PHP sucks. There's dozens of posts defending it, and some of them have a lot of great points. It does have a really low barrier to entry, and that's the primary reason that I learned it. It was easy. I could make dynamic, templated web sites in a matter of minutes, and it took away all of the pain of updating all my HTML files whenever I made a design change. It was great at the time, but now I can't stand it. I can't stand looking at my old code and seeing foreach($array as $key=>$val) anymore. The $ variable prefix makes code incredibly hard to read, and I still don't even understand the as...=> syntax. There's even more weird stuff in there, like the infamous T_PAAMAYIM_NEKUDOTAYIM error messages and function case-insensitivity (MAX(...) is the same as mAx(...)). Variables are case-sensitive, however. It's just functions that don't matter. Why? I don't know. If you're interested in a more thorough write up, I suggest you read eevee's post.

Markdown

I was mostly just sick of writing <code>...</code> every time I wanted to highlight something in monospaced font. HTML tags are pretty verbose, and when you're trying to get thoughts out of your head it can be a pain in the ass to spend an extra 13 (17 with shift!) keystrokes on formatting. Markdown, as stated in its design philosophy, is easy to read and easy to write. It's such a natural system for formatting, and it has a whole host of features I didn't even know existed until I actually checked John Gruber's official documentation. I never want to go back to writing my posts in HTML. Miserable.

Vim

Oh god. I can write my posts in Vim now, and I don't even have to cat the file, copy it, and then paste it into MySQL. This is easily my favorite feature of the new system. Paired with Markdown formatting, I've made it orders of magnitude easier to write my posts.

Open-Source

I can do this now that I won't expose any possible security holes like the database credentials, schema, server, etc. Huzzah! Beyond that, Steve Losh was a heavy influence on this decision, so I'm just going to redirect you to his article instead.

Directory Structure

Not much to say here, but my last directory structure was bad. I mean it. Insecure, horribly unorganized, unmaintainable, and straight up bad. I've reorganized and toyed with my Apache configuration to create a much more rigid organization.

Portability

Kinda? It's much easier to get my site running on a new machine when it's not tied to a database server. All I need to do is install the dependencies and set up the Apache config files. Lovely!

Maintainability, Readability, Writeability

My code is so much easier to maintain, read, and write now. Python has made my life so much easier than it was with PHP. The post database is also much more accessible and maintainable as well, so that's a giant bonus.

Publishing Method

Again, simplifying my life. I write posts in plaintext on a local copy of the site and then push them to the server when I want to publish. It's simple.

Version Control

I can store my posts in version control now! If I accidentally rm posts/* I can just restore them through Git. Paired with the improved directory structure, this feature is even better. The former structure made it possible but not easy (I still used it, however), so I'm excited for this.

That's it... I think. If you have any suggestions or comments, please feel free to contact me via email! Thanks for reading.