Announcing... pyfw

February 24th, 2013 about python and soofw

After about a week's worth of effort, I've finally got a functioning version of soofw based on a Python and mod_wsgi. I wrote my own framework system to get a better understanding of the web environment for Python, but I'm not entirely confident it's the best solution for my needs. Earlier today I finished importing all of the legacy posts to the new system and flipped the switch to transition from PHP to Python.

Overall, it was an enjoyable experience. It felt pretty satisfying every time I got a piece to click into place. Python made it easy and quick to implement all of the features of the previous version plus some new ones. I wrote the new system with the eventual goal of open-sourcing it on GitHub, but at the moment it's a little... messy. I'd like to spend some time with it clean it up and get a chance to see where it misses the mark. I can already tell that it's a much more powerful and flexible setup than my old PHP, so I'm pretty hopeful.

Database

When I first started writing the code, I decided I didn't want to use a(n) SQL database anymore. They're too clunky for such a simple project. I didn't want to use an awkward (and probably insecure) admin system every time I wanted to publish a new post, and I definitely didn't want to write that admin system. I also didn't want to use a standard MySQL admin system like phpMyAdmin and its ilk, because they're ugly and hard to use. Using a database system also comes with some other barriers, like migrating between a production and testing server. It annoyed me.

As a result, I decided to use a filesystem based "database". Within the application directory I have a posts directory, and inside of that posts directory I have a list of plaintext files. A script reads the directory and generates a post listing. Each post file contains some Markdown-formatted source text and some meta properties like publish date and title. From there, the view decides which posts to display and fills a buffer with rendered templates, which the WSGI app returns as the page content.

First, yes, I know that most Python web frameworks come with their own database management system. Yes, I know that databases are much faster than a filesystem set up. Yes, I know that I can't query the filesystem and find posts matching a certain tag, date range, etc. But I also don't need any of that. I did away with tags a while ago; Google does it better anyway. I only have 40-something posts, so the speed a non-issue. I even tested it with 1000 posts and the read times were negligible. As for the management system, I'm much more comfortable using a terminal to manage my posts. I can clone the site's Git repository to my computer, write a new post and preview it locally, and then commit and push back to the site to publish. I'm not missing out on any features, either, because I have the full power of the Linux shell to help me out. I can search my posts with grep and write them with Vim. I don't have to deal with a bad web editor or funky search syntax. At the moment, I'm much more content with the flexibility of a filesystem database than a standard one. I also don't have to store my database credentials in my source files anymore, either.

No more PHP

Where to start... PHP sucks. There's dozens of posts defending it, and some of them have a lot of great points. It does have a really low barrier to entry, and that's the primary reason that I learned it. It was easy. I could make dynamic, templated web sites in a matter of minutes, and it took away all of the pain of updating all my HTML files whenever I made a design change. It was great at the time, but now I can't stand it. I can't stand looking at my old code and seeing foreach($array as $key=>$val) anymore. The $ variable prefix makes code incredibly hard to read, and I still don't even understand the as...=> syntax. There's even more weird stuff in there, like the infamous T_PAAMAYIM_NEKUDOTAYIM error messages and function case-insensitivity (MAX(...) is the same as mAx(...)). Variables are case-sensitive, however. It's just functions that don't matter. Why? I don't know. If you're interested in a more thorough write up, I suggest you read eevee's post.

Markdown

I was mostly just sick of writing <code>...</code> every time I wanted to highlight something in monospaced font. HTML tags are pretty verbose, and when you're trying to get thoughts out of your head it can be a pain in the ass to spend an extra 13 (17 with shift!) keystrokes on formatting. Markdown, as stated in its design philosophy, is easy to read and easy to write. It's such a natural system for formatting, and it has a whole host of features I didn't even know existed until I actually checked John Gruber's official documentation. I never want to go back to writing my posts in HTML. Miserable.

Vim

Oh god. I can write my posts in Vim now, and I don't even have to cat the file, copy it, and then paste it into MySQL. This is easily my favorite feature of the new system. Paired with Markdown formatting, I've made it orders of magnitude easier to write my posts.

Open-Source

I can do this now that I won't expose any possible security holes like the database credentials, schema, server, etc. Huzzah! Beyond that, Steve Losh was a heavy influence on this decision, so I'm just going to redirect you to his article instead.

Directory Structure

Not much to say here, but my last directory structure was bad. I mean it. Insecure, horribly unorganized, unmaintainable, and straight up bad. I've reorganized and toyed with my Apache configuration to create a much more rigid organization.

Portability

Kinda? It's much easier to get my site running on a new machine when it's not tied to a database server. All I need to do is install the dependencies and set up the Apache config files. Lovely!

Maintainability, Readability, Writeability

My code is so much easier to maintain, read, and write now. Python has made my life so much easier than it was with PHP. The post database is also much more accessible and maintainable as well, so that's a giant bonus.

Publishing Method

Again, simplifying my life. I write posts in plaintext on a local copy of the site and then push them to the server when I want to publish. It's simple.

Version Control

I can store my posts in version control now! If I accidentally rm posts/* I can just restore them through Git. Paired with the improved directory structure, this feature is even better. The former structure made it possible but not easy (I still used it, however), so I'm excited for this.

That's it... I think. If you have any suggestions or comments, please feel free to contact me via email! Thanks for reading.

How to Learn Vim

February 17th, 2013 about vim

Well, at least that's what I was planning on writing today... Unfortunately, I got a little distracted by the idea of rewriting this web site to use Python instead of PHP. It's not that I hate PHP (I do) or anything, but I've recently become fascinated with Python. I've tried this before, but it ended rather quickly because I was completely unfamiliar with Python, web frameworks in general, and the specific framework I was using (Django). Since I've gained significantly more experience with Python, hopefully I'm much more prepared for attempt two.

Since I was originally going to write about learning Vim, I also wanted to show off this neat Chrome extension I just discovered: Vimium. Essentially, it adds some awesome Vim-style keybindings to the browser, such as hjkl movement, gg and G jumping, and /searching. If you're familiar with Vim, give it a shot. If you're still learning Vim, give it a shot. If you've never used Vim, give Vim a shot. If you've never heard of Vim, look it up and then give it a shot.

Terminals Can Have Colors!

February 9th, 2013 about shells

I love using terminal emulators. I think they're pretty neat, and I think they're pretty powerful too, but I also think they can be pretty hard to use without some moderate customization. Conveniently enough, I love personalizing my gadgets, tools, and programs. I've spent a lot of time in the last two years collecting tiny snippets and adding them to my .bashrc, and all of this accumulation has resulted in a rather personal terminal / prompt set up. I'll try to highlight the best features in this post so anyone else can use them.

Before starting, I would like to point out that this is my prompt and is designed to suit my needs. You might have different needs, wants, and goals, so this might not be a perfect set up for you, but I'm hoping you'll at least learn something new. I would also like to point out that (almost) all of my Linux dotfiles are available on my GitHub page.

Why bother?

I'll just quote Steve Losh's wonderful answer to the same question:

Many people use the command line every day and never bother to customize their prompts. It’s just a bit of text that’s printed before every command — why should you waste time learning how to customize it?

I feel that the most important aspect of my command line work is the prompt. Your prompt is something you’ll see literally thousands of times a day. Why not take 30 minutes and customize it into something that’s much more useful?

I firmly believe I’m right in thinking this way. 30 minutes (or even 3 hours) of customization to make your work easier for the rest of your life (or at least however long you stay with your current shell) is worth it.

That's why. Even more importantly, most default prompts are completely unusable. They usually lack any color highlighting which makes them blend into large command output. Some even lack the current working directory, and most are in such a compact form that it can be hard to distinguish one part from another without colors. For example:

john@desktop:$HOME/projects/arduino/>
vs
john desktop ~/projects/arduino master?+4 $

Without even reading my own prompt, I can determine I'm logged into my desktop and working in a git repository. After reading, I can tell you that the repository has untracked files but no modifications, I'm 4 commits ahead of the remote repository, and I'm currently tracking my project time with clk. I can't say anything good about the alternative.

Colors, Font, and Terminal

For a terminal emulator, I use GNOME Terminal or the Secure Shell Chrome Extension when on my Chromebook or a non-Linux machine. I'm using Proggy Clean (Slashed Zeros) 12pt for my font, although I find the size varies from machine to machine and terminal to terminal. Make sure you always find the appropriate size for Proggy or it will look like garbage; pixel fonts are atrocious unless they're sized correctly. My background is black with approximately 80% transparency. I use a customized 16-color palette that really likes to mess with color-highlighted output, so copying it exactly may not be the best idea unless you can put up with the occasional color mistakes. The first 8 colors are all varying shades of gray, while the next 8 are some rainbowy accent colors:

  • Default: Light gray #0 #D9D9D9
  • Color 0: Dark gray #1 #2E3338
  • Color 1: Dark gray #2 #454C54
  • Color 2: Dark gray #3 #5C6670
  • Color 3: Gray #1 #737F8C
  • Color 4: Gray #2 #8F99A3
  • Color 5: Light gray #1 #ABB3BA
  • Color 6: Light gray #2 #C7CCD1
  • Color 7: Light gray #3 #E3E6E8
  • Color 8: Red #FF4D6A
  • Color 9: Orange #FFA64D
  • Color 10: Yellow #E1FF4D
  • Color 11: Green #6AFF4D
  • Color 12: Seafoam #4DFFA6
  • Color 13: Cyan #4DE1FF
  • Color 14: Blue #4DA6FF
  • Color 15: Purple #A64DFF

Color Scheme Screenshot

Shell and Prompt

I'm using bash as my shell. I've heard great things about zsh, but I find that bash is more universally available. It's also the default shell on most machines, so this post should hopefully be applicable to most people.

My prompt leads with the history number of the current command. This seemed like an excellent feature at first, but I'm starting to use it less frequently than I used to. Essentially, having the line number be so visible makes it much easier to use bash's !### history lookup, where ### is the history line. In practice, I find myself using ctrl-r and !partial more frequently. In bash, ctrl-r allows you to reverse search through the history by typing part of a command, while !partial will automatically reverse search the history and be replaced by the first command that starts with partial. For example, frequently editing a file with vim file.c and compiling with make can be repeated with !v and !m. That can get a little tricky if you suddenly use a man page however, as !m will find the most recent command that started with m.

Immediately following the history number is my username and the hostname, color-coded according to the host. For example, my desktop prompt is purple, while my web server is yellow and my Raspberry Pi is raspberry red. If I'm logged in as root, my username will also turn red, and the command separator will turn into a #. After the hostname is my current working directory highlighted in green. The neatest part, in my opinion, is the current git repository status between the working directory and command separator. While I'm currently in a git repository, the working branch shows up in red. If the current repository has no changes but contains untracked files, a question mark will be appended after the branch. If the repository has staged or unstaged changes, an asterisk will be appended. If the repository is ahead of the remote, a +X will be appended to notify me of how long it's been since I last pushed my commits. Lastly, the command separator (normally a $, but # when root) changes colors based on my clk status: green means I have no clock data for this directory, red means I'm clocked in and tracking my time, and yellow means I'm clocked out and not tracking my time.

Prompt Screenshot

Conclusion

I'll try to elaborate more on how to add these features in the future, but I just wanted to give a nice sample of my prompt. For the adventurous, you can check out my .bashrc at GitHub.

Layout Testing

February 2nd, 2013 about soofw

So, I'm at it again. soofw has a new style. Here's a preview of all the things I plan on using. Up at the top is the standard header level 1 with the post title. Titles with a dotted underline will link to another resource. The date (and its lemniscate) provide a permalink to this article.

If viewed on the home page, the article will break after this paragraph and ask you to continue reading. If viewed as a single article, nothing special happens after this paragraph. Oh yeah, here's some inline code style.

Header level 2 and paragraph

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis suscipit tempor eleifend. Etiam id odio non leo varius tempor. Maecenas lorem mi, rhoncus eu a link for awesomeness interdum eu, sodales ac justo. Sed tellus nunc, elementum in eleifend at, hendrerit ac nisi. Morbi accumsan, nunc nec faucibus hendrerit, lectus lorem luctus diam, sit amet feugiat neque massa placerat quam. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec enim augue, pretium eu eleifend eget, venenatis in urna. Suspendisse non augue sem.

Header level 3 and paragraph

Fusce justo nibh, pharetra ac volutpat in, condimentum ac nisl. Sed sodales sodales sem nec gravida. Quisque ut ante nec augue tincidunt consectetur at nec magna. Integer fermentum neque sodales tellus semper consequat. Curabitur nec tempus erat. Donec nisl dolor, convallis nec ultricies at, lacinia non orci. Ut id ligula elit, at dignissim enim. Praesent hendrerit, lorem sed fringilla pharetra, nisi turpis consequat ligula, et placerat leo libero nec dui.

Quote

To this end, Markdown's syntax is comprised entirely of punctuation characters, which punctuation characters have been carefully chosen so as to look like what they mean. E.g., asterisks around a word actually look like emphasis. Markdown lists look like, well, lists. Even blockquotes look like quoted passages of text, assuming you’ve ever used email.

Block formatted code

Source Code Pro Regular
abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ
0123456789.:,;(*!?')
0oO()[]{}
1iIl|L

Unordered list

  • Lorem ipsum dolor sit amet, consectetur adipiscing elit.
  • Aliquam eget augue vitae enim lacinia mollis.
  • Praesent tristique turpis quis lectus aliquam at fringilla velit auctor.
  • Nulla eu augue nulla, a posuere felis.
  • Nam a ante non tortor faucibus placerat nec at metus.

Image

Alt Text

Embedded Video

Closing paragraph

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis suscipit tempor eleifend. Etiam id odio non leo varius tempor. Maecenas lorem mi, rhoncus eu interdum eu, sodales ac justo. Sed tellus nunc, elementum in eleifend at, hendrerit ac nisi. Morbi accumsan, nunc nec faucibus hendrerit, lectus lorem luctus diam, sit amet feugiat neque massa placerat quam. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec enim augue, pretium eu eleifend eget, venenatis in urna. Suspendisse non augue sem.

Pebble

February 2nd, 2013 about gadgets

So I ordered a Pebble. For those of you who don't know, it's a neat little watch with an e-paper display and Bluetooth connectivity. The basic idea is that you'll have this small device on your wrist that can connect to the your phone and the internet and provide you with a handful of subtle notifications, such as new text messages, phone calls, emails, etc. It can also communicate back to the phone for things like music playback. I chose a Pebble mostly because of the overwhelming amount of hype surrounding it; it reached its $100,000 Kickstarter goal in merely two hours. Plus, it's got a whole array of nice features: sleek design, seven day battery life, water resistance for showering/swimming, high-contrast display, vibrating motor, and accelerometer for gestures. Oh yeah, and it's completely customizable and has an SDK for writing your own apps.

From what I've gathered, you can write your own custom notifications via if-this-then-that without even touching any code. Or if you want more depth, you can write your own apps or watchfaces using their public SDK. Personally, I'm really excited to see what kind of cool things I can make it do and some of the useful apps made by other people. At the moment, I'm just hoping I can get the time + weather watchface displayed in this picture. I'm also planning on making use of the three buttons to implement some sort of timer/stopwatch app so I can instantly set up timers.

Creating Our World

January 22nd, 2013 about me

Sometimes I take a step back and actually think about everything I interact with on a daily basis. My computer is capable of billions of computations per second and can connect with millions (billions?) of other devices just like it through some magic cables that run all over the planet. I can fit an entire computer in my pocket. My phone is essentially a complete computer as well, and it can literally communicate with anywhere in the world just through the air.

I can wake up warm, dry, and safe every morning in a giant wooden structure. I can buy a digital piece of paper on my phone before I get out of bed. After getting out of bed, I can hop in the shower and have clean water rain down from a magic tube. I can eat food that was made hundreds of miles away and delivered to my house. I can then get in a self-propelled, hollow, metal box with wheels and use it to travel several miles to an airport. From there, I can climb aboard a long metal tube with wings and take off into the sky, arriving at my destination within a few hours. It's absolutely incredible to me.

How did these inventions come about? How is it even possible to create something with that magnitude of complexity? I struggle managing a mere several thousand lines of code or a dozen wires. I cannot even comprehend how people, even hundreds of them working together, could create something as complicated as an airplane, or a home, or a CPU, or an office building, or anything I have ever used. How did we fly to the moon?! I simply cannot understand it.

What's even more amazing is that so much of this advancement has taken place in less than a century. The Ford Model T, the first affordable automobile, was produced in 1908. The Wright brothers made their first flight in 1903. The first programmable computer didn't exist until 1936. We landed on the moon in 1969. How on earth have these inventions become so popular so quickly? How did we go from a computer that can compute one operation per second to one that can computer 3.47 billion operations per second in each of its six cores in less than 80 years? How did we come so far in such a small amount of time?

It's truly an incredible time to be alive. Don't forget that.