/usr/bin/website

I think it would be wonderful if Apple would add a few features to iTunes / iTunes Store.

• Full ID3 searching
• Full lyric searching
• Real regex based searching

Since Apple legally sells its songs, it carries licencing for each song it has (or at least I believe it does). Given that, why can’t I search for any field in the ID3 tag? Why can’t I view song lyrics? Why can’t I use a regex search to weed out what I want to hear and what I don’t want to hear?

Maybe not even regex searching, maybe an SQL-like syntax instead. Also include some statistics based seaarching in as well.

I know that iTunes has support for “Recently Added”, “Recently Listened To”, “90’s Music”, etc etc etc; but why can’t I access that information from the search bar?

Maybe there is and I’m just too daft to realize its there / how to use it.

At any rate, Apple, that’s what I’d like to see in iTunes.

I recently purchased an Arduino Decimila, and have been playing with it non-stop for days now. After browsing through the examples and tutorials on the Arduino project homepage, I’ve started working on some slightly different things involving the unit.

At the moment, I think it’s fair to say that all I’m really working with is a microcontroller, a breadboard, an LED and a resistor. Not much to really have fun with, right?

Well, Potentiometers are basically fancy variable voltage regulators. They control a voltage based on an analog input, say.. turning a knob. A lot of fun can be had with certain types; especially photo-sensitive or pressure-sensitive ones.

I unfortunately have not had the time to go out and get a decent arsenal of electrical components. However, what I can do in the meantime is emulate their behavior. Enter Processing and its Serial interface.

The development environment that you program for the Arduino is based on the same software that Processing is run in, so they share a lot of common ground. Likewise, they can talk to eachother very well.

I scripted a simple applet in Processing that contains a scrollbar, and it emulates the behavior of a dimmer switch. Slide it back and forth to change the brightness. I also scripted up a server-type application to run on the Arduino to listen for Serial data.

You can view the video below to see how my experiment turned out.

(notice how the led on the breadboard gets brighter and dimmer as the slider moves)

That example really isn’t all that complicated either. The Processing source code is only a page long (not counting the HScrollbar class that I borrowed), and the Arduino source code is half the size of that even.

Hopefully, after I get some more ideas flowing and get some real parts I’ll have a little bit better stuff to talk about.

While working on my JavaBot, I came to a point where simply writing out each response and storing it as a hardcoded array was, to say the least, tedious. Actually editing bot source code every time I wanted to add a new phrase to his vocabulary… it’s just horribly inconvenient.

So what are we to do? Well, I got to researching, and I found out that another bot project has an interesting way of managing this sort of thing. InfoBot, written in Perl, uses .fact files to store information. These .fact files are essentially associative arrays that get parsed at runtime.

Taking that idea and running with it, I downloaded a simple .fact file to test with. The one I used was UnixCmds.fact, and it’s safe to assume that all it contained was unix commands and their descriptions.

In the file, they look like this:

cd => change the current working directory
mkdir => create a new directory

Which, as it turns out is very easy to parse in Java:

String[] data = line.split( " => " );

This creates a new String with the commands in one index and the descriptions in another. Referencing them us easy, especially while in a loop like how I did it. data[0] is the command, data[1] is the description.

Here’s a look at the method in charge of creating the “database”:

    private static Map getDatabase( String filepath )
    {
        try {
            BufferedReader reader = getReader( filepath );//create this on a more local level
            Map _database = new HashMap();//our local database

            String line;
            while( ( line = reader.readLine() ) != null )
            {
                String[] data = line.split( " => " );// .fact files are "key => value"
                _database.put( data[0], data[1] );//populate the database
            }
            return _database;//this will be referenced over to the main database
        } catch( NullPointerException e ) {
            //3 hours of recoding to try to figure out why i was getting a NullPointerException.
            System.err.println( "Check fact file for extraneous whitespace or newlines" );
            //Turns out it was an issue with the .fact file i was using to test with
            return null;
        } catch( IOException e ) {
            System.err.println( "There was an issue generating the database" );
            return null;
        }

In there, I use a custom method to acquire the BufferedReader, and then I create a HashMap to store the various factoids. “database” is a static global that gets called later in a query() method.

The whole source code for my parser is over here. I’ll be updating it periodically to add more functionality to it.

One of the projects that I mentioned in my previous post was JavaBot (cliche name that I’m going to have to change I guess). JavaBot is an IRC Bot written in Java. He’s fairly simple to use:

C:\Users\dydx> java -jar "C:\Users\dydx\Desktop\JavaBot\dist\JavaBot.jar" irc.freenode.net 6667 nullbot #dydx

That creates an instance of the bot named ‘nullbot’ in #dydx on irc.freenode.net::6667, easy enough right?

At the time of this writing, he only has 3 user commands. Here’s an overview:

00:40 < dydx> !id
00:40 < nullbot> I am nullbot, and  I was coded by dydx!
00:40 < dydx> !time
00:40 < nullbot> The date and time are: Tue Sep 16 15:40:22 EDT 2008
00:40 < dydx> !exit
00:40 < nullbot> Mah massah sed I gotz'a go..
00:40 -!- nullbot [n=nullbot@adsl-220-69-19.ags.bellsouth.net] has quit [Client Quit]

Given the nature of how I coded it, adding new features takes no time at all. It’s just a matter of writing a regex to match your trigger and then adding some functionality for that trigger.

For instance, if I wanted it to reply every time someone said “hey everyone”, you could do this fairly simply by first writing out the regex:

Pattern heyRegex   = Pattern.compile( "hey everyone", Pattern.CASE_INSENSITIVE );

Then, in the homeostasis() method, just run a matcher and implement some functionality:

Matcher hey = heyRegex.matcher( line );
if( hey.find() )
{
    this.message( "Hey, what's up?" );
}

And the end product is some pseudo-interaction with the users:

00:55 < dydx> hey everyone
00:55 < nullbot> Hey, what's up?

There are a few conventions that ought to be followed when adding a feature. I try to keep everything as functions, so a lot of the code is reusable. As well, there are a few variables, like ‘line’ which is mysteriously used to match against. Just keep in mind how the system works.

I have the JavaDoc for JavaBot online here, and you can download the latest build here.

Stay tuned for updates and news on this little project of mine.

It’s been a bit more than a month since my last post, this is partly because I’ve been on vacation. Now that I’m back, however, lets get the party started again.

While I was gone, I took some time to get acquainted with some ideas that I had previously disregarded as being “too difficult to learn” or “stupid”. After taking the time to do this, I can’t for the life of me see how I could live without it!

Before I left I downloaded the latest copy of Processing, a hobby language developed in the MIT Media Lab. It’s namely used to teach programming and visualization to arts students at universities, and as a toy for whomever takes the time to learn it. Along with the software I got Processing: A Programming Handbook for Visual Designers and Artiststhe quintessential manual to go with Processing.

After reading it cover to cover multiple times and playing with every subject it covered, I got to thinking. Processing is basically a dumbed down Java API, so how much harder could Java itself be?

Previously I had regarded Java as immensely hard and something only college students taking Computer Science learned. My previous work with PHP and C++, combined with my meager self-training with Processing helped me pick Java up in a few days. Since I already understood the utter essentials of programming ( commenting, variables, conditionals, loops, etc ), I took some time to dig right into the meat of Java and start working on some projects.

The first such thing was trying to write Processing code in NetBeans. Normally you would use the PDE (Processing Development Environment) to work on Processing apps, but the preprocessor is limited in what it can do. A way to overcome that is to use a standard IDE and write your code that way. Loading the Processing core and extending PApplet is basically all there is to getting that to work. There are a few differences, but nothing so outrageous to make you quit. I think the hardest thing I came to find was colors had to be referenced in their true hex value.

Right now I’m working on some networking applications for a security website. The first one is a private chatting client with encryption, and the other is an IRC bot that can interface between multiple channels, even on different servers!

Excerpt from JavaBot IRC Bot project

One of the other things I got to know and love were Regular Expressions. When I was working with PHP, I would create elaborate functions to parse out text in various ways, and to filter for certain things. I never used Regex for any of it. Now, I couldn’t imagine doing that without Regex.

When I was on vacation, I also picked up a copy of Mastering Regular Expressions, 2nd Edition, I read that cover to cover several times as well. When an untrained eye looks at a regular expression, they see it as incomprehensible line noise… I know it, I’ve been there myself. The way Jeff Friedl explains Regex, how they work, and how to implement them in situations… its unparalleled in anything I’ve read before. Now, after reading the book and having some first hand experience with Regex, I’m addicted to them! If I’m bored somewhere, I’ll think of Regex’s to match words I see (very good for doctors offices or classes at college).

A big part of my ongoing work in Java revolves around regular expression matching. I think I use them in just about every project I do these days.

After my recent reinstalling of OS’s, I had been noticing a definite lag in my system during even the most meager of tasks (simply opening Firefox, or listening to Amarok). I had attributed this to just the fact that my system was getting old.

Earlier I was just poking around in my menus and opened up the System Monitor (the equivalent of Task Manager in Windows). I looked at my Resources tab and was offstruck at the fact that, well, I had no Swap partition.

Your swap partition is a resource that your computer uses when RAM gets a little tight and needs more. So, rather than disrupting everything by reallocating memory, it just allocates some hard drive space (the swap partition) until it can allocate some RAM.

A real life-saver on systems with a small amount of RAM (though, my laptop/server has 512Mb, which should be plenty).

So, here I was, with no swap space. I vividly remember creating a swap partition, so I knew I physically had one, but what had happened to it was amiss.

I checked my /etc/fstab and saw that there _was_ an entry for it there. But, I couldn’t help but think that it was in error.

To check if it was in fact the correct listing, you can list all of your disks according to their UUID with the command:

$ ls -l /dev/disk/by-uuid

From there you can look for what the supposed Swap partition’s UUID is, and compare that to the one listed in /etc/fstab.

It just so happened that the volume was mislabeled in my /etc/fstab. The fix was easy, just open vi and edit it to be right & rebot = fixed swap mounting.

Easy enough. But, why had that happened?

The way I setup my system this past time, I had reinstalled the main OS (Ubuntu Desktop Edition) first, then installed the server OS (Ubuntu Server Edition). They both share the same swap partition.

When I installed my main OS it gave the swap a unique UUID, and when I installed the server OS it gave it _another_, different one.

My main OS’s /etc/fstab was still pointed to the old UUID, so of course it wasn’t going to mount properly.

Thanks to the folks in the Ubuntu Forums. Without their help I probably wouldn’t have figured out that was the issue.

I am a fan of Refactormycode.com, and as such I try to contribute when I can, in whatever way I can. I tend to stick in the PHP section, because well, I know PHP decently and I like seeing different methods for doing things.

Someone had posted some code on there the other day, and was wanting some advice on how he could better it.

Really interesting PHP class he made, very similar to what I was trying to accomplish the other day (I’ve since switched to CoughPHP). Just a small and flexible ORM to use with database driven websites, nothing too fancy.

One of the things he was wanting was some advice on his ‘disconnect()’ method, and I gave a shot at making a ‘better?’ one. In it, I handled the fact that there might not be a mysql connection link at all (for whatever reason, there might not be one, and there could be an error due to this). So had it Throw and new Exception if there was no database connection present, and then if there was one, to just close it. However, in my code, the original author noted my use of the Error Suppression Operator (@) with my mysql_disconnect().

I like to cover my bases. There are things that ’shouldn’t’ happen, but eventually do. It happens all of the time. So you really shouldn’t get an error on that function, but since its handling a MySQL connection, I always tend to clean those up and make sure that nothing really leaks due to errors.

Apparently, the original author doesn’t like this idea of using the @ in PHP, ever. To each his own, I love it. I’ve run some benchmarks on its usage in MySQL as well, and the results are interesting.

ishkur@craptop:/var/www$ php ./benchmark.php
Error Suppression Operator Benchmark Testing

Control Average (no @): 0.396842058897 seconds
Test Average (with @): 0.40044508481 seconds
Total Queries Run: 2000000
Total Time: 797.303076982 seconds
Queries Per Second (average): 2508.45639223
Total Records Fetched: 2000000000

There is a very slight difference in the time, but it’s barely apparent in most cases where you’d normally max out at 30-40 queries on a page load (WordPress has 33 I believe).

Other than a perceived time delay, I don’t understand why you wouldn’t use Error Suppression with MySQL queries. Error handling is great, Exceptions and try…catch blocks are wonderful at dealing with shortcomings in your code and dealing with what happens when things go wrong: but they cant do everything.

They don’t handle PHP’s extraneous errors like incorrect database login info, bad array indexes, etcetera. @ ftw!

Right now I’m working on a few private pet projects, nothing big; just some things to do for fun when I’m not busy working on other people’s code. Something that I think every programmer who is devoted to their hobby should do.

Among other things I am namely working on 3 projects: a Database Wrapper Class, a PHP based implementation of the Active Record pattern, and a CMS based off of both of those. It’s a very enjoyable and involving set of projects, but at times it feels like I’m doing it without need.

There are hundreds of premade database classes out there (as well as a few integrated extensions). There are numerous attempts at the Active Record pattern. And numbers cease to mean anything when trying to explain how many different content management systems are available.

So why am I contributing to this mess of code. Reworking old problems that have been solved, time and time again?

So I know how it works.

I can use an extension like MySQLi or PDO, or a premade library such as ADOdb all day, but understanding exactly how it works is amiss. Sure, they’re open-source and I can just read the code and see what it does, but that’s no fun.

I want to know why something works over something else, why they’re calling a variable by reference, why a method is protected instead of being private.

So, I’ll look at the code from those things, read papers on the theory behind them, and spend some time trying to make my own version.

I’ve been slightly criticized for the fact that I tend to redo everything instead of rely on a 3rd party library. Sometimes I don’t need the overhead that something like PEAR or CodeIgniter, or the extendability of MySQLi or PDO. I just need what works for what I’m doing.

So I guess that I’ve made a habit of reinstalling OS every few weeks. I tend to run a muck and break things, but this time it wasn’t entirely my fault. The rooting competition that I held this past weekend left my OS in shambles.

I’ve returned it to OEM as much as possible, and I’ve beefed up the security a little bit (added Firestarter Firewall, installed Chkrootkit, and configured my sysctl.conf). But for all of that, there are still some problems that are just persistent.

For all the money and fame in the world, I couldn’t begin to understand why it’s not wanting go online, or at the very least let me install new software.

So, I’ve decided to just go ahead and reinstall OS.

This time I’m making room for a secondary OS, Ubuntu Server Edition. That will allow me to (for the most part anyway), encapsulate most of the bad-nasty stuff to one partition.

I’m hoping that the people who will be participating in the rooting challenges can respect that. :\

It’s 2:00AM on a weeknight, I have to be up at 7 for work: I’m tired and tomorrow is going to suck.

There is some light on the situation, however. After many hours of grief, my laptop is fixed. It took a lot of chatting in IRC, on AIM, emails, forums, and a rather large number of Google searches: but my computer is back the way it should be.

So what was wrong? Well, I dont entirely know. Somewhere during my forced removal of VMWare I messed up some system files. Luckily, there are people in this world that are smarter than me, and a whole lot more patient than me. So lets get into how this was rectified.

First, since my computer wasn’t getting past the login screen (all I was getting was an orange screen and a grey box in the upper left corner), I assumed it was an issue with the X Server. I was at least somewhat right.

VMware had entered some files and configurations for its own purposes into my xorg configuration, and by removing them forcefully, I messed that up. I dont have a backup of my /etc/X11/xorg.conf file (though, if my past has taught me anything, it’s to backup these little pesky things). So, to work around that, I was told that I could boot a LiveCD of Ubuntu, then just cp the xorg.conf from that directly into place of my old one

$ mkdir ~/OS
$ sudo mount /dev/sda2 ~/OS
$ sudo mv ~/OS/etc/X11/xorg.conf ~/OS/etc/X11/xorg.conf.old
$ sudo cp /etc/X11/xorg.conf ~/OS/etc/X11/xorg.conf
$ sudo shutdown -r now

So, I tried that, still nothing. At this point I was sort of gringing at the thought of possibly having to reinstall OS (sometimes it’s just easier than trying to unbreak things). After some talking in IRC, the idea of running the uninstall script for VMWare might be a good idea (if I had known there was one, I would have ran it in the first place). Apparently, during the installation VMware placed a file called ‘vmware-uninstall.pl’ in the /usr/bin directory. So in essence, all I should have had to do was run:

$ sudo vmware-uninstall.pl

But, now, that was not an option. I had deleted every file with even the slightest mention of VMware in it, including this uninstall script.

Later it was determined that I might could just reinstall VMWare temporarilly and see if the computer will try and boot.

The idea, of going through the hassle of redownloading, re-unpacking, and reinstalling this software made my skin crawl… but, some things just need to be done. I went and wget’ed a fresh, shiny new copy of VMWare Server Edition 1.0.6 from the VMWare website. Hooray.

I do have one positive thing to say to the folks at VMWare, your files are incredibly fast downloading. I averaged 730Kb/s on both of my downloads for VMWare Server. Thats just, unheard of in comparison to what I am used to.

So, after my download completed (maybe a minute and a half for 200Mb of stuff), I unpacked it, and began my reinstall.

5 or so minutes later, the reinstall was complete and I am off on my way to reboot and try this again.

To my astonishment, it does the same thing it has been doing for the past several hours: the aforementioned orange screen and grey box. I sit in my spinny office chair and sulk. Nothing seems to be working. I need this machine running as soon as possible and it just is not wanting to cooperate.

I did have another machine that was working, my recently worked on desktop. So, I go to check my email, read some RSS stuff, check out Xkcd. I try to completely put this laptop issue out of my head. I start thinking about what got me there in the first place. Originally, I was going to unstall a virtual machine for a rooting challenge for enigmagroup.org to try and hack. In it, was going to be a vulnerable version of PHP, and running on that was going to be a CMS with a gaping security hole, screaming out “EXPLOIT ME PLEASE!”.

Nine hours after that initial idea, I was sitting alone in a dark room full of computer equipment trying to salvage a fubar’d laptop.

Thinking “screw it”, I left the room and got some water. Being devoid of liquid for that long can give you a headache, and I could feel one coming one: ‘nip it in the bud’.

When I come back into the room, I notice a rather interesting thing. I see my wallpaper up on the laptop. And plastered all over the wallpaper are folders and files. My folders and files. And on the top and bottom of the wallpaper are the Gnome panels.

My laptop was fixed.

How? I still don’t know. Maybe it just needed a second to simmer in its juices. Maybe it was sick of looking at me and fixed itself in my absense. The votes are open on exactly what happened, but I do know that it feels like it did before, and that I typed this up on my laptop.

Tonight wasn’t so bad after all.