Lewies Blog

Last month I had looked into tools to notify me when any of the databases at work went off-line. I primarily looked into a beta of SQL Response by Red-Gate since I would automagically own it when it would finally be released. The SQL Toolbelt bundle that I have would include the software. I found many problems and made recommendations. After the beta was over, the Red-Gate folks contacted me and asked for my address so that they could send me a T-Shirt. Today I came home to see a 6″ cubed red cardboard box. Inside was plenty of shredded documents and a shirt (odd packaging for a shirt). I took out the shirt and discovered that it was wrapped around a coffee/tea mug - which within contained a pen, some mints, and a 1 GB USB thumb drive. I gave the drive to my wife since she loves the color red. Originally shocked that they would give me a t-shirt, I was simply impressed that they threw in all the other goodies.

As I use this little utility that I made on each computer system, I find the need to add more features, handle specific scenarios, or simply fix bugs. The software is actually becoming very stable and flexible. The latest changes to it are:

• Add the ability to cancel what it is doing.
• Disable “Copy” button, and make all text fields read-only.
• Show a list of stats (File total/copied/skipped, folder total/copied/skipped, error count)
• Add error handling on everything.
• Remove stack-trace in output since it doesn’t really help with anything.
• Provide ability to start half-way through the job.
• Give status messages of what is going on.
• Only list file name instead of full path with each file.
• List the folder path above the file.
• Change the layout to be more friendly with very long file/folder names.
• Change how files are retrieved/compared from destination to get list all at once rather than individually.

I’m debating if I should sell this thing for a few bucks. It is a useful utility that has probably saved me hours of head-ache with normal windows copy/paste operations over the network. I admit that I’ve been copying hundred of Gigabytes, but perhaps someone else would find it useful as well.

One interesting thing that I saw is that regardless of connecting through the Wirless or wired LAN, the destination seems a bit slow. Not by much I guess. I’ll need to find a way to measure the transfer speed between each computer and the NAS to be certain.

After everything is copied to the NAS, the next step will be organization. Part of this would involve comparing files to find duplicate information. I am sure that another utility will come about from this adventure as well. I guess I should start looking into making a website of some sort to make these utilities available.

In the game of tic-tac-toe, a single row has many different possible states. Each cell can either have X, O, or remain blank. It is a trinary number system. The number of unique states in one row is 3 to the power of 3 (3*3*3), or simply 27.

I am looking into this because I am interested in making a game of tic-tac-toe in an environment where image size and count mean everything. The less data that is needed to be transferred, the better. I could create an image for the entire grid, but that would result in 3 to the power of 9 images (19,683 images). I could dumb it down to just 3 images of an X, O, or empty spot, but the format of what is available would require me to use more resources to display 9 separate images at one time. The optimal method to use limited resources is to display 3 rows as 3 separate images.

I could go ahead and create 27 different images to load up in each row. However, the key idea here is the use less data that is needed to be transferred. My take on it is that I’ll use the same image for each row, but only show part of that image. So if I have an image that says “XXOXOXOXX”, and I want to show “XOX”, then I’ll stretch the image and center it on the “XOX” so that all of the other characters are outside of the visible area.

This is good, but I had one last thing to focus on. 27 different images all streamed together would result in an image with 81 characters. Surely their is a lot of repetitive characters? Take my example: “XXOXOXOXX”. You’ll notice that “XOX” is repeated three times, and “OXO” is repeated twice. I could try eye-balling it to cut it down to size, but we have computers for that sort of thing.

I started generating a random string of ” “, X, and O with 81 characters. As soon as I found a string that suited all possabilities that a row would have, I would generate another random string with 1 less character. It took much longer as I tried to generate less characters. I could only get down to 29 characters “O-OOX-O—X-XXX–OXO-XOXXOOO-”. This was very interesting to me because I was starting to see a significant pattern here. 29 is very close to 27, and as I noted earlier, the total number of possible combinations is … 27.

Doing the same for 2^2 is something I could do in my head. Possible combinations are 00, 01, 10, and 11. I came up with 10011. Only off by one (2^2 = 4), but if I wrap the text to repeat itself, I get “1001″ - 4 characters! Note “11″ is “wrapped” with the two ends. Now, with that knowledge in hand, look at the pattern that I had with 29 characters. Notice that the last 2 characters matches the first 2. From here, I can wrap the text around to reduce 2 additional characters resulting in 27! My final pattern was “O-OOX-O—X-XXX–OXO-XOXXOO”.

This reduces the total size of my image by almost 67%. The “wrapping” effect already happens with images in the environment that I work with and is known as “tiling”. If an image does not cover an entire area, the same image is placed next to itself, essentially causing a tiling effect over the whole area.

The next steps are to actually make the image, detect where a person has touched the game, and map the image to reflect the actual state of the game in each row.

At one point, I did try creating a string of values and incrementing them as a trinary number (—, –O, –X, -O-, -OO, -OX), however this actually took a lot of time to do. After letting it run for a couple hours with no results, I abandoned it all together and went for a randomized “fuzzy” approach. Randomly, I was finding different combinations of 29 characters each in about 2 minutes each. The lack of speed may have been because I was dealing with strings rather then an array of numbers. I’ll look into different possabilities next time.

Sometimes, I surpass my own geneouse.  Something got done today that was an amazing solution to a difficult problem regarding cyclic redundancy.  The side effect of this solution has sereouse potential to solve some very aged and hard problems that have been left hidden away in a corner.

All of this has to do with the implementation of Http Modules within ASP.Net.  With these special modules, you can catch the process of a web page before the web server processes it in its own normal way.  The power of them is so strong once they are understood that you can make all kinds of tweaks that were never possible before.

So today, I solved an immediate problem, impressed the manager on how I solved it, and collaborated on how we could use the method to solve some past feature requests.  When I left for home, I already had a working solution up and running on my local machine in the office.  What a day.

I’ve been tasked to learn ColdFusion this weekend. If you see any web pages on the internet where the name ends with CFM, those are ColdFusion web pages. I don’t have any software, and a server wasn’t provided for me to work with. I was a bit confused on how I was supposed to learn the stuff. We have a new guy comming in on Tuesday and I need to know ColdFusion by then so I can help him get up to speed.

I did some searching on the internet. I found a product called BlueDragon that can run web pages with ColdFusion tags in them (ColdFusion Markup Language - CFML). It can run on its own on port 8080 by default, and you can also map it to Microsoft’s Internet Information Server (IIS). The best part of the deal is that this server is free.

I found a few websites with ColdFusion tutorials. I also found that Macromedia has a website that I could use as a reference for everything CFML. I always looked down at ColdFusion as if it were almost a joke. When HomeSite started supporting ColdFusion tags, I was busy learning Active Server Pages (ASP). Thank goodness for that, as ASP has caught on greatly compared to ColdFusion.

It seems that ColdFusion has finally caught up with me. The good news is that I already know so many programming, scripting, and markup languages. This is making it easier to catch onto CFML.

I created an account for myself over at SourceForge. This is a website that many people use to provide open-source software on. Many services are available such as defect tracking, forums, tasks, screenshots, and even some source-control using CVS.

The whole reason I’m looking into SourceForge is that I want to go public with my Automagic Podcaster software soon that I use with Dreamy Audio. I’m looking into a couple of things before I do it. Besides places to host the files and collaborate with others, I also need to look into licensing.

Although I wish to distribute the code as free open-source software, I also need to protect it. I want to allow people to modify it, but I don’t want them to package it up and resell it. I also want credit where credit is due. I also don’t want people to redistribute there modified version without stating what they have modified from the version that they received.

There are many licenses out there. I just have to find the rite one that matches my intent.

I’ve started to have a look at the tags in other MP3 sources other then the files that I make myself for DreamyAudio. The purpose is to test the compatability of my software with MP3 files. It’s like throwing a monkey wrench into a running motor. I started off with DarkCompass and found a few bugs that I had fixed quickly.

During this time, I stumbled upon four ID3v2.3+ tags that I hadn’t seen before (PCST, TDES, TGID) in the DarkCompass podcast. I haven’t found any documentation on these tags when googling them, but from the content within, they appear to be related specifically to podcasts. I came upon similar tags (TID, TDS, WFD, TDR) in another podcast that was using ID3v2.2 tags. Thankfully, the 2.2 tags were documented on many sites on the internet.

One thing that grabbed my attention was the content withing the tags of an old podcast that I grabbed off of Odeo. It actually had the URL of my feed in it. I started thinking a bit about this and realized that if I redistributed the MP3 file to friends and such, that eventually it could be traced back to me. It seems that Odeo had dynamically threw in its own ID3 tags that identify my account.

Perhaps in the future, ID3 tags (or any embedded metadata) could become a privacy concern. Imagine if more sensitive data was stored within those tags such as email addresses, phone numbers, or any sort of contact information without your knowledge and you let your friends get a copy and listen to it. Further down the line after friends sending friends sending their friends the MP3 file, someone gets a hold of it who knows how to read all of that data. Now a stranger has your personal data and could sell it or make use of it in malicouse ways.

Ok, so I’m really going over the edge here and blowing a little thing up to infinite proportions. Who cares if the alien Captian Zorgon from the planet TH-87w discovers our planet and designates you as the offender of the careless audio. What if he makes use of a tool specifically crafted for him to carry out his will of removing any source of that aweful sound and targets you just because your email address was in a small tag in the file.

Back to reality. It’s just a little thing really. I’m just a bit unsettled that someone had inserted some information with my account name as part of a url feed into the ID3 tags without my knowing.  Still, maybe it isn’t Odeo that did it.  Maybe iTunes inserted this data in there behind my back when it downloaded the MP3 file from the feed.  If that is the case, iTunes is the culprit for exposing that extra information.

I’ve been having a go at improving the accuracy and format of my DreamyAudio podcast. The accuracy being the part where I read information embedded within the MP3 files.

I decided to challenge myself and display the playtime for a mp3 file encoded with a variable bitrate (VBR). It was tough just trying to detect that the file had a VBR. It’s actually simple now that I figured it out.  You need to detect if an Xing header is present in the audio data a few bytes after the MPEG header.  The tricky part is figuring out what position those bytes are at in the file.
After figuring out if the file has a VBR, I noticed that the estimated playtime is out of wack. It’s based on a constant bitrate (CBR). So now I’m plagued with a problem.

I assume that I could go ahead and read through the individual audio frames and tally the bitrates from each one and divide it by the number of frames found. The thing is, there are thousands of frames. Perhaps I can find a way to read the first few hundred and give an estimate based on the number of bytes remaining.
Bah. No time for it today.

I was looking into a way to cache my generated pages on the server with PHP. I couldn’t find anything to do it on the server-side, but I did find something to prevent a client from asking for the page via client-side caching.

When a web browser requests a page for the first time, it will save a copy into its local cache. If the person visits the web page a second time, the browser will send the date and time when the page in the local cache was originally retrieved. The server checks the date and responds with either a new page, or a message stating that the page hasn’t been modified since the last time the request was made.

This helps out servers to decrease how much bandwidth is being sent accross the internet. Some of us are charged extra if we go over so many gigabytes per month. It also helps my server from doing unnecessary processing of scripts.

I was working out a way to take care of this. PHP has a method to access the client headers individually called getallheaders(). The problem is that this only works on apache web servers. I’m testing things here at home on an Internet Information Server (iis). Although my website is hosted on apache (I think), I still would rather have support for multiple platforms.

I continued to run through some problems until I found a PHP implimation of conditional get. It was pretty easy to impliment and it worked immediately. The method expects a parameter of a date to compare. I read the date that an MP3 file was last modified and passed it in there. The method did the rest of the work telling the web browser that the information had not changed.

I didn’t realize that this term, “Conditional Get”, was coined for this pattern. The PHP implementation referred to a very good article about it called HTTP Conditional Get for RSS Hackers. It goes into detail about the whole process of it all. It’s amazing how much detail you can get into that amounts to just a few lines of code.

Tomorrow I’ll look more into caching data on the server. I found a method (md5_file()) to get a hash of a file that I can use to see if the files contents have changed since the last time I created cached xml documents and images referring to its contents. Reading all of that data from MP3 files constantly could perhaps have a toll on the servers performance that I want to avoid.

Since my interest in recording my dreams has been renewed, I’ve started fixing up the code to my audio blog. I mainly call it “Automagic Podcaster” due to the fact that all the end-user has to do is upload MP3 files onto there site. The code reads all the meta data about the MP3 files that have ID3 tags embedded within them.

Some of the MP3 tagging software that I’ve been using lack features such as Unicode support and chapters. I tried doing translations of my dreams in Japanese, Arabic, Chinease, and Portuguise and saving them as lyrics. Most other languages that I translated using google had problems with some characters. I soon fixed that by using html entities to represent the special characters. I sent an email off to the author of a particular piece of software. I’m hoping that maybe he is already working on somethng similar for the next version of his software.

Anyhow, I’m thinking that soon I may just package everything up, clean the code a little, and look into posting it as open source software. I don’t know what all is involved. I’m using a flash applet to read and play XSPF feeds on my pages to play the mp3 files, so I need to contact the author to see if I can distribute it along with my code and what additional requirements there may be.