The Monty Hall Problem Hurt My Pride

Have you ever had one of those days when you swore up and down that you knew in your gut that something was right, but the world was against you, well I just had one of those days.  One of my work buds brought up the movie "21" about these punk kids from MIT who decided to try to bring the "man" down in casinos, or something like that (never saw it). Anyway this movie goes on to explain (horribly) the "Monty Hall Problem".

What's the "Monty Hall Problem"?  I'm glad you asked, but I sure wasn't glad I asked that fateful day late December back in '63.  tl;dr:  there was this game show host named Monty Hall who used to have a 3 door game where a contestant picked a door to win a prize.  One of the 3 doors had a prize (car) and the other two had goats or some other junk. [side note] I would have been very happy to win a pet goat [/side note].  As soon as the contestant picked a door Monty would then take away one of the two other doors knowing full well that the door he was removing was the losing door.  The contestant was then asked if he/she wants to switch.

The movie tries to make us believe that the contestant should ALWAYS swap doors because it will give better odds (66%).  So I said that is total Barney Snicker-doodles.   There are only 2 doors left so your odds of actually winning the prize is 50/50 :duh:. So we had this long winded debate and even resorted to watching a Kahn Academy video.  Statistics, blah blah, statistics.

So even after watching this my gut just wouldn't let me believe it.  So being a slightly able to reason out a problem and program JavaScript, I took to the keyboard to write a program that would simulate 10,000 times a user first picking the door and STICKING with that door and then a person picking a door and SWITCHING.  My hypothesis was that no matter which option was chosen the user would have a 33% chance of winning (1/3 doors).  Well my own fingers don't lie so I wrote it up and it turns out that switching DOES give you an extra 33% possibility.

No Switch Wins count = 3316 = 33.2%
Switch Wins count = 6650 = 66.5%

Every time I ran the program it was very close to these results. Click here for the source of my dishonor (JavaScript).  I'm so unhappy at this point that I will never clean up the code and it will forever be sloppy.

Most importantly I have to state the following publicly:

You were right Frank.

OMG tmux how have I ever lived without you!

How the **** I have I never heard of/used "tmux" on a Linux box?  I mean seriously, they had me at Control + B --> D.  For those like me who had never heard of "tmux" it is a NECESSITY if you ever run any long running (or even short running) scripts that require you to be connected to a SSH session.

Let's pretend that you are upgrading a very important application on your server and this upgrade script does thousands of instructions and database queries etc etc.  Now let's pretend that you are 4 hours in and all of a sudden a squirrel nibbles the fiber optic cable connecting you to the interwebs (yes, this happened).  All of a sudden all of the processes that are running have been terminated because your SSH session was disconnected.  Now you have to pray that it either finished, somehow is still running, and didn't corrupt itself and you have to roll back to a snapshot (you did make a snapshot before you started right?).

Now let's pretend you knew about "tmux" and let's run though this situation again.  You first SSH into your server and then you immediately issue the command:
tmux
Now you get a nifty colored bar at the bottom of your SSH screen.  Mine is green, but it will probably depend on your terminal settings.  You seem to get a standard prompt like you had before.  That green bar is the only thing that really lets you know you are in tmux.  Now you issue your script commands.  4 hours in the rascally squirrel gets his teeth into your cables.  After flipping out and finally realizing what happened you then proceed to go to your mom's house for some nice good old fashioned cable internet and cookies.

From there you connect to your server (VPN + SSH) and then you issue the command (as the SAME user that you ran tmux from before):
tmux attach

Now you are brought right back where you were!!! HAPPINESS!!!

Useless example:
$ tmux
Now in tmux:
$ nano hello.txt
If you purposely want to disconnect hold down the CONTROL key and press "b" then press "d" to "detach"
Close your SSH connection completely or go to another computer and connect to your server.  Now as the same user type:
$ tmux attach
Violoa! you now are editing hello.txt!
To exit from tmux you can just type "exit" at the prompt.  Here is a nifty site dedicated to more tmux use cases like splitting screens etc: https://danielmiessler.com/study/tmux/

Happy tmux-ing!

Why I Hate/Love Free Support - I'm looking at you Piwik!

Who doesn't like free? Of course as the age old adage goes, you get what you pay for and Piwik is no different. Piwik is an analytical platform for log analysis and real time (semi) user tracking via web service calls. All in all the product is decent. We are running a local version of their software and performing log analysis only.

We (meaning me) ran into problems during a few different stages of implementation. I posted 3 different times to their forum with only crickets as a response. One topic was viewed 138 times and the others around 50ish. Not a lot, but hey someone's looking. There weren't even any responses from that one jerk who says, "I have no idea, man" or the other jerk that says "just RTFM" or the jerk that says "I'm not going to do your schoolwork for you" or the last jerk that is obviously from the java forums who says "if you are too stupid to figure it out I can't help you" (you know who you are!). 

There is one guy, Matthieu Aubry (the creator of Piwik) who does make great efforts to answer questions, but he appears to be just one guy (or ARE YOU?!?!?!). I do not fault him in any way and as a developer who also supports users I understand the lack of time he has to do both.

Piwik does have paid support contracts that will solve the world's problems, but it is rather expensive. For the little guy you are SOL. Piwik has a stackoverflow section, but that too is sparsely answered. In the end, I spent way too many hours troubleshooting and trying to figure things out, but hey you get what you pay for.

There's an Nginx on my Pi and that's how I like it!

The Raspberry Pi is just awesome.  It is perhaps one of the best CHEAP well made computers out there, especially with the model 3 available now.  However there is one HUGE problem... what the heck am I going to do with this thing once the it loses its shimmer and shine (genies divine - yea you know what I'm talking about parents)?  There are so many cool projects out there that can be done like the beloved magic mirror project or perhaps one of most useless, but technically awesome BeetBox. What is a guy to do.  I have two Pis (Pies?, Pii?, Pi's?) one is a 2 and one a 3.  I'm reserving my 3 for Retro Pi (perhaps a future post on this), but my 2 is just hanging out collecting dust.  Well I did what every devops kind of guy would do - run my own website on it!

This is the story of how the little engine that I thought couldn't actually could. Raspberry Pi 2 + Nginx 1.10.2 + PHP 7 + MariaDB 10.0.29 (screw you Oracle for corrupting everything you touch ::cough:: Java,  Solaris, MySQL, Oracle licensing in the cloud ::cough:: ).  Spoiler alert - on my small little site, this is REALLY fast.  It will never be blade server fast, but response time is mostly < 1s on static pages.  On a few php it was < 2s on fresh caches.  13ms for a static html page, 56ms for a 42k image, 248ms for a small php file.

First off make sure your Pi is up to date:

sudo apt-get update
sudo apt-get upgrade

Now make sure your Pi firmware is up to date and reboot:

sudo rpi-update
sudo reboot

Once you are up and running let's get MariaDB and client installed:

sudo apt-get install mariadb-server mariadb-client

Now secure it - PLEASE do this even if you are just futzing around.  Get used to security all the time.

mysql_secure_installation

Now comes a tricky part.  In order to install PHP 7 and the latest Nginx you will need to add the "stretch" repository.  Edit the file:

/etc/apt/sources.list
Add the line:
deb http://mirrordirector.raspbian.org/raspbian/ stretch main contrib non-free rpi
Make sure that line is just below the first line for Jesse.  It should look something like this:
deb http://mirrordirector.raspbian.org/raspbian/ jessie main contrib non-free rpi
deb http://mirrordirector.raspbian.org/raspbian/ stretch main contrib non-free rpi

Now run:

sudo apt-get update
sudo apt-get install -t stretch php7.0 php7.0-curl php7.0-gd php7.0-fpm php7.0-cli php7.0-opcache php7.0-mbstring php7.0-xml php7.0-zip
Note the "-t stretch".  If you have multiple repositories this will force a specific one.  Good practice to use anyway.

If all goes well let's now run the installer for Nginx:

sudo apt-get install -t stretch nginx

On one particular site I can't find now, a person recommended changing the user/group to pi:pi for Nginx. It wasn't explained, but I'm guessing it was so Nginx does run as a privileged account (although you still need root to start it up with a low port 80/443).

sudo nano /etc/php/7.0/fpm/pool.d/www.conf

Change:

user = pi
group = pi

My site has a top level www.domain.com as well as a few subdomains a.domain.com and b.domain.com.  To set this up you need to add a config for each.  First off is creating your domain configs.  Go to:

/etc/nginx/sites-available/

Now add in your domain file:

sudo nano domain.com

After you add your goodness, now go to:

/etc/nginx/sites-enabled
Remove the entry for default and then add a link to your new one:
sudo ln -s ../etc/nginx/sites-available/domain.com

Restart Nginx

sudo service nginx restart

Restart PHP FPM:

sudo service php7.0-fpm restart

Now what doesn't help anyone with a custom web site is DHCP on your local network.  To fix this, let's force the Pi to a static ip:

sudo nano /etc/dhcpcd.conf

At the end of the file add your information for your static local ip (leave the '/24' in the first line):

interface wlan0
static ip_address=192.168.1.200/24
static routers=192.168.1.100
static domain_name_servers=192.168.1.100

You read that correctly, I'm using my wireless connection (wlan0) for my web server MWAHAHAHAHAHA.  It should be noted that most hosting providers will BLOCK port 80 and 443 from even touching your internal network.  You may need to call them up to ask them to open it.  In my case I had to call them, but as long as you have a minimum internet speed package they will be so kind as to open it up for you.  Also you will need to forward traffic on port 80/443 from your router to your Pi (or other custom port if you desire).  If you set a static ip on your Pi I recommend either setting that same device to receive that static ip on the router side or exclude that ip from being leased by another device.  Call me Nostradamus, I've been predicting lease conflict problems for 500 years now.

Next up is something that came about due to cheapness, FREE SSL.  What's that I say 'FREE SSL' that can't be.  There is no way GoDaddy, Thawte or any of the other ridiculously priced SSL crooks Certificate Authorities would allow that.  Well they did - it is an awesome Linux Foundation Collaborative Project called Let's Encrypt: https://letsencrypt.org/.  This is a new (relatively) Certificate Authority that works on the premise that you have to renew your certificate every 90 days.  Sure that sucks, but when you save $50 a year, I can set up a small cron job that will automatically renew my certificate for me.  There are limitations like the site must have a public DNS entry (sorry, local intranet domains need not apply), IP's that are public can not be signed, 90 day expiration, no wildcard certs.  However with the wild card option, you can have up to 100 names on the same certificate (not really sure how that works) and you can also apply for multiple domains.  They cap it at 20 domain requests a week (https://letsencrypt.org/docs/rate-limits/).  This is really for Linux boxes, but you can get it to work on Windows servers with some extra work.

I followed this guide partially for the Let's Encrypt and Nginx stuff in case you are wondering:
http://www.htpcguides.com/secure-nginx-reverse-proxy-with-lets-encrypt-on-ubuntu-16-04-lts/

In the case of my Pi, I had to first install git so I could clone the repository (yes you can just download the zip and go from there, but where's the fun in that)?

sudo apt-get install git -y
sudo git clone https://github.com/certbot/certbot /home/pi/certbot

Now go into your certbot directory to issue some commands to "install" certbot-auto which does the heavy lifting:

cd /home/pi/certbot
./certbot-auto certonly -a webroot --webroot-path=/var/www/domain.com -d www.domain.com -d domain.com

The first time this will ask you for your email address.  Add a legit email in case you need to do something with your certs later (retrieve them etc).  Notice that I have two domains with two "-d" options.  If you want to secure both www.domain.com and just domain.com enter them this way.  Now it will support either.  You can issue individual certs if you like for subdomains in just the same manner like this:

./certbot-auto certonly -a webroot --webroot-path=/var/www/a.domain.com -d a.domain.com

Notice that I pointed the webroot to the actual web root of the configured site in Nginx?  This is necessary because the certbot program puts a file in there when it goes to validate your web site for issuing the cert. Once this completes it will tell you where the certs/pem that you need are located.  You can choose to leave them there (recommended) or move them.  Keep in mind that if you move the certs, when 90 days rolls around and the cert is renewed automatically you will need to then move the cert to the new location either via a script you write or manually.

Now you need to configure Nginx to play nicely with the cert. Edit the site of your choosing:

sudo nano /etc/nginx/sites-available/domain.com

Add the following to the "Server" section:

    listen 443 ssl;
    ssl_certificate /etc/letsencrypt/live/domain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/domain.com/privkey.pem;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;
    ssl_dhparam /etc/ssl/certs/dhparam.pem;

There is a lot of stuff in there, let's take it line by line.
listen 443 ssl;  - This says respond to web requests on port 443 (default ssl port)
 ssl_certificate /etc/letsencrypt/live/domain.com/fullchain.pem; - this is the location of the PEM full certificate chain that Let's Encrypt generates for you.
ssl_certificate_key /etc/letsencrypt/live/domain.com/privkey.pem; - the location of the private key that Let's Encrypt generated for you.  Necessary for SSL encryption to work
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; - This limits what SSL protocols are served and accepted by the web server.  As protocols get hacked this will need to get updated.
ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH'; - this is the latest list that I found of secure encryption ciphers that WILL NOT work on older browser like IE 6.
ssl_prefer_server_ciphers on; - This tells clients that if you are not running my ciphers, you can't connect - LOSER
ssl_session_cache shared:SSL:10m; - honestly don't know what this does.  Guessing performance caching for repeat clients.
ssl_dhparam /etc/ssl/certs/dhparam.pem; - This is the wizard that helps get the "A" SSL rating.  By default if you don't include this I got a "B" rating from Qualys SSL Labs (https://ssllabs.com/).  The reasoning is that Diffie Hellman algorithm is more secure when establishing connections.  The unfortunate downside to this is that you need to generate the DH pem file.  On my Pi 2 this took an hour, but I read in some places it took over 24 hours. It is random so it will take as long as it takes.

To generate that pem file:

cd /etc/ssl/certs/
sudo openssl dhparam -out dhparam.pem 4096

You can choose a 2048 bit key which will go MUCH faster, but that is up to you.  Now don't forget to set up a cron job (as root) to try to renew your SSL certs DAILY.  They recommend that you try to renew it twice a day in case their servers are not available.  There is no penalty for trying to renew a certificate and it doesn't count towards rate limiting.

sudo crontab -e

In the file add an entry like this:

0 1,13 * * * /home/pi/certbot/certbot-auto renew >> /var/log/letsencrypt-renew.log

Note: after a successful update you will need to restart Nginx.  You can just restart it during your maintenance window or just do a quick restart a few minutes after you try to renew your cert.  I elected for a restart at 2am every day. Just add it to your root crontab and you are all set:

0 2 * * * service nginx restart

Restart Nginx again and test to make sure you can hit your page via SSL.  Finger crossed.  If you can, now check your rating on https://ssllabs.com.  If you are bold let them post your results on their leader board.  I'm not that bold.  Right now I have an "A" rating and if any new fun SSL bugs come out, I'll have to update my cipher list.  Welcome to DevOps!

Raspberry Pi Prompting User for Lighting an LED

Project 1 - Prompting a user to light up an LED

Welcome back kinder!  Today I spend a few hours playing with the GPIO pins on the Raspberry Pi 2.  I really didn't know what to expect, so I bought a kit from Sunfounder.  The instructions as everyone else mentioned stink so I just tossed them aside and googled everything.  The kit itself is good with the exception of their labeling...there is none.  I had to use my multimeter to figure out which resistors where of which ohmage... ohmeratude... ohmwhatever.  Yes the resistors do have the little color coded bands on them, but when I went to check them, it didn't match various web sites and color keys.  I suspect either the colors on the bands are off, or I just can't tell the difference between their black and brown.  Other than that so far so good!

Step 1, figure out the resistors to use

Why is using a resistor important you ask?  I didn't know the answer either, so I googled it and apparently if you don't add resistors in front of LEDs then the LED will try to draw as much current from the Pi as possible, therefor potentially burning it out. For my first test I went with the 1k ohm resistors.  Why that amount, I have no idea, but the very project page in the manual I tossed was using that so I thought it was safe.  Apparently If you use a higher ohm resistor it will RESIST more voltage than a lower ohm resistor.  So it makes it safer it seems... [note to self, look into this later, hopefully without burning out the Pi first]

Step 2, first test with a breadboard and GPIO

What's a breadboard you say? Basically it is a circuit board that doesn't require soldering and is great for tinkerers like us.  I initially went the quick and dirty route and wired up the led to the breadboard and the LED longer leg (the anode aka the positive lead) directly to the 3.3v pin on the GPIO (pin 1) and the LED shorter leg connects to the 1k ohm resistor which connects to GPIO GND (ground pin 6).  As UNexpected, it worked on the first shot (go me!).

PS - sorry for no pictures, I didn't take any on the first round.

Step 3, turning an LED on and off

This part was a little tricky at first. The important take away was that in Step 2 you change the POSITIVE lead from pin 1 (3.3v) to pin 11 (aka GPIO17).  OK small aside here.  When programming with the Pi and interacting with the GPIO board you have 2 ways of referencing the pins on the Pi.  

1. Using the actual pin number (1,2,3,4,5...).  In python you reference this by using:
   GPIO.setmode(GPIO.BOARD)
   GPIO.setup(11, GPIO.OUT)
2. Using the pin BCM name (Broadcom SOC channel)
   GPIO.setmode(GPIO.BCM)
   GPIO.setup(17, GPIO.OUT)

More information from stackoverflow.

Now comes the fun part, using python to turn the LED on and off.  Through some googleing and self learning, but first a little rant: 

[rant] I came to the realization that I don't much care for Python as a language. I knew nothing about Python coming in and I guess this is to be expected since I'm coming from Java mostly.  I hate the tabbing structure of everything from functions, to if blocks to try blocks.  I miss curly brackets so.  Next up is the typeless language structure.  I found that when I was prompting someone for input and I wanted to make sure it was a number, I had to either check the type or try/catch the input and the force a casting like this: int(somevar).  I would prefer to declare my variable types ahead of time so that the runtime would throw errors properly.  Next up is "elif"...really?  Could type out "else if".  What is this perl?  Finally back to tabbing.  Seriously, it makes it quite the chore to know if you are still within a nested if block or the outer while loop etc.  Anyway noob statements I know, I'll get used to it or try out the Java implementation.[/rant]

I'm attaching my first program here.
To run the program use the following command in the same directory that you uploaded the file:

sudo python3 p0-led.py

If you run into an issue where it won't run at all you most likely need to set it as executable:

sudo chmod a+x p0-led.py

Step 4, multiple LEDs!

This was a little trickier.  First off I did some reading and apparently you can share the ground between multiple LEDs.  Not sure how true this is, but it seems to work, but I still put a 1k ohm resistor in front of each LED just to be safe.  I lined the 3 LEDs in a row on the breadboard and ran all 3 resistors on the negative lead (short leg) of each LED to the global ground wire on the board.  Then I ran each positive lead on the LEDs to the following GPIO pins that accept input from the Pi (11, 12, 13) aka (GPIO17, GPIO18, GPIO27 - no idea why they skipped names).  Here is a picture of the breadboard and a bad picture of the Pi pins (sorry) with Pin 1 being the top right and Pin 2 being bottom right.  Click each image for a larger version.

Now comes the fun part again Python!  I really do mean it, it is fun!  So first off I learned something.  Those warning message you get about "RintimeWarning: This channel is already in use" were nagging me so I looked it up.  They are actually important.  The trick is properly closing out your code with:

GPIO.cleanup()

That releases the resources associated with those pins and resets them back to INPUT. Now that opened a can of worms and I must be a good programmer and catch  my errors and release connections etc.  It is overkill for this kind of project, but it was fun to learn.  The first thing I realized is that I need to learn how methods are defined in python.  Simply they work like this:

def YOUR FUNCTION(yourInputs)

Everything in the method is tabbed in and you can do standard return statements too.  Next up is learning how to prompt someone for input with the "input()" method!  Here is what my method looks like to prompt a user:

def promptLED():
print ("\n\nChoose LED to light up")
print ("1 = Green")
print ("2 = Red")
print ("3 = Yellow")
print ("4 = All")
print ("0 = exit")
return input("> ")

Now when you call the promptLED() method, it will prompt the user for an input and then returns the value of the input back to the main program.  From there I pass it to another method that inspects the input and then lights up the proper LED:

def exeChoice(userchoice):
choice = 0
try:
choice = int(userchoice)

if (choice == 1):
print ("[info] Lighting green LED")
GPIO.output(ledGreen, True)
time.sleep(1)
GPIO.output(ledGreen, False)

Ignore the try up there, it is caught later below.  For now the important part is the line that says:
choice = int(userchoice)
This line forces the choosing of a NUMBER and it throws an error if it gets anything but that - hence the try.  There is also an else in there to handle cases where it is not 1,2,3,4 or 0.

The next line of importance is:
time.sleep(1)
This simply pauses execution of everything for 1 second.  It can take decimals for shorter time.  NOTE: you have to import "time" at the top of your program:
import time

In exeChoice, I return a number ranging from -1 to 4.  -1 is an error and 0 is exit.  The main program is within a while loop to constantly prompt the user.  If it sees a -1 it shows an error and reprompts the user.

# holds response code from last LED method call
lastResponse = -1

# Exit if we have a 0 response, otherwise keep prompting
while (int(lastResponse) != 0):
promptChoice = promptLED()
lastResponse = int(exeChoice(promptChoice))

There is more to the program, but half the fun is looking at code, so here you go:

Link to the full program here
To run it make sure it has execute rights and run:

sudo python3 p1-multi-led-prompts.py

WOOO HOOO and happy coding!

A Pi is Born!

Merry Christmas, I'm Back!!!

This Christmas I received a wonderful present - a Raspberry Pi 2 (Model B) via CanaKit.  Time to kick the tires!

First impressions:

I can't believe how easy it was to set this up using the CanaKit. This kit came preloaded with an 8g micro sd card with NOOBS (New Out Of the Box Software - essentially a super easy installer for Rasbian OS).  I plugged in the HDMI cable to my tv, plugged in the included WiFi USB dongle to the Pi, and then plugged in the power (which turns it on).  It loaded up and I checked the box for Raspbian OS (running version 4.1 - Jessie) and clicked "Install".  It took a while to run, probably 10+ minutes.  I walked away and came back without checking the clock. After the boat load sequence ran, I was presented with a giant raspberry wallpaper - WOO HOO!

Initial Configuration:

Now that I'm up and running it is time to do some configurations.  First up change the user "pi" password.  Go to Menu (aka Start Menu for windows people) -> Preferences -> Raspberry Pi Configuration.  Here you can do stuff like change the keyboard, timezone, change password, setup whether you want to log in to the GUI or command line.  Next up WiFi.  This was easy.  I clicked on the WiFi icon in the upper right of taskbar, clicked my router id and entered the password and I was all set.  Nice.  Now for the most important part - getting up to date with apt-get.  Open a terminal (which is conveniently in the taskbar) and type in:

sudo apt-get update

"sudo" says run this as the almighty "root" user

"apt-get" is the application that does updates (i.e. windows update)

"update" says update your repository listings so it knows what dependencies to download when you go to update/install a program

After that runs a bit type:

sudo apt-get upgrade

This upgrades everything you have installed that needs updating.

Remoting into Pi:

This was very easy with SSH, however I wanted to do this with a GUI too.  After a couple of quick googles, I found this link:
https://www.raspberrypi.org/documentation/remote-access/vnc/

Very easy to follow and spot on.  I used tightvnc viewer on windows.  The only caveat is that whatever "X" display you set it to (i.e. 0 or 1) you have to connect to that via the viewer like this:

192.168.0.99:1

As a quick note, if you left the defaults in the initial config for networking you can use the hostname of the Pi like this:

raspberrypi:1

This works swell so now I think it would be great to save the CPU cycles and only boot into CLI (command line interface).  This can be done either via the configuration GUI mentioned earlier or via a command line GUI application so I did it that way.  Open a command prompt and type in:

sudo raspi-config

Once that loads choose the option for boot and then choose command line with auto login.  Reboot and test. WOO HOO!

Final Thoughts:

I'm impressed overall.  The OS boots quite fast and shuts down fast.  General navigation even through VNC is nicely responsive with no lag.  Browsing the internet is SLOW, but that is to be expected.  The built in browser is based on "Epiphony" which I'm reading is a GNOME Web browser (WebKit based).  I'm not impressed so I tried to install Chromium, but that failed the "sudo apt-get install chromium" so I installed Midori.  It is faster, but not by much.  Either way I won't be doing much browsing on the Pi if I can help it, but heck that's not why I'm using this bad boy, this is a maker machine and I want to make something...  Now comes the hard part, picking a project.  There are some easy installs like a MAME emulator, a media center (XBMC), but all those are just premade stuff and I already have a Roku and an old school NES so it's time to build something.  

JW Player, Security and EdgeCast

Hello kinder,

I'm currently doing some work with a company that wants to stream their media. Their site does not have that kind of bandwidth nor do they have the storage space so the next best solution was to outsource. They did trials with a few companies and www.edgecast.com won out. One of the criteria for choosing a streaming media company is security of files so that any 'Joe sixpack' or 'hockey mom' couldn't save the urls and send them to their friends therefore eating up your bandwidth and money.

So what does EdgeCast do for their security? They use token authentication. Essentially you can preshare a 'token' and then encrypt that along with other data to create an encrypted string that will be used to authorize the viewer. As 'integrationman' my job was to integrate ColdFusion and their token authentication. Surely this would be simple, well I was both wrong and right.

Process 1 - the official way + ColdFusion = failboat
The official way is to use their Blowfish binaries and exe to encrypt the string. Sure this would be simple enough with cfexecute if we weren't on a shared hosing provider (I feel the failboat waning). To add insult to injury I tried just using the built in ColdFusion <cfencrypt> tag with the blowfish algorythm and SALT etc, but that ultimately produced a string that would not match their encrypted string and therefore capsized my failboat. Finally after many hours of failing, one of their wizards said, "hey why don't they just use the SOAP api we have." Well DUH to me (I really wish they told me they had this many many hours earlier :sigh: )

Process 2 - the better way (aka SOAP)
So finally after they sent me the info on the SOAP calls this was a breeze to set up. The short part of this is that you can either use standard CF SOAP calls or go the easy route and just use CFHTTP since we only need to encrypt our token.

1. Send CFHTTP request to:
   https://api.edgecast.com/v1/mediamanager.asmx/TokenEncrypt
2. Pass 3 Parameters:
   
   • strCredential = string in the format:
     c:bas:YOURUSERNAME:YOURPASSWORD
     "c:bas:" is required and is not modified. Replace the username and password with your username and password specified in the "ADMINISTRATION -> USERS" section of the Control Panel. Make sure you do not give this user any privileges like file uploads since you will be putting the username/password either in a db or in the code.
     
   • strKey = preshared token
   • strArgs = any extra parameters you need to pass (minimum is "ec_expire=#####")
3. That CFHTTP call will return an XML string that you will parse for the encrypted string and use in your JW player.

Example Code:

<!--- MAKE THE EXPIRE TIME 30 SECONDS FROM NOW --->
<CFSET VARIABLES.tmpDate = DateAdd("s", 30, Now()) />
<!--- CONVERT TO EPOCH TIME --->
<CFSET VARIABLES.expDate = DateDiff("s", "January 1 1970 00:00", VARIABLES.tmpDate) />

<!--- MAKE CFHTTP REQUEST TO GET ENCRYPTED TOKEN --->
<CFHTTP method="POST" URL="https://api.edgecast.com/v1/mediamanager.asmx/TokenEncrypt">
<CFHTTPPARAM NAME="strCredential" VALUE="c:bas:USERNAME:PASSWORD" TYPE="formfield" />
<CFHTTPPARAM NAME="strKey" VALUE="YOURSHAREDTOKEN" TYPE="formfield" />
<CFHTTPPARAM NAME="strArgs" VALUE="ec_expire=#variables.expDate#" TYPE="formfield" />
</CFHTTP>

<CFIF FindNoCase("200", CFHTTP.StatusCode)>
<!--- PARSE XML --->
<CFSET VARIABLES.xmlContent = xmlparse(CFHTTP.FileContent) />
<CFSET VARIABLES.encText = xmlContent.string.xmlText />
<!--- CREATE VIEWER --->
<script type="text/javascript" src="swfobject.js"></script>
<div id="container"><a href="http://www.macromedia.com/go/getflashplayer">Get the Flash Player</a> to see this player.</div>
<script type="text/javascript">
var s1 = new SWFObject("player.swf","ply","640","480","9","#FFFFFF");
s1.addParam("allowfullscreen","true");
s1.addParam("allowscriptaccess","always");
s1.addVariable('autostart', 'true');
s1.addVariable('type', 'video');
s1.addVariable('streamer', 'rtmp://flash.edgecastcdn.net/#####/YOURAUTHFOLDER/');
s1.addVariable("file",'mp4:YOURFILE.mp4?<CFOUTPUT>#VARIABLES.encText#</CFOUTPUT>');
s1.write("container");
</script>
</CFIF>

Notes:

• When setting the 'streamer' variable in the JW Player make sure you use your path that is in the FLASH->FLASH STREAMING section of the admin. They automatically list "/videos/example" as the folder, however you need to choose a folder yourself that will be used for the authenticated videos. To do this, go into the FLASH->TOKEN AUTH section and add a PATH like "/protectedfiles" (make sure you create the folder first via ftp or the MEDIA MANAGER). Now wait a half hour or so like it says, then you can use this.
• When setting the 'file' variable in the JW Player make sure you set it in the format: TYPE:FILENAME.EXTENSION?ENCRYPTEDSTRING
  i.e.: 'mp4:YOURFILE.mp4?asdfkl234234jsaf...

Hope this helps some other developers out there to get their job done quicker!

Happy coding!

Things I Hate About JAVA

It's ranting time and I am officially enclosing this whole blog in a big fat rant tag:

<rant>

First off I'm not an über java programmer so coming from other languages I've picked up a thing or two about how I would expect languages to be programmed based on convenience and consistency. I'll keep this short and only hit my major gripes:

Why Java is EVIL:

1. Bad plugin implementation - how many times has an applet crashed your browser or not work cross platform? Granted it has come a long way and it really depends on who programmed the applet, but in general you have to blame SUN for poor implementation. They should have fixed it way back in 1.0. If they did flash most likely would never have hit the market. Now I find client perception in Java to be slower than Flash (especially AS3).
   
2. Programmatic Inconsistencies - why is it that to get a length of a string you use: string.length() and for an array: array.length; and for a vector: vector.size(). Who programmed this crappy language? Yes I understand that a string is an object and that length is a property of that object, but why not make a function? I understand the concepts however backwards they are.
   
   I have an idea: how about using a method for everything and calling it .length() ? Just make all arrays have a default method called length(). Same with vectors (why oh why did they choose size() ). Remember if you want people to pick up your language, make it easy for them to do so.
3. Java Docs - OHHHH how evil these are. They are god awful in design and usage. Their initial examples suck (if they even have any). Why not have an example for everything in there (method specifically). If you have a method called length() give me an example. I don't care how simple, just an example. Don't make me search the web to find an example when you could easily make one. Remember if you want people to pick up your language, make it easy for them to do so.
4. Last but not least I HATE - I repeat HATE "Null pointer exception" . I give this one big WTF Sun. Yes I get the premise behind it - I am using a variable that hasn't been initialized or something else along those lines. Don't just tell me "haha sucks for you, you have a null pointer exception" TELL me where it is. Where am I calling it, what is the name of the variable/object etc that is null? Lets take a look at a JSP example (since I happen to be working on this now):
   
   [31/Mar/2008:08:29:15] failure (15460): Internal error: servlet service function had thrown ServletException (uri=/cp/www/registration.jsp):$
   at org.apache.jasper.runtime.PageContextImpl.handlePageException(PageContextImpl.java:453)
   at _jsps._www._registration_jsp._jspService(_registration_jsp.java:826)
   at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:119)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
   at org.apache.jasper.servlet.JspServlet$JspServletWrapper.service(JspServlet.java:259)
   at org.apache.jasper.servlet.JspServlet$JspServletWrapper.access$6(JspServlet.java:249)
   at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:530)
   at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:599)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
   at com.iplanet.server.http.servlet.NSServletRunner.invokeServletService(NSServletRunner.java:943)
   at com.iplanet.server.http.servlet.WebApplication.service(WebApplication.java:1094)
   at com.iplanet.server.http.servlet.NSServletRunner.ServiceWebApp(NSServletRunner.java:1005)
   , root cause: java.lang.NullPointerException
   at _jsps._www._registration_jsp._jspService(_registration_jsp.java:620)
   at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:119)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
   at org.apache.jasper.servlet.JspServlet$JspServletWrapper.service(JspServlet.java:259)
   at org.apache.jasper.servlet.JspServlet$JspServletWrapper.access$6(JspServlet.java:249)
   at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:530)
   at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:599)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
   at com.iplanet.server.http.servlet.NSServletRunner.invokeServletService(NSServletRunner.java:943)
   at com.iplanet.server.http.servlet.WebApplication.service(WebApplication.java:1094)
   at com.iplanet.server.http.servlet.NSServletRunner.ServiceWebApp(NSServletRunner.java:1005)
   
   
   
   Where is the line number in the JSP? What variable? What object? Throw me a bone here Sun! Now I have to use System.out.println to track down the freaking line that it is on. Yes I know you can go to the java file that is precompiled and try to find it in there. That sometimes works, however System.out.println 's are often just as fast.
   
   ALSO - I give kudos to Macrobe for making ColdFusion user friendly. ColdFusion is built on top of a Java runtime and when an error is throwing in CF it (usually) tells you the line number/variable etc etc. Good Job guys! I imagine your programmers went through hell to get that working.
   

Java how I hate thee...

</rant>

IE 7 + Flash 8 ExternalInterface = BUGTASTIC!

<rant>GRRRRR BROWSER INCOMPATIBILITIES!!!!</rant>

For the past few nights I have been working on and off with a very simple (or what should have been simple) Flash => JavaScript => Flash application. The plan was to use swfobject to embed a flash movie within a form and make the submit button flash. Then after validating the form when the flash button was clicked a sound would play (don't ask - client requested said ability).

So in order to accomplish something like this I created an html form; put the flash div within the form (along with the js call to render); and then used ExternalInstance to trigger the function call on(release) within flash. FireFox worked like the trooper it is. Then comes that dreaded IE 7 test. All aboard the failboat! Every time I get the same stupid error on Line 1: "savebutton" is not defined (savebutton was what I named my flash movie). WTF. I tried everything and traced every variable/function call but nothing. After much Google digging I came upon this article by Steve Kamerman:

A fix for Flash 8's ExternalInterface() in a Form!

He said that IE has a nifty little bug where it can not reference a flash movie properly within a form tag. This was both extremely disheartening as well as a great joy. My flash button was the submit button so I just moved it as well as the flash div that I call in swfobject after the closing form tag and it worked! Thanks Steve for the hint!

Success!

Cryptography samples for dummies (RSA/DSA)

Over this past year I had the pleasure (read - angst, woe, terror) of trying to figure out how to use java cryptography. At the time we were running Luminis 3.3 (we still are, but hopefully not for much longer) and we were in the process of implementing GMail for our school which would tie into Luminis authentication. In short (I'll save the gory details for another time) we decided to use SAML (Security Assertion Markup Language) to tell Google that the user is properly authenticated to Luminis and we say that he/she is OK in our book so let them read their e-mail on your servers. This is all well and good, however during the setup 2 major things came up relating to cryptography.

Problem 1: Generating DSA keys for Google

In order for this whole SAML business to work we need to send an encrypted packet with information relating to what user is properly authenticated to Google. Google being the sly folks they are, said that they don't want this information in plain text so 'you utes need to encrypt it with DSA. Now send us the keys to the palace so we can decrypt it after you send it encrypted.' Hence the first problem, how do I generate DSA keypairs? I did not see this on Google's code site so I scoured the interweb for a few nibblets of info and I came across a sample which I updated ever so slightly (see attached file genDSAKeys.java at the end of this post). I apologize for not having the links to give credit as this was about a year ago now when I was searching. I had to dig this up now for other stuff I'm doing. Well after compiling this file with JDK 1.5 (at the time) and now with 1.6, it generates the public and private keys and saves it out to 2 files.

javac genDSAKeys.java
java genDSAKeys

I ran this on a Windows XP box fyi and then sent the goods to Google and all is swell.

Problem 2: WTF Luminis 3.3 only has JRE1.3 on it and GMail APIs require 1.4+??? Cryptography stuff only started coming out with neat packages in java 1.4!!!

More background since this is relevant. We are running Luminis 3.3 on Solaris and the only JRE they (Sungard/uPortal) supports is 1.3. So now comes the fun part, when using the Google Provisioning API to set up GMail authentication via SAML, it required a few servlets and JSPs in order to generate the tokens and do encryption and their API was written specifically for JRE 1.4+. I first tried compiling it on 1.3, however it failed miserably and I didn't think rewriting Google's api would help upgrades.

What we ended up doing is setting up a second box just for Tomcat 5.5 running JRE 1.5. This worked beautifully with Google's api code. Now comes the uber problem, interserver communications sent via the browser are not secure (bleh!). So we though the usual stuff like SSL, but it really comes down to this. We want the packets secure and we want the content within the packet protected to the fullest as it is being passed back and forth. As a result we looked into encryption.

Since we had a mixed environment of 1.3 and 1.5 JREs we had problems. After bouncing for a few hours in one of those bouncy castles at a carnival (and also endless hours of searching for anything cryptographically related), I came across an awesome site:

http://www.bouncycastle.org (The Legion of the Bouncy Castle)

This site provides cryptography libraries for Java version 1.1 and up! It's a beautiful thing (and FREE). So now comes the fun part, we downloaded the jars for 1.3 and 1.5 and wrote some code to do encryption/decryption of the content sent back and forth between the servers. However back to the topic of this post - as with most encryption that has decryption you need a public/private key pair. So I took a look around and the DSA stuff I found didn't have an example (and as you can guess I'm not an evil genius when it comes to cryptography). So I bit the bull by the nose on a grindstone and pieced together numerous examples to create one single example that would generate RSA public and private keys. I chose RSA since that was an easy option that bouncy castle supported (they support a ton of stuff - check them out and did I mention it was FREE).

After a while I created the final java file that creates public/private keys, creates a string, encrypts it, decrypts it and writes out the keys to files. It is everything I ever wanted in an example and now it can be yours for the low low price of not flaming me on how easy this would be if you did A, B, or C ;-)

javac kGenerateKeys.java
java kGenerateKeys

Download source files below:

genDSAKeys.java (DSA Example)
kGenerateRSAKeys.java (RSA Example)

The code has some comments in there for the good stuff so feel free to take a look.

Help Help My Shortcut's Repressed!

Well here is another quirky thing that happened on the way to the getting work done. My friend was using Windows XP sp2 when he suddenly noticed he could not right click on the desktop and create a new shortcut. Google searches returned nothing for a fix on this, so I resorted to some trickery to try to find out what was actually launching the shortcut wizard. I fired up Spy++ (included with Visual Studio .NET) and found that the process for the shortcut window was "RunDll32.exe" . Now we're getting somewhere. RunDll32.exe is basically a kickstart for other programs (DLLs). Now a Google search on RunDll32 and shortcut returned this url:

http://www.dx21.com/SCRIPTING/RUNDLL32/VIEWITEM.ASP?OID=103&CMD=P-A

Now he was able to run the command:

RunDll32.exe AppWiz.Cpl,NewLinkHere c:\

and it actually popped up the wizard! So now we have a couple options - create a bat file on the desktop that just calls this wizard or actually get to the root of the problem and fix it. We choose the latter. Most quirky windows problems stem in the registry so I did a find for: "NewLinkHere" and the first place that popped up was:

HKEY_CLASSES_ROOT\.lnk\ShellNew

It has the following REG_SZ (string) entry:

Name: Command
Data: rundll32.exe appwiz.cpl,NewLinkHere %1

Luckily this was missing from his registry. Once he added it, it worked! Just for stuff and giggles a second find in the registry returned the same string in the location:

HKEY_LOCAL_MACHINE\SOFTWARE\Classes\.lnk\ShellNew

So if you find you are missing the Wizard of Oz er Shortcuts, check your registry.

CryptoAPI is not available - I'm boned!

I currently use Symantec PCAnywhere (10.5) on Windows XP (SP2, however this problem occurred on SP1 as well) to manage various servers. However we also use the Novell client on our work machines. What do these two have in common? Well it means that the sys admins can push junk to your machine without your knowledge. In this case I rebooted my machine today and something was pushed to my machine that caused PCAnywhere to stop functioning. Every time I launch the application and try connecting to a host using Symmetric encryption I get the following error:

In the past I had received this message and even went so far as to reinstall PCAnywhere, but to no avail that failed miserably. The first time I got this I said, "I'm boned" and did many Google searches. I came across a post that was completely unrelated to PCAnywhere, but had a brief message about CryptoAPI. I have long since lost this url, however the answer is still fresh in my mind. It turns out that PCAnywhere Symmetric encryption relies on RSA info located at:

c:\documents and settings\[USERNAME]\Application Data\Microsoft\Crypto

This folder contains another folder called "RSA" . To fix your problem, just rename the folder (or delete it, but better safe than sorry if something else breaks - you have been warned!). The next time you launch PCAnywhere and try to connect to a server, the RSA folder will be recreated and you will be all set!

Success!

Hootie Rules!!!

Well I'm in the midst of a Sungard Higher Ed conference in Las Vegas now and I have to give props to SCT for putting on a concert with Hootie and the Blowfish. Prior to their arrival on stage was a comedian Frank Caliendo who was awesome! He had actor voices down perfectly. I would highly recommend seeing him if you get a chance. Now back to Hootie. They played a good mix of their new and old stuff and came out for 2 encoures - good job guys! Here is a vid I took on my super schweet S1IS.

Cairngorm, good to the last drop, or just a drop in the bucket of other frameworks

Well tonight I, along with my comrade in arms Brian, braved the wintery weather of NYC to go to a Flex conference hosted by the New York Flex User Group. This was my first visit to the group and I am fairly new to flex so I thought that this would be a great opportunity to see what others are really doing with flex. Tonight's topic: Moving to Cairngorm, by Michal Glowacki. Right off the bat you can easily tell that Michal is quite intelligent. He was one of the cofounders of the Cairngorm framework (which was eventually bought out by Macrobe and is still in use today by Adobe).

Basically this framework works as a means of standardizing the location of code when developing projects. With this framework he breaks it up into three layers: View, Control and business logic (I know this is not what he said, I'll have to fix this later, but this is the essence of it). In short you put all your display code in one folder, the handling of 'gestures' (button clicks, mouse movements etc) in another, and any other business logic in a third. The really good part about this framework is that if you have multiple developers working on a flex project, they can all work on separate pieces without stepping on each other's toes. However, if you are a single developer or your application is very small, this is basically overkill. Michal opened by saying that if you include all of the Cairngorm libraries, your movie size will be 300k+ larger - yikes. Perhaps he meant source code size, but it wasn't clear. I know you can just import the libraries you need so that problem is out the window.

I won't delve too much into the actual presentation, but just a note for any future developer/presenters, not only does PowerPoint kill, but if you code an application from scratch in front of an audience, yes it is quite a feat, however it is boring as hell to watch. Just show the audience the end product and explain each piece, or save a few lines for the audience and then wow them with the result, or show a completed product and ask the audience for suggestions on what to build into it.

So overall here is a quick breakdown of Cairngorm:

• Great for multiple developers
• Crappy for single developer shops, or very small applications
• Don't bother retro fitting existing applications (Michal basically dodged a question as to whether it was worth rewriting existing applications)
• New applications or potentially complicated applications can benefit from a standardized framework of code - not necessarily Cairngorm, just a standard framework

[rant]NOW for the rant of the day - DO NOT come out with a radically different framework when going from version 2.0 to 2.1. That is just stupid. What good is a standardizing framework if it keeps changing??? UGGG - I won't be using this framework until version 20, or whenever they decide to make it consistant.[/rant].

OK any advocators of Cairngorm who wish to turn me around feel free to post. I just don't see it - maybe I'm just a flex neophyte, or maybe it was the cold weather (and stepping in a very cold puddle), but I just don't see it yet...

Want to play with Cairngorm:

http://labs.adobe.com/wiki/index.php/Cairngorm

Windows XPSP2 and SQL Server 2000 - Evil?

<rant>XP SP2 IS EVIL - read more to find out why<rant>

I knew there was a reason why I held off on upgrading to SP2 on XP. I know sp2 has been out for many months, but I knew that it was evil when I first upgraded my home machine months ago and it fried it (BSOD - had to do full reinstall). So naturally I was apprehensive when it came time to upgrade a desktop at work (we are being forced to upgrade our machines to sp2). Well I figured that I would run the installer myself so I could tell right away what breaks. Running the installer went ok and after several reboots I killed the windows firewall and restarted my ZoneAlarm and then started SQL Server 2k and CF 7. Suprisingly all started fine. However when you try to connect to a datasource defined in the CF admin, I kept getting an error about not being able to connect. Within my CF admin, I was connecting to 127.0.0.1 on port 1433 as a normal sane person would.

Ok so after bashing my head against the wall and trying all of the "fixes" by microsoft which all regard adding ports to the windows firewall (which is disabled I remind you) I took a stab in the dark. I added datasources to the ODBC control panel in the System tab since that can connect to (local) and CF can connect to it via ODBC connectors. One more note, I still can't connect to my server via 127.0.0.1, rather I have to use (local). Now inside the CF admin I connect via ODBC and everything is fine. This is one more annoyance that Microsoft has given me. GRRRRR.

If anyone has a solution to this I would greatly appreciate it.

SQL Server 2000 Massive Transaction Log Fix

Ahhh the sweet smell of a full hard drive - isn't wonderful, NOT (think Borat). Ok so today I was checking out a computer running SQL Server 2000 when I noticed that an application was throwing errors that the disk was full.

I took a look and sure enough it was! I knew this box was getting low on space for a while, but I ignored it (not a production box, just a test). I tried to clean up space the usual windows way, run the low disk space cleanerupper. Then I went through and looked for logs in various applications and wiped them, but this was not enough.

Then I did what is one of the more clever things to happen to the windows search - a search based on file size. It has been in windows since at least 2k, but I'm pretty sure it was there in 98. I looked for files greater than 100mb and sure enough an 8GB file popped up at the top. What was it? well my SQL Server 2k transaction log (.LDF) for a database that has fairly high traffic. Since this is a transaction log in general you don't want to just wipe them. I did a couple google searches (<rant>btw Microsoft's MSDN site sucks for SQL server - at least in this instance. They had a dozen articles on shrinking the db, but not one that actually showed a command to do it. Anyone can talk about theory, but just a simple example would be nice.</rant>).

Ok well now it has been about a half hour and I came across:

http://www.eggheadcafe.com/community/aspnet/9/10012663/very-easy-way-to-truncate.aspx

This was perfect (and at least 2 pages in on my google search). From the page:

In SQL Server 2000 Query Analyzer run the following command (doesn't matter what db you are in):
backup log [database name] with truncate_only
(don't include the square brackets)

Open Enterprise Manager and navigate to your database:
--Right click on the database
--Highlight 'All Tasks'
--Choose 'Shrink Database...'
--On the bottom of the window on the right, click the 'Files' button
--In the drop down at the top make sure you choose the '_Log' option. By default it is the '_Data' option
--Leave the shrink action as: 'Compress......'
--Click 'OK'

At this point depending on the size of the file, it may take a minute or two or more and then it will give a success message. I took a look at my LDF file again and it went from 8gb to ~2mb.

Success!! Now rejoice by clicking the play button below (this is in my head every time I do something successfully)!

MythTV Part II - Getting the remote to work

I love ravioli. What does that have to do with getting a remote to work within MythTV on CentOS - absolutely nothing, but it was really good for dinner tonight! Whew I feel better, now on with the show!

Getting my Hauppauge IR Blaster (aka remote) that came with my Hauppauge WinTV PVR 150 to work has been the bane of my existance since my last MythTV post. When I first started, I saw that Jarod Wilson explained how to set up LIRC using YUM here and suggested using the following:

yum -y install lirc-kmdl-$KVER
yum -y install lirc

I ran that and it seemed to install properly. I then ran the following:

cp /usr/share/doc/lirc-*/remotes/hauppauge/lircd.conf.hauppauge /etc/lircd.conf

and that too worked. Then as per the instructions I added code to my /etc/modprobe.conf so that lirc would load before ivtv. Now my /etc/modprobe.conf looks like the following:

alias eth0 tulip
alias snd-card-0 snd-emu10k1
options snd-card-0 index=0
install snd-emu10k1 /sbin/modprobe --ignore-install snd-emu10k1 && /usr/sbin/alsactl restore >/dev/null 2>&1 || :
remove snd-emu10k1 { /usr/sbin/alsactl store >/dev/null 2>&1 || : ; }; /sbin/modprobe -r --ignore-remove snd-emu10k1
alias usb-controller ohci-hcd

# LIRC stuff
alias char-major-61 lirc_i2c

# ivtv modules setup
alias char-major-81 ivtv
alias char-major-81-0 ivtv

# LIRC stuff
install lirc_i2c /sbin/modprobe ivtv; /sbin/modprobe --ignore-install lirc_i2c

Then finally I tried to run:

/sbin/depmod -a
/sbin/modprobe lirc_i2c

to start up the lirc stuff and no dice, I received the following errors:

lircd: could not open /dev/lirc
lircd: default_init(): No such device

After much searching it turns out that the version of lirc_i2c that you use must be compiled with the same compiler that your os was compiled with (or something to that effect). So then I thought that I should take the plunge and compile it myself. Nothing like rolling up the sleeves to get down and dirty. But before I try to get down and dirty, I want to know what I'm getting into so after uninstalling the lirc stuff via yum, I proceeded to check out this very skimpy how to guide by mythtv.org - here. I was hoping for something a little more in depth so I then checked out the install guide provided by LIRC themselves here and that was great. I downloaded version: lirc-0.8.0 (the latest stable at the time). NOTE: 9/2/2007 - these instructions work with lirc-0.8.2 as well) After extracting all of the files to a folder on my desktop (yep the desktop - it's the Windows in me), I then proceeded to follow the "Compiling" instructions on the LIRC site listed above. I ran:

./setup.sh

and the wizard popped up and I populated all of the necessary fields and chose to have it run the configure (since it autopopulates the config options). Now it was either at the "make" step where I received an error something to the effect: "lirc_dev: class_create failed" This was bad. I spent many an hour searching for the fix to this. Then I tried just going into the code of the installer where it said it had the problem and just commented out the lines. What's the worst that could happen - well it bombed in a second point and I killed that line too. Then it worked! Well before I get too happy I tried "make install" and that ran. This was too good to be true, but there had to be a reason for those lines of code so I tried to run:

/sbin/depmod -a
/sbin/modprobe lirc_i2c

but that bombed out too. Ok it looks like those lines were really meant for something. So I backed out:

make uninstall
make clean

and Googled some more.

Then I finally came across: RE: Centos 4.3/4.4 unable to compile any LIRC version (0.7.2, 0.8.0, CVS). It wasn't my version, but heck it was worth a look. The problem as I found out was in the installer scripts on the lines I commented out. "...the recent kernel has two new functions and macros (named class_create and class_destroy) which conflict with the same lirc functions." His suggestion was renaming the functions by prefixing it with "lirc_" in every instance and wouldn't you know it, that worked!

The two files that needed changing are:

[installer location]/drivers/kcompat.h
[installer location]/drivers/lirc_dev/lirc_dev.c

I ran configure/make/make install and everything went smoothly. Then I tried:

/sbin/depmod -a
/sbin/modprobe lirc_i2c

and no errors in /var/log/lircd !!! This was jolly good. I went back to the LIRC site and continued on to installation. They mentioned this and it happened to me - when you first get this running you'll find the lirc running in /dev/lirc but after a restart of services/computer etc, for me it is now running in /dev/lirc0 . This is important, because when you want to start up the daemon later you need to point to the device like this:

lircd -d /dev/lirc0

After testing with mode2 and irw it was back to Jarod's guide to continue with getting it to work with MythTV.

wget http://wilsonet.com/mythtv/lircrc-haupgrey-g3.txt
mkdir ~/.mythtv
mv lircrc-haupgrey.txt ~/.mythtv/lircrc
ln -s ~/.mythtv/lircrc ~/.lircrc

This is great at this point I tried running MythTV and my remote was working! Ok so the next logical step is to make sure this works after a restart - well for me it was a bust. I have a feeling that it has to do with order of startup and using something like rc3.d (which I don't know about yet). So what I did when I got the machine up and running was open a terminal and run the following commands manually:

/sbin/depmod -a
/sbin/modprobe lirc_i2c
lircd -d /dev/lirc0
echo 16384 > /proc/sys/vm/min_free_kbytes
/sbin/depmod -a
/sbin/modprobe ivtv

This seemed to work sometimes (sometimes the remote would only work with mode2 but not MythTV). So then I thought about doing something like a startup script and putting them in there. After searching, it appears that there is a startup script area within ~/.kde/Autostart where you can throw an executable .sh file and it will run at some point during start up (I'm not really sure when). I created: myth-load.sh with the following:

#!/bin/bash

# Only do this stuff if we're on the main display
# (i.e., don't do this in a vnc session)
if [ `echo $DISPLAY | grep -c ":0"` -ge 1 ]
then
echo "myth-load.sh starting..."

echo "loading lirc..."
# Load LIRC (remote control)
echo "running: /sbin/depmod -a"
/sbin/depmod -a
sleep 4s
echo "running: /sbin/modprobe lirc_i2c"
/sbin/modprobe lirc_i2c
sleep 4s
echo "running: lircd -d /dev/lirc0"
lircd -d /dev/lirc0
sleep 4s

echo "loading ivtv..."
# Load IVTV (capture card)
echo 16384 > /proc/sys/vm/min_free_kbytes
echo "running: /sbin/depmod -a"
/sbin/depmod -a
sleep 4s
echo "running: /sbin/modprobe ivtv"
/sbin/modprobe ivtv
sleep 4s
echo "myth-load.sh completed"
fi
exit

Now I have the "sleep" in there just to make sure everything has a good enough amount of time to run. It is probabaly not needed. Now one note about shell scripting that I fell prey to because I was borrowing an example without fully understanding what a command does - If you use an "&" after a command like this:

/sbin/depmod -a &
lircd -d /dev/lirc0

then BOTH command will run simultaneously. The one with the "&" is set as a background process. This was messing me up for a long time. As of tonight and a couple restarts later I am not having any problems with my remote.

WHEW!

MythTV - just a myth or a reality for the Linux neophyte

<rant>Rant #1 - I have to start this off with 'WHY ME, WHY ME, OH THE HUMANITY!' Why you ask - well barring the fact that this is now 2:30 am, I have been attempting to get MythTV running on my CentOS Linux box for the past 4 hours. I have to hand it to Microsoft and Windows application developers, at least when they need a dependency they either include it or provide a direct link to it (well most of the time). My limited experience with Linux makes me the perfect candidate for RTFM, however who needs a manual??? Ok well I do, but it doesn't help - provide examples that anyone can understand, not just Linux guru's!</rant>

<rant>Rant #2 - BestBuy you kinda suck. I went to BB yesterday (Saturday) to pick up an ATI TVWonder 650 card for like $130. I got it with no probs, then the next day I look at the circular and what do I see - the card for $99! However, I had a bad feeling last night so I never opened the box. I was reading some posts about people having Windows/Linux driver problems and various problems all around so I elected not to open it and go with my original plan - the Hauppauge WinTV-PVR-150. Normally it retails for about $100, however CompUSA had a special this week (Dec 10, 2006) for $40 after rebate!!! Joy! I picked one up and returned the ATI and I'm super (thanks for asking). BB, you should have had some means of telling me that the product would be on sale the next day (hmm if only BB had a computer, or a room full of Monkeys to input the sales for the next week and then compare items when purchased...) - Boooo</rant>

Ok now that that is out of the way lets dig into what happened these past for hours. First off I have to give EXTREME thanks to a guy called Jarod C. Wilson. He wrote this great tutorial on how to set up MythTV on Fedora.

Click for Tutorial

While it got me about 70% of the way there, there was another person - 22Eleven who created a tutorial for the CentOS specifics

Click for Tutorial

Ok a tutorial can only take you so far with Linux. In a perfect world (not that I'm calling Windows the perfect world) you could just run the installer and boom it would be done. Well when I installed CentOS, I chose some extended options like GIMP so I could potentially do some photo editing. Well MythTV installer wants nothing to do with GIMP or any other photo type programs. So during my installation of MythTV as per:

> yum -y install mythtv-suite

I received many errors like the following:

Missing Dependency: libstdc++.so.6(GLIBCXX_3.4.6)
Missing Dependency: perl(File::Slurp) is needed by package perl-XMLTV

Well I'm not a Linux master, but wouldn't you think that you could just install libstdc++ with a yum command? NOPE forgetaboutit! So then I had to uninstall them with commands like:

> yum erase gimp

After uninstalling anything that was in the "is needed by" list, I tried again - no good, other probs still. I ended up going into 22Eleven's tutorial again and looking at the YUM configuration again. I noticed that I had mixed some of the YUM configs from Jarod's and 22Eleven's. I wiped out the unnecessary configs and tried again - no luck. Then I went in and enabled the "atrpms-testing" and tried again. This time still no luck. Ok I was about to throw in the towel and then with dumb luck I tried installing it with the following command:

> yum -y install mythtv

Notice no "-suite" on the end. Well to my suprise it worked!!! It installed with no errors (at this time it is about 1am). Well now I'm on a roll so I figured what the heck - lets try for the home run and I ran the yum command with the -suite and it too worked properly with no errors!!! My assumption is that by installing MythTV standalone first it fixed some dependency issues.

Well now this is freaking sweet so I just followed along "10. Get and install capture card driver(s)" within Jarod's tutorial and I didn't hit any hiccups since. I was able to query my card and the picture looked like crap, but I had faith that it was just using some baseline driver for looking for video. After the tv tuner card was set up, I walked through setting up MySQL with no problems either. Then came the setup for MythTV. That too had no problems. It brought me into a wizard which walked me through its initial setup. My only problem was that my mouse had no cursor in their setup wizard so I had to either use the keyboard or eyeball when the mouse was over a field by looking for the box to lightup.

I realized the first time through the setup that I didn't have the channel guide. Upon RTFM'ing I realized I had to create a Zap2It account in order to retrieve the channel listings. After creating it with no problems, it reminds you that you have 3 months of free usage, then you have to come back to the site and take a survey before it will let you use it again. A small price to pay for something free like this.

Finally I made it through the setup and finalized the download/processing of the channel guide (which btw took about 10 minutes to grab 2 weeks worth of channel data). Now comes the juicy part - using the program! I ran:

> mythbackend

to start the service - then I ran:

> mythfrontend

to run the actual application. I immediately went in and tried the tv and........... IT WORKED!! The quality was a little off, but that could have been because of a number of reasons:

• Old cabling


• Old cable splitter


• Bad signal coming in (not likely since the tv looks fine


• Computer is too slow - could be since it is only a 700mhz, 512mb ram, 16mb Voodoo 3 3000 vid card


• Computers don't like me

Well I would love to stay up and tinker some more, but I have to get some sleep before work starts in a few hours. Tuesday or Wednesday I'll get more time to tinker again.

What did we learn today kinder - Flex!

Well here we go again - nothing like midnightish programming. Today in webdev, we watched an Adobe Flex presentation on Flex Data Services. It had some really promising ideas (webcasts/chat rooms/games potentially). I had not had a chance to dive into Flex development yet so I thought this would be a good opportunity to kill two birds with one stone - learning Flex and integrating Flex with its Data Services.

As with any new language that you learn you have to start with the equivalent "Hello World" app which I did. It simply consisted of:

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" layout="absolute">
   <mx:Panel title="My Application" width="200" height="300">
      <mx:Label text="Welcome to Flex!" mouseDownEffect="WipeRight" />
   </mx:Panel>
</mx:Application>

After saving and running the app within the bundled Flex Builder Application it worked! Ok this is super (thanks for asking)! Well I'm in a groove now so I then moved on to "How to pull in an RSS feed with Flex" I followed along with the handy dandy tutorial that came with Flex Builder and it worked perfectly (Good job Macrobe).

Well this looks like a double win for me, time to do some real testing - Data Services. First off I installed the Flex Data Services with the embedded JRUN J2EE server installed (I have ColdFusion MX, but for simplicity I chose to do it with the embedded version. So far so good. I installed it and ran the "Start Integrated Flex Server" icon; allowed it access within ZoneAlarm; hit the web page: http://127.0.0.1:8700/ and viola!! a web page shows up. I tinkered with the sample apps and they are very interesting. In general this gives me a NetMeeting-esque feeling. NM was desinged for sharing a whiteboard, computer space etc with multiple users. This was back in the 90s and now we are accomplishing this with Flash. Java had stuff like this back in the 90s as well with RMI.

[inserting wild tangent here]
My first experience with RMI was with a guy named Harry Baya. Back then in the Computer Center of Hofstra University we held a web page contest every year showcasing new technologies. At the time flash was in version 4 so you can imagine all of the timeline based movies :-) Harry decided to work on a game in Java using RMI (Remote Method Invocation). His game was almost like a Monopoly type board game where different users would control different pawns. Unfortunately the game never really panned out, but he did get the RMI stuff working and multiple users could connect with a java applet to his central server and move pieces around. It was really mind blowing at the time. After that Shockwave came out with their version of an xml socket server which is still used today with many multiplayer games. And now back to the topic at hand - Flex Data Services
[/inserting wild tangent here]

So far so good, I can run the sample applications, but can I build my own? To quote Bob the Builder, "Yes We Can!" (well I only found that out after about a half hour of trying). So here we go:

First off, it helps to RTFM. I started with: Macrobe's Note's example. It seemed like a good place to start.

Here's a little Flex Builder info that I ran into - When you configure your project to use Data Services, it asks you for the WEB-INF/flex root folder. If you are working on the samples web app, you would specify the web root of:

C:\fds2\jrun4\servers\default\samples

And a url of:

http://localhost:8700/samples

Oh one more thing - by default JRUN runs off of port 8700 with this version. Now keep in mind that Flex Builder will create a folder with the project name that you choose. So in the documentation if you want to put this into the "DSLessons" folder as per the instructions, make your project name that.

I went through page by page looking at the code and copying it into my Flex Builder. I built an application called "lesson1" as per the instructions in Flex Builder (if I truly followed the instructions I would have used DSLessons, but you know I'm a bad ass). After that the directions said to add the "destination id" to the data-management-config.xml and had no problems in the sample area (especially since it was already there for me!). After putting all the code into Flex Builder and compiling and opening two browsers, Viola, it worked! Here is my sample code:

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" layout="absolute" width="100%" height="100%" creationComplete="initApp();">
   <mx:Script>
   <![CDATA[
   /* IMPORT STATEMENTS */
      import mx.data.DataService;
      import mx.data.events.*;
      import mx.rpc.AsyncToken;
      import mx.rpc.events.*;
      import mx.messaging.events.*;
      import mx.utils.ObjectProxy;
      /* GLOBAL VARS */
      public var noteObj:Object = new Object();
      public var getToken:AsyncToken;
      private var ds:DataService;
      [Bindable]
      public var noteProxy:ObjectProxy;
      
      public function initApp():void {
         ds = new DataService("notes");
         ds.addEventListener(ResultEvent.RESULT, resultHandler);
         ds.autoCommit = false;
         noteObj.noteId = 1;
         noteObj.noteText = "Type your notes here and share them with other clients!";
         getToken = ds.getItem(noteObj, noteObj);
      }
      
      public function resultHandler(event:ResultEvent):void {
         if (event.token == getToken)
            noteProxy = ObjectProxy(event.result);
      }
      
   ]]>
   </mx:Script>
   <mx:Binding source="log.text" destination="noteProxy.noteText"/>
   <mx:TextArea id="log" width="549" height="216" text="{noteProxy.noteText}" x="27" y="24"/>
   <mx:Button x="269" y="265" label="Button" click="ds.commit();"/>   
</mx:Application>

UPDATE 1/6/2007!
Click here for the project files - including mxml

Java CFX and Jar Hell

While this topic may not come up all that often, it came up recently for me. I started to play around with building Java CFX tags for ColdFusion to add in some functionality. While I could have just invoked the java code within CF, I chose to try learning how to build a CFX tag. So first off I went with the absolute basic - the Hello World Application:

import com.allaire.cfx.*;
public class HelloWorld implements CustomTag
{
   public void processRequest(Request request, Response response) throws Exception
   {
      response.write("Hello World");
   }
}

Now comes the compiling part. This was a little tricky, but the only key to this is knowing that you have to include the classpath to your cfx.jar file in order for it to compile properly. The following example assumes javac is in your path and you are in the same directory as HelloWorld.java:

javac HelloWorld.java -classpath "C:\JRun4\servers\cfusion\cfusion-ear\cfusion-war\WEB-INF\lib\cfx.jar"

Once that compiles properly put it in the "C:\JRun4\servers\cfusion\cfusion-ear\cfusion-war\WEB-INF\ classes" folder.

Now register it within the CF Administrator by going to the Extensions -> CFX left navigation and choose "Register Java CFX". Then make the tag name CFX_HelloWorld and make the Class name "HelloWorld" (sans quotes and Case Sensitive based on the name of your class file you just compiled)

To call this on a .cfm page use:

<cfx_HelloWorld/>

(not case sensitive). That's it!!! However I started to think - hmmm applications can have more than one class file, so normally I would just jar the files. Simple enough in theory. HOWEVER, after spending several hours trying to get a JAR to work by putting it in various areas (classes folder etc and even adding it to the JRun classpath) I had to resort to asking for Macrobe (it's hard to get away from Macromedia and calling them Adobe) help through their support line. They were very helpful and in one simple e-mail they said that all you have to do to get a jar to work is put the jar in "C:\JRun4\servers\cfusion\cfusion-ear\cfusion-war\WEB-INF\lib" and restart the server. Registration within the cfadmin is the same for the class. For my simple example I took the same HelloWorld class file and jar'd it. Then I named the jar the same as the class file and put it in the lib folder. After a restart, it worked! Whew - I wish I knew that many hours ago. Just for a little background - I was using CFMX 7.0.2 with JRun 4 with the JDK 1.4.2_12 (the version of the Java SDK should match the JVM version of your ColdFusion Server)

Hopefully my little walk around the park will help someone else out since Google didn't seem to help this time. Good Luck!