« October 2003 | Main | April 2004 »

November 13, 2003

dsp returns to the elbow room

My brother's band Donner Surprise Party is coming to the Elbo Room on November 25th. Be there or be square!

Hey Everyone!

We've got some shows coming up that we wanted to let you know about.

The first is:

Tuesday November 25, 2003
The Elbo Room
2871 N. Lincoln
Chicago, IL 60657

Donner Surprise Party goes on third at 11:00.

Check out www.elboroomchicago.com for more info.

Here's an idea, join us at the Elbo Room on the 25th, take a vacation day Wednesday the 26th, and have a nice 5 day weekend!

Our next show after that:

Thursday December 11, 2003
The Lyons Den
1934 W. Irving Park
Chicago IL, 60613

Donner Surprise Party goes on first @ 9:00

Check out www.lyonsdenlive.com for more info.

Hope to see you all there!

November 05, 2003

my future tv

I think I've decided what TV I'm going to buy for my new condo. Toshiba's 57H93 looked especially nice in Best Buy's showroom compared to most of the other TVs there. Picture brightness/range was better than any other TV of it's size. It's also only 24in deep which is skinner than most other projection TVs of any size.

the twelve days of panther - part one

I've been running Panther on my Powerbook G4 for twelve days now. It's had it's ups and it's downs. Read on if you want my review of Panther, starting with the downs.

Well today just completely sucked. After eleven days of success with FileVault, I was finally hit by the FileVault screwing you up the ass bug. I'm going to detail my experience with FileVault and why I think it becomes corrupted and is causing so many problems for people.

I was very excited when I first learned about the FileVault feature coming in OS X 10.3 back in June. At the time I was manually keeping all of my important and sensitive documents on an encrypted disk image located in the home directory on my PowerBook. I also stored all my email on the image by creating a symlink from ~/Library/Mail to /Volumes/Confidential/Library/Mail. Confidential was the name I gave the disk image. It was part of my daily routine to mount the encrypted disk image whenever I needed to read email or work with any of the documents contained on it, and then to unmount the image when I was done. This works pretty well for keeping your documents secure. If you left the image mounted, when you came out of sleep, it would prompt you for the image password and if you did not supply it, it would forcibly unmount the image no matter what files may have still been open. This process worked well for me, although it was pretty tedious to be continually mounting and unmounting the image entering the password each time. At this point you may be saying, well why didn't you just store the password in Keychain so it would stop asking for it. Well if I did that, then wouldn't that defeat the purpose of having a password on it? Truth is, I did have it stored in Keychain. But I have it configured with a short timeout, so I was basically entering my Keychain password continually.

When I first learned of FileVault in Panther, I figured that Apple was doing something similar to what I was doing (creating an encrypted disk image, storing all your files on it, pointing your home directory at it instead of the usual place). And in fact I was right. But one of the differences between my method and Apple's is that they use a "sparse" image, where as I used a regular disk image. When you create a regular disk image, you specify how large it should be and the image takes up that amount of space whether it is actually being used or not. I chose a disk image size of 4GB and while I was only using about 1.8GB of it, it was still taking up 4GB of space on the harddrive. This was of little concern to me since I had plenty of space anyway. But you can see how it is a disadvantage. Sparse images on the other hand, grow in size as you use them. So the image grows automatically as you add data to it. But if you remove data, the space is not reclaimed, so sparse images do not automatically shrink. I had toyed with the idea of using a sparse image when I first created my encrypted disk image, but googling at the time I found reports from people saying that sparse images tended to be easily corrupted if your system crashes where as regular disk images rarely had any troubles. (And in fact, mine never did. I crashed a handful of times throughout my life with Jaguar and never once was my encrypted disk image not in tip-top shape when I remounted it.) So there was that, and the fact that you can only create sparse images by using the hdiutil command-line utility instead of the Disk Utility GUI I used to create my regular encrypted disk image.

On Friday, October 24th I headed to the North Michigan Apple Store for Night of the Panther and bought my shiny new copy of Mac OS X 10.3. Come Sunday morning I started my install. I decided to test my luck and chose to do an Upgrade install. First part of the install for most part went okay. I turned on the log to see what all it was doing and saw messages about journaling not being enabled because there was not enough free space on my drive. I only had a little over a gig free when I started the install. Prebinding was next which took quite a while. Viewing the log I saw that prebinding did not work with the versions of iCal and iSync I had installed the day before with Jaguar's Software Update which were newer than the versions shipped with Panther. Now was time for the first reboot. Doh! My PowerBook seemed to be hung after the Starting Installation splash rolled past. I could move the mouse and my drive seemed to be making noise, but nothing would respond. After about 20 minutes I gave it the three-finger-salute. (on a Mac that's ctrl+cmd+power) During boot I went into single-user mode and ran fsck a few times. After that the install picked backup where it left off and finished installing software from disk two and three. There was no second reboot was was surprising. It went straight to registration, and then to the login screen.

I logged in and started looking through all the new System Preferences. I created a couple user accounts and started playing around with Fast User Switching. Everything worked like clockwork. Played around with Exposť. Pretty cool. Played with the new DVD Player. Spent quite a bit of time playing with the new Activity Monitor. Turned on the new iDisk syncing. Everything worked fine and overall the entire system felt more snappy and responsive compared to Jaguar. So far so good.

With all that out of the way, it was time to ditch my old encrypted disk image and give FileVault it's time in the spotlight. But before I want to devote my data to it, I wanted to first investigate just what it's doing. I wanted to see if my assumptions were correct. So I first enabled it for one of the new accounts I created to test it out. Before it was go ahead, it said that no other users could be logged in during the conversion. So I fast-user-switched to my other logins and logged them out. Then returned back to my test account. From there, in order to turn it on for the account, I had to give an administrator password. And I was prompted to set the master password. It then confirmed I wanted to start the process and that it could not be interrupted. I went ahead. While it was processing, I logged in via ssh under my normal account to inspect what I was doing. I found in /Volumes it had created a .com.apple.FileVault folder. In there was a folder with the name of the test account ('conan'). Inside that directory was a conan.sparseimage file. That was being created by hdiutil I saw from ps. So it's using sparse images I learned. Once the process was done, it removed the original files from /Users/conan and moved the sparse image there. The login window was then displayed. I logged in as conan and then inspected what it had done upon login.

When a filevault user logs in, it moves the existing /Users/username folder holding the sparse image to /Users/.username and then creates a new /Users/username folder with mode 700 permissions so that it is only readable by the one user. It then mounts the sparse image to the /Users/username directory instead of in the usual /Volumes location. When the user logs out, it unmounts the image. Removes the new /Users/username directory, and then moves the /Users/.username directory back in it's place. Pretty straight-forward and simple actually. All the pieces to make FileVault were pretty much already in place with Jaguar. All that was needed was to automate these few tiny steps at login and add support for master passwords.

Well I was ready to give FileVault a shot for real. I had over 10GB of mp3s in iTunes and almost of gig of pictures in iPhoto and a bunch of movies in my Movies folder. I knew it would take a long time to encrypt all that data which doesn't even need to be encrypted in the first place. I also knew that it wouldn't be able to encrypt all the data anyway since you need as much free space on your drive as you have in your home directory so that it can create the disk image. So I decided to move it to /Users/Shared. All the files I was going to keep in my home directory I then quickly backed up to my iPod just to be safe. I started FileVault and let it go to work. Took about an hour to encrypt everything. I also logged in via ssh to see that it was doing all the same things I saw from before.

First login under FileVault and everything looked to be working fine. Things ran fine for a few days. I upgraded the rest of my programs to the newer pantherized versions. And everything ran smoothly. Eventually it became time to install Security Update 2003-10-28 and this was to be my first reboot since starting my life with FileVault. As my system started the reboot process, after shutting down all my apps and desktop I was presented with a dialog asking me if I wanted to reclaim left over space from using FileVault. Since I wanted to be able to see what it was going to do and I wasn't sure I would be able to ssh in since it was in the middle of rebooting. I decided to pick Skip Now. Reboot proceeded as normal and I was back up.

I logged in and then immediately logged back up. I was again presented with the dialog. It said the process could not be interrupted once I hit continue. I hit continue and then from my Beige G3 logged in over ssh to see what it was doing. Running ps revealed a "hdiutil compact" command being run. Pulling up the man page for hdiutil reveals:

compact image
scans the bands of a SPARSE type disk image with an HFS filesystem in it, removing those parts of the image file which are no longer being used by the filesystem. Depending on the layout of files in the filesystem, compact may or may not shrink the image file. Common Options: -encryption, -stdinpass, -srcimagekey, -shadow with friends, and -plist.
Ahh so that's how they are doing it. I can't remember if compact was available through hdiutil in Jaguar or not.

Now this is where things get interesting. I then noticed that my PowerBook had returned to the login screen with the compact process seemingly complete. But I could hear my harddrive was still churning pretty hard. I checked ps over ssh and indeed saw that the hdiutil compact process was still running! This can't be right I thought to myself. If I were to login right now, I bet it would completely hose the image. So I let it complete before I logged back in. Second login with FileVault and everything is working fine. I then thought...well let's just see what happens if I try to login while the compact process is still running. I logged out. It asked if I wanted to reclaim unused space. I hit continue. PowerBook returned to the login screen and then I logged in. I checked Activity Monitor and saw that the hdiutil compact process was still running. But everything seemed to be working fine regardless. iTunes played. iPhoto zoomed my pictures. Mail downloaded my new mail and let me read it. Safari worked fine as well. I guess it's okay for that process to keep going while you are logged in I thought. I even tried the same thing a few more times just to make sure. Nope. No corruption at all. A few days go by and I read about all the troubles people are having with FileVault on the Apple discussion boards. Hrmm. I wonder if it's related to the login screen not waiting for hdiutil compact to complete. Nah...mine is working fine. These people must be doing something wrong.

Fast forward to today. One of my apps was giving me some troubles so I decided a logout would probably correct it. Sure reclaim my unused space. Login screen appears. Harddrive is still churning. I decide to login. Everything should be fine right? I login. First thing I notice is that my desktop background has been reverted to the default blue instead of being a picture of my niece, Riley. Oh and the icons on my Dock have been replaced by the default set. Oh crap. I've been hit! I open up Mail.app. All my email is there and it begins to check my mail. So far so good until I notice that messages aren't being flagged as junk. Going to preferences I see that Junk Mail Filtering has been turned off. And so are all my own filters. Gone. In fact all preferences had been returned back to their defaults. Ugh. This isn't looking good. I open up AddressBook. Phew, everything is intact there. Open iCal. Everything is fine. Maybe I wasn't hit to badly. I open iTunes. Boom, library file is corrupted and it won't open. Thank God all my mp3s are in the Shared folder instead of in my directory. I removed iTunes Library file and then opened iTunes. Empty. I then selected Import from the File menu and chose the iTunes Music Library.xml file. It started the import process which tool longer than I expected. All my music is back, but now it has lost all my play counts and the last time played for each track. Quite a bummer as I had some good playlists based on play count and last time played. I opened Safari. It opened to the Apple/Netscape home page.
That's not right, I have it configured to open to a blank page. Checking preferences it turns out all Safari preferences had been reverted to their defaults. I luckily still had all my bookmarks, but the default bookmarks had been added back in. Not too bad, but a definite irritation.

Then I opened up StickyBrain.

StickyBrain can't read your data file.
A new data file has been created in the same directory as your old file (which hasn't been deleted). To see where your file is located look at the Default File location in the Preferences dialog.
NOOOOOOOOOOOO!!!! Now I was mad. Losing preferences is annoying, but it's not horribly serious. But now I was hit with dataloss. Fortunately I was able to revert to the backup from yesterday and then open the broken StickyBrain file with BBEdit and extract out most everything that was lost. I was lucky. Then I discovered that the Excel spreadsheet I keep the list of my current and future tasks in was corrupted and would not be opened by Excel or OpenOffice. Fortuneately I emailed a copy of it to my boss on Friday so I was able to pull it out of my Sent folder in Mail.app. I hadn't made very many changes to it since Friday so I was easy to recreate the changes. Again...lucky.

Needless to say FileVault is now turned off. The conversion back to normal was painless and I haven't found anything else to be in disarray. I would bet that so long as you don't say yes when it asks you to reclaim space, or that if you do, you wait and make sure that the hdiutil compact process has completed, that you can have a corruption free experience with FileVault. But I think I'll wait until Apple releases 10.3.1 that will hopefully correct these issues before I turn it back on.

That pretty much covers all of the bad. Come back later and for part two of my Panther review wherel I will describe what ROCKS! about Panther.