Consilience

You may have already heard, but Severed Fifth, Jono Bacon’s music project, has been ramping up recently.

I want to highlight a few things that I think are fairly interesting.

First, Fair Pay:
Jono is experimenting with the same model that many others have done (notably: Nine Inch Nails, Radiohead, and GirlTalk) where you let the music fan pay for the enjoyment they get from the music (that is about the best way to put that, because they aren’t really paying for the bits, those cost practically nothing). When this choice is put in front of music fans, either paying zero or some other non-zero amount, many pay some non-zero amount. My recollection is that the amount is somewhere around $8 for an album.

What would be curious to look at in the case of Severed Fifth is what that amount is. As we saw with the Humble Bundle from Wolfire, Linux users paid more per game than any other platform. Will the awareness of Jono’s project in the FLOSS world translate into a higher average “Fair Pay” for Severed Fifth? That probably isn’t measurable, unfortunately.

Second, Frets on Fire:
Boy do I love me some Rock Band, er, Frets on Fire! I think this is one of the cooler things about CC-licensed music: easier conversion into kick ass formats like Frets on Fire with no worries about copyright law (more accurately, the copyright holder) telling you that enjoying Jono’s music in a certain way is not permitted.

Third, YouTube:
This one might seem obvious, but I promise there is a bit more to it. Basically, because the Severed Fifth music is licensed under a CC license, people can us it in their YouTube videos without getting take-down notices. Pretty awesome stuff. Now the really interesting part. Jono has licensed the Severed Fifth music under CC:BY-SA (Attribution ShareAlike) license. That means, simply, you can use, redistribute, and remix the work as long as you give attribution to the author (Severed Fifth, or Jono) and share any derivative work you make under the same license, CC:BY-SA.

Now, using Severed Fifth music in your video means that your video, a derivative work of both your footage and the Severed Fifth music, is now required to be licensed CC:BY-SA. From the legal code of the license: “For the avoidance of doubt, where the Work is a musical composition or sound recording, the synchronization of the Work in timed-relation with a moving image (“synching”) will be considered a Derivative Work for the purpose of this License” (see section 1b). This is great, now all of those awesome videos Jono showcased in his blog post are now available for your reuse under the terms of the CC:BY-SA license! I think that might be my favorite part of the new happenings around Severed Fifth; Jono spreading the Freedom!

Today, Creative Commons has announced their campaign to support the new Catalyst Grants.

If you haven’t heard of it, the Catalyst Grant program is pretty awesome. It helps people who are working on great projects keep them going by providing the much needed funding. From the Catalyst Grant page:

Creative Commons is investing up to $100,000 to empower individuals and communities deeply rooted in the principles of openness and sharing. With the Catalyst Grants program, Creative Commons will seed activities around the globe that support our mission. Our goal is to scale our community’s efforts and support them in becoming self-sustainable. Through a rigorous public review and transparent evaluation process, the best proposals submitted by CC affiliates and the broader community, will be selected to receive $1,000–$10,000 to make their ideas a reality.

But, Creative Commons can’t do it all. And this is where you come in. By donating to Creative Commons you can directly help support the (no doubt) awesome projects that the grant program will select. Help support the commons by being a catalyst.

As some of you are probably (way too) aware of, I like to backup my social data across the web (see what I do for backing up my google calendars). I actually dream for the day when there is an Ubuntu package I can install, give it my credentials to a few websites (which it saves in your keyring), and then it proceeds to create an initial backup of all your data across all your services. Why do this? Well, aside from the mantra of “keep your own backups!” in case of service malfunction (remember when gmail went down for a few hours? I do, people went crazy), there is also the personal desire to have the ability to migrate to a new service should I wish in the future. If I find a better photo sharing service for some reason, I want to migrate my data/photos to it easily.

Now, backing up flickr.

There are few very important pieces of information to backup from flickr which I can do right now: my photos and my stats (views/referrals of my photos).

Photos
Tool: FlickrTouchr
Why #1: Do you backup your ~/Photos directory? If your answer is “No” or “Infrequently” you might really like this when your harddrive crashes and you don’t have local copies of those awesome photos from your awesome vacation.

Why #2: Do you take photos with your cell phone and upload them directly to flickr? Do you then clear them off your phone because they take up valuable space? This will make sure you have a copy of those on your own machine for easy editing/backup (see #1).

What it does: This one does what it does very well. It authorizes itself with your flickr account and then proceeds to download all of your photos (including your private ones, hence needing to authorize). Also, if you use the Sets feature of flickr, it keeps those associations by creating directories with the sets’ names. So, my directory structure that flickrtouchr creates for my account looks like this:

greg@rose:~/backup/flickr/photos$ ls -1
Bike Lock Fail Blog
Bike Ride – 20090628
Botanical Garden, July 4th, 2009
Bug Jam
Favourites
Gettysburg Trip
Jaunty Release Party
Mackinac Island Trip
No Set
SF – 2008
touchr.frob.cache
Traverse City – December ’09
UDS Karmic

You’ll see the “No Set” directory, which is where all the photos that are NOT part of any set.

How:
If you are going to run this script manually and your local machine with a web browser, you’ll be just fine and just follow the instructions it gives you. If, however, you are like me and want to run this via a cron job on a regular basis, you’ll need to take an extra 2 steps.
1. Start it on your local computer and it will authorize itself via your browser.
2. Kill it (CTRL+C) so you don’t have to sit there and wait for it to finish downloading all of your photos.
3. Copy the touchr.frob.cache file to your server and put it in the folder you’re going to backup your photos to.

Now when it runs it will pick up your credential information from that file and run as expected. Put 5 23 * * * python /home/greg/src/scripts/flickrtouchr.py in your crontab and you always have a backup of your photos! Don’t worry about running it every night; if the photo is already downloaded it just skips it (ie: It does The Right Thing®).

COOL! Now you have your photos backed up!

Statistics
Tool: My flickr-stats-export.sh based on this unnamed Github Gist
Why #1: I like numbers and these are the “raw” CSV files that flickr is producing for your photos. It tells you how many times your photos are viewed and what the referrer was.
Why #2: The stats are going away!

What it does: Pretty simply, it goes to the your stats download page and downloads all the CSV files linked from it. You can see that page by going to this url: http://www.flickr.com/photos/YOURUSERNAME/stats/downloads/ (fill in your username). It then makes a tar.gz of these to save space.

How:
How about we let it tell you:

greg@zen:~/src/scripts$ ./flickr-stats-export.sh –help

Usage: ./flickr-stats-export.sh DIRECTORY USERNAME COOKIES

DIRECTORY:
Directory to save the flickr-stats.tar.gz file of stats .CSVs

USERNAME:
Your flickr.com username

COOKIE:
See the -b flag from the CURL manpage.
It can be the contents of a cookie file or the full filename of the cookie file.
I recommend getting the cookie file from flickr using Firebug, then saving that
in the directory you plan to save the stats files.

If there is already a cookiejar.txt file in the download directory,
we will use that instead and this can be left blank.
See the -c flag from the CURL manpage for more on cookiejars.

As you can see, it needs your flickr cookies to run, so, 1) Install Firebug and Firecookie 2) Login to flickr 3) Go to the cookie tab in Firebug, then the Cookies dropdown and select “Export Cookies For This Site.” 4) Save that file somewhere.

I run this form my server, so I copied that cookies.txt file to the ~/backup/flickr/stats/ directory and then ran
./flickr-stats-export.sh ~/backup/flickr/stats/ grggrssmr /backup/flickr/stats/cookies.txt

I would suggest running this automatically so you don’t miss any stats. But, you only need to do it monthly as the stats csv files are only updated every first of the month. So, I have this in my crontab:
0 12 1 * * /home/greg/src/scripts/flickr-stats-export.sh /home/greg/backup/flickr/stats/ grggrssmr

Notice that I left off the cookies.txt? That is because after the first time it runs it saves the cookies in a “cookiejar.txt” file in the stats directory, and if that file is there, it uses it.

That cron job runs at Noon (Eastern time zone, where my server is) on the 1st day of every month. Why? This data will only be available until June 1st, 2010 at Noon PDT (Pacific time zone). So, I picked a time 3 hours before the data will disappear so that I A) won’t miss it and B) give it time to generate my data for the month of May. After June, you can remove this from your crontab as it won’t do much after the files are gone.

Luckily, if you forget to remove the script’s entry from your cronjob file after that date, it will just exit if it doesn’t have any .csv urls to download. So, it then won’t try to make a tar.gz of empty files and save empty data over your last good flickr-stats.tar.gz

Future Research
So, with those two things you have the photos and the statistics from your flickr account. However, that isn’t everything. I am working on extending flickrtouchr to also download the photo metadata (title, description, tags, comments, license) which it doesn’t save. With that metadata I will either create a metadata xml file associated with the jpg or embed the info INTO the jpg using the XMP standard (see: python-xmp-toolkit). You can see what I’m doing at this launchpad branch. Please feel free to branch it and help out!

I just got back from a great day at the Ubuntu Michigan LoCo edition of the Global Jam were we tested Lucid on a ton of different hardware. It was a great time. See the photos.

But, what I want to share with you right now is a quick script I whipped up to backup my Google Calendars nightly. This is one of the steps in my on-going process of making sure all of my personal data is backed up by me on machines I control with an eye to migrating to self (or friend) hosted services. Yes, I want services I use to follow the Franklin Street Statement.

Until the day that all of the services I use follow the Franklin Street Statement recommendations, I will just have to make sure I make personal backups of my information. So tonight, I finally did that for Google Calendars. It was pretty simple, really:

#!/bin/sh
# Backup my Google Calendars

# Remove files that are older than 1 week
find /home/greg/backup/google/*.ics -mtime +7 -exec rm -f {} \;

That’s it. Create the filenames for the various calendars I’m backing up, including today’s date. Then wget them. Then, delete any .ics file that is older than a week. Not sure why I need 7 days of backup, but better safe than sorry, I guess.

As you probably know, the Ubuntu Global Jam is coming up this weekend.

Well, along with this long list of US LoCo Teams, the Michigan LoCo is having an event on our east-coast. Our LoCo Directory event page has all the details.

I hope to see a lot of new faces!

[btw, that was really tedious to link to all of those US team events... glad there are so many, but ...]

And my email configuration addiction continues. As you may recall, I did an extensive (now out-dated) post about my email setup. I’m still using the basic configuration outlined there (all of the same programs) just some of my settings have changed a little. I tend to make little changes every now and then which I think will save me a tiny amount of time wet-ware processing my email. Below I outline how I am making the transition to new machines/installs easier for my email setup.

Restore Config Files

With the purchase of a new laptop, I have to re-setup my email’s (and other applications’) configuration. Luckily, I keep all of my configuration files in version control which is synced between 3 computers; my old laptop, my desktop, and my webserver (for mostly backup purposes, but also for imapfilter). All I had to do was run the following very basic commands:

mkdir src
cd src
bzr branch greg@my.server.net:/home/greg/src/dotfiles .
bzr branch greg@my.server.net:/home/greg/src/scripts .
cd scripts
./restore_email_apps.sh
./restore_symlinks_desktop.sh

The first 5 things are pretty self-explanatory: make a directory called “src,” go in there, then create 2 branches of the bzr repos for dotfiles and scripts.

Then, lets run two of the scripts in ~/src/scripts/.

restore_email_apps.sh is really just an easy way for me to remember which applications I use for email and its supported functions. The contents of that shell script is merely:
sudo aptitude install mutt-patched offlineimap msmtp abook
Yep, thats it.

The restore_symlinks_desktop.sh file is a little more involved. Basically, since I keep all of my dotfiles in a bzr repo called ~/src/dotfiles I need to tell the associated applications where to find those files. Well, I don’t really tell the applications, I trick them. I create symbolic links from where they are expecting the file to where I actually keep it. This is a common trick for these types of setups. In fact, if you didn’t want to use a traditional version control system to handle the versioning/syncing between computers you could store your dotfiles in your Ubuntu One folder and then they will be synced automatically. So, restore_symlinks_desktop.sh goes and creates all of those symlinks for me.

offlineimap

Now, you may have noticed that I use offlineimap with my email system. I won’t go into all of the benefits of offlineimap here, but the main ones that I enjoy are: local really fast storage of my mail (vs online) and brain-less email backups (its already backed up already).

However, using offlineimap creates one speed bump in this email setup; you need to redownload all of your email again, which, if you are like me and don’t delete anything (just put it in an Archive folder) then this can take a long time depending on your mail server. And, if you use gmail’s servers, they can lock you out of IMAP access if you download too much. Luckily, offlineimap is pretty robust and allows you to get around that problem fairly easily.

Note: I have only done this with offlineimap using the MailDir format so I don’t know what would be different for the other formats. Also, I haven’t seen any other guides like this online which is A) why I wrote this out and B) makes me wonder if this is bad for some reason [I did this over 5 days ago and haven't lost any email yet].

Anyway, here is what I did:

1) copy my .maildb folder from one computer which had a recent sync to my new laptop using a local LAN. I did this using rsync so it would compress during transit, because even at 10 megs a second, 6 gigs is a lot to transfer.
rsync -e 'ssh -ax' -auhvz --progress greg@192.168.1.102:/home/greg/.maildb/ /home/greg/.maildb/

2) rename the MailDir format email files to reflect the new computer’s hostname. The files look like this:
1264133355_0.24102.alexandria,U=3422,FMD5=e727b00944f81e1d0a95c12886ac4641:2,S
That “alexandria” part is the hostname of my desktop. So, I need to change that to “zen” so it looks like this:
1264133355_0.24102.zen,U=3422,FMD5=e727b00944f81e1d0a95c12886ac4641:2,S
But I need to do that for ALL of my email. find and xargs to the rescue!
find . -name '*' -type f -print0 | xargs -0 rename -v 's/alexandria/zen/'

3) setup offlineimap; ie: install and configure your email accounts. This is already done for me by running those two scripts above. Note: Make sure your ~/.offlineimap folder is empty (ie: don’t copy it over from your old computer).

3b) A big change from my last email setup is that I am now checking one of my mail account (grossmeier.net) over ssh. Basically, offlineimap ssh’s to the server, then runs an imap server (in my case, rimap). I followed the instructions set out here: http://eagain.net/blog/2007/02/09/imap-over-ssh.html (changing the pertinent parts about the imap service on the server, of course).

4) run offlineimap. You should see it SCREAM through your email archive as it adds the metadata to its .offlineimap folder from the local maildir you just copied to the machine.

There ya go. Now when you get a new machine and you use offlineimap to sync your email you don’t need to re-download all of your email again.

I finally broke down and purchased a new laptop.

My old trusty companion was a System76 Darter Ultra (revision 2). It was a great little machine with a 12.1″ screen, as much processor and RAM (4gig after purchase) as I need, and Ubuntu pre-installed. I can’t say enough about the people at System76; a small company out of Denver, Colorado where you know pretty much the whole company by first name. Carl is the owner; Tom is the guy answering your support questions; Erik is in charge of sales; and one of their daughters or nieces helps stuff envelopes on the weekends. I’m sure there are more people involved, but those three people I have had personal communication with via email and/or phone and every time it was great.

Why am I telling you how much I love System76? Because I feel bad not sticking with them for my latest purchase. I now have a Lenovo X200s. Another 12.1″ screen machine that is really light and works great.

Why did I switch my supplier? Honestly, because I wanted a machine that had a little better structural build quality (my Darter was a MSI barebones and started having structural issues near its 3rd year of life) and the X200s is just a little “better” (lighter and higher resolution).

So far, everything is working great out of the box*! The best part is I had a 160gig Intel SSD waiting to be put in it when it arrived. As soon as I replaced the harddrive I installed Lucid using a USB flash drive which was extremely quick. What that means is I never booted the original drive and thus never even saw the Windows 7 logo. :)

This machine, with the SSD drive, simply just screams. Best upgrade investment I ever made.

Action pics:

writing this blog post at my work office:

* The only thing not working correctly is two finger scrolling using the nipple and third mouse button (there is no trackpad). I tried the adding the hal quirk identified here, but it didn’t work. Haven’t messed with it too much, really.

That’s right, we be jammin’, too.

First the details from the original ubuntu-us-mi mailing list announcment:

• What: Michigan LoCo’s Global Jam
• When: October 3rd from 12 (noon) to 6pm
• Where: SRT Solutions in Ann Arbor
• Why: To rock out on Ubuntu before the release of Karmic Koala.
• Who: YOU! Your friends! Your neighbors! And all of the wonderful members of the Michigan LoCo team.

Now, the great looking banner from Leandro Gómez:

See you there!

(sourcecode is to binary as ??? is to ppt/odp/pdf)

Ted Gould just posted to the planet with his presentation that he gave at the Desktop Summit. At the end of his post you’ll notice that he uploaded his presentation to Launchpad (at lp:~ted/presentations/2009_desktop_summit/).

I think that is a great idea! Not only does it provide the ability for the community to see what others are using for their presentations but it allows anyone to branch a presentation, which has awesome potential. Especially with the presentation format that Ted chose, SVGs. The S5 presentation format (XHTML/CSS/JS based) would also be a great candidate for easy branching and editing of presentations.

But what if you need to create presentations with others who use Powerpoint or Impress and you wanted to harness the power of a Version Control System? Old powerpoint (ppt) files are binary blobs which don’t work well in version control systems (they *work* but not *well*). Impress (odp) and new Powerpoint (pptx) files are effectively zipped archives of xml and images. However, since it is zipped, bzr treats it as a binary. I only tested with bzr but don’t foresee any of the other systems behaving any differently.

Why would you want to use a VCS for your presentation files? Especially a DVCS like bzr/git/hg? COLLABORATION!

Some of you may know that I am currently working with Open.Michigan, a project at the University of Michigan that enables the creation of Open Educational Resources (OER). OER is effectively a broader term for the concept of Open CourseWare. Basically, everything used in education is a resource, not just presentations, and thus is useful for others to see, use, and remix. If you are curious to see what kinds of things we produce, see our Educommons installation.

Back to the topic at hand though: presentations and DVCS.

One of the major areas that the OER community could greatly improve upon is the area of remixing; taking the openly licensed materials and using them, adding new material, and creating something original. Remixing, in general, is enabled by having access to the source files of the material being worked with. Sure, you can use a PDF or a mp3 in a remix, but it is usually better to have the original .odt or multitrack file to work from. This is why Open.Michigan provides to the public the ppt files along with the pdfs of the presentations created through the OER program.

But lets leverage some of the tried and true methods of the FLOSS community in the OER community. One of the biggest and most fundamental benefits of the FLOSS world is that everyone has access to the source code, and can easily get it, edit it, and (hopefully) compile a new version of the program; effectively a “remix.” How does the FLOSS community lower the barriers and increase efficiency for that workflow? We provide public access to code repositories, instructions on building the software (documentation), and a bug tracker to inform what needs to be worked on next.

I want to mirror much of that to the OER community. One of the first things that needs to happen is to provide an easy way to manage multiple versions of a single resource (eg: presentation, video/audio, book). A VCS seems like the obvious choice. But there must be a better way than just managing binary blobs, right?

That is the part that I need to figure out next: how to utilize the power of a DVCS in this genre. Then I can move on to figuring out what a bug tracker for OER would look like (and if it is even needed). The documentation is actually already there, at least for Open.Michigan.

Do you have any ideas?

I am a proud member of the Ubuntu Cyclist group, just like Rich. So, today I decided to make the best of the beautiful weather and go for a ride on my Miyata 310.

Here is the route I took:

View Bike/Walk Route June 28, 2009 in a larger map

Those blue markers all correspond to places I stopped to take pictures.

This ride would have been a great time, but with 3.4 miles to get home my front tire tube popped. Note to other riders: when going down a hill on a dirt road and at the bottom is where the pavement begins, be sure to take that pretty easy or you are bound to pop your front tire.

Why did I title this post “My Ubuntu Cycling Sunday Ride?” What makes it Ubuntu? Well I had my with me the whole time which means I had 8 Jaunty CDs and a couple sheets of Ubuntu stickers, just in case!

Anyways, here are some photos, full set here:

Bike + Diesel Tank

Crane Treads

Swamps of Washtenaw County

Oliver the Tractor

Foraged Mullberries from a foreclosed home