How does iTunes 8 Genius know what sounds good together?

The rumors about iTunes 8 and it's Genius feature that automatically creates playlists from your iTunes library and suggests music to you from the iTunes store turned out to be completely true. When I first heard the rumors about this feature, I thought immediately of BeaTunes, a cross-platform third-party add-on for previous iTunes releases. Among features aimed at helping correct errors in tags for tracks imported into iTunes (such as mismatched genres or misspelled artist names), BeaTunes offers the ability to rank songs that will sound similar and likely go well together based in part on their beats per minute (an obscure tag available in iTunes that BeaTunes could populate for you if it were missing for any tracks).

Turns out that Genius has much more magic going on behind the scenes. The service actually gathers a set of information about all songs in your iTunes library, uploads it to Apple’s servers, and then returns an analysis of your library that identifies music that sounds good together. Along the way, it suggests other tracks, albums, and iTunes collections for songs (think of it as the old iTunes mini-store on steroids).

Genius is designed to update its analyzing capabilities not just by the anonymous data it collects from your iTunes library but by the data collected from all users. Theoretically, this should mean matches for everyone get better as time goes on. Privacy advocates will probably question just how anonymous this information actually is, particularly when the feature also requires an iTunes Store account/Apple ID to function. Even without that stipulation, the question of privacy could easily be raised based on a computer’s network identifying information.

What isn’t completely clear yet is just what information Apple is using to generate Genius results and how that information is being analyzed. Though the results in my initial testing are more or less spot-on, I have noticed a couple of interesting tidbits.

Genius can’t create playlists for a handful songs because it "Genius is unavailable" for them. Ironically enough, some of these songs will show results in the Genius sidebar. In a handful of other cases, the Genius Sidebar reports I'm missing songs related to an artist even though those songs exist (albeit not fully tagged) in my library. To further deepen the mystery some of the songs that show up in Genius playlists sound great next to the song used to generate the playlist (showing Genius is working pretty well) despite having misspelled artist and track names and no additional tags.

Genius definitely seems to be pulling information from more than just the expected artist, album, and genre tags as well as the more obscure beat per minute tag (which isn’t populated for anything in my library except for iTunes Store purchases). Given that each iTunes library stores a wealth of additional information about music (play counts, skip counts, ratings, playlist inclusion, and the dates a track was added and last played to name a few), Apple certainly has a great deal of information available for Genius to use – most of which seems to be encapsulated in a new file "Genius.itdb" in the folder containing an iTunes library. No doubt this file will be synced to new crop of iPods allowing them to generate Genius playlists on the fly without a computer

Whatever the magic is, it definitely seems to work pretty well (though it might prove helpful to be able to tweak some of the settings, or even know what they are) and it will be interesting to watch in the coming weeks to see the changes that occur in Genius results as Apple begins to analyze more data.