Kaleidoscribe

I used to have a YouTube account to host my public videos. (It still exists, although most of its contents have been unlisted.) The same account was what I used to subscribe to other channels and watch YouTube content. In an effort to disentangle my daily life from Google and other cloud service providers, I decided to stop using my YouTube account and replace it with a suitable feed for the YouTube channels I follow.

It turns out that YouTube still offers RSS feeds for channels as well as playlists. Just funneling those into my feed reader would get me 80% to my goal, but I also kinda wanted information like video length and video thumbnails shown on my video feed, so I decided to implement it as a custom software project.

Kaleidoscribe is a self-contained Python 3 application that uses SQLite for storage. It supports multiple user accounts and has password-based logins and session management. YouTube feeds are updated in the background. Videos on the feed show a button panel upon mouse hover where you can mark them as “Watched,” “Watch later” or “Not interested.” From the menu bar on the left side of the page you can access your subscription feed, your “Watch later” list, your watch history and your subscription management page. You can subscribe to YouTube channels and playlists.

Video information is pulled from YouTube's RSS feeds first and then enhanced with some additional scraping of video pages. Video thumbnails are fetched and stored locally. That means that even if a video is removed from YouTube, its metadata remains accessible to you in Kaleidoscribe.

If you approach YouTube purely as a platform to subscribe to channels and playlists you enjoy, Kaleidoscribe replicates that entire experience. Individual video links lead to the video page on YouTube. There is no need to be logged into a YouTube/Google account, and if you additionally block YouTube from storing cookies, it will be unable to build a profile of your watching habits.

I started this project with a strong motivation to eventually make it public, and took some time to come up with a name that would be reasonably marketable and not already in use by anything else. The showstopper in that regard was YouTube's rate limiting. Understandably but still annoyingly, YouTube doesn't particularly want people scraping their web page and building custom tools around it, so I need to be very careful about my scraping strategy to not get my server rate-limited, which has happened a few times.

I came up with a dynamic updating strategy that refreshes channels with frequent uploads more often than channels which upload rarely. Even with that though, it can take a day or two for new videos to show up on my feed. Personally that doesn't bother me all that much, but I don't think general audiences would accept it. Also, YouTube customizes pages by IP-based geolocation, which means that content which is unavailable in my country is also unavailable to Kaleidoscribe.

For these and other reasons (not least of which is that all that web scraping may technically not entirely jive with YouTube's terms of service), it has remained a private project after all.