PDA

View Full Version : Coral as a simple, workable alternative to Bittorrent


Katsushiro
Feb 6th, 2005, 08:47 AM
Hey guys. Long time lurker, first time poster. I've been podcasting for a short while now (just uploaded my 11th show this morning), and I've ben watching with interest the ideas people have been having about ways to conserve bandwith, since, as we all know, the more popular you get, the higher your bandwith usage can soar, and that number can rise pretty **** fast.

I've seen a lot of talk about using Bittorrent to fix it, and I even experimented a little with it during my first couple of shows, but I didn't care for it. It was wonky, support for it in the major podcatching clients is mediocre at best, and if you try and pick up a podcast with it after the first couple of days, you're SOL. Honestly, I don't think Bittorrent is the answer we're looking for.

So I kept searching, I've found commercial services like lybsin.org, which look promising, but I wasn't ready to pick it up. Then I ran across something I'd seen before on Slashdot: Coral ( http://www.scs.cs.nyu.edu/coral/ ).

In a nutshell, Coral is a distributed caching mechanism that's available for free use by anyone. Files distributed through Coral get cached in a series of servers spread all across the world, and when someone requests the file, it gets sent to them from the closest cache location, rather than coming directly from your site.

Why do I like it so much? Because of the simplicity with which it works. You don't have to make any modifications to existing podcatcher clients. Your listeners don't have to download any weird plugins or learn how to use bittorrent. They can download and subscribe as they always did, and they won't notice any difference. All you, as a podcaster, have to do to get your file Coralized, is include one little bit into your podcast url: '.nyud.net:8090'. Like so:

If the link to your podcast was:
http://www.yoursite.com/podcast.mp3
It now reads:
http://www.yoursite.com.nyud.net:8090/podcast.mp3

And that's it! Nothing to install, no need to sign up anywhere, and it works transparently with all web browsers and podcastcher clients I've tried it with. I've been distributing my last 5 or 6 podcasts through it and so far it's worked like a dream, no one's complained about slow downloads or not being able to get the file, and my bandwith's stayed under control despite a slowly growing listener userbase.

I know I can't be the only podcaster who's found Coral useful, but I haven't found anyone discussing it as a possibility, anywhere. Everyone's so focused on the Bittorrent solution (which I think might be a dead end), and I've seen no one paying attention to other possibilities, like Coral. Has anyone else here tried Coral, or not? And what do you think? For me at least, it seems to be a nearly ideal solution..

.Katsushiro.
http://www.techgnosisweb.com/

camilian
Feb 6th, 2005, 02:05 PM
Wow... this is a great solution. I am going to start playing with it. Thanks for the heads up!

Katsushiro
Feb 6th, 2005, 05:47 PM
You're welcome! You've got a substantially larger userbase than me, I'm sure (I have maybe a couple dozen people snagging my show.. your show, however, I've heard mentioned a couple of times on Adam Curry's podcast, so I'm *fairly* certain you've got more listeners. ;) ), so I look forward to hearing if this Coral system behaves well under a larger load and with a wider userbase trying it out. Good luck!

One thing that Coral *doesn't* solve, by the way, is the original storage. You still need to upload your file somewhere in the first place, so it will still be sitting there in yoru webserver taking up space, and that's neccesary, because if enough time passes and no one requests your podcast, Coral will eventually flush it out of its cache network. And if a couple of months down the line someone requests the file again, Coral's going to need to find that original file on your webserver so it can reinsert it into its cache once more.

However, Coral does work with any URL, so if you do find a solution to hosting space problems, you can always add the '.nyud.net:8090' to its URL, and Coral will work with it. :)

Another possible sticking point is for tracking purposes. Since the actual request for the file is sent to the Coral network, and not directly to you, it may be harder for you to track the amount of downloads/subscribers. Some sort of PHP redirect might be in order there, but then you can run into problems with podcatcher clients that don't deal well with redirects. Personally, I don't care too much about my stats, the only thing I keep an eye on them for is to make sure I don't go over my bandwith limit. :) But if you're interested in tracking your users, then that's something you might want to take into consideration.

Katsushiro
Feb 11th, 2005, 09:06 AM
Just a quick note on two major podcasts that are using Coral to distribute their files:

OpenPodcast.org

and

Leo Laporte's Show

Just checked my feeds and at the very least those two are using Coral, and they seem to be doing well under some pretty heavy downloading.

FLEB
Feb 14th, 2005, 08:59 AM
How reliable is that? I remember Coral Cache links being slow and sort of flaky on Slashdot.

Lance
Feb 14th, 2005, 01:11 PM
This seems really interesting. Are there any other podcasters who've tried this?

camilian
Feb 14th, 2005, 04:37 PM
Fleb, I tried it with a test file. Some of the times it is as fast as my server, sometimes MUCH slower. You are only as fast as the "share" the user is linked to. I am going to play a little more with it, might not be the perfect solution for everyone, but would sure help those with little bandwidth.

slapcast
Feb 15th, 2005, 07:35 AM
>Has anyone else here tried Coral, or not? And what do you think?

First, let me start by saying Coral is a fabulous service and probably close to perfect for individual podcasters. I've exchanged some messages with Mike, the project lead, and he is very supportive of podcasting.

But for me it was not a good solution. Slapcast.com just launched so we (no surprise) have a small userbase with just a few downloaders per podcast. In that scenario, using Coral it can sometimes take a couple of minutes to download a tiny file of just 500K or so (these were phone podcasts). That was unacceptable for me because it didn't provide a good user experience.

As Knuth says: Premature optimisation is the root of all evil. So if bandwidth is not a problem for you yet, don't use Coral yet. But try it out when you get popular if you are having bandwidth issues.

Cookiepuss
Feb 15th, 2005, 11:40 AM
It has come a long way since I first looked at it. I did some test with the site fallujah.us since they use it for big (130MB+) video files. As long as you are in the US it seems to push at 50Kps to 150Kbps whih is good. But if I use a European or other proxy to test the caching on "the other side of the pond" it is a lot slower.

So I think it is good if you are a US based podcaster.

FLEB
Feb 25th, 2005, 09:32 AM
I'm wondering if it isn't weighted toward larger files, since I too have had more problems and slowdowns on smaller files. It might just naturally have very high latency, though, which would make slow downloads proportionally slower.

Actually, that would make sense, since the initial processing and "set-up" time would be a significant amount.

Cookiepuss
Feb 25th, 2005, 10:20 AM
I poseted a more detailed answer after more testing HERE (http://www.podcastalley.com/phpBB2/viewtopic.php?p=1421#1421)

pzarquon
Mar 10th, 2005, 11:46 PM
I've been experimenting it with my tiny podcast. The first pulldown (i.e. the file has never been through the Coral system) is like molasses, but it seems as they day progresses and use by subscribers increases, so does the performance of the network. I don't know if I'm just imagining this, though.

I make sure my site features links to both the Coral cache and the local file, but I list the Coral cache first in the post and as the first enclosure in the feed. No complaints yet, but pretty much my listenership is my wife and two of my friends. (Actually, maybe a couple hundred pulldowns over several days.)

Ryan
HawaiiUP.com (http://www.hawaiiup.com) - Podcasting in paradise.

Cookiepuss
Mar 11th, 2005, 01:41 AM
That is what I was sayng that once the cache works its way around to numerous servers it is then distributed enough to speed up downloads.

pzarquon -

Your site looks good, but I noticed your Skype link is broken. You have http:// in front of the callto:// . . .

The funny part is in Firefox if you click on it, it takes you to the Secunia advisory for the Skype callto buffer overflow:
http://secunia.com/advisories/13191/

;)

pzarquon
Mar 11th, 2005, 06:51 AM
Your site looks good, but I noticed your Skype link is broken. You have http:// in front of the callto:// . . .D'oh! The downsides of using an automated CMS. WordPress puts that in there, without asking me. I have to keep remembering to go in to hack it out. Thanks for catching that. And thanks for the visit!

Cookiepuss
Mar 11th, 2005, 12:44 PM
D'oh! The downsides of using an automated CMS. WordPress puts that in there, without asking me. I have to keep remembering to go in to hack it out. Thanks for catching that. And thanks for the visit!

If you go to Options > Writing in WordPress Admin Section and uncheck "WordPress should correct invalidly nested XHTML automatically" it should remedy that. I never have an issue with that unchecked. Plus with it checked it screws a lot of stuff up for me. ;)