2008/4/8 Matthew Toseland <toad@???>:
> On Tuesday 08 April 2008 12:36, Matthew Toseland wrote:
> > On Tuesday 08 April 2008 00:36, Ian Clarke wrote:
> > > http://video.google.com/videoplay?docid=-2372664863607209585
> > >
> > > He mentions Freenet's use of this technique about 10-15 minutes in,
> > > they also use erasure codes, so it seems they are using a few
> > > techniques that we also use (unclear about whether we were the direct
> > > source of this inspiration).
> >
> > They use 500% redundancy in their RS codes. Right now we use 150% (including
> > the original 100%). Maybe we should increase this? A slight increase to say
> > 200% or 250% may give significantly better performance, despite the
> increased
> > overhead...
>
> In fact, I think I can justify a figure of 200% (the original plus 100%, so
> 128 -> 255 blocks to fit within the 8 bit fast encoding limit). On average,
> in the long term, a block will be stored on 3 nodes. Obviously a lot of
> popular data will be stored on more nodes than 3, but in terms of the
> datastore, this is the approximate figure. On an average node with a 1GB
> datastore, the 512MB cache has a lifetime of less than a day; stuff lasts a
> lot longer in the store, and on average data is stored on 3 nodes (by
> design).
I think the downloader would "heal" a broken file by re-inserting the
missing FEC blocks, right?
If that is the case, I think we can use 300% (or higher) redundancy,
but only insert a random portion of them. When a downloader download
this file, he insert (some other) random blocks of FEC for this file.
Under this scheme, the inserter don't have to pay for a high bandwidth
overhead cost, while increasing the redundancy.
> [snip]
Regards,
Daniel Cheng
