Messages in this thread | | | Date | Fri, 28 May 2004 15:08:09 -0400 | From | Chris Shoemaker <> | Subject | Re: ftp.kernel.org |
| |
On Fri, May 28, 2004 at 11:01:19AM -0400, Theodore Ts'o wrote: > On Fri, May 28, 2004 at 02:21:41AM -0400, Chris Shoemaker wrote: > > > Agreed - fmirror is so much more reliable than rsync (imho) that it makes > > > rsync into a worst-case option for retrieving files. > > > > bug reports to rsync@lists.samba.org are appreciated... > > > > The main problem with rsync that I can see is that it is fairly heavy > weight on the server, so many servers (including mirrors.kernel.org) > have a maximum number of connections set to something pathetically > small, like say 5 connections.
Do you mean w.r.t. memory usage or storage i/o? I know that creating the file list can take up a lot of memory for large lists.
Five connections seems pretty low. I've never personally hit any connection limit, and I make moderate use of rsync. On the server side I know of several rsync servers offering >1 million files. Not sure how hard they work, but they're highly available.
> > I remember Tridge trying to get someone to implement checksum caching > for rsync servers some 4+ years ago, which would surely help. Did > that ever get done? If so, convincing the server admins that it's OK > to up the maximum number of rsync connections would be the next step. > > - Ted
I'm sure there are some things that can be done to make rsync lighter-weight. Checksums aren't cached, but the problem is, the checksum seed is varible, so that might not be the best optimization.
Overall, I'd have to disagree with the parent-post saying that rsync is worst-case option. It's not perfect, but I much prefer rsync over fmirror. I think it's more convenient and, although I have no rigorous measurements, but it seems faster, to boot.
-chris - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |