lkml.org 
[lkml]   [2008]   [May]   [14]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: POHMELFS high performance network filesystem. Transactions, failover, performance.
Jamie Lokier wrote:
> If you have a single data forwarder elected per client, then if one
> client generates a lot of traffic, you concentrate a lot of traffic to
> one network link and one CPU. Sometimes it's better to elect several
> leaders per client, and hash requests onto them. You diffuse CPU and
> traffic, but reduce opportunities to aggregate transactions into fewer
> message. It's an interesting problem, again probably with different
> optimal results for different networks.


Definitely. "several leaders" aka partitioning is also becoming
increasing paired with efforts at enhancing locality of reference. Both
Google and Amazon sort their distributed tables lexographically, which
[ideally] results in similar data being stored near each other.

A bit of an improvement over partitioning-by-hash, anyway, for some
workloads.

Jeff




\
 
 \ /
  Last update: 2008-05-14 23:41    [W:0.079 / U:0.304 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site