Messages in this thread | | | Date | Wed, 2 Aug 2000 03:14:12 -0400 (EDT) | From | Donald Becker <> | Subject | Re: 2GIG-file |
| |
>Alan Cox wrote: >> >> > Dear Al, Linus, and Alan -- 2GB liit is probably something to look at >> > expanding in the future to support these huge SQL server database files. >> >> Indeed. Thats why there are 2.2 patches and 2.4test already zapped it. I think >> the limit is now about 1Tb for a file and hit due to block layer limits
To elaborate, for most LFS implementations: Block offsets are 32 bits. The block size is 2^10 or 2^12. Thus total file size is limited to 2^42 or 2^44.
>But not for everything. The 2.2.X VFS is limited to 2GB (since the fs >drivers were coded this way). I treat the off_t field as an unsigned >long so I'm at 4GB.
The LFS kernel patches for 2.2 change the file offset to 64 bits, and add new syscalls to the kernels with 64 bit offsets.
The LFS patches I've seen (including Scyld's) don't change the block index, which is still a 32 bit value. Changing the block index to 64 bits would be a much bigger change. While many people will soon need 5GB files, very few people need a over 1TB file.
The kernel was a small fraction of the LFS work. More effort went into library changes, and the bulk of the time was spent testing (and occasionally fixing) utilities and applications. I know that people on this mailing list focus on kernel issues, just keep in mind that adding LFS is a whole-system issue.
> 2.4.X does support big files, but this guy was >asking about "commercial" linux distributions he could get his hands on, >and most folks are still on 2.2.X.
We provide a distribution with LFS and commercial support. It includes a kernel with LFS and 'bproc' (Beowulf cluster process management) support, libraries with LFS support, and utilities modified and tested with both sparse and dense large files on ext2. It's not a low-cost, mass-market distribution that can be bought at the corner computer store, but it is available.
>global across all the fs's. He has to get the data to the system >somehow, and it's a good bet he will be using a network to do it. Which >client can support copying of a 3.4GB file on Linux in the question-- >SMBFS since it's an NT server -- I don't think so? Since the VFS is >limited to 2GB, I guess he can install large FS support, then stare at >the system and wonder how to get the file
Part of the LFS enhancement is FTP client, FTP server, and 'scp' modication. As you stated, NFS version 2 and Samba will not work with e.g. 5GB file, and we don't yet support NFS version 3.
Donald Becker becker@scyld.com Scyld Computing Corporation http://www.scyld.com 410 Severn Ave. Suite 210 Beowulf Clusters / Linux Installations Annapolis MD 21403
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |