Messages in this thread |  | | Subject | Unable to allocate large buffers for tape I/O (?) | Date | Wed, 25 Dec 1996 15:55:32 -0800 | From | "Ronald F. Guilmette" <> |
| |
Greetings and Merry Christmas to all!
I have been having a problem running the following small/trivial program which simply attempts to read physical blocks from some given device (in particular, a tape drive) then show what the actual size of the previously written physical blocks are.
The particular type of equipment I am using to do this includes:
Pentium P-100 (ASUS motherboard) 16 MB main memory ASUS SC-200 SCSI controller (ncr 53x800-based card) HP 34580A SCSI DAT tape drive
I'm having two kinds of unexpected problems when running this simple program.
The first is one that I will report in a separate message later on (only to the linux-tape mailing list, as it seems to be some sort of a strange error coming from my tape drive).
The second type of problem however seems to me to perhaps indicate either one or two separate problems in the linux kernel (and its manner of memory allocation).
This problem (or these two related problems) always manifest themselves as error messages from the program below. The messages say:
recsize: error reading `/dev/st0': (75) Value too large for defined data type
(Note that the name of the program itself is `recsize'.)
I get this message (even right after a reboot, it seems) if I define the constant `buf_size' in the program below to be anything larger than (1 << 17). (The parameter `buf_size' is use in my calls to `read' as the size parameter... i.e. as the number of bytes requested. Traditionally, UNIX systems allow you to ask for some large number of bytes from a tape device, and then they just give you back one physical record's worth of bytes, regardless of how many bytes you actually asked for.)
Is this a known limitation of the current (2.1.16) linux kernel, i.e. that no tape I/O which involves buffers or record sizes greater than 128KB in size is supported?? If so, I think that is too low of a limit, and far too restrictive, and I would like some advice on how to increase this limit. Is this limit in any way related to the (undocumented) fixed con- stant called NR_MEM_LISTS which is defined at or near line 37 of the kernel file `mm/page-alloc.c'? It seems to me that it may indeed be. (I note that that symbol, NR_MEM_LISTS, is defined to `6' and that 4KB << 6 == 128KB.)
Anyway, another equally serious problem I have found is that even when I defined the `buf_size' constant in the program below to just (1 << 17), I *still* get the same errno (EOVERFLOW) from the `st' SCSI tape driver in some cases... although not right after a reboot. It seems that after running the system for awhile (and opening several xterm windows) the kernel is no longer willing or able to supply a 128KB I/O buffer to the SCSI tape driver.
I do not understand enough linux kernel internals to be sure that this is what is happening, but that is my (educated?) guess.
Can anyone confirm or refute this guess? (please?)
Anyway, assuming that this indeed what is happening, can anyone advise me on what I might do to my kernel if I wish to insure that memory (for an I/O buffer) will *always* be made available to the SCSI tape driver when it requests it... at least as long as the request could in fact be satisfied (without overwriting the kernel itself)?
To clarify, I think that if the SCSI tape driver requests some pages for use as an I/O buffer, this request should always succeed (assuming that there is enough physical memory on the system, minus the kernel of course) even if it means that one or more user processes have to be swapped out in order to make enough free pages to satisfy the request. In other words, I think that drivers should have be given priority access to memory pages above that afforded to mere user processes.
If I have jumped to incorrect assumptions in any of the above, please do accept my apologies, and please do educate me.
Thanks in advance.
-- rfg
P.S. Note that my machine has 16 MB of memory. That isn't a reat deal by today's standards, but it definitely *should be* enough to satisfy a request for a mere 128KB for an I/O buffer!
--------------------------------------------------------------------------- #include <stdio.h> #include <errno.h> #include <stdlib.h> #include <string.h> #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> #include <unistd.h>
static char *pname;
int main (register int argc, register char *argv[]) { register int fd;
pname = strrchr (argv[0], '/'); pname = pname ? pname+1 : argv[0];
if ((fd = open (argv[1], O_RDONLY)) == -1) { fprintf (stderr, "%s: error opening `%s': %s\n", pname, argv[1], strerror (errno)); exit (1); }
for (;;) { enum { buf_size = (1 << 17) }; char buf[buf_size]; register unsigned long count;
if ((count = read (fd, buf, buf_size)) == -1) { register int e = errno;
fprintf (stderr, "%s: error reading `%s': (%d) %s\n", pname, argv[1], e, strerror (e)); exit (1); } fprintf (stdout, "%s: record size = %lu\n", pname, count); }
if (close (fd) == -1) { fprintf (stderr, "%s: error closing `%s': %s\n", pname, argv[1], strerror (errno)); exit (1); }
return 0; }
|  |