[lkml]   [1996]   [Dec]   [25]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectUnable to allocate large buffers for tape I/O (?)

Greetings and Merry Christmas to all!

I have been having a problem running the following small/trivial program
which simply attempts to read physical blocks from some given device (in
particular, a tape drive) then show what the actual size of the previously
written physical blocks are.

The particular type of equipment I am using to do this includes:

Pentium P-100 (ASUS motherboard)
16 MB main memory
ASUS SC-200 SCSI controller (ncr 53x800-based card)
HP 34580A SCSI DAT tape drive

I'm having two kinds of unexpected problems when running this simple program.

The first is one that I will report in a separate message later on (only to
the linux-tape mailing list, as it seems to be some sort of a strange error
coming from my tape drive).

The second type of problem however seems to me to perhaps indicate either
one or two separate problems in the linux kernel (and its manner of memory

This problem (or these two related problems) always manifest themselves as
error messages from the program below. The messages say:

recsize: error reading `/dev/st0': (75) Value too large for defined data type

(Note that the name of the program itself is `recsize'.)

I get this message (even right after a reboot, it seems) if I define the
constant `buf_size' in the program below to be anything larger than (1 << 17).
(The parameter `buf_size' is use in my calls to `read' as the size parameter...
i.e. as the number of bytes requested. Traditionally, UNIX systems allow
you to ask for some large number of bytes from a tape device, and then they
just give you back one physical record's worth of bytes, regardless of how
many bytes you actually asked for.)

Is this a known limitation of the current (2.1.16) linux kernel, i.e. that
no tape I/O which involves buffers or record sizes greater than 128KB in
size is supported?? If so, I think that is too low of a limit, and far
too restrictive, and I would like some advice on how to increase this
limit. Is this limit in any way related to the (undocumented) fixed con-
stant called NR_MEM_LISTS which is defined at or near line 37 of the
kernel file `mm/page-alloc.c'? It seems to me that it may indeed be.
(I note that that symbol, NR_MEM_LISTS, is defined to `6' and that
4KB << 6 == 128KB.)

Anyway, another equally serious problem I have found is that even when
I defined the `buf_size' constant in the program below to just (1 << 17),
I *still* get the same errno (EOVERFLOW) from the `st' SCSI tape driver
in some cases... although not right after a reboot. It seems that after
running the system for awhile (and opening several xterm windows) the
kernel is no longer willing or able to supply a 128KB I/O buffer to the
SCSI tape driver.

I do not understand enough linux kernel internals to be sure that this is
what is happening, but that is my (educated?) guess.

Can anyone confirm or refute this guess? (please?)

Anyway, assuming that this indeed what is happening, can anyone advise me
on what I might do to my kernel if I wish to insure that memory (for an
I/O buffer) will *always* be made available to the SCSI tape driver when
it requests it... at least as long as the request could in fact be satisfied
(without overwriting the kernel itself)?

To clarify, I think that if the SCSI tape driver requests some pages for
use as an I/O buffer, this request should always succeed (assuming that
there is enough physical memory on the system, minus the kernel of course)
even if it means that one or more user processes have to be swapped out in
order to make enough free pages to satisfy the request. In other words,
I think that drivers should have be given priority access to memory pages
above that afforded to mere user processes.

If I have jumped to incorrect assumptions in any of the above, please do
accept my apologies, and please do educate me.

Thanks in advance.

-- rfg

P.S. Note that my machine has 16 MB of memory. That isn't a reat deal by
today's standards, but it definitely *should be* enough to satisfy a request
for a mere 128KB for an I/O buffer!

#include <stdio.h>
#include <errno.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>

static char *pname;

main (register int argc, register char *argv[])
register int fd;

pname = strrchr (argv[0], '/');
pname = pname ? pname+1 : argv[0];

if ((fd = open (argv[1], O_RDONLY)) == -1)
fprintf (stderr, "%s: error opening `%s': %s\n",
pname, argv[1], strerror (errno));
exit (1);

for (;;)
enum { buf_size = (1 << 17) };
char buf[buf_size];
register unsigned long count;

if ((count = read (fd, buf, buf_size)) == -1)
register int e = errno;

fprintf (stderr, "%s: error reading `%s': (%d) %s\n",
pname, argv[1], e, strerror (e));
exit (1);
fprintf (stdout, "%s: record size = %lu\n", pname, count);

if (close (fd) == -1)
fprintf (stderr, "%s: error closing `%s': %s\n",
pname, argv[1], strerror (errno));
exit (1);

return 0;

 \ /
  Last update: 2005-03-22 13:37    [W:0.047 / U:1.768 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site