lkml.org 
[lkml]   [1999]   [Feb]   [12]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
Date
From
SubjectSCSI tape buffer allocation hitting memory allocation
We've got a Quantum DLT7000 35GB uncompressed tape drive and we're using
it to do massive amounts of data backup. We've found that the transfer
rate is much, much faster using 64KB blocks instead of smaller (4KB)
blocks, so we switched it over to use these. Unfortunately, it seems to
hit memory fragmentation occasionally and barf with this message:

Feb 12 02:56:50 alfie kernel: st: failed to enlarge buffer to 65536 bytes.

Looking at the code, it seems to do this when scsi_init_malloc() returns
NULL (fails). So, I hit alt-sysreq...Sure enough, there didn't appear to
be any large areas free. I remembered that I saw that the floppy driver
once spat out that it was resorting to virtual DMA, so I thought that
maybe I would be able to convert scsi_init_malloc() to fall back to use
vmalloc() if allocation in physical memory fails. I don't know if this
will work or not, though, recalling that DMA has some funky limitations
(has to be kept < 16MB?) I'm not sure what limitations apply to the
aic7xxx SCSI stuff, though.

I wrote a patch that at first appeared to work -- it reports that it fell
back to vmalloc(), and it seemed to get the data off the tape without
any problems. It seemed to then report that it freed as many allocations
as it had allocated with vmalloc() (good news):

Feb 12 12:03:28 alfie kernel: scsi_init_malloc(): Unable to allocate 32768 bytes of contiguous memory -- falling back on virtual allocation.
Feb 12 12:03:28 alfie last message repeated 10 times
Feb 12 12:03:28 alfie kernel: scsi_init_free(): Freeing virtual memory.
Feb 12 12:03:28 alfie last message repeated 10 times

And I was happy...But then I ran the exact same command again and the
machine froze hard (even interrupts were dead -- no sysreq). So, is this
not possible? Is there another workaround that could be done? Is there
just a dumb bug in my patch (attached)?

Simon-

| Simon Kirby | Systems Administration |
| mailto:sim@netnation.com | NetNation Communications |
| http://www.netnation.com/ | Tech: (604) 684-6892 |
diff -ur linux.orig/drivers/scsi/scsi.c linux/drivers/scsi/scsi.c
--- linux.orig/drivers/scsi/scsi.c Mon Feb 8 09:48:10 1999
+++ linux/drivers/scsi/scsi.c Fri Feb 12 09:33:35 1999
@@ -48,6 +48,7 @@
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <linux/init.h>
+#include <linux/mm.h> /* "high_memory" */

#define __KERNEL_SYSCALLS__

@@ -1884,6 +1885,18 @@
} else
retval = kmalloc(size, gfp_mask);

+ /*
+ * If either __get_free_pages() or kmalloc() failed, fall back on
+ * virtual allocation. -sim
+ */
+ if (!retval && size > PAGE_SIZE) {
+ retval = vmalloc(size);
+ printk("scsi_init_malloc(): Unable to allocate %u bytes of contiguous"
+ " memory -- falling back on virtual allocation.\n",size);
+ if (!retval)
+ printk("scsi_init_malloc(): Eek! vmalloc() also failed!\n");
+ }
+
if (retval)
memset(retval, 0, size);
return retval;
@@ -1897,15 +1910,24 @@
* page aligned data. Besides, it is wasteful to allocate
* page sized chunks with kmalloc.
*/
- if ((size % PAGE_SIZE) == 0) {
- int order, a_size;
+ /*
+ * Also, we must now check to see if we fell back on virtual
+ * allocation to allow for large allocations in the SCSI code
+ * without running into memory fragmentation problems. -sim
+ */
+ if ((unsigned int) ptr >= (unsigned int) high_memory) {
+ vfree((void *)ptr);
+ } else {
+ if ((size % PAGE_SIZE) == 0) {
+ int order, a_size;

- for (order = 0, a_size = PAGE_SIZE;
- a_size < size; order++, a_size <<= 1)
+ for (order = 0, a_size = PAGE_SIZE;
+ size < size; order++, a_size <<= 1)
;
- free_pages((unsigned long)ptr, order);
- } else
- kfree(ptr);
+ free_pages((unsigned long)ptr, order);
+ } else
+ kfree(ptr);
+ }
}

void scsi_build_commandblocks(Scsi_Device * SDpnt)
\
 
 \ /
  Last update: 2005-03-22 13:50    [from the cache]
©2003-2014 Jasper Spaans. Advertise on this site