lkml.org 
[lkml]   [1996]   [Mar]   [27]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
SubjectDEBUG_KMALLOC reinstated.
Date
From


Hi,

To be able to see what kmalloc is doing, I've added the code that allows
kmalloc to report its statistics through the proc filesystem.

To be able to find out where the kmalloced memory is used for, I've
reinstated DEBUG_MALLOC. This too reports through a file in /proc.

Lately someone was trying to find out how much memory kmalloc wastes
in not-completely-full pages and in rounding up to (almost) powers of
two. This is information that you can easily gather from the information
that this patch presents in /proc.

If you want to know where kmalloc is called from, and how much memory is
allocated there, you can enable DEBUG_MALLOC and then there is a report
in /proc that tells you when kmalloc was called for how much memory and
what the current contents are of the first 16 bytes.

The information in /proc/dmalloc is:
<time> <filename> <linenumber> <bytesallocated> <dump ....>
Time is just the number of jiffies since system boot. filename/linenumber
are the calling filename and linenumber. Sadly the filesystems have
several files with the same name in different directories. However if
you guess wrong, it is unlikely that there is a call to kmalloc on
the same line in a different file.....

I think this code is stable: I've never experienced a crash due to this
code yet..... I haven't had the time to get 1.3.78(++ ?) yet. This
patches cleanly into a 1.3.74 and a 1.3.77 tree, I expect it will also
go cleanly into newer kernels.

Comments are welcome.

Roger Wolff.

P.S: The ncr 53c7,8xx driver sadly takes the address of kfree. I can't
keep DEBUG_MALLOC transparent for this case. Anybody know why this is
necessary? (I've found only one place where the kfree call is finally
placed....)
P.S2: kmalloc/kfree change arguments and names when DEBUG_MALLOC is
turned on or off. This to ensure that you will get linking errors if
you try to mix them.... (modules will need to be recompiled if they use
kmalloc/kfree)

------------------------------------------------------------------------
diff -ur linux-orig/CREDITS linux/CREDITS
--- linux-orig/CREDITS Sun Mar 24 23:35:11 1996
+++ linux/CREDITS Sun Mar 24 23:47:41 1996
@@ -1246,7 +1246,8 @@
S: Finland

N: Roger E. Wolff
-E: wolff@dutecai.et.tudelft.nl
+E: R.E.Wolff@BitWizard.nl
+W: http://www.BitWizard.nl/wolff/
D: Written kmalloc/kfree
S: Oosterstraat 23
S: 2611 TT Delft
diff -ur linux-orig/Documentation/Configure.help linux/Documentation/Configure.help
--- linux-orig/Documentation/Configure.help Sun Mar 24 23:35:05 1996
+++ linux/Documentation/Configure.help Sun Mar 24 23:55:15 1996
@@ -2907,6 +2907,12 @@
enabled "Kernel profiling support", you must be a kernel hacker and
hence you know what this is about :-)

+Debug kmalloc/kfree
+CONFIG_DEBUG_MALLOC
+ This allows you to view which kernel source files allocate memory.
+ The results can be viewed in /proc/dmalloc. It is mostly interesting
+ to kernel developers that want to find a memory leak.
+
ISDN subsystem
CONFIG_ISDN
This allows you to use an ISDN-card for networking connections and
diff -ur linux-orig/arch/i386/config.in linux/arch/i386/config.in
--- linux-orig/arch/i386/config.in Mon Feb 26 10:58:03 1996
+++ linux/arch/i386/config.in Wed Mar 20 20:16:40 1996
@@ -97,7 +97,7 @@
mainmenu_option next_comment
comment 'Kernel hacking'

-#bool 'Debug kmalloc/kfree' CONFIG_DEBUG_MALLOC
+bool 'Debug kmalloc/kfree' CONFIG_DEBUG_MALLOC
bool 'Kernel profiling support' CONFIG_PROFILE
if [ "$CONFIG_PROFILE" = "y" ]; then
int ' Profile shift count' CONFIG_PROFILE_SHIFT 2
diff -ur linux-orig/drivers/scsi/53c7,8xx.c linux/drivers/scsi/53c7,8xx.c
--- linux-orig/drivers/scsi/53c7,8xx.c Fri Mar 1 06:50:50 1996
+++ linux/drivers/scsi/53c7,8xx.c Wed Mar 20 21:28:31 1996
@@ -257,6 +257,11 @@
};
#endif

+#ifdef CONFIG_DEBUG_MALLOC
+#define kfree deb_kfree
+#endif
+
+
static int check_address (unsigned long addr, int size);
static void dump_events (struct Scsi_Host *host, int count);
static Scsi_Cmnd * return_outstanding_commands (struct Scsi_Host *host,
@@ -6367,7 +6372,12 @@
*/
cmd->next = NULL;
if (cmd->free)
+ /* Why is a pointer to kfree needed here???? */
+#ifdef CONFIG_DEBUG_MALLOC
+ cmd->free (__FILE__,__LINE__,cmd->real);
+#else
cmd->free ((void *) cmd->real, cmd->size);
+#endif
}
if (hostdata->num_cmds)
printk ("scsi%d : leaked %d NCR53c7x0_cmd structures\n",
diff -ur linux-orig/fs/proc/array.c linux/fs/proc/array.c
--- linux-orig/fs/proc/array.c Sun Mar 24 23:35:12 1996
+++ linux/fs/proc/array.c Sun Mar 24 23:33:27 1996
@@ -54,8 +54,10 @@
#define LOAD_INT(x) ((x) >> FSHIFT)
#define LOAD_FRAC(x) LOAD_INT(((x) & (FIXED_1-1)) * 100)

-#ifdef CONFIG_DEBUG_MALLOC
+/* Malloc externals.... */
int get_malloc(char * buffer);
+#ifdef CONFIG_DEBUG_MALLOC
+int read_dmalloc (struct file * file, char * buf, int count);
#endif


@@ -989,10 +991,8 @@
case PROC_VERSION:
return get_version(page);

-#ifdef CONFIG_DEBUG_MALLOC
case PROC_MALLOC:
return get_malloc(page);
-#endif

#ifdef CONFIG_MODULES
case PROC_MODULES:
@@ -1153,10 +1153,18 @@
if (count < 0)
return -EINVAL;

- switch (type) {
- case PROC_PID_MAPS:
- return read_maps(pid, file, buf, count);
- }
+ if (pid)
+ switch (type) {
+ case PROC_PID_MAPS:
+ return read_maps(pid, file, buf, count);
+ }
+ else
+ switch (type) {
+#ifdef CONFIG_DEBUG_MALLOC
+ case PROC_DMALLOC: /* Not PID */
+ return read_dmalloc(file, buf, count);
+#endif
+ }
return -EINVAL;
}

diff -ur linux-orig/fs/proc/inode.c linux/fs/proc/inode.c
--- linux-orig/fs/proc/inode.c Wed Feb 21 09:30:09 1996
+++ linux/fs/proc/inode.c Wed Mar 20 20:15:41 1996
@@ -180,6 +180,12 @@
inode->i_op = &proc_kcore_inode_operations;
inode->i_size = (MAP_NR(high_memory) << PAGE_SHIFT) + PAGE_SIZE;
break;
+#ifdef CONFIG_DEBUG_MALLOC
+ case PROC_DMALLOC:
+ inode->i_mode = S_IFREG | S_IRUGO;
+ inode->i_op = &proc_arraylong_inode_operations;
+ break;
+#endif
case PROC_PROFILE:
inode->i_mode = S_IFREG | S_IRUGO | S_IWUSR;
inode->i_op = &proc_profile_inode_operations;
diff -ur linux-orig/fs/proc/root.c linux/fs/proc/root.c
--- linux-orig/fs/proc/root.c Thu Feb 15 13:37:11 1996
+++ linux/fs/proc/root.c Wed Mar 20 20:15:41 1996
@@ -300,9 +300,14 @@
proc_register(&proc_root, &proc_scsi);
proc_register(&proc_root, &proc_sys_root);

-#ifdef CONFIG_DEBUG_MALLOC
proc_register(&proc_root, &(struct proc_dir_entry) {
PROC_MALLOC, 6, "malloc",
+ S_IFREG | S_IRUGO, 1, 0, 0,
+ });
+
+#ifdef CONFIG_DEBUG_MALLOC
+ proc_register(&proc_root, &(struct proc_dir_entry) {
+ PROC_DMALLOC, 7, "dmalloc",
S_IFREG | S_IRUGO, 1, 0, 0,
});
#endif
diff -ur linux-orig/include/linux/malloc.h linux/include/linux/malloc.h
--- linux-orig/include/linux/malloc.h Fri Mar 1 09:37:20 1996
+++ linux/include/linux/malloc.h Wed Mar 20 20:55:55 1996
@@ -1,11 +1,24 @@
#ifndef _LINUX_MALLOC_H
#define _LINUX_MALLOC_H

+#include <linux/config.h>
#include <linux/mm.h>

-void * kmalloc(unsigned int size, int priority);
+#ifdef CONFIG_DEBUG_MALLOC
+#define kmalloc(a,b) deb_kmalloc(__FILE__,__LINE__,a,b)
+#define kfree(a) deb_kfree(__FILE__,__LINE__,a)
+
+void *deb_kmalloc (const char *file, unsigned short line, size_t size, int priority);
+void deb_kfree (const char *file, unsigned short line,void *obj);
+
+
+#else /* !debug */
+
+void *kmalloc(unsigned int size, int priority);
void kfree(void * obj);

-#define kfree_s(a,b) kfree(a)
+#endif

#endif /* _LINUX_MALLOC_H */
+
+#define kfree_s(a,b) kfree(a)
diff -ur linux-orig/include/linux/proc_fs.h linux/include/linux/proc_fs.h
--- linux-orig/include/linux/proc_fs.h Wed Mar 20 20:22:07 1996
+++ linux/include/linux/proc_fs.h Wed Mar 20 20:55:55 1996
@@ -25,6 +25,7 @@
PROC_NET,
PROC_SCSI,
PROC_MALLOC,
+ PROC_DMALLOC,
PROC_KCORE,
PROC_MODULES,
PROC_STAT,
diff -ur linux-orig/kernel/ksyms.c linux/kernel/ksyms.c
--- linux-orig/kernel/ksyms.c Sun Mar 24 23:35:07 1996
+++ linux/kernel/ksyms.c Sun Mar 24 23:33:18 1996
@@ -158,8 +158,13 @@
/* internal kernel memory management */
X(__get_free_pages),
X(free_pages),
+#ifdef CONFIG_DEBUG_MALLOC
+ X(deb_kmalloc),
+ X(deb_kfree),
+#else
X(kmalloc),
X(kfree),
+#endif
X(vmalloc),
X(vremap),
X(vfree),
diff -ur linux-orig/mm/kmalloc.c linux/mm/kmalloc.c
--- linux-orig/mm/kmalloc.c Sun Mar 24 23:35:07 1996
+++ linux/mm/kmalloc.c Sun Mar 24 23:33:18 1996
@@ -4,6 +4,7 @@
* Copyright (C) 1991, 1992 Linus Torvalds & Roger Wolff.
*
* Written by R.E. Wolff Sept/Oct '93.
+ * R.E.Wolff@BitWizard.nl
*
*/

@@ -49,6 +50,12 @@
unsigned long ubh_length;
struct block_header *fbh_next;
} vp;
+#ifdef CONFIG_DEBUG_MALLOC
+ struct block_header *dbh_next, *dbh_prev;
+ long dbh_time;
+ const char * dbh_file;
+ short dbh_line;
+#endif
};


@@ -56,6 +63,9 @@
#define bh_next vp.fbh_next
#define BH(p) ((struct block_header *)(p))

+#ifdef CONFIG_DEBUG_MALLOC
+static struct block_header first_bh = {0,{0},&first_bh,&first_bh,0,NULL,0};
+#endif

/*
* The page descriptor is at the front of every page that malloc has in use.
@@ -173,7 +183,11 @@
return -1;
}

+#ifndef CONFIG_DEBUG_MALLOC
void *kmalloc(size_t size, int priority)
+#else
+void *deb_kmalloc (const char *file, unsigned short line, size_t size, int priority)
+#endif
{
unsigned long flags;
unsigned long type;
@@ -202,8 +216,13 @@
if (intr_count && priority != GFP_ATOMIC) {
static int count = 0;
if (++count < 5) {
- printk("kmalloc called nonatomically from interrupt %p\n",
- __builtin_return_address(0));
+ if (__builtin_return_address(0) < (void *) 0x1000)
+ printk ("kmalloc called nonatomically.\n"
+ "recompile kmalloc.c without -fno-frame-pointer "
+ "for valid return address.\n");
+ else
+ printk("kmalloc called nonatomically from interrupt %p\n",
+ __builtin_return_address(0));
priority = GFP_ATOMIC;
}
}
@@ -267,9 +286,18 @@
page->nfree--;
if (!page->nfree)
*pg = page->next;
- restore_flags(flags);
+#ifdef CONFIG_DEBUG_MALLOC
+ p->dbh_file = file;
+ p->dbh_line = line;
+ p->dbh_time = jiffies;
+ p->dbh_next = &first_bh;
+ p->dbh_prev = first_bh.dbh_prev;
+ first_bh.dbh_prev->dbh_next = p;
+ first_bh.dbh_prev = p;
+#endif
sizes[order].nmallocs++;
sizes[order].nbytesmalloced += size;
+ restore_flags(flags);
p->bh_flags = type; /* As of now this block is officially in use */
p->bh_length = size;
#ifdef SADISTIC_KMALLOC
@@ -278,7 +306,12 @@
return p + 1; /* Pointer arithmetic: increments past header */
}

+
+#ifndef CONFIG_DEBUG_MALLOC
void kfree(void *ptr)
+#else
+void deb_kfree (const char *file, unsigned short line, void *ptr)
+#endif
{
int size;
unsigned long flags;
@@ -312,6 +345,10 @@
#endif
save_flags(flags);
cli();
+#ifdef CONFIG_DEBUG_MALLOC
+ p->dbh_next->dbh_prev = p->dbh_prev;
+ p->dbh_prev->dbh_next = p->dbh_next;
+#endif
p->bh_next = page->firstfree;
page->firstfree = p;
page->nfree++;
@@ -327,7 +364,7 @@
struct page_descriptor *tmp = *pg;
if (!tmp) {
printk("Ooops. page %p doesn't show on freelist.\n", page);
- break;
+ goto aargh; /* Better waste a page than crash the system */
}
if (tmp == page) {
*pg = page->next;
@@ -338,7 +375,127 @@
sizes[order].npages--;
free_pages((long) page, sizes[order].gfporder);
}
+aargh:
sizes[order].nfrees++;
sizes[order].nbytesmalloced -= size;
restore_flags(flags);
}
+
+
+
+
+int get_malloc (char *buffer)
+{
+ int i;
+ int len=0;
+
+ len += sprintf (buffer+len,"order size nblk nmalocs nfrees totbyt totpgs\n");
+ for (i = 0;(len < 4000) && sizes[i].size;i++)
+ len += sprintf (buffer+len," %2d %-6d %4d %6d %6d %9d %5d\n",
+ i,
+ sizes[i].size,
+ sizes[i].nblocks,
+ sizes[i].nmallocs,
+ sizes[i].nfrees,
+ sizes[i].nbytesmalloced,
+ sizes[i].npages);
+ return len;
+}
+
+
+
+#ifdef CONFIG_DEBUG_MALLOC
+
+#define LINE_SHIFT 12
+
+int read_dmalloc (struct file * file, char * buf, int count)
+{
+ struct block_header *bh;
+ int lineno;
+ int i,len;
+ char line[100]; /* 52 chars misc stuff + 48 for the dump */
+ unsigned char *chp;
+ char *destptr;
+
+ if (count == 0)
+ return 0;
+
+ lineno = file->f_pos >> LINE_SHIFT;
+
+ /* Quickly go to the right part of the file... */
+ for (bh = first_bh.dbh_next,i=0;
+ (bh != &first_bh) && (i < lineno);
+ bh=bh->dbh_next,i++)
+ /* Nothing */;
+
+ destptr = buf;
+
+ /*
+ * I object to using "column" around there. Anything may have
+ * happened in between, which might have altered whatever we were
+ * looking at. So if you are for example at line 5 column 34 a task
+ * switch may change whatever would be displayed on line 5. You
+ * would then see the first half of one part, and the second half
+ * of another. I consider this VERY bad. You should realize that
+ * you are always seeing a snapshot. If "top" would occasionally
+ * show a "sleep" using 50% cpu and lots of memory you would have a
+ * reason to be upset. A line is a piece of data that is internally
+ * consistent.
+ *
+ * Although the "maps" (which was used as an example) are more
+ * stable than kmalloced memory areas, I'd rather not risk
+ * this confusion.....
+ */
+
+ /*
+ * Solution used:
+ * If the next line would give more than your buffer can hold,
+ * the next line is not returned.
+ * Reading 4096 bytes is likely to give you around 4050 bytes.
+ *
+ * Gnu fileutils like dd, copy and cat behave correctly under these
+ * conditions. "less" doesn't like /proc files (cat <file>|less)
+ */
+
+ /*
+ * All the link-following here works under the condition that
+ * not the last malloced block of a page is freed. This could then
+ * free a page for reuse, which may overwrite the dbh_next pointer.
+ *
+ * This means is that it might be possible for this routine
+ * to follow a bad link. This would (VERY occasionally) result in
+ * a kenel oops, which would kill the process reading /proc/dmalloc .
+ * I think this is preferable to doing this all with interrupts
+ * off. The code for the "maps" file can assume that only the
+ * current process can modify its own maps. Here we have to assume
+ * any interrupt can cause a kfree.....
+ *
+ * In short: Problem signalled, consequences evaluated,
+ * and decided that the cure would be worse than the illness.
+ * Try to convince me otherwise.....
+ */
+ while (bh && (bh != &first_bh)) {
+ len = sprintf (line,"%ld %s %d %ld",
+ bh->dbh_time,
+ bh->dbh_file,
+ bh->dbh_line,
+ bh->bh_length);
+ chp=(unsigned char *)(bh+1);
+ for (i=0;i<16;i++)
+ len +=sprintf (line+len, " %02x", chp[i]);
+ len += sprintf (line+len, "\n");
+
+ if (len > count) break;
+
+ memcpy_tofs (destptr,line,len);
+ destptr += len;
+ count -= len;
+ lineno ++;
+ bh = bh->dbh_next;
+ }
+ file->f_pos = lineno << LINE_SHIFT;
+
+ return destptr-buf;
+}
+#endif
+


\
 
 \ /
  Last update: 2005-03-22 13:36    [W:0.047 / U:0.340 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site