Messages in this thread Patch in this message | | | Subject | DEBUG_KMALLOC reinstated. | Date | Wed, 27 Mar 1996 09:31:19 +0100 (MET) | From | (Rogier Wolff) |
| |
Hi,
To be able to see what kmalloc is doing, I've added the code that allows kmalloc to report its statistics through the proc filesystem.
To be able to find out where the kmalloced memory is used for, I've reinstated DEBUG_MALLOC. This too reports through a file in /proc.
Lately someone was trying to find out how much memory kmalloc wastes in not-completely-full pages and in rounding up to (almost) powers of two. This is information that you can easily gather from the information that this patch presents in /proc.
If you want to know where kmalloc is called from, and how much memory is allocated there, you can enable DEBUG_MALLOC and then there is a report in /proc that tells you when kmalloc was called for how much memory and what the current contents are of the first 16 bytes.
The information in /proc/dmalloc is: <time> <filename> <linenumber> <bytesallocated> <dump ....> Time is just the number of jiffies since system boot. filename/linenumber are the calling filename and linenumber. Sadly the filesystems have several files with the same name in different directories. However if you guess wrong, it is unlikely that there is a call to kmalloc on the same line in a different file.....
I think this code is stable: I've never experienced a crash due to this code yet..... I haven't had the time to get 1.3.78(++ ?) yet. This patches cleanly into a 1.3.74 and a 1.3.77 tree, I expect it will also go cleanly into newer kernels.
Comments are welcome.
Roger Wolff.
P.S: The ncr 53c7,8xx driver sadly takes the address of kfree. I can't keep DEBUG_MALLOC transparent for this case. Anybody know why this is necessary? (I've found only one place where the kfree call is finally placed....) P.S2: kmalloc/kfree change arguments and names when DEBUG_MALLOC is turned on or off. This to ensure that you will get linking errors if you try to mix them.... (modules will need to be recompiled if they use kmalloc/kfree)
------------------------------------------------------------------------ diff -ur linux-orig/CREDITS linux/CREDITS --- linux-orig/CREDITS Sun Mar 24 23:35:11 1996 +++ linux/CREDITS Sun Mar 24 23:47:41 1996 @@ -1246,7 +1246,8 @@ S: Finland N: Roger E. Wolff -E: wolff@dutecai.et.tudelft.nl +E: R.E.Wolff@BitWizard.nl +W: http://www.BitWizard.nl/wolff/ D: Written kmalloc/kfree S: Oosterstraat 23 S: 2611 TT Delft diff -ur linux-orig/Documentation/Configure.help linux/Documentation/Configure.help --- linux-orig/Documentation/Configure.help Sun Mar 24 23:35:05 1996 +++ linux/Documentation/Configure.help Sun Mar 24 23:55:15 1996 @@ -2907,6 +2907,12 @@ enabled "Kernel profiling support", you must be a kernel hacker and hence you know what this is about :-) +Debug kmalloc/kfree +CONFIG_DEBUG_MALLOC + This allows you to view which kernel source files allocate memory. + The results can be viewed in /proc/dmalloc. It is mostly interesting + to kernel developers that want to find a memory leak. + ISDN subsystem CONFIG_ISDN This allows you to use an ISDN-card for networking connections and diff -ur linux-orig/arch/i386/config.in linux/arch/i386/config.in --- linux-orig/arch/i386/config.in Mon Feb 26 10:58:03 1996 +++ linux/arch/i386/config.in Wed Mar 20 20:16:40 1996 @@ -97,7 +97,7 @@ mainmenu_option next_comment comment 'Kernel hacking' -#bool 'Debug kmalloc/kfree' CONFIG_DEBUG_MALLOC +bool 'Debug kmalloc/kfree' CONFIG_DEBUG_MALLOC bool 'Kernel profiling support' CONFIG_PROFILE if [ "$CONFIG_PROFILE" = "y" ]; then int ' Profile shift count' CONFIG_PROFILE_SHIFT 2 diff -ur linux-orig/drivers/scsi/53c7,8xx.c linux/drivers/scsi/53c7,8xx.c --- linux-orig/drivers/scsi/53c7,8xx.c Fri Mar 1 06:50:50 1996 +++ linux/drivers/scsi/53c7,8xx.c Wed Mar 20 21:28:31 1996 @@ -257,6 +257,11 @@ }; #endif +#ifdef CONFIG_DEBUG_MALLOC +#define kfree deb_kfree +#endif + + static int check_address (unsigned long addr, int size); static void dump_events (struct Scsi_Host *host, int count); static Scsi_Cmnd * return_outstanding_commands (struct Scsi_Host *host, @@ -6367,7 +6372,12 @@ */ cmd->next = NULL; if (cmd->free) + /* Why is a pointer to kfree needed here???? */ +#ifdef CONFIG_DEBUG_MALLOC + cmd->free (__FILE__,__LINE__,cmd->real); +#else cmd->free ((void *) cmd->real, cmd->size); +#endif } if (hostdata->num_cmds) printk ("scsi%d : leaked %d NCR53c7x0_cmd structures\n", diff -ur linux-orig/fs/proc/array.c linux/fs/proc/array.c --- linux-orig/fs/proc/array.c Sun Mar 24 23:35:12 1996 +++ linux/fs/proc/array.c Sun Mar 24 23:33:27 1996 @@ -54,8 +54,10 @@ #define LOAD_INT(x) ((x) >> FSHIFT) #define LOAD_FRAC(x) LOAD_INT(((x) & (FIXED_1-1)) * 100) -#ifdef CONFIG_DEBUG_MALLOC +/* Malloc externals.... */ int get_malloc(char * buffer); +#ifdef CONFIG_DEBUG_MALLOC +int read_dmalloc (struct file * file, char * buf, int count); #endif @@ -989,10 +991,8 @@ case PROC_VERSION: return get_version(page); -#ifdef CONFIG_DEBUG_MALLOC case PROC_MALLOC: return get_malloc(page); -#endif #ifdef CONFIG_MODULES case PROC_MODULES: @@ -1153,10 +1153,18 @@ if (count < 0) return -EINVAL; - switch (type) { - case PROC_PID_MAPS: - return read_maps(pid, file, buf, count); - } + if (pid) + switch (type) { + case PROC_PID_MAPS: + return read_maps(pid, file, buf, count); + } + else + switch (type) { +#ifdef CONFIG_DEBUG_MALLOC + case PROC_DMALLOC: /* Not PID */ + return read_dmalloc(file, buf, count); +#endif + } return -EINVAL; } diff -ur linux-orig/fs/proc/inode.c linux/fs/proc/inode.c --- linux-orig/fs/proc/inode.c Wed Feb 21 09:30:09 1996 +++ linux/fs/proc/inode.c Wed Mar 20 20:15:41 1996 @@ -180,6 +180,12 @@ inode->i_op = &proc_kcore_inode_operations; inode->i_size = (MAP_NR(high_memory) << PAGE_SHIFT) + PAGE_SIZE; break; +#ifdef CONFIG_DEBUG_MALLOC + case PROC_DMALLOC: + inode->i_mode = S_IFREG | S_IRUGO; + inode->i_op = &proc_arraylong_inode_operations; + break; +#endif case PROC_PROFILE: inode->i_mode = S_IFREG | S_IRUGO | S_IWUSR; inode->i_op = &proc_profile_inode_operations; diff -ur linux-orig/fs/proc/root.c linux/fs/proc/root.c --- linux-orig/fs/proc/root.c Thu Feb 15 13:37:11 1996 +++ linux/fs/proc/root.c Wed Mar 20 20:15:41 1996 @@ -300,9 +300,14 @@ proc_register(&proc_root, &proc_scsi); proc_register(&proc_root, &proc_sys_root); -#ifdef CONFIG_DEBUG_MALLOC proc_register(&proc_root, &(struct proc_dir_entry) { PROC_MALLOC, 6, "malloc", + S_IFREG | S_IRUGO, 1, 0, 0, + }); + +#ifdef CONFIG_DEBUG_MALLOC + proc_register(&proc_root, &(struct proc_dir_entry) { + PROC_DMALLOC, 7, "dmalloc", S_IFREG | S_IRUGO, 1, 0, 0, }); #endif diff -ur linux-orig/include/linux/malloc.h linux/include/linux/malloc.h --- linux-orig/include/linux/malloc.h Fri Mar 1 09:37:20 1996 +++ linux/include/linux/malloc.h Wed Mar 20 20:55:55 1996 @@ -1,11 +1,24 @@ #ifndef _LINUX_MALLOC_H #define _LINUX_MALLOC_H +#include <linux/config.h> #include <linux/mm.h> -void * kmalloc(unsigned int size, int priority); +#ifdef CONFIG_DEBUG_MALLOC +#define kmalloc(a,b) deb_kmalloc(__FILE__,__LINE__,a,b) +#define kfree(a) deb_kfree(__FILE__,__LINE__,a) + +void *deb_kmalloc (const char *file, unsigned short line, size_t size, int priority); +void deb_kfree (const char *file, unsigned short line,void *obj); + + +#else /* !debug */ + +void *kmalloc(unsigned int size, int priority); void kfree(void * obj); -#define kfree_s(a,b) kfree(a) +#endif #endif /* _LINUX_MALLOC_H */ + +#define kfree_s(a,b) kfree(a) diff -ur linux-orig/include/linux/proc_fs.h linux/include/linux/proc_fs.h --- linux-orig/include/linux/proc_fs.h Wed Mar 20 20:22:07 1996 +++ linux/include/linux/proc_fs.h Wed Mar 20 20:55:55 1996 @@ -25,6 +25,7 @@ PROC_NET, PROC_SCSI, PROC_MALLOC, + PROC_DMALLOC, PROC_KCORE, PROC_MODULES, PROC_STAT, diff -ur linux-orig/kernel/ksyms.c linux/kernel/ksyms.c --- linux-orig/kernel/ksyms.c Sun Mar 24 23:35:07 1996 +++ linux/kernel/ksyms.c Sun Mar 24 23:33:18 1996 @@ -158,8 +158,13 @@ /* internal kernel memory management */ X(__get_free_pages), X(free_pages), +#ifdef CONFIG_DEBUG_MALLOC + X(deb_kmalloc), + X(deb_kfree), +#else X(kmalloc), X(kfree), +#endif X(vmalloc), X(vremap), X(vfree), diff -ur linux-orig/mm/kmalloc.c linux/mm/kmalloc.c --- linux-orig/mm/kmalloc.c Sun Mar 24 23:35:07 1996 +++ linux/mm/kmalloc.c Sun Mar 24 23:33:18 1996 @@ -4,6 +4,7 @@ * Copyright (C) 1991, 1992 Linus Torvalds & Roger Wolff. * * Written by R.E. Wolff Sept/Oct '93. + * R.E.Wolff@BitWizard.nl * */ @@ -49,6 +50,12 @@ unsigned long ubh_length; struct block_header *fbh_next; } vp; +#ifdef CONFIG_DEBUG_MALLOC + struct block_header *dbh_next, *dbh_prev; + long dbh_time; + const char * dbh_file; + short dbh_line; +#endif }; @@ -56,6 +63,9 @@ #define bh_next vp.fbh_next #define BH(p) ((struct block_header *)(p)) +#ifdef CONFIG_DEBUG_MALLOC +static struct block_header first_bh = {0,{0},&first_bh,&first_bh,0,NULL,0}; +#endif /* * The page descriptor is at the front of every page that malloc has in use. @@ -173,7 +183,11 @@ return -1; } +#ifndef CONFIG_DEBUG_MALLOC void *kmalloc(size_t size, int priority) +#else +void *deb_kmalloc (const char *file, unsigned short line, size_t size, int priority) +#endif { unsigned long flags; unsigned long type; @@ -202,8 +216,13 @@ if (intr_count && priority != GFP_ATOMIC) { static int count = 0; if (++count < 5) { - printk("kmalloc called nonatomically from interrupt %p\n", - __builtin_return_address(0)); + if (__builtin_return_address(0) < (void *) 0x1000) + printk ("kmalloc called nonatomically.\n" + "recompile kmalloc.c without -fno-frame-pointer " + "for valid return address.\n"); + else + printk("kmalloc called nonatomically from interrupt %p\n", + __builtin_return_address(0)); priority = GFP_ATOMIC; } } @@ -267,9 +286,18 @@ page->nfree--; if (!page->nfree) *pg = page->next; - restore_flags(flags); +#ifdef CONFIG_DEBUG_MALLOC + p->dbh_file = file; + p->dbh_line = line; + p->dbh_time = jiffies; + p->dbh_next = &first_bh; + p->dbh_prev = first_bh.dbh_prev; + first_bh.dbh_prev->dbh_next = p; + first_bh.dbh_prev = p; +#endif sizes[order].nmallocs++; sizes[order].nbytesmalloced += size; + restore_flags(flags); p->bh_flags = type; /* As of now this block is officially in use */ p->bh_length = size; #ifdef SADISTIC_KMALLOC @@ -278,7 +306,12 @@ return p + 1; /* Pointer arithmetic: increments past header */ } + +#ifndef CONFIG_DEBUG_MALLOC void kfree(void *ptr) +#else +void deb_kfree (const char *file, unsigned short line, void *ptr) +#endif { int size; unsigned long flags; @@ -312,6 +345,10 @@ #endif save_flags(flags); cli(); +#ifdef CONFIG_DEBUG_MALLOC + p->dbh_next->dbh_prev = p->dbh_prev; + p->dbh_prev->dbh_next = p->dbh_next; +#endif p->bh_next = page->firstfree; page->firstfree = p; page->nfree++; @@ -327,7 +364,7 @@ struct page_descriptor *tmp = *pg; if (!tmp) { printk("Ooops. page %p doesn't show on freelist.\n", page); - break; + goto aargh; /* Better waste a page than crash the system */ } if (tmp == page) { *pg = page->next; @@ -338,7 +375,127 @@ sizes[order].npages--; free_pages((long) page, sizes[order].gfporder); } +aargh: sizes[order].nfrees++; sizes[order].nbytesmalloced -= size; restore_flags(flags); } + + + + +int get_malloc (char *buffer) +{ + int i; + int len=0; + + len += sprintf (buffer+len,"order size nblk nmalocs nfrees totbyt totpgs\n"); + for (i = 0;(len < 4000) && sizes[i].size;i++) + len += sprintf (buffer+len," %2d %-6d %4d %6d %6d %9d %5d\n", + i, + sizes[i].size, + sizes[i].nblocks, + sizes[i].nmallocs, + sizes[i].nfrees, + sizes[i].nbytesmalloced, + sizes[i].npages); + return len; +} + + + +#ifdef CONFIG_DEBUG_MALLOC + +#define LINE_SHIFT 12 + +int read_dmalloc (struct file * file, char * buf, int count) +{ + struct block_header *bh; + int lineno; + int i,len; + char line[100]; /* 52 chars misc stuff + 48 for the dump */ + unsigned char *chp; + char *destptr; + + if (count == 0) + return 0; + + lineno = file->f_pos >> LINE_SHIFT; + + /* Quickly go to the right part of the file... */ + for (bh = first_bh.dbh_next,i=0; + (bh != &first_bh) && (i < lineno); + bh=bh->dbh_next,i++) + /* Nothing */; + + destptr = buf; + + /* + * I object to using "column" around there. Anything may have + * happened in between, which might have altered whatever we were + * looking at. So if you are for example at line 5 column 34 a task + * switch may change whatever would be displayed on line 5. You + * would then see the first half of one part, and the second half + * of another. I consider this VERY bad. You should realize that + * you are always seeing a snapshot. If "top" would occasionally + * show a "sleep" using 50% cpu and lots of memory you would have a + * reason to be upset. A line is a piece of data that is internally + * consistent. + * + * Although the "maps" (which was used as an example) are more + * stable than kmalloced memory areas, I'd rather not risk + * this confusion..... + */ + + /* + * Solution used: + * If the next line would give more than your buffer can hold, + * the next line is not returned. + * Reading 4096 bytes is likely to give you around 4050 bytes. + * + * Gnu fileutils like dd, copy and cat behave correctly under these + * conditions. "less" doesn't like /proc files (cat <file>|less) + */ + + /* + * All the link-following here works under the condition that + * not the last malloced block of a page is freed. This could then + * free a page for reuse, which may overwrite the dbh_next pointer. + * + * This means is that it might be possible for this routine + * to follow a bad link. This would (VERY occasionally) result in + * a kenel oops, which would kill the process reading /proc/dmalloc . + * I think this is preferable to doing this all with interrupts + * off. The code for the "maps" file can assume that only the + * current process can modify its own maps. Here we have to assume + * any interrupt can cause a kfree..... + * + * In short: Problem signalled, consequences evaluated, + * and decided that the cure would be worse than the illness. + * Try to convince me otherwise..... + */ + while (bh && (bh != &first_bh)) { + len = sprintf (line,"%ld %s %d %ld", + bh->dbh_time, + bh->dbh_file, + bh->dbh_line, + bh->bh_length); + chp=(unsigned char *)(bh+1); + for (i=0;i<16;i++) + len +=sprintf (line+len, " %02x", chp[i]); + len += sprintf (line+len, "\n"); + + if (len > count) break; + + memcpy_tofs (destptr,line,len); + destptr += len; + count -= len; + lineno ++; + bh = bh->dbh_next; + } + file->f_pos = lineno << LINE_SHIFT; + + return destptr-buf; +} +#endif +
| |