lkml.org 
[lkml]   [2008]   [Feb]   [19]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
Date
From
SubjectRe: [LTP] [PATCH 1/8] Scaling msgmni to the amount of lowmem
Subrata Modak wrote:
>>Nadia Derbey wrote:
>>
>>>Andrew Morton wrote:
>>>
>>>
>>>>On Mon, 11 Feb 2008 15:16:47 +0100 Nadia.Derbey@bull.net wrote:
>>>>
>>>>
>>>>
>>>>>[PATCH 01/08]
>>>>>
>>>>>This patch computes msg_ctlmni to make it scale with the amount of
>>>>>lowmem.
>>>>>msg_ctlmni is now set to make the message queues occupy 1/32 of the
>>>>>available
>>>>>lowmem.
>>>>>
>>>>>Some cleaning has also been done for the MSGPOOL constant: the msgctl
>>>>>man page
>>>>>says it's not used, but it also defines it as a size in bytes (the code
>>>>>expresses it in Kbytes).
>>>>>
>>>>
>>>>
>>>>Something's wrong here. Running LTP's msgctl08 (specifically:
>>>>ltp-full-20070228) cripples the machine. It's a 4-way 4GB x86_64.
>>>>
>>>>http://userweb.kernel.org/~akpm/config-x.txt
>>>>http://userweb.kernel.org/~akpm/dmesg-x.txt
>>>>
>>>>Normally msgctl08 will complete in a second or two. With this patch I
>>>>don't know how long it will take to complete, and the machine is horridly
>>>>bogged down. It does recover if you manage to kill msgctl08. Feels like
>>>>a terrible memory shortage, but there's plenty of memory free and it
>>>>isn't
>>>>swapping.
>>>>
>>>>
>>>>
>>>
>>>Before the patchset, msgctl08 used to be run with the old msgmni value:
>>>16. Now it is run with a much higher msgmni value (1746 in my case),
>>>since it scales to the memory size.
>>>When I call "msgctl08 100000 16" it completes fast.
>>>
>>>Doing the follwing on the ref kernel:
>>>echo 1746 > /proc/sys/kernel/msgmni
>>>msgctl08 100000 1746
>>>
>>>makes th test block too :-(
>>>
>>>Will check to see where the problem comes from.
>>>
>>
>>Well, actually, the test does not block, it only takes much much more
>>time to be executed:
>>
>>doing this:
>>date; ./msgctl08 100000 XXX; date
>>
>>
>>gives us the following results:
>>XXX 16 32 64 128 256 512 1024 1746
>>time(secs) 2 4 8 16 32 64 132 241
>>
>>XXX is the # of msg queues to be created = # of processes to be forked
>>as readers = # of processes to be created as writers
>>time is approximative since it is obtained by a "date" before and after.
>>
>>XXX used to be 16 before the patchset ---> 1st column
>> --> 16 processes forked as reader
>> --> + 16 processes forked as writers
>> --> + 16 msg queues
>>XXX = 1746 (on my victim) after the patchset ---> last column
>> --> 1746 reader processes forked
>> --> + 1746 writers forked
>> --> + 1746 msg queues created
>>
>>The same tests on the ref kernel give approximatly the same results.
>>
>>So if we don't want this longer time to appear as a regression, the LTP
>>should be changed:
>>1) either by setting the result of get_max_msgqueues() as the MSGMNI
>>constant (16) (that would be the best solution in my mind)
>>2) or by warning the tester that it may take a long time to finish.
>>
>>There would be 3 tests impacted:
>>
>>kernel/syscalls/ipc/msgctl/msgctl08.c
>>kernel/syscalls/ipc/msgctl/msgctl09.c
>>kernel/syscalls/ipc/msgget/msgget03.c
>
>
> We will change the test case if need that be. Nadia, kindly send us the
> patch set which will do the necessary changes.
>
> Regards--
> Subrata
>

Subrata,

You'll find the patch in attachment.
FYI I didn't change msgget03.c since we need to get the actual max value
in order to generate an error.

Regards,
Nadia

Since msgmni now scales to the memory size, it may reach big values.
To avoid forking 2*msgmni processes and create msgmni msg queues, do not take
msgmni from procfs anymore.
Just define it as 16 (which is the MSGMNI constant value in linux/msg.h)

Also fixed the Makefiles in ipc/lib and ipc/msgctl: there was no dependency
on the lib/ipc*.h header files.

Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net>

---
testcases/kernel/syscalls/ipc/lib/Makefile | 3 +++
testcases/kernel/syscalls/ipc/lib/ipcmsg.h | 2 ++
testcases/kernel/syscalls/ipc/msgctl/Makefile | 3 +++
testcases/kernel/syscalls/ipc/msgctl/msgctl08.c | 23 ++---------------------
testcases/kernel/syscalls/ipc/msgctl/msgctl09.c | 24 ++----------------------
5 files changed, 12 insertions(+), 43 deletions(-)

Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl08.c
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/msgctl/msgctl08.c 2006-02-11 05:46:36.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl08.c 2008-02-19 18:45:27.000000000 +0100
@@ -50,6 +50,7 @@
#include <sys/msg.h>
#include "test.h"
#include "usctest.h"
+#include "ipcmsg.h"

void setup();
void cleanup();
@@ -479,26 +480,6 @@ static int get_used_msgqueues()
return used_queues;
}

-/** Get the max number of message queues allowed on system */
-static int get_max_msgqueues()
-{
- FILE *f;
- char buff[BUFSIZE];
-
- /* Get the max number of message queues allowed on system */
- f = fopen("/proc/sys/kernel/msgmni", "r");
- if (!f){
- tst_resm(TBROK,"Could not open /proc/sys/kernel/msgmni");
- tst_exit();
- }
- if (!fgets(buff, BUFSIZE, f)) {
- tst_resm(TBROK,"Could not read /proc/sys/kernel/msgmni");
- tst_exit();
- }
- fclose(f);
- return atoi(buff);
-}
-
/***************************************************************
* setup() - performs all ONE TIME setup for this test.
*****************************************************************/
@@ -520,7 +501,7 @@ setup()
*/
TEST_PAUSE;

- MSGMNI = get_max_msgqueues() - get_used_msgqueues();
+ MSGMNI = MAX_MSGQUEUES - get_used_msgqueues();
if (MSGMNI <= 0){
tst_resm(TBROK,"Max number of message queues already used, cannot create more.");
cleanup();
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl09.c
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/msgctl/msgctl09.c 2006-02-11 05:46:36.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl09.c 2008-02-19 18:46:44.000000000 +0100
@@ -49,6 +49,7 @@
#include <unistd.h>
#include "test.h"
#include "usctest.h"
+#include "ipcmsg.h"

#define MAXNREPS 1000
#ifndef CONFIG_COLDFIRE
@@ -649,26 +650,6 @@ static int get_used_msgqueues()
return used_queues;
}

-/** Get the max number of message queues allowed on system */
-static int get_max_msgqueues()
-{
- FILE *f;
- char buff[BUFSIZE];
-
- /* Get the max number of message queues allowed on system */
- f = fopen("/proc/sys/kernel/msgmni", "r");
- if (!f){
- tst_resm(TBROK,"Could not open /proc/sys/kernel/msgmni");
- tst_exit();
- }
- if (!fgets(buff, BUFSIZE, f)) {
- tst_resm(TBROK,"Could not read /proc/sys/kernel/msgmni");
- tst_exit();
- }
- fclose(f);
- return atoi(buff);
-}
-
/***************************************************************
* setup() - performs all ONE TIME setup for this test.
*****************************************************************/
@@ -689,12 +670,11 @@ setup()
*/
TEST_PAUSE;

- MSGMNI = get_max_msgqueues() - get_used_msgqueues();
+ MSGMNI = MAX_MSGQUEUES - get_used_msgqueues();
if (MSGMNI <= 0){
tst_resm(TBROK,"Max number of message queues already used, cannot create more.");
cleanup();
}
-
}


Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/lib/ipcmsg.h
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/lib/ipcmsg.h 2005-12-22 21:18:23.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/lib/ipcmsg.h 2008-02-19 18:51:38.000000000 +0100
@@ -41,6 +41,8 @@ void setup(void);
#define MSGSIZE 1024 /* a resonable size for a message */
#define MSGTYPE 1 /* a type ID for a message */

+#define MAX_MSGQUEUES 16 /* MSGMNI as defined in linux/msg.h */
+
typedef struct mbuf { /* a generic message structure */
long mtype;
char mtext[MSGSIZE + 1]; /* add 1 here so the message can be 1024 */
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/lib/Makefile
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/lib/Makefile 2006-08-21 08:58:39.000000000 +0200
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/lib/Makefile 2008-02-19 18:50:19.000000000 +0100
@@ -19,6 +19,7 @@
SRCS = libipc.c
OBJS = $(SRCS:.c=.o)
LIBIPC = ../libipc.a
+LIBIPC_HEADERS = ipcmsg.h ipcsem.h

CFLAGS += -I../../../../../include -Wall

@@ -27,6 +28,8 @@ all: $(LIBIPC)
$(LIBIPC): $(OBJS)
$(AR) -rc $@ $(OBJS)

+$(OBJS): $(LIBIPC_HEADERS)
+
install:

clean:
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/Makefile
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/msgctl/Makefile 2006-08-23 07:46:27.000000000 +0200
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/Makefile 2008-02-19 19:02:26.000000000 +0100
@@ -18,12 +18,15 @@

CFLAGS += -I../lib -I../../../../../include -Wall
LDLIBS += -L../../../../../lib -lltp -L.. -lipc
+LIBIPC_HEADERS = ../lib/ipcmsg.h

SRCS = $(wildcard *.c)
TARGETS = $(patsubst %.c,%,$(SRCS))

all: $(TARGETS)

+$(TARGETS): $(LIBIPC_HEADERS)
+
install:
@set -e; for i in $(TARGETS); do ln -f $$i ../../../../bin/$$i ; done
\
 
 \ /
  Last update: 2008-02-19 18:19    [W:0.123 / U:0.092 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site