lkml.org 
[lkml]   [2010]   [Jun]   [20]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
SubjectFW: sched_setaffinity not working with kernel 2.6.32.15
Date

From: psxlover@hotmail.com
To: mingo@elte.hu; peterz@infradead.org
Subject:
sched_setaffinity not working with kernel 2.6.32.15
Date: Sun, 20
Jun 2010 17:14:12 +0300






.ExternalClass .ecxhmmessage P
{padding:0px;}
.ExternalClass body.ecxhmmessage
{font-size:10pt;font-family:Verdana;}



On kernel 2.6.32.15 after I execute sched_setaffinity even
though
sched_getaffinity returns the right cpus the threads
of my program
are running on different cpus.

  First time I came across the bug
was on a PC on my unive-
rsity which had just been updated to Debian
'squeeze'. I was
using a library calles hwloc to get information
about the to-
pology of the system and bind threads to cpus. The
binding
while it wasn't returning any error wasn't working (I used
top
to check which cpus where being used). After contacting
with
the developers of hwloc they suggested that the pro-
blem is either
in the kernel or in glibc.
  I made a virtual machine on my pc using
the latest Debian
'squeeze' and the problem was there. So I tried an
older
kernel I found in the Synaptic Package Manager and my pro-
gram
was working fine. So the problem was in the kernel. I
e-mailed hwloc
mailing list again and he suggested I tried
the vanilla kernels and
use git bisect to find where the problem
started.
  With 2.6.34
and 2.6.33.5 I had no problem but with 2.6.32.15
sched_setaffinity
wasn't working. I used git bisect and in the
end I got this:


c6fc81afa2d7ef2f775e48672693d8a0a8a7325d
is the first bad commit
commit
c6fc81afa2d7ef2f775e48672693d8a0a8a7325d
Author: John Wright
<john.wright@hp.com>
Date:   Tue Apr 13 16:55:37 2010 -0600

   
sched: Fix a race between ttwu() and migrate_task()
   
   
Based on commit e2912009fb7b715728311b0d8fe327a1432b3f79 upstream, but
   
done differently as this issue is not present in .33 or .34 kernels due
   
to rework in this area.
   
    If a task is in the TASK_WAITING
state, then try_to_wake_up() is working
    on it, and it will place
it on the correct cpu.
   
    This commit ensures that neither
migrate_task() nor __migrate_task()
    calls set_task_cpu(p) while p
is in the TASK_WAKING state.  Otherwise,
    there could be two
concurrent calls to set_task_cpu(p), resulting in
    the task's
cfs_rq being inconsistent with its cpu.
   
    Signed-off-by:
John Wright <john.wright@hp.com>
    Cc: Ingo Molnar
<mingo@elte.hu>
    Cc: Peter Zijlstra
<peterz@infradead.org>
    Signed-off-by: Greg Kroah-Hartman
<gregkh@suse.de>

:040000 040000
a9d18950d8edddb761c0266706f671f0e9a006fe
2c3a7d7d5e616ecc276b4e93f4b6e5162a9382c8 M    kernel


  I've
attached a simple program that binds a few threads on
the first cpu.
With 2.6.32.15 when I run it I can see with
System Monitor that more
than one cpu is being used by the
threads.


--------------------------
Alexandros
Papadogiannakis
_________________________________________________________________
Hotmail: Trusted email with Microsoft's powerful SPAM protection.
https://signup.live.com/signup.aspx?id=60969
#ifndef _GNU_SOURCE
#define _GNU_SOURCE 1
#endif

#include <stdio.h>
//#include <sched.h>
//#include <hwloc.h>
#include <stdlib.h>
#include <pthread.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/syscall.h>

#define NR_CPUS 8
#define NR_THREADS 5

pid_t tid[NR_THREADS];

pthread_barrier_t init_start, barrier2;

void print_affinity(pthread_t pid, int threadno) {
int i;
cpu_set_t set;

CPU_ZERO(&set);

sched_getaffinity( pid, sizeof(set), &set);


for( i = 0; i < NR_CPUS; i++) {
if (CPU_ISSET(i, &set))
printf("Cpu #%d is in %d thread's affinity\n", i, threadno);
}
}

void set_schedaffinity(pthread_t pid, int cpu) {
cpu_set_t set;

CPU_ZERO(&set);
CPU_SET(cpu, &set);

sched_setaffinity(pid, sizeof(set), &set);
}

void set_threadaffinity(pthread_t pid, int cpu) {
cpu_set_t set;

CPU_ZERO(&set);
CPU_SET(cpu, &set);

pthread_setaffinity_np(pid, sizeof(set), &set);
}

void * threadCode(void * threadNo) {
long int thread = (long int) threadNo;

//Set current thread id
//tid[thread] = (pid_t) syscall (SYS_gettid);

//Wait for all the threads to set their thread id
//pthread_barrier_wait(&init_start);

//Wait for main to set the affinity
pthread_barrier_wait(&barrier2);
//sleep(1);

//Print affinity
print_affinity(0,thread);

//Do something so that we can see cpu being used
while (1);

pthread_exit(0);
}



int main(int argc , char *argv[]) {
long int i;
pthread_t pid[NR_THREADS];

// Initialize barriers
if(pthread_barrier_init(&init_start, NULL, NR_THREADS)) {
printf("Could not create a barrier\n");
exit(1);
}
if(pthread_barrier_init(&barrier2, NULL, NR_THREADS)) {
printf("Could not create a barrier\n");
exit(1);
}

//Create threads
for (i = 0; i < NR_THREADS; i++)
pthread_create(&pid[i], NULL, threadCode, (void *) i);

//Wait for threads to set their thread id
//pthread_barrier_wait(&init_start);

//Set affinity
for (i = 0; i < NR_THREADS; i++)
//set_schedaffinity(tid[i],1);
set_threadaffinity(pid[i],0);

//Tell the threads to continue after setting affinity
//pthread_barrier_wait(&barrier2);

getchar();

return 0;
}

\
 
 \ /
  Last update: 2010-06-20 21:13    [W:0.112 / U:0.204 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site