lkml.org 
[lkml]   [2004]   [Mar]   [10]   [last100]   RSS Feed
Views: [more markup]   [less markup]   [headers]   [forward]  
 
Messages in this thread
/
SubjectRE: 2.6.3 userspace freeze
DateWed, 10 Mar 2004 10:12:50 +0100
From"Anders K. Pedersen" <>
Hello,

"Andrew Morton" <akpm@osdl.org> wrote:
> It could be a kernel deadlock, or a memory leak, or a disk 
> device driver
> bug.
> 
> Would it be possible to run a `vmstat 1' somewhere and 
> capture the last
> thirty or so lines prior to the reboot?

I ran it again tonight - here are the last lines from 'vmstat 1' on the
serial console:

   procs                      memory      swap          io     system
cpu
 r  b  w   swpd   free   buff  cache   si   so    bi    bo   in    cs us
sy id
 1  0  0      0  55736  25252 552272    0    0    44    80 1302   614  9
2 90
 0  0  0      0  56952  25284 552308    0    0     0   244 1189   495  4
1 95
 0  0  0      0  56376  25284 552308    0    0     0     0 1205   598  4
1 95
 0  0  0      0  54832  25316 552276    0    0     0   132 1136   528  2
8 91
 0  0  0      0  51488  25324 552268    0    0     0   160 1124   384  4
3 94
 0  0  0      0  51416  25332 552260    0    0     0    48 1091   348  1
0 99
 0  0  0      0  51272  25332 552260    0    0     0     0 1055   290  1
0 98
 0  0  0      0  51072  25332 552260    0    0     0     0 1078   345  1
0 99
 0  0  0      0  50616  25340 552320    0    0     0    40 1156   491  7
9 85
 0  0  0      0  50232  25348 552312    0    0     0    68 1188   509  4
0 95
 0  0  0      0  50232  25356 552372    0    0    16    56 1101   395  1
1 99
 0  0  0      0  50232  25356 552372    0    0     0     0 1075   314  0
0 99
 1  0  0      0  50104  25356 552372    0    0    24     0 1094   323  2
0 98
 0  0  0      0  50232  25364 552364    0    0     0    40 1150   516  3
8 89
 0  0  0      0  50120  25388 552408    0    0    36   324 1151   374  2
1 98
 0  0  0      0  50056  25396 552468    0    0    32    48 1142   401  1
0 99
 2  0  0      0  49032  25396 552604    0    0   108     0 1146   380  3
1 96
 0  0  0      0  49800  25404 552596    0    0    12    32 1250   516  1
0 98
 0  0  0      0  49688  25404 552596    0    0     0     0 1209   532  6
7 87
 0  0  0      0  49368  25412 552588    0    0     0   200 1347   780  6
1 92
 0  0  0      0  49368  25432 552704    0    0    76    64 1296   696 10
2 88
   procs                      memory      swap          io     system
cpu
 r  b  w   swpd   free   buff  cache   si   so    bi    bo   in    cs us
sy id
 0  0  0      0  48024  25456 552748    0    0    52     0 1148   418  4
3 94
 0  0  0      0  41560  25456 552748    0    0    12     0 1213   483 10
3 87
 0  0  0      0  35504  25464 552740    0    0    32   116 1227   631  8
10 82
 2  0  0      0  26656  25464 552740    0    0     0     0 1274   491 16
6 78
 0  0  0      0  12040  23296 552936    0    0     0   248 1322   691 12
11 77
 4  0  1      0   6128  21452 547368    0    0     4     0 1370   704 16
10 74
 0  0  0      0   6448  17008 533520    0    0     0     0 1138   517  9
18 73
 2  0  0      0   5512  16900 531996    0    0     0   120 1200   560  6
8 86
 0  0  0      0   5832  16792 529724    0    0     8   192 1197   524  7
6 88
 1  0  0      0   6848  16804 529168    0    0    12    68 1148   411  3
2 95
 0  0  0      0   6976  16836 529136    0    0     0   212 1178   428  6
1 93
 0  0  0      0   6976  16836 529136    0    0     0     0 1051   262  0
0 100
 0  0  0      0   6912  16884 529088    0    0    12   116 1137   404  2
1 98
 0  0  0      0   6464  16892 529080    0    0     8     0 1096   364  2
8 90
 0  0  0      0   6016  16788 528504    0    0     8   180 1147   384  4
1 95
 0  0  0      0   6784  16796 527544    0    0     8     0 1105   344  4
1 95
 0  0  0      0   7104  16808 523316    0    0     8     0 1173   493 12
2 86
 0  2  0      0   5952  16608 519980    0    0  4260   200 1860   912 29
10 61
 0  1  0      0   5568  16624 520780    0    0  3852     8 1926   616 16
21 62
 0  1  0      0   5944  16676 518348    0    0  2864   976 1498   829 17
4 79
 1  0  0      0   5680  16524 516324    0    0  1072     4 1405  1003 39
8 53
   procs                      memory      swap          io     system
cpu
 r  b  w   swpd   free   buff  cache   si   so    bi    bo   in    cs us
sy id
 0  1  0      0   5936  16504 511584    0    0  1188     0 1452  1003 40
7 53
 2  0  0      0   6608  16568 508664    0    0  1116   184 1476  1034 42
6 52
 1  0  0      0   5520  16628 506156    0    0  1056     0 1475  1086 37
22 41
 0  1  0      0   5520  16716 504300    0    0  1176   408 1418  1026 32
7 61
 2  0  0      0   5648  16636 499824    0    0  1172   144 1444  1003 39
6 56
 0  1  0      0   5648  16692 498612    0    0  1176     0 1405   955 33
5 61
 0  1  0      0   6040  16708 494584    0    0  1072   344 1422  1069 33
7 59
 0  1  0      0   6056  16728 491300    0    0   852     0 1462   976 38
23 39
 0  1  0      0   7080  16660 488580    0    0  1112   336 1492  1050 27
6 67
 1  0  0      0   5864  16700 486976    0    0  1200   116 1502  1209 20
5 75
 0  1  0      0   6568  16584 482060    0    0  1592     0 1514  1204 24
8 68
 9  0  9      0 508980  16616 469516    0    0  1452   708 3347  1647 14
68 18
 6  0  0      0 476088  17520 470924    0    0  2396   316 1449  2154 80
19  1
 3  0  0      0 474096  17680 470968    0    0   140     0 1131  4116 87
13  0
 4  0  0      0 471792  17716 471000    0    0     0  1564 1080  5567 87
13  0
 1  1  0      0 463344  17804 477372    0    0  5856     0 1438  3048 87
10  3
 3  0  0      0 456880  17980 481140    0    0  4605     0 1758  2237 78
18  4
 2  0  0      0 455408  18616 481184    0    0   584  2080 1259   731 77
20  3
 2  0  2      0 442136  19096 481180    0    0   484     0 1158   467 45
55  0
 3  0  1      0 430520  19300 481248    0    0   172   284 1119   329 11
89  0
 0  2  0      0 418720  19508 481244    0    0   200     8 1085   301 12
88  0
   procs                      memory      swap          io     system
cpu
 r  b  w   swpd   free   buff  cache   si   so    bi    bo   in    cs us
sy id
 2  0  0      0 399552  20080 482032    0    0  1255   822 1232   691 44
44 12
 7  3  0      0 331768  20208 482448    0    0   588     0 1424   788 60
40  0
 0  1  0      0 293128  20464 482668    0    0   412     0 1576   980 56
34  9
 0  1  1      0 289544  21364 482856    0    0  1056   252 1526  1380  9
7 85
 2  1  0      0 285000  22376 483068    0    0  1164    24 1551  1259 16
5 79
 0  1  1      0 277976  23024 483372    0    0   924    60 1674  1442 30
7 64
 0  1  0      0 273560  23828 483520    0    0   900  1128 1480  1149  9
21 70
 0  1  0      0 268832  24656 483576    0    0   864     0 1373   844  9
5 86
SOFTDOG: Initiating system reboot.

I also got had a top running - this is the last output before it froze:

 04:02:33  up  3:33,  2 users,  load average: 1.27, 0.44, 0.35
266 processes: 265 sleeping, 1 running, 0 zombie, 0 stopped
CPU0 states:  24.4% user  15.1% system    0.1% nice  22.0% iowait  38.2%
idle
CPU1 states:  25.2% user  13.2% system    0.2% nice  49.2% iowait  11.1%
idle
Mem:  2072988k av, 1795780k used,  277208k free,       0k shrd,   23232k
buff
       573192k active,             372544k inactive
Swap: 8384880k av,       0k used, 8384880k free                  483436k
cached

  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME CPU
COMMAND
      httpd     16   0  111M 104M 11320 S    62.8  5.1   0:25   0 httpd
 4306 root      16   0  2476 1464  1936 R     3.3  0.0   5:57   1 top
29761 root      34  19  1464  520  1368 D N   2.5  0.0   0:00   0
updatedb
29889 root      16   0 29628  27M  1712 S     2.1  1.3   0:00   1
rotatelogspsoft
 1515 root      16   0  372M  65M 44444 S     0.7  3.2   1:10   1
caspeng
 1973 root      16   0  1416  472  1356 S     0.7  0.0   1:40   1 vmstat
29891 root      16   0  2584 1300  1712 S     0.5  0.0   0:00   1
rotatelogspsoft
 1559 root      16   0  2068 1064  1864 S     0.3  0.0   0:44   1
soagent
29890 root      16   0  2684 1400  1712 S     0.3  0.0   0:00   1
rotatelogspsoft
    1 root      16   0  1376  484  1328 S     0.0  0.0   0:01   0 init
    2 root      RT   0     0    0     0 SW    0.0  0.0   0:00   0
migration/0
    3 root      34  19     0    0     0 SWN   0.0  0.0   0:00   0
ksoftirqd/0
    4 root      RT   0     0    0     0 SW    0.0  0.0   0:00   1
migration/1
    5 root      34  19     0    0     0 SWN   0.0  0.0   0:00   1
ksoftirqd/1
    6 root       5 -10     0    0     0 SW<   0.0  0.0   0:00   0
events/0
    7 root       5 -10     0    0     0 SW<   0.0  0.0   0:00   1
events/1
    8 root       5 -10     0    0     0 SW<   0.0  0.0   0:00   0
kblockd/0
    9 root       5 -10     0    0     0 SW<   0.0  0.0   0:00   1
kblockd/1
   10 root      15   0     0    0     0 SW    0.0  0.0   0:00   0 khubd
   14 root      15   0     0    0     0 SW    0.0  0.0   0:07   1
kswapd0
   13 root      25   0     0    0     0 SW    0.0  0.0   0:00   0
pdflush
   12 root      15   0     0    0     0 SW    0.0  0.0   0:00   1
pdflush
   11 root      15   0     0    0     0 SW    0.0  0.0   0:00   0 kirqd
   15 root      15 -10     0    0     0 SW<   0.0  0.0   0:00   0 aio/0
   16 root      10 -10     0    0     0 SW<   0.0  0.0   0:00   1 aio/1
   17 root      25   0     0    0     0 SW    0.0  0.0   0:00   1
scsi_eh_0
   18 root      19   0     0    0     0 SW    0.0  0.0   0:00   0
kseriod
   19 root      15   0     0    0     0 SW    0.0  0.0   0:02   1
kjournald
  155 root      15   0     0    0     0 SW    0.0  0.0   0:00   1
kjournald
  156 root      15   0     0    0     0 DW    0.0  0.0   0:00   1
kjournald
  157 root      15   0     0    0     0 SW    0.0  0.0   0:03   1
kjournald
  236 root      18   0  1352  404  1312 S     0.0  0.0   0:00   0
mingetty
  303 root       6 -10  1356  420  1308 S <   0.0  0.0   0:00   0
watchdogd
  591 root      16   0  1428  540  1368 S     0.0  0.0   0:01   1
syslogd
  596 root      16   0  1368  444  1324 S     0.0  0.0   0:00   0 klogd
  760 ntp       16   0  1896 1892  1740 S     0.0  0.0   0:00   1 ntpd
  781 named     15   0 52648  50M  2056 S     0.0  2.4   0:12   1 named
  799 root      15   0  3468 1484  3184 S     0.0  0.0   0:00   1 sshd
  902 root      16   0  3708 1200  3416 S     0.0  0.0   0:00   1 master
If you need anything further, please let me know.

Med venlig hilsen - Best regards

Anders K. Pedersen
Network Engineer
------------------------------------------------ 
Cohaesio A/S - Maglebjergvej 5D - DK-2800 Lyngby
Phone: +45 45 880 888  - Fax: +45 45 880 777
Mail: akp@cohaesio.com - http://www.cohaesio.com
------------------------------------------------
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
\
 
 \ /
  Last update: 2005-03-22 13:01    [W:0.781 / U:0.430 seconds]
©2003-2008 Jasper Spaans