[lkml]   [2006]   [Feb]   [28]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: sg regression in 2.6.16-rc5
Kai Makisara wrote:
> On Tue, 28 Feb 2006, Douglas Gilbert wrote:
>>Mark Rustad wrote:
>>>We have encountered some kind of sg regression with kernel 2.6.16-rc5
>>>relative to 2.6.15. We have a small program that demonstrates the
>>>failure. On 2.6.15 it produces the output:
>>>Alloced dataptr 0 -> 0xb7d07008
>>>IOS: 0
>>>ios 100
>>>indicating that it did 100 operations successfully. On 2.6.16-rc5, it
>>>produces the output:
>>>Alloced dataptr 0 -> 0xa7d10008
>>>SG_IO ioctl error 12 Cannot allocate memory
>>>ios 0
>>>indicating that it did 0 operations successfully. This program is
>>>attempting to do 1MB reads on a SCSI device.
>>You can stop right there with the 1 MB reads. Welcome
>>to the new, blander sg driver which now shares many
>>size shortcomings with the block subsystem.
>>In lk 2.6.15 the sg driver (and the st driver) did its
>>own scatter gather list allocations. The sg driver
>>used 32 KB segments (8 times the normal page size)
>>in each scatter gather element. The maximum number
>>of scatter gather elements depends on the LLD but
>>can be no more than 256. That meant the sg driver
>>allowed a maximum single IO size of 8 MB. There was
>>also a define in sg.h (SG_SCATTER_SZ and it is still
>>there) that allowed the 32KB per segment to be increased
>>allowing larger single command transfers (then 8 MB).
> This is still possible but it needs some changes to most SCSI HBA drivers.
> The big requests are split into bios supporting 256 pages. For 4 kB pages,
> this limits i/o to 1 MB. The scsi_execute_async() path used by st and sg
> can chain bios and this enables large request at the ULD level. At lower
> level, the request consists of pages and now we hit the s/g list maximum
> length _unless_ the HBA driver enables clustering. In this case the
> adjacent pages are coalesced and the large requests fit into the HBA s/g
> limits. Well, now we hit another limit: the max_sectors default for SCSI
> drivers is 1024 and this limits requests to 512 kB _unless_ the HBA driver
> increases max_sectors.
> The aic79xx driver enables clustering but does not increase max_sectors.
> This makes the maximum request size 512 kB. If it is possible to set
> .max_sectors = 0xFFFF,
> in linux/drivers/scsi/aic7xxx/aic79xx_osm.c without breaking the driver,
> this should enable requests up to 8 MB - 256 B. (I don't have the hardware
> to test this.)
> Several SCSI HBA drivers currently have similar problems.

I applied the above changes to my scsi_debug (plus
extended .sg_tablesize to SG_ALL (it was 64)) and
my single command READs topped out at 4 MB exactly
(bs=512 bpt=8192). When I tried bpt=8193 the
SG_IO ioctl (via a sg device) yielded ENOMEM which
is much more informative than EIO.

That is an improvement.

Doug Gilbert
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
More majordomo info at
Please read the FAQ at

 \ /
  Last update: 2006-03-01 03:09    [W:0.111 / U:7.344 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site