Messages in this thread | | | Date | Wed, 26 Jan 2011 19:10:34 +0530 | Subject | Re: [PATCH] oom: handle overflow in mem_cgroup_out_of_memory() | From | Balbir Singh <> |
| |
On Wed, Jan 26, 2011 at 1:59 PM, Greg Thelen <gthelen@google.com> wrote: > mem_cgroup_get_limit() returns a byte limit as a unsigned 64 bit value, > which is converted to a page count by mem_cgroup_out_of_memory(). Prior > to this patch the conversion could overflow on 32 bit platforms > yielding a limit of zero. >
Why would the overflow occur? Due to the right shift being used with unsigned long long? I am afraid I don't have quick access to a 32 bit box to test the patch. May be I too drained to understand the problem, can you post the corresponding assembly for analysis?
Balbir -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |