diff mbox

slub: Make CONFIG_PAGE_ALLOC work with new fastpath

Message ID alpine.DEB.2.00.1105131614440.913@router.home
State Not Applicable, archived
Delegated to: David Miller
Headers show

Commit Message

Christoph Lameter (Ampere) May 13, 2011, 9:15 p.m. UTC
I ran this with hackbench under KVM with CONFIG_DEBUG_PAGEALLOC and I saw
no failures.


Subject: slub: Make CONFIG_PAGE_ALLOC work with new fastpath

Fastpath can do a speculative access to a page that CONFIG_PAGE_ALLOC may have
marked as invalid to retrieve the pointer to the next free object.

Use probe_kernel_read in that case in order not to cause a page fault.

Signed-off-by: Christoph Lameter <cl@linux.com>
---
 mm/slub.c |   14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Comments

Eric Dumazet May 13, 2011, 9:26 p.m. UTC | #1
Le vendredi 13 mai 2011 à 16:15 -0500, Christoph Lameter a écrit :
> I ran this with hackbench under KVM with CONFIG_DEBUG_PAGEALLOC and I saw
> no failures.
> 
> 
> Subject: slub: Make CONFIG_PAGE_ALLOC work with new fastpath
> 
> Fastpath can do a speculative access to a page that CONFIG_PAGE_ALLOC may have
> marked as invalid to retrieve the pointer to the next free object.
> 
> Use probe_kernel_read in that case in order not to cause a page fault.
> 
> Signed-off-by: Christoph Lameter <cl@linux.com>

Reported-by: Eric Dumazet <eric.dumazet@gmail.com>
Acked-by: Eric Dumazet <eric.dumazet@gmail.com>

Thanks !


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

Index: linux-2.6/mm/slub.c
===================================================================
--- linux-2.6.orig/mm/slub.c	2011-05-10 14:31:28.000000000 -0500
+++ linux-2.6/mm/slub.c	2011-05-10 14:31:35.000000000 -0500
@@ -261,6 +261,18 @@  static inline void *get_freepointer(stru
 	return *(void **)(object + s->offset);
 }

+static inline void *get_freepointer_safe(struct kmem_cache *s, void *object)
+{
+	void *p;
+
+#ifdef CONFIG_DEBUG_PAGEALLOC
+	probe_kernel_read(&p, (void **)(object + s->offset), sizeof(p));
+#else
+	p = get_freepointer(s, object);
+#endif
+	return p;
+}
+
 static inline void set_freepointer(struct kmem_cache *s, void *object, void *fp)
 {
 	*(void **)(object + s->offset) = fp;
@@ -1933,7 +1945,7 @@  redo:
 		if (unlikely(!irqsafe_cpu_cmpxchg_double(
 				s->cpu_slab->freelist, s->cpu_slab->tid,
 				object, tid,
-				get_freepointer(s, object), next_tid(tid)))) {
+				get_freepointer_safe(s, object), next_tid(tid)))) {

 			note_cmpxchg_failure("slab_alloc", s, tid);
 			goto redo;