| Message ID | 1357723959-5416-3-git-send-email-tangchen@cn.fujitsu.com |
|---|---|
| State | Not Applicable |
| Delegated to: | David Miller |
| Headers | show |
On Wed, 9 Jan 2013 17:32:26 +0800 Tang Chen <tangchen@cn.fujitsu.com> wrote: > We remove the memory like this: > 1. lock memory hotplug > 2. offline a memory block > 3. unlock memory hotplug > 4. repeat 1-3 to offline all memory blocks > 5. lock memory hotplug > 6. remove memory(TODO) > 7. unlock memory hotplug > > All memory blocks must be offlined before removing memory. But we don't hold > the lock in the whole operation. So we should check whether all memory blocks > are offlined before step6. Otherwise, kernel maybe panicked. Well, the obvious question is: why don't we hold lock_memory_hotplug() for all of steps 1-4? Please send the reasons for this in a form which I can paste into the changelog. Actually, I wonder if doing this would fix a race in the current remove_memory() repeat: loop. That code does a find_memory_block_hinted() followed by offline_memory_block(), but afaict find_memory_block_hinted() only does a get_device(). Is the get_device() sufficiently strong to prevent problems if another thread concurrently offlines or otherwise alters this memory_block's state? -- To unsubscribe from this list: send the line "unsubscribe sparclinux" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Andrew, On 01/10/2013 07:11 AM, Andrew Morton wrote: > On Wed, 9 Jan 2013 17:32:26 +0800 > Tang Chen<tangchen@cn.fujitsu.com> wrote: > >> We remove the memory like this: >> 1. lock memory hotplug >> 2. offline a memory block >> 3. unlock memory hotplug >> 4. repeat 1-3 to offline all memory blocks >> 5. lock memory hotplug >> 6. remove memory(TODO) >> 7. unlock memory hotplug >> >> All memory blocks must be offlined before removing memory. But we don't hold >> the lock in the whole operation. So we should check whether all memory blocks >> are offlined before step6. Otherwise, kernel maybe panicked. > > Well, the obvious question is: why don't we hold lock_memory_hotplug() > for all of steps 1-4? Please send the reasons for this in a form which > I can paste into the changelog. In the changelog form: Offlining a memory block and removing a memory device can be two different operations. Users can just offline some memory blocks without removing the memory device. For this purpose, the kernel has held lock_memory_hotplug() in __offline_pages(). To reuse the code for memory hot-remove, we repeat step 1-3 to offline all the memory blocks, repeatedly lock and unlock memory hotplug, but not hold the memory hotplug lock in the whole operation. > > > Actually, I wonder if doing this would fix a race in the current > remove_memory() repeat: loop. That code does a > find_memory_block_hinted() followed by offline_memory_block(), but > afaict find_memory_block_hinted() only does a get_device(). Is the > get_device() sufficiently strong to prevent problems if another thread > concurrently offlines or otherwise alters this memory_block's state? I think we already have memory_block->state_mutex to protect the concurrently changing of memory_block's state. The find_memory_block_hinted() here is to find the memory_block corresponding to the memory section we are dealing with. Thanks. :) > -- To unsubscribe from this list: send the line "unsubscribe sparclinux" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/drivers/base/memory.c b/drivers/base/memory.c index 987604d..8300a18 100644 --- a/drivers/base/memory.c +++ b/drivers/base/memory.c @@ -693,6 +693,12 @@ int offline_memory_block(struct memory_block *mem) return ret; } +/* return true if the memory block is offlined, otherwise, return false */ +bool is_memblock_offlined(struct memory_block *mem) +{ + return mem->state == MEM_OFFLINE; +} + /* * Initialize the sysfs support for memory devices... */ diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h index 4a45c4e..8dd0950 100644 --- a/include/linux/memory_hotplug.h +++ b/include/linux/memory_hotplug.h @@ -247,6 +247,7 @@ extern int add_memory(int nid, u64 start, u64 size); extern int arch_add_memory(int nid, u64 start, u64 size); extern int offline_pages(unsigned long start_pfn, unsigned long nr_pages); extern int offline_memory_block(struct memory_block *mem); +extern bool is_memblock_offlined(struct memory_block *mem); extern int remove_memory(u64 start, u64 size); extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn, int nr_pages); diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 62e04c9..5808045 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1430,6 +1430,54 @@ repeat: goto repeat; } + lock_memory_hotplug(); + + /* + * we have offlined all memory blocks like this: + * 1. lock memory hotplug + * 2. offline a memory block + * 3. unlock memory hotplug + * + * repeat step1-3 to offline the memory block. All memory blocks + * must be offlined before removing memory. But we don't hold the + * lock in the whole operation. So we should check whether all + * memory blocks are offlined. + */ + + mem = NULL; + for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) { + section_nr = pfn_to_section_nr(pfn); + if (!present_section_nr(section_nr)) + continue; + + section = __nr_to_section(section_nr); + /* same memblock? */ + if (mem) + if ((section_nr >= mem->start_section_nr) && + (section_nr <= mem->end_section_nr)) + continue; + + mem = find_memory_block_hinted(section, mem); + if (!mem) + continue; + + ret = is_memblock_offlined(mem); + if (!ret) { + pr_warn("removing memory fails, because memory " + "[%#010llx-%#010llx] is onlined\n", + PFN_PHYS(section_nr_to_pfn(mem->start_section_nr)), + PFN_PHYS(section_nr_to_pfn(mem->end_section_nr + 1)) - 1); + + kobject_put(&mem->dev.kobj); + unlock_memory_hotplug(); + return ret; + } + } + + if (mem) + kobject_put(&mem->dev.kobj); + unlock_memory_hotplug(); + return 0; } #else