| Message ID | 20200207183341.12582-1-dwilder@us.ibm.com |
|---|---|
| Headers | show |
| Series | Support for non-consecutive numa nodes and core ids. | expand |
> Ovs-numa currently makes the assumption that numa node ids and cpu core ids > will be numbered consecutively. Current Power systems don't always follow this > model. Furthermore, cpus on Power may be on/off lined based the setting of > Simultaneous multithreading (SMT). The result can be gaps in the numbering of > the cores. For example, a 2 socket system with 20 Core(s) per socket configured > with 4 thread per core (smt=4) has the following topology: > > NUMA node0 CPU(s): 0-79 > NUMA node8 CPU(s): 80-159 > > When set to smt=2 the following topology is created. > > NUMA node0 CPU(s): 0,1,4,5,8,9,12,13,16,17,20,21,24,25,28,29,32,33,36,37,40, > 41,44,45,48,49,52,53,56,57,60,61,64,65,68,69,72,73,76,77 > NUMA node8 CPU(s): 80,81,84,85,88,89,92,93,96,97,100,101,104,105,108,109,112, > 113,116,117,120,121,124,125,128,129,132,133,136,137,140, > 141,144,145,148,149,152,153,156,157 > > The first patch in this series allows ovs-numa to work with non-consecutive > node and cpu ids. I updated pmd and dpif-netdev multi-node tests to simulate > a numa topology with non-consecutive node ids. > > The second patch in the series updates lib/dpdk:construct_dpdk_socket_mem() > to correctly build the EAL options: --socket-mem and --socket-limit on systems > with non-consecutive node ids. > > v2 changes: > 0-day Robot suggested changes. > v3 changes: > re-wrote cpu_detected() to address memory leak. > Tested-by: David Christensen <drc@linux.vnet.ibm.com>
Ovs-numa currently makes the assumption that numa node ids and cpu core ids will be numbered consecutively. Current Power systems don't always follow this model. Furthermore, cpus on Power may be on/off lined based the setting of Simultaneous multithreading (SMT). The result can be gaps in the numbering of the cores. For example, a 2 socket system with 20 Core(s) per socket configured with 4 thread per core (smt=4) has the following topology: NUMA node0 CPU(s): 0-79 NUMA node8 CPU(s): 80-159 When set to smt=2 the following topology is created. NUMA node0 CPU(s): 0,1,4,5,8,9,12,13,16,17,20,21,24,25,28,29,32,33,36,37,40, 41,44,45,48,49,52,53,56,57,60,61,64,65,68,69,72,73,76,77 NUMA node8 CPU(s): 80,81,84,85,88,89,92,93,96,97,100,101,104,105,108,109,112, 113,116,117,120,121,124,125,128,129,132,133,136,137,140, 141,144,145,148,149,152,153,156,157 The first patch in this series allows ovs-numa to work with non-consecutive node and cpu ids. I updated pmd and dpif-netdev multi-node tests to simulate a numa topology with non-consecutive node ids. The second patch in the series updates lib/dpdk:construct_dpdk_socket_mem() to correctly build the EAL options: --socket-mem and --socket-limit on systems with non-consecutive node ids. v2 changes: 0-day Robot suggested changes. v3 changes: re-wrote cpu_detected() to address memory leak. A travis build with these two patches can be found here: https://travis-ci.org/djlwilder/ovs/builds/647118555 David Wilder (2): ovs-numa: Non-consecutively numa nodes and cores ovs-numa: Dpdk options with non-consecutive nodes. lib/dpdk.c | 26 ++++++++++++++++++-------- lib/ovs-numa.c | 44 ++++++++++++++++++++++++++++++++------------ lib/ovs-numa.h | 2 ++ tests/dpif-netdev.at | 12 ++++++------ tests/pmd.at | 20 ++++++++++---------- 5 files changed, 68 insertions(+), 36 deletions(-)