| Message ID | 20191127082613.22903-4-tao3.xu@intel.com |
|---|---|
| State | New |
| Headers | show |
| Series | Build ACPI Heterogeneous Memory Attribute Table (HMAT) | expand |
Tao Xu <tao3.xu@intel.com> writes: > From: Liu Jingqi <jingqi.liu@intel.com> > > Add -numa hmat-cache option to provide Memory Side Cache Information. > These memory attributes help to build Memory Side Cache Information > Structure(s) in ACPI Heterogeneous Memory Attribute Table (HMAT). Please mention this requires -machine hmat=on. > Reviewed-by: Igor Mammedov <imammedo@redhat.com> > Reviewed-by: Daniel Black <daniel@linux.ibm.com> > Signed-off-by: Liu Jingqi <jingqi.liu@intel.com> > Signed-off-by: Tao Xu <tao3.xu@intel.com> [...] > diff --git a/qapi/machine.json b/qapi/machine.json > index c741649d7b..3d0ba226a9 100644 > --- a/qapi/machine.json > +++ b/qapi/machine.json > @@ -428,10 +428,12 @@ > # > # @hmat-lb: memory latency and bandwidth information (Since: 5.0) > # > +# @hmat-cache: memory side cache information (Since: 5.0) > +# > # Since: 2.1 > ## > { 'enum': 'NumaOptionsType', > - 'data': [ 'node', 'dist', 'cpu', 'hmat-lb' ] } > + 'data': [ 'node', 'dist', 'cpu', 'hmat-lb', 'hmat-cache' ] } > > ## > # @NumaOptions: > @@ -447,7 +449,8 @@ > 'node': 'NumaNodeOptions', > 'dist': 'NumaDistOptions', > 'cpu': 'NumaCpuOptions', > - 'hmat-lb': 'NumaHmatLBOptions' }} > + 'hmat-lb': 'NumaHmatLBOptions', > + 'hmat-cache': 'NumaHmatCacheOptions' }} > > ## > # @NumaNodeOptions: > @@ -647,6 +650,77 @@ > '*latency': 'uint64', > '*bandwidth': 'size' }} > > +## > +# @HmatCacheAssociativity: > +# > +# Cache associativity in the Memory Side Cache > +# Information Structure of HMAT > +# > +# For more information of @HmatCacheAssociativity see > +# the chapter 5.2.27.5: Table 5-147 of ACPI 6.3 spec. # Cache associativity in the Memory Side Cache Information Structure # of HMAT # # For more information of @HmatCacheAssociativity, see chapter # 5.2.27.5: Table 5-147 of ACPI 6.3 spec. > +# > +# @none: None What does cache associativity @none mean? A none-associative cache? I guess it makes sense to people familiar with the ACPI spec... > +# > +# @direct: Direct Mapped > +# > +# @complex: Complex Cache Indexing (implementation specific) > +# > +# Since: 5.0 > +## > +{ 'enum': 'HmatCacheAssociativity', > + 'data': [ 'none', 'direct', 'complex' ] } > + > +## > +# @HmatCacheWritePolicy: > +# > +# Cache write policy in the Memory Side Cache > +# Information Structure of HMAT > +# > +# For more information of @HmatCacheWritePolicy see > +# the chapter 5.2.27.5: Table 5-147: Field "Cache Attributes" of ACPI 6.3 spec. Break lines around column 70, please. > +# > +# @none: None What does cache write policy @none mean? > +# > +# @write-back: Write Back (WB) > +# > +# @write-through: Write Through (WT) > +# > +# Since: 5.0 > +## > +{ 'enum': 'HmatCacheWritePolicy', > + 'data': [ 'none', 'write-back', 'write-through' ] } > + > +## > +# @NumaHmatCacheOptions: > +# > +# Set the memory side cache information for a given memory domain. > +# > +# For more information of @NumaHmatCacheOptions see > +# the chapter 5.2.27.5: Table 5-147: Field "Cache Attributes" of ACPI 6.3 spec. > +# > +# @node-id: the memory proximity domain to which the memory belongs. > +# > +# @size: the size of memory side cache in bytes. > +# > +# @level: the cache level described in this structure. > +# > +# @assoc: the cache associativity, none/direct-mapped/complex(complex cache indexing). > +# > +# @policy: the write policy, none/write-back/write-through. > +# > +# @line: the cache Line size in bytes. > +# > +# Since: 5.0 > +## > +{ 'struct': 'NumaHmatCacheOptions', > + 'data': { > + 'node-id': 'uint32', Ignorant question: you use 'uint16' for other "proximity domains". Is 'uint32' intentional here? > + 'size': 'size', > + 'level': 'uint8', > + 'assoc': 'HmatCacheAssociativity', > + 'policy': 'HmatCacheWritePolicy', > + 'line': 'uint16' }} > + > ## > # @HostMemPolicy: > # [...]
On 11/27/2019 5:56 PM, Markus Armbruster wrote: > Tao Xu <tao3.xu@intel.com> writes: > >> From: Liu Jingqi <jingqi.liu@intel.com> >> >> Add -numa hmat-cache option to provide Memory Side Cache Information. >> These memory attributes help to build Memory Side Cache Information >> Structure(s) in ACPI Heterogeneous Memory Attribute Table (HMAT). > > Please mention this requires -machine hmat=on. OK I will add these for 3 related patches. > >> Reviewed-by: Igor Mammedov <imammedo@redhat.com> >> Reviewed-by: Daniel Black <daniel@linux.ibm.com> >> Signed-off-by: Liu Jingqi <jingqi.liu@intel.com> >> Signed-off-by: Tao Xu <tao3.xu@intel.com> > [...] >> diff --git a/qapi/machine.json b/qapi/machine.json >> index c741649d7b..3d0ba226a9 100644 >> --- a/qapi/machine.json >> +++ b/qapi/machine.json >> @@ -428,10 +428,12 @@ >> # >> # @hmat-lb: memory latency and bandwidth information (Since: 5.0) >> # >> +# @hmat-cache: memory side cache information (Since: 5.0) >> +# >> # Since: 2.1 >> ## >> { 'enum': 'NumaOptionsType', >> - 'data': [ 'node', 'dist', 'cpu', 'hmat-lb' ] } >> + 'data': [ 'node', 'dist', 'cpu', 'hmat-lb', 'hmat-cache' ] } >> >> ## >> # @NumaOptions: >> @@ -447,7 +449,8 @@ >> 'node': 'NumaNodeOptions', >> 'dist': 'NumaDistOptions', >> 'cpu': 'NumaCpuOptions', >> - 'hmat-lb': 'NumaHmatLBOptions' }} >> + 'hmat-lb': 'NumaHmatLBOptions', >> + 'hmat-cache': 'NumaHmatCacheOptions' }} >> >> ## >> # @NumaNodeOptions: >> @@ -647,6 +650,77 @@ >> '*latency': 'uint64', >> '*bandwidth': 'size' }} >> >> +## >> +# @HmatCacheAssociativity: >> +# >> +# Cache associativity in the Memory Side Cache >> +# Information Structure of HMAT >> +# >> +# For more information of @HmatCacheAssociativity see >> +# the chapter 5.2.27.5: Table 5-147 of ACPI 6.3 spec. > > # Cache associativity in the Memory Side Cache Information Structure > # of HMAT > # > # For more information of @HmatCacheAssociativity, see chapter > # 5.2.27.5: Table 5-147 of ACPI 6.3 spec. > >> +# >> +# @none: None > > What does cache associativity @none mean? A none-associative cache? I > guess it makes sense to people familiar with the ACPI spec... > This means this proximity domain has no memory cache, thus none for Cache associativity, I will add more description about this. >> +# >> +# @direct: Direct Mapped >> +# >> +# @complex: Complex Cache Indexing (implementation specific) >> +# >> +# Since: 5.0 >> +## >> +{ 'enum': 'HmatCacheAssociativity', >> + 'data': [ 'none', 'direct', 'complex' ] } >> + >> +## >> +# @HmatCacheWritePolicy: >> +# >> +# Cache write policy in the Memory Side Cache >> +# Information Structure of HMAT >> +# >> +# For more information of @HmatCacheWritePolicy see >> +# the chapter 5.2.27.5: Table 5-147: Field "Cache Attributes" of ACPI 6.3 spec. > > Break lines around column 70, please. > >> +# >> +# @none: None > > What does cache write policy @none mean? This means this proximity domain has no memory cache, thus none for cache write policy. > >> +# >> +# @write-back: Write Back (WB) >> +# >> +# @write-through: Write Through (WT) >> +# >> +# Since: 5.0 >> +## >> +{ 'enum': 'HmatCacheWritePolicy', >> + 'data': [ 'none', 'write-back', 'write-through' ] } >> + >> +## >> +# @NumaHmatCacheOptions: >> +# >> +# Set the memory side cache information for a given memory domain. >> +# >> +# For more information of @NumaHmatCacheOptions see >> +# the chapter 5.2.27.5: Table 5-147: Field "Cache Attributes" of ACPI 6.3 spec. >> +# >> +# @node-id: the memory proximity domain to which the memory belongs. >> +# >> +# @size: the size of memory side cache in bytes. >> +# >> +# @level: the cache level described in this structure. >> +# >> +# @assoc: the cache associativity, none/direct-mapped/complex(complex cache indexing). >> +# >> +# @policy: the write policy, none/write-back/write-through. >> +# >> +# @line: the cache Line size in bytes. >> +# >> +# Since: 5.0 >> +## >> +{ 'struct': 'NumaHmatCacheOptions', >> + 'data': { >> + 'node-id': 'uint32', > > Ignorant question: you use 'uint16' for other "proximity domains". Is > 'uint32' intentional here? > Yes, because ACPI 6.3 spec define the domain as 4 byte(32 bit), and for HmatCacheOptions we directly use this QAPI struct when building HMAT. But for other "proximity domains" we use local variable (such as "uint32_t initiator")
On 11/28/2019 10:46 AM, Tao Xu wrote: > On 11/27/2019 5:56 PM, Markus Armbruster wrote: >> Tao Xu <tao3.xu@intel.com> writes: >> >>> From: Liu Jingqi <jingqi.liu@intel.com> >>> >>> Add -numa hmat-cache option to provide Memory Side Cache Information. >>> These memory attributes help to build Memory Side Cache Information >>> Structure(s) in ACPI Heterogeneous Memory Attribute Table (HMAT). >> >> Please mention this requires -machine hmat=on. > > OK I will add these for 3 related patches. >> >>> Reviewed-by: Igor Mammedov <imammedo@redhat.com> >>> Reviewed-by: Daniel Black <daniel@linux.ibm.com> >>> Signed-off-by: Liu Jingqi <jingqi.liu@intel.com> >>> Signed-off-by: Tao Xu <tao3.xu@intel.com> >> [...] >>> diff --git a/qapi/machine.json b/qapi/machine.json >>> index c741649d7b..3d0ba226a9 100644 >>> --- a/qapi/machine.json >>> +++ b/qapi/machine.json >>> @@ -428,10 +428,12 @@ >>> # >>> # @hmat-lb: memory latency and bandwidth information (Since: 5.0) >>> # >>> +# @hmat-cache: memory side cache information (Since: 5.0) >>> +# >>> # Since: 2.1 >>> ## >>> { 'enum': 'NumaOptionsType', >>> - 'data': [ 'node', 'dist', 'cpu', 'hmat-lb' ] } >>> + 'data': [ 'node', 'dist', 'cpu', 'hmat-lb', 'hmat-cache' ] } >>> ## >>> # @NumaOptions: >>> @@ -447,7 +449,8 @@ >>> 'node': 'NumaNodeOptions', >>> 'dist': 'NumaDistOptions', >>> 'cpu': 'NumaCpuOptions', >>> - 'hmat-lb': 'NumaHmatLBOptions' }} >>> + 'hmat-lb': 'NumaHmatLBOptions', >>> + 'hmat-cache': 'NumaHmatCacheOptions' }} >>> ## >>> # @NumaNodeOptions: >>> @@ -647,6 +650,77 @@ >>> '*latency': 'uint64', >>> '*bandwidth': 'size' }} >>> +## >>> +# @HmatCacheAssociativity: >>> +# >>> +# Cache associativity in the Memory Side Cache >>> +# Information Structure of HMAT >>> +# >>> +# For more information of @HmatCacheAssociativity see >>> +# the chapter 5.2.27.5: Table 5-147 of ACPI 6.3 spec. >> >> # Cache associativity in the Memory Side Cache Information Structure >> # of HMAT >> # >> # For more information of @HmatCacheAssociativity, see chapter >> # 5.2.27.5: Table 5-147 of ACPI 6.3 spec. >> >>> +# >>> +# @none: None >> >> What does cache associativity @none mean? A none-associative cache? I >> guess it makes sense to people familiar with the ACPI spec... >> > > This means this proximity domain has no memory cache, thus none for > Cache associativity, I will add more description about this. Read again about ACPI spec, there is no description about 'none'. In linux kernel HMAT code, this is handle as "other", maybe means not provided. I will also add a check when level is none, the associativity, policy and line_size should be none or 0.
diff --git a/hw/core/numa.c b/hw/core/numa.c index 2183c8df1f..3d87fcbf2d 100644 --- a/hw/core/numa.c +++ b/hw/core/numa.c @@ -366,6 +366,71 @@ void parse_numa_hmat_lb(NumaState *numa_state, NumaHmatLBOptions *node, g_array_append_val(hmat_lb->list, lb_data); } +void parse_numa_hmat_cache(MachineState *ms, NumaHmatCacheOptions *node, + Error **errp) +{ + int nb_numa_nodes = ms->numa_state->num_nodes; + NodeInfo *numa_info = ms->numa_state->nodes; + NumaHmatCacheOptions *hmat_cache = NULL; + + if (node->node_id >= nb_numa_nodes) { + error_setg(errp, "Invalid node-id=%" PRIu32 ", it should be less " + "than %d", node->node_id, nb_numa_nodes); + return; + } + + if (numa_info[node->node_id].lb_info_provided != (BIT(0) | BIT(1))) { + error_setg(errp, "The latency and bandwidth information of " + "node-id=%" PRIu32 " should be provided before memory side " + "cache attributes", node->node_id); + return; + } + + if (node->level >= HMAT_LB_LEVELS) { + error_setg(errp, "Invalid level=%" PRIu8 ", it should be less than or " + "equal to %d", node->level, HMAT_LB_LEVELS - 1); + return; + } + assert(node->assoc < HMAT_CACHE_ASSOCIATIVITY__MAX); + assert(node->policy < HMAT_CACHE_WRITE_POLICY__MAX); + if (ms->numa_state->hmat_cache[node->node_id][node->level]) { + error_setg(errp, "Duplicate configuration of the side cache for " + "node-id=%" PRIu32 " and level=%" PRIu8, + node->node_id, node->level); + return; + } + + if ((node->level > 1) && + ms->numa_state->hmat_cache[node->node_id][node->level - 1] && + (node->size >= + ms->numa_state->hmat_cache[node->node_id][node->level - 1]->size)) { + error_setg(errp, "Invalid size=%" PRIu64 ", the size of level=%" PRIu8 + " should be less than the size(%" PRIu64 ") of " + "level=%" PRIu8, node->size, node->level, + ms->numa_state->hmat_cache[node->node_id] + [node->level - 1]->size, + node->level - 1); + return; + } + + if ((node->level < HMAT_LB_LEVELS - 1) && + ms->numa_state->hmat_cache[node->node_id][node->level + 1] && + (node->size <= + ms->numa_state->hmat_cache[node->node_id][node->level + 1]->size)) { + error_setg(errp, "Invalid size=%" PRIu64 ", the size of level=%" PRIu8 + " should be larger than the size(%" PRIu64 ") of " + "level=%" PRIu8, node->size, node->level, + ms->numa_state->hmat_cache[node->node_id] + [node->level + 1]->size, + node->level + 1); + return; + } + + hmat_cache = g_malloc0(sizeof(*hmat_cache)); + memcpy(hmat_cache, node, sizeof(*hmat_cache)); + ms->numa_state->hmat_cache[node->node_id][node->level] = hmat_cache; +} + void set_numa_options(MachineState *ms, NumaOptions *object, Error **errp) { Error *err = NULL; @@ -417,6 +482,19 @@ void set_numa_options(MachineState *ms, NumaOptions *object, Error **errp) goto end; } break; + case NUMA_OPTIONS_TYPE_HMAT_CACHE: + if (!ms->numa_state->hmat_enabled) { + error_setg(errp, "ACPI Heterogeneous Memory Attribute Table " + "(HMAT) is disabled, enable it with -machine hmat=on " + "before using any of hmat specific options"); + return; + } + + parse_numa_hmat_cache(ms, &object->u.hmat_cache, &err); + if (err) { + goto end; + } + break; default: abort(); } diff --git a/include/sysemu/numa.h b/include/sysemu/numa.h index 70f93c83d7..ba693cc80b 100644 --- a/include/sysemu/numa.h +++ b/include/sysemu/numa.h @@ -91,6 +91,9 @@ struct NumaState { /* NUMA nodes HMAT Locality Latency and Bandwidth Information */ HMAT_LB_Info *hmat_lb[HMAT_LB_LEVELS][HMAT_LB_TYPES]; + + /* Memory Side Cache Information Structure */ + NumaHmatCacheOptions *hmat_cache[MAX_NODES][HMAT_LB_LEVELS]; }; typedef struct NumaState NumaState; @@ -98,6 +101,8 @@ void set_numa_options(MachineState *ms, NumaOptions *object, Error **errp); void parse_numa_opts(MachineState *ms); void parse_numa_hmat_lb(NumaState *numa_state, NumaHmatLBOptions *node, Error **errp); +void parse_numa_hmat_cache(MachineState *ms, NumaHmatCacheOptions *node, + Error **errp); void numa_complete_configuration(MachineState *ms); void query_numa_node_mem(NumaNodeMem node_mem[], MachineState *ms); extern QemuOptsList qemu_numa_opts; diff --git a/qapi/machine.json b/qapi/machine.json index c741649d7b..3d0ba226a9 100644 --- a/qapi/machine.json +++ b/qapi/machine.json @@ -428,10 +428,12 @@ # # @hmat-lb: memory latency and bandwidth information (Since: 5.0) # +# @hmat-cache: memory side cache information (Since: 5.0) +# # Since: 2.1 ## { 'enum': 'NumaOptionsType', - 'data': [ 'node', 'dist', 'cpu', 'hmat-lb' ] } + 'data': [ 'node', 'dist', 'cpu', 'hmat-lb', 'hmat-cache' ] } ## # @NumaOptions: @@ -447,7 +449,8 @@ 'node': 'NumaNodeOptions', 'dist': 'NumaDistOptions', 'cpu': 'NumaCpuOptions', - 'hmat-lb': 'NumaHmatLBOptions' }} + 'hmat-lb': 'NumaHmatLBOptions', + 'hmat-cache': 'NumaHmatCacheOptions' }} ## # @NumaNodeOptions: @@ -647,6 +650,77 @@ '*latency': 'uint64', '*bandwidth': 'size' }} +## +# @HmatCacheAssociativity: +# +# Cache associativity in the Memory Side Cache +# Information Structure of HMAT +# +# For more information of @HmatCacheAssociativity see +# the chapter 5.2.27.5: Table 5-147 of ACPI 6.3 spec. +# +# @none: None +# +# @direct: Direct Mapped +# +# @complex: Complex Cache Indexing (implementation specific) +# +# Since: 5.0 +## +{ 'enum': 'HmatCacheAssociativity', + 'data': [ 'none', 'direct', 'complex' ] } + +## +# @HmatCacheWritePolicy: +# +# Cache write policy in the Memory Side Cache +# Information Structure of HMAT +# +# For more information of @HmatCacheWritePolicy see +# the chapter 5.2.27.5: Table 5-147: Field "Cache Attributes" of ACPI 6.3 spec. +# +# @none: None +# +# @write-back: Write Back (WB) +# +# @write-through: Write Through (WT) +# +# Since: 5.0 +## +{ 'enum': 'HmatCacheWritePolicy', + 'data': [ 'none', 'write-back', 'write-through' ] } + +## +# @NumaHmatCacheOptions: +# +# Set the memory side cache information for a given memory domain. +# +# For more information of @NumaHmatCacheOptions see +# the chapter 5.2.27.5: Table 5-147: Field "Cache Attributes" of ACPI 6.3 spec. +# +# @node-id: the memory proximity domain to which the memory belongs. +# +# @size: the size of memory side cache in bytes. +# +# @level: the cache level described in this structure. +# +# @assoc: the cache associativity, none/direct-mapped/complex(complex cache indexing). +# +# @policy: the write policy, none/write-back/write-through. +# +# @line: the cache Line size in bytes. +# +# Since: 5.0 +## +{ 'struct': 'NumaHmatCacheOptions', + 'data': { + 'node-id': 'uint32', + 'size': 'size', + 'level': 'uint8', + 'assoc': 'HmatCacheAssociativity', + 'policy': 'HmatCacheWritePolicy', + 'line': 'uint16' }} + ## # @HostMemPolicy: # diff --git a/qemu-options.hx b/qemu-options.hx index 6a9f738dc5..108ee3b553 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -169,7 +169,8 @@ DEF("numa", HAS_ARG, QEMU_OPTION_numa, "-numa node[,memdev=id][,cpus=firstcpu[-lastcpu]][,nodeid=node][,initiator=node]\n" "-numa dist,src=source,dst=destination,val=distance\n" "-numa cpu,node-id=node[,socket-id=x][,core-id=y][,thread-id=z]\n" - "-numa hmat-lb,initiator=node,target=node,hierarchy=memory|first-level|second-level|third-level,data-type=access-latency|read-latency|write-latency[,latency=lat][,bandwidth=bw]\n", + "-numa hmat-lb,initiator=node,target=node,hierarchy=memory|first-level|second-level|third-level,data-type=access-latency|read-latency|write-latency[,latency=lat][,bandwidth=bw]\n" + "-numa hmat-cache,node-id=node,size=size,level=level[,assoc=none|direct|complex][,policy=none|write-back|write-through][,line=size]\n", QEMU_ARCH_ALL) STEXI @item -numa node[,mem=@var{size}][,cpus=@var{firstcpu}[-@var{lastcpu}]][,nodeid=@var{node}][,initiator=@var{initiator}] @@ -177,6 +178,7 @@ STEXI @itemx -numa dist,src=@var{source},dst=@var{destination},val=@var{distance} @itemx -numa cpu,node-id=@var{node}[,socket-id=@var{x}][,core-id=@var{y}][,thread-id=@var{z}] @itemx -numa hmat-lb,initiator=@var{node},target=@var{node},hierarchy=@var{hierarchy},data-type=@var{tpye}[,latency=@var{lat}][,bandwidth=@var{bw}] +@itemx -numa hmat-cache,node-id=@var{node},size=@var{size},level=@var{level}[,assoc=@var{str}][,policy=@var{str}][,line=@var{size}] @findex -numa Define a NUMA node and assign RAM and VCPUs to it. Set the NUMA distance from a source node to a destination node. @@ -280,11 +282,19 @@ NUM byte (or MB/s, GB/s or TB/s depending on used suffix). And if input bandwidt value without any unit, the unit will be B/s. Note that if latency or bandwidth value is 0, means the corresponding latency or bandwidth information is not provided. +In @samp{hmat-cache} option, @var{node-id} is the NUMA-id of the memory belongs. +@var{size} is the size of memory side cache in bytes. @var{level} is the cache +level described in this structure. @var{assoc} is the cache associativity, +the possible value is 'none/direct(direct-mapped)/complex(complex cache indexing)'. +@var{policy} is the write policy. @var{line} is the cache Line size in bytes. + For example, the following options describe 2 NUMA nodes. Node 0 has 2 cpus and a ram, node 1 has only a ram. The processors in node 0 access memory in node 0 with access-latency 5 nanoseconds, access-bandwidth is 200 MB/s; The processors in NUMA node 0 access memory in NUMA node 1 with access-latency 10 nanoseconds, access-bandwidth is 100 MB/s. +And for memory side cache information, NUMA node 0 and 1 both have 1 level memory +cache, size is 10KB, policy is write-back, the cache Line size is 8 bytes: @example -machine hmat=on \ -m 2G \ @@ -298,7 +308,9 @@ nanoseconds, access-bandwidth is 100 MB/s. -numa hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-latency,latency=5 \ -numa hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-bandwidth,bandwidth=200M \ -numa hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=access-latency,latency=10 \ --numa hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=access-bandwidth,bandwidth=100M +-numa hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=access-bandwidth,bandwidth=100M \ +-numa hmat-cache,node-id=0,size=10K,level=1,assoc=direct,policy=write-back,line=8 \ +-numa hmat-cache,node-id=1,size=10K,level=1,assoc=direct,policy=write-back,line=8 @end example ETEXI