| Message ID | 4C8F9FDE.8050004@codemonkey.ws |
|---|---|
| State | New |
| Headers | show |
On 09/14/2010 11:28 AM, Avi Kivity wrote: > On 09/14/2010 06:16 PM, Anthony Liguori wrote: >> >> Right, it should only freeze if the L2 table needs to be allocated, >> not if it only needs to be updated. IOW, >> >> diff --git a/block/qed.c b/block/qed.c >> index 4c4e7a2..0357c03 100644 >> --- a/block/qed.c >> +++ b/block/qed.c >> @@ -948,7 +948,7 @@ static void qed_aio_write_data(void *opaque, int >> ret, >> } >> >> /* Freeze this request if another allocating write is in >> progress */ >> - if (need_alloc) { >> + if (ret == QED_CLUSTER_L1) { >> if (acb != QSIMPLEQ_FIRST(&s->allocating_write_reqs)) { >> QSIMPLEQ_INSERT_TAIL(&s->allocating_write_reqs, acb, next); >> } >> >> It's being a bit more conservative than it needs to be. > > Yes, I hit this too. So without this patch, it does serialize all > allocating writes? Yes, but my patch is not enough as it turns out. When dealing with O_DIRECT, we have to handle RMW on our own which means we need to serialize access to the same sector. The way we're planning on addressing this in the short term is to break the single allocator queue into a per-L2 table queue. So writes to the same L2 would be serialized but writes to different L2s would not be serialized. Regards, Anthony Liguori > If multiple requests need to update pointers in L2, will those updates > generate one write per request, or just two writes (one write from the > first request, another from all those that serialized after it)? >
On 09/14/2010 07:08 PM, Anthony Liguori wrote: >> Yes, I hit this too. So without this patch, it does serialize all >> allocating writes? > > > Yes, but my patch is not enough as it turns out. > > When dealing with O_DIRECT, we have to handle RMW on our own which > means we need to serialize access to the same sector. > > The way we're planning on addressing this in the short term is to > break the single allocator queue into a per-L2 table queue. So writes > to the same L2 would be serialized but writes to different L2s would > not be serialized. > So at least I read the code correctly. The next step (also addressed in the qcow2 performance plan) is to batch writes to L2. You'd actually expect to have many concurrent allocating writes to one L2. The first is sent to disk, but the following ones just mark the L2 dirty. When the write returns, it sees it's still dirty and goes back to disk again.
On 09/14/2010 12:23 PM, Avi Kivity wrote: > On 09/14/2010 07:08 PM, Anthony Liguori wrote: >>> Yes, I hit this too. So without this patch, it does serialize all >>> allocating writes? >> >> >> Yes, but my patch is not enough as it turns out. >> >> When dealing with O_DIRECT, we have to handle RMW on our own which >> means we need to serialize access to the same sector. >> >> The way we're planning on addressing this in the short term is to >> break the single allocator queue into a per-L2 table queue. So >> writes to the same L2 would be serialized but writes to different L2s >> would not be serialized. >> > > So at least I read the code correctly. > > The next step (also addressed in the qcow2 performance plan) is to > batch writes to L2. You'd actually expect to have many concurrent > allocating writes to one L2. The first is sent to disk, but the > following ones just mark the L2 dirty. When the write returns, it > sees it's still dirty and goes back to disk again. Yeah, I have to think through batching quite a bit more but I agree that batching should be a natural next step and can further reduce the cost of updating metadata in a streaming workload. Regards, Anthony Liguori
diff --git a/block/qed.c b/block/qed.c index 4c4e7a2..0357c03 100644 --- a/block/qed.c +++ b/block/qed.c @@ -948,7 +948,7 @@ static void qed_aio_write_data(void *opaque, int ret, } /* Freeze this request if another allocating write is in progress */ - if (need_alloc) { + if (ret == QED_CLUSTER_L1) { if (acb != QSIMPLEQ_FIRST(&s->allocating_write_reqs)) { QSIMPLEQ_INSERT_TAIL(&s->allocating_write_reqs, acb, next); }