[4/6] cifs: prevent starvation in wait_for_free_credits for multi-credit requests
diff mbox series

Message ID 20190308025823.24382-5-lsahlber@redhat.com
State New
Headers show
Series
  • cifs: simplify handling of credits for compounds
Related show

Commit Message

Ronnie Sahlberg March 8, 2019, 2:58 a.m. UTC
Reserve the last MAX_COMPOUND credits for any request asking for >1 credit.
This is to prevent future compound requests from becoming starved while waiting
for potentially many requests is there is a large number of concurrent
singe-credit requests.

However, we need to protect from servers that are very slow to hand out
new credits on new sessions so we only do this IFF there are 2*MAX_COMPOUND
(arbitrary) credits already in flight.

Signed-off-by: Ronnie Sahlberg <lsahlber@redhat.com>
---
 fs/cifs/transport.c | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

Comments

Pavel Shilovsky March 9, 2019, 12:55 a.m. UTC | #1
чт, 7 мар. 2019 г. в 19:35, Ronnie Sahlberg <lsahlber@redhat.com>:
>
> Reserve the last MAX_COMPOUND credits for any request asking for >1 credit.
> This is to prevent future compound requests from becoming starved while waiting
> for potentially many requests is there is a large number of concurrent
> singe-credit requests.
>
> However, we need to protect from servers that are very slow to hand out
> new credits on new sessions so we only do this IFF there are 2*MAX_COMPOUND
> (arbitrary) credits already in flight.
>
> Signed-off-by: Ronnie Sahlberg <lsahlber@redhat.com>
> ---
>  fs/cifs/transport.c | 28 ++++++++++++++++++++++++++++
>  1 file changed, 28 insertions(+)
>
> diff --git a/fs/cifs/transport.c b/fs/cifs/transport.c
> index baf15194aa3d..f4ec9635dab2 100644
> --- a/fs/cifs/transport.c
> +++ b/fs/cifs/transport.c
> @@ -520,6 +520,34 @@ wait_for_free_credits(struct TCP_Server_Info *server, const int num_credits,
>                         }
>
>                         /*
> +                        * For normal commands, reserve the last MAX_COMPOUND
> +                        * credits to compound requests.
> +                        * Otherwise these compounds could be permanently
> +                        * starved for credits by single-credit requests.
> +                        *
> +                        * To prevent spinning CPU, block this thread until
> +                        * there are >MAX_COMPOUND credits available.
> +                        * But only do this is we already have a lot of
> +                        * credits in flight to avoid triggering this check
> +                        * for servers that are slow to hand out credits on
> +                        * new sessions.
> +                        */
> +                       if (!optype && num_credits == 1 &&
> +                           server->in_flight > 2 * MAX_COMPOUND &&
> +                           *credits <= MAX_COMPOUND) {
> +                               spin_unlock(&server->req_lock);
> +                               cifs_num_waiters_inc(server);
> +                               rc = wait_event_killable(server->request_q,
> +                                       has_credits(server, credits,
> +                                                   MAX_COMPOUND + 1));
> +                               cifs_num_waiters_dec(server);
> +                               if (rc)
> +                                       return rc;
> +                               spin_lock(&server->req_lock);
> +                               continue;
> +                       }
> +
> +                       /*
>                          * Can not count locking commands against total
>                          * as they are allowed to block on server.
>                          */
> --
> 2.13.6
>

We need to watch how the the approach above work in real workloads but
it looks promising.

Reviewed-by: Pavel Shilovsky <pshilov@microsoft.com>

--
Best regards,
Pavel Shilovsky

Patch
diff mbox series

diff --git a/fs/cifs/transport.c b/fs/cifs/transport.c
index baf15194aa3d..f4ec9635dab2 100644
--- a/fs/cifs/transport.c
+++ b/fs/cifs/transport.c
@@ -520,6 +520,34 @@  wait_for_free_credits(struct TCP_Server_Info *server, const int num_credits,
 			}
 
 			/*
+			 * For normal commands, reserve the last MAX_COMPOUND
+			 * credits to compound requests.
+			 * Otherwise these compounds could be permanently
+			 * starved for credits by single-credit requests.
+			 *
+			 * To prevent spinning CPU, block this thread until
+			 * there are >MAX_COMPOUND credits available.
+			 * But only do this is we already have a lot of
+			 * credits in flight to avoid triggering this check
+			 * for servers that are slow to hand out credits on
+			 * new sessions.
+			 */
+			if (!optype && num_credits == 1 &&
+			    server->in_flight > 2 * MAX_COMPOUND &&
+			    *credits <= MAX_COMPOUND) {
+				spin_unlock(&server->req_lock);
+				cifs_num_waiters_inc(server);
+				rc = wait_event_killable(server->request_q,
+					has_credits(server, credits,
+						    MAX_COMPOUND + 1));
+				cifs_num_waiters_dec(server);
+				if (rc)
+					return rc;
+				spin_lock(&server->req_lock);
+				continue;
+			}
+
+			/*
 			 * Can not count locking commands against total
 			 * as they are allowed to block on server.
 			 */