From patchwork Fri Mar 1 05:03:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirsher, Jeffrey T" X-Patchwork-Id: 1049967 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=intel.com Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 449cly47n0z9s47 for ; Fri, 1 Mar 2019 16:04:30 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728147AbfCAFE3 (ORCPT ); Fri, 1 Mar 2019 00:04:29 -0500 Received: from mga09.intel.com ([134.134.136.24]:42691 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725978AbfCAFEC (ORCPT ); Fri, 1 Mar 2019 00:04:02 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Feb 2019 21:04:01 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,426,1544515200"; d="scan'208";a="128139012" Received: from jtkirshe-desk1.jf.intel.com ([134.134.177.96]) by fmsmga008.fm.intel.com with ESMTP; 28 Feb 2019 21:04:00 -0800 From: Jeff Kirsher To: davem@davemloft.net Cc: Jacob Keller , netdev@vger.kernel.org, nhorman@redhat.com, sassmann@redhat.com, Jeff Kirsher Subject: [net-next 04/12] i40e: save PTP time before a device reset Date: Thu, 28 Feb 2019 21:03:44 -0800 Message-Id: <20190301050352.14978-5-jeffrey.t.kirsher@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190301050352.14978-1-jeffrey.t.kirsher@intel.com> References: <20190301050352.14978-1-jeffrey.t.kirsher@intel.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Jacob Keller In the case where PTP is running on the hardware clock, but the kernel system time is not being synced, a device reset can mess up the clock time. This occurs because we reset the clock time based on the kernel time every reset. This causes us to potentially completely reset the PTP time, and can cause unexpected behavior in programs like ptp4l. Avoid this by saving the PTP time prior to device reset, and then restoring using that time after the reset. Directly restoring the PTP time we saved isn't perfect, because time should have continued running, but the clock will essentially be stopped during the reset. This is still better than the current solution of assuming that the PTP HW clock is synced to the CLOCK_REALTIME. We can do even better, by saving the ktime and calculating a differential, using ktime_get(). This is based on CLOCK_MONOTONIC, and allows us to get a fairly precise measure of the time difference between saving and restoring the time. Using this, we can update the saved PTP time, and use that as the value to write to the hardware clock registers. This, of course is not perfect. However, it does help ensure that the PTP time is restored as close as feasible to the time it should have been if the reset had not occurred. During device initialization, continue using the system time as the source for the creation of the PTP clock, since this is the best known current time source at driver load. Signed-off-by: Jacob Keller Signed-off-by: Jeff Kirsher --- drivers/net/ethernet/intel/i40e/i40e.h | 4 ++ drivers/net/ethernet/intel/i40e/i40e_main.c | 5 ++ drivers/net/ethernet/intel/i40e/i40e_ptp.c | 58 +++++++++++++++++++-- 3 files changed, 63 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/intel/i40e/i40e.h b/drivers/net/ethernet/intel/i40e/i40e.h index 3f998f4cf033..3a93a0782c4e 100644 --- a/drivers/net/ethernet/intel/i40e/i40e.h +++ b/drivers/net/ethernet/intel/i40e/i40e.h @@ -612,6 +612,8 @@ struct i40e_pf { struct sk_buff *ptp_tx_skb; unsigned long ptp_tx_start; struct hwtstamp_config tstamp_config; + struct timespec64 ptp_prev_hw_time; + ktime_t ptp_reset_start; struct mutex tmreg_lock; /* Used to protect the SYSTIME registers. */ u32 ptp_adj_mult; u32 tx_hwtstamp_timeouts; @@ -1106,6 +1108,8 @@ void i40e_ptp_rx_hwtstamp(struct i40e_pf *pf, struct sk_buff *skb, u8 index); void i40e_ptp_set_increment(struct i40e_pf *pf); int i40e_ptp_set_ts_config(struct i40e_pf *pf, struct ifreq *ifr); int i40e_ptp_get_ts_config(struct i40e_pf *pf, struct ifreq *ifr); +void i40e_ptp_save_hw_time(struct i40e_pf *pf); +void i40e_ptp_restore_hw_time(struct i40e_pf *pf); void i40e_ptp_init(struct i40e_pf *pf); void i40e_ptp_stop(struct i40e_pf *pf); int i40e_is_vsi_uplink_mode_veb(struct i40e_vsi *vsi); diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c index f013a3e3c27a..de7a92197668 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_main.c +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c @@ -9281,6 +9281,11 @@ static void i40e_prep_for_reset(struct i40e_pf *pf, bool lock_acquired) dev_warn(&pf->pdev->dev, "shutdown_lan_hmc failed: %d\n", ret); } + + /* Save the current PTP time so that we can restore the time after the + * reset completes. + */ + i40e_ptp_save_hw_time(pf); } /** diff --git a/drivers/net/ethernet/intel/i40e/i40e_ptp.c b/drivers/net/ethernet/intel/i40e/i40e_ptp.c index 5fb4353c742b..66147c3f5f2c 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_ptp.c +++ b/drivers/net/ethernet/intel/i40e/i40e_ptp.c @@ -724,9 +724,56 @@ static long i40e_ptp_create_clock(struct i40e_pf *pf) pf->tstamp_config.rx_filter = HWTSTAMP_FILTER_NONE; pf->tstamp_config.tx_type = HWTSTAMP_TX_OFF; + /* Set the previous "reset" time to the current Kernel clock time */ + pf->ptp_prev_hw_time = ktime_to_timespec64(ktime_get_real()); + pf->ptp_reset_start = ktime_get(); + return 0; } +/** + * i40e_ptp_save_hw_time - Save the current PTP time as ptp_prev_hw_time + * @pf: Board private structure + * + * Read the current PTP time and save it into pf->ptp_prev_hw_time. This should + * be called at the end of preparing to reset, just before hardware reset + * occurs, in order to preserve the PTP time as close as possible across + * resets. + */ +void i40e_ptp_save_hw_time(struct i40e_pf *pf) +{ + /* don't try to access the PTP clock if it's not enabled */ + if (!(pf->flags & I40E_FLAG_PTP)) + return; + + i40e_ptp_gettimex(&pf->ptp_caps, &pf->ptp_prev_hw_time, NULL); + /* Get a monotonic starting time for this reset */ + pf->ptp_reset_start = ktime_get(); +} + +/** + * i40e_ptp_restore_hw_time - Restore the ptp_prev_hw_time + delta to PTP regs + * @pf: Board private structure + * + * Restore the PTP hardware clock registers. We previously cached the PTP + * hardware time as pf->ptp_prev_hw_time. To be as accurate as possible, + * update this value based on the time delta since the time was saved, using + * CLOCK_MONOTONIC (via ktime_get()) to calculate the time difference. + * + * This ensures that the hardware clock is restored to nearly what it should + * have been if a reset had not occurred. + */ +void i40e_ptp_restore_hw_time(struct i40e_pf *pf) +{ + ktime_t delta = ktime_sub(ktime_get(), pf->ptp_reset_start); + + /* Update the previous HW time with the ktime delta */ + timespec64_add_ns(&pf->ptp_prev_hw_time, ktime_to_ns(delta)); + + /* Restore the hardware clock registers */ + i40e_ptp_settime(&pf->ptp_caps, &pf->ptp_prev_hw_time); +} + /** * i40e_ptp_init - Initialize the 1588 support after device probe or reset * @pf: Board private structure @@ -734,6 +781,11 @@ static long i40e_ptp_create_clock(struct i40e_pf *pf) * This function sets device up for 1588 support. The first time it is run, it * will create a PHC clock device. It does not create a clock device if one * already exists. It also reconfigures the device after a reset. + * + * The first time a clock is created, i40e_ptp_create_clock will set + * pf->ptp_prev_hw_time to the current system time. During resets, it is + * expected that this timespec will be set to the last known PTP clock time, + * in order to preserve the clock time as close as possible across a reset. **/ void i40e_ptp_init(struct i40e_pf *pf) { @@ -765,7 +817,6 @@ void i40e_ptp_init(struct i40e_pf *pf) dev_err(&pf->pdev->dev, "%s: ptp_clock_register failed\n", __func__); } else if (pf->ptp_clock) { - struct timespec64 ts; u32 regval; if (pf->hw.debug_mask & I40E_DEBUG_LAN) @@ -786,9 +837,8 @@ void i40e_ptp_init(struct i40e_pf *pf) /* reset timestamping mode */ i40e_ptp_set_timestamp_mode(pf, &pf->tstamp_config); - /* Set the clock value. */ - ts = ktime_to_timespec64(ktime_get_real()); - i40e_ptp_settime(&pf->ptp_caps, &ts); + /* Restore the clock time based on last known value */ + i40e_ptp_restore_hw_time(pf); } }