diff mbox

[V3,02/19] memory: tegra: Add MC flush support

Message ID 1436791197-32358-3-git-send-email-jonathanh@nvidia.com
State Deferred, archived
Headers show

Commit Message

Jon Hunter July 13, 2015, 12:39 p.m. UTC
The Tegra memory controller implements a flush feature to flush pending
accesses and prevent further accesses from occurring. This feature is
used when powering down IP blocks to ensure the IP block is in a good
state. The flushes are organised by software groups and IP blocks are
assigned in hardware to the different software groups. Add helper
functions for requesting a handle to an MC flush for a given
software group and enabling/disabling the MC flush itself.

This is based upon a change by Vince Hsu <vinceh@nvidia.com>.

Signed-off-by: Jon Hunter <jonathanh@nvidia.com>
---
 drivers/memory/tegra/mc.c | 110 ++++++++++++++++++++++++++++++++++++++++++++++
 drivers/memory/tegra/mc.h |   2 +
 include/soc/tegra/mc.h    |  34 ++++++++++++++
 3 files changed, 146 insertions(+)

Comments

Thierry Reding July 17, 2015, 9:57 a.m. UTC | #1
On Mon, Jul 13, 2015 at 01:39:40PM +0100, Jon Hunter wrote:
> The Tegra memory controller implements a flush feature to flush pending
> accesses and prevent further accesses from occurring. This feature is
> used when powering down IP blocks to ensure the IP block is in a good
> state. The flushes are organised by software groups and IP blocks are
> assigned in hardware to the different software groups. Add helper
> functions for requesting a handle to an MC flush for a given
> software group and enabling/disabling the MC flush itself.
> 
> This is based upon a change by Vince Hsu <vinceh@nvidia.com>.
> 
> Signed-off-by: Jon Hunter <jonathanh@nvidia.com>
> ---
>  drivers/memory/tegra/mc.c | 110 ++++++++++++++++++++++++++++++++++++++++++++++
>  drivers/memory/tegra/mc.h |   2 +
>  include/soc/tegra/mc.h    |  34 ++++++++++++++
>  3 files changed, 146 insertions(+)

Do we know if this is actually necessary? I remember having a discussion
with Arnd Bergmann a while ago, and the Linux driver model kind of
assumes that by the time a device is disabled all outstanding accesses
will have stopped.

Do we have a way to determine that this even makes a difference? Can we
trigger a case where not doing this would cause breakage and see that
adding this fixes that particular issue?

Thierry
Peter De Schrijver July 17, 2015, 10:20 a.m. UTC | #2
On Fri, Jul 17, 2015 at 11:57:55AM +0200, Thierry Reding wrote:
> * PGP Signed by an unknown key
> 
> On Mon, Jul 13, 2015 at 01:39:40PM +0100, Jon Hunter wrote:
> > The Tegra memory controller implements a flush feature to flush pending
> > accesses and prevent further accesses from occurring. This feature is
> > used when powering down IP blocks to ensure the IP block is in a good
> > state. The flushes are organised by software groups and IP blocks are
> > assigned in hardware to the different software groups. Add helper
> > functions for requesting a handle to an MC flush for a given
> > software group and enabling/disabling the MC flush itself.
> > 
> > This is based upon a change by Vince Hsu <vinceh@nvidia.com>.
> > 
> > Signed-off-by: Jon Hunter <jonathanh@nvidia.com>
> > ---
> >  drivers/memory/tegra/mc.c | 110 ++++++++++++++++++++++++++++++++++++++++++++++
> >  drivers/memory/tegra/mc.h |   2 +
> >  include/soc/tegra/mc.h    |  34 ++++++++++++++
> >  3 files changed, 146 insertions(+)
> 
> Do we know if this is actually necessary? I remember having a discussion
> with Arnd Bergmann a while ago, and the Linux driver model kind of
> assumes that by the time a device is disabled all outstanding accesses
> will have stopped.
> 
> Do we have a way to determine that this even makes a difference? Can we
> trigger a case where not doing this would cause breakage and see that
> adding this fixes that particular issue?
> 

Most likely it is. The memory controller can still be processing requests
when the peripheral domain is powergated. This would mean the response cannot
be delivered in that case. So we need to be sure there are no outstanding
requests before shutting down the domain.

Cheers,

Peter.
--
To unsubscribe from this list: send the line "unsubscribe linux-tegra" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Thierry Reding July 17, 2015, 11:31 a.m. UTC | #3
On Fri, Jul 17, 2015 at 01:20:49PM +0300, Peter De Schrijver wrote:
> On Fri, Jul 17, 2015 at 11:57:55AM +0200, Thierry Reding wrote:
> > * PGP Signed by an unknown key
> > 
> > On Mon, Jul 13, 2015 at 01:39:40PM +0100, Jon Hunter wrote:
> > > The Tegra memory controller implements a flush feature to flush pending
> > > accesses and prevent further accesses from occurring. This feature is
> > > used when powering down IP blocks to ensure the IP block is in a good
> > > state. The flushes are organised by software groups and IP blocks are
> > > assigned in hardware to the different software groups. Add helper
> > > functions for requesting a handle to an MC flush for a given
> > > software group and enabling/disabling the MC flush itself.
> > > 
> > > This is based upon a change by Vince Hsu <vinceh@nvidia.com>.
> > > 
> > > Signed-off-by: Jon Hunter <jonathanh@nvidia.com>
> > > ---
> > >  drivers/memory/tegra/mc.c | 110 ++++++++++++++++++++++++++++++++++++++++++++++
> > >  drivers/memory/tegra/mc.h |   2 +
> > >  include/soc/tegra/mc.h    |  34 ++++++++++++++
> > >  3 files changed, 146 insertions(+)
> > 
> > Do we know if this is actually necessary? I remember having a discussion
> > with Arnd Bergmann a while ago, and the Linux driver model kind of
> > assumes that by the time a device is disabled all outstanding accesses
> > will have stopped.
> > 
> > Do we have a way to determine that this even makes a difference? Can we
> > trigger a case where not doing this would cause breakage and see that
> > adding this fixes that particular issue?
> > 
> 
> Most likely it is. The memory controller can still be processing requests
> when the peripheral domain is powergated. This would mean the response cannot
> be delivered in that case. So we need to be sure there are no outstanding
> requests before shutting down the domain.

My point is that that's the driver's responsibility anyway, hence making
the explicit flush unnecessary.

Thierry
Jon Hunter July 20, 2015, 8:46 a.m. UTC | #4
On 17/07/15 12:31, Thierry Reding wrote:
> * PGP Signed by an unknown key
> 
> On Fri, Jul 17, 2015 at 01:20:49PM +0300, Peter De Schrijver wrote:
>> On Fri, Jul 17, 2015 at 11:57:55AM +0200, Thierry Reding wrote:
>>>> Old Signed by an unknown key
>>>
>>> On Mon, Jul 13, 2015 at 01:39:40PM +0100, Jon Hunter wrote:
>>>> The Tegra memory controller implements a flush feature to flush pending
>>>> accesses and prevent further accesses from occurring. This feature is
>>>> used when powering down IP blocks to ensure the IP block is in a good
>>>> state. The flushes are organised by software groups and IP blocks are
>>>> assigned in hardware to the different software groups. Add helper
>>>> functions for requesting a handle to an MC flush for a given
>>>> software group and enabling/disabling the MC flush itself.
>>>>
>>>> This is based upon a change by Vince Hsu <vinceh@nvidia.com>.
>>>>
>>>> Signed-off-by: Jon Hunter <jonathanh@nvidia.com>
>>>> ---
>>>>  drivers/memory/tegra/mc.c | 110 ++++++++++++++++++++++++++++++++++++++++++++++
>>>>  drivers/memory/tegra/mc.h |   2 +
>>>>  include/soc/tegra/mc.h    |  34 ++++++++++++++
>>>>  3 files changed, 146 insertions(+)
>>>
>>> Do we know if this is actually necessary? I remember having a discussion
>>> with Arnd Bergmann a while ago, and the Linux driver model kind of
>>> assumes that by the time a device is disabled all outstanding accesses
>>> will have stopped.
>>>
>>> Do we have a way to determine that this even makes a difference? Can we
>>> trigger a case where not doing this would cause breakage and see that
>>> adding this fixes that particular issue?
>>>
>>
>> Most likely it is. The memory controller can still be processing requests
>> when the peripheral domain is powergated. This would mean the response cannot
>> be delivered in that case. So we need to be sure there are no outstanding
>> requests before shutting down the domain.
> 
> My point is that that's the driver's responsibility anyway, hence making
> the explicit flush unnecessary.

I see your point and it is interesting. The trouble is that we would
need to test every memory client in every power domain to prove this. So
I don't think that is a trivial thing to do. Furthermore, looking at
what we have done in kernel used for android products (which probably
stress PM the most) this is done and so I don't know of any shipping
product that stresses PM that does not do this. May be someone else
might. I personally would not be comfortable removing this without
testing, but as I mentioned it is not a trivial thing to test correctly.
However, I will let you and the other maintainers decide what's best here.

Jon
--
To unsubscribe from this list: send the line "unsubscribe linux-tegra" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Thierry Reding July 20, 2015, 9:17 a.m. UTC | #5
On Mon, Jul 20, 2015 at 09:46:02AM +0100, Jon Hunter wrote:
> 
> On 17/07/15 12:31, Thierry Reding wrote:
> > * PGP Signed by an unknown key
> > 
> > On Fri, Jul 17, 2015 at 01:20:49PM +0300, Peter De Schrijver wrote:
> >> On Fri, Jul 17, 2015 at 11:57:55AM +0200, Thierry Reding wrote:
> >>>> Old Signed by an unknown key
> >>>
> >>> On Mon, Jul 13, 2015 at 01:39:40PM +0100, Jon Hunter wrote:
> >>>> The Tegra memory controller implements a flush feature to flush pending
> >>>> accesses and prevent further accesses from occurring. This feature is
> >>>> used when powering down IP blocks to ensure the IP block is in a good
> >>>> state. The flushes are organised by software groups and IP blocks are
> >>>> assigned in hardware to the different software groups. Add helper
> >>>> functions for requesting a handle to an MC flush for a given
> >>>> software group and enabling/disabling the MC flush itself.
> >>>>
> >>>> This is based upon a change by Vince Hsu <vinceh@nvidia.com>.
> >>>>
> >>>> Signed-off-by: Jon Hunter <jonathanh@nvidia.com>
> >>>> ---
> >>>>  drivers/memory/tegra/mc.c | 110 ++++++++++++++++++++++++++++++++++++++++++++++
> >>>>  drivers/memory/tegra/mc.h |   2 +
> >>>>  include/soc/tegra/mc.h    |  34 ++++++++++++++
> >>>>  3 files changed, 146 insertions(+)
> >>>
> >>> Do we know if this is actually necessary? I remember having a discussion
> >>> with Arnd Bergmann a while ago, and the Linux driver model kind of
> >>> assumes that by the time a device is disabled all outstanding accesses
> >>> will have stopped.
> >>>
> >>> Do we have a way to determine that this even makes a difference? Can we
> >>> trigger a case where not doing this would cause breakage and see that
> >>> adding this fixes that particular issue?
> >>>
> >>
> >> Most likely it is. The memory controller can still be processing requests
> >> when the peripheral domain is powergated. This would mean the response cannot
> >> be delivered in that case. So we need to be sure there are no outstanding
> >> requests before shutting down the domain.
> > 
> > My point is that that's the driver's responsibility anyway, hence making
> > the explicit flush unnecessary.
> 
> I see your point and it is interesting. The trouble is that we would
> need to test every memory client in every power domain to prove this. So
> I don't think that is a trivial thing to do. Furthermore, looking at
> what we have done in kernel used for android products (which probably
> stress PM the most) this is done and so I don't know of any shipping
> product that stresses PM that does not do this. May be someone else
> might. I personally would not be comfortable removing this without
> testing, but as I mentioned it is not a trivial thing to test correctly.
> However, I will let you and the other maintainers decide what's best here.

Quite frankly, I'm fairly sure most of this is untested anyway. There
isn't a good way to test gr3d for example.

Generally I lean towards not merging any code that we don't know is
needed. Merely because Android kernels do something doesn't mean it is
necessary to do it (or even sane in some cases).

Now I understand that it might be difficult to test this, but that's all
the more reason not to include the code. If we find that we have bugs
that this can fix, then we can come up with tests to trigger it and
validate that the fix actually fixes something.

In other words, I don't think it makes sense to implement mechanisms to
recover from situations that we have no way of triggering in the first
place.

Thierry
Peter De Schrijver July 20, 2015, 9:59 a.m. UTC | #6
On Fri, Jul 17, 2015 at 01:31:24PM +0200, Thierry Reding wrote:
> * PGP Signed by an unknown key
> 
> On Fri, Jul 17, 2015 at 01:20:49PM +0300, Peter De Schrijver wrote:
> > On Fri, Jul 17, 2015 at 11:57:55AM +0200, Thierry Reding wrote:
> > > > Old Signed by an unknown key
> > > 
> > > On Mon, Jul 13, 2015 at 01:39:40PM +0100, Jon Hunter wrote:
> > > > The Tegra memory controller implements a flush feature to flush pending
> > > > accesses and prevent further accesses from occurring. This feature is
> > > > used when powering down IP blocks to ensure the IP block is in a good
> > > > state. The flushes are organised by software groups and IP blocks are
> > > > assigned in hardware to the different software groups. Add helper
> > > > functions for requesting a handle to an MC flush for a given
> > > > software group and enabling/disabling the MC flush itself.
> > > > 
> > > > This is based upon a change by Vince Hsu <vinceh@nvidia.com>.
> > > > 
> > > > Signed-off-by: Jon Hunter <jonathanh@nvidia.com>
> > > > ---
> > > >  drivers/memory/tegra/mc.c | 110 ++++++++++++++++++++++++++++++++++++++++++++++
> > > >  drivers/memory/tegra/mc.h |   2 +
> > > >  include/soc/tegra/mc.h    |  34 ++++++++++++++
> > > >  3 files changed, 146 insertions(+)
> > > 
> > > Do we know if this is actually necessary? I remember having a discussion
> > > with Arnd Bergmann a while ago, and the Linux driver model kind of
> > > assumes that by the time a device is disabled all outstanding accesses
> > > will have stopped.
> > > 
> > > Do we have a way to determine that this even makes a difference? Can we
> > > trigger a case where not doing this would cause breakage and see that
> > > adding this fixes that particular issue?
> > > 
> > 
> > Most likely it is. The memory controller can still be processing requests
> > when the peripheral domain is powergated. This would mean the response cannot
> > be delivered in that case. So we need to be sure there are no outstanding
> > requests before shutting down the domain.
> 
> My point is that that's the driver's responsibility anyway, hence making
> the explicit flush unnecessary.
> 

The peripheral driver doesn't know how long a request is queued in the memory
controller. So it can't be responsible for ensuring this.

Cheers,

Peter.
--
To unsubscribe from this list: send the line "unsubscribe linux-tegra" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Thierry Reding July 20, 2015, 1:14 p.m. UTC | #7
On Mon, Jul 20, 2015 at 12:59:41PM +0300, Peter De Schrijver wrote:
> On Fri, Jul 17, 2015 at 01:31:24PM +0200, Thierry Reding wrote:
> > * PGP Signed by an unknown key
> > 
> > On Fri, Jul 17, 2015 at 01:20:49PM +0300, Peter De Schrijver wrote:
> > > On Fri, Jul 17, 2015 at 11:57:55AM +0200, Thierry Reding wrote:
> > > > > Old Signed by an unknown key
> > > > 
> > > > On Mon, Jul 13, 2015 at 01:39:40PM +0100, Jon Hunter wrote:
> > > > > The Tegra memory controller implements a flush feature to flush pending
> > > > > accesses and prevent further accesses from occurring. This feature is
> > > > > used when powering down IP blocks to ensure the IP block is in a good
> > > > > state. The flushes are organised by software groups and IP blocks are
> > > > > assigned in hardware to the different software groups. Add helper
> > > > > functions for requesting a handle to an MC flush for a given
> > > > > software group and enabling/disabling the MC flush itself.
> > > > > 
> > > > > This is based upon a change by Vince Hsu <vinceh@nvidia.com>.
> > > > > 
> > > > > Signed-off-by: Jon Hunter <jonathanh@nvidia.com>
> > > > > ---
> > > > >  drivers/memory/tegra/mc.c | 110 ++++++++++++++++++++++++++++++++++++++++++++++
> > > > >  drivers/memory/tegra/mc.h |   2 +
> > > > >  include/soc/tegra/mc.h    |  34 ++++++++++++++
> > > > >  3 files changed, 146 insertions(+)
> > > > 
> > > > Do we know if this is actually necessary? I remember having a discussion
> > > > with Arnd Bergmann a while ago, and the Linux driver model kind of
> > > > assumes that by the time a device is disabled all outstanding accesses
> > > > will have stopped.
> > > > 
> > > > Do we have a way to determine that this even makes a difference? Can we
> > > > trigger a case where not doing this would cause breakage and see that
> > > > adding this fixes that particular issue?
> > > > 
> > > 
> > > Most likely it is. The memory controller can still be processing requests
> > > when the peripheral domain is powergated. This would mean the response cannot
> > > be delivered in that case. So we need to be sure there are no outstanding
> > > requests before shutting down the domain.
> > 
> > My point is that that's the driver's responsibility anyway, hence making
> > the explicit flush unnecessary.
> > 
> 
> The peripheral driver doesn't know how long a request is queued in the memory
> controller. So it can't be responsible for ensuring this.

Surely whenever the peripheral reports that the operation is done (be
that via some DMA controller interrupt or syncpoint increment) the
operation would have flushed from the memory controller. Drivers are
already supposed to make sure this is the case when they are removed or
suspended, so this would mean that we'd be adding all this code for no
real gain.

It would also explain why we're currently not seeing any such problems.

Also note that I'm not saying no to this, I merely want to make sure
that it's necessary, and in order to prove so we need tests to produce
enough traffic for this to be exposed, and then we can also verify that
the patches actually do what they're supposed to.

Thierry
Peter De Schrijver July 21, 2015, 10:57 a.m. UTC | #8
On Mon, Jul 20, 2015 at 03:14:25PM +0200, Thierry Reding wrote:
> * PGP Signed by an unknown key
> 
> On Mon, Jul 20, 2015 at 12:59:41PM +0300, Peter De Schrijver wrote:
> > On Fri, Jul 17, 2015 at 01:31:24PM +0200, Thierry Reding wrote:
> > > > Old Signed by an unknown key
> > > 
> > > On Fri, Jul 17, 2015 at 01:20:49PM +0300, Peter De Schrijver wrote:
> > > > On Fri, Jul 17, 2015 at 11:57:55AM +0200, Thierry Reding wrote:
> > > > > > Old Signed by an unknown key
> > > > > 
> > > > > On Mon, Jul 13, 2015 at 01:39:40PM +0100, Jon Hunter wrote:
> > > > > > The Tegra memory controller implements a flush feature to flush pending
> > > > > > accesses and prevent further accesses from occurring. This feature is
> > > > > > used when powering down IP blocks to ensure the IP block is in a good
> > > > > > state. The flushes are organised by software groups and IP blocks are
> > > > > > assigned in hardware to the different software groups. Add helper
> > > > > > functions for requesting a handle to an MC flush for a given
> > > > > > software group and enabling/disabling the MC flush itself.
> > > > > > 
> > > > > > This is based upon a change by Vince Hsu <vinceh@nvidia.com>.
> > > > > > 
> > > > > > Signed-off-by: Jon Hunter <jonathanh@nvidia.com>
> > > > > > ---
> > > > > >  drivers/memory/tegra/mc.c | 110 ++++++++++++++++++++++++++++++++++++++++++++++
> > > > > >  drivers/memory/tegra/mc.h |   2 +
> > > > > >  include/soc/tegra/mc.h    |  34 ++++++++++++++
> > > > > >  3 files changed, 146 insertions(+)
> > > > > 
> > > > > Do we know if this is actually necessary? I remember having a discussion
> > > > > with Arnd Bergmann a while ago, and the Linux driver model kind of
> > > > > assumes that by the time a device is disabled all outstanding accesses
> > > > > will have stopped.
> > > > > 
> > > > > Do we have a way to determine that this even makes a difference? Can we
> > > > > trigger a case where not doing this would cause breakage and see that
> > > > > adding this fixes that particular issue?
> > > > > 
> > > > 
> > > > Most likely it is. The memory controller can still be processing requests
> > > > when the peripheral domain is powergated. This would mean the response cannot
> > > > be delivered in that case. So we need to be sure there are no outstanding
> > > > requests before shutting down the domain.
> > > 
> > > My point is that that's the driver's responsibility anyway, hence making
> > > the explicit flush unnecessary.
> > > 
> > 
> > The peripheral driver doesn't know how long a request is queued in the memory
> > controller. So it can't be responsible for ensuring this.
> 
> Surely whenever the peripheral reports that the operation is done (be
> that via some DMA controller interrupt or syncpoint increment) the
> operation would have flushed from the memory controller. Drivers are
> already supposed to make sure this is the case when they are removed or
> suspended, so this would mean that we'd be adding all this code for no
> real gain.
> 

That highly depends on how the peripheral is implemented. I'm not sure we
can assume things work this way.

> It would also explain why we're currently not seeing any such problems.
> 

I don't think we are doing agressive enough powergating today to see any
problems. That might also mean we're just lucky.

Cheers,

Peter.
--
To unsubscribe from this list: send the line "unsubscribe linux-tegra" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/drivers/memory/tegra/mc.c b/drivers/memory/tegra/mc.c
index c71ede67e6c8..fb8da3d4caf4 100644
--- a/drivers/memory/tegra/mc.c
+++ b/drivers/memory/tegra/mc.c
@@ -7,6 +7,7 @@ 
  */
 
 #include <linux/clk.h>
+#include <linux/delay.h>
 #include <linux/interrupt.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
@@ -71,6 +72,107 @@  static const struct of_device_id tegra_mc_of_match[] = {
 };
 MODULE_DEVICE_TABLE(of, tegra_mc_of_match);
 
+const struct tegra_mc_flush *tegra_mc_flush_get(struct tegra_mc *mc,
+						unsigned int swgroup)
+{
+	const struct tegra_mc_flush *flush = NULL;
+	int i;
+
+	mutex_lock(&mc->lock);
+
+	for (i = 0; i < mc->soc->num_flushes; i++) {
+		if (mc->soc->flushes[i].swgroup == swgroup) {
+			if (mc->flush_reserved[i] == false) {
+				mc->flush_reserved[i] = true;
+				flush = &mc->soc->flushes[i];
+			}
+			break;
+		}
+	}
+
+	mutex_unlock(&mc->lock);
+
+	return flush;
+}
+EXPORT_SYMBOL(tegra_mc_flush_get);
+
+static bool tegra_mc_flush_done(struct tegra_mc *mc,
+				const struct tegra_mc_flush *flush)
+{
+	unsigned long timeout = jiffies + msecs_to_jiffies(100);
+	int i;
+	u32 val;
+
+	while (time_before(jiffies, timeout)) {
+		val = mc_readl(mc, flush->status);
+
+		/*
+		 * If the flush bit is still set it
+		 * is not done and so wait then retry.
+		 */
+		if (val & BIT(flush->bit))
+			goto retry;
+
+		/*
+		 * Depending on the tegra SoC, it may be necessary to read
+		 * the status register multiple times to ensure the value
+		 * read is correct. Some tegra devices have a HW issue where
+		 * reading the status register shortly after writing the
+		 * control register (on the order of 5 cycles) may return
+		 * an incorrect value.
+		 */
+		for (i = 0; i < mc->soc->metastable_flush_reads; i++) {
+			if (mc_readl(mc, flush->status) != val)
+				goto retry;
+		}
+
+		/*
+		 * The flush is complete and so return.
+		 */
+		return 0;
+retry:
+		udelay(10);
+	}
+
+	return -ETIMEDOUT;
+}
+
+int tegra_mc_flush(struct tegra_mc *mc, const struct tegra_mc_flush *flush,
+		   bool enable)
+{
+	int ret = 0;
+	u32 val;
+
+	if (!mc || !flush)
+		return -EINVAL;
+
+	mutex_lock(&mc->lock);
+
+	val = mc_readl(mc, flush->ctrl);
+
+	if (enable)
+		val |= BIT(flush->bit);
+	else
+		val &= ~BIT(flush->bit);
+
+	mc_writel(mc, val, flush->ctrl);
+	mc_readl(mc, flush->ctrl);
+
+	/*
+	 * If activating the flush, poll the
+	 * status register until the flush is done.
+	 */
+	if (enable)
+		ret = tegra_mc_flush_done(mc, flush);
+
+	mutex_unlock(&mc->lock);
+
+	dev_dbg(mc->dev, "%s bit %d\n", __func__, flush->bit);
+
+	return ret;
+}
+EXPORT_SYMBOL(tegra_mc_flush);
+
 static int tegra_mc_setup_latency_allowance(struct tegra_mc *mc)
 {
 	unsigned long long tick;
@@ -359,6 +461,12 @@  static int tegra_mc_probe(struct platform_device *pdev)
 	mc->soc = match->data;
 	mc->dev = &pdev->dev;
 
+	mc->flush_reserved = devm_kcalloc(&pdev->dev, mc->soc->num_flushes,
+					  sizeof(mc->flush_reserved),
+					  GFP_KERNEL);
+	if (!mc->flush_reserved)
+		return -ENOMEM;
+
 	/* length of MC tick in nanoseconds */
 	mc->tick = 30;
 
@@ -410,6 +518,8 @@  static int tegra_mc_probe(struct platform_device *pdev)
 		return err;
 	}
 
+	mutex_init(&mc->lock);
+
 	value = MC_INT_DECERR_MTS | MC_INT_SECERR_SEC | MC_INT_DECERR_VPR |
 		MC_INT_INVALID_APB_ASID_UPDATE | MC_INT_INVALID_SMMU_PAGE |
 		MC_INT_SECURITY_VIOLATION | MC_INT_DECERR_EMEM;
diff --git a/drivers/memory/tegra/mc.h b/drivers/memory/tegra/mc.h
index b7361b0a6696..0f59d49b735b 100644
--- a/drivers/memory/tegra/mc.h
+++ b/drivers/memory/tegra/mc.h
@@ -14,6 +14,8 @@ 
 
 #include <soc/tegra/mc.h>
 
+#define MC_FLUSH_METASTABLE_READS	5
+
 static inline u32 mc_readl(struct tegra_mc *mc, unsigned long offset)
 {
 	return readl(mc->regs + offset);
diff --git a/include/soc/tegra/mc.h b/include/soc/tegra/mc.h
index 1ab2813273cd..b634c6df79eb 100644
--- a/include/soc/tegra/mc.h
+++ b/include/soc/tegra/mc.h
@@ -45,6 +45,13 @@  struct tegra_mc_client {
 	struct tegra_mc_la la;
 };
 
+struct tegra_mc_flush {
+	unsigned int swgroup;
+	unsigned int ctrl;
+	unsigned int status;
+	unsigned int bit;
+};
+
 struct tegra_smmu_swgroup {
 	const char *name;
 	unsigned int swgroup;
@@ -96,6 +103,10 @@  struct tegra_mc_soc {
 	const struct tegra_mc_client *clients;
 	unsigned int num_clients;
 
+	const struct tegra_mc_flush *flushes;
+	unsigned int num_flushes;
+	unsigned int metastable_flush_reads;
+
 	const unsigned long *emem_regs;
 	unsigned int num_emem_regs;
 
@@ -117,9 +128,32 @@  struct tegra_mc {
 
 	struct tegra_mc_timing *timings;
 	unsigned int num_timings;
+
+	bool *flush_reserved;
+
+	struct mutex lock;
 };
 
 void tegra_mc_write_emem_configuration(struct tegra_mc *mc, unsigned long rate);
 unsigned int tegra_mc_get_emem_device_count(struct tegra_mc *mc);
 
+#ifdef CONFIG_TEGRA_MC
+const struct tegra_mc_flush *tegra_mc_flush_get(struct tegra_mc *mc,
+						unsigned int swgroup);
+int tegra_mc_flush(struct tegra_mc *mc, const struct tegra_mc_flush *s,
+		   bool enable);
+#else
+const struct tegra_mc_flush *tegra_mc_flush_get(struct tegra_mc *mc,
+						unsigned int swgroup)
+{
+	return NULL;
+}
+
+int tegra_mc_flush(struct tegra_mc *mc, const struct tegra_mc_flush *s,
+		   bool enable)
+{
+	return -ENOTSUPP;
+}
+#endif
+
 #endif /* __SOC_TEGRA_MC_H__ */