Patchwork on-die ECC support

login
register
mail settings
Submitter David Mosberger-Tang
Date March 20, 2013, 5:46 p.m.
Message ID <CACwUX0OuirXy27nOAjZ3zxw2qrBVbXC+fwKXASs1_R=CSkX_FQ@mail.gmail.com>
Download mbox | patch
Permalink /patch/229444/
State New
Headers show

Comments

David Mosberger-Tang - March 20, 2013, 5:46 p.m.
Here is an updated patch that seems to work well so far.  Some comments:

- nand_oob_64_on_die is probably correct for Micron MT29F4G part only.  Perhaps
 there is a way to auto-detect this based on ONFI parameters?
- the handling of the raw ops is a bit ugly, since we have to
temporarily turn off
 the on-die ECC (and then turn it back on); this can't be done in the
ecc callbacks
 because those usually get called after the main command (e.g., READ) has been
 issued already

The patch is relative to kernel 3.7.0.  I'm providing it mostly in
case someone else needs the
functionality.  Feedback is always welcome, of course.  Hopefully at
least some of this can eventually
make it into main-line.

At the very least I'd like to see a warning added to the kernel which
kicks in when someone is
trying to run an ECC algorithm that is too weak (e.g., NAND_ECC_SOFT
when 4-bit ECC is
required).  Looks like that could be done easily based on ONFI parameters.

  --david

On Fri, Mar 15, 2013 at 10:08 AM, David Mosberger-Tang
<dmosberger@gmail.com> wrote:
> We need to be able to support 4-bit-correcting ECC with a
> micro-controller that doesn't have hardware-support for that.  We are
> planning to use the on-die ECC controller supported by Micron flash
> chips.  I suppose we could use the BCH swecc support in the Linux
> kernel, but I'm concerned about the performance implication of that
> and we'd also have to add BCH ecc to our bootloader, which would mean
> more development and testing.
>
> For backwards-compatibility, we need to be able to support a
> subpage-size of 512 bytes.  From the Micron data sheets, it's not
> clear to me whether the on-die controller really supports reading 512
> byte subpages but the attached patch certainly *seems* to work fine so
> far.
>
> I'd love to get some feedback as to whether I'm on the right track
> here or whether I should pursue a different path.  The patch does the
> following:
>
>  - Add NAND_ECC_HW_ON_DIE ecc mode
>  - Add nand_read_subpage_raw() to read a subpage without worrying about ECC
>  - Hack atmel_nand.c to use on-die ECC.
>
> Thanks,
>
>   --david
commit afee2a1b319943d449f09e087369537469870b67
Author: David Mosberger <davidm@egauge.net>
Date:   Wed Mar 20 11:19:59 2013 -0600

    [MTD] Add support for NAND flash with on-die ECC controllers.
    
    * drivers/mtd/nand/nand_base.c (nand_oob_64_on_die): New struct for
    		ECC layout of Micron MT29F4G16ABADAWP.
    	(set_column): Properly handle READID, GET_FEATURES, and SET_FEATURES
    		commands.  The column is not a word address for those commands
    		and always single byte.
    	(nand_command): Use set_column() to set column in lieu of open code.
    	(nand_command_lp): Likewise.
    	(set_on_die_ecc): New function.
    	(check_read_status_on_die): Likewise.
    	(nand_read_subpage_on_die): Likewise.
    	(nand_read_page_on_die): Likewise.
    	(nand_do_read_ops): Temporarily disable on-die ECC for raw operations.
    	(nand_do_read_oob): Likewise.
    	(nand_write_page): Likewise.
    	(nand_do_write_oob): Likewise.
    	(nand_onfi_set_features): Make it work on 16-bit devices: parameters
    		need to be read in byte-mode even on 16-bit devices.
    	(nand_onfi_get_features): Likewise.
    	(nand_flash_detect_onfi): Likewise.
    	(nand_get_flash_type): Always try to detect ONFI, even if chip
    		is already known.  Otherwise, chip->onfi_get_features and
    		chip->onfi_set_features won't work on chips that do support
    		ONFI.
    	(nand_scan_tail): Check if chip has on-die ECC enabled (either by
    		hardware or by bootloader) and, if so, force on-die ECC mode.
    		Handle NAND_ECC_HW_ON_DIE.
    		Enable sub-page reads for on-die ECC as well.
    * drivers/of/of_mtd.c (nand_ecc_modes): Add NAND_ECC_HW_ON_DIE/hw_on_die.
    * include/linux/mtd/nand.h (NAND_STATUS_REWRITE): New macro.
    	(nand_ecc_modes_t): Add NAND_ECC_HW_ON_DIE.

Patch

diff --git a/drivers/mtd/nand/nand_base.c b/drivers/mtd/nand/nand_base.c
index 1a03b7f..fa85dd5 100644
--- a/drivers/mtd/nand/nand_base.c
+++ b/drivers/mtd/nand/nand_base.c
@@ -79,6 +79,20 @@  static struct nand_ecclayout nand_oob_64 = {
 		 .length = 38} }
 };
 
+static struct nand_ecclayout nand_oob_64_on_die = {
+	.eccbytes = 32,
+	.eccpos = {
+		    8,  9, 10, 11, 12, 13, 14, 15,
+		   24, 25, 26, 27, 28, 29, 30, 31,
+		   40, 41, 42, 43, 44, 45, 46, 47,
+		   56, 57, 58, 59, 60, 61, 62, 63},
+	.oobfree = {
+		{.offset =  4,	 .length = 4},
+		{.offset = 20,	 .length = 4},
+		{.offset = 36,	 .length = 4},
+		{.offset = 52,	 .length = 4}}
+};
+
 static struct nand_ecclayout nand_oob_128 = {
 	.eccbytes = 48,
 	.eccpos = {
@@ -509,6 +523,32 @@  void nand_wait_ready(struct mtd_info *mtd)
 }
 EXPORT_SYMBOL_GPL(nand_wait_ready);
 
+static int
+set_column (struct mtd_info *mtd, struct nand_chip *chip,
+	    unsigned int command, int column,
+	    unsigned int column_width, int ctrl)
+{
+	switch (command) {
+	case NAND_CMD_READID:
+	case NAND_CMD_GET_FEATURES:
+	case NAND_CMD_SET_FEATURES:
+		chip->cmd_ctrl(mtd, column, ctrl);
+		ctrl &= ~NAND_CTRL_CHANGE;
+		break;
+
+	default:
+		/* Adjust columns for 16 bit buswidth */
+		if (chip->options & NAND_BUSWIDTH_16)
+			column >>= 1;
+		chip->cmd_ctrl(mtd, column, ctrl);
+		ctrl &= ~NAND_CTRL_CHANGE;
+		if (column_width > 8)
+			chip->cmd_ctrl(mtd, column >> 8, ctrl);
+		break;
+	}
+	return ctrl;
+}
+
 /**
  * nand_command - [DEFAULT] Send command to NAND device
  * @mtd: MTD device structure
@@ -548,13 +588,8 @@  static void nand_command(struct mtd_info *mtd, unsigned int command,
 	/* Address cycle, when necessary */
 	ctrl = NAND_CTRL_ALE | NAND_CTRL_CHANGE;
 	/* Serially input address */
-	if (column != -1) {
-		/* Adjust columns for 16 bit buswidth */
-		if (chip->options & NAND_BUSWIDTH_16)
-			column >>= 1;
-		chip->cmd_ctrl(mtd, column, ctrl);
-		ctrl &= ~NAND_CTRL_CHANGE;
-	}
+	if (column != -1)
+		ctrl = set_column(mtd, chip, command, column, 8, ctrl);
 	if (page_addr != -1) {
 		chip->cmd_ctrl(mtd, page_addr, ctrl);
 		ctrl &= ~NAND_CTRL_CHANGE;
@@ -640,14 +675,9 @@  static void nand_command_lp(struct mtd_info *mtd, unsigned int command,
 		int ctrl = NAND_CTRL_CHANGE | NAND_NCE | NAND_ALE;
 
 		/* Serially input address */
-		if (column != -1) {
-			/* Adjust columns for 16 bit buswidth */
-			if (chip->options & NAND_BUSWIDTH_16)
-				column >>= 1;
-			chip->cmd_ctrl(mtd, column, ctrl);
-			ctrl &= ~NAND_CTRL_CHANGE;
-			chip->cmd_ctrl(mtd, column >> 8, ctrl);
-		}
+		if (column != -1)
+			ctrl = set_column (mtd, chip, command, column, 16,
+					   ctrl);
 		if (page_addr != -1) {
 			chip->cmd_ctrl(mtd, page_addr, ctrl);
 			chip->cmd_ctrl(mtd, page_addr >> 8,
@@ -1221,6 +1251,124 @@  static int nand_read_subpage(struct mtd_info *mtd, struct nand_chip *chip,
 	return max_bitflips;
 }
 
+static int
+set_on_die_ecc (struct mtd_info *mtd, struct nand_chip *chip, int on)
+{
+	u8 data[ONFI_SUBFEATURE_PARAM_LEN] = { 0, };
+
+	if (chip->ecc.mode != NAND_ECC_HW_ON_DIE)
+		return 0;
+
+	if (on)
+		data[0] = 0x8;
+
+	return (*chip->onfi_set_features)(mtd, chip, 0x90, data);
+}
+
+static int check_read_status_on_die (struct mtd_info *mtd, struct nand_chip *chip)
+{
+	unsigned int max_bitflips = 0;
+	uint8_t status;
+
+	/* Check ECC status of page just transferred into NAND's page buffer: */
+	chip->cmdfunc(mtd, NAND_CMD_STATUS, -1, -1);
+	status = chip->read_byte(mtd);
+
+	/* Switch back to data reading: */
+	chip->cmd_ctrl(mtd, NAND_CMD_READ0,
+		       NAND_NCE | NAND_CLE | NAND_CTRL_CHANGE);
+	chip->cmd_ctrl(mtd, NAND_CMD_NONE,
+		       NAND_NCE | NAND_CTRL_CHANGE);
+
+	if (status & NAND_STATUS_FAIL)
+		/* Page has invalid ECC. */
+		mtd->ecc_stats.failed++;
+	else if (status & NAND_STATUS_REWRITE) {
+		/*
+		 * Bit flip(s); we don't know how many, but since a rewrite is recommended
+		 * we should assume ecc.strength bitflips so that the higher-level software
+		 * is going to do something about it.
+		 */
+		mtd->ecc_stats.corrected += chip->ecc.strength;
+		max_bitflips = chip->ecc.strength;
+	}
+	return max_bitflips;
+}
+
+/**
+ * nand_read_subpage_on_die - [REPLACEABLE] raw sub-page read function
+ * @mtd: mtd info structure
+ * @chip: nand chip info structure
+ * @data_offs: offset of requested data within the page
+ * @readlen: data length
+ * @bufpoi: buffer to store read data
+ */
+static int nand_read_subpage_on_die(struct mtd_info *mtd, struct nand_chip *chip,
+			uint32_t data_offs, uint32_t readlen, uint8_t *bufpoi)
+{
+	int start_step, end_step, num_steps, ret;
+	int data_col_addr;
+	int datafrag_len;
+	uint32_t failed;
+	uint8_t *p;
+
+	/* Column address within the page aligned to ECC size */
+	start_step = data_offs / chip->ecc.size;
+	end_step = (data_offs + readlen - 1) / chip->ecc.size;
+	num_steps = end_step - start_step + 1;
+
+	/* Data size aligned to ECC ecc.size */
+	datafrag_len = num_steps * chip->ecc.size;
+	data_col_addr = start_step * chip->ecc.size;
+	p = bufpoi + data_col_addr;
+
+	failed = mtd->ecc_stats.failed;
+
+	ret = check_read_status_on_die (mtd, chip);
+	if (ret < 0 || mtd->ecc_stats.failed != failed) {
+		memset (p, 0, datafrag_len);
+		return ret;
+	}
+
+	/* If we read not a page aligned data */
+	if (data_col_addr != 0)
+		chip->cmdfunc(mtd, NAND_CMD_RNDOUT, data_col_addr, -1);
+
+	chip->read_buf(mtd, p, datafrag_len);
+
+	return ret;
+}
+
+/**
+ * nand_read_page_on_die - [INTERN] read raw page data without ecc
+ * @mtd: mtd info structure
+ * @chip: nand chip info structure
+ * @buf: buffer to store read data
+ * @oob_required: caller requires OOB data read to chip->oob_poi
+ * @page: page number to read
+ */
+static int nand_read_page_on_die(struct mtd_info *mtd, struct nand_chip *chip,
+				 uint8_t *buf, int oob_required, int page)
+{
+	uint32_t failed;
+	int ret;
+
+	failed = mtd->ecc_stats.failed;
+
+	ret = check_read_status_on_die (mtd, chip);
+	if (ret < 0 || mtd->ecc_stats.failed != failed) {
+		memset (buf, 0, mtd->writesize);
+		if (oob_required)
+			memset (chip->oob_poi, 0, mtd->oobsize);
+		return ret;
+	}
+
+	chip->read_buf(mtd, buf, mtd->writesize);
+	if (oob_required)
+		chip->read_buf(mtd, chip->oob_poi, mtd->oobsize);
+	return ret;
+}
+
 /**
  * nand_read_page_hwecc - [REPLACEABLE] hardware ECC based page read function
  * @mtd: mtd info structure
@@ -1474,17 +1622,21 @@  static int nand_do_read_ops(struct mtd_info *mtd, loff_t from,
 		if (realpage != chip->pagebuf || oob) {
 			bufpoi = aligned ? buf : chip->buffers->databuf;
 
+			if (unlikely(ops->mode == MTD_OPS_RAW))
+			    set_on_die_ecc(mtd, chip, 0);
+
 			chip->cmdfunc(mtd, NAND_CMD_READ0, 0x00, page);
 
 			/*
 			 * Now read the page into the buffer.  Absent an error,
 			 * the read methods return max bitflips per ecc step.
 			 */
-			if (unlikely(ops->mode == MTD_OPS_RAW))
+			if (unlikely(ops->mode == MTD_OPS_RAW)) {
 				ret = chip->ecc.read_page_raw(mtd, chip, bufpoi,
 							      oob_required,
 							      page);
-			else if (!aligned && NAND_HAS_SUBPAGE_READ(chip) &&
+				set_on_die_ecc(mtd, chip, 1);
+			} else if (!aligned && NAND_HAS_SUBPAGE_READ(chip) &&
 				 !oob)
 				ret = chip->ecc.read_subpage(mtd, chip,
 							col, bytes, bufpoi);
@@ -1778,9 +1930,11 @@  static int nand_do_read_oob(struct mtd_info *mtd, loff_t from,
 	page = realpage & chip->pagemask;
 
 	while (1) {
-		if (ops->mode == MTD_OPS_RAW)
+		if (ops->mode == MTD_OPS_RAW) {
+			set_on_die_ecc(mtd, chip, 0);
 			ret = chip->ecc.read_oob_raw(mtd, chip, page);
-		else
+			set_on_die_ecc(mtd, chip, 1);
+		} else
 			ret = chip->ecc.read_oob(mtd, chip, page);
 
 		if (ret < 0)
@@ -2045,6 +2199,9 @@  static int nand_write_page(struct mtd_info *mtd, struct nand_chip *chip,
 {
 	int status;
 
+	if (unlikely(raw))
+		set_on_die_ecc(mtd, chip, 0);
+
 	chip->cmdfunc(mtd, NAND_CMD_SEQIN, 0x00, page);
 
 	if (unlikely(raw))
@@ -2052,8 +2209,11 @@  static int nand_write_page(struct mtd_info *mtd, struct nand_chip *chip,
 	else
 		status = chip->ecc.write_page(mtd, chip, buf, oob_required);
 
-	if (status < 0)
+	if (status < 0) {
+		if (unlikely(raw))
+			set_on_die_ecc(mtd, chip, 1);
 		return status;
+	}
 
 	/*
 	 * Cached progamming disabled for now. Not sure if it's worth the
@@ -2073,11 +2233,16 @@  static int nand_write_page(struct mtd_info *mtd, struct nand_chip *chip,
 			status = chip->errstat(mtd, chip, FL_WRITING, status,
 					       page);
 
+		if (unlikely(raw))
+			set_on_die_ecc(mtd, chip, 1);
+
 		if (status & NAND_STATUS_FAIL)
 			return -EIO;
 	} else {
 		chip->cmdfunc(mtd, NAND_CMD_CACHEDPROG, -1, -1);
 		status = chip->waitfunc(mtd, chip);
+		if (unlikely(raw))
+			set_on_die_ecc(mtd, chip, 1);
 	}
 
 	return 0;
@@ -2386,9 +2551,11 @@  static int nand_do_write_oob(struct mtd_info *mtd, loff_t to,
 
 	nand_fill_oob(mtd, ops->oobbuf, ops->ooblen, ops);
 
-	if (ops->mode == MTD_OPS_RAW)
+	if (ops->mode == MTD_OPS_RAW) {
+		set_on_die_ecc(mtd, chip, 0);
 		status = chip->ecc.write_oob_raw(mtd, chip, page & chip->pagemask);
-	else
+		set_on_die_ecc(mtd, chip, 1);
+	} else
 		status = chip->ecc.write_oob(mtd, chip, page & chip->pagemask);
 
 	if (status)
@@ -2709,13 +2876,27 @@  static int nand_block_markbad(struct mtd_info *mtd, loff_t ofs)
 static int nand_onfi_set_features(struct mtd_info *mtd, struct nand_chip *chip,
 			int addr, uint8_t *subfeature_param)
 {
-	int status;
+	uint16_t buf[ONFI_SUBFEATURE_PARAM_LEN];
+	size_t len = ONFI_SUBFEATURE_PARAM_LEN;
+	int status, i;
 
 	if (!chip->onfi_version)
 		return -EINVAL;
 
 	chip->cmdfunc(mtd, NAND_CMD_SET_FEATURES, addr, -1);
-	chip->write_buf(mtd, subfeature_param, ONFI_SUBFEATURE_PARAM_LEN);
+	if (chip->options & NAND_BUSWIDTH_16) {
+		/*
+		 * ONFI says parameters are always transferred on the
+		 * lower 8-bits of the databus.  Since there is no
+		 * chip->write_byte callback, we have to convert
+		 * subfeature_param to 16-bit data.
+		 */
+		for (i = 0; i < ONFI_SUBFEATURE_PARAM_LEN; ++i)
+			buf[i] = subfeature_param[i];
+		subfeature_param = (uint8_t *) buf;
+		len = sizeof (buf);
+	}
+	chip->write_buf(mtd, subfeature_param, len);
 	status = chip->waitfunc(mtd, chip);
 	if (status & NAND_STATUS_FAIL)
 		return -EIO;
@@ -2732,6 +2913,8 @@  static int nand_onfi_set_features(struct mtd_info *mtd, struct nand_chip *chip,
 static int nand_onfi_get_features(struct mtd_info *mtd, struct nand_chip *chip,
 			int addr, uint8_t *subfeature_param)
 {
+	int i;
+
 	if (!chip->onfi_version)
 		return -EINVAL;
 
@@ -2739,7 +2922,13 @@  static int nand_onfi_get_features(struct mtd_info *mtd, struct nand_chip *chip,
 	memset(subfeature_param, 0, ONFI_SUBFEATURE_PARAM_LEN);
 
 	chip->cmdfunc(mtd, NAND_CMD_GET_FEATURES, addr, -1);
-	chip->read_buf(mtd, subfeature_param, ONFI_SUBFEATURE_PARAM_LEN);
+	/*
+	 * ONFI says parameters are always transferred on the
+	 * lower 8-bits of the databus.  Use read_byte() since
+	 * that works even on 16-bit devices.
+	 */
+	for (i = 0; i < ONFI_SUBFEATURE_PARAM_LEN; ++i)
+	  subfeature_param[i] = chip->read_byte(mtd);
 	return 0;
 }
 
@@ -2846,10 +3035,10 @@  static int nand_flash_detect_onfi(struct mtd_info *mtd, struct nand_chip *chip,
 					int *busw)
 {
 	struct nand_onfi_params *p = &chip->onfi_params;
-	int i;
+	uint8_t *bp;
+	int i, j;
 	int val;
 
-	/* Try ONFI for unknown chip or LP */
 	chip->cmdfunc(mtd, NAND_CMD_READID, 0x20, -1);
 	if (chip->read_byte(mtd) != 'O' || chip->read_byte(mtd) != 'N' ||
 		chip->read_byte(mtd) != 'F' || chip->read_byte(mtd) != 'I')
@@ -2857,7 +3046,10 @@  static int nand_flash_detect_onfi(struct mtd_info *mtd, struct nand_chip *chip,
 
 	chip->cmdfunc(mtd, NAND_CMD_PARAM, 0, -1);
 	for (i = 0; i < 3; i++) {
-		chip->read_buf(mtd, (uint8_t *)p, sizeof(*p));
+		/* Must read with 8-bit transfers even on 16-bit devices: */
+		bp = (uint8_t *)p;
+		for (j = 0; j < sizeof (*p); ++j)
+			*bp++ = chip->read_byte(mtd);
 		if (onfi_crc16(ONFI_CRC_BASE, (uint8_t *)p, 254) ==
 				le16_to_cpu(p->crc)) {
 			pr_info("ONFI param page %d valid\n", i);
@@ -3202,11 +3394,9 @@  static struct nand_flash_dev *nand_get_flash_type(struct mtd_info *mtd,
 			break;
 
 	chip->onfi_version = 0;
-	if (!type->name || !type->pagesize) {
-		/* Check is chip is ONFI compliant */
-		if (nand_flash_detect_onfi(mtd, chip, &busw))
-			goto ident_done;
-	}
+	/* Check is chip is ONFI compliant */
+	if (nand_flash_detect_onfi(mtd, chip, &busw))
+		goto ident_done;
 
 	if (!type->name)
 		return ERR_PTR(-ENODEV);
@@ -3362,6 +3552,7 @@  EXPORT_SYMBOL(nand_scan_ident);
 int nand_scan_tail(struct mtd_info *mtd)
 {
 	int i;
+	u8 features[ONFI_SUBFEATURE_PARAM_LEN];
 	struct nand_chip *chip = mtd->priv;
 
 	/* New bad blocks should be marked in OOB, flash-based BBT, or both */
@@ -3409,6 +3600,16 @@  int nand_scan_tail(struct mtd_info *mtd)
 	if (!chip->onfi_get_features)
 		chip->onfi_get_features = nand_onfi_get_features;
 
+	if ((*chip->onfi_get_features)(mtd, chip, 0x90, features) >= 0) {
+		if (features[0] & 0x08) {
+			/*
+			 * If the chip has on-die ECC enabled, we kind of have to use it.
+			 */
+			chip->ecc.mode = NAND_ECC_HW_ON_DIE;
+			pr_info("NAND device: Using on-die ECC\n");
+		}
+	}
+
 	/*
 	 * Check ECC mode, default to software if 3byte/512byte hardware ECC is
 	 * selected and we have 256 byte pagesize fallback to software ECC
@@ -3530,6 +3731,25 @@  int nand_scan_tail(struct mtd_info *mtd)
 			chip->ecc.bytes * 8 / fls(8 * chip->ecc.size);
 		break;
 
+	case NAND_ECC_HW_ON_DIE:
+		/* nand_bbt attempts to put Bbt marker at offset 8 in
+		   oob, which is used for ECC by Micron
+		   MT29F4G16ABADAWP, for example.  Fixed by not using
+		   OOB for BBT marker.  */
+		chip->bbt_options |= NAND_BBT_NO_OOB;
+		chip->ecc.layout = &nand_oob_64_on_die;
+		chip->ecc.read_page = nand_read_page_on_die;
+		chip->ecc.read_subpage = nand_read_subpage_on_die;
+		chip->ecc.write_page = nand_write_page_raw;
+		chip->ecc.read_oob = nand_read_oob_std;
+		chip->ecc.read_page_raw = nand_read_page_raw;
+		chip->ecc.write_page_raw = nand_write_page_raw;
+		chip->ecc.write_oob = nand_write_oob_std;
+		chip->ecc.size = 512;
+		chip->ecc.bytes = 8;
+		chip->ecc.strength = 4;
+		break;
+
 	case NAND_ECC_NONE:
 		pr_warn("NAND_ECC_NONE selected by board driver. "
 			   "This is not recommended!\n");
@@ -3602,8 +3822,10 @@  int nand_scan_tail(struct mtd_info *mtd)
 	/* Invalidate the pagebuffer reference */
 	chip->pagebuf = -1;
 
-	/* Large page NAND with SOFT_ECC should support subpage reads */
-	if ((chip->ecc.mode == NAND_ECC_SOFT) && (chip->page_shift > 9))
+	/* Large page NAND with SOFT_ECC or on-die ECC should support subpage reads */
+	if (((chip->ecc.mode == NAND_ECC_SOFT)
+	     || (chip->ecc.mode == NAND_ECC_HW_ON_DIE))
+	    && (chip->page_shift > 9))
 		chip->options |= NAND_SUBPAGE_READ;
 
 	/* Fill in remaining MTD driver data */
diff --git a/drivers/of/of_mtd.c b/drivers/of/of_mtd.c
index a27ec94..79ea026 100644
--- a/drivers/of/of_mtd.c
+++ b/drivers/of/of_mtd.c
@@ -23,6 +23,7 @@  static const char *nand_ecc_modes[] = {
 	[NAND_ECC_HW_SYNDROME]	= "hw_syndrome",
 	[NAND_ECC_HW_OOB_FIRST]	= "hw_oob_first",
 	[NAND_ECC_SOFT_BCH]	= "soft_bch",
+	[NAND_ECC_HW_ON_DIE]	= "hw_on_die",
 };
 
 /**
diff --git a/include/linux/mtd/nand.h b/include/linux/mtd/nand.h
index 24e9159..53e99c7 100644
--- a/include/linux/mtd/nand.h
+++ b/include/linux/mtd/nand.h
@@ -129,6 +129,7 @@  extern int nand_unlock(struct mtd_info *mtd, loff_t ofs, uint64_t len);
 /* Status bits */
 #define NAND_STATUS_FAIL	0x01
 #define NAND_STATUS_FAIL_N1	0x02
+#define NAND_STATUS_REWRITE	0x08
 #define NAND_STATUS_TRUE_READY	0x20
 #define NAND_STATUS_READY	0x40
 #define NAND_STATUS_WP		0x80
@@ -143,6 +144,7 @@  typedef enum {
 	NAND_ECC_HW_SYNDROME,
 	NAND_ECC_HW_OOB_FIRST,
 	NAND_ECC_SOFT_BCH,
+	NAND_ECC_HW_ON_DIE,
 } nand_ecc_modes_t;
 
 /*