[1/4] Support for checking and reading block grade information in kernel

Message ID CAExFE6m_jeBzmTUaYnEA7cZ-jbfogYXWPmjRPPJfR_5MvEWU5w@mail.gmail.com
State New
Headers show
Series
  • RFC : Support for data gradation of a single file.
Related show

Commit Message

Sayan Ghosh April 6, 2018, 11:41 a.m.
This introduces the different functions in order to get the grades as
the extended attributes while pre-allocating a new file. The grades
are stored as extended attributes while the file gets created. The
grades can be used by different user space applications as necessary.
The functions introduced are read_grade_xattr(), is_file_graded(),
read_count_xattr() which aim to read the extended attribute for grade
array and also to know whether the file is graded. The detailed
descriptions of the functions are provided as comments in the patch.
The patch is on top of Linux Kernel 4.7.2.

Signed-off-by: Sayan Ghosh <sgdgp.2014@gmail.com>
---
 fs/ext4/ext4.h    | 15 +++++++++++++++
 fs/ext4/extents.c | 35 +++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+)

+    return total;
+}
+
 static __le32 ext4_extent_block_csum(struct inode *inode,
                      struct ext4_extent_header *eh)
 {
‌

Comments

Randy Dunlap April 6, 2018, 5:09 p.m. | #1
On 04/06/2018 04:41 AM, Sayan Ghosh wrote:
> This introduces the different functions in order to get the grades as
> the extended attributes while pre-allocating a new file. The grades
> are stored as extended attributes while the file gets created. The
> grades can be used by different user space applications as necessary.
> The functions introduced are read_grade_xattr(), is_file_graded(),
> read_count_xattr() which aim to read the extended attribute for grade
> array and also to know whether the file is graded. The detailed
> descriptions of the functions are provided as comments in the patch.
> The patch is on top of Linux Kernel 4.7.2.

Well, it's up to Ted if he wants to merge a patch that is based on 4.7.2.
In general, patches are made to the current mainline or -next tree,
while 4.7.2 was released on 2016-Aug-20.

> 
> Signed-off-by: Sayan Ghosh <sgdgp.2014@gmail.com>
> ---
>  fs/ext4/ext4.h    | 15 +++++++++++++++
>  fs/ext4/extents.c | 35 +++++++++++++++++++++++++++++++++++
>  2 files changed, 50 insertions(+)
> 
> diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
> index b84aa1c..b9ec0ca 100755
> --- a/fs/ext4/ext4.h
> +++ b/fs/ext4/ext4.h
> @@ -136,6 +136,18 @@ enum SHIFT_DIRECTION {
>  /* Use blocks from reserved pool */
>  #define EXT4_MB_USE_RESERVED        0x2000
> 
> +/* Structure of a grade - starting block number
> + * and length of contiguous blocks with same higher
> + * grade (inclusive of starting block)
> + * example : if blocks 2,3,4 are higher graded,
> + * then block_num = 2 and len = 3
> + * Only high grade information is stored by this struct.
> + */

Wrong multi-line format style (unless ext4 is like netdev :).
Sayan Ghosh April 9, 2018, 5:27 p.m. | #2
Hi Randy,

Thank you for looking into the patchset and providing your suggestions
and comments.
We are working on the implementation for the current version of kernel
and will post the code for that soon.

We will definitely keep in mind the style and indentation issues that
you pointed out.

Regards,
Sayan Ghosh
‌On Fri, Apr 6, 2018 at 10:39 PM, Randy Dunlap <rdunlap@infradead.org> wrote:
> On 04/06/2018 04:41 AM, Sayan Ghosh wrote:
>> This introduces the different functions in order to get the grades as
>> the extended attributes while pre-allocating a new file. The grades
>> are stored as extended attributes while the file gets created. The
>> grades can be used by different user space applications as necessary.
>> The functions introduced are read_grade_xattr(), is_file_graded(),
>> read_count_xattr() which aim to read the extended attribute for grade
>> array and also to know whether the file is graded. The detailed
>> descriptions of the functions are provided as comments in the patch.
>> The patch is on top of Linux Kernel 4.7.2.
>
> Well, it's up to Ted if he wants to merge a patch that is based on 4.7.2.
> In general, patches are made to the current mainline or -next tree,
> while 4.7.2 was released on 2016-Aug-20.
>
>>
>> Signed-off-by: Sayan Ghosh <sgdgp.2014@gmail.com>
>> ---
>>  fs/ext4/ext4.h    | 15 +++++++++++++++
>>  fs/ext4/extents.c | 35 +++++++++++++++++++++++++++++++++++
>>  2 files changed, 50 insertions(+)
>>
>> diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
>> index b84aa1c..b9ec0ca 100755
>> --- a/fs/ext4/ext4.h
>> +++ b/fs/ext4/ext4.h
>> @@ -136,6 +136,18 @@ enum SHIFT_DIRECTION {
>>  /* Use blocks from reserved pool */
>>  #define EXT4_MB_USE_RESERVED        0x2000
>>
>> +/* Structure of a grade - starting block number
>> + * and length of contiguous blocks with same higher
>> + * grade (inclusive of starting block)
>> + * example : if blocks 2,3,4 are higher graded,
>> + * then block_num = 2 and len = 3
>> + * Only high grade information is stored by this struct.
>> + */
>
> Wrong multi-line format style (unless ext4 is like netdev :).
>
> --
> ~Randy

</sgdgp.2014@gmail.com></rdunlap@infradead.org>
Jan Kara April 19, 2018, 3:40 p.m. | #3
On Fri 06-04-18 17:11:40, Sayan Ghosh wrote:
> This introduces the different functions in order to get the grades as
> the extended attributes while pre-allocating a new file. The grades
> are stored as extended attributes while the file gets created. The
> grades can be used by different user space applications as necessary.
> The functions introduced are read_grade_xattr(), is_file_graded(),
> read_count_xattr() which aim to read the extended attribute for grade
> array and also to know whether the file is graded. The detailed
> descriptions of the functions are provided as comments in the patch.
> The patch is on top of Linux Kernel 4.7.2.
> 
> Signed-off-by: Sayan Ghosh <sgdgp.2014@gmail.com>

Thanks for the patch! The fact that this is based on rather old kernel has
been already mentioned - you really need to base on much newer kernel to
get this merged. Another problem I see is that there's no description of
the design of this feature. I.e., What this feature is good for? And how is
it supposed to work? Probably before investing too much time into rebasing
you can start with sending the high level design of the feature for
discussion. From quickly glancing through the patches I gather it is some
kind of HSM but I'm not completely sure...

								Honza

> ---
>  fs/ext4/ext4.h    | 15 +++++++++++++++
>  fs/ext4/extents.c | 35 +++++++++++++++++++++++++++++++++++
>  2 files changed, 50 insertions(+)
> 
> diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
> index b84aa1c..b9ec0ca 100755
> --- a/fs/ext4/ext4.h
> +++ b/fs/ext4/ext4.h
> @@ -136,6 +136,18 @@ enum SHIFT_DIRECTION {
>  /* Use blocks from reserved pool */
>  #define EXT4_MB_USE_RESERVED        0x2000
> 
> +/* Structure of a grade - starting block number
> + * and length of contiguous blocks with same higher
> + * grade (inclusive of starting block)
> + * example : if blocks 2,3,4 are higher graded,
> + * then block_num = 2 and len = 3
> + * Only high grade information is stored by this struct.
> + */
> +struct grade_struct {
> +    ext4_lblk_t block_num;
> +    unsigned long long len;
> +};
> +
>  struct ext4_allocation_request {
>      /* target inode for block we're allocating */
>      struct inode *inode;
> @@ -3186,6 +3198,9 @@ extern int ext4_check_blockref(const char *, unsigned int,
>  /* extents.c */
>  struct ext4_ext_path;
>  struct ext4_extent;
> +extern unsigned long long read_count_xattr(struct inode *inode);
> +extern void read_grade_xattr(struct inode *inode,struct grade_struct
> *grade_array);
> +extern int is_file_graded(struct inode *inode);
> 
>  /*
>   * Maximum number of logical blocks in a file; ext4_extent's ee_block is
> diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
> index d7ccb7f..de9194f 100755
> --- a/fs/ext4/extents.c
> +++ b/fs/ext4/extents.c
> @@ -57,6 +57,41 @@
>  #define EXT4_EXT_DATA_VALID1    0x8  /* first half contains valid data */
>  #define EXT4_EXT_DATA_VALID2    0x10 /* second half contains valid data */
> 
> +/*
> + * read_grade_xattr() is used to read the grade array from the
> extended attribute.
> + */
> +void read_grade_xattr(struct inode *inode,struct grade_struct *grade_array)
> +{
> +    const char *xattr_name = "grade_array";
> +    int xattr_size = ext4_xattr_get(inode,
> EXT4_XATTR_INDEX_USER,xattr_name, NULL,0);
> +    xattr_size = ext4_xattr_get(inode,
> EXT4_XATTR_INDEX_USER,xattr_name, (void *)grade_array,xattr_size);
> +    return;
> +}
> +
> +/*
> + * is_file_graded() returns whether the file has a grade information or not.
> + * It takes the inode number as a parameter.
> + */
> +int is_file_graded(struct inode *inode)
> +{
> +    const char *xattr_name = "is_graded";
> +    int is_graded = 0;
> +    int xattr_size = sizeof(int);
> +    xattr_size = ext4_xattr_get(inode,
> EXT4_XATTR_INDEX_USER,xattr_name, (void *)&is_graded,xattr_size);
> +    return is_graded;
> +}
> +
> +/*
> + * read_count_xattr() used to get the number of the elements in the
> grade array.
> + */
> +unsigned long long read_count_xattr(struct inode *inode)
> +{
> +    const char *xattr_name = "grade_array";
> +    unsigned long long xattr_size = ext4_xattr_get(inode,
> EXT4_XATTR_INDEX_USER,xattr_name, NULL,0);
> +    unsigned long long total = xattr_size/sizeof(struct grade_struct);
> +    return total;
> +}
> +
>  static __le32 ext4_extent_block_csum(struct inode *inode,
>                       struct ext4_extent_header *eh)
>  {
> ‌

Patch

diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index b84aa1c..b9ec0ca 100755
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -136,6 +136,18 @@  enum SHIFT_DIRECTION {
 /* Use blocks from reserved pool */
 #define EXT4_MB_USE_RESERVED        0x2000

+/* Structure of a grade - starting block number
+ * and length of contiguous blocks with same higher
+ * grade (inclusive of starting block)
+ * example : if blocks 2,3,4 are higher graded,
+ * then block_num = 2 and len = 3
+ * Only high grade information is stored by this struct.
+ */
+struct grade_struct {
+    ext4_lblk_t block_num;
+    unsigned long long len;
+};
+
 struct ext4_allocation_request {
     /* target inode for block we're allocating */
     struct inode *inode;
@@ -3186,6 +3198,9 @@  extern int ext4_check_blockref(const char *, unsigned int,
 /* extents.c */
 struct ext4_ext_path;
 struct ext4_extent;
+extern unsigned long long read_count_xattr(struct inode *inode);
+extern void read_grade_xattr(struct inode *inode,struct grade_struct
*grade_array);
+extern int is_file_graded(struct inode *inode);

 /*
  * Maximum number of logical blocks in a file; ext4_extent's ee_block is
diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index d7ccb7f..de9194f 100755
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -57,6 +57,41 @@ 
 #define EXT4_EXT_DATA_VALID1    0x8  /* first half contains valid data */
 #define EXT4_EXT_DATA_VALID2    0x10 /* second half contains valid data */

+/*
+ * read_grade_xattr() is used to read the grade array from the
extended attribute.
+ */
+void read_grade_xattr(struct inode *inode,struct grade_struct *grade_array)
+{
+    const char *xattr_name = "grade_array";
+    int xattr_size = ext4_xattr_get(inode,
EXT4_XATTR_INDEX_USER,xattr_name, NULL,0);
+    xattr_size = ext4_xattr_get(inode,
EXT4_XATTR_INDEX_USER,xattr_name, (void *)grade_array,xattr_size);
+    return;
+}
+
+/*
+ * is_file_graded() returns whether the file has a grade information or not.
+ * It takes the inode number as a parameter.
+ */
+int is_file_graded(struct inode *inode)
+{
+    const char *xattr_name = "is_graded";
+    int is_graded = 0;
+    int xattr_size = sizeof(int);
+    xattr_size = ext4_xattr_get(inode,
EXT4_XATTR_INDEX_USER,xattr_name, (void *)&is_graded,xattr_size);
+    return is_graded;
+}
+
+/*
+ * read_count_xattr() used to get the number of the elements in the
grade array.
+ */
+unsigned long long read_count_xattr(struct inode *inode)
+{
+    const char *xattr_name = "grade_array";
+    unsigned long long xattr_size = ext4_xattr_get(inode,
EXT4_XATTR_INDEX_USER,xattr_name, NULL,0);
+    unsigned long long total = xattr_size/sizeof(struct grade_struct);