diff mbox

ioctl() numbers are unsigned (the man page lies)

Message ID 1328761361-23119-1-git-send-email-david@gibson.dropbear.id.au
State New
Headers show

Commit Message

David Gibson Feb. 9, 2012, 4:22 a.m. UTC
In kvm-all.c we store an ioctl cmd number in the irqchip_inject_ioctl field
of KVMState, which has type 'int'.  This seems to make sense since the
ioctl() man page says that the cmd parameter has type int.

However, the kernel treats ioctl numbers as unsigned - sys_ioctl() takes an
unsigned int, and the macros which generate ioctl numbers expand to
unsigned expressions.  Furthermore, some ioctls (IOC_READ ioctls on x86
and IOC_WRITE ioctls on powerpc) have bit 31 set, and so would be negative
if interpreted as an int. This has the surprising and compile-breaking
consequence that in kvm_irqchip_set_irq() where we do:
    return (s->irqchip_inject_ioctl == KVM_IRQ_LINE) ? 1 : event.status;
We will get a "comparison is always false due to limited range of data
type" warning from gcc if KVM_IRQ_LINE is one of the bit-31-set ioctls,
which it is on powerpc.

So, despite the fact that the man page and posix say ioctl numbers are
signed, they're actually unsigned.  The kernel uses unsigned, the glibc
header uses unsigned long, and FreeBSD, NetBSD and OSX also use unsigned
long ioctl numbers in the code.

Therefore, this patch changes the variable to be unsigned, fixing the
compile.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 kvm-all.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

Comments

Jan Kiszka Feb. 9, 2012, 8:30 a.m. UTC | #1
Looks like the subject would warmly welcome a "kvm:" tag.

On 2012-02-09 05:22, David Gibson wrote:
> In kvm-all.c we store an ioctl cmd number in the irqchip_inject_ioctl field
> of KVMState, which has type 'int'.  This seems to make sense since the
> ioctl() man page says that the cmd parameter has type int.
> 
> However, the kernel treats ioctl numbers as unsigned - sys_ioctl() takes an
> unsigned int, and the macros which generate ioctl numbers expand to
> unsigned expressions.  Furthermore, some ioctls (IOC_READ ioctls on x86
> and IOC_WRITE ioctls on powerpc) have bit 31 set, and so would be negative
> if interpreted as an int. This has the surprising and compile-breaking
> consequence that in kvm_irqchip_set_irq() where we do:
>     return (s->irqchip_inject_ioctl == KVM_IRQ_LINE) ? 1 : event.status;
> We will get a "comparison is always false due to limited range of data
> type" warning from gcc if KVM_IRQ_LINE is one of the bit-31-set ioctls,
> which it is on powerpc.
> 
> So, despite the fact that the man page and posix say ioctl numbers are
> signed, they're actually unsigned.  The kernel uses unsigned, the glibc
> header uses unsigned long, and FreeBSD, NetBSD and OSX also use unsigned
> long ioctl numbers in the code.
> 
> Therefore, this patch changes the variable to be unsigned, fixing the
> compile.
> 
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> ---
>  kvm-all.c |    5 ++++-
>  1 files changed, 4 insertions(+), 1 deletions(-)
> 
> diff --git a/kvm-all.c b/kvm-all.c
> index 0b87658..681ad15 100644
> --- a/kvm-all.c
> +++ b/kvm-all.c
> @@ -78,7 +78,10 @@ struct KVMState
>      int pit_in_kernel;
>      int xsave, xcrs;
>      int many_ioeventfds;
> -    int irqchip_inject_ioctl;
> +    /* The man page (and posix) say ioctl numbers are signed int, but
> +     * they're not.  Linux, glibc and *BSD all treat ioctl numbers as
> +     * unsigned, and treating them as signed here can break things */

What about naming the problem instead:

/* Comparison with IOCTL macros on 32-bit hosts requires unsigned. */

> +    unsigned irqchip_inject_ioctl;
>  #ifdef KVM_CAP_IRQ_ROUTING
>      struct kvm_irq_routing *irq_routes;
>      int nr_allocated_irq_routes;

Jan
David Gibson Feb. 9, 2012, 9:58 a.m. UTC | #2
On Thu, Feb 09, 2012 at 09:30:09AM +0100, Jan Kiszka wrote:
> Looks like the subject would warmly welcome a "kvm:" tag.
> 
> On 2012-02-09 05:22, David Gibson wrote:
[snip]
> 
> What about naming the problem instead:
> 
> /* Comparison with IOCTL macros on 32-bit hosts requires unsigned. */

Just once, it would be nice to post something to this list and get a
substantive comment _before_ the bitching about minutiae.

Oh, and it's not just 32-bit hosts, it's anywhere sizeof(int)==4,
which includes ppc64 amongst other 64-bit hosts.
Jan Kiszka Feb. 9, 2012, 10:06 a.m. UTC | #3
On 2012-02-09 10:58, David Gibson wrote:
> On Thu, Feb 09, 2012 at 09:30:09AM +0100, Jan Kiszka wrote:
>> Looks like the subject would warmly welcome a "kvm:" tag.
>>
>> On 2012-02-09 05:22, David Gibson wrote:
> [snip]
>>
>> What about naming the problem instead:
>>
>> /* Comparison with IOCTL macros on 32-bit hosts requires unsigned. */
> 
> Just once, it would be nice to post something to this list and get a
> substantive comment _before_ the bitching about minutiae.
> 
> Oh, and it's not just 32-bit hosts, it's anywhere sizeof(int)==4,
> which includes ppc64 amongst other 64-bit hosts.

Then state "if int is 32 bits". The comment is unfortunately not helpful
without the commit log.

Jan
Paul Brook Feb. 9, 2012, 12:07 p.m. UTC | #4
> >> What about naming the problem instead:
> >> 
> >> /* Comparison with IOCTL macros on 32-bit hosts requires unsigned. */
> > 
> > Just once, it would be nice to post something to this list and get a
> > substantive comment _before_ the bitching about minutiae.
> > 
> > Oh, and it's not just 32-bit hosts, it's anywhere sizeof(int)==4,
> > which includes ppc64 amongst other 64-bit hosts.
> 
> Then state "if int is 32 bits". The comment is unfortunately not helpful
> without the commit log.

Int is 32 bits on every host we're ever likey to care about, so clearly 
there's something else going on.

Paul
Jan Kiszka Feb. 9, 2012, 2:11 p.m. UTC | #5
On 2012-02-09 13:07, Paul Brook wrote:
>>>> What about naming the problem instead:
>>>>
>>>> /* Comparison with IOCTL macros on 32-bit hosts requires unsigned. */
>>>
>>> Just once, it would be nice to post something to this list and get a
>>> substantive comment _before_ the bitching about minutiae.
>>>
>>> Oh, and it's not just 32-bit hosts, it's anywhere sizeof(int)==4,
>>> which includes ppc64 amongst other 64-bit hosts.
>>
>> Then state "if int is 32 bits". The comment is unfortunately not helpful
>> without the commit log.
> 
> Int is 32 bits on every host we're ever likey to care about, so clearly 
> there's something else going on.

Yep. So a better explanation could be "Required for comparison with
unsigned IOCTL macro values". This wasn't noticed so far as the original
code was never build on anything but IA32/64.

Jan
David Gibson Feb. 10, 2012, 6:43 a.m. UTC | #6
On Thu, Feb 09, 2012 at 11:06:57AM +0100, Jan Kiszka wrote:
> On 2012-02-09 10:58, David Gibson wrote:
> > On Thu, Feb 09, 2012 at 09:30:09AM +0100, Jan Kiszka wrote:
> >> Looks like the subject would warmly welcome a "kvm:" tag.
> >>
> >> On 2012-02-09 05:22, David Gibson wrote:
> > [snip]
> >>
> >> What about naming the problem instead:
> >>
> >> /* Comparison with IOCTL macros on 32-bit hosts requires unsigned. */
> > 
> > Just once, it would be nice to post something to this list and get a
> > substantive comment _before_ the bitching about minutiae.
> > 
> > Oh, and it's not just 32-bit hosts, it's anywhere sizeof(int)==4,
> > which includes ppc64 amongst other 64-bit hosts.
> 
> Then state "if int is 32 bits". The comment is unfortunately not helpful
> without the commit log.

It'd also be nice if people didn't drop me from the CC list on my own
patches.

I have resent the patch with a revised summary.
David Gibson Feb. 10, 2012, 6:45 a.m. UTC | #7
On Thu, Feb 09, 2012 at 03:11:13PM +0100, Jan Kiszka wrote:
> On 2012-02-09 13:07, Paul Brook wrote:
> >>>> What about naming the problem instead:
> >>>>
> >>>> /* Comparison with IOCTL macros on 32-bit hosts requires unsigned. */
> >>>
> >>> Just once, it would be nice to post something to this list and get a
> >>> substantive comment _before_ the bitching about minutiae.
> >>>
> >>> Oh, and it's not just 32-bit hosts, it's anywhere sizeof(int)==4,
> >>> which includes ppc64 amongst other 64-bit hosts.
> >>
> >> Then state "if int is 32 bits". The comment is unfortunately not helpful
> >> without the commit log.
> > 
> > Int is 32 bits on every host we're ever likey to care about, so clearly 
> > there's something else going on.
> 
> Yep. So a better explanation could be "Required for comparison with
> unsigned IOCTL macro values". This wasn't noticed so far as the original
> code was never build on anything but IA32/64.

Right.  The problem still exists on x86 and x86_64, but it occurs for
a different set of ioctls (IOC_READ instead of IOC_WRITE).  I guess we
never did this sort of signed comparison on an IOC_WRITE ioctl number
(comparing ioctl numbers for equality isn't a particularly obvious
thing to do from userspace, after all).
Jan Kiszka Feb. 10, 2012, 7:27 a.m. UTC | #8
On 2012-02-10 07:43, David Gibson wrote:
> On Thu, Feb 09, 2012 at 11:06:57AM +0100, Jan Kiszka wrote:
>> On 2012-02-09 10:58, David Gibson wrote:
>>> On Thu, Feb 09, 2012 at 09:30:09AM +0100, Jan Kiszka wrote:
>>>> Looks like the subject would warmly welcome a "kvm:" tag.
>>>>
>>>> On 2012-02-09 05:22, David Gibson wrote:
>>> [snip]
>>>>
>>>> What about naming the problem instead:
>>>>
>>>> /* Comparison with IOCTL macros on 32-bit hosts requires unsigned. */
>>>
>>> Just once, it would be nice to post something to this list and get a
>>> substantive comment _before_ the bitching about minutiae.
>>>
>>> Oh, and it's not just 32-bit hosts, it's anywhere sizeof(int)==4,
>>> which includes ppc64 amongst other 64-bit hosts.
>>
>> Then state "if int is 32 bits". The comment is unfortunately not helpful
>> without the commit log.
> 
> It'd also be nice if people didn't drop me from the CC list on my own
> patches.

Sorry, that was not intentional. My mail client possibly followed a tag
like in this mail:

Mail-Followup-To: Jan Kiszka <jan.kiszka@web.de>, anthony@codemonkey.ws,
	avi@redhat.com, mtosatti@redhat.com, qemu-devel@nongnu.org

I had to add you by hand to the list again. Maybe you wanna check what
is causing this on your side.

Jan
diff mbox

Patch

diff --git a/kvm-all.c b/kvm-all.c
index 0b87658..681ad15 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -78,7 +78,10 @@  struct KVMState
     int pit_in_kernel;
     int xsave, xcrs;
     int many_ioeventfds;
-    int irqchip_inject_ioctl;
+    /* The man page (and posix) say ioctl numbers are signed int, but
+     * they're not.  Linux, glibc and *BSD all treat ioctl numbers as
+     * unsigned, and treating them as signed here can break things */
+    unsigned irqchip_inject_ioctl;
 #ifdef KVM_CAP_IRQ_ROUTING
     struct kvm_irq_routing *irq_routes;
     int nr_allocated_irq_routes;