[RFC] allow low HZ values?

I'm not necessarily wanting to open up the age old question of "what is
a good HZ", but we were doing some testing on timer tick overheads for
HPC applications and this came up...

Below is a minimal hack at enabling lower HZ values.  The kernel builds
and boots for me on x86_64 (simple laptop and kvm configs) and ppc64
(misc. IBM System p) with each of the added HZ options.

There's explicit code checking HZ down to 12, but HZ<100 wasn't a config
option.  We collected some data at 10, 12 and 25.  There'd been some
question of whether 10 would even work or not but it looks fine in the
relatively minimal testing we did.  We tried 12 since the code seemed
to allow for it.  And 25 as a "safe" lower value.  The only difference
observed under load (ie: no no idle HZ in play) was the expected timer
tick happening less often.  There was definitely surprise that nothing
else seemed to break anywhere, especially at 10.

Do people feel it is reasonable to have Kconfig bits to allow some lower
HZ values?

If so, then there's the question of what breaks.  It's reasonable to
think there are other going to be subtleties buried in code around
assumptions on the likely range of HZ:

- I'm not sure that what I did in inet_timewait_sock.h and jiffies.h is
  reasonable.
- arch/x86/kernel/i8253.c throws a warning at line 43 (v2.6.36-rc7):
      warning: large integer implicitly truncated to unsigned type
- drivers/char/cyclades.c's cy_ioctl() warns:
      drivers/char/cyclades.c:2761: warning: division by zero
- drivers, drivers, drivers across all the arch's could use sanity checking

On Mon 11 Oct at 22:32:06 +0200 tglx@linutronix.de said:
> On Mon, 11 Oct 2010, Tim Pepper wrote:
> 
> > I'm not necessarily wanting to open up the age old question of "what is
> > a good HZ", but we were doing some testing on timer tick overheads for
> > HPC applications and this came up...
> 
> Yeah. This comes always up when the timer tick overhead on HPC is
> tested. And this patch is again the fundamentally wrong answer.

Yep.  Long term no hz is definitely the goal.  I'm not sufficiently
connected to the -rt space I guess to have followed that there's somebody
again looking in that direction.  The rfc patch was mostly just a minimal
is there anything simple we can do in the meantime exercise.

> We have told HPC folks for years that we need a kind of "NOHZ" mode
> for HPC where we can transparently switch off the tick when only one
> user space bound thread is active and switch back to normal once this
> thing terminates or goes into the kernel via a syscall.

I'd not heard of this in between NOHZ-y idea...sounds promising.
We'd talked about different non-idle no hz approaches in the past year
or so, some of which were on the veeery complicated side of the spectrum.

> Sigh, nothing
> happened ever except for repeating the same crap patches over and
> over.

I'll check out what Frederic is doing.  Thanks for the pointer and
apologies for the noise.

On 10/11/2010 03:33 PM, Thomas Gleixner wrote:
> On Tue, 12 Oct 2010, Benjamin Herrenschmidt wrote:
> 
>> On Mon, 2010-10-11 at 13:11 -0700, Tim Pepper wrote:
>>> I'm not necessarily wanting to open up the age old question of "what is
>>> a good HZ", but we were doing some testing on timer tick overheads for
>>> HPC applications and this came up...
>>
>> Note that this is also very useful when working on CPU prototypes
>> implemented in FPGAs and running at something like 12Mhz :-)
> 
> /me hands benh 0.5$ for a FPGA upgrade

That's often not possible if the CPU cannot be mapped onto a single FPGA
(either because the core is too large, multiple cores are tested, or
because there is debugging logic is included.)  The interconnects slows
things down tremendously.

	-hpa

Thomas Gleixner <tglx@linutronix.de> writes:

> On Mon, 11 Oct 2010, Tim Pepper wrote:
>
>> I'm not necessarily wanting to open up the age old question of "what is
>> a good HZ", but we were doing some testing on timer tick overheads for
>> HPC applications and this came up...
>
> Yeah. This comes always up when the timer tick overhead on HPC is
> tested. And this patch is again the fundamentally wrong answer.

That's a unfair description of the proposal.

> We have told HPC folks for years that we need a kind of "NOHZ" mode
> for HPC where we can transparently switch off the tick when only one
> user space bound thread is active and switch back to normal once this
> thing terminates or goes into the kernel via a syscall. Sigh, nothing
> happened ever except for repeating the same crap patches over and
> over.

Jan Blunck posted a patch for this exactly few months ago.
Unfortunately it didn't get the accounting right, but other than
that it seemed like a reasonable starting point.

-Andi

On Tue, 12 Oct 2010, Andi Kleen wrote:
> Thomas Gleixner <tglx@linutronix.de> writes:
> > We have told HPC folks for years that we need a kind of "NOHZ" mode
> > for HPC where we can transparently switch off the tick when only one
> > user space bound thread is active and switch back to normal once this
> > thing terminates or goes into the kernel via a syscall. Sigh, nothing
> > happened ever except for repeating the same crap patches over and
> > over.
> 
> Jan Blunck posted a patch for this exactly few months ago.
> Unfortunately it didn't get the accounting right, but other than
> that it seemed like a reasonable starting point.

Unfortunately it did not get a lot of other things right either.

Thanks,

	tglx