[SGVLUG] Time to boot getting longer with Debian

[David Lawyer]

On Mon, May 01, 2006 at 01:48:07PM -0700, Emerson, Tom wrote:
> > -----Original Message----- Of David Lawyer
> > On Sat, Apr 22, 2006 at 12:58:58PM -0700, John E. Kreznar wrote:
> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > David Lawyer <dave at lafn.org> writes:
> > > 
> > > > I just installed kernel 2.6.15 on my 11-year-old PC and it takes 
> > > > over 3 min. to boot. ... I could track down the delays ....
> > > 
> > > 2.6.16 (and maybe 2.6.15 as well?) has a new configuration parm
> > > PRINTK_TIME described as follows.
> > [snip]
> > Thanks for the info but if I watch the boot-time messages I 
> > can see a few places where it pauses far too long (I think) 
> > so I'll work on these first without using any timing tools.  
> > I've sent in the first bug report on this [...]
> 
> Speaking as someone who has manned the other end of the phone on calls
> like that, saying "I think it's taking too long" when your only proof is
> subjective will be resolved in the bit bucket.

For my first bug report re boot-time, I found that the kernel was probing
something that the kernel command line parameters said didn't exist.
Hopefully, I'll have similar clues to mention so it will be more than
just subjective timing that I report.  Of course I could use a stop
watch and report the results.

> that some (presumably) unchanged process now takes anywhere from 5 to 50
> times longer than before will get some attention.
> 
[snip]
> 
> > I've gotten such responses from Debian maintainers in the 
> > past. A bug was introduced in the latest revision of module 
> > load tools so that the ppp module of kernel 2.2.20 doesn't 
> > load (but other modules for 2.2.20 do load).
> 
> If an older module was written in a way that was marginally acceptable
> in the past but is now incompatable, then I'd support the "upgrade or
> die" philosophy [of course,

I should have explained a suspected reason why it doesn't load.  If
when running 2.4.27 (I run 2.2, 2.4, and 2.6 on my PC from the same
disk partition) I do "modprobe ppp" it doesn't load the ppp module
since the module with the exact name "ppp" seems to have become
obsolete after kernel 2.2.  Instead some other modules like
ppp_generic get loaded.  So the problem may be due to making other
modules an alias for ppp.  Thats why ppp doesn't load.  Since I don't
have source code, I just now did a "strings" command on the binary of
what modprobe calls (it's actually a program insmod.modutils which is a
lot different from the insmod program).  I found a new ppp string in the
latest binary of insmod.modutils and I suspect that an alias for ppp
was hard-coded in to support kernel 2.4.  But I think it's bad policy
to hardcode aliases since it could be done via a file.

So it's not the module ppp that's causing the problem.  It's likely
that the loader program (insmod...) tries to load an alias of ppp when
it should load the real ppp module when the version is 2.2.  I could
be wrong of course since my evidence is only circumstantial.

> I don't know offhand the quality of the
> module in question nor what actually "broke" between 2.2.20 and now, but
> I suspect that any fix to the 2.2.20 version would be significant.

I'm not asking for a fix to 2.2.20.  I'm asking for a fix to the
latest version of insmod.modutils which is used only for versions
prior to 2.6 (the modutils package is not for 2.6 but for 2.4 and
2.2).  Here's a description of this package.  Well, if they won't fix
it, they'll at least need to remove the reference to 2.2 below.

Description: Linux module utilities
 These utilities are intended to make a Linux modular 2.2 or 2.4
 kernel manageable for all users, administrators and distribution
 maintainers. For 2.6 kernels, you should use the module-init-tools

> Also, I just realized what you're saying (or implying?) is that you're
> trying to load a 2.2 version of a module in a 2.6 kernel

I'm loading ppp for 2.2 into the 2.2 kernel.  As we all should know, a
module must be used in the kernel version it was intended for (unless
you're lucky).  Try to load it in the wrong kernel and it often will
not load due to unresolved symbols.  I'm now using 2.2 and ppp.o is
now loaded due to my reverting to the old version of insmod.modutils.

> -- that sounds extremely suspect to me.  Is there no "version 2.6"
> of this module?]
> 
> > Without the ppp module I can't connect to the Internet 
> 
> Hmmm... Given how critical this module would be to "everyone", I'd
> imagine there is a proper 2.6 version of the module -- is there a
> particular reason you are avoiding it's use?

Yes.  2.2 boots much faster.  If I want to wait over 3 min. and boot
2.6.15, then ppp works OK, but the modules used for ppp in 2.6 have
names like ppp_generic, etc.  But if I've got something else to do
that will take a few min. then I may boot 2.6.15.

> > ...  The response was 
> > that 2.2.20 is no longer supported.  But I'm appealing this 
> > and will let this list know if I succeed in getting this bug fixed.

> If I understand this, you're asking that the 2.6 kernel fully support an
> older version of a module when there may be a perfectly good "current"
> version of that same module, correct?
No.
> 
> > And when Tom E. wrote that the longer boot times might be a 
> > price we pay for progress, I think it's a price we pay for 
> > not having people like me that complain about it, try to find 
> > the causes for it, and get them fixed.

I should have added that there are a number of others also concerned
about longer boot times and some of them are trying to fix the
situation (and doing a lot more than I am).  It's especially important
for embedded stuff.

> 
> My previous comment about the "price of progress" is the fact that the
> kernel [should be] doing /significantly/ more now than in the past.
> Overall strategies and methods may have changed, so it's not surprising
> to see different (and generally longer) times to accomplish the same
> thing as in the past.  The article I referrenced did point out, however,
> that something as simple as an "fopen" was being performed some 80,000+
> times!  Each fopen, in turn, would end up calling a "walk the directory
> tree" routine, which then gets down to (possibly un-cached) disk reads
> and associated I/O.  Some investigation into this revealed that things
> like .conf files were being repeatedly opened, parsed, and closed by the
> same process (as opposed to perhaps caching the information retrieved to
> avoid further fopens), so there was an underlying philosophy problem [of
> referrencing a file on disk /every time/ to evaluate a configuration
> parameter instead of storing the result locally] that is now bordering
> on the "extreme" when spread across /every/ kernel initialization
> process.
> 
> But, as pointed out elsewhere on this thread, there is a trade off being
> taken into consideration: if the /only/ time a procedure ends up being
> "wasteful" is during startup, then it is unlikely to be optimized or
> "repaired" because the net effect of fixing that process is negligable
> over the "uptime" of a given system.

For home use my "uptime" is often short, sometimes as short as 10 min.
to 1 hr. when I need to look up something either on the Internet or in
my personal files.  If I leave my terminal for several minutes, my
wife is likely to turn off my computer to save electricity.  There's a
problem with kernel 2.2 in this situation since it doesn't support the
ext3 file system for fast recovery from "pulling the plug".

> OTOH, if improving some kernel startup process /also/ improves
> day-to-day operation of that process, then by all means optimizing
> or "fixing" that process should become a priority.

Since most Linux use is for servers, you're right.  But for desktop
use by people who turn it off when not in use, then boot-times become
important, especially if one needs to get info from a PC (or the
Internet) in a hurry.

A reason for keeping 2.2.20 is to have a benchmark for fast booting.
Perhaps I'm not being fair, since the 2.2.20 I'm using was put
together by Debian and named 2.2.20-compact so it was intentionally made
small and mainly uses modules instead of having support for various
devices compiled into the kernel.  So perhaps I'm comparing apples and
oranges since 2.6.15 is not compact.  But I'm convinced that there is
great inefficiency in booting my PC with 2.6.15.

I hope that people are aware of the computer situation for most of the
6+ billion people in the world.  Many of those that do have PCs can't
afford new ones and thus a niche for Linux on the desktop is to insure
that it supports old PCs and can run the latest kernel on many of them.

I also just now checked my email archive for June 1999 and find that I
complained to this lug that 2.2 was too slow to boot and asked for
help but didn't get any.  As it turned out from further investigation,
it was a race condition and only people with old computers had the
problem.  But it got fixed.  Over a hundred processes were created,
all trying to do the same thing: load a certain module.  As I seem to
recall, the problem solved itself when the PC ran out of memory and
stopped the "process explosion" as I called it.  But it took almost a
minute for this to happen on a 486 PC.

			David Lawyer