Subject: Re: makefile(.abi) vs. autoconf/automake/libtool/etc.
From: Paul Rohr (paul@abisource.com)
Date: Fri Feb 16 2001 - 01:47:23 CST
At 06:29 PM 2/15/01 -0600, Sam TH wrote:
>On Thu, Feb 15, 2001 at 11:03:11AM -0800, Paul Rohr wrote:
>Well, I hope they aren't disjoint sets. I like to think I fall in
>both A and B. I like the current system a lot. It's quite
>impressive, and it does its job well. And I do like, and like to
>think I understand, the autotools system.
Excellent. That makes you the perfect person to help us all through this
transition.
>There's one more category I would like to propose, because it's the
>main reason that I want to use autoconf:
>
>9. Allows the elimination of lines 176-188 of ut_types.h.
>
>That's the section where ICONV_CONST is defined. It's the worst hack
>I've ever written, and it's going to break some system sometime. But
>worse than all of those is this:
> It's a solved problem. People have dealt with just that sort
>of issue before, and figured out how to deal with it. Dynamic tests
>can be constructed, so that we don't have to keep adding defines. And
>that's available with autoconf.
Oh dear. Sam, you've got too much time on your hands. If that hack alone
is enough to get you to switch build environments...
I personally would never work that hard to save so little. That's exactly
the right kind of hack -- it makes clear that just about every "reasonable"
platform concurs about the constness of that library call. ;-)
Not that I'm objecting, mind you. More power to you.
>This point is related to 1, but I'll make it here. The other major
>requirement for the Makefile system is the Bourne Shell. This is a
>requirement for running sed and make, and also for a number of the
>tests. It's why we don't work on MPW (since that isn't a compatible
>shell, although I'm not sure why it can't be ported. Leonard?)
>
>The shell is present or available on basically every non-(classic Mac)
>computer in the world today.
Thanks for the clarification. I forgot about the bash/sh distinction here.
To be honest, I just tend to install as little of Cygwin as I can get away
with and then ignore everything else about that shell except my few favorite
commands, all of which Just Work:
cd
make
make clean
make tidy
make canonical
make realclean
All my real work happens elsewhere.
>However, the reason that it's not usually neccessary is that this
>problem is solved in a very different way by autotools build systems.
>The standard way of building with a configure script is to do it in
>different directory than the source tree.
>
> [...]
>
>Then, when you want to build with gnome enabled, or pspell, or
>debugging, or whatever, you create a different build directory, and
>repeat the process.
>
>making clean in one of the build directories clean that directory, and
>so on.
Gotcha. I knew there had to be a way. So do all the build targets get
sprinkled throughout that shadow copy of the tree?
>This poses a few complexities having to do with our peer directories,
>but I think they are solvable.
Yeah, I guess they'd need to be built in a per-config set of directories,
too, huh? Could we just have a script that setup an appropriate subtree and
fired off configure runs in each? For example:
buildabi/gnome_debug/abi
buildabi/gnome_debug/expat
buildabi/gnome_debug/psiconv
...
buildabi/win32_rel/abi
etc.
>It should be noted that the current system involves exactly one
>Makefile.abi, the one in wv. Both psiconv and expat use their native
>build systems, as do the various other libraries (I believe).
It looks like there's still a Makefile.abi in libiconv, and I'm pretty sure
we modified the Makefiles in our copies of libpng and zlib as well.
>> 7. rebuild speed
>> -----------------
>> (strength) Because this is a diving make system, rebuilds can be
localized
>> by diving to the appropriate level of the tree and doing the appropriate
>> make variants (tidy, clean, realclean) there.
>>
>> The scale factors are nice here, because this mirrors and reinforces the
>> modularity of the code. Localized API changes which only affect a small
>> part of the tree can be rebuilt quickly. API changes which affect the
>> entire tree require massive clean rebuilds of the tree (and usually get
>> mentioned as such during commits).
>
>Fundamentally, it works like this: if you change any significant
>header file, you have to rebuild the whole abi tree. If you don't,
>and it segfaults on something, then you rebuild the whole tree just to
>make sure. At least that's how it ends up for me.
What? You mean the rest of you don't keep the dependencies in your head?
Silly me. Jeff and I were the ones who originally decided which headers
should be exposed outside of the relevant modules, which is where the
capital vs. lowercase prefix originated.
Thus, I got used to being aware of the potential impact of any header file
change. I'd just go rebuild whatever portions of the tree needed it. Most
of the time, it wasn't much, and we always sent commit mails to each other
with that info. Full rebuilds of, say, af/xap or text/fmt were pretty
common, but that goes pretty quickly. Full rebuilds of the entire tree were
actually pretty rare.
However, there hasn't been anyone policing the spread of #includes for quite
a while now, so I wouldn't be surprised if rebuilds are needed more often.
>Actually, they can be used to handle the presence or abscence of
>virtually any feature of the target system. For example, instead of
>defining ABI_OPT_LIBXML2, an autoconf macro could detect what xml
>libraries you had on your system, define the appropriate symbols, and
>build with that library. You could, of course, override that choice.
>For a real look a lots of different macros, for lots of different
>things, check out
> http://cryp.to/autoconf-archive/
Yeah. The Unix world has a tradition of multiple slight incompatibilities
of this sort, which is what autoconf is all about. From outside that world,
even the most XP savvy of us tend to see that kind of "maybe this or maybe
that" phenomenon as just plain broken.
>Well, personally I think that our hardest-to-run-makefiles-on platform
>is Windows,
Really? If you're willing to run command-line make at all, then there's
very little functional difference (to us Win32 heathen) between opening a
DOS box to type nmake and opening a Cygwin shell to type make. They both do
the same job, and you get out of there as soon as the compiler has rendered
its verdict on your code. ;-)
All the annoyance comes from the fact that Cygwin keeps moving around its
Unix-ish view of a working filesystem and sending new weirdness to uname,
but we wind up begging Unix folks for help on that anyhow.
>but definitely a close second is BeOS/PPC. And while it
>still had telnet running (which for some unknown reason was closed,
>causing us to lose a shipping platform), I compiled GNU Make, GNU
>Autoconf, GNU Automake, GNU Libtool, and several projects using all of
>the above.
>
>Furthermore, people make significant effort to get the autotools to
>work with Cygwin, both with gcc and with MSVC.
That's promising.
>Well, I have no idea what dependency tracking looks like for
>cl.exe. If it has facilities to support this, they could be worked
>in.
I'll let the compiler gurus confirm this, but AFAIK, dependencies are
tracked externally via project files. I've never looked to see whether
cl.exe is used to extract the dependency information, though.
>Since I'm the one who seems to do most of the Makefile hacking
>recently, that sounds good to me. And the generated makefiles are
>quite simple. Here's src/wp/ap/GNUmakefile.am:
>
>LIBTOOL = @LIBTOOL@ --silent
>
>SUBDIRS= xp unix
>
>noinst_LTLIBRARIES= libAp.la
>
>libAp.la: xp/libWpAp_xp.la @PLATFORM@/libWpAp_@PLATFORM@.la
> $(LIBTOOL) --mode=link $(CXX) -o libAp.la xp/libWpAp_xp.la \
> @PLATFORM@/libWpAp_@PLATFORM@.la
>
>Pretty easy.
Yeah, the equivalent src/wp/ap/Makefile is more verbose, because each of the
OBJs gets mentioned explicitly. It's brain-dead simple work, though.
Is the difference as pronounced when you go down another level?
abi/src/wp/ap/xp/Makefile
abi/src/wp/ap/<platform>/Makefile
>Passing those enviroment variables is definitely busted. We want to
>pass options to configure that say "please use cl.exe as the
>compiler", among other things. I'm sure this can be done.
Cool.
>The real speed increase, however, comes because auto* generates
>Makefiles which can be run in parallel. This means that you can use
>the -j option to make, which allows it to run multiple processes at
>once. On a fast machine, where lots of time is spent in I/O, this can
>be a significant speedup.
Hmm. Is that what all those lock files are about?
>This doesn't work on the Makefile system, I've tried it. I don't know
>enough about parallel make right now to change that, or to know if it
>can be changed. I could, of course, learn. :-)
:-)
>Other than Hubert, who I feel bad for, I think that these aren't ever
>going to be neccessary. However, more power to Mike for keeping them
>up to date. I'm impressed.
Yeah. Ditto.
>Well, that was long. I hope it was informative, too.
Very. Thanks.
Paul
This archive was generated by hypermail 2b25 : Fri Feb 16 2001 - 07:36:08 CST