About a week ago John Neil wrote an e-mail to the GAP forum asking about
the possibilities of byte compiling the library.

There are two different issues addressed in his e-mail, which I like to
discuss a little bit. First let us take a look at the libraries and the
time it takes to load them. Then in another e-mail I will discuss the
issue of the interpretation of GAP programs.

When you start GAP, it only reads '<gap-dir>/lib/init.g'. This file does
not contain any library functions (except some random stuff like
'ReadLib', 'ReadGrp', etc.). However, it contains lines of the form

AUTO( ReadLib( "combinat" ),
Factorial, Binomial, Bell, Stirling1, Stirling2, CombinationsA,
... );

Such a statement makes the declares the functions (actually variables)
'Factorial', 'Binomial', etc. When such a variable, e.g., 'Factorial', is
evaluated GAP executes the statement 'ReadLib("combinat");'. This
statement will cause the file '<gap-lib>/lib/combinat.g' to be read, which
in turn contains the definition of 'Factorial' etc. The next time
'Factorial' is called it is already read, and need not be read again.

This does *not* take very long. On the DECstation 5000/120 I am using,
reading 'combinat.g' takes about 350 msecs. Note that 'combinat.g' is a
file with 1500 lines, defining 56 functions.

Of course you all know that there must be more to it, because you all know
that something as simple as 'Group( (1,2), (1,2,3) )' will take more than
5 secs for the first time. The problem is that this simple statement will
cause a lot of files to be read. If you want to trace this, set
'InfoRead1:=Print;' before executing 'Group( (1,2), (1,2,3) )' (I have
this set in my '.gaprc' file). This will give the following output (the
indentation is used to show which file is read from within which other
file. For example 'grphomom' is read from within 'group', and in turn
reads 'mapping'.)

#I  ReadLib( "group" )
#I    ReadLib( "domain" )
#I    ReadLib( "grpelms" )
#I    ReadLib( "grphomom" )
#I      ReadLib( "mapping" )
#I    ReadLib( "operatio" )
#I    ReadLib( "grpcoset" )
#I    ReadLib( "grpprods" )
#I    ReadLib( "lattgrp" )
#I  ReadLib( "group" ) done
#I  ReadLib( "list" )
#I  ReadLib( "list" ) done
#I  ReadLib( "permutat" )
#I    ReadLib( "permgrp" )
#I      ReadLib( "permoper" )
#I      ReadLib( "permbckt" )
#I      ReadLib( "permnorm" )
#I      ReadLib( "permhomo" )
#I      ReadLib( "permprod" )
#I      ReadLib( "lattperm" )
#I  ReadLib( "permutat" ) done

Those files account together for 23500 lines, and about 700 Kbyte. If one
looks closer at the profile one sees that about 30 % of the 5 secs are
spend reading the text files. This part could be improved by about a
factor of 2. Another 30 % are spent in the scanner assembling the single
characters into symbols. This part could also be improved by a factor of
2. (This figures. 'wc', which does something somewhat simpler, needs
about 1 second to count the lines, words, and characters in those files).
The final 30 % are spent in the reader building the syntax trees. I see
no easy way to improve the performance of this part. Thus the whole time
could be reduced to about 3--4 seconds.

Now let us compare this with EMACS, which Neil mentiones in his e-mail.
EMACS takes about 16 seconds to read the file 'gnus.el' 3 times, which
amounts to about the same number of lines and bytes. Bytecompiling this
file and loading the byte compiled file 3 times still takes about 8
seconds.

The point I want to make is that it is not the mechanism by which GAP
reads its library that makes this process slow. It is the pure amount of
code in the library. The total GAP library is currently about 86000 lines
and 2.8 MBytes. This is larger than the EMACS library (which, as of
version 18.58, is about 61000 lines and 2.2 MByte, including stuff like
'ada.el', 'vi.el', 'vip.el', etc.). And while you do most of your editing
with only a few of the library files of EMACS, almost any nontrivial
application of GAP will use (read) about half of the library.

Thus simply saving the syntax trees GAP builds to a disk file and
recalling them when needed would not help that much. Also note that this
is not what EMACS does. EMACS's byte compiled files contain the bytecode
for each function proper. However, they also contain the name of the
function defined, the description string of this function, and the
constant vector. In fact the bytecode is usually only a small part of
such a definition. I think that the most important reason why EMACS's
byte compiled files are read in faster than the corresponding source files
is that they contain no comments.

This is of course also an option for GAP. Take the library and remove all
the comments and unnecessary blanks. However, the effect is not that
dramatic. While the space is reduced by a factor of 2, the time is
reduced from 5.3 seconds to 4.3 seconds. This is probably because the
part of the scanner that drops the comments is pretty efficient.

Anyhow, there is another option, which EMACS and TeX use. One could load
(most) of the library, and then dump a core file. Then you modify this
core dump file (in an unfortunately very operating system dependent way)
and obtain a new executable. When you execute this file all the parts of
the library that where loaded when you produced this core dump will be
automatically loaded again. This is very much faster, because no
processing needs to be done at all, and also because the routines in the
UNIX kernel that copy a file into memory for execution are pretty much
optimized. We are investigating this possibility further, also in
connection with workspace files.

Tommorow more on the issue of interpreting GAP programs.

Martin.

--
Martin Sch"onert, Martin.Schoenert@Math.RWTH-Aachen.DE, +49 241 804551
Lehrstuhl D f"ur Mathematik, Templergraben 64, RWTH, D 51 Aachen, Germany