Dear Mr Alvarez,

allow me first to make some general remarks about your computation
before I explain what happened in your GAP session.

I tried to compute a presentation of the normal closure of
the subgroup generated by the commutator of the two generators of a free
group on two letters with gap, and it appears to be enumerating something
which is infinite.

Let F be the free group on a and b. The normal closure of [a,b]
is the commutator subgroup F' of F and F/F' is a free abelian
group of rank two. This is an infinite group, of course.
Furthermore, F' is a free group of infinite rank. Therefore, any
presentation for F' has infinitely many generators.

The following is an outline of the method used in GAP for
computing a presentation for a subgroup H of a finitely presented
group G. First GAP attempts to compute a coset table for the
cosets of H in G via the modified Todd-Coxeter method. This can
only succeed if H has finite index in G. Once the coset table is
completed, a generating set and a set of defining relations is
constructed for H. In your situation, the computation of the
coset table is not possible, since the index of F' in F is
infinite.

Even if one could tell GAP somehow that the normal closure of
[a,b] is a free subgroup of F of infinite rank, there is nothing
interesting one could do with this subgroup in GAP because there
is practically no support for groups given by an infinite
generating set.

gap> F := FreeGroup("a",' "b") ;;
gap> E := F / [] ;;
gap> a := E.1 ;; b := E.2 ;;
gap> R := Subgroup( E, [a*b*a^-1*b^-1] ) ;;
gap> PresentationNormalClosure( E, R ) ;
Error, the coset enumeration has defined more than 64000 cosets:
type 'return;' if you want to continue with a new limit of 128000 cosets,
type 'quit;' if you want to quit the coset enumeration,
type 'maxlimit := 0; return;' in order to continue without a limit,
in
CosetTableFpGroup( F, TrivialSubgroup( F ) ) called from
PresentationNormalClosure( E, R ) called from
main loop
brk> return ;

At this stage, the Todd-Coxeter method has computed 64 000 cosets
an asks for confirmation to compute more. If you modify your
example slightly by adding the elements a^2 and b^2 as generators
to R, you will get a subgroup whose normal closure has finite
index (although R itself doesn't).

With kind regards,
Werner Nickel.

------------------------------------------------------------------------------
   Dr (AUS) Werner Nickel                 Mathematics with Computer Science
                                                               TU Darmstadt 
   Tel: +49 (0)6151 163486                     Fachbereich Mathematik, AG 2
   Fax: +49 (0)6151 166535                              Schlossgartenstr. 7
   nickel@mathematik.tu-darmstadt.de                      D-64289 Darmstadt
------------------------------------------------------------------------------