[GAP Forum] Multitasking with GAP

[Alan Hylton]

Howdy,

This is very helpful! I had not heard of SCSCP before.

It sounds like the idea is to have a GAP-to-GAP transfer via a TCP socket -
so really the "threading" comes from the ability to run a procedure on
"many" GAP instances. I see there is configpars.g, which suggests
approaches to how many ports/servers might be open, and then I can create
all of my processes at once using ParListWithSCSCP. Does this sound right?

I am curious - if I want to run on n cores, do I need to open GAP n+1 times?

Thank you again,
Alan


On Tue, May 18, 2021 at 2:26 AM Sergio Siccha <sergio at mathb.rwth-aachen.de>
wrote:

> Hi Alan,
>
> unfortunately `RunTask` in GAP is more or less only a wrapper for
> `CallFuncListWrap` and is indeed blocking. `RunTask` in GAP only is a
> mock-up of the proper `RunTask` function from HPC-GAP. HPC-GAP didn't
> make it out of alpha stage though and if I'm not mistaken nobody is
> working on it ATM.
>
> I'm not sure whether that's the best option as I have never used it, but
> in principle you should be able to do parallel computations by spawning
> several GAP processes via the SCSCP package, see chapter 8 "Parallel
> computing with SCSCP" of the SCSCP manual. So I guess as long as your
> computations don't need a lot of memory that should be fine.
>
> Hope this helps! :-)
>
> Best,
> Sergio
>
>
> On 18.05.21 07:35, Alan Hylton wrote:
> > Howdy,
> >
> > Suppose I have a list of lists, and I wish to run some time-consuming
> > process on each of these sub-list (each sub-list is independent, so I am
> > not worried about race conditions).
> >
> > I think the easiest way to demonstrate my thought process is with code:
> >
> > I have a list of lists and the number of cores I wish to use -
> > master_list := [ [...], [...], ..., [...]];
> > n := Length(master_list);
> > cores:=15;
> >
> > I have a time consuming function whose arguments are ranges into
> > master_list -
> > time_consuming_func := function(start_index, stop_index)
> > ...
> > end;
> >
> > I portion out master_list, creating a list a tasks -
> > task_list:=[];
> > start:=1;
> > for i in [1..cores] do
> >  flag:=0;
> >  if i <= n mod cores then
> >   flag:=1;
> >  fi;
> >  if i > n then
> >   break;
> >  fi;
> >  len:=Int(n/cores)+flag;
> >
> >  Add(task_list, RunTask( time_consuming_func , start, start+len-1));
> >
> >  start:=start+len;
> > od;
> >
> > I had several hopes:
> > 1: I could get a list of tasks, and then use something like TaskFinished
> to
> > see if each are done
> > 2: Store the result of each time_consuming_func in the global master_list
> >
> > But I ran into one problem: RunTask seems to be blocking. Instead of
> > spawning a process and continuing with my loop, it waits until each task
> > finishes. I considered DelayTask instead of RunTask so that I could just
> > use WaitTask, but DelayTask does not seem to exist (similarly for
> > asynchronous tasks). Is there an alternative? To follow the
> documentation,
> > I'd like to avoid the lower-level CreateThread if I can.
> >
> > Also, number 2 makes some assumptions on how memory works. Is it actually
> > valid to have a thread working on element i of master_list modify the
> > global master_list[i]?
> >
> > I'd greatly appreciate any insight!
> >
> > Thanks,
> > Alan
> > _______________________________________________
> > Forum mailing list
> > Forum at gap-system.org
> > https://mail.gap-system.org/mailman/listinfo/forum
> >
>