29 Apr
2020
29 Apr
'20
7:32 a.m.
Hi Juan,
Creating connections in complex networks can take time. Sometimes, it is possible to
improve on connection times by tweaks to the way in which the network is constructed.
Given that you have a quite large network, I assume you have a considerable number of
layers and thus also a quite large number of calls to ConnectLayers(). In that case, the
forthcoming NEST 3 will most likely reduce construction times noticeably, because layers
passed to Connect in a much more efficient way. We currently also have not fully
thread-parallelised connection construction for "divergent" connections, in
contrast to "convergent". We could look into that if switching between
"convergent" and "divergent" gives you noticeable improvements in
speed.
Please DO NOT USE MULTIPROCESSING with NEST. NEST internally parallelizes network
construction and maintains internal data structures in this process. Running several
ConnectLayers() calls simultaneously will lead to unpredictable results.
Best,
Hans Ekkehard
On 28 Apr 2020, at 20:27, Juan Manuel Vicente
<juanma.v82@gmail.com<mailto:juanma.v82@gmail.com>> wrote:
Hi all,
I'm trying to understand some inner workings of Nest. Rigth now I'm running
simulations with close half millons elements, using mpirun in a cluster with 25 nodes. The
problem I am having is that the "setup" (layer creation and connections) phase
takes close to 8min and the simulation only takes 1min.
So I tried to use python's multiprocessing package to speed it up, with the following
code:
nest.ResetKernel()
nest.SetKernelStatus({"local_num_threads": 1})
#...
connections = [
(layer1, layer1, conn_ee_dict, 1),
(layer1, layer2, conn_ee_dict, 2),
(layer2, layer2, conn_ee_dict, 3),
(layer2, layer1, conn_ee_dict, 4)
]
# Process the connections.
def parallel_topology_connect(parameters):
[pre, post, projection, number] = parameters
print(f"Connection number: {number}")
topology.ConnectLayers(pre, post, projection)
pool = multiprocessing.Pool(processes=4)
pool.map(parallel_topology_connect, connections)
The above example takes around 0.9s, but if the last two to lines are changed for a
sequential call, it takes 2.1s:
for [pre, post, projection, number] in connections:
print(f"Connection number: {number}")
topology.ConnectLayers(pre, post, projection)
So far the multiprocessing works great, the problem comes when the
"local_num_threads" parameters is changed from 1 to 2 or more, in the cluster it
could be 32. The code gets stuck in the topology.Connect without any error, after a while
I just stopped it.
Also I realised that the tolopoly.ConnectLayers just spawn one thread to connects layers
despite the local_num_threads is setted more than one.
Any idea what is going on?
Thanks in advance
Juan Manuel
PD: The full example code is attached (60 lines of code). The local_num_threads and
multiprocessing_flag variables change the behaviors of the code.
<smalltestcase.py>_______________________________________________
NEST Users mailing list --
users@nest-simulator.org<mailto:users@nest-simulator.org>
To unsubscribe send an email to
users-leave@nest-simulator.org<mailto:users-leave@nest-simulator.org>
--
Prof. Dr. Hans Ekkehard Plesser
Head, Data Science Section
Faculty of Science and Technology
Norwegian University of Life Sciences
PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560
Email hans.ekkehard.plesser(a)nmbu.no
Home http://arken.nmbu.no/~plesser