[NEST Users] Re: local_num_threads, multiprocessing and topology.ConnectLayers

Hi Juan, Creating connections in complex networks can take time. Sometimes, it is possible to improve on connection times by tweaks to the way in which the network is constructed. Given that you have a quite large network, I assume you have a considerable number of layers and thus also a quite large number of calls to ConnectLayers(). In that case, the forthcoming NEST 3 will most likely reduce construction times noticeably, because layers passed to Connect in a much more efficient way. We currently also have not fully thread-parallelised connection construction for "divergent" connections, in contrast to "convergent". We could look into that if switching between "convergent" and "divergent" gives you noticeable improvements in speed. Please DO NOT USE MULTIPROCESSING with NEST. NEST internally parallelizes network construction and maintains internal data structures in this process. Running several ConnectLayers() calls simultaneously will lead to unpredictable results. Best, Hans Ekkehard On 28 Apr 2020, at 20:27, Juan Manuel Vicente &lt;juanma.v82(a)gmail.com&gt; wrote: Hi all, I'm trying to understand some inner workings of Nest. Rigth now I'm running simulations with close half millons elements, using mpirun in a cluster with 25 nodes. The problem I am having is that the "setup" (layer creation and connections) phase takes close to 8min and the simulation only takes 1min. So I tried to use python's multiprocessing package to speed it up, with the following code: nest.ResetKernel() nest.SetKernelStatus({"local_num_threads": 1}) #... connections = [ (layer1, layer1, conn_ee_dict, 1), (layer1, layer2, conn_ee_dict, 2), (layer2, layer2, conn_ee_dict, 3), (layer2, layer1, conn_ee_dict, 4) ] # Process the connections. def parallel_topology_connect(parameters): [pre, post, projection, number] = parameters print(f"Connection number: {number}") topology.ConnectLayers(pre, post, projection) pool = multiprocessing.Pool(processes=4) pool.map(parallel_topology_connect, connections) The above example takes around 0.9s, but if the last two to lines are changed for a sequential call, it takes 2.1s: for [pre, post, projection, number] in connections: print(f"Connection number: {number}") topology.ConnectLayers(pre, post, projection) So far the multiprocessing works great, the problem comes when the "local_num_threads" parameters is changed from 1 to 2 or more, in the cluster it could be 32. The code gets stuck in the topology.Connect without any error, after a while I just stopped it. Also I realised that the tolopoly.ConnectLayers just spawn one thread to connects layers despite the local_num_threads is setted more than one. Any idea what is going on? Thanks in advance Juan Manuel PD: The full example code is attached (60 lines of code). The local_num_threads and multiprocessing_flag variables change the behaviors of the code. <smalltestcase.py>_______________________________________________ NEST Users mailing list -- users(a)nest-simulator.org To unsubscribe send an email to users-leave(a)nest-simulator.org -- Prof. Dr. Hans Ekkehard Plesser Head, Data Science Section Faculty of Science and Technology Norwegian University of Life Sciences PO Box 5003, 1432 Aas, Norway Phone +47 6723 1560 Email hans.ekkehard.plesser(a)nmbu.no Home http://arken.nmbu.no/~plesser _______________________________________________ NEST Users mailing list -- users(a)nest-simulator.org To unsubscribe send an email to users-leave(a)nest-simulator.org