[NEST Users] Re: DumpLayerConnections() enhancement request

Dear Xavier, I think the most efficient way would be to drop the entires “src_iter” loop and simply place the entire source node collection in conn_filter, just as the target collection. Then, one would only need a single GetConnections() call and would avoid a lot of duplicate parsing of the connection tables. The one downside with that approach is that one may need quite some memory to store the connectome returned by GetConnections(). If we assume a layer size of 10000 neurons and a connectivity of 10% within the layer, we get 10 million connections. Each of those requires 40B in the representation returned by GetConnections(), so 400 MB in total. But this is manageable in most cases. And if the simulation is distributed on say 8 MPI ranks, it would only be 50 MB per rank, and only until DumpLayerConnections completes. It would be nice if you could try to implement this. Could you try to use modern C++-style loops ( for ( auto& conn : …) )? The proper solution to avoid the memory overhead would be for GetConnections not to return the actual collection of connections, but actually a iterator over it so that we can avoid the memory overhead entirely. But that would be a larger change to the code. Best, Hans Ekkehard -- Prof. Dr. Hans Ekkehard Plesser Department of Data Science Faculty of Science and Technology Norwegian University of Life Sciences PO Box 5003, 1432 Aas, Norway Phone +47 6723 1560 Email hans.ekkehard.plesser@nmbu.no<mailto:hans.ekkehard.plesser@nmbu.no> Home http://arken.nmbu.no/~plesser From: Xavier Otazu &lt;xotazu(a)cvc.uab.cat&gt; Date: Saturday, 16 March 2024 at 12:35 To: users(a)nest-simulator.org &lt;users(a)nest-simulator.org&gt; Subject: [NEST Users] Re: DumpLayerConnections() enhancement request [Some people who received this message don't often get email from xotazu(a)cvc.uab.cat. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] Hi Hans, I have taken a look at the cpp code implementing DumpLayerConnections(). I have seen that, for every pre-synaptic neuron, it looks for the corresponding connections with all the post-sytnaptic neurons. Would it be faster to do it the other way around? That is, for every post-synaptic, looking for the corresponding pre-synaptic neurons. I was thinking about it because GetConnections() find connections on every MPI rank for every post-synaptic neuron. If you believe it could bring some speedup, I could try to modify the cpp code. Thanks a lot in advance, Xavier _______________________________________________ NEST Users mailing list -- users(a)nest-simulator.org To unsubscribe send an email to users-leave(a)nest-simulator.org