Hi!
I am using DumpLayerConnections() with two layers with around 100x100 nodes each layer. Every layer node is connected to around 100 nodes on the other layer. Calling the DumpLayerConnections() takes around 10-15 minutes to save connections to file. I know it is a too general request, but it would be nice if this function is faster.
In my particular case, I obtain these connections in a previous step using GetConnections(). I guess (hence I am probably wrong) that DumpLayerConnections() calls GetConnections(). If I am right, I believe it could be an interesting enhancement to implement a DumpLayerConnections() function with input parameters that are a set of connections, that is, something like:
myconn = nest.GetConnections(layer1, layer2, ...) nest.DumpLayerConnections(myconn)
Thanks a lot in advance!
Xavier
Hello Xavier,
A 100 x 100 layer with 100 connections per neuron would give about 1 million neurons. I just tried it on my computer and dumped 1 million connections in 50 seconds. That still seems very long, given that GetConnections() completes in 0.6s with no arguments and in 2.6s when source, target and synapse model are given.
Could you open an issue about the slow DumpLayerConnection on Github?
Best, Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science Faculty of Science and Technology Norwegian University of Life Sciences PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560 Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no Home http://arken.nmbu.no/~plesser
From: Xavier Otazu xotazu@cvc.uab.cat Date: Friday, 8 March 2024 at 14:06 To: users@nest-simulator.org users@nest-simulator.org Subject: [NEST Users] DumpLayerConnections() enhancement request [Some people who received this message don't often get email from xotazu@cvc.uab.cat. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
Hi!
I am using DumpLayerConnections() with two layers with around 100x100 nodes each layer. Every layer node is connected to around 100 nodes on the other layer. Calling the DumpLayerConnections() takes around 10-15 minutes to save connections to file. I know it is a too general request, but it would be nice if this function is faster.
In my particular case, I obtain these connections in a previous step using GetConnections(). I guess (hence I am probably wrong) that DumpLayerConnections() calls GetConnections(). If I am right, I believe it could be an interesting enhancement to implement a DumpLayerConnections() function with input parameters that are a set of connections, that is, something like:
myconn = nest.GetConnections(layer1, layer2, ...) nest.DumpLayerConnections(myconn)
Thanks a lot in advance!
Xavier _______________________________________________ NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
Hi Hans,
I have taken a look at the cpp code implementing DumpLayerConnections(). I have seen that, for every pre-synaptic neuron, it looks for the corresponding connections with all the post-sytnaptic neurons. Would it be faster to do it the other way around? That is, for every post-synaptic, looking for the corresponding pre-synaptic neurons. I was thinking about it because GetConnections() find connections on every MPI rank for every post-synaptic neuron.
If you believe it could bring some speedup, I could try to modify the cpp code.
Thanks a lot in advance,
Xavier
Dear Xavier,
I think the most efficient way would be to drop the entires “src_iter” loop and simply place the entire source node collection in conn_filter, just as the target collection. Then, one would only need a single GetConnections() call and would avoid a lot of duplicate parsing of the connection tables.
The one downside with that approach is that one may need quite some memory to store the connectome returned by GetConnections(). If we assume a layer size of 10000 neurons and a connectivity of 10% within the layer, we get 10 million connections. Each of those requires 40B in the representation returned by GetConnections(), so 400 MB in total. But this is manageable in most cases. And if the simulation is distributed on say 8 MPI ranks, it would only be 50 MB per rank, and only until DumpLayerConnections completes.
It would be nice if you could try to implement this. Could you try to use modern C++-style loops ( for ( auto& conn : …) )?
The proper solution to avoid the memory overhead would be for GetConnections not to return the actual collection of connections, but actually a iterator over it so that we can avoid the memory overhead entirely. But that would be a larger change to the code.
Best, Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science Faculty of Science and Technology Norwegian University of Life Sciences PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560 Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no Home http://arken.nmbu.no/~plesser
From: Xavier Otazu xotazu@cvc.uab.cat Date: Saturday, 16 March 2024 at 12:35 To: users@nest-simulator.org users@nest-simulator.org Subject: [NEST Users] Re: DumpLayerConnections() enhancement request [Some people who received this message don't often get email from xotazu@cvc.uab.cat. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
Hi Hans,
I have taken a look at the cpp code implementing DumpLayerConnections(). I have seen that, for every pre-synaptic neuron, it looks for the corresponding connections with all the post-sytnaptic neurons. Would it be faster to do it the other way around? That is, for every post-synaptic, looking for the corresponding pre-synaptic neurons. I was thinking about it because GetConnections() find connections on every MPI rank for every post-synaptic neuron.
If you believe it could bring some speedup, I could try to modify the cpp code.
Thanks a lot in advance,
Xavier _______________________________________________ NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
Hans,
I have experience with C++, but not with this modern auto reference loops (although taking a quick look it doesn't look like a very difficult concept). I can try to implement your approach. Will tell you in the next days.
Xavier
Hi Xavier,
Great! Don’t hesitate to get in touch if you have any questions. BTW, a nice compact book on modern C++ is Stroustrup’s A tour of C++ (https://stroustrup.com/Tour.html).
Best, Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science Faculty of Science and Technology Norwegian University of Life Sciences PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560 Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no Home http://arken.nmbu.no/~plesser
From: Xavier Otazu xotazu@cvc.uab.cat Date: Sunday, 17 March 2024 at 23:45 To: users@nest-simulator.org users@nest-simulator.org Subject: [NEST Users] Re: DumpLayerConnections() enhancement request [Du mottar ikke ofte e-post fra xotazu@cvc.uab.cat. Finn ut hvorfor dette er viktig p? https://aka.ms/LearnAboutSenderIdentification ]
Hans,
I have experience with C++, but not with this modern auto reference loops (although taking a quick look it doesn't look like a very difficult concept). I can try to implement your approach. Will tell you in the next days.
Xavier _______________________________________________ NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
Hans,
I know this book. I read it many years ago :-) ... but I do not remember reading about auto references on loops (probably I skipped it).
Xavier
Hans Ekkehard Plesser wrote:
Hi Xavier,
Great! Don’t hesitate to get in touch if you have any questions. BTW, a nice compact book on modern C++ is Stroustrup’s A tour of C++ (https://stroustrup.com/Tour.html).
Best, Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science Faculty of Science and Technology Norwegian University of Life Sciences PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560 Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no Home http://arken.nmbu.no/~plesser
From: Xavier Otazu <xotazu(a)cvc.uab.cat> Date: Sunday, 17 March 2024 at 23:45 To: users(a)nest-simulator.org <users(a)nest-simulator.org> Subject: [NEST Users] Re: DumpLayerConnections() enhancement request [Du mottar ikke ofte e-post fra xotazu(a)cvc.uab.cat. Finn ut hvorfor dette er viktig p? https://aka.ms/LearnAboutSenderIdentification ]
Hans,
I have experience with C++, but not with this modern auto reference loops (although taking a quick look it doesn't look like a very difficult concept). I can try to implement your approach. Will tell you in the next days.
Xavier _______________________________________________ NEST Users mailing list -- users(a)nest-simulator.org To unsubscribe send an email to users-leave(a)nest-simulator.org
Hans,
I have modified the dump_connections() method. For a particular test code, with the "old" code it took 1m13s. With the "new" code it takes 14 seconds. I checked results are exactly the same.
In my implementation, I have to look for the source node position iterating (for every source layer node) over the src_vec (that contains all the source node positions). May be there could be an even more efficient way to find it?
This is my new code. I have not deleted the previous code (I have commented it).
template < int D > void Layer< D >::dump_connections( std::ostream& out, NodeCollectionPTR node_collection, AbstractLayerPTR target_layer, const Token& syn_model ) { std::vector< std::pair< Position< D >, size_t > >* src_vec = get_global_positions_vector( node_collection );
// Dictionary with parameters for get_connections() DictionaryDatum conn_filter( new Dictionary ); def( conn_filter, names::synapse_model, syn_model ); def( conn_filter, names::target, NodeCollectionDatum( target_layer->get_node_collection() ) );
// Avoid setting up new array for each iteration of the loop // std::vector< size_t > source_array( 1 );
// for ( typename std::vector< std::pair< Position< D >, size_t > >::iterator src_iter = src_vec->begin(); // src_iter != src_vec->end(); // ++src_iter ) // {
// const size_t source_node_id = src_iter->second; // const Position< D > source_pos = src_iter->first;
// source_array[ 0 ] = source_node_id; // def( conn_filter, names::source, NodeCollectionDatum( NodeCollection::create( source_array ) ) ); def( conn_filter, names::source, NodeCollectionDatum( node_collection ) ); ArrayDatum connectome = kernel().connection_manager.get_connections( conn_filter );
// Print information about all local connections for current source for ( size_t i = 0; i < connectome.size(); ++i ) { ConnectionDatum con_id = getValue< ConnectionDatum >( connectome.get( i ) ); const size_t source_node_id = con_id.get_source_node_id();
// Search source_pos for source node Position< D > source_pos = src_vec->begin()->first;
for ( typename std::vector< std::pair< Position< D >, size_t > >::iterator src_iter = src_vec->begin(); src_iter != src_vec->end() && source_node_id!=src_iter->second; ++src_iter, source_pos = src_iter->first);
// DictionaryDatum result_dict = kernel().connection_manager.get_synapse_status( con_id.get_source_node_id(), DictionaryDatum result_dict = kernel().connection_manager.get_synapse_status( source_node_id, con_id.get_target_node_id(), con_id.get_target_thread(), con_id.get_synapse_model_id(), con_id.get_port() );
long target_node_id = getValue< long >( result_dict, names::target ); double weight = getValue< double >( result_dict, names::weight ); double delay = getValue< double >( result_dict, names::delay );
// Print source, target, weight, delay, rports out << source_node_id << ' ' << target_node_id << ' ' << weight << ' ' << delay;
Layer< D >* tgt_layer = dynamic_cast< Layer< D >* >( target_layer.get() );
out << ' '; const long tnode_lid = tgt_layer->node_collection_->get_lid( target_node_id ); assert( tnode_lid >= 0 ); tgt_layer->compute_displacement( source_pos, tnode_lid ).print( out ); out << '\n'; } // } }
Hans Ekkehard Plesser wrote:
Hi Xavier,
Great! Don’t hesitate to get in touch if you have any questions. BTW, a nice compact book on modern C++ is Stroustrup’s A tour of C++ (https://stroustrup.com/Tour.html).
Best, Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science Faculty of Science and Technology Norwegian University of Life Sciences PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560 Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no Home http://arken.nmbu.no/~plesser
From: Xavier Otazu <xotazu(a)cvc.uab.cat> Date: Sunday, 17 March 2024 at 23:45 To: users(a)nest-simulator.org <users(a)nest-simulator.org> Subject: [NEST Users] Re: DumpLayerConnections() enhancement request [Du mottar ikke ofte e-post fra xotazu(a)cvc.uab.cat. Finn ut hvorfor dette er viktig p? https://aka.ms/LearnAboutSenderIdentification ]
Hans,
I have experience with C++, but not with this modern auto reference loops (although taking a quick look it doesn't look like a very difficult concept). I can try to implement your approach. Will tell you in the next days.
Xavier _______________________________________________ NEST Users mailing list -- users(a)nest-simulator.org To unsubscribe send an email to users-leave(a)nest-simulator.org
Hans,
Here I send an even more improved version. It onlys look for node position if the node is different from the one of the previous iteration. It also considers (I do not know if it is safe -I included an sssert-) that source node positions (src_vec) and connectome are ordered by source node. It completely avoids the use of the for loop on the source node positions.
The final execution time now is 9 seconds.
Instead of sending more messages in this list, do you prefer I send all this information through the github issue?
Xavier
template < int D > void Layer< D >::dump_connections( std::ostream& out, NodeCollectionPTR node_collection, AbstractLayerPTR target_layer, const Token& syn_model ) { std::vector< std::pair< Position< D >, size_t > >* src_vec = get_global_positions_vector( node_collection );
// Dictionary with parameters for get_connections() DictionaryDatum conn_filter( new Dictionary ); def( conn_filter, names::synapse_model, syn_model ); def( conn_filter, names::target, NodeCollectionDatum( target_layer->get_node_collection() ) );
// Avoid setting up new array for each iteration of the loop // std::vector< size_t > source_array( 1 );
// for ( typename std::vector< std::pair< Position< D >, size_t > >::iterator src_iter = src_vec->begin(); // src_iter != src_vec->end(); // ++src_iter ) // {
// const size_t source_node_id = src_iter->second; // const Position< D > source_pos = src_iter->first;
// source_array[ 0 ] = source_node_id; // def( conn_filter, names::source, NodeCollectionDatum( NodeCollection::create( source_array ) ) ); def( conn_filter, names::source, NodeCollectionDatum( node_collection ) ); ArrayDatum connectome = kernel().connection_manager.get_connections( conn_filter );
size_t previous_source_node_id = getValue< ConnectionDatum >( connectome.get( 0 ) ).get_source_node_id(); typename std::vector< std::pair< Position< D >, size_t > >::iterator src_iter = src_vec->begin(); Position< D > source_pos = src_iter->first;
// Print information about all local connections for current source for ( size_t i = 0; i < connectome.size(); ++i ) { ConnectionDatum con_id = getValue< ConnectionDatum >( connectome.get( i ) ); const size_t source_node_id = con_id.get_source_node_id();
// Search source_pos for source node only if it is a different node if(source_node_id != previous_source_node_id) { // source_pos = src_vec->begin()->first; // // for ( typename std::vector< std::pair< Position< D >, size_t > >::iterator src_iter = src_vec->begin(); // src_iter != src_vec->end() && source_node_id!=src_iter->second; // ++src_iter, source_pos = src_iter->first);
assert((src_iter++)!=src_vec->end()); source_pos = src_iter->first;
previous_source_node_id = source_node_id; }
// DictionaryDatum result_dict = kernel().connection_manager.get_synapse_status( con_id.get_source_node_id(), DictionaryDatum result_dict = kernel().connection_manager.get_synapse_status( source_node_id, con_id.get_target_node_id(), con_id.get_target_thread(), con_id.get_synapse_model_id(), con_id.get_port() );
long target_node_id = getValue< long >( result_dict, names::target ); double weight = getValue< double >( result_dict, names::weight ); double delay = getValue< double >( result_dict, names::delay );
// Print source, target, weight, delay, rports out << source_node_id << ' ' << target_node_id << ' ' << weight << ' ' << delay;
Layer< D >* tgt_layer = dynamic_cast< Layer< D >* >( target_layer.get() );
out << ' '; const long tnode_lid = tgt_layer->node_collection_->get_lid( target_node_id ); assert( tnode_lid >= 0 ); tgt_layer->compute_displacement( source_pos, tnode_lid ).print( out ); out << '\n'; } // } }
________________________________ From: Xavier Otazu xotazu@cvc.uab.cat Sent: Monday, March 18, 2024 12:46 PM To: users@nest-simulator.org users@nest-simulator.org Subject: [NEST Users] Re: DumpLayerConnections() enhancement request
Hans,
I have modified the dump_connections() method. For a particular test code, with the "old" code it took 1m13s. With the "new" code it takes 14 seconds. I checked results are exactly the same.
In my implementation, I have to look for the source node position iterating (for every source layer node) over the src_vec (that contains all the source node positions). May be there could be an even more efficient way to find it?
This is my new code. I have not deleted the previous code (I have commented it).
template < int D > void Layer< D >::dump_connections( std::ostream& out, NodeCollectionPTR node_collection, AbstractLayerPTR target_layer, const Token& syn_model ) { std::vector< std::pair< Position< D >, size_t > >* src_vec = get_global_positions_vector( node_collection );
// Dictionary with parameters for get_connections() DictionaryDatum conn_filter( new Dictionary ); def( conn_filter, names::synapse_model, syn_model ); def( conn_filter, names::target, NodeCollectionDatum( target_layer->get_node_collection() ) );
// Avoid setting up new array for each iteration of the loop // std::vector< size_t > source_array( 1 );
// for ( typename std::vector< std::pair< Position< D >, size_t > >::iterator src_iter = src_vec->begin(); // src_iter != src_vec->end(); // ++src_iter ) // {
// const size_t source_node_id = src_iter->second; // const Position< D > source_pos = src_iter->first;
// source_array[ 0 ] = source_node_id; // def( conn_filter, names::source, NodeCollectionDatum( NodeCollection::create( source_array ) ) ); def( conn_filter, names::source, NodeCollectionDatum( node_collection ) ); ArrayDatum connectome = kernel().connection_manager.get_connections( conn_filter );
// Print information about all local connections for current source for ( size_t i = 0; i < connectome.size(); ++i ) { ConnectionDatum con_id = getValue< ConnectionDatum >( connectome.get( i ) ); const size_t source_node_id = con_id.get_source_node_id();
// Search source_pos for source node Position< D > source_pos = src_vec->begin()->first;
for ( typename std::vector< std::pair< Position< D >, size_t > >::iterator src_iter = src_vec->begin(); src_iter != src_vec->end() && source_node_id!=src_iter->second; ++src_iter, source_pos = src_iter->first);
// DictionaryDatum result_dict = kernel().connection_manager.get_synapse_status( con_id.get_source_node_id(), DictionaryDatum result_dict = kernel().connection_manager.get_synapse_status( source_node_id, con_id.get_target_node_id(), con_id.get_target_thread(), con_id.get_synapse_model_id(), con_id.get_port() );
long target_node_id = getValue< long >( result_dict, names::target ); double weight = getValue< double >( result_dict, names::weight ); double delay = getValue< double >( result_dict, names::delay );
// Print source, target, weight, delay, rports out << source_node_id << ' ' << target_node_id << ' ' << weight << ' ' << delay;
Layer< D >* tgt_layer = dynamic_cast< Layer< D >* >( target_layer.get() );
out << ' '; const long tnode_lid = tgt_layer->node_collection_->get_lid( target_node_id ); assert( tnode_lid >= 0 ); tgt_layer->compute_displacement( source_pos, tnode_lid ).print( out ); out << '\n'; } // } }
Hans Ekkehard Plesser wrote:
Hi Xavier,
Great! Don’t hesitate to get in touch if you have any questions. BTW, a nice compact book on modern C++ is Stroustrup’s A tour of C++ (https://stroustrup.com/Tour.html).
Best, Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science Faculty of Science and Technology Norwegian University of Life Sciences PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560 Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no Home http://arken.nmbu.no/~plesser
From: Xavier Otazu <xotazu(a)cvc.uab.cat> Date: Sunday, 17 March 2024 at 23:45 To: users(a)nest-simulator.org <users(a)nest-simulator.org> Subject: [NEST Users] Re: DumpLayerConnections() enhancement request [Du mottar ikke ofte e-post fra xotazu(a)cvc.uab.cat. Finn ut hvorfor dette er viktig p? https://aka.ms/LearnAboutSenderIdentification ]
Hans,
I have experience with C++, but not with this modern auto reference loops (although taking a quick look it doesn't look like a very difficult concept). I can try to implement your approach. Will tell you in the next days.
Xavier _______________________________________________ NEST Users mailing list -- users(a)nest-simulator.org To unsubscribe send an email to users-leave(a)nest-simulator.org
_______________________________________________ NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
Computer Vision Centerhttp://www.cvc.uab.cat CONFIDENTIALITY WARNINGhttp://www.cvc.uab.es/?page_id=7475
Dear Xavier,
That’s great news!
Could you create a pull request on Github? And could you mail me via direct email a NEST contributor agreement (https://nest-simulator.readthedocs.io/en/stable/_downloads/9b65adbdacba6bfed...
Best, Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science Faculty of Science and Technology Norwegian University of Life Sciences PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560 Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no Home http://arken.nmbu.no/~plesser
From: Xavier Otazu Porter xotazu@cvc.uab.cat Date: Monday, 18 March 2024 at 14:30 To: NEST User Mailing List users@nest-simulator.org Subject: [NEST Users] Re: DumpLayerConnections() enhancement request Hans,
Here I send an even more improved version. It onlys look for node position if the node is different from the one of the previous iteration. It also considers (I do not know if it is safe -I included an sssert-) that source node positions (src_vec) and connectome are ordered by source node. It completely avoids the use of the for loop on the source node positions.
The final execution time now is 9 seconds.
Instead of sending more messages in this list, do you prefer I send all this information through the github issue?
Xavier
template < int D > void Layer< D >::dump_connections( std::ostream& out, NodeCollectionPTR node_collection, AbstractLayerPTR target_layer, const Token& syn_model ) { std::vector< std::pair< Position< D >, size_t > >* src_vec = get_global_positions_vector( node_collection );
// Dictionary with parameters for get_connections() DictionaryDatum conn_filter( new Dictionary ); def( conn_filter, names::synapse_model, syn_model ); def( conn_filter, names::target, NodeCollectionDatum( target_layer->get_node_collection() ) );
// Avoid setting up new array for each iteration of the loop // std::vector< size_t > source_array( 1 );
// for ( typename std::vector< std::pair< Position< D >, size_t > >::iterator src_iter = src_vec->begin(); // src_iter != src_vec->end(); // ++src_iter ) // {
// const size_t source_node_id = src_iter->second; // const Position< D > source_pos = src_iter->first;
// source_array[ 0 ] = source_node_id; // def( conn_filter, names::source, NodeCollectionDatum( NodeCollection::create( source_array ) ) ); def( conn_filter, names::source, NodeCollectionDatum( node_collection ) ); ArrayDatum connectome = kernel().connection_manager.get_connections( conn_filter );
size_t previous_source_node_id = getValue< ConnectionDatum >( connectome.get( 0 ) ).get_source_node_id(); typename std::vector< std::pair< Position< D >, size_t > >::iterator src_iter = src_vec->begin(); Position< D > source_pos = src_iter->first;
// Print information about all local connections for current source for ( size_t i = 0; i < connectome.size(); ++i ) { ConnectionDatum con_id = getValue< ConnectionDatum >( connectome.get( i ) ); const size_t source_node_id = con_id.get_source_node_id();
// Search source_pos for source node only if it is a different node if(source_node_id != previous_source_node_id) { // source_pos = src_vec->begin()->first; // // for ( typename std::vector< std::pair< Position< D >, size_t > >::iterator src_iter = src_vec->begin(); // src_iter != src_vec->end() && source_node_id!=src_iter->second; // ++src_iter, source_pos = src_iter->first);
assert((src_iter++)!=src_vec->end()); source_pos = src_iter->first;
previous_source_node_id = source_node_id; }
// DictionaryDatum result_dict = kernel().connection_manager.get_synapse_status( con_id.get_source_node_id(), DictionaryDatum result_dict = kernel().connection_manager.get_synapse_status( source_node_id, con_id.get_target_node_id(), con_id.get_target_thread(), con_id.get_synapse_model_id(), con_id.get_port() );
long target_node_id = getValue< long >( result_dict, names::target ); double weight = getValue< double >( result_dict, names::weight ); double delay = getValue< double >( result_dict, names::delay );
// Print source, target, weight, delay, rports out << source_node_id << ' ' << target_node_id << ' ' << weight << ' ' << delay;
Layer< D >* tgt_layer = dynamic_cast< Layer< D >* >( target_layer.get() );
out << ' '; const long tnode_lid = tgt_layer->node_collection_->get_lid( target_node_id ); assert( tnode_lid >= 0 ); tgt_layer->compute_displacement( source_pos, tnode_lid ).print( out ); out << '\n'; } // } }
________________________________ From: Xavier Otazu xotazu@cvc.uab.cat Sent: Monday, March 18, 2024 12:46 PM To: users@nest-simulator.org users@nest-simulator.org Subject: [NEST Users] Re: DumpLayerConnections() enhancement request
Hans,
I have modified the dump_connections() method. For a particular test code, with the "old" code it took 1m13s. With the "new" code it takes 14 seconds. I checked results are exactly the same.
In my implementation, I have to look for the source node position iterating (for every source layer node) over the src_vec (that contains all the source node positions). May be there could be an even more efficient way to find it?
This is my new code. I have not deleted the previous code (I have commented it).
template < int D > void Layer< D >::dump_connections( std::ostream& out, NodeCollectionPTR node_collection, AbstractLayerPTR target_layer, const Token& syn_model ) { std::vector< std::pair< Position< D >, size_t > >* src_vec = get_global_positions_vector( node_collection );
// Dictionary with parameters for get_connections() DictionaryDatum conn_filter( new Dictionary ); def( conn_filter, names::synapse_model, syn_model ); def( conn_filter, names::target, NodeCollectionDatum( target_layer->get_node_collection() ) );
// Avoid setting up new array for each iteration of the loop // std::vector< size_t > source_array( 1 );
// for ( typename std::vector< std::pair< Position< D >, size_t > >::iterator src_iter = src_vec->begin(); // src_iter != src_vec->end(); // ++src_iter ) // {
// const size_t source_node_id = src_iter->second; // const Position< D > source_pos = src_iter->first;
// source_array[ 0 ] = source_node_id; // def( conn_filter, names::source, NodeCollectionDatum( NodeCollection::create( source_array ) ) ); def( conn_filter, names::source, NodeCollectionDatum( node_collection ) ); ArrayDatum connectome = kernel().connection_manager.get_connections( conn_filter );
// Print information about all local connections for current source for ( size_t i = 0; i < connectome.size(); ++i ) { ConnectionDatum con_id = getValue< ConnectionDatum >( connectome.get( i ) ); const size_t source_node_id = con_id.get_source_node_id();
// Search source_pos for source node Position< D > source_pos = src_vec->begin()->first;
for ( typename std::vector< std::pair< Position< D >, size_t > >::iterator src_iter = src_vec->begin(); src_iter != src_vec->end() && source_node_id!=src_iter->second; ++src_iter, source_pos = src_iter->first);
// DictionaryDatum result_dict = kernel().connection_manager.get_synapse_status( con_id.get_source_node_id(), DictionaryDatum result_dict = kernel().connection_manager.get_synapse_status( source_node_id, con_id.get_target_node_id(), con_id.get_target_thread(), con_id.get_synapse_model_id(), con_id.get_port() );
long target_node_id = getValue< long >( result_dict, names::target ); double weight = getValue< double >( result_dict, names::weight ); double delay = getValue< double >( result_dict, names::delay );
// Print source, target, weight, delay, rports out << source_node_id << ' ' << target_node_id << ' ' << weight << ' ' << delay;
Layer< D >* tgt_layer = dynamic_cast< Layer< D >* >( target_layer.get() );
out << ' '; const long tnode_lid = tgt_layer->node_collection_->get_lid( target_node_id ); assert( tnode_lid >= 0 ); tgt_layer->compute_displacement( source_pos, tnode_lid ).print( out ); out << '\n'; } // } }
Hans Ekkehard Plesser wrote:
Hi Xavier,
Great! Don’t hesitate to get in touch if you have any questions. BTW, a nice compact book on modern C++ is Stroustrup’s A tour of C++ (https://stroustrup.com/Tour.html).
Best, Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science Faculty of Science and Technology Norwegian University of Life Sciences PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560 Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no Home http://arken.nmbu.no/~plesser
From: Xavier Otazu <xotazu(a)cvc.uab.cat> Date: Sunday, 17 March 2024 at 23:45 To: users(a)nest-simulator.org <users(a)nest-simulator.org> Subject: [NEST Users] Re: DumpLayerConnections() enhancement request [Du mottar ikke ofte e-post fra xotazu(a)cvc.uab.cat. Finn ut hvorfor dette er viktig p? https://aka.ms/LearnAboutSenderIdentification ]
Hans,
I have experience with C++, but not with this modern auto reference loops (although taking a quick look it doesn't look like a very difficult concept). I can try to implement your approach. Will tell you in the next days.
Xavier _______________________________________________ NEST Users mailing list -- users(a)nest-simulator.org To unsubscribe send an email to users-leave(a)nest-simulator.org
_______________________________________________ NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
Computer Vision Centerhttp://www.cvc.uab.cat CONFIDENTIALITY WARNINGhttp://www.cvc.uab.es/?page_id=7475
-
Hans Ekkehard Plesser -
Xavier Otazu -
Xavier Otazu Porter