Hi!
After using some PyNEST functions and modifying the C++ implementation of DumpLayerConnections(), I was thinking on a possible further improvement of this function (or creating a new function). But since I do not know the design and architecture of C++ nestkernel code, I prefer to ask.
In my code I normalize the presynaptic connections of every neuron. To do this, I originally do something like
for i in range(len(target_layer)) conn[i] = nest.GetConnections(source_layer, target_layer, synapse_model)
normalize(conn)
Since I am using MPI and the loop iterates over all the neurons, I modified the previous code to
local_nodes = nest.GetLocalNodeCollection(target_layer) local_nodes = nest.NodeCollection(local_nodes)
for i in range(len(local_nodes)) conn[i] = nest.GetConnections(source_layer, local_nodes, synapse_model)
normalize(conn)
This last code is faster that the previous one (I guess that local_nodes variable is local to every MPI process and, as a consequence, GetConnections() is more efficient because it only works with local nodes instead of with all of them).
Using the idea of the GetLocalNodeCollection(), I was thinking it could be used into the C++ implementation of DumpLayerConnections(). Presently, this function obtain all the connections between source and target layers. Could it be possible to call the equivalent C++ implementation of GetLocalCollection()? I understand that the problem is that, since DumpLayerConnections() needs the spatial information of the layers, the node collections obtained with GetLocalNodeCollection() (I guess similarly its C++ implementation) does not have this spatial information.
As a second possibility, I was thinking on adding a new method (GetSpatialInformation()) and enhancing DumpLayerConnections() (or add a new function). The GetSpatialInformation() function could return the spatial information of a collection of nodes (the ones obtained by GetLocalNodeCollection() ). The DumpLayerConnections() (or a new function) could be enhanced by changing the target layer parameter to a pair of parameters that contain the local nodes collection and their spatial information. Something like DumpLaterConnections(source_layer, local_nodes_collection, spatial_information, synapse_model). This way DumpLayerConnections() will only used local nodes and would be much faster.
Sorry for the technical email.
Xavier
Dear Xavier,
Thank you for your detailed email—do not worry about writing technical here! Overall your thoughts are sound, but even better solutions should be feasible once I have fixed https://github.com/nest/nest-simulator/pull/3132.
BTW, you should not need the second line in
local_nodes = nest.GetLocalNodeCollection(target_layer) local_nodes = nest.NodeCollection(local_nodes)
since GetLocalNodeCollection should return a NodeCollection object.
Best, Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science Faculty of Science and Technology Norwegian University of Life Sciences PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560 Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no Home http://arken.nmbu.no/~plesser
From: Xavier Otazu xotazu@cvc.uab.cat Date: Monday, 8 April 2024 at 16:43 To: users@nest-simulator.org users@nest-simulator.org Subject: [NEST Users] Enhancement of C++ implementation of DumpLayerConnections() Hi!
After using some PyNEST functions and modifying the C++ implementation of DumpLayerConnections(), I was thinking on a possible further improvement of this function (or creating a new function). But since I do not know the design and architecture of C++ nestkernel code, I prefer to ask.
In my code I normalize the presynaptic connections of every neuron. To do this, I originally do something like
for i in range(len(target_layer)) conn[i] = nest.GetConnections(source_layer, target_layer, synapse_model)
normalize(conn)
Since I am using MPI and the loop iterates over all the neurons, I modified the previous code to
local_nodes = nest.GetLocalNodeCollection(target_layer) local_nodes = nest.NodeCollection(local_nodes)
for i in range(len(local_nodes)) conn[i] = nest.GetConnections(source_layer, local_nodes, synapse_model)
normalize(conn)
This last code is faster that the previous one (I guess that local_nodes variable is local to every MPI process and, as a consequence, GetConnections() is more efficient because it only works with local nodes instead of with all of them).
Using the idea of the GetLocalNodeCollection(), I was thinking it could be used into the C++ implementation of DumpLayerConnections(). Presently, this function obtain all the connections between source and target layers. Could it be possible to call the equivalent C++ implementation of GetLocalCollection()? I understand that the problem is that, since DumpLayerConnections() needs the spatial information of the layers, the node collections obtained with GetLocalNodeCollection() (I guess similarly its C++ implementation) does not have this spatial information.
As a second possibility, I was thinking on adding a new method (GetSpatialInformation()) and enhancing DumpLayerConnections() (or add a new function). The GetSpatialInformation() function could return the spatial information of a collection of nodes (the ones obtained by GetLocalNodeCollection() ). The DumpLayerConnections() (or a new function) could be enhanced by changing the target layer parameter to a pair of parameters that contain the local nodes collection and their spatial information. Something like DumpLaterConnections(source_layer, local_nodes_collection, spatial_information, synapse_model). This way DumpLayerConnections() will only used local nodes and would be much faster.
Sorry for the technical email.
Xavier _______________________________________________ NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
Hi Hans,
When you fixed that bug, tell me and I could take a look at a new possible solution (after you tell me what this better solution can be 🙂 ).
I tried not to use the second line, but an error occurred on GetConnections() telling me to use a NodeCollection as input parameter instead of a list. In fact, I see an example using GetLocalNodeCollection and the second line in the following python example code:https://nest-simulator.readthedocs.io/en/v3.3/auto_examples/hpc_benchmark.ht...
Anyway, in the next few days I am going to try again my code without that second line.
Thanks!
Xavier
________________________________ From: Hans Ekkehard Plesser hans.ekkehard.plesser@nmbu.no Sent: Tuesday, April 9, 2024 2:40 PM To: NEST User Mailing List users@nest-simulator.org Subject: [NEST Users] Re: Enhancement of C++ implementation of DumpLayerConnections()
Dear Xavier,
Thank you for your detailed email—do not worry about writing technical here! Overall your thoughts are sound, but even better solutions should be feasible once I have fixed https://github.com/nest/nest-simulator/pull/3132.
BTW, you should not need the second line in
local_nodes = nest.GetLocalNodeCollection(target_layer) local_nodes = nest.NodeCollection(local_nodes)
since GetLocalNodeCollection should return a NodeCollection object.
Best,
Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science
Faculty of Science and Technology
Norwegian University of Life Sciences
PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560
Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no
Home http://arken.nmbu.no/~plesser
From: Xavier Otazu xotazu@cvc.uab.cat Date: Monday, 8 April 2024 at 16:43 To: users@nest-simulator.org users@nest-simulator.org Subject: [NEST Users] Enhancement of C++ implementation of DumpLayerConnections()
Hi!
After using some PyNEST functions and modifying the C++ implementation of DumpLayerConnections(), I was thinking on a possible further improvement of this function (or creating a new function). But since I do not know the design and architecture of C++ nestkernel code, I prefer to ask.
In my code I normalize the presynaptic connections of every neuron. To do this, I originally do something like
for i in range(len(target_layer)) conn[i] = nest.GetConnections(source_layer, target_layer, synapse_model)
normalize(conn)
Since I am using MPI and the loop iterates over all the neurons, I modified the previous code to
local_nodes = nest.GetLocalNodeCollection(target_layer) local_nodes = nest.NodeCollection(local_nodes)
for i in range(len(local_nodes)) conn[i] = nest.GetConnections(source_layer, local_nodes, synapse_model)
normalize(conn)
This last code is faster that the previous one (I guess that local_nodes variable is local to every MPI process and, as a consequence, GetConnections() is more efficient because it only works with local nodes instead of with all of them).
Using the idea of the GetLocalNodeCollection(), I was thinking it could be used into the C++ implementation of DumpLayerConnections(). Presently, this function obtain all the connections between source and target layers. Could it be possible to call the equivalent C++ implementation of GetLocalCollection()? I understand that the problem is that, since DumpLayerConnections() needs the spatial information of the layers, the node collections obtained with GetLocalNodeCollection() (I guess similarly its C++ implementation) does not have this spatial information.
As a second possibility, I was thinking on adding a new method (GetSpatialInformation()) and enhancing DumpLayerConnections() (or add a new function). The GetSpatialInformation() function could return the spatial information of a collection of nodes (the ones obtained by GetLocalNodeCollection() ). The DumpLayerConnections() (or a new function) could be enhanced by changing the target layer parameter to a pair of parameters that contain the local nodes collection and their spatial information. Something like DumpLaterConnections(source_layer, local_nodes_collection, spatial_information, synapse_model). This way DumpLayerConnections() will only used local nodes and would be much faster.
Sorry for the technical email.
Xavier _______________________________________________ NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
Computer Vision Centerhttp://www.cvc.uab.cat CONFIDENTIALITY WARNINGhttp://www.cvc.uab.es/?page_id=7475
Dear Hans,
Thinking about this topic, I thought about another possible solution.
DumpLayerConnections() needs to know nodes ID, weight, delay and relative position. All this data (specially relative position -which is the slowest part of DumpLayerConnection()- ) is available (and used) at Connect() function ... and Connect() is MUCH faster than DumpLayerConnection().
Hence, could it be possible to implement DumpLayerConnection() using a C++ implementation very similar to Connect()?
Best,
Xavier
Xavier Otazu Porter wrote:
Hi Hans,
When you fixed that bug, tell me and I could take a look at a new possible solution (after you tell me what this better solution can be 🙂 ).
I tried not to use the second line, but an error occurred on GetConnections() telling me to use a NodeCollection as input parameter instead of a list. In fact, I see an example using GetLocalNodeCollection and the second line in the following python example code:https://nest-simulator.readthedocs.io/en/v3.3/auto_examples/hpc_benchmark.ht...
Anyway, in the next few days I am going to try again my code without that second line.
Thanks!
Xavier
From: Hans Ekkehard Plesser <hans.ekkehard.plesser(a)nmbu.no> Sent: Tuesday, April 9, 2024 2:40 PM To: NEST User Mailing List <users(a)nest-simulator.org> Subject: [NEST Users] Re: Enhancement of C++ implementation of DumpLayerConnections()
Dear Xavier,
Thank you for your detailed email—do not worry about writing technical here! Overall your thoughts are sound, but even better solutions should be feasible once I have fixed https://github.com/nest/nest-simulator/pull/3132.
BTW, you should not need the second line in
local_nodes = nest.GetLocalNodeCollection(target_layer) local_nodes = nest.NodeCollection(local_nodes)
since GetLocalNodeCollection should return a NodeCollection object.
Best,
Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science
Faculty of Science and Technology
Norwegian University of Life Sciences
PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560
Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no
Home http://arken.nmbu.no/~plesser
From: Xavier Otazu <xotazu(a)cvc.uab.cat> Date: Monday, 8 April 2024 at 16:43 To: users(a)nest-simulator.org <users(a)nest-simulator.org> Subject: [NEST Users] Enhancement of C++ implementation of DumpLayerConnections()
Hi!
After using some PyNEST functions and modifying the C++ implementation of DumpLayerConnections(), I was thinking on a possible further improvement of this function (or creating a new function). But since I do not know the design and architecture of C++ nestkernel code, I prefer to ask.
In my code I normalize the presynaptic connections of every neuron. To do this, I originally do something like
for i in range(len(target_layer)) conn[i] = nest.GetConnections(source_layer, target_layer, synapse_model)
normalize(conn)
Since I am using MPI and the loop iterates over all the neurons, I modified the previous code to
local_nodes = nest.GetLocalNodeCollection(target_layer) local_nodes = nest.NodeCollection(local_nodes)
for i in range(len(local_nodes)) conn[i] = nest.GetConnections(source_layer, local_nodes, synapse_model)
normalize(conn)
This last code is faster that the previous one (I guess that local_nodes variable is local to every MPI process and, as a consequence, GetConnections() is more efficient because it only works with local nodes instead of with all of them).
Using the idea of the GetLocalNodeCollection(), I was thinking it could be used into the C++ implementation of DumpLayerConnections(). Presently, this function obtain all the connections between source and target layers. Could it be possible to call the equivalent C++ implementation of GetLocalCollection()? I understand that the problem is that, since DumpLayerConnections() needs the spatial information of the layers, the node collections obtained with GetLocalNodeCollection() (I guess similarly its C++ implementation) does not have this spatial information.
As a second possibility, I was thinking on adding a new method (GetSpatialInformation()) and enhancing DumpLayerConnections() (or add a new function). The GetSpatialInformation() function could return the spatial information of a collection of nodes (the ones obtained by GetLocalNodeCollection() ). The DumpLayerConnections() (or a new function) could be enhanced by changing the target layer parameter to a pair of parameters that contain the local nodes collection and their spatial information. Something like DumpLaterConnections(source_layer, local_nodes_collection, spatial_information, synapse_model). This way DumpLayerConnections() will only used local nodes and would be much faster.
Sorry for the technical email.
Xavier _______________________________________________ NEST Users mailing list -- users(a)nest-simulator.org To unsubscribe send an email to users-leave(a)nest-simulator.org
Computer Vision Centerhttp://www.cvc.uab.cat CONFIDENTIALITY WARNINGhttp://www.cvc.uab.es/?page_id=7475
Hi Xavier,
I have picked up work again on DumpLayerConnections and sent you a PR on Github. With the changes I sent the code should be in good shape and if we could integrate all by the middle of next week, it can make it into NEST 3.8.
Please get in touch if you have any questions!
Best, Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science Faculty of Science and Technology Norwegian University of Life Sciences PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560 Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no Home http://arken.nmbu.no/~plesser
From: Xavier Otazu Porter xotazu@cvc.uab.cat Date: Tuesday, 9 April 2024 at 18:50 To: NEST User Mailing List users@nest-simulator.org Subject: [NEST Users] Re: Enhancement of C++ implementation of DumpLayerConnections() Hi Hans,
When you fixed that bug, tell me and I could take a look at a new possible solution (after you tell me what this better solution can be 🙂 ).
I tried not to use the second line, but an error occurred on GetConnections() telling me to use a NodeCollection as input parameter instead of a list. In fact, I see an example using GetLocalNodeCollection and the second line in the following python example code:https://nest-simulator.readthedocs.io/en/v3.3/auto_examples/hpc_benchmark.ht...
Anyway, in the next few days I am going to try again my code without that second line.
Thanks!
Xavier
________________________________ From: Hans Ekkehard Plesser hans.ekkehard.plesser@nmbu.no Sent: Tuesday, April 9, 2024 2:40 PM To: NEST User Mailing List users@nest-simulator.org Subject: [NEST Users] Re: Enhancement of C++ implementation of DumpLayerConnections()
Dear Xavier,
Thank you for your detailed email—do not worry about writing technical here! Overall your thoughts are sound, but even better solutions should be feasible once I have fixed https://github.com/nest/nest-simulator/pull/3132.
BTW, you should not need the second line in
local_nodes = nest.GetLocalNodeCollection(target_layer) local_nodes = nest.NodeCollection(local_nodes)
since GetLocalNodeCollection should return a NodeCollection object.
Best,
Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science
Faculty of Science and Technology
Norwegian University of Life Sciences
PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560
Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no
Home http://arken.nmbu.no/~plesser
From: Xavier Otazu xotazu@cvc.uab.cat Date: Monday, 8 April 2024 at 16:43 To: users@nest-simulator.org users@nest-simulator.org Subject: [NEST Users] Enhancement of C++ implementation of DumpLayerConnections()
Hi!
After using some PyNEST functions and modifying the C++ implementation of DumpLayerConnections(), I was thinking on a possible further improvement of this function (or creating a new function). But since I do not know the design and architecture of C++ nestkernel code, I prefer to ask.
In my code I normalize the presynaptic connections of every neuron. To do this, I originally do something like
for i in range(len(target_layer)) conn[i] = nest.GetConnections(source_layer, target_layer, synapse_model)
normalize(conn)
Since I am using MPI and the loop iterates over all the neurons, I modified the previous code to
local_nodes = nest.GetLocalNodeCollection(target_layer) local_nodes = nest.NodeCollection(local_nodes)
for i in range(len(local_nodes)) conn[i] = nest.GetConnections(source_layer, local_nodes, synapse_model)
normalize(conn)
This last code is faster that the previous one (I guess that local_nodes variable is local to every MPI process and, as a consequence, GetConnections() is more efficient because it only works with local nodes instead of with all of them).
Using the idea of the GetLocalNodeCollection(), I was thinking it could be used into the C++ implementation of DumpLayerConnections(). Presently, this function obtain all the connections between source and target layers. Could it be possible to call the equivalent C++ implementation of GetLocalCollection()? I understand that the problem is that, since DumpLayerConnections() needs the spatial information of the layers, the node collections obtained with GetLocalNodeCollection() (I guess similarly its C++ implementation) does not have this spatial information.
As a second possibility, I was thinking on adding a new method (GetSpatialInformation()) and enhancing DumpLayerConnections() (or add a new function). The GetSpatialInformation() function could return the spatial information of a collection of nodes (the ones obtained by GetLocalNodeCollection() ). The DumpLayerConnections() (or a new function) could be enhanced by changing the target layer parameter to a pair of parameters that contain the local nodes collection and their spatial information. Something like DumpLaterConnections(source_layer, local_nodes_collection, spatial_information, synapse_model). This way DumpLayerConnections() will only used local nodes and would be much faster.
Sorry for the technical email.
Xavier _______________________________________________ NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
Computer Vision Centerhttp://www.cvc.uab.cat CONFIDENTIALITY WARNINGhttp://www.cvc.uab.es/?page_id=7475
Hans,
I think I merged your PR with my code (I am not sure I did it correctly). Please, could you check it?
Xavier ________________________________ From: Hans Ekkehard Plesser hans.ekkehard.plesser@nmbu.no Sent: Friday, June 21, 2024 5:12 PM To: NEST User Mailing List users@nest-simulator.org Subject: [NEST Users] Re: Enhancement of C++ implementation of DumpLayerConnections()
Hi Xavier,
I have picked up work again on DumpLayerConnections and sent you a PR on Github. With the changes I sent the code should be in good shape and if we could integrate all by the middle of next week, it can make it into NEST 3.8.
Please get in touch if you have any questions!
Best,
Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science
Faculty of Science and Technology
Norwegian University of Life Sciences
PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560
Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no
Home http://arken.nmbu.no/~plesser
From: Xavier Otazu Porter xotazu@cvc.uab.cat Date: Tuesday, 9 April 2024 at 18:50 To: NEST User Mailing List users@nest-simulator.org Subject: [NEST Users] Re: Enhancement of C++ implementation of DumpLayerConnections()
Hi Hans,
When you fixed that bug, tell me and I could take a look at a new possible solution (after you tell me what this better solution can be ?? ).
I tried not to use the second line, but an error occurred on GetConnections() telling me to use a NodeCollection as input parameter instead of a list. In fact, I see an example using GetLocalNodeCollection and the second line in the following python example code:https://nest-simulator.readthedocs.io/en/v3.3/auto_examples/hpc_benchmark.ht...
Anyway, in the next few days I am going to try again my code without that second line.
Thanks!
Xavier
________________________________
From: Hans Ekkehard Plesser hans.ekkehard.plesser@nmbu.no Sent: Tuesday, April 9, 2024 2:40 PM To: NEST User Mailing List users@nest-simulator.org Subject: [NEST Users] Re: Enhancement of C++ implementation of DumpLayerConnections()
Dear Xavier,
Thank you for your detailed email—do not worry about writing technical here! Overall your thoughts are sound, but even better solutions should be feasible once I have fixed https://github.com/nest/nest-simulator/pull/3132.
BTW, you should not need the second line in
local_nodes = nest.GetLocalNodeCollection(target_layer) local_nodes = nest.NodeCollection(local_nodes)
since GetLocalNodeCollection should return a NodeCollection object.
Best,
Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science
Faculty of Science and Technology
Norwegian University of Life Sciences
PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560
Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no
Home http://arken.nmbu.no/~plesser
From: Xavier Otazu xotazu@cvc.uab.cat Date: Monday, 8 April 2024 at 16:43 To: users@nest-simulator.org users@nest-simulator.org Subject: [NEST Users] Enhancement of C++ implementation of DumpLayerConnections()
Hi!
After using some PyNEST functions and modifying the C++ implementation of DumpLayerConnections(), I was thinking on a possible further improvement of this function (or creating a new function). But since I do not know the design and architecture of C++ nestkernel code, I prefer to ask.
In my code I normalize the presynaptic connections of every neuron. To do this, I originally do something like
for i in range(len(target_layer)) conn[i] = nest.GetConnections(source_layer, target_layer, synapse_model)
normalize(conn)
Since I am using MPI and the loop iterates over all the neurons, I modified the previous code to
local_nodes = nest.GetLocalNodeCollection(target_layer) local_nodes = nest.NodeCollection(local_nodes)
for i in range(len(local_nodes)) conn[i] = nest.GetConnections(source_layer, local_nodes, synapse_model)
normalize(conn)
This last code is faster that the previous one (I guess that local_nodes variable is local to every MPI process and, as a consequence, GetConnections() is more efficient because it only works with local nodes instead of with all of them).
Using the idea of the GetLocalNodeCollection(), I was thinking it could be used into the C++ implementation of DumpLayerConnections(). Presently, this function obtain all the connections between source and target layers. Could it be possible to call the equivalent C++ implementation of GetLocalCollection()? I understand that the problem is that, since DumpLayerConnections() needs the spatial information of the layers, the node collections obtained with GetLocalNodeCollection() (I guess similarly its C++ implementation) does not have this spatial information.
As a second possibility, I was thinking on adding a new method (GetSpatialInformation()) and enhancing DumpLayerConnections() (or add a new function). The GetSpatialInformation() function could return the spatial information of a collection of nodes (the ones obtained by GetLocalNodeCollection() ). The DumpLayerConnections() (or a new function) could be enhanced by changing the target layer parameter to a pair of parameters that contain the local nodes collection and their spatial information. Something like DumpLaterConnections(source_layer, local_nodes_collection, spatial_information, synapse_model). This way DumpLayerConnections() will only used local nodes and would be much faster.
Sorry for the technical email.
Xavier _______________________________________________ NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
Computer Vision Centerhttp://www.cvc.uab.cat CONFIDENTIALITY WARNINGhttp://www.cvc.uab.es/?page_id=7475
Computer Vision Centerhttp://www.cvc.uab.cat CONFIDENTIALITY WARNINGhttp://www.cvc.uab.es/?page_id=7475
Hi Xavier,
Yes, all worked well, thank you!
Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science Faculty of Science and Technology Norwegian University of Life Sciences PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560 Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no Home http://arken.nmbu.no/~plesser
From: Xavier Otazu Porter xotazu@cvc.uab.cat Date: Friday, 21 June 2024 at 20:11 To: NEST User Mailing List users@nest-simulator.org Subject: [NEST Users] Re: Enhancement of C++ implementation of DumpLayerConnections() Hans,
I think I merged your PR with my code (I am not sure I did it correctly). Please, could you check it?
Xavier ________________________________ From: Hans Ekkehard Plesser hans.ekkehard.plesser@nmbu.no Sent: Friday, June 21, 2024 5:12 PM To: NEST User Mailing List users@nest-simulator.org Subject: [NEST Users] Re: Enhancement of C++ implementation of DumpLayerConnections()
Hi Xavier,
I have picked up work again on DumpLayerConnections and sent you a PR on Github. With the changes I sent the code should be in good shape and if we could integrate all by the middle of next week, it can make it into NEST 3.8.
Please get in touch if you have any questions!
Best,
Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science
Faculty of Science and Technology
Norwegian University of Life Sciences
PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560
Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no
Home http://arken.nmbu.no/~plesser
From: Xavier Otazu Porter xotazu@cvc.uab.cat Date: Tuesday, 9 April 2024 at 18:50 To: NEST User Mailing List users@nest-simulator.org Subject: [NEST Users] Re: Enhancement of C++ implementation of DumpLayerConnections()
Hi Hans,
When you fixed that bug, tell me and I could take a look at a new possible solution (after you tell me what this better solution can be 🙂 ).
I tried not to use the second line, but an error occurred on GetConnections() telling me to use a NodeCollection as input parameter instead of a list. In fact, I see an example using GetLocalNodeCollection and the second line in the following python example code:https://nest-simulator.readthedocs.io/en/v3.3/auto_examples/hpc_benchmark.ht...
Anyway, in the next few days I am going to try again my code without that second line.
Thanks!
Xavier
________________________________
From: Hans Ekkehard Plesser hans.ekkehard.plesser@nmbu.no Sent: Tuesday, April 9, 2024 2:40 PM To: NEST User Mailing List users@nest-simulator.org Subject: [NEST Users] Re: Enhancement of C++ implementation of DumpLayerConnections()
Dear Xavier,
Thank you for your detailed email—do not worry about writing technical here! Overall your thoughts are sound, but even better solutions should be feasible once I have fixed https://github.com/nest/nest-simulator/pull/3132.
BTW, you should not need the second line in
local_nodes = nest.GetLocalNodeCollection(target_layer) local_nodes = nest.NodeCollection(local_nodes)
since GetLocalNodeCollection should return a NodeCollection object.
Best,
Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science
Faculty of Science and Technology
Norwegian University of Life Sciences
PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560
Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no
Home http://arken.nmbu.no/~plesser
From: Xavier Otazu xotazu@cvc.uab.cat Date: Monday, 8 April 2024 at 16:43 To: users@nest-simulator.org users@nest-simulator.org Subject: [NEST Users] Enhancement of C++ implementation of DumpLayerConnections()
Hi!
After using some PyNEST functions and modifying the C++ implementation of DumpLayerConnections(), I was thinking on a possible further improvement of this function (or creating a new function). But since I do not know the design and architecture of C++ nestkernel code, I prefer to ask.
In my code I normalize the presynaptic connections of every neuron. To do this, I originally do something like
for i in range(len(target_layer)) conn[i] = nest.GetConnections(source_layer, target_layer, synapse_model)
normalize(conn)
Since I am using MPI and the loop iterates over all the neurons, I modified the previous code to
local_nodes = nest.GetLocalNodeCollection(target_layer) local_nodes = nest.NodeCollection(local_nodes)
for i in range(len(local_nodes)) conn[i] = nest.GetConnections(source_layer, local_nodes, synapse_model)
normalize(conn)
This last code is faster that the previous one (I guess that local_nodes variable is local to every MPI process and, as a consequence, GetConnections() is more efficient because it only works with local nodes instead of with all of them).
Using the idea of the GetLocalNodeCollection(), I was thinking it could be used into the C++ implementation of DumpLayerConnections(). Presently, this function obtain all the connections between source and target layers. Could it be possible to call the equivalent C++ implementation of GetLocalCollection()? I understand that the problem is that, since DumpLayerConnections() needs the spatial information of the layers, the node collections obtained with GetLocalNodeCollection() (I guess similarly its C++ implementation) does not have this spatial information.
As a second possibility, I was thinking on adding a new method (GetSpatialInformation()) and enhancing DumpLayerConnections() (or add a new function). The GetSpatialInformation() function could return the spatial information of a collection of nodes (the ones obtained by GetLocalNodeCollection() ). The DumpLayerConnections() (or a new function) could be enhanced by changing the target layer parameter to a pair of parameters that contain the local nodes collection and their spatial information. Something like DumpLaterConnections(source_layer, local_nodes_collection, spatial_information, synapse_model). This way DumpLayerConnections() will only used local nodes and would be much faster.
Sorry for the technical email.
Xavier _______________________________________________ NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
Computer Vision Centerhttp://www.cvc.uab.cat CONFIDENTIALITY WARNINGhttp://www.cvc.uab.es/?page_id=7475
Computer Vision Centerhttp://www.cvc.uab.cat CONFIDENTIALITY WARNINGhttp://www.cvc.uab.es/?page_id=7475
Dear Hans,
After modifying the C++ code of this method (layer_impl.h), I was thinking on another improvement. The bottleneck in this method is the get_global_positions_vector( node_collection ) call that obtain source layer nodes positions. It takes the most execution time.
Could it be possible to add a new Python function (and its corresponding C++ implementation) with an input parameter that is these nodes positions? User can initially obtain these positions using GetPosition() PyNEST function, and afterthat use the obtained position with DumpLayerConnections().
It could be a great improvement when user call DumpLayerConnections() several times for the same source nodes.
Best,
Xavier
-
Hans Ekkehard Plesser -
Xavier Otazu -
Xavier Otazu Porter