Results of simulation using `fixed_indegree` seem to depend on `total_num_virtual_procs`
Hello everyone,
I am writing to the NEST mailing list as I have encountered an issue in my simulation script, and I would greatly appreciate your insights.
The problem I am facing is that the membrane potential time course for a simulation with a fixed indegree seems to be dependent on the number of virtual processes (`total_num_virtual_procs`). Interestingly, this behavior does not occur for simulations with all-to-all connectivity.
The simulation involves a network with the following architecture: 5 `spike_generator` nodes connected to 5 `parrot_neuron` nodes with (all-to-all), which are further connected to 5 `iaf_psc_delta` neurons (all-to-all). The connections between the `iaf_psc_delta` neurons have an `fixed_indegree` of 2. A voltmeter is used to monitor the membrane potentials of these 5 neurons, and the simulation runs for 100 ms.
The issue arises when I vary the number of virtual processes (`total_num_virtual_procs`). I have tested the values 1, 2, and 4, and I have noticed that the plot of the membrane potentials at the end of the simulation differs for each `total_num_virtual_procs` value. Additionally, when I print the table of connections, I observe that the weights between the `iaf_psc_delta` neurons are also influenced by the number of virtual processes.
I have attached the simulation script and three images showing the plot for each `total_num_virtual_procs` value. The experiments were conducted on my personal Ubuntu machine.
The version of NEST that I am using is the latest at the time of writing this message, with commit `b9514cce789fb0b2c0dfbd9d0894d3986f743021`, authored by Abigail Morrison morrison@fz-juelich.de on Tue Jul 18 23:46:22 2023 +0200.
I would be grateful for any suggestions or explanations regarding this issue, as it hinders the reproducibility of my results.
Thank you in advance for your valuable help and support.
Best regards, Jesus Espinoza.
-- Jesús Espinoza Bergische Universität Wuppertal Fakultät für Mathematik und Naturwissenschaften Wissenschaftliches Rechnen und Hochleistungsrechnen Gaußstraße 20 42119 Wuppertal Germany E-Mail: espinozavalverd{at}uni-wuppertal.de
Dear Jesús,
The NEST Simulator documentation on parallel computing has a section regarding reproducibility:
https://nest-simulator.readthedocs.io/en/stable/hpc/parallel_computing.html
"To achieve the same simulation results even when using different parallelization strategies, the number of virtual processes has to be kept constant. A simulation with a specific number of virtual processes will always yield the same results, no matter how they are distributed over threads and processes, given that the seeds for the random number generators of the different virtual processes are the same (see Random numbers in NEST)."
Each virtual process has its own instance (and unique seed) in the random number generator. This will result in different typologies and random current/spikes generated. What you are describing is expected behavior.
Sincerely,
Wouter
On 26-Jul-23 12:10, Jesus Espinoza wrote:
Hello everyone,
I am writing to the NEST mailing list as I have encountered an issue in my simulation script, and I would greatly appreciate your insights.
The problem I am facing is that the membrane potential time course for a simulation with a fixed indegree seems to be dependent on the number of virtual processes (`total_num_virtual_procs`). Interestingly, this behavior does not occur for simulations with all-to-all connectivity.
The simulation involves a network with the following architecture: 5 `spike_generator` nodes connected to 5 `parrot_neuron` nodes with (all-to-all), which are further connected to 5 `iaf_psc_delta` neurons (all-to-all). The connections between the `iaf_psc_delta` neurons have an `fixed_indegree` of 2. A voltmeter is used to monitor the membrane potentials of these 5 neurons, and the simulation runs for 100 ms.
The issue arises when I vary the number of virtual processes (`total_num_virtual_procs`). I have tested the values 1, 2, and 4, and I have noticed that the plot of the membrane potentials at the end of the simulation differs for each `total_num_virtual_procs` value. Additionally, when I print the table of connections, I observe that the weights between the `iaf_psc_delta` neurons are also influenced by the number of virtual processes.
I have attached the simulation script and three images showing the plot for each `total_num_virtual_procs` value. The experiments were conducted on my personal Ubuntu machine.
The version of NEST that I am using is the latest at the time of writing this message, with commit `b9514cce789fb0b2c0dfbd9d0894d3986f743021`, authored by Abigail Morrison morrison@fz-juelich.de on Tue Jul 18 23:46:22 2023 +0200.
I would be grateful for any suggestions or explanations regarding this issue, as it hinders the reproducibility of my results.
Thank you in advance for your valuable help and support.
Best regards, Jesus Espinoza.
-- Jesús Espinoza Bergische Universität Wuppertal Fakultät für Mathematik und Naturwissenschaften Wissenschaftliches Rechnen und Hochleistungsrechnen Gaußstraße 20 42119 Wuppertal Germany E-Mail: espinozavalverd{at}uni-wuppertal.de
NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
Thank you for your kind response.
My primary concern revolves around weak scaling. I am specifically interested in understanding how the weak scaling results for a very large and sparse network (e.g. with a small `fixed_indegree`) would be affected by the emerging differences in sparsity patterns and overall dynamics due to an increasing number of virtual processes. My intuition suggests that these differences may "balance out," thereby making the scaling results reliable. However, I am curious to know if there exists any mathematical guarantee or theory that either supports or disproves this hypothesis. Any insights or references on this matter would be greatly appreciated.
Best, Jesus Espinoza.
Hello Jesus,
Several aspects come into play in when using simulations to explore the dynamics of network models.
First, any given simulation will result in one specific realisation of the model dynamics. If you run a simulation twice under identical circumstances (same random seed, same number of parallel processes), you must get identical results, otherwise something is wrong with the simulator.
If you change the random seed in your simulation, then you will get model dynamics that differ in detail in most cases (see below for exceptions). For networks with "boring" dynamics such as a Brunel network with asynchronous irregular activity, this variation will only be in details of spike trains and voltage traces. For models with more complex dynamics, e.g., switching between "up" and "down" states at larger, random, intervals, the differences can be larger. But if proper statistical measures are applied over long enough traces and sufficiently many realisation, consistent result shall be obtained.
As Wouter already pointed out, changing the number of virtual processes in a NEST simulation implicitly changes the random number sequences provided to specific neurons (and used to randomise connections), so even if the rng_seed set in NEST is the same, varying the number of virtual processes will yield different realisations.
Given that you use `spike_generator`s with fixed input spike trains and then make all-to-all connections throughout your network, there is no randomness left in your network (assuming you do not randomize connection weights or delays). Then, random numbers play no role and you get identical results for any number of VPs.
Overall, changing the number of VPs should only influence the time required to simulate your model, not the dynamics the model exhibits.
Best regards, Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser Head, Department of Data Science
Faculty of Science and Technology Norwegian University of Life Sciences PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560 Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no Home http://arken.nmbu.no/~plesser
From: espinozavalverd@uni-wuppertal.de espinozavalverd@uni-wuppertal.de Date: Wednesday, 26 July 2023 at 17:31 To: users@nest-simulator.org users@nest-simulator.org Subject: [NEST Users] Re: Results of simulation using `fixed_indegree` seem to depend on `total_num_virtual_procs` Sorry, I meant to say Strong scaling. _______________________________________________ NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
-
espinozavalverd＠uni-wuppertal.de -
Hans Ekkehard Plesser -
Jesus Espinoza
-
Wouter Klijn