Hi everyone,

It seems like there is no straight-forward method of saving the network as a single object, so I think the simplest solution would be to save the relevant connection information and reconnect it accordingly when I reinitialise the network again. Since my network is pretty simple, this is not excessively demanding...the only real issue I've had is saving SLILiteral typed objects, but that's usually solved by turning it into a string.

Just in case someone else unfamiliar with nest finds themselves in this situation too, here are some details on the method to extract the information:

Ensure that the handles of all distinct populations of created neurons are stored in variables (e.g. pop1 = nest.Create("iaf_psc_alpha", 50) retains a list of node handles in pop1).
Using nest.GetConnections(args):

Without args; returns all connections
args = pop1; returns all connections coming from the specified nodes
args = pop1, pop2; returns connections between the two populations

Using nest.GetStatus(conn, keys="weight") to get the parameters of the connection.
Save the parameters, there are many numbers of ways e.g. pickle.dump(), numpy.save()

If you know the populations in the network, you can get the connections between them (in both directions) and obtain their parameters with GetStatus (available keys: ['delay', 'receptor', 'sizeof', 'source', 'synapse_model', 'target', 'weight']).

To recreate the network, I choose to first create all the same populations then reinstate the connections using nest.Connect(pop1, pop2, 'one_to_one', syn_spec), with syn_spec being {'delay': [d1, d2,...,dn], 'weight': [w1, w2,...,wn], etc.} reconstructed from the saved data.

Cheers,

Sock

On Wed, 5 Aug 2020 at 09:28, Wouter Klijn <w.klijn@fz-juelich.de> wrote:

Hello all,

I remember seeing a presentation some years back where a large
connectome was read directly into NEST from disk. I think it was stored
in HDF5. One of the main complications was the round robin neuron
placement complicating manners when a MPI rank size different then the
stored version was used: all synapses needed to be transported to a
different node. It was expensive.

Looking ahead, how some connectomes are generated in EBRAINS. Large
pre-processing pipelines with complicated touch detectection and 3d
placement of neurons (Markram Cell 2015) having this functionality might
come in handy in the future.

2cents.

Wouter

On 04-Aug-20 10:33, Benedikt S. Vogler wrote:
> Hey all,
>
> this sound similarly to my later suggestion in the reset issue:
> https://github.com/nest/nest-simulator/issues/1618#issuecomment-633972232
>
> I wanted to do some more research on how loading and saving is already
> currently supported in the library (PyNest and c++ back-end side) before
> working on the implementation. I would be interested in working on this
> implementation or gladly join anyone, who wants to work on this.
>
> Kind regards,
> Benedikt
>
>> Am 04.08.2020 um 10:15 schrieb LOW, SOCK CHING <sockching.low@upf.edu
>> <mailto:sockching.low@upf.edu>>:
>>
>> Hi Charl,
>>
>> Thanks for your prompt reply. What I mean to say is that I'd like to
>> save the network parameters, specifically the connections (i.e.
>> weights, delays etc.) and neuron parameters. There is no plasticity in
>> the network, as I am simply initialising a network of neurons with
>> probabilistic, random connections between each other (a liquid state
>> machine, in other words).
>>
>> My intention is to use the network in a real-time control
>> architecture, so I don't need to replicate /exactly /the same output
>> for a given input stimulation but I do need an output that is
>> comparable. Currently, I am using the states of the network to train a
>> classifier offline, but this classifier is fitted to the behaviour of
>> the network that it trains on. To be able to use the classifier again,
>> I will need to recreate that network, which leads me to the question
>> of "saving the entire network". Once I can do so, it becomes possible
>> to run simulations episode by episode rather than queueing all the
>> episodes into a single batch for simulation, which is critical for my
>> purpose. I hope that clarifies the question.
>>
>> Cheers,
>> Sock Ching
>>
>> On Tue, 4 Aug 2020 at 09:01, Charl Linssen <nest-users@turingbirds.com
>> <mailto:nest-users@turingbirds.com>> wrote:
>>
>> __
>> Dear Sock Ching,
>>
>> When you say "I would like to save the network", could you
>> elaborate on what exact part of the state you want to save and
>> recall? For instance, in case of simulating plasticity, all the
>> weights can be (re)set by calling SetStatus() on each connection
>> object. The same goes for neuronal state variables. There are a
>> few things that cannot be reset at the time of writing, such at
>> the random number generators, but this is only an issue if you try
>> to reproduce the same run *exactly*, multiple times in a row. The
>> biggest issue might be that spike buffers (spikes in transit)
>> cannot be reset. Potentially, you could work around this by
>> allowing a small "startup transient", i.e. ignoring whatever your
>> network produces in the first few hundred or so milliseconds after
>> starting simulation. If this startup transient is causing you
>> trouble, however, please feel invited to submit a feature request
>> on our GitHub repository for spike buffer reset functionality.
>> (Please provide as much technical detail as possible.)
>>
>> There is already some prior discussion on this topic at
>> https://github.com/nest/nest-simulator/issues/1618, where it was
>> decided that just having a global "ResetNetwork()" function was
>> not feasible, because it is not clear for the general case what
>> this function would do. So in case you open a new GitHub issue,
>> please try to specify as precisely as possible what parts of your
>> network state need to be (re)set.
>>
>> Hope this helps, please don't hesitate to share your further
>> experiences.
>>
>> Best regards,
>> Charl
>>
>>
>> On Mon, Aug 3, 2020, at 19:19, LOW, SOCK CHING wrote:
>>> Hi,
>>>
>>> I am new to pynest and am using it to implement a liquid state
>>> machine. It works marvelously for generating the network. My
>>> workflow is as follows:
>>> - Initialise kernel
>>> - Get stimulation episodes
>>> - Create neurons (including spike generators)
>>> - Connect neurons
>>> - nest.Simulate()
>>> - nest.GetStatus(recording_neurons) for readout at relevant
>>> timepoints
>>>
>>> I would like to save the network so I can run nest.Simulate() on
>>> other episodes in a real-time application, is there a
>>> straight-forward way to do so?
>>>
>>> I have tried pickling
>>>
>>> conn = nest.GetConnections()
>>>
>>> but to my knowledge that does not include a lot of details about
>>> the connection, like the weight. It also does not return where
>>> there is /no/ connections, which means simply using
>>>
>>> nest.SetConnections(conn)
>>>
>>> prior to nest.Simulate() will not work to reproduce the
>>> previously generated network even if I initialise with all the
>>> same variables. I have found the functions GetNetwork() and
>>> GetNodes() but I'm not sure how I can use them, or if they are
>>> even useful for what I need.
>>>
>>> Cheers,
>>> Sock Ching
>>> _______________________________________________
>>> NEST Users mailing list -- users@nest-simulator.org
>>> <mailto:users@nest-simulator.org>
>>> To unsubscribe send an email to users-leave@nest-simulator.org
>>> <mailto:users-leave@nest-simulator.org>
>>>
>>
>> _______________________________________________
>> NEST Users mailing list -- users@nest-simulator.org
>> <mailto:users@nest-simulator.org>
>> To unsubscribe send an email to users-leave@nest-simulator.org
>> <mailto:users-leave@nest-simulator.org>
>>
>> _______________________________________________
>> NEST Users mailing list -- users@nest-simulator.org
>> <mailto:users@nest-simulator.org>
>> To unsubscribe send an email to users-leave@nest-simulator.org
>> <mailto:users-leave@nest-simulator.org>
>
>
> _______________________________________________
> NEST Users mailing list -- users@nest-simulator.org
> To unsubscribe send an email to users-leave@nest-simulator.org
>

--
Wouter Klijn <w.klijn@fz-juelich.de>

Team Leader Multiscale simulation and design
SimLab Neuroscience
Jülich Supercomputing Centre
Institute for Advanced Simulation
Forschungszentrum Jülich
http://www.fz-juelich.de/ias/jsc/slns

Office: +49 2461 61-3523
Fax # : +49 2461 61-6656

------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
_______________________________________________
NEST Users mailing list -- users@nest-simulator.org
To unsubscribe send an email to users-leave@nest-simulator.org