Lecture 8: Coding and computing with balanced spiking networks

Lecturer: Sophie Denève

Cortical spike trains

Spike trains: highly variable ⇒ really hard to guess if there has been a stimulus based on ONE spike train

Count variance vs. Count means ⟶ ≃ linear

\text{Proba to fire:} \underbrace{f}_{\text{firing rate}} dt
• But where does the variability come from? External noise, internal dynamics?

• How does the brain deal with such a variability? ⟹ What matters is not spikes, but rather firing rates

Integrate-and-Fire

Very naive neuron model:

τ V = - V + I_{exc} + I_{inh}

where

• $I_{exc}$: excitatory synapse current (Poisson)
• $I_{inh}$: inhibition synapse current (Poisson)

In practice: there’s also a noise term ⟶ but if there is a large number of inputs at the synapse, the noise averages out.

⇒ How does Poisson-like variability survives?

1. One possibility: a lot of excitatory synaptic weights, but it’s rare that there is a spike, and several of them have to accumulate for the output neuron to spike

2. Other possibility: $I_{inh}$ and $I_{exc}$ almost compensate one another ⇒ then, random walk → the variance increases over time, until it reaches the threshold potential, and then the neuron fires

In this case: exponentially-distributed interspike interval → indicate Poisson process

E/I balance

Stimulus driven response

But in practice: not as simple as that: sometimes both excitation and inhibition increase, and then it results in the overall input current increasing ⟹ spike

Spontaneous activity

Two neighboring cells have very correlated inh/exc currents ⟶ you can measure

• exc on one
• inh on the other

as if it was for the same cell.

Observation: each time the neuron receives an exc current, it receives a strongly correlated inh current at the same time.

Two types of balanced E/I

• feedforward inhibition: input (from the thalamus for ex) ⟶ exc and inh population neurons, then from inh to exc (delay for inhibition, as there are two links)

• recurrent inhibition: exc and inh population neurons are linked

Balanced neural networks generate their own variability

Constant $I_{ext}$ ⟶ Balance in the network:

J_{EE} ν_E - J_{IE} ν_I = 0\\ J_{EI} ν_E - J_{II} ν_I = 0

And then integrate-and-fire:

\frac{dV_E^i}{dt} = - EV_E + J_{EE} \sum\limits_{ j ∈ \lbrace \text{input spikes} \rbrace } o_j^E + + J_{JE} \sum\limits_{ j ∈ \lbrace \text{input spikes} \rbrace } o_j^J

(the firing rates $ν$ are computed out of the spikes $o$)

Asynchronous irregular regime: if you shift one spike by $0.1$ ms, it changes everything else!

Chaotic system dynamics: not satisfactory, as any slight change in initial condition leads to completely different results ⟹ very hard to code information

I/E variability: the system dynamics is located in a low-dimensional submanifold (Lorentz attractor: 2D-manifold)

Efficient coding by sensory neurons

Example: real image $\textbf{x}$, reconstructed one: $\hat{\textbf{x}}$

\hat{x}_i = \sum\limits_{ j } Γ_{ij} r_j
• One neuron ≃ one feature
• You want to minimize the cost

⟹ Neural network: linear decoder

A pure top-down approach

Two types of constraints:

1. bilogical ones (synpactic, etc…)
2. optimization one: reduce cost
\dot{\textbf{x}} = f(\textbf{x}) + \textbf{c}(\textbf{t})

where

• $\textbf{x}$ is an internal state variable governed by an unknown dynamical system
• $\textbf{c(t)}$ is the input (or command variable) ⟶ controlled externally

Ex:

• $\textbf{c(t)}$: Motor stimuli / Motor command
• $\textbf{x}$: Direction of motion / State of your arm

The state variable is decoded linearly from the output spike trains:

\hat{\textbf{x}} = D \textbf{r}

where

• $D$: decoding weights
• $\textbf{r}$: filtered spike trains

\dot{\textbf{r}} = - \textbf{r} + \textbf{s}\\ ⟹ \textbf{r}_j = \textbf{s}_j \exp(-t)
• $\textbf{s}$ is a spike train ($\textbf{s}_j ∈ \lbrace 0, 1 \rbrace$)

Goal: minimize:

L = 𝔼(\underbrace{\Vert \textbf{x} - \hat{\textbf{x}}\Vert^2}_{\text{error}} + \underbrace{μ \Vert \textbf{r} \Vert^2}_{\text{cost}})

Example of a dynamical system

\dot{\textbf{x}} = - \textbf{x} + \text{c(t)}

So in if you multiply by $D$ the $\textbf{r}$-defining equation:

\underbrace{D\dot{\textbf{r}}}_{= \dot{\hat{\textbf{x}}}} = -\underbrace{D\textbf{r}}_{=\hat{\textbf{x}}} + D \textbf{s}

So with the input:

I = \textbf{c} - D\textbf{s} = \textbf{c} - (\dot{\hat{\textbf{x}}} + \hat{\textbf{x}})

Integrate-and-fire:

\dot{V} = -V + I

Tags:

Updated: