The unit-level specifications contain most of the RBp-specific
parameters, although some important ones are also present in the
RBpTrial process. Note that the dt
parameter should be the
same for all unit specs used in a given network. Also, this parameter
is copied automatically to the RBpTrial process, which also needs to
know the value of this parameter. Thus, the unit spec is the place to
change dt
, not the trial process.
The unit object in RBp is essentially the same as the BpUnit, except for the addition of variables to hold the previous values of all the state variables, and special circular buffers to hold the entire record of activation state variables over the update trajectory. These are described in greater detail in section 14.2.9 RBp Implementation Details.
float dt
ACTIVATION
mode, the
activations are updated towards the raw activation value computed as a
sigmoid function of the current net input by an amount proportional to
dt
:
u->da = dt * (u->act_raw - u->prv_act); u->act = u->prv_act + u->da;Similarly, in
NET_INPUT
mode, the net-inputs are moved towards
the current raw net input proportional to the size of dt
:
u->da = dt * (u->net - u->prv_net); u->net = u->prv_net + u->da;
TimeAvgType time_avg
ACTIVATION
based time-averaging, as shown above, adapts the
current activations towards the raw activation based on the current net
input, while NET_INPUT
based time-averaging, also shown above,
adapts the net input towards the current raw value. The latter is
generally preferred since it allows networks with large weights to update
activations quickly compared to activation-based updates, which have a
strict ceiling on the update rate since the maximum activation value is
1, while the maximum net input value is unbounded.
bool soft_clamp
float soft_clamp_gain
bool teacher_force
bool store_states
Random initial_act