It is often useful to monitor the values of network variables, such as the activation of the output units, in your log files. For example, you may wish to watch how the network gradually differentiates its output in response to the four input patterns. We can do this by creating what is called a MonitorStat statistic.
Conceptually, a monitor statistic is a special type of statistic that simply monitors values already computed by the processes that are applied to the network. For example, activations are calculated by the TrialProcess during training. To record these activations in a log we need to monitor them and that's what a Monitor Statistic will do. Think of it as an electrode for recording data at specified times inside your network.
To create a Monitor statistic, first select the units and the variable
you want to monitor in the NetView. To monitor with activation of
the output unit, for example, we would make sure act
is selected
as the variable, and then we would select the output unit. In this case
we can select either the unit or the whole layer, it won't make any
difference. To select the layer, make sure the Select button is
highlighted in the NetView and then click over the output layer itself.
The layer's border will be highlighted. Now Select Monitor Values /
New (located in the middle-left region of the netview), and a pop-up
will appear with some necessary information displayed.
The popup indicates that you are creating a type of Statistic called a
MonitorStat, and it indicates which process will contain this
Statistic. In this case, the In Process
button will indicate
that the statistic will be placed at the end of the "TrainTrial"
process, which is what we want (we already have something monitoring
activations in the "TestTrial" process). The For Variable
should
be act
. The Using Operator
in this case is COPY
,
which is also what we want -- we just want to copy the values of all the
selected units into the statistic. The last aspect of this is
conceptually the most challenging. What we need to do is to specify
that we want to Create Aggregates
of this variable at higher
levels. In this case, what we really want to do is just copy the
activation from all 4 patterns to the Epoch level, so we can display the
activations of the output unit for the different events in our Graph.
So click the CreateAggregates toggle, then click Ok. Another
popup will then appear, asking you what form of aggregation you want.
In this case you must select COPY, to indicate that we are copying
each event's output unit activation into the epoch graph, not averaging
or summing or anything like that. Do this, click Ok, and you are
done.
Before you restart the simulation, you'll need to reinitialize the weights. We do this in the Train Control Panel. You reinitialize with the same starting weights by clicking Re Init, or you can initialize with new starting weights by clicking New Init. Either way, the network is reinitialized and the epoch counter is set to 0. Click Re Init this time so we can see trace the learning process over the very same trajectory as before. To run the simulation again we can simply press Run or Step in the "Train" process control panel. You'll see the new results graphed on the same graph along with the earlier sum of sum-squared-error results.
The graph may seem a little confusing at first. Overlaying the
sum_sum_se
results will be four new lines, which will seem to be
oscillating rather extremely. Buttons identifying these new lines will
be shown on the left of the View. Note that the color of the graph line
associated with a variable that is displayed is shown to the left of the
Button, and the color of the axis that is associated with the variable
is displayed on the right. All four new lines are associated with a new
Y-axis, which you can see covers a rather narrow range of values (about
.46 to .54). This axis is tied to the entire group of activations that
are being monitored, and it is auto-scaled to the largest and smallest
value in the display.
You may want to fix the scale to something reasonable like 0-1. To do
this, click with the right mouse button -- the Edit button
-- on the lead (top) element of the set of new elements (it may be
labeled output.act
or cpy_output.act_0
. A popup will
appear. You want to set min mode
and max mode
to
FIXED
so click where it says AUTO GROUP
and select
FIXED
, then enter 1.0 in the max field next to range
. To
make this take effect, click Update View, then click Ok. The
new Y axis will now span the 0-1 range, and the oscillations will appear
more moderate on this scale. This axis adjustment can be done while the
simulation is running.
In any case, you'll see that after thirty epochs or so the oscillations level off. The simulation appears to be flatlined until about epoch 120, when the activations of the output unit begin to differentiate the four input patterns. (The outputs to patterns 1 and 2, which are the 01 and 10 patterns, are nearly identical and the latter lies on top of the former much of the time). In any case, you should see that the network tends to produce the strongest output for the '11' pattern (pattern 3) until about epoch 210, well after it has completely nailed the '00' case. After epoch 210 the performance changes rapidly for the 10, 01, and 11 patterns (patterns 1, 2, and 3) until about epoch 270 or so, where they begin to level off at their correct values.