5.2 Questions about Networks

For information about how to build a network using the network viewer, see section 10.8 Building Networks Using the Viewer and the tutorial section 4.3 Configuring the Encoder Problem. Also, many questions can be answered by looking at the chapter on networks section 10 Networks (Layers, Units, etc).

How to I get certain units to use a different parameter than others?
For example, if you want certain units to use a different learning rate, or activation function, or gain, etc... This is done by making a new UnitSpec or ConSpec (depending on where the relevant parameter is), and telling the relevant units or connections to use this new spec. It is recommended that you create the spec as a child of an existing spec (see section 8.4 Specifications), so that all the other parameters will automatically be inherited by the new spec (except for the one you change). The easiest way to apply a different spec is to select the relevant units or projections, and use the Selections menu on the NetView (see section 10.6 Network Viewer) to set the spec. The tutorial now has an example of how to do this (see section 4.3.7 Training and Testing Your Network).
What is the difference between Projections and Connections?
Projections specify the broad patterns of connectivity between layers. Connections are the actual unit-to-unit weights and other parameters which actually implement this connectivity. Thus, there is always a projection associated with a set of connections. See section 10.3 Projections for more details.
How do I implement weight sharing/linking?
The TesselPrjnSpec (see section 10.3.3.2 Tesselated (Repeated) Patterns of Connectivity) and the LinkPrjnSpec (see section 10.3.3.5 Miscellaneous other Projection Types) are two types of projection specifications (see section 10.3 Projections) that implement weight sharing. The TesselPrjnSpec generates repeated patterns of connectivity, and it can automatically link all of the repeated patterns together with the same set of weights. Thus, a set of units in a receiving layer can all have the same receptive field from a given sending layer, and all of the units can use the same set of weights to define their receptive field. The LinkPrjnSpec allows individual or small groups of weights to be specifically linked together, even if these connections are in different layers in the network. It does not generate any connectivity itself, it simply causes existing connections to share weights.
Can I temporarily lesion a layer in my network?
Sometimes, one wants to pre-train part of a network on one task, and then subject the rest of the network to some more complex task. This process is greatly facilitated by being able to create the entire network at the outset, and then temporarily "lesion" certain layers during pre-training. This can be accomplished by simply checking the lesion flag on the Layer object (see section 10.2 Layers and Unit Groups).
Are there functions for lesioning the weights or units in the network?
Yes, LesionCons, AddNoiseToWeights, TransformWeights, and PruneCons all perform various manipulations on the weights in a network, and could be used to simulation "lesions" of the network. LesionUnits lesions units. These functions, like most in the network, can be called at various levels of granularity from a single group of weights (or units) up to the entire network. See (see section 10.1 The Network Object) for details.
How can I use a specified (non-random) set of initial weight values?
There are several ways to do this. One is to write a CSS script to set the weight values by reading them in from a file or from values coded into the script itself. This script can be attached to a ScriptPrjnSpec so it is run automatically when the network is connected (see section 10.3.3.5 Miscellaneous other Projection Types). It is also possible to use a TesselPrjnSpec (see section 10.3.3.2 Tesselated (Repeated) Patterns of Connectivity) or RandomPrjnSpec (see section 10.3.3.3 Random Patterns of Connectivity) in conjunction with the init_wts flag to specify initial weight patterns, which are used instead of the random ones whenever the network is initialized (see section 10.3.2 The Projection Specification). You could also construct a "donor" network that had the intial weights set as you wanted them (by hand or whatever), and then use the CopyFrom or Copy_Weights function to initialize your training net from the donor net (see section 10.1 The Network Object). Similarly, you could use WriteWeights and ReadWeights to save and load weights from a file.
Is there a way to view the weights for a set of multiple units at
the same time? Yes -- the function GridViewWeights on the network (in the Actions menu) will display the entire weight matrix between two layers of the network on a GridLog. Also, you can plot a matrix of events from an environment using the EnvToGrid function on an environment (Generate menu). This is useful for activity-based receptive fields computed via the UnitActRFStat (see section 12.8.5 Activity-based Receptive Fields), which are stored in an Environment.