Instead of putting all the control necessary to iterate over the several levels of processing needed to train a network (training, epochs, trials, settles, cycles, etc.) into one process object, we have divided up the processing hierarchy into a set of nested scheduling processes, each of which handles one level of processing. This results in a lot of flexibility, since one can then change only the code controlling the trial, for example, or easily extend it using a CSS script.
The Schedule Process (or SchedProcess) is an extension of the basic Process type. It provides more variables and functions with which one can control the execution of the Process. It has fields for the parent and child processes in its hierarchy, support for counters that control the iteration of the process over time, places to link in View objects and Log objects to be updated, and groups to hold the various sub-processes and statistics that can be associated with a given level of processing.
In order to support all of its extended functionality, the schedule process has a somewhat complicated execution structure. However, understanding how a schedule process runs should make it easier to figure out how to get them to do what you want them to.
The central function a schedule process performs is one of looping, where the process repeated performs some function. In most cases, this function simply involves telling the process below it in the hierarchy to run. Thus, an epoch process repeatedly loops over the trial process, for example. The functions in a schedule process center around the main loop of processing,
The main loop is written so as to be re-entrant. Thus, something can cause the process to pop out of the loop (i.e., the user pressing Stop), and when it runs again, it will fall back down to the point where it was last running and pick up again where it left off.
The places where the things that hang off of a schedule process, like statistics, logs and displays, can all be seen in the main schedule process loop code, which is reproduced here:
void SchedProcess::C_Code() { bool critval; if(re_init) { // if its time to re-initialize, then do it Init(); // this sets the counters to zero, etc. InitProcs(); // this runs any initialization processes } do { Loop(); // user defined code goes here UpdateCounters(); // increment the counters (before logging) LoopProcs(); // check/run loop procs (use mod of counter) LoopStats(); // update in-loop statistics if(log_loop) // can log inside loop or after it... UpdateLogs(); // generate log output and update logs UpdateState(); // update process state vars (current event..) critval = Crit(); // check if stopping criterion was reached if(!critval) { // if at critera, going to quit anyway, so don't StepCheck(); // check for stepping StopCheck(); // check for stopping } } while(!critval); // loop until we reach criterion (ctr > max) // or stat criterion met Final(); // user defined code at end of loop FinalProcs(); // call the final procs FinalStats(); // run final_stats at end of loop if(!log_loop) UpdateLogs(); // if not logging in loop, logging at end UpdateDisplays(); // update displays after the loop SetReInit(true); // made it through the loop, so Init next time }
The fall-through character of processing is made possible by storing all
of the counter state variables on the process object itself, so it is
preserved even when we pop out of the loop, and by only initializing
once we make it through the loop (by setting the re_init
flag).
As you can see, there are two places where statistics get updated,
inside the loop (LoopStats()
) and after the loop
(FinalStats()
). While it is more natural to think of computing
statistics at the end of the loop (e.g., at the end of the trial or the
end of the epoch), the need to aggregate statistic values over time
(e.g., compute the sum of the squared-errors over an entire epoch)
necessitates inside-the-loop statistics.
Finally, it should be noted that all schedule processes are written to allow any number of intervening processes to be added in to the hierarchy at any point. Thus, new and unanticipated levels of processing can be introduced by the user without breaking the assumptions of the existing process objects. Basically, any time there is a dependency of one level of processing on another (e.g., the trial process looks to its parent epoch process to determine if it should be testing or training the network), the dependent process searches through the entire hierarchy for the type of process it depends on.