This is the second Tlearn assignment: the same rules for running the software in the lab B21 Dwinelle apply.

In this problem, we will experiment with a standard back-prop example, the auto-encoder. The basic idea is very simple, get Tlearn to produce output that is as close as possible to its input. As is conventional, we will restrict ourselves to binary strings. The catch is that the network will have a hidden layer that is significantly smaller than the input and output layers.

The basic case will have 4 binary input units, 2 hidden units and 4 output units. Only one input unit will be "on" ( =1) at a time and we want the network to learn weights that will make the corresponding output unit to turn on.

We can envision the network as doing a kind of compact encoding. For example, if some language had only 4 phonemes, the auditory system could get by with just 2 fibers for transmitting one phoneme at a time. More realistically, a phonemes from a language with 64 phonemes could be transmitted by just 6 nerve fibers from one brain region to another. One could imagine a complex neural structure that computed which phoneme was most likely at each moment and another complex structure that made use of phonemes to make up words. Since each phoneme has different uses, we would need a separate unit for each one at the receiving end, but the transmission could be done more compactly using the idea above.

The assignment is to experiment with how well back-propagation learning can do at finding weights that will produce a good encoding.

1. Create a Tlearn project file for the 4-2-4 encoder problem. The first part of this assignment is to analyze how the system does on this task. As with the previous assignment, you should experiment with different values for the learning rate and momentum. We suggest using random initialization and sampling. Explain how the parameter values affected the final error rate and number of trials needed. Notice that, since the initial weights are random, separate runs might have wildly different outcomes. How does this relate to learning in animals?

2. For the second part of the assignment, you are to expand the task to solve the 8-3-8 encoder problem. Again experiment with and report on the effects of different parameter values. Do you notice any differences in moving to a larger task?

3. Finally, try Tlearn on the 9-3-9 encoder problem. Why can it learn this despite the fact that 3 bits is not enough to encode 9 values?

Homework 4 is due Monday, February 22.