This assignment is to be run in the lab B21 Dwinelle. Don't forget to check the open hours schedule, and to copy the Tlearn software to the Hard Drive before you begin. This assignment might take some time, so bring a disk in order to be able to save the files you create or change in case you need more than one lab session to complete the assignment.
Refer to #8 in you reader (Plunkett & Elman: Ch 1. & Appendix B - User Manual) for general instructions on how to run Tlearn. If you would like to consult the whole Plunkett & Elman book, you can find it on reserve for the class in the Engineering library.
The assignment is to explore the capabilities of some very simple network by design or training. You are to build networks using Tlearn and to describe their behavior.
The first task is the logical AND function: AND(0,0)=AND(0,1)=AND(1,0)=0;
AND(1,1)=1.
(start by opening the AND project in ch. 3 of Tlearn
and studying its files, then opening the architecture, weights and node
activation display).
1. You are to set the weights and the activation output
function in such a way that the 4 possible AND inputs (01, 11, 00, 10)
produce the correct distribution of the AND outputs. Keep in mind
that the bias node (b) can fulfill the function of a threshold for the
output units; assign its weight accordingly.
- Fix the weights at the values that you think would
work best for this problem; try setting both the sigmoid (default) or linear
output function of propagating activation (both are done by making changes
to the and.cf file).
- Set the # of training sweeps (in "Training Options")
to 1, run the "Train the Network" command. (It turns out that Tlearn requires
this dummy training run for no good reason)
- Now run the "Verify the Network has learned" command
to evaluate your solution for the AND function.
2. Now try to make Tlearn learn the AND function
(learning will involve the system setting random initial weights which
are adjusted during training):
- "Unfix" the weights (set then range from -1 to 1 by
setting the weight_limit = 1 in the .cf file)
- Set the # of training sweeps to 5000 this time, and
the function to sigmoid (you don't need to say anything but to remove the
line assigning a linear function if it is present in the .cf file).
- When you are done configuring the files, open the "Error"
display and run the "Train the Network" command.
- Tlearn allows you to change a number of parameters
such as learning rate, momentum, etc (in "Training Options"). See if you
can find settings that will enable Tlearn to learn AND essentially every
time (after a number of training runs).
- For what range of settings will it learn 75% of the
time?
- Try learning the same AND function starting with a
net that has two intermediate units between the input and output, but no
direct input/output links (you have to reconfigure the and.cf file again
in order to change the network architecture). Experiment with adjusting
the same parameters. How much does this new network help?
3.The second task is the logical SAME function:
SAME(0,0)=SAME(1,1)=1; SAME(0,1)=SAME(1,0)=0.
Using the AND project as an example, create the necessary
training
and configuration files for the SAME project (or simply edit the
.data and .teach files to perfom the SAME function -- the architecture
will be the same as for AND).
Perform a similar set of experiments with designed weights
and with learning, using networks without and with two extra "hidden layer"
units. Explain why the results are quite different in this case.
You should turn a record of the (hand-recorded)
parameters that you tried on each example as well as an account of what
happened.
A printed or hand-written network architecture with drawn-in
weights is reccomended.
For those who want to use the software at home, 3 options:
-You can get the program for PC or Mac from the following
ftp sites:
ftp://crl.ucsd.edu/pub/
ftp://ftp.psych.ox.ac.uk/pub/
-Mac users can bring a disk to the lab and copy the software (the whole Tlearn folder) from there;
-PC users (WINDOWS 95 REQUIRED!) -- download and
uncompress the software from the Tlearn sight - http://crl.ucsd.edu/innate/;
then copy these files to the same directory, and create a new project using
files below (refer to the user manual for instructions on creating new
projects). and.data
and.teach
and.cf
This assignment is due in class on Wednesday, February
17 (Monday the 15th is a holiday).
Readings:
Reader: 8; Regier: Ch 3-4