Homework 7. Bailey model exercises

Please note: after you have generated the scenarios, make sure to save them before starting to label them. In the label window, you may have to reload your labels (by changing language, for instance) to register the new scenarios in the list). Proceed with the rest of the assignment.

Introduction

This assignment involves the use of David Bailey's model for learning hand-motion verbs. Please read #11 in your reader before starting this assignment.

The software you will be working with is installed on the Solaris PC machines in 275-277 Soda, just as the previous homework was. The homework requires some time-consuming tasks (scenario labeling), so please allow yourself at least 3 hours in the lab.

Getting started

To set things up, type the following commands:

Then any time you want to run the system, type

In a few moments you should see a window pop up, with a picture in the center and a row of "tabs" across the top.

Note: Later on, once you've modified the various data files, if you want to review how things worked with the original files, you can do so by running the system this way:

cd /home/hh/cs182/verblearn
verblearn

Everything except saving files should work when you run the program this way.

Running the program

First, familiarize yourself with the system by training it on the set of "scenarios" (actions) which are already labelled with English verbs.

Here is a guide showing one way to use the software for this purpose. The system can do many things -- and you are free to play around -- but the user interface is not always obvious or "idiot-proof", so you may want to stick closely to this script at first:

Note: The program is not very fast. Try to wait a few seconds after each operation for the screen to update, before proceeding.

Choose the "Label" tab to bring up the Labelling window.
Use the "Pick a language" menu to choose "English1".
In the large window, you can observe the feature values for one particular scenario (whose name, such as "sc12", is shown below the feature values). Further below, is a small window showing the label given to this action by an informant.
You can browse through the set of scenarios using the "Browse Forward" and "Browse Back" buttons. (You can change the labels too, though this is not required yet.)

Switch to the Training window by clicking on the "Train" tab up top.
Create a new lexicon to train, by choosing "New/Unnamed" from the "Pick a lexicon" menu.
Pick the "original-50" dataset from the "Pick a dataset" menu.
Click "Do It!" and training will occur, spewing output to the main window which you may scroll through to inspect, once training finishes in a minute or so. Each word sense is displayed as a collection of probability distributions, one for each feature. Other information in the window explains the reasons for the creation or merging of these word senses.

You can inspect the trained lexicon by clicking on the "Lexicon" tab.
Click on the buttons for individual verbs to see the word senses learned for each.
Click the "Toggle Verbosity" button to oscillate between displaying full probability distributions for each feature, versus displaying only those feature values which have been learned to be important for the verb.

Test the trained lexicon using the "Recognize" and "Obey" tabs up top.
In either of these windows, just click "Do It!" to test the lexicon on a collection of test examples. The window will display the errors and the overall success rate.
You can study an individual test scenario more closely by selecting the "Selected w/more detail" button, then clicking on the scenario name in the choice box, and then clicking "Do It!" again.

Your assignment

Now that you are familiar with the program functions, your task is to teach the system a new set of verbs (along with some of the old ones). The key idea is to study verbs which might use a somewhat different set of features than those already built in to the program: you'll want to add some features, and perhaps remove a few.

Here is the set of verbs encompassed by the original scenarios in the system:

You should keep at least half of these, plus add the following:

You will have to add new features to be able to learn these.

Also add a verb of your choice which make use of one of the following features:

intention
object type, shape or weight
orientation (such as in Regier's Mixtec "cii" example)
other features that you can justify applying to a novel verb

Please do not add more than 4 new features, so that the system's capability to learn does not diminish due to a small number of examples per feature / word label.

To proceed, first generate a new set of scenarios which reflect your added and removed features:

Click on the "Generate" tab.
Click "Load scenarios"; you should see a collection of scenarios, labelled sc1 through sc50, show up in the lower-right window.
To remove unwanted features or add new features, click on the "Add or Remove features" button. You'll see a popup window. Click on features to be removed (or restored, if you removed them earlier). To add a new feature, click on "Add feature" button. You'll get a second popup window. Specify the name for your feature, and whether it's an action-oriented feature ("motor") or a contextual feature to be used for disambiguation ("world"). Enter the possible values for the feature in the designated box, pressing <enter> after each one. When finished, click "OK" in this second popup. When done adding/removing, click the "OK" button to apply all the changes. You'll be prompted for a name for the new "parameter file". Enter a new name, such as "myparams". Use this same name if you make further changes to the feature set.

Warning:

Back in the Generate window, click on "load restrictions" in the upper right region of the window. In the popup that follows, click on the file "standardrestrictions".
If you have new restrictions to add, involving your new features, click on "make new restriction" and fill out the popup window. (Eg: "weight = heavy", "force = low", and "accel = high", are an unlikely combination; so are "schema = lift" and "dir = down".)
Back in the Generate window, set the "first index" field to 51, not the current setting of 1 (so the existing 50 scenarios won't be overwritten). Set the "how many" field to 50.
Click on "generate scenarios. You should see your new scenarios added to the window in the lower right.
Important: Click on "save scenarios" so that the rest of the program will see your changes. Next, label your new scenarios with the appropriate verbs:

Choose the "Label" tab up top to switch to the Labelling window.
Click on "unlabelled" to select all unlabelled scenarios. Scenarios number sc51 through sc100 should be highlighted.
Select "start label cycle" to begin labelling.
One by one, scenarios will be presented in the main window. Use your judgement to determine an appropriate verb label, enter it in the little window toward the bottom, and then press an appropriate button. You can interrupt this process and finish it later if you get tired.

with one caveat

Questions

Also explain whether you think the system would be able to learn the following verbs (and what needs to be added in order to learn them): give, take, catch, dribble (the ball), pass (the ball), remove, clean.

This assignment is due Monday, March 29.