Adam's [1] sproxy Home Page


My hands have improved enough in the last six months that I'm no longer actively developing sproxy.

There's a program called Web Keyboard by Nelson Sproul that has similar functionality as sproxy. I haven't used it, but it appears to be much easier to install, and more complete. It's shareware, however, and I do not believe it comes with source.

Web Keyboard can be found at http://www.adyn.com/web_key/.


Overview

Sproxy allows nearly hands-free navigation of the Web using a browser and generic speech recognition software. Unlike other programs of this kind, sproxy allows navigation by saying the index of a link. The indices are added automatically.

This document has been manually processed with sproxy (see the little numbers after all the links? They were added by sproxy). Once you have sproxy running, this will happen automatically.

If you want to visit Dragon System's Home Page [9] , try typing "g9[Enter]". If you're using a JavaScript aware browser, off you'll go! If you add a macro in DragonDictate to send the keys "g9[Enter]" when you say "link 9", you could navigate this document with voice only!

You can also type "s9[Enter]", which will display "http://www.dragonsys.com" in the status line.

Requirements

Sproxy requires:
  1. a browser that supports JavaScript and the new event capture mechanism (e.g. Netscape 4 [10] )
  2. perl [11]
  3. libwww-perl [12]
  4. my own patches to libwww-perl
  5. a scriptable speech recognition engine (e.g. DragonDictate [13] )
The proxy and the browser need not run on the same machine.

I have only tested sproxy with Netscape 4.03, DragonDictate 3.01, Perl 5.004, and libwww 5.18 on a Pentium PC running Windows NT4sp3. I have no idea what else it will run on, although I tried to make it generic.

Installation

To install sproxy:
  1. Be sure you meet the requirements [14]
  2. Download and save patch.pl [15]
  3. Install the patch:
  4. Find where HTML::TreeBuilder::text is defined. It should be in the same directory as Element.pm, and called TreeBuilder.pm. Around line 315, you need to replace:
      if ($pos->is_inside(qw(pre xmp listing))) {
    with:
      if ($pos->is_inside(qw(script pre xmp listing))) {
  5. Create a macro for the "link" command. In DragonDictate:
  6. Create a macro for the "show" command. See above, but replace [link ... with [show ... and SendKeys "g"+... with SendKeys "s"+...
  7. Download and save sproxy.pl [17]
  8. Edit the top of sproxy.pl, especially the @accesslist and @denylist.
  9. Configure your browser to send http requests to the proxy. In Netscape:
  10. Start the proxy. On a PC:
  11. Browse!

How it Works

Sproxy is a very simple proxy server that parses an html document, inserts the labels after each link, and generates a little JavaScript code to perform the hotkey operations. When you request a document, the request gets forwarded to the proxy. The proxy does the actual request. When it gets the document, it first checks if its really "text/html". If it isn't, sproxy just copies it verbatim back to you. If it is, it parses the document (using HTML::TreeBuilder, a class from libwww-perl).

Next, sproxy looks through the entire document, searching for <a href=...> tags. It adds a bit of html code after each one. In particular, it adds <sup>[n]</sup> , where n is the index of the link. It also keeps track of each such link for the next step.

Finally, sproxy adds a JavaScript program to the start of the page. The program simply sits around waiting for keyboard input. If the user enters g followed by a sequence of numbers followed by [Enter] , the JavaScript program tries to jump to the link with that index. If the user enters s followed by a sequence of numbers followed by [Enter] , it displays the link URL in the status line. It also writes the currently entered text to the status line. Note that the hotkey is ignored if a user-interface element like a text box has focus.

Limitations, Bugs, etc.

Comments, suggestions, patches, and extensions are all welcomed. However, since I'm a grad student, I doubt I'll have much time to fiddle with this much more. Contact me at janin@icsi.berkeley.edu [19] .

Legal Stuff

Copyright (C) 1998 Regents of the University of California. All rights reserved.

Disclaimer: This software is provided "as is". I take no responsibility for any problems resulting from its use.

You may use this software in any way as long as:

Acknowledgments

Special thanks to Roy Fielding [20] , Gisle Aas [21] , and the rest of the libwww-perl [22] folks. Also, my appreciation to Jerry LeVan [23] for phd, a proxy server written in perl on which sproxy was based.