My hands have improved enough in the last six months that I'm no
longer actively developing sproxy.
There's a program called Web Keyboard by Nelson Sproul that has
similar functionality as sproxy. I haven't used it, but it appears to
be much easier to install, and more complete. It's shareware, however,
and I do not believe it comes with source.
Web Keyboard can be found at http://www.adyn.com/web_key/.
Sproxy allows nearly hands-free navigation of the Web using a browser and generic speech recognition software. Unlike other programs of this kind, sproxy allows navigation by saying the
index
of a link. The indices are added automatically.
This document has been manually processed with sproxy (see the little numbers after all the links? They were added by sproxy). Once you have sproxy running, this will happen automatically.
If you want to visit Dragon System's
Home Page
[9]
, try typing "g9[Enter]". If you're using a JavaScript aware browser, off you'll go! If you add a macro in DragonDictate to send the keys "g9[Enter]" when you say "link 9", you could navigate this document with voice only!
You can also type "s9[Enter]", which will display "http://www.dragonsys.com" in the status line.
Sproxy requires:
-
a browser that supports JavaScript and the new event capture mechanism (e.g.
Netscape 4
[10]
)
-
perl
[11]
-
libwww-perl
[12]
-
my own patches to libwww-perl
-
a scriptable speech recognition engine (e.g.
DragonDictate
[13]
)
The proxy and the browser need not run on the same machine.
I have only tested sproxy with Netscape 4.03, DragonDictate 3.01, Perl 5.004, and libwww 5.18 on a Pentium PC running Windows NT4sp3. I have no idea what else it will run on, although I tried to make it generic.
To install sproxy:
-
Be sure you meet the
requirements
[14]
-
Download and save
patch.pl
[15]
-
Install the patch:
-
Find where HTML::Element::as_HTML is defined. It should be in something like /usr/local/lib/site/HTML/Element.pm. If you can't find it, you probably need to install libwww-perl. See
Requirements
[16]
, above.
-
Backup Elements.pm (e.g. cp Elements.pm Elements-orig.pm).
-
Near the bottom is the function as_HTML. Replace from the line starting
sub as_HTML
{
all the way to
join('', @html, "\n");
}
with the contents of patch.pl.
-
Find where HTML::TreeBuilder::text is defined. It should be in the same directory as Element.pm, and called TreeBuilder.pm. Around line 315, you need to replace:
if ($pos->is_inside(qw(pre xmp listing))) {
with:
if ($pos->is_inside(qw(script pre xmp listing))) {
-
Create a macro for the "link" command. In DragonDictate:
-
"Voice Menu" -> "Vocabulary Manager"
-
Click on "Netscape" in the left pane
-
"Word" -> "Add Word"
-
Word Name:
[link <Number/1 to 99>]
-
Execute Following Script:
SendKeys "g"+Str$(_1_to_99_1)+"{Enter}"
-
Create a macro for the "show" command. See above, but replace
[link ...
with
[show ...
and
SendKeys "g"+...
with
SendKeys "s"+...
-
Download and save
sproxy.pl
[17]
-
Edit the top of sproxy.pl, especially the @accesslist and @denylist.
-
Configure your browser to send http requests to the proxy. In Netscape:
-
"Edit" -> "Preferences", "Advanced" -> "Proxies"
-
"Manual Proxy Configuration" -> "View"
-
Under HTTP, fill in the machine name and the selected port.
-
Start the proxy. On a PC:
-
"Start Menu" -> "Programs" -> "Command Prompt"
-
C:> cd [
correct directory
]
-
C:> perl sproxy.pl
-
Browse!
Sproxy is a very simple proxy server that parses an html document, inserts the labels after each link, and generates a little JavaScript code to perform the hotkey operations. When you request a document, the request gets forwarded to the proxy. The proxy does the actual request. When it gets the document, it first checks if its really "text/html". If it isn't, sproxy just copies it verbatim back to you. If it is, it parses the document (using HTML::TreeBuilder, a class from libwww-perl).
Next, sproxy looks through the entire document, searching for
<a href=...>
tags. It adds a bit of html code after each one. In particular, it adds
<sup>[n]</sup>
, where
n
is the index of the link. It also keeps track of each such link for the next step.
Finally, sproxy adds a JavaScript program to the start of the page. The program simply sits around waiting for keyboard input. If the user enters
g
followed by a sequence of numbers followed by
[Enter]
, the JavaScript program tries to jump to the link with that index. If the user enters
s
followed by a sequence of numbers followed by
[Enter]
, it displays the link URL in the status line. It also writes the currently entered text to the status line. Note that the hotkey is ignored if a user-interface element like a text box has focus.
-
It's a hack.
-
The proxy is really simple. It doesn't fork subprocesses, and it doesn't cache. Therefore, it's only appropriate for one user at a time.
-
The hotkey code is primitive. The keys silently disappear if they're not valid (i.e. "g123x" is ignored).
-
The JavaScript code that gets added can be lengthy, since it includes all the links in the page. For a page consisting of mostly links (e.g.
Yahoo
[18]
), this is very wasteful.
-
The JavaScript code can interfere with an existing page's JavaScript. If you get weird JavaScript error messages, this is probably the problem.
-
The parser is more picky than most browsers. Pages which work fine without the proxy may fail with the proxy.
-
I violate some of the libwww data abstractions to make things work. sproxy may fail if libwww changes.
-
It only enables navigation of links, not image maps or forms. These shouldn't be too hard to add, but I haven't gotten around to it yet.
-
Frames are not well supported. You must manually change focus to the frame containing the link you're interested in.
-
Proxy-to-proxy is not supported. If you must use a proxy normally, sproxy won't work as-is. It should be pretty easy to forward the request.
Comments, suggestions, patches, and extensions are all welcomed. However, since I'm a grad student, I doubt I'll have much time to fiddle with this much more. Contact me at
janin@icsi.berkeley.edu
[19]
.
Copyright (C) 1998 Regents of the University of California. All rights reserved.
Disclaimer: This software is provided "as is". I take no responsibility for any problems resulting from its use.
You may use this software in any way as long as:
-
You do not charge for it.
-
The original author and copyright information is retained.
Special thanks to
Roy Fielding
[20]
,
Gisle Aas
[21]
, and the rest of the
libwww-perl
[22]
folks. Also, my appreciation to
Jerry LeVan
[23]
for phd, a proxy server written in perl on which sproxy was based.