The good news: CGI scripts let us create Web sites that are more than just static brochures.
The bad news: CGI is sluggish.
It's true -- CGI can be slow. Users of CGI-heavy sites get a very different experience from users of native applications. Sites that use CGI tend to use it for any nontrivial computation. CGI is also slow because of the common technique of spawning a new process on the server side for each CGI request. JavaScript can be used to speed things up by moving some computation to the client side, but it's no help if you need to hit the server for data.
All of this adds up to a site in which every single action can take a few seconds. Admittedly, we're used to it, but does it have to be this way?
The solution
We can create a more responsive experience if we look at the available technology in a different way. In particular, we are going to forget the assumption that a Web page is loaded directly from a Web server. Instead, we are going to use the request/response structure of the Web in a slightly different way. We are going to have the server script send the browser a page that is nothing but JavaScript code. This code, when executed, will stuff new information into data structures which can be used and displayed by the rest of the program. That is, the purpose of the server call is to simply to get raw data from the server. Since the page has nothing but JavaScript code in it, there is nothing in it to see. Thus, the frame into which it is loaded can be made very small, so that it doesn't take up any room on the screen. This frame, instead of being used to display HTML to the user, is being used as a receiving window for server data. This approach makes the server request cycle almost entirely invisible. (It's not entirely invisible because the browser itself will let you know that it is loading something by running the animation in the upper right hand corner.)
Not only will we load this data invisibly, but we are also going to do it automatically. Instead of waiting for the user to initiate a data request by clicking on a link, we are going to set up a timer that repeatedly initiates the request/response cycle. Once the response is received, the timer is started again and the cycle continues. This setup acts like a kind of ad-hoc background "thread" which is periodically loading new data from the server. (I say "thread" even though JavaScript doesn't have threads the way Java, for example, does.)
While all this is going on behind the scenes, the user navigates around the user interface as with any other application. However, the links the user clicks on do not make requests to the server. Instead, the links are tied to JavaScript functions which create and display HTML pages in frames. These JavaScript functions access and display data that has been loaded by the background thread. Since the HTML is being rendered on the client side, and since the data has already been loaded by the background thread, the HTML is displayed immediately. This is what gives the user a heightened impression of responsiveness. With the request/response cycle nearly invisible, and data loaded automatically, delays caused by slow connections can be "hidden" from the user.
The technique can be outlined more clearly if we contrast it with the manual request/response cycle that is usually used.
Manual request/response cycle:
- user decides to request new information
- user clicks on a link
- browser sends request to CGI script
- CGI script prepares page of HTML
- browser receives Web page and displays it in a frame
- user sees page in frame after a delay
Asynchronous (background) request/response cycle:
- browser, under the control of a looping timer, sends request to CGI script
- CGI script prepares page of JavaScript code
- browser receives JavaScript code in a "hidden" frame
- browser executes JavaScript code
- JavaScript code stuffs new data into JavaScript variables, ready to be seen by the user
- timer is reset to start the process over again
- user decides to request new information
- user clicks on a link
- link calls a JavaScript routine
- JavaScript routine renders page of HTML to a frame
- user sees page in frame immediately
Example program
To illustrate this technique, we will use an ersatz stock-market application. This application allows the user to buy and sell shares of three different stocks. The background thread continually requests stock price updates from the server. The user can browse through the stocks, as well as buy or sell shares of them.
The screen is divided into four main windows.
The Select frame allows you to select which stock is currently being displayed. Pressing any of these links will bring up the corresponding stock. Since the program does not have to hit the server synchronously to get this information, you see the information immediately.
The Portfolio frame shows you the value of the currently selected stock, as well as the number of shares you own. When you select a stock in the Select frame, this frame is updated immediately, without any need to wait for a server hit.
The Buy/Sell frame allows you to buy and sell shares of the currently selected stock. This transaction also happens instantaneously, and the Portfolio window is updated immediately, without any need to wait for a server hit. (In a real application, the purchase orders would have to be sent to the server in the background thread.)
Finally, the strip at the bottom is used to display messages. When the background thread hits the server, this fact is displayed here.
Oh, you'll notice that at the top of the window there are three tiny frames. These frames are used for the internal workings of the system and do not need to be seen by the user. (In a real application, these frames could be made less obtrusive by coloring them and removing their borders.)
The first tiny frame, codeFrame, contains most of the JavaScript code in the system. This contains all the data variables that hold the information the user sees, and it's from here that everything is controlled.
The second and third tiny frames, known as sendFrame and rcvFrame, respectively, are used by the background thread to make the asynchronous server requests. They are described in more detail below.
The background timer
In order to periodically and automatically request new data from the server, we need to have something that acts like a background thread. This "thread" would hit the server, pause for some length of time, hit the server again, pause again, and so on, forever. JavaScript doesn't currently have threads per se, but you can get the same effect with the call. The call takes two arguments:
setTimeout( "<i>JavaScript code</i>", <i>delay_in_milliseconds</i> )<BR>
|
Following a call to , after <i>delay_in_milliseconds</i> |
has elapsed, the code fragment gets run. Here is how this is used to set up a repeating background thread:
// Start the background "thread"<BR>
function startAsynch() {<BR>
setTimeout( "runAsynch()", pause*1000 );<BR>
}<P>
// This function is called repeatedly by our background "thread".<BR>
// It simply calls the hitServer() and then calls startAsynch() to<BR>
// restart the timer and keep the "thread" going.<BR>
function runAsynch() {<BR>
<UL>
hitServer();<BR>
startAsynch();<BR>
</UL>
}<BR>
|
In this example, is called to set everything in motion. sets a timer to call after a pause. It's like a job scheduler, where is the job.
When is called, it does whatever work it has to do -- in this case, the work is to call . When this is done, it calls to reset the timer -- this way, it sets itself up to be called again, after a pause. As long as no code error occurs, this function will run over and over.
(Note: client-side JavaScript doesn't seem to have any synchronization mechanism, which is not surprising since it's not really multithreaded. In any case, it is important to take care to make changes to data atomically so that the program does not get into an inconsistent state.)
Data communication
Now that we are sure that is going to run periodically and automatically, we have to create . Remember, 's job is to call a CGI script on the server; the server sends back a page of JavaScript code and this code, when executed, stuffs the new data into variables for use by the rest of the program. To make the request, we are going to use a hidden form. To do this, we create a form in the tiny frame called . This form has as it's action, which means that, when it is submitted, its contents will be sent to the CGI script . The target of this form is , which means that anything that is sent back by will be put in the tiny frame called . Finally, the form also contains hidden variables -- one for each of the stock prices.
The form is submitted by calling its method. Here is the full implementation of :
// Hit the server to get the new stock prices.<BR>
// This is done by creating an invisible form that is submitted<BR>
// automatically to the server program.<BR>
// The current stock prices are sent to the server program,<BR>
// as an example of how to send parameters using this technique.<BR>
function hitServer() {<BR>
<UL>
message( "Hitting server...." );<P>
// Get ready to write to the "secret" send frame<BR>
var doc = parent.sendFrame.document;<BR>
doc.clear();<BR>
doc.open();<P>
// Write the form, including the parameter as a hidden element<BR>
doc.writeln( "<form name=\"sendForm\" target=\"rcvFrame\""+<BR>
" action=\"server.cgi?\" method=\"GET\">" );<BR>
doc.writeln( "<input type=\"hidden\" name=\"blap\" value=\""+<BR>
pps.blap+"\">" );<BR>
doc.writeln( "<input type=\"hidden\" name=\"xomt\" value=\""+<BR>
pps.xomt+"\">" );<BR>
doc.writeln( "<input type=\"hidden\" name=\"smrz\" value=\""+<BR>
pps.smrz+"\">" );<BR>
doc.writeln( "</form>" );<P>
// Done writing<BR>
doc.close();<P>
// Automatically submit the frame to the server program<BR>
parent.sendFrame.document.sendForm.submit();<P>
</UL>
}<BR>
|
When the form's method is called, the browser sends the request and the CGI script sends the response. It's not important what technology is used to implement -- the only thing that matters is the structure of the response. Here is an example of a response:
Content-type: text/html<P>
<html><BR>
<script language="javascript"><P>
parent.codeFrame.rcvPPS( 23.45, 32.8, 44.4 );<P>
</script><BR>
</html><BR>
</pre><BR>
|
In fact, if you view the source of the tiny at any time, you should see something like the above (sans "Content-type" line, of course).
When this response is loaded into the tiny rcvFrame, any and all JavaScript contained by it is executed. In this case, there is a single call to . , which lives in the JavaScript code in codeFrame, and takes three arguments -- the new values for all three stocks. These values were generated by the server-side script, and, in a real application, they would have been acquired from some authoritative data source.
Note that the full "path" to is specified. This is necessary because this code fragment is being executed in rcvFrame, but the code being called is in codeFrame. Specifying parent.codeFrame.rcvPPS() |
allows the call to cross the frame boundary.
Here is the function :
// Store the new Price per Share values in our global variable<BR>
// This function is called, not from any code here, but from the<BR>
// Javascript code generated by the server; the new values are passed<BR>
// in here as an argument<BR>
function rcvPPS( blap, xomt, smrz ) {<BR>
<UL>
pps.blap = blap;<BR>
pps.xomt = xomt;<BR>
pps.smrz = smrz;<BR>
showPortfolio();<BR>
message( "Done." );<BR>
</UL>
}<BR>
|
This function takes the new values and stores them in global variables. Then, since the value of the currently selected stock has surely changed, it calls to redisplay this information. Finally, it lets the user know that the asynchronous call is over by displaying "Done" in the message strip at the bottom.
User interactivity
Now that we have fresh data coming from the server at regular intervals, we need to provide a way for the user to browse through the data. The Select frame contains three links -- one for each of our stocks. When the user clicks on one of these links, the information about that stock is displayed in the Porfolio frame. Again, because a JavaScript routine sends a full page of HTML to the Portfolio frame, it is displayed immediately, without a server hit to slow it down. This routine reads the data that had previously been acquired from the server, and generates a page containing the data along with text and formatting tags.
Here is the routine that generates the contents of the Portfolio frame:
// Display the player's portfolio information in the portfolio frame function showPortfolio() {<BR>
<UL>
// Get the current values<BR>
var curpps = round( pps[curstock] );<BR>
var curnumshares = numshares[curstock];<P>
// Calculate the total value of the player's portfolio<BR>
var totalvalue = round( curpps*curnumshares );<P>
// Get ready to write to the portfolio frame<BR>
var doc = top.marketFrame.document;<BR>
doc.clear();<BR>
doc.open();<P>
// Write some HTML containing the portfolio information<BR>
doc.writeln( "<html>" );<BR>
doc.writeln( "<center><h1>" );<BR>
doc.writeln( "<i>Portfolio</i>" );<BR>
doc.writeln( "</h1></center>" );<BR>
doc.writeln( "<h2>" );<BR>
doc.writeln( "Stock: "+curstock+" <p>" );<BR>
doc.writeln( "</h2>" );<BR>
doc.writeln( "<table>" );<BR>
doc.writeln( "<tr><td>Price per Share</td> <td> $"+curpps+"</td></tr>" );<BR>
doc.writeln( "<tr><td>Number of Shares</td><td>"+curnumshares+"</td></tr>" );<BR>
doc.writeln( "<tr><td>Total Value</td><td>$"+totalvalue+"</td></tr>" );<BR>
doc.writeln( "</table>" );<BR>
doc.writeln( "</html>" );<P>
// Done writing<BR>
doc.close();<BR>
</UL>
}<BR>
|
The important thing about this function is that it can be called at any time, regardless of what the background thread is doing. The background thread's activities and user navigation are effectively decoupled, which was our original goal. Whenever the user clicks on a link in the Select frame, this routine is called, and the most recent data, as of the last set of data loaded by the background thread, is displayed immediately in the Portfolio window.
The Buy/Sell frame is not central to the techniques described here, but has been added to demonstrate the possibilities for client-side user interactivity. This frame allows you to buy or sell shares of the currently selected stock.
Conclusion
Admittedly, this is an unusual use of JavaScript. But all we have done is to treat JavaScript like the thing it is -- a fully functional programming language. If JavaScript doesn't have what you need -- threading, or remote procedure calls, for example -- you can always create it yourself. The best way to get a feel for the results of this technique is to try out the example program. You can do this by clicking here. Navigate around by pressing the links in the Select frame, or buy and sell shares in the Buy/Sell frame. Watch the message strip at the bottom -- it will alert you when the background thread is active and loading data from the server.
It should be clear that this interface has a more responsive feel than what is usually encountered on the Web. It doesn't suffer from any of those annoying CGI pauses we are so used to. It's not perfect, however -- the Portfolio frame does tend to flicker each time the data is loaded. In the example, the server script is being called every five seconds, but for many applications, a longer delay can be used, which will decrease the visible flickering. The delay depends heavily on the particular application and on the data.
Greg Travis is a freelance programmer in New York City. He can be contacted at mito@panix.com.
Events
