Browser, Browser Not

Originally published at O’Reilly

Recently, O’Reilly published a set of articles (Netscape Navigator 6.0 to Fail Standards ComplianceAn Update, and Netscape 6.0 Released), written by the popular author David Flanagan, about the release of Netscape 6.0, Netscape’s newest entry in the browser marketplace.

David presented several valid concerns about bugs still present in the release of Netscape 6.0. And it is true, Netscape 6.0 did release with several unfixed bugs, many of which will have an impact on support for W3C specifications.

Our reaction to the release, however, was somewhat different. Along with other application developers, we’ve been waiting for the public release of an application that uses Mozilla’s XPToolkit, a set of software components from which Netscape 6.0 and the upcoming Mozilla 1.0 were built. Now that Netscape 6.0, which uses this framework, has been publicly released, we’re delighted: testing of XPToolkit may begin in earnest.

While many are focused on the release of Netscape 6.0, some of us aren’t. We’re more interested in the application environment created by the Mozilla team to support the implementation of browsers in general. To us, this framework is more important than the release of a new browser will ever be.

The reason for this is the changing face of the Internet, itself.

The Changing Face of Internet Applications

Current Internet applications rely on a centrally located Web server to distribute HTML over HTTP to clients. Each client, or Web browser, renders the source and displays a human-readable page.

This architecture has become so popular that you can’t pick up a magazine or a newspaper without hearing about Web servers or the new business models based on them. Although this architecture is based around universally located resources, most application-level resources are centralized and many other resources are hard to find. Some Web sites help you find other Web sites or “resources.” Others go so far as to offer completely centralized applications, as Application Services Providers (ASPs).

New technologies will soon force us to rethink the way we use the Internet. Distributed systems, mobile agents, and peer-to-peer (P2P) applications may completely undermine the need for browser-based Internet access.

P2P applications are already stepping around the browser. The next step will be around the Web server.

Consider this: a P2P application that locates and downloads a new function. The simplest example here may be provided by a P2P execution framework that uses XML-based remote procedure calls between peers to marshal XML-encoded functions. Instead of hitting Web pages, each peer locates and accesses both data and functions among a network of peers. No Web servers.

This scenario is not going to be best served by the traditional browser. Why?

The Limitations of Browsers

The things that made the Web browser a success in the beginning are the things that make it ineffective for new application models.

The browser was built to render files stored on Internet sites so we didn’t have to muck about with FTP. As soon as content became more visible, people started publishing yet more content, so browsers rendered HTML, then XML, formatted with CSS or XSLT. However, the browser itself has a very limited interface, even with new advances in W3C specifications. Sophisticated browser pages mean using either complicated object models–leading to cross-platform and cross-browser idiosyncrasies that are usually the result of standards initiatives–or using page-embedded applications, such as Java applets and plug-ins.

Even when the browser follows standard specifications, working within a browser page to create a sophisticated interface isn’t a simple or uncomplicated task.

In addition to the browser becoming increasingly complex as the nature of content becomes so, use of it implies that applications ought to be served from one location, and in one manner. To do something such as make a remote procedure call, you would need to use a digitally signed Java applet or some other browser-specific and limited technique. This is something that won’t bother newer P2P applications.

Finally, browsers were designed to be safe, and operate in a protective sandbox. Web-based applications served via a browser have difficulty getting at the user’s machine. Though safe, this restriction also prevents behaviors that would have the application modify its user interface. And this dynamism is going to be necessary in an environment where new services require new application interfaces that can be downloaded as data.

An Internet Application Framework?

Mozilla made a tough decision a few years ago–to scrap the Netscape 4.x architecture in favor of one built from the ground up. In the process, this open source team created an application environment based on reusable and interchangeable components.

With this application environment in place, the team then proceeded to build a sophisticated browser. They threw in Internet Chat, a Web page composer, and other complex things, all of which were released recently as Netscape 6.0. Often forgotten is that a powerful application environment came with it. This environment is now usable by developers of other Internet applications.

What types of applications? Well, ActiveState, the company that provides popular implementations of Perl and Python for various operating systems, used Mozilla to create itsKomodo product, a visual IDE for working with Python and Perl code. The user interface provides, among other things, colored syntax, syntax checking, and source-level debugging.

So, we have a browser and an application that can be used to create and test Perl and Python applications, all built from the same application architecture.

This is exciting stuff! Much has been written about reusable code and component-based design, and now we have an open source application environment we can all use to build our own applications.

Even more exciting is the extensible user-interface language from Mozilla called XUL (pronounced “zool”). It’s based on XML, which means you can use XML to create a user interface. Combine this with the ability to make remote procedure calls, and you have a perfect place from which to commence building a bunch of P2P applications, based on the scenario mentioned above.

Now, instead of opening a browser, you can open an application built on the same framework as your browser, but with a sophisticated interface of dropdown menus and tabbed pages–all created using XML. You can access remote procedure calls at the touch of a button and when you’re ready to access a new service, click another button, and in a couple of minutes restart your application. New entries will be added to new or existing menus providing access to the new service. All this is accomplished without Java bytecode, a new plug-in, or a DLL.

You’ve just downloaded XML.

When you explore the possibilities of the XPToolkit from Mozilla maybe you’ll agree that Netscape 6.0 is more than just a standards-based, better-than-Navigator-4.x-browser. It’s the start of a new new way of doing things on the Internet.

Digital Play Dough: Designing Applications with XUL

Originally published in Web Techniques

The XML-Based User Interface Language (XUL) made its first appearance with the release of Mozilla, the Open Source browser used as the foundation for Netscape 6. Pronounced “zool,” the language gives developers and designers an easy way to describe the design and layout of application windows in Web browsers. By modifying a few files, you can change the entire look of your Web browser or of the windows that pop open while a visitor browses your site. Prior to XUL, this was only possible by modifying and re-compiling the browser’s underlying source code. And in that case, you would have to distribute the modified browser to all your site’s visitors — an unlikely event. Fortunately, all you need to change the look and feel of a Web browser today is an understanding of the XML and CSS specifications and a little ingenuity.

Architecture

XUL applications consist of XML files created with .xul extensions. The files define the content of the application. Additional application data is located in Resource Description Framework (RDF) files. CSS files provide formatting, style, and some behavior, for the application. JavaScript files provide scripting support. Multimedia files, such as PNG images and other audio/visual files, might also be needed for additional user interface information. All of the file types are specifications recommended by the W3C, and collectively are referred to as the XUL application’s “chrome” — the contents, behavior, and appearance of the application’s user interface.

The Mozilla browser is itself designed as an XUL application. To manage the chrome for your browser, both Mozilla and Navigator have subdirectories labeled chrome, located off each browser’s main directory. Within the chrome directory, separate XUL applications are packaged into separate subdirectories. Within each application directory, subdirectories further divide the application into content (containing the XUL, JS, and RDF files), skin (CSS files), and locale (DTD files).

To deploy your own XUL application on the Web, you can either place all of the files within the same subdirectory on a Web server, or use the suggested chrome directory structure on the server. Note though that you may lose some functionality, such as localization, when your application is not using the chrome directory structure. Also, all files should be local to the URL of the main XUL file, otherwise the application may not work due to violations of built-in security.

A Simple Application

To demonstrate the structure of an XUL implementation, I created an application that is essentially a window with two panes — one on the left with a menu of hypertext links, and one on the right where a Web page can be viewed. Figure 1 shows the application.

To understand how I created this application, look at the sample XUL file shown in Listing 1. The first line of an XUL file is the XML declaration, which includes a reference to the version of XML used. Following that, the file must include a reference to a CSS file, to provide formatting for the XUL contents. The file then defines the application window and sets several properties — namely, the title, width, and height. Namespaces are provided for the application’s elements. By default, all elements in the application are XUL elements, identified by the following namespace (required for all XUL files):

xmlns=”http://www.mozilla.org/keymaster/
gatekeeper/there.is.only.xul”

When completed, the application will use some standard HTML elements, and these are identified with the html namespace. Additionally, the application will have a few elements unique to the specific application, and these are identified by the Webtechniques namespace WT.

Widgets

The different sections of a XUL application are contained in widgets known as “boxes.” Boxes don’t have any visual appearance themselves; their only purpose is to encapsulate several widgets as a whole, and to provide layout orientation for their contents. The orientation is defined with the orient attribute, and other attributes can be used to set the box width and height.

To demonstrate the use of boxes and page layout, Figure 1 shows a first version of the application, with buttons on the left, and a space to display the content on the right. The XUL file for this page is found in Listing 2.

In the listing, I added a box to the window contents and gave it an orientation value of vertical. This orientation forces all of the content to be positioned vertically in the pane. Additionally, the box is given a flex value of 1. The flex attribute can be applied to several different types of XUL elements. It’s used to help the layout engine determine how to expand an element to fill available space. For the outermost box in the page, the contents expand to fill the entire window.

In Listing 2, the outer box contains a toolbar and another, nested box. The toolbar and this new box are layered vertically inside the outermost box.

The nested box is given a horizontal orientation and contains two more nested boxes. These boxes are displayed left to right, and form the main panes of the application. Each of these boxes is given a vertical orientation, which means their contents will be aligned from top to bottom. A XUL splitter is used to separate the contents of the two boxes. This allows the user to resize either side of the application page by dragging the splitter either to the left or to the right.

Notice in the code that the first content box doesn’t have a set flex value. This means the box will be sized according to the width and height of its contents. The second box has a flex value of 1, and it expands to fill the remaining horizontal space.

The first box contains several XUL buttons, surrounded by widgets called springs. The spring element is used to help position elements, and has no visual characteristic of its own other than to occupy space. In this example, a spring is used on either side of the page buttons, to center the buttons within the box.

Also notice in the listing that the second box contains an HTML IFRAME element. As each listing for an article is accessed, the contents of the listing (a Web page) are opened in this IFRAME element. Because the element is an HTML element and not a defined XUL widget, the html namespace must precede the element tag.

More Widgets

The application terminates when the user clicks on the Close Window button in the application’s toolbar. To add a button widget to a page, you’d use the button element as follows:

<button value="Close Window" onclick="window.close();" />

The default appearance for the button is provided by CSS files that are included with Mozilla/Navigator.

To provide feedback to the user whenever a page is being loaded in the right pane, I have decided to extend the code by adding a progressmeter widget (see Figure 2). The progressmeter widget can have a mode of determined or undetermined. A determined meter is one in which the developer knows the exact length of time of an operation, and controls the meter accordingly. The undeterminedmeter is used when the length of time for an operation is unknown. Accessing a Web page falls into the undetermined category.

To support the use of the meter, I add JavaScript code to activate the meter. To do this, I put my code in a JavaScript .js file, and save the file in the XUL application’s contents directory. The code is shown in Listing 3.

I then create a reference to the JavaScript file in the XUL document. When doing this, I have to modify the onclick event handlers for the menu buttons so that they call the JavaScript loadItem()function instead of jumping directly to the URL. Listing 4 contains the contents of the second version of the XUL application. Now, when the application loads a Web page, the meter signals that a page is loading.

Integrating RDF Files

In the previous versions of the XUL application, the URLs of the target Web sites were hard coded into the page. A better approach would be to add the URLs as application data from RDF files. RDF files are used with XML to define application specific data, and can be altered easily without the need to modify the main XUL application pages.

As shown in Listing 5, many of the pages in an RDF list can be from the same site, and are nested by URL location.

To facilitate working with RDF data, XUL has a template engine that integrates external data with the XUL application. The template engine is activated when the layout engine finds opening and closing template elements, and all of the contents contained within the tags then form the template pattern. This pattern is used to determine how to lay out repeating values found in the RDF file.

We’ll add a template to the XUL application and set the pattern to be one button for every listed item in the RDF file. To do this, the RDF data source file must first be attached to the XUL file, and this is done in the box surrounding the template with the following command:

<box orient="vertical" flex="1" datasources="sites.rdf" ref="urn:source:data" >

The datasources attribute specifies the RDF file, and the ref attribute refers to the location in the RDF file to pull in the sequence of data.

The template, given in Listing 6, contains a button widget with the attribute of uri="rdf:*", which indicates where in the RDF file to begin the template matching. An application specific attribute (with the WT namespace) is added to the button to capture the URL associated with the Web page. The page’s title will be shown as the button’s display value.

The JavaScript loadItem() function from Listing 3 must be modified to pull the URL for the Web site as an attribute of the item passed in from the event handler. The new function is given inListing 7.

Figure 3 shows the page that results from using the RDF file and the XUL template. Note that each nested item from the RDF file is actually embedded within the button of its containing parent. So, you see a top button containing two other buttons, and so on.

Trees

Buttons aren’t necessarily the best widget to use when designing an application that uses a repeating template. Other widgets that are better are menus and trees. An XUL tree is similar to an HTML table in that the tree is delimited with one tag (tree), the rows with another (treerow), and the tree contents with yet another (treecell). The treehead tag defines a header for the tree, and the treeitem tag is used to allow users to click and select tree items.

Unlike an HTML table, though, XUL trees provide sophisticated processing for nested items. By default nested items aren’t displayed until the user clicks on an item’s containing parent.

In Listing 8, I’ve replaced the buttons with an XUL tree structure. Individual tree cells hold each hypertext link, and nested data items are displayed as nested tree items. Clicking on the graphic associated with the top-level tree items (known as the “twisty”) displays the contained items. Clicking on any contained item opens the associated Web page in the right pane of the application, as displayed in Figure 4.

Skinning

One of the more controversial aspects of XUL is skinning — the ability to change the appearance of any of the application’s chrome, including the frame, the buttons, the toolbars, menus, and so on. You can change the entire look and feel of Mozilla and Navigator 6 just by downloading new chrome packages and installing their contents. With XUL, the only limit to skin design is the designer’s imagination.

So why is giving Web developers the ability to design different interface skins so controversial? Imagine for a moment that Salvador Dali or Picasso painted frames rather than the canvases. Now, try to visualize what painting could hold its own within these frames. Not many could. Application designers argue that well-designed user interfaces could be ruined by a third party’s poorly-designed skins. The other side of this argument is that being able to create a custom user interface is an attractive concept for developers and designers who don’t want to have to work within the rigid confines of the widgets defined by current operating systems.

To create a new skin for a XUL application, you provide a custom CSS file that defines the styles for the widgets in your application. If you wish to have access to the default appearance and behaviors for the widgets, you must import the global styles into your new CSS file with the following command:

@import url(chrome://global/skin/);

You can then add styles for the widgets used in your application.

Listing 9 shows the CSS file for the XUL application used in this article. In the file, I changed the style of the splitter so that it has a maroon background when inactive, and a teal one when active. I also changed the appearance of the button on the splitter (called the “grippy”). I modified the meter to have a gold background, and did the same for the selection highlight on the tree. I even added new GIF images for the toolbar grippy (the little downward arrow that’s used to hide or show the toolbar).

In this example, I modified the styles for XUL widgets directly in the CSS file. However, the preferred approach is to add new widget classes, and use them in your application as follows:

button.wt_button { ... }
<button class="wt_button".../>

With this approach, you still have access to the default appearance and behavior for each widget.

Once I modified the widgets, I opened my main XUL application file and replaced the reference to the global CSS file with a reference to the new CSS file, using the following line:

<?xml-stylesheet href="wt.css" type="text/css"?>

To access the completed application, create a page that has the following link on it, and load that page in Mozilla or Navigator 6:

<a href="#" onclick="window.open('wt.xul','example','chrome');">test</a>

If you use PR 1 of Navigator 6, note that the toolbar won’t show properly due to changes in the XUL schema that occurred after Navigator 6 PR 1 was released. It should work correctly with PR 2, and works with Mozilla build M16 (and up, hopefully).

One more thing to note is that you may have to add the XUL MIME type to your server’s configuration in order to make sure the application XUL page downloads correctly. The MIME type is simple text/xul.

Conclusion

You’ll find the complete application with all necessary files in the wt.zip file. With just a few changes of the XML file associated with the XUL application — adding a meter and using a tree instead of buttons — I changed application functionality and provided better user feedback and a better design. With a few adjustments to the CSS file for the application, I was able to create a new and different look to go with my application’s functionality. XUL promises to be a powerful tool for Web application development and deployment. I look forward to seeing future iterations and refinements of the technology in future browser versions.

PerlScript: A hot ingredient for ASP

Originally published at Web Techniques

Microsoft’s Active Server Pages (ASP) technology has grown very popular as a Web server-side technique, combining HTML, embedded scripts, and external components to deliver dynamic content. One feature of ASP is that different languages can be used for its scripting portion, though the most widely used ASP scripting language is VBScript—a subset of Microsoft’s Visual Basic.

However, just because VBScript is the most popular language used by ASP developers doesn’t mean it’s the only one, or even the best one to use in a particular circumstance. For instance, Perl has long been synonymous with Web server development, beginning with the earliest uses of the language by CGI, and is still one of the most popular languages among Web developers. But there hasn’t been much discussion of using Perl with ASP.

If your organization has been working with Perl, and you’re interested in developing for the ASP environment, you don’t have to give up your favorite language (or your existing Perl code) to make the transition to the new technology—you can employ Perl for ASP scripting through the use of PerlScript.

A (Very) Brief Overview of ASP

Microsoft originally introduced ASP technology was with the company’s own Web server, Internet Information Server (IIS). However, ASP has now been ported to other Web servers through the use of software such as ChiliSoft’s platform-independent version of ASP. In addition, ASP was created originally to work in a Windows environment, but again thanks to ChiliSoft and other companies, ASP now runs in non-Windows environments such as UNIX and Linux.

Still, the most popular use of ASP is within the Windows environment, with pages hosted on IIS. This environment—specifically Windows NT/IIS 4.0—is the one I’ll discuss.

ASP pages have an .asp extension, and are a mix of HTML and embedded script. When a client requests the page, the embedded script is accessed and processed. The results generated by the script are embedded into the Web page, which is then returned to the client browser.

The Response object—along with the other built-in ASP objects Server, Session, Request, and Application—provides access to the ASP and application environment for the ASP developer. The Response object provides a way to send information back to the Web-page client; the Request object provides access to information sent from the client, such as form contents; the Application object contains information that persists for the lifetime of the ASP application; the Session object contains information that persists for the lifetime of the particular ASP user session; and the Server object, among other things, lets the ASP developer create external components.

ActivePerl and PerlScript

A company named ActiveState was formed in 1997 to provide Perl tools for developers for all platforms. Among some of ActiveState’s more popular products is ActivePerl, a port of Perl for the Win32 environment.

ActivePerl is a binary installation containing a set of core Perl modules, a Perl installer (Perl Package Manager), an IIS plug-in for use with Perl CGI applications, and PerlScript. What is PerlScript? PerlScript is Perl, but within an ASP environment—it’s a Perl implementation of the Microsoft Scripting Engine environment, which allows Perl within ASP pages.

ActivePerl can be downloaded for free from ActiveState’s Web site (see the ” Online Resources“). To try it for yourself, access the ActiveState Web site and find the ActivePerl product page.

The installation process requires very little user input. You’ll need to specify where you want to put the files, and be sure to select the option to install PerlScript.

Using PerlScript Within ASP Pages

By default in IIS, all ASP scripting is VBScript, but you can change this using one of three different techniques. First, you can change the default scripting language for the entire ASP application using the IIS Management Console, and accessing the Properties dialog box for the Virtual Directory or Site where the ASP application resides. Click on the Home Directory (or Virtual Directory) tab, and then click on the Configuration button on the page. From the dialog box that opens, select the App Options page. Change the default scripting language from “VBScript” to “PerlScript.”

A second technique is to specify the scripting language in the scripting block itself. With this technique you could actually choose more than one scripting language in the page:

<SCRIPT language=”PerlScript” RUNAT=”Server”> . . . </SCRIPT>

A third technique is to specify the scripting language directly at the beginning of the Web page. This is the approach we’ll use for examples. Add the following line as the first line of the ASP page:

<%@ LANGUAGE = PerlScript %>

All scripting blocks contained in the page are now handled as PerlScript.

Accessing Built-In ASP Objects From PerlScript

To assist the developer, PerlScript provides access to several objects available only within the ASP environment. As mentioned earlier, these are the Application, Session, Response, Request, and Server objects.

The Application object is created when an ASP application first starts, and lasts for the lifetime of the application. The object has two COM collections, both of which can be accessed from script: the StaticObjects collection, with values set using the <OBJECT> tag within an application file called global.asa; and the Contents collection, which contains values set and retrieved at runtime. A COM collection is a group of similar objects treated as a whole, with associated features that let the developer access individual elements directly, or iterate through the entire collection.

In addition to the two collections, the Application object also has two methods, Lock and UnLock, which are used to protect the object against attempts by more than one application user at a time to change its values.

We’ll take a closer look at using the Lock and UnLock methods by setting a value in the Application’s collection in one ASP page, and then retrieving that same value from another page.

First, Listing 1 contains a page that sets a new value to Application Contents by first locking down the object, setting the value, and then unlocking the object. Notice that you don’t have to create the Application object—it’s created for you, and exists in the main namespace of the PerlScript within the ASP page (the same holds true for all of the ASP objects).

If you’ve worked with VBScript you’ve probably noticed that you have to use a different technique to set the value in the Contents collection with PerlScript. VBScript allows direct setting of collection values and properties using a shorthand technique, similar to the following:

Application.Contents("test") + val

PerlScript, on the other hand, doesn’t support this shorthand technique. Instead, you have to use the SetProperty method to set the Contents item:

$Application->Contents->SetProperty('Item', 'test', $val);

Additionally, you have to use SetProperty to set ASP built-in object properties with PerlScript, or you can use the Perl hash dereference to set (and access) the value:

my $lcid = $Session->{codepage};

Listing 2 contains another ASP page that accesses the variable set in Listing 1, prints out the value, increments it, and resets it back to the Application object. It then accesses this value and prints it out one more time. Accessing this page from any number of browsers, and from any number of separate client sessions, increments the same Application item because all of the sessions that access the ASP application share the same Application object.

In addition to the Application object, there is also a Session object, which persists for the lifetime of a specific user session.

This object is created when the ASP application is accessed for the first time by a specific user, and lasts until the session is terminated or times out, or the user disconnects in some way from the application. It, too, has a StaticObjects and a Contents collection, but unlike the Application object, you don’t lock or unlock the Session object when setting a value in Contents. But you do access the Contents collection in the exact same manner, both when setting a value:$Session->Contents->SetProperty('Item', 'test', 0);

as well as when retrieving the value:

my $val = $Session->Contents('test');

Additional Choices

There are also other properties and methods available with Session, including the Timeout property, used to set the session’s timeout value, and the Abandon method, used to abandon the current session. Each session is given a unique identifier, SessionID, and this value can be accessed in a script. But use caution when accessing this value if you hope to identify a unique person—the value is only unique and meaningful for a specific session.

In support of internationalization, there are Session properties that control the character set used with the page, CodePage, and to specify the Locale Identifier, LCID. The Locale Identifier is an international abbreviation for system-defined locales, and controls such things as currency display (for instance, dollars versus pounds).

Listing 3 shows an ASP page that sets the Timeout property for the Session object, and accesses and prints out both the CodePage and the LCID values. Retrieving this ASP page in my own environment and with my own test setup, the value printed out for CodePage is 1252—the character mapping used with American English and many European languages. The value printed out for LCID is 2048—the identifier for the standard U.S. locale.

The Application and Session examples used a third ASP built-in object, the Response object. This object is responsible for all communication from the server application back to the client. This includes setting Web cookies on the client (with the Cookies collection), as well as controlling whether page contents are buffered before sending back to the client, or sent as they’re generated, through the use of the Buffer property:

$Response->{Buffer} = 1;

You can use the Buffer property in conjunction with the End, Flush, and Clear methods to control what is returned to the client. By setting Buffer to true (Perl value of 1), no page contents are returned until the ASP page is finished. If an error occurs in the page, calling Clear erases all buffered contents up to the point where the method was called. Calling the End method terminates the script processing at that point and returns the buffered content; calling the Flush method flushes (outputs) the buffered contents, and ends buffering.

Listing 4 shows an ASP page with buffering enabled. In the code, the Clear method is called just after the first Response Write method, but before the second. The page that’s returned will then show only the results of the second Write method call.

The Buffer property must be set before any HTML is returned to the client, and this restriction also applies to several other Response properties, such as the Charset property (alters the character setting within the page header); the ContentType property (alters the MIME type of the content being returned, such as “text/html”); the AddHeader method, which lets you specify any HTTP header value; and the Status property, which can be used to set the HTTP status to values such as “403 Forbidden” or “404 File not found”. For example:

$Response->{Status} = "403 Forbidden";

If buffering is enabled for a Web page, then properties such as Charset and ContentType, as well as the AddHeader method can be used anywhere within the ASP page.

You can redirect the Web page using the Redirect method call to specify a new URL. As with the other properties and methods just mentioned, Redirect must also occur before any HTML content:

$Response->Redirect("http://www.somesite.com");

In addition to manipulating HTTP headers, the Response object also generates output to the page using the Write method, as demonstrated in the previous examples. You can also return binary output to the client using BinaryWrite. You can override whether an ASP page is cached with proxy servers through the use of the CacheControl property, as well as set cache expiration for the page by setting the Expires or the ExpiresAbsolute properties:

$Response->{Expires} = 20 # expires in 20 minutes

You can test to see if the client is still connected to the setting with the IsClientConnected property. Communication doesn’t just flow just from the server to the client. The Request object handles all client-based communication in either direction. This object, as with Response, has several different methods, properties, and collections. The collections interest us the most.

You can read Web cookies using the Request Cookies collection. And it’s possible to set and get information about client certificates using the ClientCertificate collection.

You can also process information that’s been sent from a client page using an HTML form, or appended as a query string to the Web page’s URL:

<a href=”http://www.newarchitectmag.com/documents/s=5106/new1013637317/somepage.asp?test=one&test2;=two”>Test</a>

The two collections that hold information passed to the ASP page from a form or a query string are: the QueryString collection, and the Form collection. Use QueryString when data is passed via the GET method, and use Form when data is passed via the POST method.

Regardless of which technique you use to send the name/value pairs, accessing the information is similar. Listing 5 shows an ASP page that processes an HTML form that has been POSTed:

The example ASP page pulls out the values of three text fields in the form: lastname, firstname, and email. It then uses these to generate a greeting page, including adding a hypertext link for the email address. If the form had been sent using the GET method (where the form element name/value pairs get concatenated onto the form processing page’s URL), then the page contents would be accessed from QueryString:

my $firstname = $Request->QueryString('firstname')->item;In addition to the Form and QueryString collections, the Request object also has a ServerVariables collection, containing information about the server and client environment. The ServerVariables collection is comparable to accessing %ENV in CGI.

You can access individual elements in the Server Variables collection by specifying the exact variable name when accessing the collection:

my $val = $Request->ServerVariables('PATH_TRANSLATED')->item;Or you can iterate through the entire collection. To do this, you can use the Win32::OLE::Enum Perl module to help you with your efforts. The Enum class is one of the many modules installed with ActivePerl, and provides an enumerator created specifically to iterate through COM collections such as ServerVariables.

Listing 6 shows an ASP page that uses the Enum class to pull the elements of the ServerVariables collection into a Perl list. You can then use the Perl foreach statement to iterate through each ServerVariables element, printing out the element name and its associated value.

If an HTML form contains a File input element—used to upload a file with the form—you can use the Request BinaryRead method to process the form’s contents. The TotalBytes property provides information about the total number of bytes being uploaded.

Break Out with COM

All of the examples up to this point have used objects that are built in to the ASP environment. You can also create COM-based components within your ASP pages using the built-in Server object. The Server object has one property, ScriptTimeout, which can be used to determine how long a script will process—a handy property if you want to make sure scripting processes don’t take more than a certain length of time.

The Server object also has a couple of methods that can be used to provide encoding, such as HTML encoding, where all HTML-relevant characters (like the angle bracket) get converted to their display equivalents. MapPath maps server paths relative to their machine locations. URLEncode maps URL-specific characters into values that are interpreted literally rather than procedurally:my $strng = $Server->URLEncode("Hello World");The result of this method call is a string that looks like:

Hello+World%21Although these methods and the one property are handy, Server is known generally as the object used to instantiate external ASP components, through the use of the CreateObject method. This method takes as its parameter a valid PROGID for the ASP component. A PROGID is a way of identifying a specific COM object, using a combination of object.component (sometimes with an associated version number):

simpleobj.smplcmpntAs an example, I created a new Visual Basic ActiveX DLL, named the project simpleobj, and the associated class smplcmpnt. The component has one method, simpleTest, which takes two parameters and creates a return value from them, based on the data type of the second parameter. This component method, shown in Example 1, has a first parameter defined as a Visual Basic Long value (equivalent to a Perl integer), and a second parameter of type Variant, which means the parameter could be of any valid COM data type—Visual Basic functions are used to determine the data type of the value.

A new page uses the Server CreateObject method to create an instance of this ASP component, and tests are made of the component method. As shown in Listing 7, the first test passes two integers to the external VB component method. The component tests the second parameter as a Long value, adds the two parameters, and returns the sum.

The next test passes a string as the second parameter. The component tests this value, finds it is not a number, and concatenates the value onto the returned string.

The script for the final test creates a date variable using the Win32::OLE::Variant Perl module, included with ActivePerl. The standard localtime Perl method is used to create the actual date value, but if this value isn’t “wrapped” using the Variant module, the Visual Basic ASP component will receive the variable as a string parameter rather than as an actual date—PerlScript dates are treated as strings by ASP components.When the Visual Basic component receives the date as the second parameter, the component finds that it is not a number, and concatenates the value onto the string returned from the function. When displayed, the returned string looks similar to the following:

3/15/1905 100I could have passed the date value directly instead of using Variant, but as I mentioned, the COM-based VB component sees the Perl date as a string rather than as a true date type. The Variant Perl module provides techniques to ensure that the data types we create in PerlScript are processed in specific ways in ASP components.

Summary

Perl is a mature language that has been used for many years for Web development. As such, there is both expertise with, and a preference for, using this language for future development with the newer Web development techniques such as ASP.

ActivePerl and PerlScript are the key tools for using Perl within the ASP environment. Perl can be used for ASP scripting through PerlScript, but the Perl developer also has full access to the objects necessary to work within the ASP environment: namely the ASP built-in objects such as the Response and Request objects.

Additional modules to assist Perl developers—such as Win32::OLE::Enum and Win32::OLE::Variant—are included with the ActivePerl installation, and help make PerlScript as fully functional within the ASP scripting environment as VBScript.

Best of all, with ActivePerl and PerlScript you can develop within an ASP environment and still have access to all that made Perl popular in the first place: pattern matching and regular expressions, the Perl built-in functions, and a vast library of free or low-cost Perl modules to use in your code. Interest in ASP is growing, and with PerlScript you can work with this newer Web technology and still program in your favorite programming language.

Creating a Shopping Cart ASP Component

Originally published in ASP Today, October 20, 1999

As soon as HTML forms were added to the HTML specification, and CGI use extended to server-side applications, folks immediately thought of using the Web for online stores – the concept of the shopping cart was born. If you’ve ever done any online shopping, you’ve used a shopping cart.

A shopping cart is basically a small application that maintains a list of the Web shopper’s selections in such a way that they can be viewed and modified at any time. By itself, the cart is a fairly simple application, but folks end up rolling an inventory control system, an order processing system, a customer service system and the overall Web site maintenance into one application with the misnomer “shopping cart”. What should be a small, compact system sprawls into something large and difficult to manage.

The way a cart keeps tracks of items and persists the list of those items, tends to differ from implementation to implementation: they might be tracked for a single session, or persist from session to session. The cart itself can be created on a specific computer or be accessible from many computers. Although a shopping cart application can interface with other applications, like an order system, an inventory system, or general Web maintenance, it does not implement this functionality itself:

This article will look at the creation of a simple ASP-based shopping cart application that uses a Visual Basic shopping cart component – all of which can be found in the download at the end of the article, with a text file telling you what you’ll need and how to use it.

Shopping Cart Implementation Requirements

First and foremost, a shopping cart has to persist from Web page to Web page, so some sort of technique needs to be used to associate an identifier with a shopping cart and then persist that identifier between Web pages. The items contained in the cart do not need to be accessible from all pages, but the cart identifier does. Because of this essential functionality, shopping carts are usually dependent on Web client Cookies to maintain the link between the cart and the shopper.

There are other techniques that can be used to persist information about the cart between pages. Some developers use hidden form fields (containing the shopping cart identifier or possibly a string of the cart’s items), or add shopping cart information to the end of the URL of the new page that is being accessed. Both of these techniques make the cart information available in the new page. To use these techniques in an ASP shopping cart, you could get the hidden form fields from the Forms collection of the Request built-in ASP object; or the appended URL information from the QueryString collection that is, again, part of the Request object.

If a store chooses to support a shopping cart for a session only (carrying information from the user’s initial store access until they log out, close their browser or disconnect from the Internet), then either hidden form fields or query string method will work fine. However, if cart needs to persist beyond the session, then Cookies are the way forward. Using Cookies, the Web developer can persist the shopping cart for the session or for a specified period of time.

Using Cookies alone, the shopping cart and its contents are maintained solely on the client, so the cart’s contents can be accessed quickly. There is, unfortunately, a major limitation with this approach – Cookies can only hold so many items, usually not many more than 50 – 75. Should the cart need to hold more than this, then Cookie technology on it’s own simply isn’t a viable approach, because the Cookie string can get too large. Even a couple of items can create a large Cookie string.

There is another problem with Cookie technology: the cart can’t follow the shopper. So if a shopper starts a cart on their laptop then they have to keep it there: they can’t access the same cart from their desktop computer at home, or a computer at work, because Cookies don’t travel. The shopper could export the cookies to all of their computers if the browser they’re using supports this technology, but most folks don’t consider exporting cookies and many aren’t even aware of this capability.

The solution, then is to bring the shopping cart to the computer and have it create whatever Cookies it needs to be supported in the new environment.

The Ideal Shopping Cart

The ideal Shopping cart that we could implement using ASP and Visual Basic, for the purposes of this article, will allow:

  • An item to be added to a shopping cart at the touch of a button
  • Shopping cart items to persist for more than one shopping session
  • Some indication that there are items in the shopping cart to be displayed, at least on the site’s home page
  • The Web shopper to view the shopping cart contents at any time, and the contents to be displayed whenever an object is added to the cart
  • The store to provide a means to modify the shopping cart items: to remove an item / all items
  • A running total to be maintained each time the shopping cart contents are reviewed
  • The shopping cart to follow the client
  • Support for an indefinite number of items

In order to create the shopping cart, we must first create the cart database support and the cart Visual Basic component project.

Setting up the Cart’s Environment

Instead of implementing all the aspects of the shopping cart within ASP script, we’ll implement the business logic within a Visual Basic component, use stored procedures for database access, and integrate the shopping cart into the ASP infrastructure within the script. Using this separation of functionality will isolate the data access and database structures from the business logic, and isolate the business logic from the implementation environment.

The tables to support the shopping cart are simplified to include only that information necessary to implement the cart: CART , CARTITEM , CUSTOMER , and CUSTOMERCART , and WIDGET . The WIDGET table represents the product table for this example. The table CARTITEM is dependent on both CART and WIDGET , and the CUSTOMERCART is dependent on both CUSTOMER and CART , so foreign key relationships exist between these tables.

The SQL to create the tables within a SQL Server 7.0 database, and the associated indexes and foreign keys are included with the download example code attached to this article.

In addition to the tables, several stored procedures are used to manage data access. Each of these procedures will be described as they are accessed by specific business routines in the sections ahead; but before we can add the methods to implement the business logic, we’ll need to create the Visual Basic project.

Creating the Cart Project

The shopping cart component is created as a new, ActiveX DLL Visual Basic project named, appropriately enough, shopcart . References to the Microsoft ActiveX Data Objects (ADO 2.1 for this example) Library and the Microsoft Transaction Server (MTS) Type Library are added to the project. We’re adding in support for MTS so the component will be able to access the ObjectControl interface, and we’re adding in support for ObjectControl in order to enable just-in-time activation for the shopping cart component.

Some basic component functionality is added, including the use of Option Explicit at the top of the class file, and the ObjectControl implementation. The only functionality added to the ObjectControl methods (Activate, Deactivate, CanBePooled) at this time is to define a member that holds the Database connection string.

Option Explicit

' The connection string is available to all shopping cart methods
Private m_connString As String

'Implementation of ObjectControl interface
Implements ObjectControl

' ObjectControl Methods
Private Sub ObjectControl_Activate()
    m_connString = "driver=
{SQLServer};server=FLAME;database=writing;uid=sa;pwd="

End Sub

Private Function ObjectControl_CanBePooled() As Boolean
    ObjectControl_CanBePooled = False
End Function

Private Sub ObjectControl_Deactivate()
 ' no activity
End Sub

Because there is a lot to get through in this article, I’m not going to go into the ObjectControls method (these areas will be covered in another article, coming soon – Ed ). The next thing to add is the functionality specific to our implementation of the shopping cart, starting with the methods to create the shopping cart and to add an item to the cart.

Adding Methods to Create a Cart and Add Items

The first requirement of the shopping cart is that users can add items to it, and implicitly, the ability to create a cart itself. We’ll implement both of these requirements as methods.

A cart can be created either when a shopper first accesses a site, or when the shopper makes an initial move to adding an item to the cart. The cart I’m going to build here will take the second approach, will only be created if none already exists, and will be created through a method called in the page that displays the shopping cart contents.

You can add a new cart through a stored procedure called SP_NEWCART , which adds a new record to the CART table, and returns a unique cart identifier:

CREATE PROCEDURE [SP_NEWCART] 
AS
BEGIN
insert into cart (date_created)
values (getdate())
select max(cart_id) from cart
END

To access this stored procedure, a function named createCart , having no parameters and returning a LONG value, is added to the cart component. The returned value is the new shopping cart identifier returned from SP_NEWCART .

Within createCart , new Command and Recordset objects are created and a connection string is added to the ActiveConnection property of the Command object. In addition, the Command CommandType is set to adCmdStoredProc and the stored procedure name is assigned to the Command object’s CommandText property.

' createCart
' Generate shopping cart ID
' create cart without customer association
'
Function createCart() As Long

  Dim comm As New Command
  Dim rs As New Recordset

  ' open connection, attach to Command object
  comm.ActiveConnection = m_connString
  
  ' set Command object properties
  comm.CommandText = "SP_NEWCART"
  comm.CommandType = adCmdStoredProc
 
  ' execute command and get output value (cartid)
  Set rs = comm.Execute
  
  ' get cartid
  rs.MoveFirst
  createCart = rs(0)

  rs.Close

End Function

When the Command object is executed, a record is returned and assigned to the Recordset object. Only one value is returned with the record, the cart identifier, which is then assigned to the function name and returned to ASP application.

To integrate this new component method and associated functionality into the ASP shopping cart application as a whole, the Cookies collection of the ASP built-in Request object is accessed, and the contents examined for an already identified cart. If one is found then it is used as the cart identifier for displaying cart contents. Otherwise, an instance of the shopping cart component is created and the createCart function is called. The newly returned cart identifier is then assigned to the Cookies collection of the Response built-in ASP object, and the cart identifier is created as a client-side cookie. Doing this persists the cart identifier between pages of the shopping cart application, and even beyond the current shopping session. In the example, the Cookie persists until the date set in the Expires property of the Cookie, which is December 31, 2001 in our code.

cartid = cart.createCart()
Response.Cookies("cartid") = cartid
Response.Cookies("cartid").Expires = "December 31, 2001"

We’ve implemented the functionality to add a new cart, but of course it isn’t very useful unless we can add items to it:

First, a stored procedure is created, SP_ADDITEM , to handle the addition of a new cart item. Within this procedure, a check is made of the table CARTITEM to see if a record already exists for the specific cart and product item. If found, the quantity passed to the stored procedure is added to the quantity for the cart item. If a record is not found, a new entry to CARTITEM is made for the specific cart and product.

CREATE PROCEDURE [SP_ADDITEM]
(@cartid int, @itemid int, @qty int)
 AS
BEGIN
IF (select count(*) from cartitem where cart_id=@cartid and 
      widget_id = @itemid) > 0 
   update cartitem
     set quantity = quantity + @qty where 
      cart_id = @cartid and widget_id = @itemid
ELSE
    insert into cartitem values (@cartid, @itemid, @qty, getdate())
END

The stored procedure SP_ADDITEM is called from within a new method, addItem , and added to the shopping cart component. In this method, the ADO Connection object is used both to connect to the database and invoke the stored procedure.

' addItem
' Adds item to shopping cart
' If more than one item, update quantity in SP
'
Sub addItem(ByVal lCartID As Long, ByVal lItemID As Long)

  Dim conn As New Connection
  
  ' connect to database
  conn.ConnectionString = m_connString
  conn.Open
 
  ' build command string
  Dim strComm As String
  strComm = "SP_ADDITEM " & CStr(lCartID) & "," & CStr(lItemID) & ",1"
                
  ' execute command
  conn.Execute strComm

  conn.Close

End Sub

The cart identifier discussed earlier and the product identifier are passed as parameters to addItem . We already have the cart identifier, and so the product identifier is passed to the shopping cart page from a form on a product page. The value is accessed from the Form collection of the built-in ASP Request object.

Dim itemid
itemid = Request.Form("itemid")

If itemid <> "" Then 
   cart.addItem cartid, itemid
End If

So, at this point we’ve created a cart and added an item to it. The next logical step to take in developing the shopping cart component and application is to provide a technique for displaying the cart contents:

Displaying the Cart Contents

The shopping cart display is the most visual aspect of a shopping cart application, and it is also one of the easiest to implement. Basically, the shopping cart items are accessed and displayed, as rows, usually within an HTML table.

A new stored procedure is created, SP_GETITEMS , which gets information from the CARTITEM and the WIDGET tables. The items that the cart contains are located in CARTITEM , but the information about the item, such as product name, price, and quantity per unit are found inWIDGET , hence the join between both tables. Additionally, a total price is calculated from the quantity of items ordered and the price per item, and this total is added as a “column” to the record being returned.

CREATE PROCEDURE [SP_GETITEMS]
(@cartid int) AS
select widget.widget_id, 
          short_name, 
          qty_unit,
          price,
          quantity,
          price * quantity total
    from cartitem,widget where cart_id = @cartid and 
    widget.widget_id = cartitem.widget_id

A new function is created, getItems , which calls SP_GETITEMS and returns the resulting recordset as a disconnected recordset . By returning the entire recordset to the ASP page, we can use built-in Recordset functionality to access and display the returned records and the individual fields. By using a disconnected recordset, the database connection is released before the recordset is returned to the ASP page, and valuable database resources aren’t being tied up unnecessarily.

' getItems
' return list of items, short decriptions, quantity
' as disconnected recordset
'
Function getItems(ByVal lCartID As Long) As ADODB.Recordset

  Dim conn As New Connection
  Dim rs As New Recordset
  conn.ConnectionString = m_connString

  ' connect to database
  conn.Open
  Set rs.ActiveConnection = conn
  
  ' set and open recordset
  rs.CursorLocation = adUseClient
  rs.Source = "SP_GETITEMS " & CStr(lCartID)
  rs.Open

  ' disconnect recordset
  Set rs.ActiveConnection = Nothing
  conn.Close

  Set getItems = rs.Clone

  rs.Close
End Function

An HTML table is created within the body of the ASP shopping cart page, and table headers are used to provide column labels for the individual recordset fields. Because the shopping cart can be updated — new quantities can be added for an item or an item can be removed – the HTML table displaying the cart items is contained within an HTML form, so the changes can be submitted back to the shopping cart application.

Following the HTML table and form definitions, ASP script is used to access the recordset with the cart items and output the recordset rows as table rows (records) and cells (columns).

Dim total
Do While rs.EOF = False
     total = total + rs(5)
     Response.Write("<TR>")
     Response.Write("<TD align='middle'><input type='hidden' name='itemid' value='" & rs(0) & "' size=10>")
     Response.Write("<strong>" & rs(0) & "</strong></TD>")
     Response.Write("<TD align='left'><strong>" & rs(1) & "</strong></TD>")
     Response.Write("<TD align='middle'><strong>" & rs(2) & "</strong></TD>")
     Response.Write("<TD align='right'><strong>" & FormatNumber(rs(3),2) & "</strong></TD>")
     Response.Write("<TD bgcolor='white' align='middle'><input type='text' name='quantity' value='" & rs(4) & "' size=10></TD>")
     Response.Write("<TD align='right' ><strong>" & FormatNumber(rs(5),2) & "</strong></TD>")
     Response.Write("</TR>")
     rs.MoveNext
Loop  
Response.Write("<TR><TD align=right colspan=6><strong>Cart Subtotal is: $" & FormatNumber(total,2) & "</strong></td></tr>")
 

Notice in the ASP script that a hidden form field holds the product item identifier for each row, and another form input element holds the quantity of items added to the cart for the item. The hidden field is used to tie a product identifier to quantity, and the quantity field is a text input element, giving the Web shopper to ability to modify the quantity of a specific item in the cart.

Following the ASP script, traditional HTML is again used to provide handling of form submission, including options to submit the shopping cart to the order processing system, return to the main store page, continue shopping, and to update the shopping cart to process a quantity change. Updating quantities is discussed in the next section.

Updating Shopping Cart Contents

Imagine for a moment going into a grocery store and adding several items to your shopping cart. Now imagine not being able to remove an item from the cart once the item is placed there, or being unable to change the quantity of a specific item in the cart. If you couldn’t modify the cart contents at a “real” store you probably wouldn’t return to the store and the same applies to the shopping cart implemented at a virtual store. Shopping carts must provide the capability for Web shoppers to modify their cart contents after the contents have been added.

Modifying cart items includes being able to change the quantity of an item in the cart and to remove an item from the cart altogether — two different functions that really only requires one stored procedure, SP_UPDATEQTY . The quantity being passed is checked within the stored procedure : if the value is zero (0), the cart item is deleted from CARTITEM ; otherwise the value is updated. In addition, the stored procedure checks to see if a row exists for the cart and item in CARTITEM . If it does, the value is updated; otherwise the stored procedure SP_ADDITEM is called to create a new cart item with the new quantity. Another approach to removing the item could be to add a button to delete the item from the cart. This could set the quantity to zero, or even call submit the cart for update immediately – either approach works.

CREATE PROCEDURE [SP_UPDATEQTY]
(@cartid int, @itemid int, @qty int)
 AS
BEGIN
   IF @qty = 0
      delete from cartitem where cart_id = @cartid and widget_id = @itemid
   ELSE IF (select count(*) from cartitem where cart_id = @cartid and 
         widget_id = @itemid) > 0
      update cartitem
      set quantity =  @qty where 
          cart_id = @cartid and widget_id = @itemid
  ELSE
      exec sp_additem @cartid, @itemid, @qty
END

The SP_UPDATEQTY stored procedure is called from a new method added to shopping cart component and called updateItemQty . This method has three parameters, the cart identifier, the product identifier and the quantity. It checks to make sure the quantity isn’t negative, and then builds a call to SP_UPDATEQTY .

' updateItemQty
' Update quantity of item
' return new count of items
'
Sub updateItemQty(ByVal lCartID As Long, _
                        ByVal lItemID As Long, _
                        ByVal lQuantity As Long)
                        
  Dim conn As New Connection
  
  ' quantity cannot be less than zero
  If lQuantity < 0 Then
      Err.Raise 5 ' invalid argument error
  End If
       
  ' connect to database
  conn.ConnectionString = m_connString
  conn.Open
 
  ' build command string
  Dim strComm As String
  strComm = "SP_UPDATEQTY " & CStr(lCartID) & "," & CStr(lItemID) & _
                "," & CStr(lQuantity)
                
  ' execute command
  conn.Execute strComm
  conn.Close

End Sub

The updates to the quantities occur in the same shopping cart display page that receives new product items, so way of determining whether an item is being added or the quantity is being updated needs to be added in. For our example, a hidden field is added to the update quantity form on the shopping cart display page, and to the product form on the product display page. The hidden field has a name of startpos , and the value attached to this field determines what action is taken when the shopping cart page is accessed. If a value of update is accessed, then an update is to be made.

<input type="hidden" name="startpos" value="update">

If a value of additem is found, then the add item functionality is used.

<input type="hidden" name="startpos" value="additem">

With the addition of the new hidden fields, and the update quantity component method, the ASP script in the shopping cart page is amended to allow for adding new items and updated quantities.

Dim action
 action = Request.Form("startpos")
 If action = "additem" Then
    Dim itemid
    itemid = Request.Form("itemid")

    If itemid <> "" Then 
      cart.addItem cartid, itemid
    End If

  ElseIf action = "update" Then
    For i = 1 to Request.Form("quantity").Count
       cart.updateItemQty cartid, Request.Form("itemid")(i), _
				 Request.Form("quantity")(i)
    Next
  End If

We now need to add the functionality that associates the cart with a specific customer, so that the customer can access this cart from any computer, and that empties the cart if the customer decides not to place an order. These are detailed next.

Associating Cart to Customer and Emptying Cart

At this time we have all the functionality necessary to create a cart, add items to the cart, persist the cart beyond a specific session and update the cart contents. However, to make the cart callable, which means a client can access it from any computer, we need to associate the cart to a customer.

A new stored procedure, named SP_CUSTOMERCART is created, which simply updates the cart’s customer identifier field with a specific customer identifier. How the customer identifier is accessed and the login procedure for the customer is outside the scope of the shopping cart application.

CREATE PROCEDURE [SP_CUSTOMERCART]
(@cartid int, @custid int)
 AS
BEGIN
  update cart set customer_id = @custid where 
  cart_id = @cartid
END

The stored procedure is accessed from a new shopping cart component method, addCartToCust, which does a couple of tasks. First, the method checks to see if the customer already has a cart and if so, the contents of the new cart are transferred to the older cart and the new cart is destroyed; otherwise the customer is assigned to the new cart.

' addCartToCust
' associate customer to cart
'  -- shipping cost and tax comes from customer state
'     these values cannot be calculated without customer
'
Function addCartToCust(ByVal lCartID As Long, ByVal lCustomerID As Long) As Long

  Dim lcart As Long
  
  ' check for existing customer cart
  ' if found, merge contents
  lcart = getCartID(lCustomerID)
  
  If lcart > 0 Then   ' existing customer cart found
     Dim rs As New Recordset
     Set rs = getItems(lcart)
     Dim i As Integer
     
     ' transfer new quantities to existing cart items
     For i = 1 To rs.RecordCount
        updateItemQty lcart, rs(0), rs(4)
     Next i
     
     ' destroy 'new' cart, use existing
     If lcart <> lCartID Then
        clearCart (lCartID)
     End If
  ElseIf lcart = 0 Then   ' no existing cart
    lcart = lCartID
    ' connect to data store
    Dim conn As New Connection
    conn.ConnectionString = m_connString
    conn.Open
    
    ' build command string and execute command
    Dim strCmd As String
    strCmd = "SP_CUSTOMERCART " & CStr(lCartID) & "," & CStr(lCustomerID)
    conn.Execute strCmd
    conn.Close
  End If
  
  ' return cart id
  addCartToCust = lcart

End Function

The addCartToCust method itself calls other component methods. The updateItemQty, discussed earlier, is used to transfer the contents of the new cart to the existing cart. In addition, a couple of new methods are created and used: one, getCartID, is used to return a cart identifier given a customer identifier; the other, clearCart, will remove the cart and its contents.

The getCartID uses a stored procedure called SP_GETCARTID to get any cart identifier for a given customer.

CREATE PROCEDURE [SP_GETCARTID]
(@customerid int)
 AS
BEGIN
select cart_id from cart where customer_id = @customerid
END

The method is fairly simple, basically little more than a call to the stored procedure, and validation checks to make sure a value of zero(0) is returned if no cart identifier is found.

'
' getCustomerID
' get customer id given cart id
'
Function getCustomerID(ByVal lCartID As Long) As Long

  Dim conn As New Connection
  Dim rs As New Recordset
  conn.ConnectionString = m_connString

  ' connect to database
  conn.Open
  Set rs.ActiveConnection = conn
  
  ' set and open recordset
  rs.CursorLocation = adUseClient
  rs.Source = "SP_GETCUSTOMERID " & CStr(lCartID)
  rs.Open
  
  ' get cartid
  If rs.RecordCount > 0 Then
    rs.MoveFirst
    If IsNull(rs(0)) Then
       getCustomerID = 0
    Else
       getCustomerID = rs(0)
    End If
  Else
    getCustomerID = 0
  End If
  
  rs.Close
  conn.Close

End Function

The clearCart method calls a stored procedure called SP_CLEARCART that deletes the cart items associated with a cart first, and then deletes the cart.

CREATE PROCEDURE [SP_CLEARCART] 
(@cartid int)
AS
BEGIN
   delete from cartitem where cart_id = @cartid
   delete from cart where cart_id = @cartid

END

The clearCart method itself is literally nothing more than a ASP component wrapper for the stored procedure call.

' clearCart
' Clears cart, removes all items
' disassociates customer from cart
'
Sub clearCart(ByVal lCartID As Long)

  Dim conn As New Connection
  
  ' connect to database
  conn.ConnectionString = m_connString
  conn.Open
  
  ' call stored procedure
  conn.Execute ("SP_CLEARCART " & CStr(lCartID))
  conn.Close

End Sub

Now, the ASP script to add a customer to a cart can be run when the person first logs into the store, or when an order is made – this is up to the individual Web store developer. In the example we’re working with, the cart is added to the customer the first time an item is added to the cart. The script itself is fairly simple.

' create the component instance
Dim cart
Set cart = Server.CreateObject("shopcart.cart1")

Dim cartid
cartid = Request.Cookies("cartid")
If cartid = "" Then
    cartid 0
End If

Dim customerid
customerid = Request.Cookies("customerid")
Dim action

If customerid <> "" Then
	cartid = cart.getCartID(customerID)
End If

action = Request.Form("startpos")
If cartid = 0 AND action <> "" Then
    cartid = cart.createCart()
    Response.Cookies("cartid") = cartid
    Response.Cookies("cartid").Expires = "December 31, 2001"

    If customerid <> "" Then
	cart.addCartToCust cartid, customerid
    End If
End If

In the code, the “action” variable is accessed from the Form collection. If an empty string is returned then we know that the shopping cart is not being called as a result of a quantity update, nor is it being called as a result of adding a new item. The shopping cart page is being called purely to display the cart, as a request from the Web shopper.

At this time, we have a shopping cart component, stored procedures, and supporting ASP pages to create a cart, add items to the cart, modify items in the cart, associate the cart with a customer, and destroy the cart. Are we finished? Not quite yet, we have one more requirement left to implement: we need to show how many items a person has in a cart from the store’s home page.

Displaying Summary Information about the Cart

Displaying information about a cart on the home page of a store is relatively simple. A stored procedure is created to return summary information about the cart such as the quantity of items ordered and the total cost (without shipping and tax). This stored procedure is named SP_GETITEMTOTALS .

CREATE PROCEDURE [SP_GETITEMTOTALS]
(@cartid int)
 AS
BEGIN
   select sum(quantity) items, sum(price * quantity) total from 
   cartitem, widget where 
   cart_id = @cartid and 
   widget.widget_id = cartitem.widget_id
END

This stored procedure is then called from within a method, getItemTotals that has a cart identifier as a parameter and returns a disconnected recordset containing the cart information to the ASP page.

'
' getItemTotals
' Returns count of items currently in basket
'
Function getItemTotals(ByVal lCartID As Long) As ADODB.Recordset

  Dim conn As New Connection
  Dim rs As New Recordset
  
  ' connect to database
  conn.ConnectionString = m_connString
  
  conn.Open
  Set rs.ActiveConnection = conn
  
  ' set and open recordset
  rs.CursorLocation = adUseClient
  rs.Source = "SP_GETITEMTOTALS " & CStr(lCartID)
  rs.Open

  ' disconnect recordset
  Set rs.ActiveConnection = Nothing
  conn.Close

  Set getItemTotals = rs.Clone
  
  rs.Close
End Function

In the ASP page, the disconnected recordset is then used to access the number of items in the cart and the total, which are then displayed to the page.

Dim cart
Set cart = Server.CreateObject("shopcart.cart1")

Dim cartid
cartid = Request.Cookies("cartid")
Dim customerid
customerid = Request.Cookies("customerid")
Dim rs

If customerid <> "" Then   
  Dim firstname
  Dim lastname
  Set rs = cart.getCustomer(customerid)
  If rs.RecordCount > 0 Then
    Response.Write ("Hello " & rs(0) & " " & rs(1))
    Response.Write(" - If this isn't you, please <a href='getnewcust.asp'>Login to your account</a>")
    cartid = cart.getCartID(customerID)
    If cartid = 0 Then
   	cartid = ""
    End If 
   End If
ElseIf cartid = "" AND customerid = "" Then 
  Response.Write("<a href='getcust.htm'>Login to your Account to retrieve an existing cart</a>")
End If

If cartid = "" Then
  	Response.Write("<br>Currently, your shopping cart is empty")
Else
    Set rs = cart.getItemTotals(cartid)
    If rs.RecordCount > 0 Then
       rs.MoveFirst
       Response.Write("<br>Currently you have <strong>" & rs(0) & "</strong> items in your cart ")
       Response.Write("for a total of <strong>$" & FormatNumber(rs(1),2) & "</strong> dollars.")
    End If
 End If

If the cart Cookie is empty (no cart is set on the host computer), a message to this effect is shown in the page; otherwise the summary of the cart contents is printed out. In addition, the name of the person who owns the current cart is displayed, and the Web shopper is given the option of logging into the system.

Information about the customer is returned with a new stored procedure, SP_GETCUSTOMER . The login portion of this shopping cart application is included – as an extra bit of bonus code! – with the sample code attached to this article.

In Summary

Web stores can be complicated applications, but the best approach to create a Web store is to break the store’s functionality into individual pieces, or applications, and implement each of these in turn.

A key aspect to the implementation strategy of the shopping cart is that it’s business logic should be kept as separate as possible from implementation and database details. Implementation details, such as ASP specific functionality, are handled within the ASP scripts, and data access is processed within stored procedures. With this approach, changes to either the implementation or the data schema impact little if at all on the shopping cart component itself.

This article took a look at one specific application of an online store, the shopping cart, and demonstrated how a cart can be implemented without a lot of complicated code using an ASP component written in Visual Basic, some ASP pages, and stored procedures. The sample code for download contains all of the code discussed in the article as well as other functionality to handle pulling the entire application together.

Shopping Carts

Recently, someone, I’ll call him Joe, sent me an email and asked a question about maintaining shopping cart items using client-side cookies.

A shopping cart is basically a program that maintains a list of items that a person has ordered, a list that can be viewed and reviewed and usually modified.

Joe had run into a problem in that browsers limit both the size and quantity of cookies that can be set from a Web application, and this limited the number of items that could be added to his company’s online store’s shopping carts. He asked whether there was a client-side Javascript technique he could use that would update a file on the server instead of on the client so that customers could add an unlimited number of items to their carts.

Instead of trying to maintain the company’s shopping cart using cookies, I recommended that Joe check out his Web server’s capabilities, and see what type of server-side scripting and applications his server supported. Then he could use this technology to maintain the shopping cart items. I also recommended that he limit the use of the client-side cookie to setting some form of session ID so that the connection between the cart items and the shopper was maintained.

Shopping Cart Technology

Joe’s email did get me to thinking about how online stores use different techniques to maintain their shopping carts.

For instance, all of the stores I shop at, except one, use some form of a client-side cookie, but each store uses these cookies differently. Additionally, the stores use server-side techniques to support online shopping, though this support can differ considerably between the stores.

Client-side cookies were originally defined by Netscape for use in Navigator, though most browsers support cookies. Cookies are small bits of information stored at the client that can be maintained for the length of a session, a day, a week, or even longer.

The use of client-side cookies is rigidly constrained in order to prevent security violations. You can turn off cookies with your browser, but be aware that cookies do not violate the security of your machine and without the use of cookies your online shopping will be severely curtailed.

This YASD article does a little technology snooping of four online shopping sites and snoops out how each site uses a combination of server-side and client-side processing to maintain its carts.

Covered in this article are the shopping cart techniques used at the following online stores:

  • Amazon.com
  • Beyond.com
  • Catalogcity.com
  • Reel.com

Shop Til You Drop

To understand shopping cart maintenance, its important to understand customer shopping habits. We’ll start this article by looking at some online shopping habits.

First, online shopping has grown enormously in the last few years. Once folks found out that it was safer to send your credit card number over the Net at a secure site then to give it over a wireless phone, much of the hesitation about online shopping vanished.

What are the types of things that people buy online? Books, CDs, and Videos are popular, but so are kitchen utensils, computer hardware and software, photography equipment and film, food gifts, bathroom towels, and even irons.

People shop online because of necessity, convenience, and cost. We purchase books, CDs, and videos online because the online stores have a much larger selection than any local store could possibly have. We know that when we are looking for a book, even an out of print book, we will most likely be able to get the book from one of the online bookstores such as Amazon.

Some businesses we shop at, such as Dell, have no physical store location. This type of business provides service for their customers through mail or phone order, only. Many of us prefer to use online shopping for these types of stores rather than having to call someone up and go through a long list of items or manually fill out an order form, all the while hoping we put down the right item number. It is a whole lot simpler to look at an item and click on an image that says something like “Add to Shopping Cart”. An added benefit to online shopping is that we can review my order before it is sent, and can get a hard copy printout of the order for our records.

Normally, most of us only shop for a few items at a time, but the number of items can become large, especially with online grocery stores — a rising phenomena. However, it isn’t unusual for us to add some items to our shopping cart one day, a couple of more items another day, and so on, until we’re ready to actually place the order. At times we may even forget we have items in a shopping cart, until we add another item to the cart and the previous items show up.

We also change our mind at times, and may need to remove items from the shopping cart, or change the quantity of an item ordered. It’s also handy to have a running total for the order so we can make at least an attempt to stay within our budgets. If the shipping charges are also shown, that’s an added bonus.

Many of us may have more than one computer and may start a shopping session with a laptop and finish it at a desktop computer, though as you will see later in this article, this sometimes isn’t that easy. In addition, the use of both Netscape Navigator and Microsoft’s Internet Explorer on the same machine isn’t all that unusual for heavy Net users, and we may start a shopping cart with one browser and add to the cart from another browser.

Pulling requirements from these patterns of use, we come up with the following:

  • An item can be added to a shopping cart with the touch of a button
  • Shopping cart items need to persist for more than one shopping session
  • Some indication that there are items in the shopping cart should show, at least on the home page for the site
  • The store needs to provide a means to modify the shopping cart items, or to remove an item or all items
  • A running total needs to be maintained each time the shopping cart contents are reviewed
  • Showing standard shipping charges and other costs when the shopping cart is reviewed is an added bonus
  • The shopping cart needs to follow the client
  • Stores need to provide the ability to review an order before placed
  • Stores also need to provide the ability print out the contents of the shopping cart
  • Shopping carts should support an indefinite number of items, or the number of items should be limited by company policy, not Web technology limitations.

A pretty good list of requirements. Now, how do each of the stores measure up?

To determine when cookies are used at each of the sites evaluated, I set my browsers to prompt me when the online store wants to set a cookie. Using this approach I can see what kind of cookies the store uses, and get an idea of the cookie purpose.

Amazon.com

Probably the undisputed king of online bookstores is Amazon.com. This company began as a pure Net-based business, and has shown the world that online commerce not only works, it works very well, thank you.

Amazon has one of the better store interfaces, and some of the best account and order maintenance setups, but does it meet all of our shopping cart requirements? Let’s check it out.

First, all items that Amazon sells can be added to the existing shopping cart with the touch of a button, even items that are on order but not yet in stock. In addition, the shopping cart contents will persist even if you leave the site and return at a later time. In fact, Amazon tells you that the item will remain in the shopping cart for 90 days, if I read this correctly, a feature I found to be very nice.

Bonus technique: Let people know how long the shopping cart items will remain in the cart. The only surprise to pull on a customer is to let them know an item is on sale, or that they are the millionth customer and have won something. Anything else will lose you business.

Amazon also provides a feature to save the item for purchasing at a later time. This removes the item from the cart, but still keeps the item on a list for later purchase.

The shopping cart can be reviewed at any time, and there is an icon on every page that allows you easy access the shopping cart. You can also modify the cart contents by changing the quantity of an item you’re ordering, or removing an item altogether.

Amazon makes use of standard HTML technology, so the shopping cart review page should print out fairly well. Unfortunately, the shopping cart does not display shipping charges and does not display a running total for the items. However, Amazon does show a total, including shipping, that you can review before you place the order. This page can also be printed out.

So far so good. Amazon has met most of our requirements, but the real test of Amazon’s supremacy in shopping cart technology is whether the cart can follow the customer. Unfortunately, the company does not support this capability.

When you access Amazon from a browser at a specific machine for the first time, Amazon sets an ID that is used to track your shopping cart items. Access Amazon from the same browser and the same machine, and you will get the same shopping cart items. However, if you access Amazon from another machine or even another browser, you will not get access to these shopping cart items.

Is it important to maintain shopping cart persistence from browser to browser, machine to machine? You bet it is.

I, as with other folks involved with Web development and authoring, use both Navigator and IE. In addition, there are some sites geared more towards one of these browsers, so most folks who spend a lot of time on the Net have both browsers.

There are times when I am sure I have placed an item in the shopping cart, only to find out I did, but using a different browser or from a different machine. This happens more often than I would like, and is an annoyance every time.

Now the online stores have to ask themselves the question: Are people like myself part of a customer base they want to attract? Think of this: Who is more likely to be purchasing items online than folks who spend a large amount of their time, online. And who is likely to use more than one machine and more than one browser? Folks who spend a lot of time, online.

To summarize, Amazon uses client-side cookies to establish a persistent ID between the customer and the shopping cart. The company also uses this ID to establish a connection from the customer to the customer’s account information. The shopping cart items, however, are maintained on the server, persist for 90 days, and there is no limit to the number of items that can be ordered at Amazon, at least as far as I could see. Where Amazon does not meet the requirements is by not providing a running total on the shopping cart review page, and by not providing a shopping cart that moves with the customer.

Based on the requirements met by Amazon, I give them a score of 8 our of 10, for their support of shopping cart techniques.

Beyond.com

Beyond.com sells computer software and hardware and is a Net-only based company.

Beyond.com maintains a client ID in client-side cookies, which is used to track shopping cart contents for the session, only. Beyond.com does not persist the shopping cart contents outside of a specific browser session. Once you close the browser, the current shopping cart contents are gone.

In addition, it does look as if Beyond.com maintains the shopping cart totally within one client-side cookie, tagged with the name “shopcart”.

By maintaining the shopping cart on the client, Beyond.com has chosen one of the simplest approaches to maintain a shopping cart, and simple can be good. There is little or no burden on the server other than accessing the original item that is added to the cart. There is also less maintenance to this type of system, primarily because the Web developers and administrators do not have to worry about issues of storage of information on the server, or cleaning up shopping carts that become orphaned somehow. Additionally, Beyond.com is taking advantage of a client-side storage technique that is safe and simple to use.

However, there is a limitation with this approach in that the cookie is limited to a size of 4 kilobytes. It may seem that 4K is more than large enough to support a cart, but when you store information for each item such as cart item number, name of product, version, price, store identification number, quantity and price, you can reach an upper limit more quickly then you would think. Additionally, a limit is a limit, and you have to ask yourself if you really want to limit how many items a person can add to their shopping cart.

Most online stores could probably get away with shopping carts that have number of items limitations. After all, it might be a bit unusual to purchase 40 or 50 items from a software company at any one time.

If a store’s customers tend to purchase only a few items at a time, then it might not be cost effective to provide shopping cart technology that provides for virtually unlimited items.

Beyond.com also provides a quick look at the shopping basket from every page of the site. This quick view provides the first few letters of the cart item, the quantity ordered, and a running total for the cart. As much as I appreciate having this information, I found that I would have preferred having just the quantity of items in the shopping cart and the running total: listing all of the items actually became a distraction when I had more than a few.

Beyond.com uses standard HTML and the shopping cart page did print out using the browser’s print capability. In addition, you can review the order, including the shipping charges, before the order is fully placed.

To summarize, I liked Beyond.com’s support for shopping cart status display on the other Web pages. I also liked Beyond.com’s running total. The biggest negative to Beyond.com’s shopping cart was the lack of persistence outside of the browser session. I may not order more than 5 or 10 items at a time, but it isn’t unusual for me to add a couple of items at one time, close down the browser and return at a later time to add more items and finalize the order. In addition, it isn’t impossible for people to accidentally close down their browsers, which means they lose all of the items from their cart and have to start over. Based on the lack of persistence, I would have to give Beyond.com a 6 in shopping cart technology.

Catalogcity.com

CatalogCity is an interesting online business that presents the contents of catalogs from several mail order firms, allowing you to shop for everything from a new jacket to kitchen knives. Once you have placed your order for all of the items you’re interested in, CatalogCity then submits the orders for the individual items to the specific catalog company.

Of all the online shops I have seen, CatalogCity is one of the most clever. It provides both goods and a service, but without the hassle of maintaining inventories and supporting warehouses. I am sure that CatalogCity charges a fee to use their services to the catalog companies listed, but it is most likely more profitable for these companies not to hassle with online ecommerce issues. Even for those companies that have their own site and that use CatalogCity, they will get access to people who are looking to shop, but don’t necessarily know the catalog company’s name or Web site URL.

I do like to see effective and innovative uses of Web commerce. If I have a problem with the site, it is that not all of the catalog companies support online shopping in the catalog through CatalogCity. You can review the catalog and use the phone number provided to place your order. However, it’s just not the same as one button shopping.

CatalogCity uses cookies to set a customer id the first time you access their site. However, after that, all information about the shopping cart is stored on the server. There is no indication in the pages that you have shopping cart items, but you can access the shopping cart from a well placed icon on each site page.

The shopping cart page lists all of the items, provides the ability to change or remove an item, and provides a running total — sans shipping charges. It also provides a hypertext link from the item to the page where the item was originally listed, so you can review the item before making a final purchase.

The technology that CatalogCity uses for their shopping cart is fairly standard, so the cart page should print out easily. In addition, the company does provide the customer the ability to review the order, including shipping charges, before the order is placed.

The CatalogCity shopping cart is the most persistent of all of the shopping carts that I have seen. First, if you access the site but don’t set up an account, the cart will persist from browser session to browser session, but only with the same browser and machine. However, if you create an account with CatalogCity and sign in each time you access the site, the shopping cart will follow you from one browser to another, and from one machine to another. In fact, of all the sites I reviewed for this article, CatalogCity is the only one that provided this functionality.

To summarize the CatalogCity shopping cart technology, the company has provided the best support for shopping cart persistence of all the sites visited. In addition, the company also provides easy access to the cart, and provides a running total on the shopping cart page. CatalogCity also provides you with a change to review and modify your order as well as review the shipping charges before the order is placed. About the only non-positive aspect I found with this site’s shopping cart technology is that the site does not provide information that the shopping cart has items on the first page. If CatalogCity had provided this, I would have given the site a score of 10, but I’ll have to give it a score of 9.

Reel.com

Reel.com is an online video store that sells new, and used, VHS and DVD videos. It has an excellent selection of movies and a nicely organized site.

Reel.com uses cookies to set a user id when you first access the site. When you access a specific item, the site uses ASP pages and ASP (Microsoft’s server-side technology) sets a cookie with a session id when the first ASP page is accessed. After that, no other cookies are set. All shopping cart items are maintained on the server.

ASP or Active Server Pages, was the technology that seemed to be most used at the different online stores. ASP technology originated with the release of Microsoft’s Internet Information Server (IIS), but has since been ported to other Web servers and even to Unix from a company called ChiliSoft.

ASP provides for both server-side scripting as well as server-side programming with ASP components. In addition, ASP provides full support for data access through Microsoft’s Active Data Object technology.

One cookie that is set with ASP is the Session ID. When you access a site for the first time during a browser session, Microsoft tries to set a Session ID, to maintain a connection between your browser and the Web server. Without this, it is very difficult to maintain information about the session, such as shopping cart contents, from Web page to Web page.

Reel.com does not provide a running total for the purchases on the shopping cart page, and does not provide a visual indicator that items are in the shopping cart from any of the pages, except the shopping cart page. The company does provide easy text-based access to the shopping cart from each page and does allow you to change the quantity of an item ordered, as well as remove an item from the cart.

Reel.com provides shipping and full order information for the customer to review and modify before the order is placed, and the order page, as well as the shopping cart, can be printed out.

Reel.com does not provide persistence beyond the session. Once you close the browser, the shopping cart is gone.

To summarize, Reel.com did not score very high by meeting many of the requirements for a shopping cart. It didn’t provide a visual cue about shopping cart contents, at least on the first page of the site, nor did it provide a running total on the shopping cart page. The biggest negative, though, was that the site did not maintain the shopping cart persistently outside of the browser session. Reel.com did provide the ability to review and modify the order before the order was placed, but based on the requirements met, I would have to give Reel.com only a 4 for shopping cart technology.

Summary

Four different online store sites, each using different techniques to support the site’s shopping cart.

All of the sites used cookies to establish a connection between the browser session and the shopping cart. In addition, each site provided shopping cart pages that could be printed out, and provided the ability for the customer to review and modify, or cancel, the order before it was placed.

The highest scorer in my evaluation is CatalogCity, primarily because of their support for persistence across different browsers and machines. This was followed by Amazon, which provided for browser and machine specific persistence.

Both Reel.com and Beyond.com dropped drastically in points because of their lack of shopping cart persistence, in any form. However, Beyond.com did provide feedback as to shopping cart contents, something that CatalogCity and Amazon did not. Beyond.com may want to consider dropping their line item display of the shopping cart as this can be distracting. They were also the only online store maintaining their shopping cart in client-side cookies. While this approach has an advantage of being the quickest technique to displaying the shopping cart contents when the customer wants to review the shopping cart, and is the simplest technique to use, I still don’t see this approach as the way to go for online shopping.

If we could take CatalogCity’s persistence and add to it a running total with estimated shipping charges, and provided feedback on the cart contents in at least the home page of the site, we would have, in my opinion, the perfect shopping cart.

The next article in the Technology Snoop series will be on order processing and order status maintenance. In this you’ll see that Amazon shines. You’ll also read about how not to win customers and to lose the ones you have when I add some new online stores to the review.