Sometimes you feel like RDF, sometimes you don’t

Semaview came out with this illustrated RDF vs XML graphic showing the ‘differences’ between RDF and XML. At least one assumes this is the purpose of a graph so titled. This might be confusing for some people that assume RDF is XML, which isn’t entirely true: RDF is a model, RDF/XML is one serialization of that model.

(Still, when you have XML on one side and RDF/XML on the other, one does wonder where the concept of ‘versus’ enters the picture.)

Based on this illustration, Leigh Dobbs asked the question:

I’m working with RDF tools now, but thats because FOAF is an RDF vocabulary. I’m just using the right tools for the job. If I was given a task to design a new system I don’t have any feel for why I might choose RDF over XML. I haven’t had that “aha” moment yet.

Personally, I doubt there will ever be an ‘aha’ moment associated with any W3C specification, but that’s beside the point.

In response to Leigh’s question, I wrote the following in his comments:

Pat Hayes actually grabbed a quote from the book and posted at the RDF WG core mail list about RDF’s usefulness:

“RDF is a technique to record statements about resources so that machines can easily pick up the statements. Not only that, but RDF is based on a domain-neutral model that allows one set of statements to be merged with another set of statements, even though the information contained in each set of statements may differ dramatically.”

XML gives us the format to record domain-neutral data, but RDF gives us the methodology to record complete domain-neutral statements — data in action as it were.

Ontologies are then domain-specific views built on top of the domain-neutral model that is RDF.

It’s all layers. Taking a cross-section:

Knowledge can be split into domain-specific views (ontology) based on complete statements (RDF) consisting of separate pieces of syntactically valid data (XML).

Since the first moment that XML appeared a few years back, the first thing I, as a data, not a markup person, looked for was a data model making use of XML. To me, XML would never be anything more than interesting data formatted in an interesting manner; without rules to help that data form some cohesive pattern outside of the rules defined for each implementation of an XML vocabulary I doubted its usefulness. Still, the bugger caught on and achieved wide-spread use.

Such waste — all that machine accessible data and absolutely no way of merging it into one data store in any meaningful way. Worse, having to create algorithms to manage each specific XML vocabulary rather than having one set for all vocabularies.

To me, all these XML vocabularies are equivalent to throwing out our relational databases and going back to proprietary data structures in each of our applications. Change jobs, learn an all-new structure. Buy a new application, and face a huge learning curve just understanding the underlying data and how it’s interrelated.

I knew when RDF came out it was the missing link between XML as bits and pieces of data, and XML as information; this though RDF wasn’t necessarily created specifically to be used as a model for XML.

(Some would say that the marriage between RDF and XML was a shotgun wedding at best. I don’t care. The cake was good and the band played on; I had a good time.)

In an effort to answer Leigh, Dorothea Salo wrote:

First. If you must end up with something XML-valid, don�t bother with RDF. Just don�t. Yes, you can restrict the RDF/XML you produce to a specific syntax form; you just can�t expect anything you receive to be similarly restricted, because RDF/XML-generating tools can�t be made to give a damn about which form they output of the many possible syntax forms of a given set of RDF/XML statements.

What Dorothea is referencing specifically is that there are different forms of XML that can be used for a specific type of RDF construct, which means that the same RDF model can be serialized in four different forms, and each would be an accurate and valid rendition of that model. True, but that doesn’t preclude that all of the XML is valid and that all of it can be restricted through DTD’s and XML Schemas, and still be valid RDF/XML.

However, Dorothea is right in that RDF is not magic pixie dust. RDF is nothing more than a way of recording domain-neutral statements in such a way that they are merged with other domain-neutral statements, each statement adding to the others in a mounting knowledge base.

When she says:

Computers only know what you tell �em. They don�t automagically know foo from bar any more than humans do. Inference only gets you so far. Sure, it might be further than we�ve been yet; I�m inclined to think so, myself. At some point, though, somebody�s got to know what the bits of the vocabularies mean, and all the inferential power in the world won�t get that across.

XML gives us the ability to record bits and pieces of data in a valid manner. RDF then builds on the data, piecing the bits and pieces together into complete statements. Ontologies then take these statements and build machine-understandable inferential rules based around them. The result of all this working together is the wine scenario:

Information from a vineyard is recorded as XML, and the names of wine are recorded as XML Schema datatype strings. The XML ensures that the names are valid, and the data is accessible with any XML compliant parser. Another format could be used, but then if someone else wanted to access the data, they’d have to build parsers that can understand the proprietary format.

The RDF model then provides the means to incorporate those names into facts, such as “Chianti is a red wine”, using a serialization technique molded on to XML:

<rdf:Description rdf:ID=”chianti”>
<wine:category rdf:datatype=
“http://www.w3.org/2001/XMLSchema#string>white</wine:category>
</rdf:Description>

We could build the model on which the facts are based directly into the XML vocabulary. But then, we’d have to make sure the model and the facts were consistent regardless of use. And since it was proprietary, other tools would also have to build in the ability to produce or consume facts based on this proprietary model.

Finally, the ontology, such as DAML+OIL and the W3C’s OWL, pieces together the separate statements and facts into domain-specific knowledge, by applying rules that allow machines to make inferences on the data, such as the fact that a cheese and nut dessert course is a part of a formal meal and is an alternative to a sweet dessert, and wines served during this course should be red:

<rdfs:Class rdf:ID=”CHEESE-NUTS-DESSERT-COURSE”>
<daml:intersectionOf rdf:parseType=”daml:collection”>
<daml:Restriction>
<daml:onProperty rdf:resource=”#FOOD”/>
<daml:hasClass rdf:resource=”#CHEESE-NUTS-DESSERT”/>
</daml:Restriction>
<rdfs:Class rdf:about=”#MEAL-COURSE”/>
</daml:intersectionOf>
<rdfs:subClassOf rdf:resource=”#DRINK-HAS-RED-COLOR-RESTRICTION”/>
</rdfs:Class>
<rdfs:Class rdf:ID=”CHEESE-NUTS-DESSERT”>
<rdfs:subClassOf rdf:resource=”#DESSERT”/>
<daml:disjointWith rdf:resource=”#SWEET-DESSERT”/>
</rdfs:Class>

This type of information can never be recorded in ‘straight’ RDF/XML because you’d have to have the ability to record the inferential rules, and RDF focuses on recording statements. Additionally, the information could never be discovered in straight XML because you have to have the ability to record not only the rules but the statements, too. You would literally have to build a model and then find a way to serialize that model in XML. Just like RDF. If you used XML, you’d have to define the ability to record facts, and then on top of that, the ability to record the necessary information to perform inferential queries — something more esoteric than “what is a white wine”.

But using XML as a data format, and using RDF as a statement/model methodology and using OWL to record the domain-specific rules, you can go to the application such as the Wine Agent, ask for recommendations for a cheese desert wine within a certain region and get the following answer:

“Pairs well with sweet red varieties. Full-bodied wines featuring strong flavors match especially well.”

The local knowledge base particularly recommends the following:

TAYLOR PORT

The recommended wines can be found below, along with some comparable selections: (with link to selection)

The frosting on this particular layer cake is that anyone associated in some way with the wine industry — producing or consuming — could use the wine ontology, based on RDF, persisted in XML for their own applications and functionality. Better yet, another industry, let’s say the chocolate industry, can use the same XML/RDF/ontology combination, and the same tools that work with each, as a way of recording their domain-specific data.

And you’ll be able to know exactly which champagne to buy to go with that dark bitter-sweet chocolate covered orange peel you bought for that special someone.

Web Services Working Group

I’m extremely pleased to see the formation of a new working group at the W3C. It will be responsible for generating some meaning of the chaos that is Web Services. I may not be an adherent of the “Standards at all cost” club, but we do need to have standardization in three specific areas:

  • Protocols
  • Syntax
  • Semantics

Syntax is XML, Semantics is RDF, and a  public domain Protocol will, hopefully, come quickly and efficiently out of this new group.

XML as a syntax is safely within the public domain. However, there is too much implied ownership of the protocols (openly available or not), and there’s too little understanding of the importance of the semantics.

We don’t need centralized enforcement agencies such as UDDI., or centralized security paradigms such as Passport if we only have a standardized syntax, semantics, and protocol. Give me these, and I can build you an interface to anything, from anything.

My strongest hope is that the W3C moves swiftly — they’re late in the protocol game.

Browser, Browser Not

Originally published at O’Reilly

Recently, O’Reilly published a set of articles (Netscape Navigator 6.0 to Fail Standards ComplianceAn Update, and Netscape 6.0 Released), written by the popular author David Flanagan, about the release of Netscape 6.0, Netscape’s newest entry in the browser marketplace.

David presented several valid concerns about bugs still present in the release of Netscape 6.0. And it is true, Netscape 6.0 did release with several unfixed bugs, many of which will have an impact on support for W3C specifications.

Our reaction to the release, however, was somewhat different. Along with other application developers, we’ve been waiting for the public release of an application that uses Mozilla’s XPToolkit, a set of software components from which Netscape 6.0 and the upcoming Mozilla 1.0 were built. Now that Netscape 6.0, which uses this framework, has been publicly released, we’re delighted: testing of XPToolkit may begin in earnest.

While many are focused on the release of Netscape 6.0, some of us aren’t. We’re more interested in the application environment created by the Mozilla team to support the implementation of browsers in general. To us, this framework is more important than the release of a new browser will ever be.

The reason for this is the changing face of the Internet, itself.

The Changing Face of Internet Applications

Current Internet applications rely on a centrally located Web server to distribute HTML over HTTP to clients. Each client, or Web browser, renders the source and displays a human-readable page.

This architecture has become so popular that you can’t pick up a magazine or a newspaper without hearing about Web servers or the new business models based on them. Although this architecture is based around universally located resources, most application-level resources are centralized and many other resources are hard to find. Some Web sites help you find other Web sites or “resources.” Others go so far as to offer completely centralized applications, as Application Services Providers (ASPs).

New technologies will soon force us to rethink the way we use the Internet. Distributed systems, mobile agents, and peer-to-peer (P2P) applications may completely undermine the need for browser-based Internet access.

P2P applications are already stepping around the browser. The next step will be around the Web server.

Consider this: a P2P application that locates and downloads a new function. The simplest example here may be provided by a P2P execution framework that uses XML-based remote procedure calls between peers to marshal XML-encoded functions. Instead of hitting Web pages, each peer locates and accesses both data and functions among a network of peers. No Web servers.

This scenario is not going to be best served by the traditional browser. Why?

The Limitations of Browsers

The things that made the Web browser a success in the beginning are the things that make it ineffective for new application models.

The browser was built to render files stored on Internet sites so we didn’t have to muck about with FTP. As soon as content became more visible, people started publishing yet more content, so browsers rendered HTML, then XML, formatted with CSS or XSLT. However, the browser itself has a very limited interface, even with new advances in W3C specifications. Sophisticated browser pages mean using either complicated object models–leading to cross-platform and cross-browser idiosyncrasies that are usually the result of standards initiatives–or using page-embedded applications, such as Java applets and plug-ins.

Even when the browser follows standard specifications, working within a browser page to create a sophisticated interface isn’t a simple or uncomplicated task.

In addition to the browser becoming increasingly complex as the nature of content becomes so, use of it implies that applications ought to be served from one location, and in one manner. To do something such as make a remote procedure call, you would need to use a digitally signed Java applet or some other browser-specific and limited technique. This is something that won’t bother newer P2P applications.

Finally, browsers were designed to be safe, and operate in a protective sandbox. Web-based applications served via a browser have difficulty getting at the user’s machine. Though safe, this restriction also prevents behaviors that would have the application modify its user interface. And this dynamism is going to be necessary in an environment where new services require new application interfaces that can be downloaded as data.

An Internet Application Framework?

Mozilla made a tough decision a few years ago–to scrap the Netscape 4.x architecture in favor of one built from the ground up. In the process, this open source team created an application environment based on reusable and interchangeable components.

With this application environment in place, the team then proceeded to build a sophisticated browser. They threw in Internet Chat, a Web page composer, and other complex things, all of which were released recently as Netscape 6.0. Often forgotten is that a powerful application environment came with it. This environment is now usable by developers of other Internet applications.

What types of applications? Well, ActiveState, the company that provides popular implementations of Perl and Python for various operating systems, used Mozilla to create itsKomodo product, a visual IDE for working with Python and Perl code. The user interface provides, among other things, colored syntax, syntax checking, and source-level debugging.

So, we have a browser and an application that can be used to create and test Perl and Python applications, all built from the same application architecture.

This is exciting stuff! Much has been written about reusable code and component-based design, and now we have an open source application environment we can all use to build our own applications.

Even more exciting is the extensible user-interface language from Mozilla called XUL (pronounced “zool”). It’s based on XML, which means you can use XML to create a user interface. Combine this with the ability to make remote procedure calls, and you have a perfect place from which to commence building a bunch of P2P applications, based on the scenario mentioned above.

Now, instead of opening a browser, you can open an application built on the same framework as your browser, but with a sophisticated interface of dropdown menus and tabbed pages–all created using XML. You can access remote procedure calls at the touch of a button and when you’re ready to access a new service, click another button, and in a couple of minutes restart your application. New entries will be added to new or existing menus providing access to the new service. All this is accomplished without Java bytecode, a new plug-in, or a DLL.

You’ve just downloaded XML.

When you explore the possibilities of the XPToolkit from Mozilla maybe you’ll agree that Netscape 6.0 is more than just a standards-based, better-than-Navigator-4.x-browser. It’s the start of a new new way of doing things on the Internet.

The Tyranny of Standards

Originally published at O’Reilly

Before proceeding into the core of this article, I want to say one thing to you: challenge your assumptions.

Challenge your assumption that all Internet services are provided by a Web server and consumed by a browser Challenge your assumption that chaos within a development environment is a bad thing. And challenge your assumption that standards must take precedence over innovation.

Several years ago, when the concepts of Web server and browser were first implemented, the Internet was introduced to a new state of chaos and, as the explosive growth of technologies that are “Web-enabled” demonstrates, innovation was not only the rule, it was the norm.

Over time, people decided that standards were a necessary adjunct to the growth of the Web, something with which I completely agree. Enter the W3C, the World Wide Web Consortium.

As the W3C organization will attest, they are not a standards body. As such, they don’t issue “standards” per se. Instead, the W3C issues recommended specifications. The only enforcement of these specifications has been through voluntary compliance on the part of the technology providers, and demand for said compliance on the part of technology consumers.

Thanks to the efforts of the W3C, we have specifications for HTML, XML, CSS, HTTP, and a host of other Web-enabling technologies. Thanks to those following the specifications, we have Web pages that can be viewed by different browsers and served by different servers.

Somewhere along the way, however, standards became less of a means for providing stability and more a means of containment. In some cases, standards have become a weapon used to bludgeon organizations for practicing the very thing that started the growth of Web applications in the first place: innovation.

The Importance of Innovation

Innovation is the act of improving what exists and creating something new. Though innovation does not always lead to something better (Remember push technology?), it is the thing that keeps us moving forward, always searching for a better way of doing things.

Innovation can work comfortably with standards; new XML-based specifications, such as MathML, are a case in point. There are also times when innovation actually bucks the standards.

For instance, Microsoft has been long criticized for adding its own “innovations” to a specification, particularly with its popular Web browser, Internet Explorer. One innovation was the support of a property called innerHTML that is used to access or easily replace the contents of a specific HTML element. Though innerHTML is not part of any of the W3C specifications, its use is so popular that Mozilla, the open source effort behind the new Netscape 6.0 browser, has adopted the use of innerHTML within its own layout engine.

Should Microsoft and Mozilla be bashed for lack of standards compliance because innerHTML is not a property supported by the W3C? Or should both organizations be commended for providing a useful tool that has become very popular with Web developers?

This leads to an additional question: How does one measure standards compliance? For example, if Internet Explorer and Mozilla both supported CSS attributes such as font size and color, and they also supported new attributes and properties like innerHTML, would both browsers be compliant? Or are they noncompliant because they’ve added new features to the underlying CSS/DOM/XML/HTML specifications? How exactly do we define “standards compliance,” especially when there are groups like the WSP (Web Standards Project) enforcing this compliance?

The WSP

I’ve long been a fan of the W3C, and I think that the Web and the Internet would be a much more chaotic environment without this organization. However, my fondness for the W3C does not necessarily extend itself to the WSP.

If you haven’t heard of the WSP, it is an example of what happens when standards enforcement is left to the masses. This organization’s intentions are pure: It’s a nonprofit organization of Web developers, designers, and artists who encourage browsers to support standards equally and completely. However, somewhere along the way, the WSP took on the aspect of a holy war, a Web jihad.

The WSP’s behavior is tantamount to lynch mob justice. After all, there are no gray areas of justice: only black and white, right or wrong. The same can be said of support for the enforcement of standards: A company supports standards 100 percent, or the company is noncompliantand, therefore, evil.

Note that I agree with the WSP in spirit: Our lives would be much easier if Microsoft and Mozilla and Netscapewould support the W3C specifications fully and equally. I’m more than aware of the cost of having to write different Web pages for different browsers because each has implemented technologies in a different way. I’ve been doing this for years.

However, I’ve also benefited when an organization has expressed an innovation that exists outside of a specification, such as the aforementioned innerHTML, or Mozilla’s support for XUL (Extensible User Interface Language). If having all browsers be 100 percent standards compliant means not having access to these innovations, then I’ll take noncompliance even if it does mean extra effort to compensate for differences.

I encourage Microsoft and Mozilla and Netscape to support the W3C specifications and other standards, but I also encourage these same organizations to continue their innovative efforts, even if the result is a bit of chaos in a world that would otherwise run smoothly, and without a wrinkle.

And who’s to say that a little chaos is such a bad thing?

The Chaos of Innovation or the Sameness of Compliance

In August 2000, CNET.com featured an article titled Why Open Standards are a Myth. The author of the article, Paul Festa, wrote that open standards only work when a company has a lead in a technology and then uses the standard as a means of ensuring that its competition doesn’t exceed its own ability. The support for standards, then, becomes a means of disabling a competitor’s innovation.

In this context, the sameness of compliance to standards becomes less a tool to help developers and businesses and more a weapon against competition. The sameness of compliance also becomes a measure of ensuring that all participants reach one level, are kept on this level, and that there are no bumps in the road of compatibility.

Is this smooth path of total compliance the Internet of the past? And is this the Internet we want in the future?

In the End

Standards are essential to doing business between companies. They are necessary to ensure that, for example, CD players can play all CDs, and elevators don’t crash to the first floor from the tenth. Our lives are protected by standards and our laws are based on them.

However, standards were never meant to be a weapon against innovation, as a tool for beating a company into submission, particularly within the free-spirited environment of the Internet.

Should we encourage the adoption of standards? A resounding yes! But not at the expense of what makes working on the Internet so challenging and exciting: The promise of something new coming through the router. Something different. Something interesting. Something innovative.

XML Expectations

Originally published at Netscape Enterprise Developer, now archived at Wayback Machine

Extensible Markup Language is a language that defines other languages. It also has the potential to give structure and meaning to the information contained in HTML documents or any other data form — making such information naturally as searchable and structured as the information locked into a database. Such capabilities mean that XML can turn our current view of data upside down — instead of a static, impenetrable lump of information, a file that uses XML suddenly has a logical structure that can be manipulated, queried, and changed without delving down into the data itself. The potential of such a meta-language is huge — if it’s implemented as an open standard. Right now, XML remains such a standard, and if it continues to evolve along open lines, it could drastically improve Web-based development.XML is similar to SQL in a number of ways; SQL is also an example of a multi-purpose language used to define data structures and query those same data structures without concern as to how the information is displayed or used. The only guarantees are that the information is defined in structures, the structures follow certain rules, and the information contained within the structures can be accessed automatically or manually. Both SQL and XML define structures for information in the form of elements, element attributes, and element content. The main difference is that instead of defining data that is stored in a physical storage medium usually only accessible by a database engine, XML describes data that is stored and accessed from within documents.

XML’s parent: SGML
XML is a subset of SGML (Standardized General Markup Language), a generalized markup language that was passed as an ISO standard in the 1980s. Rather than specifying a language’s elements directly, SGML is used to define the rules that constrain the elements of a specific language.

SGML grew out of the need to define a document’s structure and to define rules used to determine whether the document is valid and well formed. The document’s structure is defined through the use of markup tags, which delimit the elements, and Document Type Definition (DTD) files that define each element’s structure and content, providing a sort of grammar for the document.

For example, using SGML, a customer element within a document could have the following structure:

<CUSTOMER name="Shelley Powers" id="CUST011A1">
<PO id="PO23349008">
<POITEM id="POI1">
<ITEM id="14453">
Item ID: 14453
Item Desc: some description
</ITEM>
</POITEM>
</PO>
</CUSTOMER>

To validate the markups used to define the structure of the document, an associated DTD would have to be created, with statements similar to the following:

<!ELEMENT customer - - (POITEM)+><!ATTLIST customer    name CDATA   id CDATA>

This extremely simplified and abbreviated DTD uses an Extended Backus-Naur Form (EBNF) syntactic notation to create the grammar.

Using a standardized meta-language to define entities within a document allows SGML parsers to pull out the individual entities (such as the customer entity just described) and any associated attributes and content. An application can then use that information for a number of purposes, including the following possibilities:

  • To define information in a database-neutral format for transport between unlike databases.
  • To provide a search engine that allows a person to query on the entity type as well as the data.
  • For report generation, or even an online hypertext order processing form that allows the reader to drill down within the document to find the desired information.
  • To define a standard language for a specific industry or science, such as the petroleum industry or chemistry, which includes special notational conventions.

The concept of SGML is very attractive: Define a language that in turn defines a document structure used for a specific group of documents and which can be extended without impacting the underlying language generation mechanism. Unfortunately, the downside to SGML is that it is far from trivial to define the DTD for a language. SGML is a complex standard that is difficult to implement.

HTML, a derivative of SGML
SGML did, however, provide the roots of the first Web document specification, HTML. HTML was derived from SGML, except that it predefined a group of elements that controlled the delivery of a Web page’s content. In addition, the original HTML elements were expanded to include suggested presentation elements that controlled the appearance of the Web page. SGML does not control the presentation of elements, only the element structure and semantics.

The following code defines an HTML unnumbered list element, which is defined by the DTD associated with the HTML 4.0 specification as having a start and end tag and containing at least one list item:

<!ELEMENT UL - - (LI)+>

According to the EBNF associated with SGML, this DTD states that UL is an element, the double dashes assert that the element requires a start and end tag, and the element consists of at least one, and possibly more than one, list item (LI). When a user agent such as a browser parses an HTML element, it knows to look for both beginning and ending UL tags and at least one LI element contained within those tags.

Associated with the DTD for HTML 4.0 is an implied visual presentation of an unnumbered list, which is that each list item has a specified list graphic, each list item is on a separate line, and each list item lines up beneath the previous list item. However, not all user agents (such as browsers) are visual, so presentation can only be a suggestion, not a mandate.

After the first releases of the HTML specification, new elements crept into the language to provide control over page presentation. One such element is the FONT element, which controls the size, color, type, and font family of any text the element contains. The problem with using a specific tag like FONT, however, is that non-standardized tags can lead to different Web page presentations depending on the user agents.

To help differentiate an element’s structure and its presentation in HTML, the W3C issued a recommendation for CSS1, or Cascading Style Sheet Level 1, a specification that provides presentation information for HTML elements.

The real advantage of HTML was that it was relatively easy to code and display, even in different browsers. The ease of HTML was directly responsible for the massive growth of the Web. If Web document access had begun with XML, chances are you wouldn’t be reading this article right now and Web access would probably be limited to the scientific community. We initially needed a simple mechanism to create Web documents, and HTML was it. The very lack of flexibility was the language’s strength.

Now that the Web and Internet-based technologies have matured to a certain extent, enterprise developers are increasingly demanding a way to build flexibility into documents like Web pages in order to increase their effectiveness and ease of access.

Enter XML
XML arose from a need to create more generalized markup languages without having to follow the large and complex SGML standard. The XML standard still demands that a markup language be defined as well-formed, but it makes the validation step optional, which means that an associated DTD is not required (though one can be included). Additionally, XML uses only a subset of the rules for SGML, letting developers understand the principles and implementation of the technology more quickly.

Like SGML, XML is a meta-language that provides rules to define a set of tags that can be used within a document. These tags are then used to delimit an XML entity, its attributes, and its contents, and to define the elements’ syntax. These tags are read by a XML processor, which in turn provides an application with access to the entities. The application can then perform one or more actions on the XML entities.

XML processors can either be validating, which means that they make use of an associated DTD in order to ensure valid structures, or non-validating. Regardless of whether or not they are validated, XML documents can be considered to be well-formed as long as they match the XML syntax overall and as long as each entity within the document meets the syntax for a well-formed XML entity.

The main requirements for a well-formed Extensible Markup Language include the following:

  • The language may begin with a valid XML declarative statement or prolog.
  • There is one element that acts as the root element and which acts as parent to all other elements.
  • Elements are either not empty or, if they are empty, they have a “hint” encoded within the element that defines this information to the XML parser.
  • Non-empty elements must have start and ending tags.
  • All elements except the root element are contained within some element, referred to as the element’s parent; all contained elements are referred to as the parent element’s children.
  • Elements can contain character data, other elements, CData sections, processing instructions or comments.
  • Each parsed element within the document is well-formed.
  • Character data that may be processed as XML is enclosed within CData sections.
  • Documents can include comments, white space, and processing instructions.

Consider that a valid and well-formed XML document consists of the following EBNF format non-terminating symbols (non-terminating meaning that the symbols are themselves expanded elsewhere):

document::= prolog element Misc*

A complying document could be as simple as:

<?XML VERSION="1.0" ENCODING="UTF-8"?>
<ARTICLE name="XML" author="Shelley Powers"/>

This document consists of the prolog section which includes the XML declaration (“<?XML”) and includes the version number of the XML definition, as well as the encoding declaration. It also contains one element, ARTICLE, which has two attributes, NAME and AUTHOR. Since the element is an empty element, it ends with a backslash to signal to the processor that the element contains no other content. This is necessary for a non-DTD (non-validating) document. Otherwise the XML processor would not know when to look ahead in the parsing for required element content. This is one of the key features of XML: Forward processing information is embedded directly within the document, negating the necessity of creating an associated DTD.

The example just provided is a well-formed document, but not a valid one, since no DTD is provided for validation. The example also demonstrates the simplicity of XML. An even simpler version of the language would be:

<ARTICLE name="XML" author="Shelley Powers"/>

To make the document a valid one, I could have added a DTD for the ARTICLE element directly into the document, or linked to a DTD external file:

<?XML VERSION="1.0" ENCODING="UTF-8"?>
<!DOCTYPE article SYSTEM "article.dtd">

<ARTICLE name="XML" author="Shelley Powers"/>

XML in action
Though the standard is relatively new, there are several XML parsers that validate whether an XML document and its associated DTD fit the rules for a valid XML document. In addition, these same parsers may return the elements within a document exposed in their tree-like form — a form that can be used by applications.

XML is being used in the real world already. For example, Microsoft has defined an XML application it terms “Channel Definition Format,” or CDF. CDF files contain entities that describe the contents of an active channel. Following the accepted technique for XML, CDF files do not contain reference to a DTD file and instead use clues embedded within the tags and tag definitions to provide forward-looking information for the XML parser.

CDF’s purpose is to provide a document that defines the use of push technology at a specific Web site, including which pages are to be displayed as channels, what icons to display, what the update schedules are, etc. With this information, the XML processor provides the key elements that a channels-based application can use to control channel access on the Web site.

The following code shows the CDF file I have defined for use at my personal Web site. The root element for the file is the CHANNEL element. It is the parent element for several other elements, such as an ICON element, an ITEM element, and an ABSTRACT element. Each of the elements within the document may or may not have attributes, and a child element may in turn be the parent for another element:

<?XML VERSION="1.0" ENCODING="UTF-8"?>
<CHANNEL HREF="http://www.yasd.com/plus/index.htm" 
        BASE="http://www.yasd.com/plus/">
    <TITLE>YASD+</TITLE>
    <ABSTRACT>YASD+ pages, using the newest technologies</ABSTRACT>
    <LOGO HREF="http://www.yasd.com/mm/wide_logo.gif" STYLE="IMAGE-WIDE"/>
    <LOGO HREF="http://www.yasd.com/mm/logo.gif" STYLE="IMAGE"/>
    <LOGO HREF="http://www.yasd.com/mm/icon.gif" STYLE="ICON"/>
    <SCHEDULE>
        <INTERVALTIME DAY="1"/>
        <EARLIESTTIME HOUR="0"/>
        <LATESTTIME HOUR="12"/>
    </SCHEDULE>
    <ITEM HREF="http://www.yasd.com/samples/bytes/daily.htm">
        <LOGO HREF="http://www.yasd.com/mm/icon.gif" STYLE="ICON"/>
        <ABSTRACT>YASD Code Byte</ABSTRACT>
    </ITEM>
    <ITEM HREF="http://www.yasd.com/samples/bytes/cheap.htm">
        <LOGO HREF="http://www.yasd.com/mm/icon.gif" STYLE="ICON"/>
        <ABSTRACT>Cheap Page Tricks</ABSTRACT>
    </ITEM>
</CHANNEL>

Notice that the first line contains the XML declaration element, a version number, and an encoding declaration. The main entity within the document is the CHANNEL entity, enclosing other elements such as TITLE, ITEM, ABSTRACT, and LOGO. Each of these elements falls within the allowable XML definition for elements:

element ::= EmptyElemTag | STag content ETag 
EmptyElemTag ::= '<' Name (S Attribute)* S? '/>'
STag :: = '<' Name (S Attribute)* S? '>'
ETag::= '</' Name S? '>'
content ::= (element | CharData | Reference | CDSect | PI | Comment )*

Without continuing to resolve the non-terminating references, what the syntax just shown states is that each element is either an empty element, in which case it ends with a backslash/angle bracket combination (‘/>’), or it has start and end tags which enclose content. A “well-formed” constraint is placed on the start and end tags in that the NAME used in both is the same. The enclosed content can include other elements, comments, processing instructions, or other well-formed XML entities. Both empty and non-empty elements can have zero or more attributes, as the following demonstrates:

<CHANNEL HREF="http://www.yasd.com/plus/index.htm" 
        BASE="http://www.yasd.com/plus/">
...
</CHANNEL>

or

<INTERVALTIME DAY="1"/>

Internet Explorer 4.0 has an associated XML parser that pulls the element information out of the document. IE 4.0 uses this parsed element information to create the channel for the Web site, including the two sub-channel items, as shown below:

This site has a main Web page channel, denoted by the top-level graphic, and two sub-channels, with the second sub-channel loaded into the browser.

Accessing the CDF file directly with IE 4.0 opens a dialog box asking the individual how they would like to subscribe to the site’s channel, and allowing the reader to determine how and when the channel contents are downloaded to their client machine.

Other uses of XML
In addition to CDF, Microsoft and Marimba, Inc. have also proposed XML-based technology called the Open Software Description (OSD) format, which can be used to control software downloads and installations over a corporate network. A major IS expense for larger corporations, especially those that are geographically distributed, is installing and maintaining software upgrades on employees’ desktops. One small upgrade to a popular piece of software can take days of planning and weeks of actual implementation (i.e., walking around to each person’s desk and installing the upgrade). During the upgrade rollout, employees will have different versions of the same software, which can create problems. With OSD, software upgrades can be handled automatically using push technology, reducing both IS staff hours and logistical problems.

SGML and XML have both been used to create a Chemical Markup Language (CML) for the chemistry community. With the CML vocabulary, molecular structures can be defined within a document and the information can be either posted or transmitted. XML processors can pull out the CML elements and pass these to applications that perform actions like preparing a print-out of the information, either textually or graphically, or creating an online three-dimensional model of the information using VRML or some other 3D technology.

Netscape, Apple, and others have proposed a Meta Content Framework (MCF) created in XML that can expose a Web site structure for navigation or online exploration. MCF can be used to do such things as generating a three-dimensional site map which can be used for Web site publication and administration. The technology is currently used by Apple’s ProjectX/HotSauce browser, and “Xspace”-compatible content can also be viewed using a plug-in available from Apple.

XML can also be used to define a relational database meta-language, which can then be used to describe documents containing relational database information. These same documents can be easily generated from the relational database dictionaries, which are repositories of information about the information stored in the database. The extensible markup language can then be used to create context-centered documents like “all information pertaining to any purchases, week of January 16 through January 23,” rather than using the context-neutral database format. In addition, supporting information that is not part of the data in the database, like images or reference material, can be pulled into the document.

An XML processor can process this context-based data document and use the information therein to present reports, perform online research and queries, or even to create interactive three-dimensional models of the data. Instead of issuing a SQL statement such as:

select customer_name, customer_address, city, state, zip_code from customer, 
purchase_orderwhere purchase_order.order_id = 32245 and 
customer.customer_id = purchase_order.customer_id;

I could enter a three-dimensional VRML world at a purchase_order portal and scan a virtual filing cabinet for my purchase order number. Once I find it, I can open the door into another room with doors labeled “Purchase Order items” and “Customer” and open the Customer door into another room containing the information I am looking for. Best of all, the documents containing the context-based data could be generated automatically, processed automatically, and presented automatically. This means a change in the database table could be handled automatically.

Besides three-dimensional database applications, defining data in an XML document could be used as a method to convert database data in one format, such as relational data, into another format, such as object- based database records. The resources section at the end of this article has a reference to a preliminary XML representation of a relational database.

In addition, with XML processors (or XML parsers, if you prefer), the most difficult aspect of XML has already been implemented: pulling the entities out of the document.

Returning to the CDF example, not only can the XML document be used by Internet Explorer 4.0 to provide information about the structure of a Web site’s channels, I can also access the XML entities using JavaScript, C++, or Java and use the information for other purposes. For example, the following JavaScript functions open a CDF file, pull out information about the elements contained within the CDF file, and print out this information in a newly opened window.

<script language="jscript">
<!--
var doc = new ActiveXObject("msxml");
var wndw = null;

// display elements in CDF file
// file reference must be fully resolved Internet reference
function DisplayElements(cdffile)
{
// Display this with an appropriate message in a popup window
wndw = window.open("","CDFFile",
"resizable,scrollbars=yes");
wndw.document.open();
doc.URL = cdffile;

// begin displaying elements at root
displayElement(doc.root);

wndw.document.write("</body>");
wndw.document.close();

}

// display element tagname, if any
// and information about element such as any attributes (even 
// if undefined for element) and text and element type
function displayElement(elem) {
if (elem == null) return;
wndw.document.writeln("<p>");
if (elem.type == 0)
    wndw.document.writeln("Document contains element with 
                           tagname: " + elem.tagName);
else
    wndw.document.writeln("Document contains element with no tagname");
wndw.document.writeln("<br>Element is of type: " + 
                                GetType(elem.type) +"<br>");
wndw.document.writeln("Element text: " 
                                + elem.text + "<br>");
wndw.document.writeln("Element href: " 
                                + elem.getAttribute("href") + "<br>");
wndw.document.writeln("Element base: " 
                                + elem.getAttribute("base") + "<br>");
wndw.document.writeln("Element style: " 
                                + elem.getAttribute("style") + "<br>");
wndw.document.writeln("Element day: " 
                                + elem.getAttribute("day") + "<br>");
wndw.document.writeln("Element hour: " 
                                + elem.getAttribute("hour") + "<br>");
wndw.document.writeln("Element minute: " 
                                + elem.getAttribute("min") + "<br>");

// check to see if element has children
var elem_children = elem.children;
if (elem_children != null)
   for (var i = 0; i < elem_children.length; i++) {
      element_child = elem_children.item(i);
        displayElement(element_child);
   }

}

// element type
function GetType(type) { 
if (type == 0) 
        return "ELEMENT"; 
if (type == 1) 
        return "TEXT"; 
if (type == 2) 
        return "COMMENT"; 
if (type == 3) 
        return "DOCUMENT"; 
if (type == 4) 
        return "DTD"; 
else 
        return "OTHER";
}

//-->
</script>

See the Resources section for a pointer to an XML demonstration.

Creating an XML document
A key to the true usefulness of XML is that once an XML parser has been created to process an XML document, you can use it to parse out entity information from any document containing any well-formed XML content.

In the last section, I used Internet Explorer’s ability to parse XML entities, attributes, and content to create a Web page that listed the entities, their attributes, and some content. An interesting example, but not really useful. But what if I were to define my own XML document, including my own XML entities and attributes, and then use IE’s built-in XML parser to create my own graphic menu Web page application? This is fairly simple and only took a couple of hours of playing around to accomplish.

First, I defined my own CDF file and created my own entities, as shown here:

<?XML VERSION="1.0" ENCODING="UTF-8"?>

<DOCUMENT >
    <TITLE>YASD+</TITLE>
    <STYLESHEET HREF="http://www.yasd.com/css/daily.css" />
    <ITEM HREF="http://www.yasd.com/plus/plus.htm">
        <IMAGE HREF="http://www.yasd.com/plus/logo.jpg">
        <ALT>YASD+ Main Page</ALT>
        </IMAGE>
    </ITEM>
    <ITEM HREF="http://www.yasd.com/samples/bytes/daily.htm">
        <IMAGE HREF="http://www.yasd.com/plus/logo.jpg">
        <ALT>YASD Code Byte</ALT>
        </IMAGE>
    </ITEM>
    <ITEM HREF="http://www.yasd.com/samples/bytes/cheap.htm">
        <IMAGE HREF="http://www.yasd.com/plus/logo.jpg">
        <ALT>YASD Cheap Page Tricks</ALT>
        </IMAGE>
    </ITEM>
</DOCUMENT>

I redefined what ITEM is, created a new root element called “DOCUMENT,” and added some new elements of IMAGE, STYLESHEET, and ALT. I followed the XML convention for well-formed entities — opening up this document for parsing within IE 4.0 generates no errors.

I then created an application, consisting of two frames, that uses the images associated with the items to create a graphical menu bar in the top frame of the window and set the link associated with each image to open in the bottom frame of the window. The window originally opens with the form to access the CDF file and process its contents. This form is then overwritten with the processing results. The code for the form and to process the form contents is as follows:

 
<script language="jscript">
<!--
var doc = new ActiveXObject("msxml");
var wndw = null;

var title = "";
var stylesheet = "";
items = new Array();
itemimages = new Array();
itemalts = new Array();
ct = -1;

function createWindow(cdffile)
{
doc.URL = cdffile;

// find main document and any associated item documents
findElements(doc.root);

// if associated documents
if (ct > 0) {
  var strng = "<HTML><HEAD><TITLE>" + title + 
        "</TITLE><LINK REL=STYLESHEET TYPE='text/css'" +  
        " HREF='" + stylesheet + "'></HEAD><BODY>";
  for (var i = 0; i <= ct; i++) 
     strng+="<a href='" + items[i] + 
                "' target='Body'><IMG src='" + itemimages[i] + "' ALT='" + 
                itemalts[i] + "' border=0>" + 
                "</a>"; 
  strng+="</BODY></HTML>";
  document.open();
  document.writeln(strng);
  document.close();
  }
}

// display element tagname, if any
// and information about element such as any attributes (even if undefined for element)
// and text and element type
function findElements(elem) {
if (elem == null) return;
if (elem.type == 0) {
    if (elem.tagName == "TITLE")
        title = elem.text;
    if (elem.tagName == "STYLESHEET")
        stylesheet = elem.getAttribute("href");
    if (elem.tagName == "ITEM") {
        ct++;
          items[ct] = elem.getAttribute("href");
        }
    if (elem.tagName == "ALT") 
        itemalts[ct] = elem.text;
    if (elem.tagName == "IMAGE")
        itemimages[ct] = elem.getAttribute("href");
    }
        
// check to see if element has children
var elem_children = elem.children;
if (elem_children != null)
   for (var i = 0; i < elem_children.length; i++) {
      element_child = elem_children.item(i);
        findElements(element_child);
   }
}
//-->
</script>

I could have defined any elements within the XML document as long as I used well-formed XML entities, and I could process the results in virtually any way I desired just by using simple scripting techniques.

Linking and style information
In addition to the XML specification, other efforts are currently underway to add supporting specifications. The first is XML part 2, which includes linking. Another is XSL, the Extensible Style Language, which defines an XML stylesheet.

Linking has been extended considerably with XML. You can specify an attribute that determines how a resource is displayed, specify whether the resource is displayed automatically, and even specify multiple layers of linkage. Of particular interest is the capability to define a group of links, associating documents together in such a way that the person following the links does not have to hunt around for related documents. If you have ever jumped to a Web site page by following a link from another site, you know how frustrating it can be to try establish the context of the link in order to find related documents.

XSL would be specified using XML and would provide a way to define presentation elements, such as those used currently in HTML. For example, HTML includes the Emphasis element, delimited with <EM> </EM> tags, the Strong element, delimited with <STRONG> </STRONG>, and others. With XSL, you could create styles to provide recommendations for how an XML entity is rendered.

The downside to XML
While XML’s implementation-neutral technique allows parsed information to be used for multiple purposes in multiple applications, it is this same flexibility that may cause problems.

Returning one last time to my CDF example, I created a simple JavaScript application that opens the main channel page and all the associated pages into a frames-based Web page. The main page opens into the top-most frame, and each individual CDF ITEM element opens into one of the smaller frames located along the bottom of the document.

This isn’t a problem for my own CDF file, which is relatively simple. Applying the same application to another CDFfile, however — one I neither created nor control — creates a Web page that probably does not meet the expectations of the page’s designer. The following screen shot shows the result of using the frames-generation application on the IDG.net channel:

To create this page, I used a publicly accessible file, IDG.net’s CDF file, and exposed the XML elements to create a presentation neither Microsoft nor IDG.net intended. Even with the new effort on XSL, currently only a W3C proposal, there is no guarantee that the information exposed with XML will be used for anything approaching the intended purpose of the XML document’s original creator.

Another potential problem area with XML is the CDF specification. CDF’s potential is great; you could use it to build an XML-based document that could be used by different push technology vendors with relatively comparable results. But what happens if a vendor supports channels but doesn’t want to use CDF? Do we end up with different “flavors” of channels? Does the W3C then create a different standards specification for channels, another for chemistry, another for math, another for finance, and so on in order to ensure that only one specification for each “topic” or “business” is created? Or can we design tools for translating between each of the XML document definitions?

In conclusion
Even with these issues at stake, XML is a terrific addition to Web and other application development. One of the most difficult aspects in application programming is extracting the structure as well as the contents of documents. XML has made this process a whole lot easier.

During the recent XML/SGML conference in Washington DC (December10-12), XML became a proposed recommendation of the W3C, the last remaining step before becoming a real recommendation. It may be only a matter of time before XML is just as common as SQL is today.