<< Politics: Profiles in Liberal Hypocrisy | Home | Politics: Bush - 2000+ vs. Clinton 3000+>>
Design: Dreamweaving Domino and XML into an AJAX Feed Reader
Fri 28 Oct 2005
What do you get when you combine Domino, Dreamweaver, AJAX and XML in a bowl, mix it up and take a big healthy slurp? Not indigestion... ;-)
How to have your cake and eat it.
By Mark Barton and Jerry Carter
By now, you should know what AJAX is. If not, you should read an introduction to the concept. If you've been thinking there must be a cool way in which AJAX can be used with Domino, you should take a look at Richard Schwartz's list of Domino AJAX projects and articles. In this article, we (Mark Barton and Jerry Carter) have teamed up to show you a light-weight implementation of AJAXin Domino that allows you to incorporate information from other data
sources in your pages (think 'portal'). Mark has put together a nice
demo that allows you to view multiple RSS /
Application feeds from a single page, all served from separate servers.
We've written this article for web developers who are comfortable with
their knowledge of CSS, server side scripting concepts, and an API or two, but who may be unfamiliar with AJAX, XML, XSLT, or the finer points of roasting duck. Buckle up and strap on the feed bag!
More and more web applications are exposing rich data sources via XML streams. Though RSS in all of its flavors is probably the best known incarnation of XML, this is not the only source we can make use of. Notes Views or MS Office 12 documents are other examples with potential. As Notes Applications can easily expose XML streams via views, using an AJAX
client in one application presents a cheap way to integrate Domino
applications from across your domain, and even from without your
domain. Add to that the possibility of mixing feeds from other, non
Domino, sources and we start to have something really compelling on our
hands (prior art nod belongs to this delightful quote from Tim
O'Reilly's Web 2.0: Compact Definition"consuming
and remixing of data from multiple sources, while providing their own
data and services in a form that allows remixing by others".)
The demo Mark has cooked up is a simple, low-calorie, feed reader. Like any feed reader, we'll be pulling unformatted XML
as a stream with minimal intervention. This means formatting should
happen on the fly. Also, since the external feeds are of an unknown
size, we don't want to warehouse any data in our application either
(except for caching purposes). To keep our implementation generic and
handle the formatting, an XSLT style sheet was selected for use. If you're like most casual diners, the mere mention of XSLT
brings to mind cryptic standards and black-box magic (or deep, dark,
Turkish coffee.) But this really is a quick and simple way to avoid
heavy processing while allowing us a reasonable representation of
divergent source data (read as: "just make it work".)
NB This
article and subsequent demo concentrates on merging and displaying
feeds from other applications but does not try and parse the XML into data we can understand and utilise e.g. Bob Obringers AJAX View Object
XML converted to an HTML fragment is not new technology - then again nor is AJAX ;-) So, what's triggered the interest? Macromedia's tasty new release, Dreamweaver 8, which provides relative ease of creating XSLT stylesheets in a cheerful, vegematic sort of way. The Notes R6 LS Classes combine with Dreamweaver's XSLT to do the grunt work for us leaving us free to ponder important things, like which wine to serve with boiled muskrat.
NB While the LS Classes are specifically for DXL processing, they will work with any XML. We'll tackle that in a moment.
So, what is this mysterious bit of goo, XSLT?
The Dreamweaver help has this helpful description:
Extensible Stylesheet Language Transformations (XSLT) is a subset language of XSL that actually lets you display XML data on a web page, and "transform" it, along with XSL styles, into readable, styled information in the form of HTML. You can use Dreamweaver to create XSLT pages that let you perform XSL transformations using an application server or a browser. When you perform a server-side XSL transformation, the server does the work of transforming the XML and XSL,
and displaying it on the page. When you perform a client-side
transformation, a browser (such as Internet Explorer) does the work.
So, all together, "T is for transformation". We still have no idea what XSL is though. Webopedia to the rescue.
Short for Extensible Style Language, a specification for separating style from content when creating HTML or XML pages. The specifications work much like templates, allowing designers to apply single style documents to multiple pages. XSL is the second style specification to be offered by the World Wide Web Consortium (W3C). The first, called Cascading Style Sheets (CSS), is similar to XSL but does not include two major XSL's innovations-allowing developers to dictate the way Web pages are printed, and specifications allowing one to transfer XML documents across different applications. W3C released the first draft of XSL in August 1998, and promotes the specifications as helpful to the Web's speed, accessibility, and maintenance.
Ok, so XSL is essentially a scripted version of CSS. The options we have to choose from then boil down to server side or client side?
The approach you ultimately take (server-side transformations versus client-side transformations) depends on many factors:
- What you are trying to achieve as an end result
- the technologies available to you
- the level of access you have to XML source files, and other factors.
- What is the expected number of visitors/users of the application?
- How much bandwidth, per transaction, am I going to consume?
- Do you wish to embroil (or broil) yourself in more testing of more complex client side code or lean towards less code with more performance and configuration overhead in a server side solution?
Both approaches have their own benefits and limitations.
For example: Server-side transformations work in all browsers while client-side transformations are restricted to modern browsers only (Internet Explorer 6, Netscape 8, Mozilla 1.8, and Firefox 1.0.2). Also, server-side transformations let you display XML data dynamically from your own server or from anywhere else on the web, but require that you deploy your pages to a configured application server.
Client-side transformations, on the other hand, must use XML data that is locally hosted on your own web server but only require access to a web server.
Clearly, there are several architectural decisions to make.
Given our objective to integrate data from various locations, we will use server-side transformation of the XML for performance to the end user, and AJAX to update the page with any new content, thus splitting the work into UI and backend components.
Into the Kitchen
Our example application does the following:
- Create and store configuration documents in a Notes Database (settings include: XSL, URL to the XML, any cache settings).
- When an appropriate page is opened in a browser 1 or more AJAX calls are made to return the required HTML fragments.
- The Domino Server will determine if the cached content requires refreshing and if so will get the required XML.
- The Server will then perform the transformation using the stored XSL.
- The Server will then store the HTML fragment on the page and finally return it back to the AJAX call.
Ingredients for a successful AJAX / XML / XSLT Server Transformation
- Either take 1 full size R6 / R7 Domino Server if planning for a feast, or a R6 / R7 client for a simple tasting.
- Add a dash of Windows to allow you to use the Microsoft's XMLHTTP Object
- Sprinkle in some Lotus script to use the NotesXSLTransformer class.
- Get that tattered old cookbook out and use the following Javascript libraries: onLoad busted script , CSS Query . Keep an eye on the oven temp with A List Apart article
- Finish off by stirring in some good old fashioned Notes configuration documents and a smidge of LS Custom class.
- Present using your favorite browser, with a pint of %favorite_beverage.
Step by Step Preparations for "ready to eat"™ XSL
(0. Get Dreamweaver 8 or download the 30 day evaluation from here http://www.macromedia.com/software/dreamweaver/?promoid=BINR)
- In Dreamweaver create new XSLT Fragment.
- Enter location of XML Feed at the prompt - we'll use the BBC News RSS Feed example
If you have problems connecting through Dreamweaver then download the XML file to your local machine and use this.
- Find the item that repeats, in this case the news article - going to skip the channel information to keep the example simple.
- Use Insert - XSLT Object - Repeat Region
- Select Item - NB Not the Channel item, we're only after the articles, as we've said
- Inside "repeat region" select Insert ? XSLT Object ? Dynamic Text
- Select Title
- You can preview from Dreamweaver to test out your freshly baked transformation.
Tip To make things more functional, you will probably
want to link back to the article as well since most feeds only show
summary information and / or display images from the feed e.g. Flickr.
Here is a sample XSL to get you started (the example database has more.)
< div class="item">
< h3>
< a> < xsl:attribute name="href"> < /xsl:attribute>
< xsl:value-of select="title"/> < /a> < /h3>
< p> < xsl:value-of select="description" disable-output-escaping="yes"/> < /p>
< /div>
This will result in:
For images, this styling works nicely (taken from the Flickr feed.)
<img>
< xsl:attribute name="src">
< xsl:value-of select="media:thumbnail/@url"/>
< /xsl:attribute>
< /img>
The "tastes just like chicken!"™ Example Database
The example database was built in R6 and can be downloaded here
The
database should be signed by you, or someone with sufficient agent
execution privileges, and watch out for the ProcessRequest Agent as it
needs unrestricted access to run correctly - this will not be so much
of an issue if you run the example database on your local machine.
Either way it needs to reside on a Windows machine.
Important - If you decide to implement the various components in your own database the code expects the JS Logging
part to be included (though it does not have to be displayed to the
user). A complete list of dependent resources appears at the end of
this article.
The database will display this page if you launch the database in a browser and when you click on one of the top links it will perform an AJAX
call to populate the corresponding content. Smooth (like pudding!).
Notice the loading image being displayed to the user - this is
important from an end user point of view as it let them knows something
is going on. You could also change the cursor to an hour glass. To
display the image I added an additional argument to the AJAX
library method getXMLText. If true the image is displayed. You may in
some cases not want to display the loading image (as in the case of the
onLoadTest demo below).
The onLoadTest
(also available as a link on the home page) displays a simulation of a
possible real world implementation. Notice the nicely color coded
output at the bottom. This logging comes to you courtesy of the self
sacrifice and hard work of the fvlogger Javascript library taken from
the A List Apart article.
Configuration Documents
There is one configuration document per XML feed and is modified / created in the Notes client (sorry, not a pure web tool, but come on, you miss the client sometimes, don't you? It misses you!)
The
configuration document is used to hold not only the cached version of
the transformation but the various settings as well. Cached? Yes. More
on that in a moment.
Explanation of fields
Title
This is used for your reference - not referred to in the code.
XSL
This Rich text field holds the XSL you created in Dreamweaver from the above steps. This field needs to be rich text as the LS NotesXSLTransformer class takes a stream as an input and you can create a stream from Rich Text.
XML URL
This field holds the URL which points to the XML file that you wish to use.
Cached Content is valid for
These 3 fields determine how long to reuse the cached transformed content before fetching new XML content. Leave blank if you want the transformation to be performed every time the user requests the page.
HTML Target Container (ID)
This field should be the ID of the HTML
container that will hold the resultant transformed content. We use this
value as a unique key. The presence of a container with this ID on a
page (with the appropriate Javascript) will result in it being
automatically populated.
Cache Date
This display field holds the date and time when the cache content was built.
Cached XML
For reference purposes a rich text field holds a copy of the XML that was retrieved
Transformed HTML
For reference purposes a rich text field holds a copy of the transformed XML (HTML)
NB There
are a series of hidden plain text fields which are created when the
transformation takes place. These hold the transformed HTML
and need to be split to allow for the 64Kb internal limit. These fields
are used in lookups on the web page to display the cache version of the
content. Mark previously discussed this technique here
Site Configuration Document
This
is a staple from the Mark Barton pantry often included in Domino
applications to allow for the specification of various items which are
then included on every web page.
Have a look if you're interested, else you can ignore it for this demo.
onLoad Demo Page
The onLoad demo uses the OnLoadTest form and displays a possible real world use of this technique.
The form contains a series of HTML
containers (DIVs in this case) which are initially populated by
performing a @dblookup to the appropriate Configuration Documents.
The formula looks like this:
REM{The first Plain Text Field will contain the number of lookups we need to do.};
id:="bbc";
x:=@DbLookup("":"";"";"(LUHTMLTarget)";id;"PlainHTML_1";[FailSilent]);
numberLoops:=@TextToNumber(@Left(x;"%"));
FirstLookup:=@RightBack(x;"%");
temp := FirstLookup;
@If(x="";"";numberLoops>1;@Do(
@For(n := 2; n <= numberLoops; n := n + 1;
x:=@DbLookup("":"";"";"(LUHTMLTarget)";id;"PlainHTML_" + @Text(n);[FailSilent]);
temp:=temp+ @RightBack(x;"%%")
));"");
@Trim(temp)
In this case we are going to load up a BBC RSS Feed. The only part you would need to change in this formula for a different feed is the id:="bbc" part. This refers to the HTML Target ID field on the Configuration Document.
The
rest of the formula loops through the appropriate number of lookups to
retrieve all of the plain text fields which hold the transformed HTML. The number of iterations are held as the first parameter of the first field and is delimited with a '%'.
The resultant cached content is displayed to the user immediately.
Once the page has loaded (we use Dean Edwards onLoad busted script so we do not have to wait for images to load first) we iterate through all of the elements which have a CSS Class of update-single again using a Dean Edwards script - CSS Query
We then make an AJAX call for each element - we use Richard Schwarz's AJAX framework for this. (I know you can't technically eat pot-puree, but…) The AJAX call passes the HTML Elements
ID to an agent. The agent then refers to the configuration document and
determines whether the cached content is out of date (or even there).
If new content is required the following happens:
- The XML
is fetched using the Microsoft Microsoft.XMLHTTP Object (This is the
reason why it needs to be a windows machine - converting to Java would
get around this issue, but for most folks, finding a windows machine to
cook with shouldn't be an issue.) The XML is stored in a rich text field.
- The XML is transformed using XSL (taken as a stream from the rich text item).
- The resultant HTML is printed out and therefore returned to the AJAX call.
- The AJAX framework ensures the content ends up in the right container.
To clarify, to 'enable' a content container to be automatically updated, just make sure it has a CSS class of "update-single" (& the configuration document is in place).
Side Notes
Error Checking
The demo database contains no error checking - We had to leave something for you to do :-)
Javascript Debugging Script - from A List Apart
To help with debugging the AJAX scripts I found it easier to use a Javascript debugger as featured in the A List Apart article.
This debugger is included in the demo and is turned on in the site configuration document.
XMLHTTP Object
As
mentioned this object makes this a Windows only demo but the object
does have some useful properties which are easy to use. e.g. Caching or
Authentication for secure feeds.
XSLT
Though Dreamweaver can get you started there will still be tricky XSLT hurdles you need to get over - Google can be your friend here - example the A Link.
The Final Tasting
A veritable cornucopia of technologies comes together to make for a steaming plate of (no, not fermented shark)
yummy goodness. ;-) Another fine example of the flexibility (and hence,
power) of Domino. Right, off to the pub. mmmm… shepherds pie sounds
strangely good and wholesome after contemplating that boiled muskrat.
These are the design elements you will need to copy into your own database if you want to reuse the same functionality:
Forms
- NOTES - XML - XSL Definition
Views
- Definitions (Notes View)
- (LUDefinitions)
- (LUHTMLTarget)
Agents
- (ProcessRequest)
Subforms
- WEB - Javascript Logging
Script Libraries
- AJAX.js
- class:TransformXML
- General
- logger.js
- URL_utils
Images
- loading.gif
Files
- cssQuery-p.js
- ie_onload.js
Mark Barton
has been an IT consultant for 12 years and is currently living in
London, England. He enjoys taking pictures of small fuzzy animals. Jerry Carter has been an IT Consultant for ten years and is presently living in rural Ohio, USA. He enjoys hunting for and subsequently cooking small fuzzy animals.