The system is based on the following fundamental principles:



Developing Dynamic, Database-Driven

Web Pages by Demonstration

David Wolber, Yingfing Su, Yih Tsung Chiang,

University of San Francisco

2130 Fulton St., San Francisco, CA. 94117

(415) 422-6451 wolber@usfca.edu

ABSTRACT

This paper describes an exploratory environment for the end-user development of dynamic web pages. The system combines techniques from query by example (QBE), programming by example (PBE), and the spreadsheet paradigm. In combination, these techniques allow a designer to visually specify web pages that display and manipulate persistent (database) data.

INTRODUCTION

Many WYSIWYG editors allow designers to create static web pages without writing code. But with the proliferation of dynamic pages, and specifically those that allow access and manipulation of server databases, these editors are not sufficient.

A number of tools have emerged which allow the presentation of dynamic web content to be specified in similar way to how static pages are constructed. With these tools, the designer specifies formatting and layout on representative data. Representative data is a sample of what will actually be presented when a browser renders a page.

For instance, with Enhydra's XMLC tool, the designer enters and formats data for a representative row in an HTML table. The programmer then writes code which replaces the sample row with real records from a database, keeping the formatting intact.

Such tools improve the development process by separating designer and programmer concerns. But they restrict the designer, at least those that do not program, to presentation concerns only. This is unfortunate, especially since non-programmers have long used database management systems, spreadsheets, and programming by example systems to specify non-visual computations.

Macromedia's UltraDev 4 (formerly Allaire's Cold Fusion) is a commercial system that does allow non-programmers to visually specify more than just presentation. UltraDev 4 provides dialogs, similar to those found in a DBMS, which allow a developer to map a visual component to a data source. Without any programming, a designer can create a complete dynamic web page that displays live database data, and dynamic web pages that allow end-users to manipulate that data.

UltraDev4 dialog-based specification eliminates coding syntax and thus increases the pool of designers capable of developing dynamic web pages. But is dialog-based specification the optimal method for visually specifying dynamic web pages?

Our research has focused on exploring alternative visual techniques. We have built a development system, DemoWeb, that combines techniques from a number of areas, including query by example (QBE), programming by example (PBE), and spreadsheet applications. Informal user studies suggest that these techniques can improve the process of developing dynamic web pages.

OVERVIEW

DemoWeb is based on a WYSIWYG HTML/XML editor. Designers can use it to create static pages, or they can add dynamic components to a page, in which case the system generates a servlet to render the page at run-time.

Dynamic Components

One of the most common dynamic components is the dynamic table. The designer inserts it into a page just like any static component. The designer can then use a dialog to map some or all of its columns to data in a pre-existing database. Alternatively, the designer can enter sample data into the table, then specify that DemoWeb should create a new database table with fields matching the sample data columns.

The granularity of the mapping is not dynamic table to database table, but dynamic table column to database table column. Thus, a dynamic table component can display data from various database tables. It can also contain “formula” columns not mapped to a database table (see below).

Formatting By Example

Once a dynamic table is mapped to a database table, the designer specifies the formatting by example. The system displays the first row of live data (data in the mapped database table) in the first non-header row of the dynamic table component. The designer can then apply fonts, colors, and other formatting to the data in each column. At run-time, all of the rows of the database table are displayed with the formatting the designer has demonstrated.

For instance, if the designer italicizes the name “Wolber”, as in Figure 1:

Table: STUDENTS

Name Test1 Test2

|Wolber |87 |90 |

| | | |

the rendered page (created by the system generated servlet) would display all names with italics.

Query By Example

The designer also specifies queries by example in the second non-header row of the dynamic table. The designer enters a boolean expression (e.g., ">85") in a particular cell.

Table: STUDENTS

Name Test1 Test2 Average

|Wolber |87 |90 | |

| | |>85 | |

The system generates the SQL query (Select * From STUDENTS WHERE TEST2>85) so that, at run-time, only selected rows of the mapped table are displayed in the table.

Spreadsheet Functionality

If a column in a dynamic table component is not mapped to a database, the designer maps it to a spreadsheet-like formula. The formula can reference any of the other columns in the row. For example, a designer could specify "=AVERAGE(C1:C4)" for column 5. At run-time, the fifth column of each row will display the average of the dynamic database data in the previous four columns of that row.

Table: STUDENTS

Name Test1 Test2 Average

|Wolber |87 |90 |=Average(C2:C3) |

| | |>85 | |

Programming By Example

The QBE and spreadsheet techniques described above allow for the development of web-based database views, but they don't provide a means for developing pages that manipulate a database.

Consider, for instance, a simple form page that allows the end-user to add a record to an existing database. UltraDev4 …

DemoWeb provides a PBE technique for creating such a page. The designer first enters sample data in the components of the form. The designer then selects the form's submit button and chooses Record Response from the DemoWeb toolbar. Record Response is similar to the macro recorders in common desktop applications. However, in this case, the purpose of recording operations is not so this new "macro" can be replayed in the development environment. Instead, the designer is specifying the operations that should be executed at run-time when a submit button is selected. Also, the recorded operations will not be executed verbatim-- the system will infer generalized operations.

After selecting record mode, the designer enters new data in the dynamic table component for which the record should be added. For this example, let's assume the designer enters data in the table that matches the data in the form (he might even copy and paste). When the designer chooses "Stop Recording", the system compares the before and after states of the interface to identify the changes. Since the designer demonstrated entering the new data in a dynamic table, the system interprets the operation as a not just a change to the display, but a change to the underlying persistent store (database), i.e. a Table.AddRecord(d1,d2..) operation. And, since the designer entered data in the table that matches the data in the form, the system generates a generalized AddRecord command where d1,d2,etc. are the names of form elements.

The PBE technique described eliminates the need for some of the abstraction necessary if programming or even dialog-based specification is used. From the designer's perspective, no name is necessary to refer to the AddRecord operation, and names are not necessary for the parameters of that operation (i.e. the names of the form elements). Instead, the designer demonstrates the AddRecord operation by providing an example of what should occur when the submit button is clicked. The system performs the pattern matching and generates code for AddRecord and its parameters. The designer need only view this code if modification of it is desired.

Generated Code

WebDemo generates an HTML document and a Java servlet from the designers WYSIWYG and visual specifications. The HTML document is funneled through Enhydra's XMLC compiler. XMLC creates Java code from an HTML file. When executed within a servlet, the code renders the same exact page as if the HTML file was sent directly to a browser.

Of course the purpose of XMLC is to create a basis data structure so that a programmer can easily add dynamic content to the page. The Java code created by XMLC is based on a tree data structure fitting the Document Object Model (DOM) standard. A programmer, or in this case, the WebDemo code generator, creates code that manipulates the DOM tree to insert dynamic content.

WebDemo generates a servlet that contains both JDBC code to access any mapped databases, and DOM manipulation code that puts that data into the HTML sent to the browser.

Servlet including JDBC and DOM manipulation code

HTML Java DOM code

Figure 3. Architecture of Generated Code

For example, if a dynamic table component is mapped to a database table, WebDemo generates code like in the following example:

RELATED WORK

WebDemo combines ideas from numerous areas of computer science: the human-computer interface field (in particular PBE), the database field, those studying applications and extensions to spreadsheets, and those interested in web development.

PBE

Numerous PBE systems have been developed and documented in two collections of research papers[]. One fundamental idea behind this research is that of programming in the interface[], that is, allowing the designer to create programs in the same interface in which the program is run. Instead of formulating abstractions like variables, the designer demonstrates operations on sample data. The system records the examples then infers or deduces a general program. PBE has been applied to numerous applications, including text and graphics editors[] and spreadsheets. Some research efforts have also focused on application-independent PBE.

A number of systems have applied PBE principles to dynamic web pages, mostly focusing on the induction of wrapper programs from examples. A Wrapper program is a web page that displays data from one or more other web pages. It generally collects input from the user, then sends it a set of one or more web pages, e.g. a number of airline pricing pages. The wrapper program then identifies the desired data on the resultant page(s) and displays this collection of data on the wrapper program page.

Wrapper induction systems eliminate coding by allowing the designer to point out the pieces of data in an existing page that should be collected by the wrapper program. The system then infers a description of the data, often using contextual and landmark information so that, if the existing page changes slightly in the future, the wrapper will still work.

As far as we know, PBE has not previously been applied to the development of new web pages that connect directly to databases. However, examples have been used in databases for years, as the following section explains.

QBE

Back in 1971, Zloof presented QBE [], a query by example system. With QBE, the designer enters data or an expression in a column of a table, and the system uses the "example" to formulate a query. For instance, in Figure 1, the designer has entered "39" in column 2 and ">56" in column 4. The system generates the query:

Select * from Table Where age>39 and x>56

Some form of QBE is now included in most Database Management Systems (DBMS). This idea of example-based database specification has been extended in numerous research efforts, including efforts focused on Query by Templates[] and Triggers by Example[]. It has not, however, been extended to allow database manipulation by example, as is the purpose of WebDev.

Spreadsheet Technology

Spreadsheets are the most popular form of programming by example. A designer can enter a formula for a particular cell, then copy that cell to other cells in the same column (row). The system then generalizes the example (the copied cell) and generates formulas that compute data in the pasted row (column).

Research efforts such as [] have extended the spreadsheet paradigm to non-tabular interface components, and have provided more complex generalization techniques, than that described above, to handle complex non-tabular components.

Commercial Web Development

The efficient development of dynamic web pages is of critical importance to all types of companies, and new tools are emerging daily to help the process.

A major concern has been the separation of designer and programmer concerns. For instance, in the Java world, servlets alone were first used to create dynamic pages. With straight servlets, both the presentation and content of a page are specified in Java. Presentation must thus be specified by a programmer, or at least a designer that is code-savvy enough to enter HTML code within Java code.

Java Server Pages (JSP) then emerged. JSP documents are HTML based but allow Java code to be interlaced with the HTML. JSP eases the designers task somewhat, but a designer is still faced with Java code, and the designer and programmer must still work on the same document.

Template systems such as WebMacro [] provided the next generation tool. A template document is stored separately from the Java code that adds the dynamic content, so designer and programmer can work on their parts separately. WebMacro provides a simple scripting language that allows the designer to mark the places in the template where dynamic content should be put.

WebMacro library code then processes the script so that the Java programmer need only associate code with the names of the placeholders specified by the designer.

Though WebMacro provides a template, it doesn't quite allow the designer to format sample data. The scripting language introduces variables (any identifier preceded with a '$') and for loops, which are difficult for some designers. The designer can supply sample values for the script variables, however, so a sample page can be viewed.

Enhydra's XMLC provides what we felt was the best solution for WebDemo's underlying mechanism, mainly because it requires no scripting language other than normal HTML. With XMLC, the designer creates normal HTML, including regular tables with sample data in the cells. To allow the page to be dynamic, the designer need only provide id tags for important HTML elements. The XMLC compiler then parses the HTML and provides manipulation methods for the identified elements. The programmer can then easily modify elements or add new elements, such as the rows of a table.

With XMLC, the designer need not learn any special scripting other than HTML. It is also conducive to WYSIWYG development since users are comfortable with using property sheets to name items, but not so comfortable writing for loops and switching between a visual and non-visual environment.

UltraDev4-Cold Fusion

Because these emerging template tools are so new, the visual tools built on top of them are just now emerging. Cold Fusion, which has now been integrated into Macromedia's UltraDev 4, allows much of a dynamic database-intensive web site to be developed without coding. With Cold Fusion, a developer uses dialogs to map graphical tables and other visual components to pre-defined database tables. UltraDev 4 then generates server code in several different forms, including ASP, JSP, and CFML which is a proprietary scripting language containing SQL tags.

Design in UltraDev 4 is based on a model-view paradigm. The designer first maps a data source to a construct called a RecordSet (the model) This RecordSet can then be mapped to one or more visual components.[note that in our system, model-view is eliminated in design…user works with the visual components]

UltraDev 4 provides a live data view in which data from the database is loaded into a visual component at development time. The normal view, however, displays variable names in cells of visual components, e.g., Employee.name.

By default, when a record set is mapped to a visual component, only the first record in the record set will be dumped to the component at run-time. The user must specify a "repeated region" to have the generated server code dump all records in the record set.

IMPLEMENTATION

The system was built using the Java II Enterprise Edition. The Java Swing HTML editing framework was used for the base HTML editor. COM is used to access Excel for spreadsheet evaluation both at development time and from the server at run-time. The system automatically registers a developed servlet with BEA’s WebLogic Server so that it can be executed immediately during development.

ISSUES

How can QBE applied to interactive operations, e.g., operations such as the end-user entering data in a form, then clicking on a submit button to see a view of a database table dependent on form data.

In the QBE described in Section x, the designer specified, by example, a query that determined the data displayed in a table when a page is first rendered. But often the data in a table changes when the user inputs and submits data. How should the development interface be modified to allow the specification of such an interactive behavior.

First, the QBE formulas entered by the user must be able to reference not only static data or data from another column, but also data from form components in the same or other pages. Consider the example page in Figure X. In the QBE row of the table, column 3’s QBE formula is “>text1”. When this page is rendered, the table will display only those rows of the mapped database table which have a field3 value greater than the default value of text1. When the user enters data in text1 and clicks submit, the generated servlet will once again select rows, this time dependent on the new value of text1.

But what if the designer wanted different criteria depending on whether the page is first being rendered, or if it being rendered as the result of a submit. The default value can help—the designer could give it a very small or very large value to have the system show all or none of the records, for instance. But the default value of text1 is also going to appear in the page, so this is not a very good solution.

Clearly, the designer needs to be able to specify a query for the original rendering, and different queries for the response to interactive submissions. The PBE technique of recording a response can be applied here. The designer selects the desired submit button and selects the Record Response choice in the toolbar. He then modifies the desired QBE cell. Because he is Record Response mode, the system will interpret the action to mean: when the button is clicked, repopulate the table with records using the query defined by the modified QBE cell. When the designer leaves Record Response mode, the former value of the QBE cell is restored.

[A different page?]

User Tests/Lessons Learned

Status and Future Work

Summary

References

-----------------------

XMLC

WebDemo Code Generator

Server

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download