This license applies to the Digital Document Discourse Environment Version 4.1.1 and later (henceforth referred to as "this software").

Portions of this software were developed by the Knowledge Media Institute at The Open University, UK (OU/KMI) and at The University of Colorado at Boulder (CU).

Access and use of this software shall impose the following obligations and understandings on the user. The user is granted the right, without any fee or cost, to use, copy, modify, alter, enhance and distribute this software, and any derivative works thereof, and its supporting documentation for any purpose whatsoever, provided that this entire notice appears in all copies of the software, derivative works and supporting documentation. Further, OU/KMI and CU requests that the user credit OU/KMI and CU in any publications that result from the use of this software or in any product that includes this software. The names CU, OU and/or KMI, however, may not be used in any advertising or publicity to endorse or promote any products or commercial entity unless specific written permission is obtained from OU/KMI and CU. The user also understands that OU/KMI and CU are not obligated to provide the user with any support, consulting, training or assistance of any kind with regard to the use, operation and performance of this software nor to provide the user with any updates, revisions, new versions or "bug fixes."

THIS SOFTWARE IS PROVIDED BY OU/KMI AND CU "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL OU/KMI BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE ACCESS, USE OR PERFORMANCE OF THIS SOFTWARE.

Contents

1. Overview

2. Installing and launching the toolkit

3. Basic requirements for source HTML documents

4. Using the toolkit to process a demonstration file

5. The Setup tab

6. The Document tab

6.1 What the toolkit generates

6.2 Bibliographic references and footnotes

7. Template files for tailoring the look and feel of publications

7.1 How D3E searches for templates

7.2 Content of a template file

7.3 Calling one template from another

8. Customising the toolkit's forms: Toolkit Parameter Files

8.1 The <input...> tag

8.2 The <setting...> tag

Appendix 1: The Overlapping Windows interface

Appendix 2: List of template files

Appendix 3: Template files: other toolkit variables

Appendix 4: Known bugs and other processing tips

Appendix 5: Contacting D3E Project

Acknowledgements

D³E Publisher's Toolkit: User Guide

1. Overview

The D3E Publisher's Toolkit converts a hierarchical HTML document (as typical of technical and scholarly publications) into a frames-based environment that supports quick navigation around the document, and structured annotation and discussion. The document is tightly linked to a discussion space with threads dedicated to user-defined topics, and each document section.

When a document is processed by the toolkit, it is transformed as follows:

• The document is split into smaller files: navigational links between sections are provided, existing internal links are preserved, and an active table of contents is generated from the section headings.
• Bibliographic references and footnotes are turned into bi-directional hyperlinks.
• All of the generated files are placed into a frames-based document interface. The demonstration we have provided provides frames for contents, the article, comments/discussion, header information, footnotes/references window, and links to the publisher's homesite. (See Appendix 1 for an annotated screenshot).
• A complete set of files for a web-discussion environment is produced, with discussion threads for each section in the document, and any other topics that have been specified in the toolkit. The toolkit can currently generate files for either of two discussion environments: D3E-Phorum and D3E-HyperNews.

Some important features of the toolkit are:

• ëTemplate' files are used to provide maximum flexibility for the ëlook and feel' of the final publication.
• The toolkit's forms can be redefined to request information specific to the purpose to which you are putting D3E.

At present the toolkit runs as a stand-alone application on any version of Windows. Ports of earlier versions to other platforms have not been troublesome, but we haven't got round to this yet. (Note: the files it generates are platform and browser independent; it's just this toolkit that only runs on Windows at present).

A version using a web-upload form and ëlite' server-side processing also exists, called Ubiquitous D3E.

It is assumed that you will install a D3E discussion system on a webserver in order to make use of the document-specific discussion files generated by the toolkit. See the D3E site for details.

2. Installing and launching the toolkit

• The top level folder, called D3E here, is the one which you specify during the installation process.
• The articles folder contains an in folder containing a folder for each HTML file to be processed by the toolkit (we have provided a Demo Document), and an out folder, where the files generated by the toolkit are placed. The in and out folder actually contain subfolders, one for each "Setup" (e.g. we have provided a default called Standard).
• A "Setup" is a collection of files which will all be processed by the toolkit using similar parameters, resulting in output documents with the same "look and feel".
• The D3EToolKit folder contains the setup-specific data: a folder for setup-specific templates (e.g. Standard_templates), and (when the toolkit has been used) a folder for the setup parameters will be created.
• The java folder contains the Java interpreter.
• The prog folder contains the toolkit program (comprising a number of .class files). [The current version of the toolkit is written in Java 1.0. Upgrades to Java 2 are in progress.]
• The templates folder contains all the ëstandard' templates which will be used for all setups, unless there are setup-specific templates in the D3EToolKit folder.

3. Basic requirements for source HTML documents

The basic requirements for a document to be processed by D3E are as follows:

• The document should have section names formatted as HTML headings, i.e. using <H1></H1> etc. tags (to whatever depth of heading required). (This requirement will be met if the file is produced using Save As Web PageÖ command in Microsoft Word 2000, assuming that Word's heading styles have been used. It can cope with most Microsoft HTML, such as this document).
• Preferably, these will be numbered headings so that they can be easily referred to within the toolkit (see below).
• If you have numeric or author/date citations in the text that you wish the toolkit to link to entries in a references section, then a section named References is required, with the bibliography.
• If you have numeric footnotes that you wish to be automatically linked, they need to be grouped in a named section in the document, whose name you enter in the toolkit.

4. Using the toolkit to process a demonstration file

Instant Test:

The screenshot below shows what you should be able to generate from the demonstration HTML document (DemoDoc) that we have provided (installed by the toolkit). If the toolkit has installed correctly, you should be able to generate this immediately by clicking the Go button in the toolkit's Article tab, and then opening: D3E\articles\out\Standard\Demo Document\d3e-demodoc-t.html

You need to be online to see and test the demo threaded discussion space on the right-hand side (this is on an OU webserver).

The screen below, when viewing part of the above document, illustrates the main features of the D3E document discussion interface.

D3E's document discussion interface. On the left is the Document Window, on the right the Discussion Window showing the top level outline view of discussion about the document. Key: 1. active contents list extracted from the section headings; 2. links to print versions, e.g. HTML and PDF; 3. numeric or author/date citation automatically linked to corresponding reference in footnote window; 4. a reverse hyperlink is inserted for each citation of a reference; 5. comment icon embedded in each section heading, linked to the discussion thread for section-specific comments; 6. a comment that has been posted on this section; 7. indicator of where the currently displayed comment is in the thread.

The above screen shows the Tiled Window interface that D3E generates for larger screens (e.g. ≥800x600). The document and discussion are displayed in the same window. For standard sized screens, you can use the Overlapping Window interface, which places the article and discussion in separate windows (see screenshots in Appendix 2).

See the following sections for the step-by-step walkthrough.

5. The Setup tab

When you launch the toolkit, you are presented with two tabs labelled Document and Setup. The Document tab presents a form for selecting a document, entering its details (metadata), and defining the parameters which will affect its processing. The Setup tab allows you to switch between groups of documents which may have different processing requirements, or look and feel.

We will step through the use of these two tabs using a demonstration that we have prepared.

The drop-down box, Select Setup is used to select from among the setups that you can work on. You can also use the New button alongside, to begin working on a new setup.

We are using a Setup called Standard, supplied with the installer. The table below describes how this setup has been defined, and how it uses the information entered in the form. (You may wish also to refer to the information on Template files for further information).

6. The Document tab

When you view the Document tab, it shows information for the Demo Document.

The box labelled Select Document is a menu from which you can choose a document to be processed.

Note: To create a new document, you have two options:

1. click the New button and give it a name (e.g. UnitPlan-draft1) and place the new HTML file to be processed in the folder that the toolkit creates, e.g. D3E\articles\in\Standard\UnitPlan-draft1

or

2. go to the File menu and Save AsÖ in order to reuse metadata for an existing document (e.g. when saving a new version).

The Document tab will automatically scan the folder D3E\articles\in\Standard\, and offer you the documents available. When the toolkit is first installed, the menu will offer the two documents D3E Test Doc and Demo Document.

Fields for article details (title, authors, etc.) just have their contents placed in the output files wherever a template specifies.

Parameter fields affect toolkit processing as described below:

• The field Header Level is used to specify at what points the article should be broken into subfiles. If it is set at 2 (the default) then every section heading of level <H2> or higher in the input file results in a new subfile (and a new entry in the contents frame).
• The field Joined Sections is used to specify which sections of the article should be left in the same subfile. Many articles have some short sections whose separation into a separate subfile can be annoying to the end-reader. Entries in here should be a comma-separated list of hyphenated begin-end pairs of section numbers, for example if the field contains ì3-3.2, 4.3-4.4î then sections 3 to 3.2 inclusive will be in the same subfile, as will sections 4.3 and 4.4. You will probably find it convenient to leave this field blank on a first pass, and to modify it after inspecting the output produced.
• The buttons labeled Draft and Final are used so that you can specify a different look and feel to differentiate documents with different status.
• The checkbox Debate on first section allows you to specify whether or not the first section (as defined using HTML <Hn></Hn> tags) is included as one of the topics in the discussion space. For instance, if the first section is simply the document's title, then you would probably leave this unchecked unless you wanted a dedicated thread named after the title.
• The checkboxes Insert break markers and Split files, both checked by default, make it possible to pause toolkit processing between its two separate phases. The first phase inserts markers where the file will be split, and the second does the splitting. If, exceptionally, you need to make changes to the exact position of the splits then you can do so between the phases. (For example this can be useful if you have a very long section with many images that you wish to subdivide into smaller, faster loading filesóyou could then insert extra split-markers).
• The Go button initiates processing. A window appears showing progress (and any problems); finally showing ìTask Completeî, as shown below. It notifies you as it splits files, and warns of any citations that cannot be matched to entries in the Reference list. (Note: if this window has already been opened in a previous run but is now covered, it will remain hidden until you manually bring it to the front)

• The Quit button exists the toolkit application. This is the recommended way to quit ëcleanly' (in contrast to force quitting from the Task Manager or shutting down)

6.1 What the toolkit generates

All the files produced by processing an article, plus any others that were in the input folder (such as images) are copied to the output folder corresponding to the particular Setup you're using, e.g. D3E\articles\out\Standard

There are two kinds of files: for the document, and for the discussion, which should be uploaded to different places on your webserver:

• Document: These files should be copied to an appropriate place on your webserver. The HTML files will go to one place (the website you have specified in the Setup tab). You should also copy over the Schemes folder to this area.
• The files for the discussion will appear in a sub-folder called either D3E-Phorum or D3E-HyperNews. They should be uploaded to the appropriate location on the discussion server. The procedure to do this will vary depending on which system you are using. See the documentation for each.

There is also a Reports folder which includes a report summarising where the toolkit found a possible mismatch between a citation and a reference.

The names of some of the HTML files generated are based on the original input filename; in the list below this is assumed to have been d3e-demodoc.html, the file used for the Demo Document. The HTML files produced are:

• d3e-demodoc.html and d3e-demodoc-t.html: these are HTML ëframeset' files, enclosing other HTML files. d3e-demodoc.html is the frameset for standard-sized screens in which document and discussion are in separate windows (see Appendix 2), and d3e-demodoc-t.html is the tiled-window version for larger screens that can display document and discussion in the same window.

The default frameset (you can design your own) is designed as follows:

• d3e-demodoc-01.html, d3e-demodoc-02.html, etc: these are the subfiles into which the original input document has been split. Each file has the content of one section (or more as specified in Document tab: ìJoined sectionsî), plus navigation buttons to the previous and next sections, and the first page.
• d3e-demodoc-paper.html: this is a composite file containing the whole text of the original article, enhanced with citation-reference links (useful for readers who want to print a single HTML document)
• contents.html: this subfile appears in the ëcontents' frame, and has a link to each section of the article.
• bubble.html (links to printer and discussion area), and banner.html (article header information) correspond to further ìtitleî frames in the frameset.
• references.html (or numRefs.html for numbered references) corresponds to the bottom frame used to display bibliographic references at the bottom of the frameset

6.2 Bibliographic references and footnotes

The toolkit can process two different formats of reference: author-year or numeric references. You specify which in the Setup tab (see above). In either case, the process is driven by the presence in the input document of a section called References (or other title as specified in the Setup tab). Each paragraph of this section it taken as a target reference. Within the main document, either the mention of a year, or else the specified delimiter, is taken as a probable citation and triggers a search for a corresponding reference in the reference section. If one is found, then a link to it is created from the citation, and also a back-link from the reference back to each citation.

Footnotes, if any, are treated in a similar way: you specify in the Setup tab, the title (if any) of a Footnotes section, and footnote delimiters, e.g. { and }.

For author-year format references, numerous citation formats are resolved:

• Multiple years with the same author(s): Jones (1968, 1978)
• Letters to distinguish same-year publications: Jones (1968a)
• Abbreviated author lists: Jones et al (1972)
• Alternative bracketing: ìBalderdashî (Jones 1968) or ìas Jones (1968) claimedî.

Matching is done using the year and all authors' surnames. A references report is produced listing any occurrences of a year which appears to represent missing references (the surrounding text is also reproduced to provide some context). Some of these may be spurious, but some at least will generally correspond to real errors. It will not report citations which it did not detect at all. The guaranteed format to use is of course numeric citations (once you have defined the delimiters), since these can be directly matched to reference entries.

A sample of an entry in the reference section when numeric references are in use is shown below. Appended after the reference are [cited] reverse-hyperlinks to each citation in the text:

[4] Jones, A.B. (2001) Proving I'm Right. Journal of the Bleeding Obvious, 1 (1), 100-101 [cited] [cited]

While the document is still in a single file before processing, it is worth quickly checking that citations are ëwell formed' in the document (authors have all sorts of idiosyncracies!).

The typographic formatting of the reference is not something that the toolkit manages for you. D3E's function is to make the links from each citation to the reference, and back (via the [cited] links that follow the reference). However, the formatting of the [cited] links can be changed since this is defined by a template (e.g. inserting an icon for each citation). (See listing of template files at end)

7. Template files for tailoring the look and feel of publications

The HTML and other files generated by the D3E Toolkit are determined by templates, meaning that you can make wide-ranging changes to the kind of output it produces. At each point where the toolkit has to generate some output, the HTML generated is taken from a template file. Advanced toolkit users can thus change the content of these files and thereby change the toolkit products.

For example, the top-level frameset file which the toolkit produces is produced using the contents of the template file frameset.html. By changing this file, you can redesign the look and feel of the user interface (from our demonstration publication).

7.1 How D3E searches for templates

The toolkit searches for template files first for a publication-specific template folder (in the D3EToolkit folder). If none is found, it looks in the general D3E\templates folder. Thus, a publication-specific template will override a standard template.

Example: templates specific to the Standard Setup would be placed in the D3E\D3EToolKit\Standard_templates folder. The templates used by Demo Publication are listed in Appendix 3.

7.2 Content of a template file

Template files consist mainly of text and HTML tags. Template files use special XML-like variables (in the sense that they are delimited by the characters < and >), but the toolkit distinguishes them by the first character (after the opening <) being a percent sign.

Example: the template file frameset.html specifies that the value of the HTML Title field of the generated frameset with the line:

<TITLE><%shortTitle></TITLE>

<%shortTitle> is substituted by the text entered in the Article tab field labelled Short Title.

Appendix 4 lists the variables generated internally by the toolkit, as opposed to being defined by template files or fields in the tabs.

7.3 Calling one template from another

For further power, templates can refer to other templates using <%include Ö> and <%foreach Ö>. This enables you to define new templates and integrate them using existing ones.

7.3.1. The <%include Ö> construct

The construct <%include Ö> results in a further template file being read and its contents inserted at this point in the file. For example <%include item.html> would result in a search for the template file item.html using the normal search route described above. This technique is used for example, to determine which icon to display depending on whether the Setup tab's Draft checkbox is ticked or not. The template banner.html has the lines:

<%include file="bannerDraft.html" ifchecked="isPreprint">
<%include file="bannerFinal.html" ifunchecked="isPreprint">

If the checkbox was ticked, then the template file bannerDraft.html is inserted, which inserts a ìdraftî icon in the final banner.html file:

<IMG ALIGN=Left SRC="<%scheme-path>/schemes/<%scheme>/draft.gif">

If the checkbox was not ticked, then the template file bannerFinal.html is inserted, which inserts a ìfinalî icon.

You can also see from the above that a template may include the value of fields in the tabs:

<%scheme-path>/schemes/<%scheme>

...picks up the values of two fields specified in the Setup tab, to generate the directory path for the icons.

7.3.2. The <%foreachÖ> construct

The <%foreachÖ> construct is processed as follows. If the construct reads <%foreach variable=varName separator=sep setVariable=setVar include=inc>, then the contents of the variable called %<varName> are parsed using the given separator, and each resulting token is used in turn as the value to which setVar is set prior to including the template file inc.

In addition, trim can be specified, in which case leading and trailing spaces are removed before setting the variable.

The template file mail.list uses all of these constructs to build a file of email addresses from the information entered in the Article tab for authors', editor's and reviewers' emails:

<%foreach variable="authorEmail" separator="," setVariable="emailItem" trim="yes" include="emailListEntry.list">

<%foreach variable="editorEmail" separator="," setVariable="emailItem" trim="yes" include="emailListEntry.list">

<%foreach variable="reviewerEmail" separator="," setVariable="emailItem" trim="yes" include="emailListEntry.list">

8. Customising the toolkit's forms: Toolkit Parameter Files

If you are an advanced toolkit user, you may well need new template files, and new Setup and Document tabs for the user to provide the information required by the templates. We have described templates already. We now describe how to change the toolkit's forms to gather the information you need for use in your customised templates.

Note first that the names of the two input tabs: Setup and Document can be changed. (Users of previous versions of the toolkit will know them as Publication and Article, and these names are in fact still used internally by the toolkit.) The names of the tabs as displayed to users, however, are specified in the file labels.tpf (in the templates directory), along with other key toolkit parameters.

Most of the contents of the forms are not hard-coded into the toolkit; instead they are specified in a Toolkit Parameter File. The toolkit uses three of these, the first is labels.tpf as mentioned above, and the other two (using the internal toolkit convention) are article_panel.tpf and publication_panel.tpf. The files are searched for in the same directories as the template files, so it is possible for forms to look different with different setups.

These files are in an HTML-like format, where each tag describes one line of the appropriate form. The possible elements are input and setting.

8.1 The <input...> tag

An input tag must have a type attribute, which must be one of separator, line or checkbox.

<input type=separator> has no further attributes, and results in a blank line appearing on the form:

<input type=separator>

<input type=line> and <input type=checkbox> have an attribute label. The value of this attribute will be the text label for a field or checkbox on the form (enclose the label with double quotes if it contains any spaces):

<input type=line label="Reviewer Email Address/es" variable=reviewerEmail>

<input type=checkbox label=Preprint? variable=isPreprint>

As shown, <input type=line> and <input type=checkbox> also have an attribute variable. For <input type=line>, this is used to specify the variable which will be set to contain the text which the user enters. The variable name should be specified here without surrounding it by <% and >. This value can then be used in a template file.

For <input type=checkbox>, the most likely context for using the variable attribute is in the <%include filename iftrue=varName> construct (or of course iffalse), where it can affect which template file is included, e.g.

<%include file="bannerTitlePublished.html" iffalse="isPreprint">

<input type=line> may also have an attribute default. This is used to specify the default value to appear in the form's field before the user enters any text:

<input type=line label="Header level" variable=headerLevel default="2">

8.2 The <setting...> tag

This is used to set a variable directly, without reference to a field on the form. Attributes are variable and value, so that the tag:

<setting variable=publication value="Journal of Taste">

will set the variable publication to the given value, for use in a template file.

Appendix 1: The Overlapping Windows interface

Comments and discussion on a section are opened in a fresh window from the document. (designed for users with standard sized screens, e.g. ≤ 800x600)

Appendix 2: List of template files

Below are listed all the template files used in the demonstration publication, indicating where its content appears in the final output.

Appendix 3: Template files: other toolkit variables

Certain toolkit variables are not set from information entered via the toolkit's forms or from parameter files, but directly by toolkit code as it is processing the files. These variables, with their values, are listed below.

Appendix 4: Known bugs and other processing tips

Unfortunately there are still a few bugs that we aim to iron out as and when we can.

Appendix 5: Contacting D3E Project

Design requests, questions or comments can be sent to Simon Buckingham Shum: sbs@acm.org

Technical problems with D3E-Phorum can be sent to Gary Li: G.Li@open.ac.uk

Acknowledgements

The D3E Publisher's Toolkit is implemented by Malcolm Story, Learning & Teaching Solutions, Open University, with contributions from Mike Wright. D3E as a whole is the product of creative collaboration with Tamara Sumner (University of Colorado, Boulder, USA) and Mike Wright (UCAR Unidata, Boulder, USA). Full design team over D3E's history on the website. We gratefully acknowledge the support of The Open University, University of Colorado, Boulder, London Mathematical Society.