]> Augmentation of base schemas for the ATMO Project C. M. Sperberg-McQueen Annotated Turki Manuscripts from the Jarring Collection Online Arienne M. Dwyer C. M. Sperberg-McQueen Henry Luce Foundation Annotated Turki Manuscripts from the Jarring Collection Online Lawrence, Kansas 2018

http://uyghur.linguistics.indiana.edu/2015/ns/0.1

Created in electronic form; no source to describe.

 Some trivial copy edits. Start list of differences. Quick and dirty description of process of augmenting our base schemas. Augmentation of base schemas for the ATMO Project C. M. Sperberg-McQueen, Black Mesa Technologies LLC 21 May 2018, most recently revised 22 May 2018 ATMO Technical report 2018-03
Nearby documents Home

This document provides a terse description of the process of generating schemas for use in the ATMO project.

The primary imaginary reader is the individual responsible for future maintenance of the schemas, who is assumed to have a working knowledge of TEI customization, XSLT, Make, and the Bash shell. Readers without such working knowledge are welcome to examine the document, but they are likely to find it heavy sledding.

In its current state, this document is incomplete even by its own not very demanding standards of completeness. It may be regarded as a promissory note, a commitment to documenting for future reference the steps involved in modifying the base schemas and re-generating the augmented schemas used in the project.

Background

As described more fully in ATMO Technical Report 2018-02, the ATMO project defines and uses several different schemas, of several different kinds.

First, there are the base schemas which define the document as a set of pages, or as a structure of paragraphs and verse, or as a sequence of sentences (in paragraphs). These base schemas, called P-base, T-base, and S-base, respectively, are defined in ATMO TR 2018-02.

Second, there are the augmented schemas which augment each view with elements to record the traces of the other views. The traces are written as Trojan-Horse elements, essentially as defined by : empty elements marking the location of what in the other view are start- or end-tags.

The three augmented schemas (PTS, STP, and TPS) are constructed from the base schemas by an automated process. When things go right, no adjustments by hand should be needed. Things do not always go right, so this document lists the hand adjustments known or thought to be needed at the time of writing.

Generating the base schemas

The base schemas are defined by a TEI customization document (an ODD file, as the TEI calls them; ODD = One Document Does it all). In principle any method of invoking the TEI's ODD processor can be used; in practice, this is not always possible. Three schemaSpec elements are included in the document; one for each base schema. And some ODD processing interfaces are not prepared for input with more than one schemaSpec element.

Running the ODD processor locally does work; it requires that some infrastructure be installed, but given that infrastructure it works reasonably well.

Necessary infrastructure

The following need to be available:

The TEI's stylesheets (https://github.com/TEIC/Stylesheets).

Some version of Saxon; Saxon HE (home edition) should suffice. If Oxygen is installed, jar files for Saxon will be available within the Oxygen directory.

Some version of Java; the ODD processor organizes its work using Ant processes.

Optionally, the TEI Guidelines (https://github.com/TEIC/TEI). The only bit actually used appears to be the file p5subset.xml; if Oxygen is installed, this file is present in Oxygen's library directories.

Running the ODD processor

To generate a given base schema in RelaxNG's XML syntax, the basic form of command is $(TEISS)/teitorelaxng --oxygenlib=$(OXLIB) --localsource=$(P5SUB) --schema=$(SCHEMASPEC-ID) $(INPUT) $(OUTPUT), where

$(TEISS) is the path to the bin directory of the local installation of the TEI Stylesheet package. On the author's work machine, for example, this is /Users /cmsmcq /2017 /TEIC-Stylesheets /Stylesheets /bin. (N.B. whitespace has been added here to allow line-breaking. Take the blanks back out.)

$(OXLIB) is the path to the lib directory of the local installation of Oxygen. On the author's work machine, for example, this is /Applications/Oxygen-17.1/oxygen/lib.

$(P5SUB) is the path to the local copy of the file p5subset.xml. (It is optional; if the --localsource option is not specified, the process will read the file from the TEI Consortium's web server. The processing will take a little longer owing to network delays, and results may not be consistent from day to day as the TEI server is updated.) On the author's work machine, this is /Applications /Oxygen-17.1 /oxygen /frameworks /tei /xml /tei /odd /p5subset.xml.)

$(SCHEMASPEC-ID) is the ID value of the schemaSpec element to be processed. In TR 2018-02, this will be one of P-base, T-base, or S-base.

$(INPUT) is the filename of the ODD document: here, atmo-schemas-PTS.xml.

$(OUTPUT) is the filename of the desired RNC document: here, $schemaSpec-id + .rnc.

A concrete invocation of the process for base T might read: OXAPP=/Applications/Oxygen-17.1/oxygen TEIFW=frameworks/tei/xml/tei/odd /Users/cmsmcq/2017/TEIC-Stylesheets/Stylesheets/bin/teitornc \ --oxygenlib=${OXAPP}/lib \ --localsource=${OXAPP}/${TEIFW}/p5subset.xml \ --schema=T-base \ atmo-schemas-PTS.xml T-base.rnc

For the compact syntax, the form of the command is very similar, but uses a different shell script: $(TEISS)/teitornc --oxygenlib=$(OXLIB) --localsource=$(P5SUB) --schema=$schemaSpec-id $input-filename $output-filename

Such long commands are tedious to type. To make the process more convenient, a Makefile has been installed in directory /2018/05; if the variables in the directory are correctly set for the current machine, then the process is simpler. To update the base schemas in the /2018/05 directory, navigate to that directory, edit the ODD file as needed, and then type make all or just make. All three base schemas will be generated.

To propagate the newly generated schemas to the /lib directory, it is necessary to copy them. Again, a Makefile has been provided. Navigate to the /lib directory and type make base.

Blind alleys

Two other ways of running the TEI's ODD processor should be mentioned, if only to explain why they are not described above.

The TEI maintains a Web interface for ODD processing at http://www.tei-c.org/Roma/. Using it, one can upload one's ODD document (in this case, the ODD document is TR 2018-02, atmo-schemas-PTS.xml in the same directory as this document) and generate the schemas. The Roma interface currently provides no mechanism for specifying the identifier of the schemaSpec to be processed, and it chokes on the ODD document in TR 2018-02.

The Oxygen XML editor from SyncRO Soft SRL ships with pre-installed schemas and transformation scenarios for TEI documents, including TEI ODD documents. Unfortunately, the predefined transformation scenario currently provides no mechanism for specifying the identifier of the schemaSpec to be processed.

Generating the augmented schemas

The augmented schemas could in principle (and in a perfect world would in practice) be generated from ODD documents. It has proven simpler and quicker in practice to create the augmented schemas by writing an XSLT stylesheet to generate them from the base schemas.

The stylesheet is called augment-base-schema.xsl and it is located in the /lib directory. It takes a target parameter to specify which augmented schema to generate: PTS, STP, or TPS.

The simplest way to run it, is to adjust the variables in the Makefile in the /lib directory and then type make base; make all.

Adjustments

At this writing, the version of STP in /lib differs from the automatically generated form in the following ways. Where possible, the ODD should be modified to make these changes in the base schemas, instead of requiring hand changes.

The ODD file sometimes omits the necessary namespace prefix when generating Schematron rules for ATMO elements. The rule for atmo:s-wrap comes out (after elision of some unnecessary namespace declarations) as Do not use atmo:s-wrap within the TEI Header. ]]>

A search in the RNG file for =": will find all the cases. At the moment, atmo:s-wrap appears to be the only sufferer; atmo:seg et al. do not exhibit the problem.

So far I have not figured out where the problem is or how to work around it, so a hand fix is the only option.
References DeRose, Steven. 2004. Markup overlap: A review and a Horse. Paper given at Extreme Markup Languages 2004, Montréal, sponsored by IDEAlliance. On the Web at http://conferences.idealliance.org / extreme / html / 2004 / DeRose01 / EML2004DeRose01.html.