Transcriber's survival guide

Transcriber's survival guide C. M. Sperberg-McQueen, Black Mesa Technologies LLC 12 December 2015 ATMO Technical report 2015-04

Nearby documents Home

This document provides a short introduction to the TEI tag set developed by the project "Annotated Turki Manuscripts from the Jarring Collection Online" for transcription of Central Asian manuscripts in Chaghatay and Uyghur.

In its current state, this document is known to be incomplete and it is quite likely that it doesn't explain things very well and is badly organized, owing to the haste within which it was written. Questions directed to the author will help him to improve it.

Some prerequisites

Some things a reader of this document will need to know, or learn very quickly. This document will be much easier to understand if you the reader know: the rules of XML syntax the difference between well-formedness and validity how to edit XML in an XML editor such as Emacs or Oxygen how to find and fix well-formedness and validity errors the syntax of namespace declarations and namespace prefixes what TEI stands for and what the TEI Guidelines are how to find information in the TEI Guidelines (or elsewhere) about how to use a given TEI element how hyperlinks work in HTML and in TEI (specifically, what '#' means, what a target without a '#' means, and what a target beginning with '#' means) I'll try to explain some of these topics in the appendix, but you may find better explanations elsewhere.

Basic TEI workflow for transcriptions

In the TEI-based workflow to which we are now moving (or attempting to move), ATMO transcriptions are made in several steps, which involve several different roles.

Initial intake

From the catalog record supplied by the Lund University Library, we create a TEI document containing a TEI header, with a msDesc (manuscript description) as the content of the sourceDesc element (which describes the source of the TEI document).

The result is essentially a free-standing TEI header.

Digital facsimile

After a manuscript has been scanned, we create a TEI digital facsimile from the free-standing TEI header by adding pointers to the page images; the resulting document follows the description in section 11.1 of the TEI Guidelines fairly closely.

Each image is listed twice, once in thumbnail form and once in a full-size image (full-size in the sense that it's not a thumbnail; these images will normally be scaled down significantly from the original TIFFs, for the sake of better response time); the two images of a given page or opening are grouped into a surface element.

Skeleton transcript

When a manuscript is to be transcribed, we convert the digital facsimile into a skeleton transcription, by changing the facsimile element into a sourceDoc and adding elements within each surface to hold the text.

Transcription

The actual transcription step consists in adding character data to the appropriate elements in the skeleton; in the normal case, no structural changes to the document should be needed.

If as a transcriber you run across anything that needs to be changed in the markup, you should feel free to make the change, if you know how to fix the problem and the change does not make the document invalid. If you don't know how to fix the problem while keeping the document valid, add a note or a comment so that we can come back to the problem later and fix it.

At the moment, and for the foreseeable future, the initial intake, digital facsimile, and skeleton transcript steps will be performed by the author of this document. Notes and comments will be added to the documents to try to help the transcribers keep their place and understand the structure of the XML document they are working in.

Basic structure of transcriptions

Like any TEI document

There are exceptions to these and many other generalizations made here about TEI documents, but they do not need to concern us here.

, ATMO project transcriptions consist of TEI element, which in turn will contain metadata in a TEI header (tagged teiHeader) and the actual representation of the document. The TEI allows for several different ways of representing the document; our transcriptions will use the TEI sourceDoc element.

So the basic structure of our transcriptions is: ... metadata here to describe (a) the TEI document and (b) the manuscript ... ... transcription here ... ]]>

The TEI header

The initial TEI header is prepared as part of the initial intake step. In the course of our work we may modify the header, and may provide special interfaces for doing so. For the immediate purposes of transcribing a manuscript, however, the header does not need to be (and normally should not be) modified, with two very important exceptions.

First: if you work on a transcription, your name should be included in a statement of responsibility in the fileDesc / titleStmt (that is: the titleStmt element contained as a child of the fileDesc element) in the TEI header.

The fileDesc element is the first child of the teiHeader; it contains a bibliographic description of the TEI document. Your name needs to be included, since you are helping to create the TEI document. (If the preparation of the skeleton transcript was done correctly, your name should already be listed. But be prepared to correct mistakes.)

While we are looking at the fileDesc element, we can notice that in addition to bibliographic information about the TEI document itself, it also contains a description of its source. In our case, the source is a manuscript in the Lund University Library, so the fileDesc element contains a manuscript description (msDesc) for the manuscript. (As mentioned above, the initial version of the msDesc is derived from the LUB catalogue.)

If we expand our outline to include the fileDesc and some of its contents, it may look something like this: Digital facsimile of manuscript Jarring Prov. 6 Annotated Turki Manuscripts from the Jarring Collection Online (ATMO) Arienne M. Dwyer C. M. Sperberg-McQueen Manuscript held and scanned by Lund University Library Manuscript transcribed by Gülnar Eziz Henry Luce Foundation ... other bibliographic information ... ... manuscript description here ... ... other kind of metadata here ... ... transcription here ... ]]>

As is shown in the example, a transcriber should be identified in a respStmt which contains first a resp element with the label Manuscript transcribed by, and then a persName element containing the transcriber's name and carrying an xml:id attribute with the transcriber's initials. If more than one transcriber works on a manuscript, there can be more than one persName: Manuscript transcribed by Gülnar Eziz Akbar Amat ]]>

Second: every modification of the document should be recorded in a change element in the header's revisionDesc (revision description) element.

The revisionDesc element is the last child of teiHeader, following fileDesc and possibly other elements. It contains a work log showing the history of changes to the document. An (imaginary) example might look like this: ... bibliographic metadata ... Checked validity, made some cosmetic changes. Finished transcription and proofread. Transcribed pages 3r - 7r. Transcribed pages 1r - 2v. Skeleton document ready. Digital facsimile made semi-automatically by C. M. Sperberg-McQueen. ... transcription here ... ]]> In the usual case, a change element carries two attributes: when gives the date of the change, in yyyy-mm-dd form, and who provides a hyperlink to an element naming the person who made the change (here either #MSM or #GE). The content of the element is a prose description of the change made or work performed.

The source document (sourceDoc) element

The actual transcription of the manuscript is contained (in ATMO documents) in a sourceDoc element. The sourceDoc element consists of a series of surface elements (short for writing surface), and each surface element contains (a) links to scanned images of one surface, and (b) a transcription of what is written there.

What is described here is our current understanding of the normal structure. We may encounter special situations that require different structures, and we may change our minds about what the default structure should be.

We will have normally have one surface element for each scan; sometimes the scan is for a single page and sometimes it covers an opening or spread: one right and one left page.

The links to scanned images will normally be TEI graphic elements giving URIs of the scans. The attribute rend="thumbnail" marks the small thumbnail images provided in the overview of a digital facsimile; other images are full-size.

After the links to the scans, a surface element will contain one or more elements representing writing zones: distinct areas of the writing surface which carry writing. For scans representing a single page of the manuscript, the surface element will usually contain an ATMO page element, and for scans representing an opening of the manuscript it will contain two page elements: ... transcription of 3v ... ... transcription of 4r ]]> In some cases the page element may be inappropriate, and a TEI zone element will be used instead. ]]>

Individual pages

Within a page element, there should be one element for each main area of writing on the page. In our customization of TEI, these areas are named main, top, right, left, and bottom and represent the main text block and the four edges of the page outside the main text block.

If you consult the TEI documentation, you will need to know that all of these elements correspond to the TEI zone element. We give them more specialized names for our own convenience; users of our data will be able to use our customized TEI, or they can request a translation into vanilla TEI without extensions or changes.

These elements will occur only if there is writing in the corresponding area of the page; when they do occur, they must occur in the order top, right, main, left, and bottom. TEI zone elements can occur before, after, or between them, if needed for unusual cases.

Within the element for each page zone, the writing in that zone is transcribed within TEI line elements. Each line element represents one line of writing in the manuscript. The top element can also contain folio-num and pagenum elements: catchword, or other specialized elements instead of or in addition to line elements. The general structure will be as shown below. ... ... ... ... ... ... ]]>

Specific special cases and questions

Endorsements, foliation, other library-made marks

Most of the Jarring manuscripts have penciled foliations made by Gunnar Jarring: these are penciled numbers on the a-side of each leaf.

Tag them using atmo:folio-num (usually as part of atmo:top).

When page numbers are given, they are generally not made by the library but by the scribe. Tag them using atmo:pagenum (usually as part of atmo:top).

The penciled labels identifying the manuscript by Prov. number (Prov. 24 on the first or last page, for example) we regard as endorsements. Tag them using tei:zone type="endorsement" (usually as part of atmo:top).

Tables and tabular material

Some manuscripts have material presented in tabular form. In the text-oriented vocabulary, we'll tag this material as tables, but tables aren't available in the transcription vocabulary.

Tag the table itself using tei:zone type="table" (usually as part of atmo:main, between the tei:line elements for the lines before and after the table).

Tag individual table rows as tei:zone type="row", individual cells as tei:zone type="cell".

If a cell spans several columns, say so in the rend attribute (e.g. cols 3 for a cell which occupies the same space that three cells on other lines do).

The tables will probably require hand work in translating into the text vocabulary.

Some basic information

The following sections provide some basic background information on topics you will need to be familiar with. Please contact the author if you have questions not answered here.

TEI

The Text Encoding Initiative (TEI) is a project to develop and maintain a set of guidelines for encoding and interchanging electronic texts created for use in research. It contains pre-defined XML markup for many text types and textual features of interest to linguists, literary scholars, historians, philosophers, art historians, computational linguists, and on and on. Since very few people actually want to use all the XML markup defined by the TEI, most users select a subset of TEI for use in a project. We do that, too, in ATMO.

And because TEI is designed to be very general, it tries to assume very little about document structure or annotation practice. To make the markup fit the practice of a particular project more closely, the TEI allows (actually, encourages) users to adapt the TEI markup to the specific situation of a project. In ATMO, we adapt the TEI by defining a number of project-specific elements and by changing the formal definitions of some existing TEI elements.

Namespaces

To use the project customization of TEI, you will need to understand the basics of XML namespaces.

In our case, the schema used for transcription has two kinds of elements:

Elements defined by the TEI itself. These are in the TEI namespace.

Elements defined by the ATMO project. These are in the ATMO namespace.

Every ATMO transcription will need to declare the two namespaces used. To keep things simple, it's a good idea to declare all namespaces on the outermost or root element (the TEI element). Namespace declarations take the form of attributes whose name begins with xmlns: and ends with a string of characters called a namespace prefix. The value of a namespace declaration is a URI or namespace name. The name of the TEI namespace is http://www.tei-c.org/ns/1.0 and the name for the current version of the ATMO namespace is http://uyghur.linguistics.indiana.edu/2015/ns/0.1.

The namespace declarations will typically take something like the following form: ... ]]> In some cases, people make the TEI namespace the default namespace, so that they don't need to use a prefix. In that case, the namespace declarations will typically look something like this: ... ]]>

Most documents in the ATMO project will use either the prefix tei or the prefix p5 for the TEI namespace, and atmo for the ATMO 0.1 namespace. But neither the TEI nor ATMO impose any requirement that particular namespace prefixes be used, or that any be used at all. Some people (including the author of this document) typically it easier to understand documents which involve more than one namespace if every namespace used has an explicit prefix, so they tend to avoid declaring a default namespace when they can. (XHTML does require that it be the default namespace; if it's not, Web browsers often don't understand it or process it correctly.)