Metadata

Senior Editor
Baylor University Institute for Oral History

In its Principles for Oral History and Best Practices for Oral History, the Oral History Association includes the following:

1.  Interviewers, sponsoring institutions, and institutions charged with the preservation of oral history interviews should understand that appropriate care and storage of original recordings begins immediately after their creation.
2.  Interviewers should document their preparation and methods, including the circumstances of the interviews and provide that information to whatever repository will be preserving and providing access to the interview.
3. Information deemed relevant for the interpretation of the oral history by future users, such as photographs, documents, or other records should be collected, and archivists should make clear to users the availability and connection of these materials to the recorded interview.
4. The recordings of the interviews should be stored, processed, refreshed and accessed according to established archival standards designated for the media format used. Whenever possible, all efforts should be made to preserve electronic files in formats that are cross platform and nonproprietary. Finally, the obsolescence of all media formats should be assumed and planned for.
5.  In order to augment the accessibility of the interview, repositories should make transcriptions, indexes, time tags, detailed descriptions or other written guides to the contents.[1]

All of these governing principles depend upon meticulous, consistent documentation and record-keeping. At the heart of record-keeping is metadata—data that describes, tracks, and manages information resources. This essay will discuss the principles of creating, collecting, and managing metadata for oral history in accordance with the principles outlined above, as well as the systems and tools currently available for accomplishing these tasks.

Metadata and The Curator’s Mission

In her book Curating Oral Histories: From Interview to Archive, Nancy MacKay writes, “Curation refers to the long-term care and management of historical documents, in order to ensure maximum access for the present and the future.”[2] The complexity of the curator’s task in the digital era is suggested by the definition offered by The Digital Curation Centre; it defines digital curation as

maintaining and adding value to a trusted body of digital research data for current and future use; it encompasses the active management throughout the research lifecycle.[3]

What does this entail for oral history? It involves documenting the context of an interview, set of interviews, or interview project, in part to establish its status as a “trusted body” of information.[4] It involves caring for oral history interviews in all their forms from the moment of their creation (or even before) into the indefinite future; it involves caring for originals and derivatives, maintaining their integrity and reliability, making them accessible through changes in technology for disparate uses and users, and, increasingly, adding value to them through application of both sophisticated and informal analytic tools. Most significant, perhaps, is the fact that as a primary historical resource, an oral history interview has an indefinitely long “research lifecycle.” This requires that every curatorial step be taken with the aim of ensuring that the oral history survives changes in technology, organization, management practice, and use.

As suggested above, at the center of all of curatorial activities is a reliance on metadata. Metadata consists of all of the sets of terms used in the curating processes and all of the lexical and semantic systems by which these records are kept in a readily useable form with widely understood words and symbols. In this digital era, metadata is created for computer systems that can share and save these records about oral history for the present and the future.

Understanding metadata challenges oral historians to understand the greatly enhanced opportunities that digital documentation offers to add value to the interviews they record. Broadly understood, metadata makes possible the discovery of themes and meaningful relationships within interviews, among sets of interviews, and with other digitally represented resources. New digital tools are being developed to enhance the accessibility of oral history interviews in various media and forms and to analyze and understand them in ever-greater contexts. Metadata is crucial to all of these new ways to add value to the recorded oral history interview.

Oral histories are complicated.

Curating oral history is greatly complicated by the fact that oral history practice generates a complicated set of objects. To begin with, the basic unit of an oral history is very nearly impossible to define in a way that is satisfactory for all creators, all projects, and all curators. The basic unit can be one or more of the following—and this list is not exhaustive:

• Single recorded interview, recorded in one session, with one or more interviewees and one or more interviewers; original recording may be contained on one or more tapes, or may consist of one or more digital audio files;
• Series of interviews recorded with the same interview participants recorded in separate sessions on a single day or over a period of time;
• Series of interviews recorded or collected with various interview participants related by family ties, community membership, topical interest, common experience, or other factor;
• Series of interviews recorded or collected by a single organizational entity.

Further complicating the effort to identify the basic unit of an oral history is the myriad objects, physical and digital, which may comprise it. Among these may be one or more of the following (again, this list is not exhaustive):

• Original audio and/or video recordings on one or a series of magnetic tapes, on open reels or in cassettes;
• Digitally recorded audio and/or video recordings in one or more audio or video files in one or more file formats;
• Analog audio and/or video recordings transferred to digital files in one or more formats;;
• Typewritten transcript, bound or not;
• Typewritten transcript transferred to a digital file by scanning, with or without optical character recognition and subsequent generation of a searchable and/or editable electronic document;
• Electronic transcript document in one or more word-processing file formats, in one or more versions with various levels of editing, from raw verbatim transcript to a heavily edited version created by one or more individuals working either independently or in collaboration with one or more of the interview participants;
• Electronic transcript document and/or digital video and/or audio file with embedded codes such as elapsed recording time stamps, subject or theme keywords or tags, or other structural, descriptive, or analytical markup;
• Abstract, log, and/or index of content;
• Legal documents, such as deeds of gift and copyright assignments;
• Appended material, such as original and/or digitized images, comments and notes, introductory essays or prefaces, among others.

Defining sets of terms—metadata elements—to document all of these disparate kinds of objects and their relationships to each other poses significant challenges to oral history curators. It requires access to technical expertise and analytical tools and processes from several disciplines, as well as acquaintance with a broad range of standards and best practices for metadata formulation, collection, and use. At the same time, because oral history must be accessible by users and researchers with a broad range of technical expertise and resources, curators are obligated to make access simple in spite of the complexities they must master to do so.

Metadata Standards

It is frequently asserted that metadata conventions, standards, and best practices are best governed by the communities which need and use the metadata. What needs to be described, by what terms, for what audience and use, in what context, are among the many matters which are best determined by the users’ community. Because oral history is a practice which spans many communities, both academic and popular, both professional and amateur, and of a wide range of size and resource endowment, generally-agreed-on metadata standards have not evolved. Curators of oral history, including those who develop systems to collect, manage, and preserve oral history metadata, have relied on a wide variety of guidelines, standards, and forums to inform their practice. In the following pages, many metadata systems and tools will be described, and reference is made to many organizations whose work and publications have contributed to these systems and tools.

Functions of metadata for oral history

Metadata for oral history, as for almost all information resources, needs to serve several sets of often overlapping functions. These can be described as follows:

• Creation, multiversioning, reuse, and recontextualization
  Oral history enters a digital information system in several ways. Original audio or video recordings on analog media such as film and magnetic tape may be digitized, or they may be created using digital recording technology. Typewritten transcripts may be scanned, or transcript documents may be created with computer word-processing applications. Once created, oral history materials may be converted to other digital formats. Original digital audio files in WAV format may be broken up into shorter segments, converted to MP3 format, or burned to CD. Extracts of audio or video recordings, with or without transcripts, may be made accessible on Web pages, or included in documentary films, performances, or art works of various kinds. Interviews represented by audio, video, and print media may be used not only for their intellectual content, their original subject matter, but also as teaching tools. They may be assessed by various linguistic, sociological, or other analytic tools that have little relation to the original historical interests of the interview project. Through all of these transformations and uses, metadata must be maintained to keep track of the items, their provenance, and their relationship to the original, to ensure the integrity and trustworthiness of the sources and protect the intellectual and property rights of those concerned.
• Organization and description.
  An oral history must be described to be discoverable and useful. This description includes the basic facts about it—the identities of the interview participants, the project of which it was a part or the organizational or personal context in which it was created, the date and place of the interview, and the original format of the interview recording, transcript, and other associated parts. Additional metadata, such as subject headings, indexes, and time logs may be created, either by information professionals such as librarians, archivists, and curators, or by user communities via such digital means as online tagging, creating so-called folksonomies, and other forms of user-contributed metadata.
• Validation.
  Closely related to the first item described above, validation metadata serves to establish authoritativeness and trustworthiness. Users of an oral history transcript, for example, need to be able to rely on the transcript to be a faithful and accurate representation of the original interview. Metadata should document the relationship of the transcript to the recording, including information about how the transcript was made, who reviewed it and according to what criteria, and how its integrity has been preserved. Similar concerns with audio and video recording can likewise be addressed, and the information documented, with appropriate metadata elements.
• Searching and retrieval.
  Descriptive metadata serves to aid researchers and community members in finding oral histories relevant to their needs and interests. Oral histories stored in libraries and archives, and more commonly, in online content management systems, must be searchable to be useful. Searching functions are provided by many computerized systems, including Internet-wide ones using sophisticated semantic algorithms. Helping interested users find oral history materials relevant to their interests relies completely on accurate and thorough content description via metadata. In addition, libraries, archives, and others who take care of oral history access and preservation usually need to be able to track usage of their materials for administrative support, data storage and planning, access control, and other purposes, and search and retrieval systems often need to include usage-tracking metadata, as well.
• Utilization and preservation.
  Closely related to several of the functions described above are the utilization and preservation functions of metadata. User tagging, rights tracking, version control, as well as the processes required for preservation and migration processes and integrity checking all generate metadata which needs to be preserved and managed.
• Disposition.
  It is occasionally appropriate or necessary to dispose of, or destroy, oral history interview recordings and/or their associated materials such as transcripts, abstracts, or even most of the metadata associated with them. This may be a consequence of interview participants’ wishes or other legal complications, quality control concerns, or other unusual circumstances. Metadata is crucial to document not only the accession of oral history interview materials, but also their deaccessioning and disposal.[5]

Types of metadata

Metadata systems for oral history materials (as for all other information resources) are designed to perform the functions described above with sets of metadata elements. These elements are usually classified as follows:

Descriptive

• Basic
  These metadata elements describe an item in all of the ways needed to identify it. Descriptive metadata forms the basis of cataloging records and finding aids; it differentiates between versions of an item and provides curatorial information necessary for evaluating it, caring for it, and making it accessible.
• Enhanced
  These metadata elements describe the content of the item in greater detail. They can show relationships between items, as well, with links to related items. Annotations, in-text tagging, and other means of content analysis also generate descriptive metadata. This is an area in which much new work is being done for oral history, to allow both information professionals and communities of users to enhance the understanding of the content of interviews.

Administrative

These metadata elements document the following aspects of the items they describe:

• Provenance
• Rights management
• Acquisition information
• Rights and reproduction tracking
• Documentation of legal access requirements
• Location information
• Selection criteria for digitization
  

Technical
These elements document the technical characteristics and processing history of items; much of this metadata is generated automatically by the computer systems used to process the items, is embedded in the digital files which constitute the item, and can be harvested and imported into external metadata systems for management and tracking:

• Hardware and software documentation
• Technical digitization information
• formats, compression ratios, scaling routines
• Tracking of system response times

Authentication and security dataExamples of this category of metadata include:

• encryption keys
• passwords

PreservationThese metadata elements document all aspects of the processes required to maintain and preserve items, including migration to new formats:

• Documentation of physical condition of tapes, discs,  and documents, and integrity of digital files (e.g., checksums)
• Documentation of actions taken to preserve physical and digital versions of resources, e.g., data refreshing and migration
• Documentation of any changes occurring during digitization or preservation (e.g., tape breakage or other damage)

StructuralThis metadata is of particular usefulness in curating oral history materials because of the complex nature of these materials. These metadata elements can serve to link audio and video in various format to transcripts, indexes, and time logs, and they can link interview segments to each other or within series of interviews or thematically related interviews or interview segments. In general, this category of elements:

• certifies the authenticity and degree of completeness of the content;
• establishes and documents the context of the content;
• identifies and exploits the structural relationships that exist within and between items;
• provides a range of meaningful access points for users

Creating and Managing Metadata

Embedded metadata

For born-digital oral history objects, and for digitized ones, much metadata, particularly technical metadata, is generated by the computer systems which are used to process the items and is actually embedded in the digital files which represent the item. This automatically generated metadata can be harvested or exported to external metadata systems for management.

Embedded metadata for digital audio and video files

Digital audio and video files created according to oral history standards and best practices usually contain much of the technical information necessary to use and preserve them. This metadata is created during the digital recording process and is embedded in the audio and video files. Audio and video editing software often provides functions for inserting more metadata into the file, including some descriptive and administrative metadata.

For audio recording, the waveform audio file format, or WAV file, is considered one of the standard acceptable recording formats for oral history. This audio file format includes the ability to embed metadata following widely recognized file-structure standards. An extension of this file format, the broadcast wave format (also designated with the WAV file name extension, although it is often referred to as a BWF file), is widely used in the radio, television, and cinema industries, and is specifically designed to accommodate additional metadata necessary for identifying, tracking, and describing audio files and synchronize them with other recordings. Audio editing software, such as Wavelab[6] and Sony Sound Forge[7], provide for the insertion of metadata into WAV files, taking advantage of the metadata capabilities of these audio file formats. The challenges for good metadata practice in using these features of proprietary software are twofold. First, although metadata inserted using the software may be of widely recognized format, not all metadata management systems may be capable of extracting the data and using it predictably and meaningfully. Concern for the interoperability of audio editing and metadata management applications, and the conformance of both to nonproprietary standards, should be a governing factor in making decisions about what applications to use. Secondly, audio editing software, particularly proprietary applications, can leave a processing footprint in every audio file they open; this footprint is a small change in the configuration and structure of the audio file which may affect its integrity or accessibility in the long run. The effect can be all but undetectable, and its consequences may not be discernible for several technological generations, but it is nonetheless a concern for those invested with responsibility for the long-term preservation of audio files. This concern, too, should be considered when evaluating software applications for metadata use. There are some applications available whose only function is to insert metadata, and these are generally reported to be minimally intrusive and safe for preservation purposes. In general, these specialized metadata insertion applications are designed only for BWF files.

Technical metadata for video recordings is greatly more complicated than for audio files, and the video industry is still far from having universally recognized and implemented standards for file formats and, consequently, for technical metadata. Many of the same concerns for interoperability and long-term stability of files and metadata pertain equally to video, and the same criteria should be applied to the selection of applications to edit and manage digital video oral history materials.

Standards for metadata for audio have been developed by a number of organizations. Contributors to this effort have included audio engineers and scientists, members of the broadcasting and recording industries, academic library preservationists, and various governmental sponsors. These include:

• Audio Engineering Society (AES)

The Audio Engineering Society is an international organization of audio engineers, creative artists, and scientists that works to advance audio technology and research. An important aspect of the work of the AES is its involvement in the creation and maintenance of international standards in the areas of digital and analog audio engineering, communications technology, acoustics, media preservation and creative practice.[8]

AES standards of particular interest to those working with oral history metadata are:

• AES57-2011: AES standard for audio metadata – Audio object structures for preservation and restoration

Printing Date: 2011-09-21

Publication History: Pub. 2011

Abstract: This standard provides a vocabulary to be used in describing structural and administrative metadata for digital and analog audio formats for the purpose of enabling audio preservation activities on those objects. Some implementations also refer to this metadata as technical metadata. The characteristics of the audio objects captured by this standard may be of use to audio communities beyond the audio preservation community.[9]

• AES60-2011: AES standard for audio metadata – Core audio metadata

Printing Date: 2011-09-22

Publication History: Pub. 2011

Abstract: This specification addresses the creation, management and preservation of material that can be re-used as originally produced, or may provide input material for new production projects. Material is expected to be exchanged between various organisations or between production facilities in a distributed environment. The core set of metadata presented in this specification is a co-publication of EBU Tech3293-2008 EBU Core, itself an extension to and a refinement of the Dublin Core [see full description of Dublin Core in following section of this essay]. EBUCore is a minimum list of attributes characterizing video and / or audio media resources. An XML representation is also provided in case this metadata would be implemented, for example in archive exchange projects using the Open Archive Initiative’s Protocol for Metadata Harvesting (OAI-PMH)[see full description of OAI in following section of this essay.].[10]

• Association for Recorded Sound Collections (ARSC)

The Association for Recorded Sound Collections is “a nonprofit organization dedicated to the preservation and study of sound recordings—in all genres of music and speech, in all formats, and from all periods…Archivists, librarians, and curators representing many of the world’s leading audiovisual repositories participate in ARSC alongside record collectors, record dealers, researchers, historians, discographers, musicians, engineers, producers, reviewers, and broadcasters.”[11]

Of particular interest to oral history curators is the work of the ARSC to provide guidance in documenting and cataloging sound recordings, detailed in the ARSC publication, Rules for Archival Cataloging of Sound Recordings.[12]

• International Association of Sound and Audiovisual Archives (IASA)

The International Association of Sound and Audiovisual Archives functions “as a medium for international co-operation between archives that preserve recorded sound and audiovisual documents.”[13]

IASA offers publications which aid in the selection of appropriate terms and elements for metadata systems for audiovisual materials. These include:

• The IASA Cataloguing Rules
  This text provides detailed guidance for describing sound recordings. Although it is primarily aimed at describing recorded performances of music, theater, and other such works, some of the specifications of descriptive elements are applicable to other kinds of recorded material, such as oral history interviews.[14] Guidelines on the Production and Preservation of Digital Audio Objects
• Guidelines on the Production and Preservation of Digital Audio Objects
  This document includes a significant chapter on metadata, giving “detailed advice on approaches to the management of data and metadata for the purposes of preservation, reformatting, analysis, discovery and use. The chapter ranges widely across the subject area from schemas through to structures to manage and exchange the content and considers the major building blocks of data dictionaries, schemas, ontologies, and encodings.” [15]

Embedded Metadata for transcripts and other documents

Digital versions of transcripts, abstracts, indexes, and other documents created along with interview recordings can also contain embedded metadata which document many aspects of their content and provenance. Word processing software usually provides a function for entering information via a “file properties” or similarly named function. This metadata can be standardized in content and format and used to help preserve important information about the documents without reference to external metadata systems. Ideally, it should be exportable to external metadata systems for access and preservation, as well. However, as many of the most frequently used word processing programs are proprietary, the portability of embedded metadata in the files they create is by no means guaranteed, and its long-term usefulness may be doubtful.

Documents in the preservation standard format of PDF (Portable Document Format, PDF/A, a document preservation standard adopted by the International Standards Organization as ISO 19005-1:2005), may also include embedded metadata. Because PDF/A has been widely adopted as a preservation standard, there is better expectation that metadata embedded in PDF/A documents will be accessible and portable for preservation and access management systems.

Standards, including metadata standards, for preservation of documents are developed by a number of organizations. Chief among these is:

• Association of Information and Image Management

The AIIM is an international association that provides a range of educational, research and certification programs for information professionals who manage and share information assets. The AIIM was instrumental in the development of the PDF/A format, which is now the widely adopted international standard for archive preservation and exchange of documents. The PDF/Archive Committee continues to be responsible “for specifying PDF tags for the archival or long-term preservation of electronic documents,”[16] and produces guidelines and forums to help those who create and manage PDF/A documents. Metadata systems which include management of such oral-history-related documents as transcripts require interoperability with PDF/A standards, and therefore the guidance of AIIM can be of interest to oral history curators.

Metadata Management: Systems, Tools, and Documents

Metadata Documents

Making oral history materials accessible and useful generally requires the creation of machine-readable documents which include all of the metadata needed for computer systems to store and display these materials. The machine-readable documents also serve to make the interviews in all their forms (audio, video, transcript, and others) discoverable by the various computer-based systems used by both amateur and professional researchers. Such systems range from highly structured and tightly controlled library and archives catalogs to Internet search engines, with a great and increasing variety of systems between and beyond these.

Exploiting the power of all of these systems almost universally requires the use of one or another variety of a markup language.

What is markup language?

Markup is a term familiar to many who have edited oral history transcripts. Before computerized word processing, editors traditionally used a blue pencil to mark changes that needed to be made to a transcript—corrections of spelling, punctuation, and such—as well as to specify formats such as italics and boldface. There were (and still are) standard symbols and notations used in the copyediting and typesetting professions for indicating such corrections. Today’s word-processing programs usually include a markup function, by which textual corrections and comments can be displayed in a document through specially formatted, colored text, characters, and shapes. This markup, whether penciled in or input by computer, is not intended to be seen in the final document; it provides instructions on how to correct the document or shows the corrections that were made.

Another kind of markup might also be familiar to oral historians who work with transcripts:  Sometimes explanatory notes or comments are inserted into transcripts, usually with a consistently used marker such a square brackets. Such additions as [laughs] or [recording paused] are common examples. Unlike editor’s or proofreader’s marks, these specially demarcated additions to the transcript are meant to appear in the final version.

In the realm of digital metadata, markup is a means of encoding a document such as an oral history transcript or a description of an interview in one or several formats and media (for example, audio or video recording, transcript, library catalog record, archive finding aid, to name but a few common ones). This encoding makes it possible for a computer program to do something useful with the item, such as store information about it in an orderly and meaningful way in a catalog or on a Web site, relate it to similar items in disparate locations, or display it on a Web page. A markup language, a kind of metalanguage, is a set of codes and a set of rules which govern how the markup codes will be distinguishable from the text or item itself, what markup codes will be permitted, what markup codes are required, and what the markup codes mean.

Below is a very simple example of how basic information about an oral history interview—metadata—might look using a markup system to encode the various parts of the description:

<interviewee>Jonathon Quinius Deaux</interviewee>

<interviewer>Jean-Jacques de Crocquigny</interviewer>

<date>January 6, 2002</date>

<duration>00:45:30</duration>

<audiomedium>analog audio cassette tape</audiomedium>

The symbols “<” and “>” serve to separate the data itself, such as the names of the interview participants and the date of the interview, from the markup codes which identify the data. Each code has both a beginning marker and an ending marker, and they are used in pairs; thus, the codes <interviewee> and </interviewee> mark the beginning and the end of the description text comprised of the interviewee’s name. The whole encoded metadata document is bracketed by a pair of codes—<oralhistoryinterview> and </oralhistoryinterview>—which serve to indentify the whole set of metadata elements between them as parts of a specified type of item, distinguishable in specified ways from others such as a book, a journal, movie, or other kind of item. The codes, both those that define kinds of items to be described as well as the elements used to describe each kind of item, are listed and standardized in a metadata schema. The schema specifies the lexicon, i.e. the codes and their proper spelling and form, and the syntax, i.e. how the codes may be used. Syntax rules might govern, for example, whether particular elements can be repeated, such as <interviewee>, for interviews in which two or more people are interviewed together, or <audiomedium>, for recordings that exist not only in analog form but also as digital audio files. The codes in this example of descriptive metadata use words or terms whose meaning is fairly obvious, but for other metadata elements, such as technical information describing digital audio or video files, the meaning of codes—the schema’s semantics—will require definition in an accompanying document. A schema’s lexicon, syntax, and semantics will also serve to provide the rules by which the metadata can be interrelated or structured within larger contexts and how it can work together with other encoded documents which specify how the metadata should be transmitted or imported into systems for cataloging, displaying, or preserving both the metadata and the physical or digital items it represents.

XML

Metadata of all kinds is, for the most part, created, managed, stored, and displayed by using the XML (Extensible Markup Language) markup language or one based on it. Developed by the World Wide Web Consortium, the XML metalanguage is similar to the markup language HTML, with which Web pages are written for display in Internet browsers. XML is not primarily a language for describing how a document should look on a page or on a computer screen, however. The language can be used to do that, but it can also be used in many other ways, such as to describe the structure of a document, or classify or describe its content by a defined set of terms. The most important quality of XML is that it is not dependent on any particular computer operating system, software program, or hardware configuration. It is almost universally useable and recognizable.

XML is used in conjunction with other metadata documents to ensure that it is used in a consistent way for particular types of documents. Stylesheets are metadata documents which describe how XML-encoded objects should be displayed; schemas define and specify the encoding standards for a particular type of XML document. For example, the TEI (Text Encoding Initiative) schema specifies how to mark up texts in the humanities and social sciences. TEI markup elements can show the structure of texts, identify subjects and themes, index key words and phrases, as well as identify such facts as authorship, publication, revision, and so on. Schemas are formally defined in a document type definition (DTD), “a machine-readable set of rules that specify how a particular metadata document, such as a TEI or EAD (Encoded Archival Description, described in a following section) XML document— formally called an instance— is to be written.”[17]

There are many resources on the Internet that provide guidance in the use of XML. One excellent general introduction is that provided in “A Gentle Introduction to XML,” provided online by the Text Encoding Initiative.[18]

For curators of oral history, there are many software tools available, both free and proprietary, for creating and editing XML documents and for the making specific types of metadata management and display documents such as those described below. Curators working in organizational contexts with computer-based bibliographic or archival catalog systems will often find in-house tools available for metadata harvesting, creating, and managing. Curators working with digital content management systems, either proprietary ones such as CONTENTdm[19] or open-source, multipurpose ones such as Omeka,[20] WordPress,[21] and Drupal,[22] will usually be able to find tools and instructions for importing their own XML-type metadata documents or adapting their metadata for entry into the systems.

Metadata Systems for Oral History

There are a number of metadata systems which have been used in curating oral history. Although no overall metadata system has been developed for oral history, the systems described in the following sections are readily adaptable to the widely ranging needs of oral history collections and are in use in a large number of organizational settings.

Dublin Core

The Dublin Core (the Dublin Core Metadata Element Set) was developed in the mid-1990s by an international group of librarians, archivists, curators, computer scientists, and others. It was conceived as a concise set of fifteen metadata elements for simple and generic description of electronic resources. Dublin Core developers have since followed the lead of the World Wide Web Consortium’s work on a generic model for metadata, the Resource Description Framework (RDF). Dublin Core metadata has broadened to encompass more than electronic resources; now, in principle, any object, whether electronic, physical, or conceptual, can be described by Dublin Core metadata.[23] Further, through work of the Dublin Core Metadata Initiative, the Dublin Core has evolved become a style of metadata, which includes not only the fifteen elements, but also a larger set of terms, the DCMI Metadata Terms,[24] a specification for Dublin Core application profiles,[25] and guidelines for using Dublin Core in XML schemas.[26]

The Dublin Core functions as part of the Open Archives Initiative Protocol for Metadata Harvesting (discussed further below),[27] and has been ratified by such international standards bodies as the Internet Engineering Task Force (IETF),[28] the American National Standards Institute (ANSI)[29] and its associated National Information Standards Organization (NISO),[30] and the International Organization for Standardization (ISO).[31] The Dublin Core Metadata Initiative Conference and Workshop Series continues to bring together information professionals from the private and public sectors to continue the development of the Dublin Core metadata standards. The goal continues to be the development of a simple, interoperable, and flexible set of metadata elements and protocols that can be used by very diverse communities and in the widest variety of applications.

The fifteen Dublin Core elements are:

• Creator
• Contributor
• Publisher
• Title
• Date
• Language
• Format
• Subject
• Description
• Identifier
• Relation
• Source
• Type
• Coverage
• Rights

The key to the metadata system’s flexibility and adaptability is that the elements are optional–one, all, or a selection may be used—and repeatable. For example, to describe an oral history interview, the creator element can be assigned to the name of one or more interviewers as well as one or more interviewees. In an application of the Dublin Core metadata set, the elements can be matched with labels as appropriate. In this example, one creator element would be labeled interviewer and another interviewee, so that end-users viewing the description of the interview in an online catalog or finding aid would see the information in appropriate, meaningful form and context. But because the widely used Dublin Core standard is used in creating the metadata template, the descriptive data in one online catalog can be shared easily and meaningfully with other systems without loss of meaning or the need to re-encode the documents used to transmit the data.

The Dublin Core metadata elements form the basis of many systems for describing, displaying, and preserving historical resources in all media, including oral history. Proprietary digital content management systems such as CONTENTdm and open-source, freely available ones such as Omeka, as well as such metadata systems and schema as Encoded Archival Description (EAD), Metadata Object Description Schema (MODS), and Text Encoding Initiative (TEI) have all been developed on a Dublin Core framework or are designed to be interoperable with it.

PBCore

The Corporation for Public Broadcasting launched its metadata schema for audiovisual media, PBCore, in 2005. PBCore “provides a rich set of descriptors for radio and television programs, in both analog and digital formats.”[32] It is based on Dublin Core, but adds a number of elements specifically appropriate for describing audiovisual media. The schema has been widely adopted beyond the broadcasting community, and now is finding usefulness in a great variety of archives and collections which include audiovisual materials, including oral history collections.

The PBCore includes an XML Schema Definition (XSD) which defines the structure and content of PBCore. The elements of the metadata set cover the following main types of information:

Intellectual Content    encompasses such matters as title, subject, and genre (oral history might be defined as one such genre)

Intellectual Property   encompasses creators, contributors, publishers, and rights information

Instantiations              encompasses all of the technical information needed to identify and keep track of the versions of a media file that are created as it moves from original recording through editing, access, and preservation processes. This set of elements makes PBCore particularly helpful for managing oral history recordings.

The Public Broadcasting Metadata Dictionary Project provides tools and guidelines online for using PBCore,[33] including a cataloging tool based on the database management application FileMaker.[34]

MARC (Machine-Readable Cataloging)

The MARC system of metadata was created to make possible the entry and maintenance of descriptive information in computer-based bibliographic catalogs. The MARC metadata system works in conjunction with other standards, such as the Anglo-American Cataloging Rules Revision 2 (AACR2),[35] to standardize the creation and encoding of cataloging records for all kinds of commonly collected library materials. Using the MARC system, curators of oral history can describe their materials in ways that will be almost universally recognizable and accessible in library catalogs and shared library-type online services worldwide.

The current version of MARC, MARC21, is jointly developed and maintained by the U. S. Library of Congress, the Library and Archives Canada, and the British Library.

A wide variety of resources describing MARC bibliographic cataloging for beginners as well as providing detailed guidance for experiences catalogers is available from the Library of Congress Web site.[36]

The MARC21 metadata system provides a large number of structured codes and symbols, or tags, which denote various details of description and access for library items. MARC tag 245, for example, indicates the metadata element title; tag 100 indicates author or creator (in cataloging terms, this is considered the catalog record’s main entry or principal access point; note that for oral history, the main entry is usually considered to be the interviewee).

Example: From the University of North Texas Libraries, transcript of an interview with Abe C. Cooper, conducted on April 3, 2006, by Benjamin P. Hegi.

The basic reference for MARC cataloging of oral history remains the Oral History Cataloging Manual, complied by Marion Matters for the Society of American Archivists.[37] The manual provides many examples of oral history units (single interviews, series of interviews, oral history projects and collections) described in MARC format, with Anglo-American Cataloging rules explained and applied to the composition of descriptive terms, and all of the MARC tags—markup—illustrated and defined. The version of MARC illustrated in Matters’s manual is USMARC, a predecessor of MARC21, but the changes for subsequent MARC versions would be minimal.

MARCXML

The Network Development and MARC Standards Office of the U. S. Library of Congress is developing an XML schema for “working with MARC data in a XML environment.” This XML framework for MARC “is intended to be flexible and extensible to allow users to work with MARC data in ways specific to their needs. The framework will contain many components such as schemas, stylesheets, and software tools developed and maintained by the Library of Congress.”[38] Complete information on the MARCXML schema is available from the Library of Congress Web site.[39]

Resource Description and Access (RDA)

Librarians and other information professionals who use MARC schema for managing metadata will need to stay abreast of new standards development governing descriptive cataloging. Most important of these at present is Resource Description and Access (RDA). RDA is a “new cataloging standard that will replace the Anglo-American Cataloguing Rules” as the guide for creating machine-readable descriptive metadata for bibliographic records. Developed by the Joint Steering Committee for Development of RDA, comprised of representatives from national libraries and information professionals’ organizations, RDA has been developed in part to enhance guidelines on cataloguing digital resources. It aims for a “stronger emphasis on helping users find, identity, select, and obtain the information they want.” The standard includes features for making clear how a particular item may exist in multiple formats, and this is of special interest to oral historians coping with the complex and changing set of objects which comprise an oral history. The Joint Steering Committee continues work to ensure interoperability of RDA with other standards and schema, such as MARC21, MODS, and Dublin Core.[40] Widespread implementation of RDA has not yet begun, but the U. S. Library of Congress has announced that it will implement RDA cataloging in early spring 2013, and the partnered national libraries have made similar announcements. Widespread adoption of RDA will certainly follow, and vendors of library cataloging and management software will implement it, as well. Detailed documentation of the standard is available online via the RDA Toolkit.[41]

Encoded Archive Description (EAD)

The archives and museum communities have traditionally focused on establishing the context of the materials they collect, documenting provenance of collections (often of disparate types of items) and the relationship of items to each other and to the whole. For example, the collected papers of an individual may include letters, diaries, journals, images, newspaper clippings, video and audio recordings (including, perhaps, oral history interviews), originals or copies of official documents, to name only a few of the possibilities. Archivists have traditionally created finding aids for such collections, rather than item-by-item catalogs. The finding aid model gives rise to a different model of metadata collection, preservation, and transmission than the item-level cataloging model of libraries, exemplified by the MARC record illustrated above.

For oral history metadata management, it would be a mistake to view the choice as clear-cut between the collection-level model of the finding aid and the item-level model of cataloging. For some contexts, in some collections, for some organizations doing oral history projects, an oral history is a collection of interviews, either with a single individual or with a group of individuals closely related by family ties, by interest or affiliation, or other factors. In other instances, an oral history may indeed be a single interview with a single individual, recorded in one sitting on a particular day. But even in the latter case, the oral history may come to be viewed as a collection of related objects. The original interview may have been recorded on an analog medium—perhaps even more than one, as multiple cassette or open-reel tapes were required to record the interview. Transcribing, indexing, transferring the original analog recording to one or more digital formats, creating other versions of the documents and recordings for preservation and access, all create new pieces which together transform into a collection what began as a seemingly simple, single object.

The metadata system most often used for archive collections is Encoded Archives Description (EAD). EAD is a “non-proprietary de facto standard for the encoding of finding aids for use in a networked (online) environment.”[42] The standard is designed to allow “the standardization of collection information in finding aids within and across repositories.”[43] The EAD markup is usually encoded in XML, following the EAD DTD (document type definition). Complete information on EAD, including tutorials, implementation tools (including XML editors, EAD templates, and other EAD implementation software), reference manuals, and links to user forums, can be found on the EAD Web site maintained by the Society of American Archivists.[44]

Among the organizations that have been active in the implementation of EAD is the California Digital Library (CDL). To encourage the development of the Online Archive of California (OAC), the CDL has developed guides and tools to help participating institutions submit EAD finding aids to CDL’s digital special collections. Information about the CDL’s recommended best practices, its guides and tools, can be found on the CDL Web site. [45]

Open Archives Initiative (OAI)

The Open Archives Initiative is a project of the Digital Library Federation and the Coalition for Networked Information, with funding support from the Andrew W. Mellon Foundation and the National Science Foundation. The OAI “develops and promotes interoperability standards that aim to facilitate the efficient dissemination of content.”[46] Its original impetus was to enhance access to e-print archives, but its work has evolved to support standards and practices through which content of almost any kind can be widely shared through standard protocols of metadata sharing and harvesting.[47] Librarians, archivists, curators, and others who maintain collections of digital resources can work with OAI standards and tools to ensure that their metadata systems produce information in a way that can be seamlessly shared among repositories, regardless of the specific software systems in use for storing and displaying that information.

The OAI uses the Dublin Core as its common metadata format, which, as noted above, has evolved as “a de facto standard for simple cross-discipline metadata.” The OAI and the Dublin Core Metadata Initiative have cooperated to create a common XML schema. However, the metadata harvesting protocol of the OAI “supports the notion of multiple metadata sets, allowing communities to expose metadata in formats that are specific to their applications and domains.  The technical framework places no limitations on the nature of such parallel sets, other than that the metadata records be structured as XML data, which have a corresponding XML schema for validation.”[48]

MODS

The Metadata Object Description Schema is an XLM standard for descriptive metadata. Developed by the Network Development and Standards Office of the Library of Congress and made public in 2002, and has been designed mainly for describing bibliographic items in libraries. It encompasses a subset of the fields in MARC21, it uses language-based codes, or tags, rather than numeric codes, for creating descriptive records. It is intended to be simpler than MARC21, accessible to a broader range of librarians, archivists, and others who collect and manage information resources with varying levels of programming skill and technology support. It includes more elements and structural complexity than Dublin Core, but it has been designed to be interoperable with Dublin Core with easily managed mapping, or rules for translating MODS items to Dublin Core-based ones.

The official MODS Web site provides the complete documentation for the metadata schema, as well as links to training materials, guidelines, and examples.[49]

METS

Maintained by the Library of Congress’s Network Development and MARC Standards Office and developed by the Digital Library Federation, the Metadata Encoding and Transmission Standard (METS) “is a data encoding and transmission specification, expressed in XML, that provides the means to convey the metadata necessary for both the management of digital objects within a repository and the exchange of such objects between repositories (or between repositories and their users).”[50] Metadata Encoding and Transmission Standard: Primer and Reference Manual for the METS XML schema is available online.

A METS XML document includes these metadata element sections:

• header
• descriptive metadata section
• administrative metadata section
• file section
• structural map section
• structural link section
• behavior section

Metadata element sets for each of these sections provide the means of describing in detail various salient aspects of a digital item.

An extensive list of software tools for creating and generating METS-encoded XML metadata documents is available from the METS Web site.[51]

An excellent source of information about how the METS schema can be applied to audio recordings is the Indiana University Digital Library Program Sound Directions project. Documentation available online provides a complete example of a METS XML document for a music recording.[52] Many parts of this document may be helpful in understanding the kind of metadata needed and useful for describing digital audio recordings, particularly those that result from the digitization of analog originals.

Preservation Metadata: Implementation Strategies (PREMIS)

The need for metadata systems to support the preservation of digital resources has been addressed by an international working group of information specialists. Originally sponsored by the library cooperative, OCLC, and Research Libraries Group (RLG, now merged with OCLC), the group produced a data dictionary and a set of XML schema for use in documenting and managing the great complexities of digital ojects. The final PREMIS document, Data Dictionary for Preservation Metadata: Final Report of the PREMIS Working Group, provides “ a comprehensive, practical resource for implementing preservation metadata in digital archiving systems.” The XML schema published by the group “support implementation of the Data Dictionary in digital archiving systems.” The U. S. Library of Congress subsequently initiated the PREMIS Maintenance Activity, which continues to maintain the data dictionary. The PREMIS Maintenance Activity “also operates the PREMIS Implementers Group (PIG) discussion list and wiki, conducts tutorials on the Data Dictionary and its use, and commissions focused studies on preservation metadata topics. The Maintenance Activity also established an Editorial Committee responsible for further development of the Data Dictionary and the XML schema and promoting their use.”[53]

The Preservation and Digital Conversion Division of the Columbia University Libraries is one of the organizations that have done considerable work in digital preservation of oral history recordings and associated materials, based in part on METS and PEMIS schema. Their work has included contributions to the development of “a standard, replicable approach to digitizing historic audio collections that other institutions can also use.”[54]

Text Encoding Initiative (TEI)

The Text Encoding Initiative began in the late 1980s as a cooperative effort of the Association for Computers and the Humanities, the Association for Computational Linguistics, and the Association for Literary and Linguistic Computing. Today, TEI is “a consortium which collectively develops and maintains a standard for the representation of texts in digital form.” TEI metadata standards are described in a set of guidelines which “specify encoding methods for machine-readable texts, chiefly in the humanities, social sciences and linguistics.”[55] Using TEI, texts can be encoded to show their structure (for example, for an oral history transcript, codes might indicate the title page, other elements of the front matter, changes in speaker between interview participants, sections of the interview separated by pauses in recording, changing of media—tapes or sides of tapes—or tracks of digital audio files, and others), as well as details of their content (as, for example, subject tags, names of persons, organizations, geographic locations, and others). The TEI metadata schema thus allows digital texts to be searchable, both internally and in relation to other texts. The schema is flexible and highly customizable, and has been widely adopted internationally by all kinds of repositories and other information-managing organizations. The TEI Web site provides complete documentation on the schema, as well as links to implementation tools and examples.[56]

The Regional Oral History Office (ROHO), part of the Bancroft Library of the University of California, Berkeley, has implemented TEI markup for its collection of oral history transcripts. Transcripts are encoded with subject index tags, including implementation of authority lists for personal and corporate names. In addition, “photos, prefaces, contents pages and in some cases, appendices, are included, so that the distant reader has full access to the complete volumes of the oral histories.”[57]

Documenting the American South (DocSouth), a project of the Carolina Digital Libraries and Archives, has implemented TEI for encoding all of its digital texts, including its “Oral Histories of the American South” collection. This has included markup with tags not only for subject searching but also for synchronizing transcripts of interviews with audio recordings. Further work is underway to incorporate georeferencing and geotagging (GIS) metadata to relate oral history interviews to maps and other geographical representations of context.[58]

Tagging, Indexing, and Commenting

User-created metadata is becoming a significant addition to the effort to document the context and meaning of digital materials. Popular Web sites have for some time facilitated the addition of tags and comments to user-generated content. From this user-created metadata emerge so-called folksonomies, which can be included in metadata schemas that describe, analyze, and relate digital materials. An advantage of such user-created metadata is its intimate relevance to user communities, which should be of special interest to oral history curators. The disadvantages are the lack of quality control, where there is no screening or moderation to ensure relevance and adherence to community standards of appropriateness, and relative inefficiency, as multiple terms distinguishable only in minor variations of spelling or grammatical form usually proliferate and make search and retrievable less precise. It seems likely that the advantages will outweigh the disadvantages, as search tools become increasingly sophisticated. Digital content management systems, such as the proprietary CONTENTdm, have implemented tagging and commenting features.

Professionally designed systems for tagging and indexing audio and video recordings of oral history interviews have been developed, using proprietary and open-source software tools. An example of the application of such a system is the Audio-Video Barn, a production of the Illinois State Museum’s Oral History of Illinois Agriculture (OHIA) project, which “contains audio and video recordings of more than 130 oral-history interviews with people involved in agriculture and rural life in Illinois.” The project employed the services of a private company, Randforce Associates,[59] to make the Audio-Video Barn oral history collection “searchable by theme, topic, and keyword.” The company also “assisted with the subdivision of interviews into segments and clips, the development of a hierarchy of control words and other indexing terms, and the application of search terms to the interview recordings.”[60] The audio and video segmenting, indexing, and cross-referencing tools developed by Randforce Associates have been demonstrated and discussed at meetings of the Oral History Association, but the standards models and metadata schema used or created by these tools have not yet been made widely available.

A similar system for segmenting and annotating audio and video recordings has been developed by the Concordia Digital History Lab, a part of the Centre for Oral History and Digital Storytelling at Concordia University in Montreal. Named Vertov, the tool is “a free media annotating plugin for Zotero[61]…Vertov allows you to cut video and audio files into clips, annotate the clips, and integrate your annotations with other research sources and notes stored in Zotero. Vertov is a free, open source Firefox 2.0 extension/Zotero plugin written in Java and JavaScript.”[62] Because the software components of the Concordia system are open-source and supported by Web-accessible developers’ forums, there may be considerable opportunity for enhancing metadata portability and standards compliance. A section of the Zotero Web site addresses aspects of metadata exposure for harvesting and importing in various bibliographic metadata systems.[63]

The Systems Approach

The principles of good oral history practice, as well as those of responsible curation, require the pursuit of the highest standards in documenting all aspects of oral history interviews and maintaining that documentation in stable, accessible, and easily managed and shared systems. This obligation to interview participants, their communities, and researchers, truly begins at the inception of every oral history project. By the time an oral history is handed over to a repository for access and preservation, documentation must already exist, in trustworthy form and complete detail. This suggests that there is need in oral history practice not only for best practices in metadata management for curatorial purposes, but also for best practices in oral history project management and record-keeping, as well as systems to support such management. Scalable, flexible, adaptable, and interoperable project management systems for oral history have yet to emerge. When they do, they will provide oral history practitioners with the means to do their work and accomplish their mission with long-term access and preservation in mind from the beginning. These systems will strike a balance—adjustable for differing levels of resources and support—between usability and sophistication, and between exhaustiveness and cost. They will also include measures of quality, tools by which their usability, their suitability for particular circumstances, their anticipated longevity and capacity for technological upgrade, and their cost can be assessed.

In short, true metadata management for oral history requires a systems view, an overall perspective on oral history practice, from start to finish, with cognizance of the fact that for historical resources such as oral history, there should never be a finish: as these resources are created to endure.