| Internet-Draft | AI Preferences | June 2025 |
| Keller & Thomson | Expires 7 December 2025 | [Page] |
This document proposes a standardized vocabulary for expressing preferences related to how digital assets are used by automated processing systems. This vocabulary allows for the creation of structured declarations about restrictions or permissions for use of digital assets by such systems. The vocabulary is agnostic to the means by which it is conveyed. The definitions in the vocabulary facilitate a shared understanding between entities that express such preferences and those that use the associated digital assets.¶
This note is to be removed before publishing as an RFC.¶
The latest revision of this draft can be found at https://ietf-wg-aipref.github.io/drafts/draft-ietf-aipref-vocab.html. Status information for this document may be found at https://datatracker.ietf.org/doc/draft-ietf-aipref-vocab/.¶
Discussion of this document takes place on the AI Preferences Working Group mailing list (mailto:ai-control@ietf.org), which is archived at https://mailarchive.ietf.org/arch/browse/ai-control/. Subscribe at https://www.ietf.org/mailman/listinfo/ai-control/.¶
Source for this draft and an issue tracker can be found at https://github.com/ietf-wg-aipref/drafts.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 7 December 2025.¶
Copyright (c) 2025 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
This document defines a common vocabulary of terms for automated systems that process digital assets. The primary purpose of this vocabulary is to enable machine-readable expressions of preferences about how digital assets are used by automated processing systems, in the context of training AI models and other forms of text and data mining (TDM).¶
The terms defined by the vocabulary can be used to describe, in a standardized way, the types of uses that a declaring party may wish to explicitly restrict or allow. Preferences are then expressed as a grant or denial of permission concerning each of the types of use defined in the vocabulary. This ensures that preferences can be communicated, processed, and stored in a consistent and interoperable manner.¶
The vocabulary is neutral to the technical details of how systems act on preferences. It is designed to ensure that preference information can be exchanged between different systems and consistently understood.¶
The vocabulary is intended to govern the use of digital assets for the training of AI models and other forms of automated processing. It does not concern itself with the mechanisms involved in obtaining digital assets (i.e., crawling).¶
The vocabulary is intended to work in contexts where such preferences result in legal obligations (such as rights reservations made by rightholders in jurisdictions with conditional TDM exceptions), and in contexts where this is not the case. It is without prejudice to applicable laws and the applicability of exceptions and limitations to copyright.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
This document uses the following terms:¶
The vocabulary is a set of categories, each of which is defined to cover a class of usage for assets. Section 4 defines these categories in more detail.¶
A statement of preference is made about an asset. Statements of preferences can assign preferences to each of the categories of use in the vocabulary. Preferences regarding each category can be expressed either to allow or disallow the usage associated with the category.¶
A statement of preferences can express preferences about some, all, or none of the categories from the vocabulary. This can mean that no preference is expressed for a given usage category.¶
Some categories describe a proper subset of the usages of other categories. A preference that is expressed for the more general category applies if no preference is expressed for the more specific category.¶
For example, the TDM category might be assigned a preference that allows the associated usage. In the absence of any statement of preference regarding the AI Training category, that usage would be also be allowed, as AI Training is a subset of the TDM category. In comparison, an explicit preference regarding AI Training might disallow that usage, while permitting other usage within the TDM category.¶
After processing a statement of preferences the recipient can assume that each category of use has a preference in one of three states: "allowed", "disallowed", or "unknown".¶
This section defines the categories of use in the vocabulary.¶
Figure 1 shows the relationship between these categories:¶
This list of specific use cases may be expanded in the future, should a consensus emerge between stakeholders, to include categories that address additional use cases as they emerge. In addition to these categories defined in the vocabulary, it is also expected that some systems implementing this vocabulary may extend this list with additional categories for their particular needs.¶
The act of using one or more assets in the context of any automated analytical technique aimed at analyzing text and data in digital form in order to generate information which includes but is not limited to patterns, trends and correlations.¶
The use of assets for TDM encompasses all the subsequent categories.¶
The act of training machine learning models or artificial intelligence (AI).¶
The use of assets for AI Training is a proper subset of TDM usage.¶
The act of training General Purpose AI models that have the capacity to generate text, images or other forms of synthetic content, or the act of training other types of AI models that have the purpose of generating text, images or other forms of synthetic content.¶
The use of assets for Generative AI Training is a proper subset of AI Training usage.¶
The act of using one or more assets as input to a trained AI/ML model as part of the operation of that model (as opposed to the training of the model).¶
The vocabulary is used by referencing the terms defined in the Section 4 section above, directly or via mappings, in accordance with how they are defined in this document.¶
A recipient of a statement of preferences that follows this model might receive more specific instructions in two ways:¶
Extensions to the vocabulary might define more specific categories of usage. Preferences about more specific categories override those of any more general category.¶
Statements of preferences are general purpose, machine-readable statements that cannot override contractual agreements or more specific statements.¶
For instance, a statement of preferences might indicate that the use of an asset is disallowed for AI Training. If arrangements, such as contracts exist that explicitly permit the use of that asset, those arrangements likely apply, unless the terms of the arrangement explicitly say otherwise.¶
The vocabulary does not preclude the use of other specific categories. Any statement of preference based on this vocabulary shall not be interpreted as restricting the use of the work(s) strictly for the purpose of search and discovery as long as no restriction is declared through search-specific means such as [RFC9309].¶
Systems referencing the vocabulary must not introduce additional categories that include existing categories defined in the vocabulary or otherwise include additional hierarchical relationships.¶
This section defines an exemplary serialization format for preferences. The format describes how the abstract model could be turned into Unicode text or sequence of bytes.¶
The format relies on the Dictionary type defined in Section 3.2 of [FIELDS].
The dictionary keys correspond to usage categories
and the dictionary values correspond to explicit preferences,
which can be either y or n; see Section 6.2.¶
For example, the following is a preference to allow AI training (Section 4.2), disallow generative AI training (Section 4.3), and and state no preference for other categories other than subsets of these categories:¶
ai=y, genai=n¶
Each usage category in the vocabulary (Section 4) is mapped to a short textual label. Table 1 tabulates this mapping.¶
| Category | Label | Reference |
|---|---|---|
| Text and Data Mining | tdm | Section 4.1 |
| AI Training | ai | Section 4.2 |
| Generative AI Training | genai | Section 4.3 |
Any mapping for a new usage category can only use lowercase latin characters (a-z), digits (0-9), "_", "-", ".", or "*". These are encoded using the mappings in [ASCII].¶
The abstract model used has two options for preferences associated with each category:
allow and disallow.
These are mapped to single byte Tokens (Section 3.3.4 of [FIELDS])
of y and n, respectively.¶
Structured Fields [FIELDS] describes a byte-level encoding of information, not a text encoding. This makes this format suitable for inclusion in any protocol or format that carries bytes.¶
Some formats are defined in terms of strings rather than bytes. These formats might need to decode the bytes of this format to obtain a string. As the syntax is limited to ASCII [ASCII], an ASCII decoder or UTF-8 decoder [UTF8] can be used. This results in the strings that this document uses.¶
Processing (see Section 6.5) requires a sequence of bytes, so any format that uses strings needs to encode strings first. Again, this process can use ASCII or UTF-8.¶
There are two ways by which this syntax might be extended: the addition of new labels and the addition of parameters.¶
New labels might be defined to correspond to new usage categories. Section 5.2 addresses the considerations for defining new categories. New labels might also be defined for other types of extension that do not assign a preference to a usage category. In either case, when processing a parsed Dictionary to obtain preferences, any unknown labels MUST be ignored.¶
The Dictionary syntax (Section 3.2 of [FIELDS]) can associate parameters with each key-value pair. This document does not define any semantics for any parameters that might be included. When processing a parsed Dictionary to obtain preferences, any unknown parameters MUST be ignored.¶
To process a series of bytes to recover the expressed preferences, those bytes are parsed into a Dictionary (Section 4.2.2 of [FIELDS]), then preferences are assigned to each usage category in the vocabulary.¶
The parsing algorithm for a Dictionary produces a keyed collection of values, each with a possibly-empty set of parameters. The parsing process guarantees that each key has at most one value and parameters.¶
To obtain preferences for each of the categories in the vocabulary, iterate through the categories. For the label that corresponds to that category (see Table 1), obtain the corresponding value from the collection, disregarding any parameters. A preference is assigned as follows:¶
If the value is a Token with a value of y,
the associated preference is to allow that category of use.¶
If the value is a Token with a value of n,
the associated preference is to disallow that category of use.¶
Otherwise, a preference is not expressed for that category of use.¶
Note that this last alternative includes
the key being absent from the collection,
values that are not Tokens,
and Token values that are other than y or n.
All of these are not errors,
they only result in no preference being inferred.¶
An important note about this process and format is that, if the same key appears multiple times, only the last value is taken. This means that duplicating the same key could result in unexpected outcomes. For example, the following expresses no preferences:¶
ai=y, ai="n", genai=n, genai, tdm=n, tdm=()¶
If the parsing of the Dictionary fails, no preferences are expressed. This includes where keys include uppercase characters, as this format is case sensitive (more correctly, it operates on bytes, not strings).¶
This format is only an exemplary way to represent preferences. The model described in Section 3, can be used without this serialization.¶
TODO Security¶
This document has no IANA actions.¶