Scribes: Chris Needham, Martin Thomson
Mark introduced the meeting, as a checkpoint before IETF San Francisco, to continue the conversation.
The most recent draft is vocabulary-08. It doesnât have consensus, but we want to get to consensus soon. This meeting will go through the issues list.
Can we close 208? The word âsubstantiveâ doesnât appear in the draft. No-one voiced a concern about closing.
Mark: Issue 172, RAG and grounding category, we now have the ai-use category. Closing.
Mark: Issue 150, substitutive use category, from Brad. Brad confirms this can be closed.
Keep open the display based preferences issue.
Two issues from EKR, 153 and 154. EKR wants to re-read those.
Issue 163: Asset terminology. Martin has waited for other discussion to settle down, but has now created a PR to replace âassetâ with âartifactâ, and suggests leaving until other issues are settled. Timid pointed out that the PR needs updating to change the terminology section. Mark suggested sending objections to the list.
Issue 167. Distinguishing access from use. Martin created a PR to edit section 3.3. Mark suggests the word âpreventâ might be too strong, an alternative might be ârestrictâ or âconstrainâ. EKR suggested how it correspond to things like robots.txt, and not to focus so much on the mechanisms. Kevin agreed with EKR.
Issue 192, relation to existing laws. Max should come back with a revised proposal if he still wants to pursue this.
Issue 217, use of âsyntheticâ. The draft still uses that word. Chris explain that synthetic opens a distinction where material thatâs quoted. Mark suggested that the word should do work, if we included.
Kevin:
Timid: Itâs about trying to create a line between classifier type uses and more generative, or where content is completely new and not cited. We need to make the line clearer. I prefer to keep âsyntheticâ, but itâs not successful drawing the line.
EKR: The sentence below talks about classifiers, so more load-bearing than the word âsyntheticâ. Identify which things are or arenât synthetic. Try to agree whatâs in scope vs not, as a suggestion. Iâm uncomfortable with it being load-bearing.
Mark: For people who want to keep it, can they create a definition?
Brad: Agree with Chris, and share EKRâs concern. The main rationale thatâs already covered. There needs to be a line between a generative and non-generative AI mode. I worry about keeping the word, your suggestion to define it makes sense. Iâd vote for deleting it.
Nate: I also think we should delete it. In copyright context, AI companies have been willing to argue that the content is original content. Absence evidence the word is load-bearing, we should remove it.
Kevin: The output Nate describes is what the definition intends to cover. Weâre not trying to create a loophole, there needs to be trust.
Nate: I recognise your good intent. But we need this to work for companies not represented here.
EKR: Itâs not an issue of trust, but we need to be clear for people who havenât participated in this process. The document should stand on its own feet.
Timid: Maybe better to remove it and ask people to make the case to add it back in. Iâm hearing more investment from the people who want it removed.
Chris: Happy to work with others offline, or go with Timidâs proposal
Paul: Concur with Timid
Matthew: Also agree with Timidâs proposal and EKR.
Mark ran a straw poll, âDo you feel strongly that we should keep the âsyntheticâ term in the draftâ?
Farzaneh: We donât have enough background on why synthetic is there, can we defer a poll to the IETF meeting?
Poll result: Yes: 3, No: 15, No Opinion: 1
Mark and Suresh asked that people coordinate to make the case, and to come up with a definition, and to flesh out the argument for removing, going into San Francisco.
Issue 231, de-minimis content generation. Erin suggested that it may have been folded into the draft already, but wasnât sure. Martin pointed to the last comment in the issue, which explains where itâs addressed. There was discussion about whether the issue operates at a higher level across the draft, so will need to be reviewed carefully.
Mark: Issue 245, redistribution of models.
Kevin: I think we agree on the intent, redistributing a model shouldnât be a way to launder preferences, but how to phrase it in the vocabulary?
Martin: There are two PRs that address this. One from Timid, which was closed, but thereâs 253 that includes text that could address the issue, albeit not directly. Thereâs text in the current draft, that Timid proposed, on conveying.
Timid: Thatâs old, look at draft 08 section 3.3. Iâm happy to close 245.
EKR: If I scrape sites and convey the data, I donât have to convey preferences. But if I train a model I do. Itâs an odd asymmetry.
Mark: If youâre training models, wilful blindness isnât compliance, is what weâre saying.
Eric: If A scrapes and distributes the data, and B trains on it. B can simply elide them.
Mark: Agree, A scraping and eliding the preferences should not protect B.
Martin: EKRâs concern is reasonable. I opened PR 252 and 253 to address a different point, but that text would address the retention of preferences with crawling and scraping, and then transformation. Leonard and Nate have voiced concerns. If you want symmetry, look at those PRs.
Brad: For us, separating use preference from crawling through to distribution of the model. Itâs important that it does travel. Donât like introducing the idea of wilful blindness, legal constructs.
Mark: As written itâs a requirement on someone not doing AI training, which is weird. You canât comply with preferences if you donât have full information about them.
Brad: Thereâs substantial activity in crawling that arenât those developing or distributing models. It would leave a big gap if we require the crawling and model developing to be the same party.
Mark: Weâll revisit in San Francisco.
Timid: Common Crawl already bundles preferences with headers and robots.txt. They help with internet archive, so a significant part of the archiving community is already doing this.
Mark: Issue 246, caching and embedding.
Martin: There was a concern that someone would collect stuff and apply a transformation, e.g., HTML to markdown or creating an embedding and distributing it. We expect preferences to be carried forward. Leonard raised 240, which is an alternate spelling of it.
Timid: I like this text (in 253). It empowers people to honour preferences if thatâs their intent.
Suresh: I prefer the term âensureâ from PR 252.
Martin: Sounds good, we can align the two.
Kevin: What does 253 change? In an AI system, text is tokenised and transformed into embeddings in an implementation of the system. It seems that trying to identify the component parts of the system if problematic, things not invented or thought of now might be considered not in scope.
Timid: This tries to target processes pre-training and pre-tokenisation. Nate spoke about existing markets where third parties selling data APIs, where the data has been transformed from the data crawled or collected.
Nate: There are commercial APIs that will sell the embeddings, drawn from the materials. Itâs important that the preferences applies to those embeddings. Tyler also raised the concerns.
Kevin: Does this make sense as part of the crawling conversation, if thatâs the case you want to cover?
Nate: Maybe, Iâd need to review the whole thing. Also, someone with more technical knowledge.
Matt: Would it address Kevinâs concern to say that it happens between different parties?
Kevin: In the past weâve tried to avoid caring about organisational boundaries.
Martin: A system could span organisations. We could say different parties if it would help clarify things.
Mark: We nee more discussion, e.g., on the list.
Mark: Issue 247 on creating a data model, from Leonard.
Martin: I see this as an editorial issue. It doesnât change the meaning. But the discussion is whether the draft describes the data model clearly enough.
Martin: This is where Chris and Kevinâs proposal to come in.
Kevin: The RAG/grounding distinction. We landed on a two part definition. The user has to express specific intent to interact with a resource and it has to be a specific resource. If you ask a question and the AI system finds resources, itâs in scope. If the user gives the URL, thatâs outside.
Other categories are worth talking about also.
Chris: Thatâs right, this is an attempt to define whatâs directly provided.
Timid: I think itâs a good definition of what was previously called system use. I think people want to express a preference of whatâs excluded here. That shouldnât hold this up, but the naming needs to be more specific.
Brad: I find this to be in the opposite direction. Rather than provide the asset to interact with the asset. I thought providing it was the dividing line. Whatâs excluded is the part thatâs more damaging: I want you to go find this article and summarise it (what does interact mean?) This is the intermediation that we want to express a preference about. Just because youâve identified a specific asset it shouldnât bypass.
Alissa: Two questions. Who is doing the interacting? Implies the intent is for the AI system, but not clear. Hypothetically, I talk to a chatbot and say go to the bbc.co.uk homepage, is that providing the asset.
Kevin: In our discussion, that would not be interacting with a specific asset.
Chris: To Bradâs point, my intent was itâs for cases where the user has already seen the asset, so the intent isnât to include substitutive use.
Nate: Need categories to cover all use. On line drawing, I gave an example of clicking a link in search. This is a difficult line to draw in practice, and hard for courts to determine. The line still is unclear.
Eric: What does specific intent mean here? If itâs unambiguous and you donât need AI to distinguish? Drawing the line between a general asset and specific asset.
Mark: The question is where to draw the line, then how to. A couple of options, have a definition with the carve-out or all use. Weâll discuss the other aspects, but the task here is the line drawing. Encourage work on proposals on how to draw the line.
Martin: This is an admirable attempt with economical use of words. The line requires some precision, on how the resource is identified. There could be different interpretations.
Mark: Agree. Write proposals with more words that we could cut down later, include explanatory cases to communicate intent.
Alissa: Would it be easier to define the exception? Previous discussion of things that donât change weights. Figure out what the exception is.
Martin: A specific category?
Alissa: Agree youâll need more words to describe it, but honing in on the exception might be more straightforward conceptually might be easier. It would be a defined term, then has relationship to prior definition of things that donât change the weights, that was discussed in London.
Nate: If we have an exception, we need to be clear about what it is. I proposed two expressed categories. What do we intend to exclude, and for what purpose? Example of a dance video that people use AI to make copycat videos. It happens at scale. Spammers are users too. Why are we drawing the line? What do we actually want to exclude?
Farzaneh: In London we talked about carve-outs for accessibility and text to speech. We should discuss that as well, similar to what we have for search.
Mark: Thatâs still to do, yes.
Mark: Good discussion today. Thereâs things to work on for San Francisco. Bring to the mailing list. We havenât declared consensus on anything yet. We want too run some consensus calls soon. If issues need more consideration, bring them to our attention soon.
Suresh: Farzaneh, you were going to write a document on the use case. Please do for San Francisco if you can.
Nate: Important we donât abandon artists, and make this a publisher focused protocol. Iâve repeatedly raised examples of uses covered by this exception that should be expressible, but just been told to add to the use case wiki, but itâs not being used in the discussion. If we have a carve out for user intent, the onus needs to be on those proposing it to explain why they want such a broad carve-out. If weâre saying you canât express a preference about some use, then that means weâre saying that usage should always be non-consensual. I have heard arguments for accessibility usage being non consensual. But, beyond that, Iâve not heard anyone make actual arguments for why other generative user use should be non-consensual. If thereâs other usage that animates it this exception, letâs discuss in public for all to hear. Iâve put forward examples in the wiki that are harmful, but thereâs no engagement. Some of us want to preserve a human internet. Weâre talking about abandoning artists and creators. That perspective is under represented here. Iâm concerned about the direction here.
Mark: Weâre at time. Take thoughts to the mailing list. Be ready for San Francisco. Thanks all.