Ex-Meta Researcher Torrented Pirated Books, Deleted It 'Due to Litigation'

In a series of lawsuits in California federal court, Meta is being sued over the use of pirated books to train its Llama AI models. The lawsuits are separate, but Judge Vince Chhabria handles them together. The plaintiffs include authors such as Sarah Silverman and Ta-Nehisi Coates, and publishers such as Entrepreneur Media and Chicken Soup for the Soul. In August, Meta told Entrepreneur that it did not believe it had obtained the pirated Books3 dataset through BitTorrent. Books3 is a collection of nearly 200,000 books, taken from the library of the private torrent tracker Bibliotik. However, recent deposition testimony from a former Meta researcher tells a different story, according to the rightsholders. Books3 Torrenting Testimony In a joint status report filed with Meta on Tuesday, the rightsholders say that Stephen Roller admitted during a deposition last week that he torrented Books3 and knowingly distributed it. Roller worked at Meta between 2017 and 2023, and the download took place during that period, they say. Roller didn’t recall exactly when he acquired the dataset, the plaintiffs note. He did acknowledge that Books3 was created in 2020, and that he didn’t work for any other company while at Meta. This is relevant, as the court ruled last year that training Llama on books was fair use, leaving Meta’s BitTorrent downloads as the main open issue in the authors’ case. Earlier this month, a judge ordered Meta to name the employees who approved torrenting from shadow libraries. Meta has acknowledged using Books3 before. In early 2024, it admitted that it used portions of the dataset to train its Llama 1 and Llama 2 models. That admission didn’t say how Meta obtained the dataset. Roller also testified that Meta had copies of Books3 by the time he left its AI research group FAIR, the plaintiffs write. He also named two research scientists who, they say, likely acquired their own copies. Roller no longer has his own copy. According to the plaintiffs, he deleted it about a year ago, possibly after he was subpoenaed for the second time. He did so “[d]ue to all this litigation,” as “it seem[ed] contentious to have copies of The Pile [which includes Books3].” The claim that Meta had multiple copies, at least one of them torrented, was news to the plaintiffs, they write. They plan to ask Meta how many copies of Books3 it acquired and how, and will ask the court to step in if that fails. Meta: Could Have Been for Himself Meta sees things differently. The company notes that the status report is the first time the rightsholders raised these concerns, and the tech giant argues that Roller’s deposition doesn’t support them. “Mr. Roller testified that he did not know when he downloaded a copy of the ‘The Pile’ via torrent, and did not testify that his download of the ‘The Pile’ was for Meta or at Meta, noting it could have been for himself or for another employer,” Meta writes. According to Meta, Roller also said that he didn’t know whether more than one copy was downloaded, or whether the two researchers he mentioned acquired Books3. He left Meta in January 2023 and has since worked for three other companies working on AI. The deleted copy was stored on Roller’s personal computer, Meta adds. He removed it more than two years after leaving the company because he “didn’t need it, so [he’d] rather not have it.” A (Pirated) Book Club Books3 is not the only pirated material that comes up in the status report. The plaintiffs also suggest that pirated books may have been used at Meta for things other than AI training. They want Meta to produce the contents of an internal Google Drive folder titled “ML Books”. In addition, they want records of a Meta book club that may have used books from shadow libraries. In one of the documents Meta produced, an employee shared a “pirated book” with a colleague. “i know for vellum sake this is prob very ironic,” the employee wrote. When asked whether the book was pirated, the employee replied with a link to the “ML Books” folder. The recipient told another colleague, “bro she tryna get me fired.” He added that he “may need her to delete those messages.” The plaintiffs see this as relevant. If Meta employees used shadow library books for a book club, that would impact Meta’s fair use defense. However, Meta argues that this request comes too late. The plaintiffs first asked for the linked files on October 9, which was more than a month after the discovery period had closed. As for the folder itself, Meta says it is investigating and will discuss it with the plaintiffs when it knows more. Missing Server Logs Roller’s copy of Books3 is not the only material the plaintiffs say is missing. Meta still hasn’t handed over all command history files from its torrenting, they note, despite court orders to do so. According to the plaintiffs, these logs show how Meta configured its torrent software and which torrent files it targeted, and thus which works. Specifically, Meta has yet to hand over command history files from any of the servers it used during its torrenting periods, the plaintiffs say. They now ask the court to order Meta to say whether any of these files were deleted, and when. Meta, in turn, counters that it has produced substantial quantities of logs from those servers and is still searching backups, handing over new files on a rolling basis. According to Meta, there is no reason to believe any command history files were deleted. Entrepreneur’s case is scheduled to go to trial in the spring of 2027, with the other publishers lined up behind it, should it settle before then. Meanwhile, the list of related cases keeps growing. This month, publishers including Elsevier, Hachette and Macmillan, as well as more authors, joined the queue. — A copy of the joint case management statement, filed at the U.S. District Court for the Northern District of California, is available here (pdf).