Rare Books Destroyed for AI Training Datasets
Data broker ISBNdb has been shipping up to a million pre-2022 books to AI labs for training large language models. The physical copies are destroyed using spine-cutting scanners to create digital datasets. Critics argue this practice risks irreversible cultural losses, particularly for rare books.
Key points
- ISBNdb supplies physical books in bulk to AI developers for human-written material unaffected by modern chatbot output.
- Pre-2022 books are considered 'dense, edited, authoritative' and offer cleaner datasets compared to internet content.
- The practice aims to avoid model collapse, where AI systems lose quality after training on synthetic content.
- Critics argue the destruction of rare books for AI training datasets risks irreversible cultural losses.
- ISBNdb's actions have attracted growing criticism and concerns about the preservation of cultural heritage.
The practice of destroying rare books for AI training datasets has sparked controversy in the tech industry. Data broker ISBNdb has been shipping up to a million pre-2022 books to AI labs for training large language models. The physical copies are destroyed using spine-cutting scanners to create digital datasets.
The approach aims to provide cleaner datasets for AI models by avoiding text generated by contemporary large language models. Pre-2022 books are considered 'dense, edited, authoritative' and offer a more reliable source of human-written material. This is in contrast to internet content, which is increasingly filled with machine-generated material of uncertain quality.
However, critics argue that this practice risks irreversible cultural losses, particularly for rare books. The destruction of these books may result in the loss of valuable historical and cultural information. As AI technology continues to advance, the preservation of cultural heritage becomes increasingly important.
The debate surrounding the use of rare books for AI training datasets highlights the need for a more nuanced approach to the preservation of cultural heritage. While the use of AI technology can provide numerous benefits, it must be balanced with the need to preserve our cultural heritage for future generations.
Sources
The WireByte editorial team synthesises technology news from multiple primary sources, verifies the facts, and links every source. Articles are produced with AI assistance and reviewed under our editorial policy.