Chapter 6 Shadow Libraries and Pirate Infrastructures

What We Do in the Shadow Libraries

Two decades of calls for OA have made a difference. Much research work is now available openly. New business models have emerged to support the publisher labor that underwrites such publications (although not all of these have met with academic favor) (Digital Science et al.). Nonetheless, substantial portions of the research literature remain behind paywalls or in paid-for books, with no open, free digital correlate. Even if the OA movement succeeded in its goals tomorrow, vast historical reservoirs of scholarship would remain locked away for the duration of the international copyright term (seventy years after the author’s death in many cases). OA gets us partially toward a dream of free accessibility of the research literature, but it is not the whole story.

Frustrated by the slow growth of OA and the realization that even OA will not achieve 100 percent access, various groups and individuals worldwide have built a series of digital infrastructures, called shadow libraries, that circumvent publisher paywalls. Shadow is an apt term here. In Star’s definition, we recall, one of the crucial features of infrastructure is that, when they are working, they are invisible. The emergence of their “shadows” implies that these infrastructures, with all their flaws, are no longer invisible; they now occlude the light. The most well known and significant of these libraries are Sci-Hub, Library Genesis, Anna’s Archive, and Memory of the World, each hosting substantial volumes of material.1 In addition, UbuWeb, which explicitly describes itself as a “pirate shadow library,” hosts hundreds of thousands of specifically avant-garde digital artifacts. Other shadow libraries include Monoskop (“an independent web-based educational resource and research platform for arts, culture and humanities founded in 2004”) and Aaaaarg (originally AAARG, an acronym of Artists, Architects, and Activists Reading Group), both of which bill themselves specifically in educational terms.

These libraries exist on a spectrum of legal antagonism. UbuWeb, for instance, traffics in specifically avant-garde material. These artifacts sit outside the conventional sphere of commercial exchange, quite simply because, although they may hold historical and cultural significance, nobody is willing to pay for them. As such, despite the fact that “by the letter of the law, the site is questionable,” UbuWeb has “never been sued—never even come close” (UbuWeb). Sites like Monoskop tend also to function more as aggregators than actual hosts of pirated material. Again, the legal status of such linking is unclear.2 Memory of the World, by contrast, veers much more deeply into clearly illicit territory in many jurisdictions, hosting copies of in-copyright works that have commercial value. Yet the site situates itself within a specific lineage of free and public libraries as a public good that exists for the education of the people (Doran). At the very extreme end of the potentially illegal spectrum sit Library Genesis, Sci-Hub, and Anna’s Archive (the latter of which only came to prominence in the later stages of writing this chapter).

Library Genesis holds 33 terabytes of books and more than 60 terabytes of scientific journal articles on behalf of Sci-Hub (Eve, “Lessons from the Library”).3 These pirate libraries are extensive, providing access to almost all published academic journal output (Banks; Himmelstein et al.). However, it is worth noting that national deposit libraries, backed by government mandates and legal structures, are substantially more extensive than the shadow archives. For instance, the British Library has a central collection of approximately 170 million items total and 13.5 million books, with a digital collection over a petabyte in size (British Library). By contrast, Library Genesis stores fewer than 3 million books, although it constantly grows. While shadow libraries are large, they struggle to compete on a scale with formal, resourced institutions.

The terminology of shadows that surrounds these archives is itself of interest (as is the designation of “pirate”). As historians studying color have shown, the idea of the “shadow” has changed throughout history. Until the eighteenth century, shadows were not defined as colorless opposites of objects. Instead, before then, shadows were partial transparencies and outlines, primarily in art and heraldry. Perhaps most important, though, in heraldic terms, shadows were used to reveal a hidden part of the family tree, not a mirror of it as might be expected (Pastoureau 26–27). Shadows, in heraldry, show illegitimate family links—children born out of wedlock—rather than formal reflections of the main subjects. Hence, as Nanna Bonde Thylstrup observes, it is worth reflecting on the “inherently unstable form of shadow libraries as a cultural construct” (Thylstrup 98). While we can think of shadow libraries as “mirror libraries” that ape mainstream, legitimate bibliographic venues, the terminology of shadows allows us to consider them within a more complex lineage of subversion and counternorms. Shadow libraries, the bastards of archival worlds, upset our relationships with traditional information retrieval.

Shadow libraries define themselves against a standard of morality that attempts to transcend local lawmaking and national-level jurisprudence; copyright laws, they say, are wrong. “Within decades,” writes one operator of Library Genesis, “generations of people everywhere in the world will grow up with access to the best scientific texts of all time. [. . .] the quality and accessibility of education to the poor will grow dramatically too. Frankly, I see this as the only way to naturally improve mankind: we need to make all the information available to them at any time” (Bodó 25). The amateur “archivists” who have taken it upon themselves to seed Library Genesis on BitTorrent believe, indeed, that their “initiative fulfills United Nations/UNESCO world development goals that mandate the removal of restrictions on access to science” and “limiting and delaying humanity’s access to science isn’t a business, it’s a crime, one with an untold number of victims and preventable deaths” (u/shrine, “Library Genesis Project Update”). But, as before, the ethical foundations on which these initiatives rest are rooted in the discourse of profit against lives.

Shadow libraries straightforwardly appear, from the perspective of the Global North, to be illegal. They are premised, after all, on a total disregard for copyright law and the contracts that academics have signed. Regardless of whether they do so from the standpoint of any moral superiority, courts in the United States have continually found against Alexandra Elbakyan, the founder of Sci-Hub (“The Library of Alexandra,” as a recent podcast quipped [WNYC Studios]), awarding millions of dollars in compensation to publishing companies such as Elsevier (Schiermeier; Harris and Barrett). However, an uncomplicated designation of illegality for these shadow libraries does not hold. In many jurisdictions worldwide, substantially more liberal laws permit the distribution of educational material, even when it is under copyright.

A good example is India, where Elbakyan is currently defending a lawsuit filed by Elsevier, Wiley India Pvt. Ltd and the American Chemical Society at the Delhi High Court (Reddy and Mishra). It is currently unclear whether she will succeed in defending Sci-Hub in this jurisdiction, but the court did not simply throw the case out as a clear-cut verdict.

Several complex legitimation discourses circle projects such as Sci-Hub and chime with longer-term economic rationalities. Even the terminologies of archive—so contested already—and library, let alone pirate, confer various statuses of veridiction upon these infrastructures.4 However, one of the perpetually resurfacing accusations is that Sci-Hub is run by the Russian state and is designed to harvest university credentials from unwitting users for malicious foreign agents (Harris and Barrett; Erkal; Masnick). The argument here goes that users who wish to support the (they believe) laudable goals of Sci-Hub donate their library credentials, but Russian government agents are then using these for other purposes. Such rumors—while remaining unverified—are fueled in part by the fact that Sci-Hub grew, according to Balázs Bodó and Elbakyan, from the sharing cultures of Soviet communism. This stands in stark contrast to other piracy cultures, such as the warez scene, which thrives on cutthroat competition that looks more akin to a capitalist economy (Bodó; Elbakyan, “A Robin Hood in the World of Science”; Eve, Warez). As such, elements of Sci-Hub’s ideology are economic anathema to U.S., U.K., and European governments operating under the political rationalities of neoliberalism (defined as the disenchantment of politics by economics, as per Davies). In this political climate, and particularly since the Russian invasion of Ukraine in 2022, it then becomes difficult to distinguish whether these allegations have merit or if they are mere smear campaigns made by powerful economic actors. Elbakyan denies these accusations (Elbakyan, “How the Chronicle Is Trying to Malign Sci-Hub”).

Disciplines and Hierarchy

There is, of course, a specific disciplinary resonance to the rationale for shadow libraries. Although Library Genesis is a nonspecific library/archive, Sci-Hub explicitly encodes science in its name (although it is notable that science, in the European context, includes the humanities disciplines). Research that “saves lives” implies the fields of biomedicine, medical physics, virology, and other disciplines that contribute toward the extension and preservation of human vitality, rather than “merely” its enrichment. Research in other spaces cannot posit the same urgency. That “nobody will die” if they cannot read the latest literary criticism seems somewhat self-evident. Hence, an argumentative terrain emerges from shadow libraries in which it is harder to justify the pirating of literary criticism than, say, cancer research. In the case of the latter, if a for-profit publisher refuses to make this research available, it is easy to argue that people may die as a result, and this would be, in some moral sense, a criminal outcome. The same cannot be said of the humanities disciplines (and much social-scientific research).

Thus, according to the arguments of these pirate sites, there comes a specific hierarchy of disciplinary values that places the humanities far lower than many of their scientific counterparts. Indeed, the arguments for access and preservation, whether they come from the formal open-access movement to scholarship or from pirate archives, are usually centered around scholarship’s impact on the world. When the value of that scholarship is more centered on cultural understanding, it becomes harder to justify a copyright-violating crusade because the external moral and legal blame is significantly less. That is to say that arguments for the ethical necessity of OA to research in the medical space often center on the fact that it can be seen as morally unacceptable to profit from the restriction of medical scholarship, which could lead to people dying. This argument has more or less traction depending on the stance toward health care in the nation in question. Economic advancement—a concept that embeds a specific telos of progress—is undoubtedly, if the United States is anything to go by, no sign of a move toward universal health care. Nevertheless, universal health care is a sign of accepting a “right to health,” included in several declarations, such as the Universal Declaration of Human Rights, the International Covenant on Economic, Social, and Cultural Rights, and the Convention on the Rights of Persons with Disabilities. In cultures where people have a right to health or health care—and where it is morally unacceptable to deny such things to people based on their wealth—blocking access to the medical literature with paywalls becomes a social ill.

Perhaps it is simply true that preserving and improving life and health are fundamentally more critical tasks than understanding culture. Suppose that we return to taxonomies such as Abraham Maslow’s well-known hierarchy of needs. In that case, it seems clear that without health and security, one is unlikely to profit significantly from cultural and humanistic endeavors.5 However, this is also not straightforwardly true. It is not possible to advance medically without comprehending medical history and medical ethics, for example. Indeed, basic medical ethics training is part of any medical degree, which includes the infamous history of human experimentation during World War II and the Tuskegee Syphilis Study. Science fiction has also often served as the primary moral compass by which the efforts of science are hypothesized; warnings against the tide of unmitigated progress or imaginary explorations of human interactions with technological advancement (e.g., McLane). To hierarchize the humanities at the top of a pyramid of needs—as though they are merely a luxury—is to neglect their embeddedness in how we understand and measure the natural sciences.

Nevertheless, shadow libraries predicate their ethical existence on access to the scientific and medical literature. They justify their international presence by gesturing toward the vast global inequalities in current scholarly communications practices. Their missions are couched in discourses of scientific enrichment and civil rights–era formulations. As the authors of one letter of solidarity put it: “This is the time to recognize that the very existence of our massive knowledge commons is an act of collective civil disobedience” (Barok et al.). However, while the rhetoric of civil disobedience here at once invokes freedom rides and other racially liberating protests in the U.S. context, the primary objection that stands in the way of shadow libraries is the civil contract/tort law against individuals. While civil rights protests were primarily against statutes, the primary alleged wrong of shadow libraries is to violate the copyright of individuals. Indeed, it may be true that Library Genesis and Sci-Hub operate at such a scale that they count as conspiracies to violate copyright (a criminal offense) rather than merely infringements on individual rights. But the fundamental logic to which this civil disobedience objects is the individual ownership of published research in many jurisdictions: tort copyright law. The protest is, to some extent, against the very idea of intellectual property (see Vaidhyanathan; Johns; Stallman).

Technical Capacity and Decentralization: Shadow Libraries as Open Scholarly Infrastructure

As with many pirate cultures, shadow libraries play a game of hide-and-seek (or whack-a-mole) with law enforcement. They also face a fundamental paradox. To be most useful, a shadow library must be well known and prominent. There is little point in establishing a pirate archive designed to liberate all human knowledge if nobody can use it. On the other hand, all increases in exposure lead to additional attention from law enforcement and the ever-prevalent risk of shutdown. To counter this threat, shadow libraries adopt principles of technological and, to some extent, organizational decentralization.

These resilience principles have also been well articulated in the formal research space. Geoffrey Bilder et al., for instance, present a set of Principles of Open Scholarly Infrastructure (POSI) designed to promote the durability of research publishers. Some of these principles are directly applicable to (and applied by) shadow libraries, whereas others are relevant only to legitimate organizations. For instance, the idea that shadow libraries “cannot lobby” would be strongly resisted by those who believe that such archives have a political function. By contrast, the idea that there are formal incentives to “fulfill mission and wind down” is a gray area. This clause of POSI is meant to ensure that organizations do not prioritize their own ongoing existence as a fundamental necessity, if the mission for which they were established has been completed. Yet if we achieved 100 percent OA tomorrow (including access to the back archive of in-copyright scholarship), would shadow libraries continue to exist? This is not clear.

However, in the context of the principle that POSI calls “insurance,” which means that the infrastructure and its data can be reconstructed by a third party, shadow libraries most viably demonstrate adherence to some of the tenets of POSI where they most strongly converge with durability requirements. Library Genesis and Anna’s Archive, for instance, offer torrent archives of all the material that they cache. Boldly, in 2019, several “seedbox” firms stepped up to the challenge of preserving this large and distributed digital archive of books and journal articles (u/shrine, “Charitable Seeding Update”). As of 2022, the entire archive of Library Genesis files and its associated metadata database is available in distributed downloadable form. The source code for the platform is also openly available. That said, this is not universally the case of different shadow libraries. For example, while POSI stipulates that “all software required to run the infrastructure should be available under an open-source license,” Sci-Hub does not currently offer its source code. In a move that seems to go somewhat against the open principle of the site, Elbakyan launched a fundraising drive in 2021 with the promise of open-sourcing the platform, alongside the development of various neural network–based text and data-mining facilities (Maxwell). The slightly problematic logic here resides in demanding payment to open the platform, even while stipulating that it is unethical to pay publishers that do not publish openly. This fundraising drive does, also and ironically, place Sci-Hub’s activities back within POSI’s frame of sustainable infrastructure. The basic fact is that, without a revenue stream, people cannot eat. Sci-Hub’s need for fundraising, to pay for the time spent working on the platform, counts, to some extent, as a sustainability plan.

That said, Sci-Hub’s functionality rests on a certain level of security by obscurity. The universal proxy system that Sci-Hub uses can retrieve any scholarly article given just a DOI. However, this comes with a particular download “signature.” To remain undetected by publishers that seek to block it, Sci-Hub’s background fetching mechanism must log in to publisher websites and appear to mimic a legitimate user request. It should not appear as a rogue agent. Security by obscurity is not a recommended practice in the information security industry. However, in Sci-Hub’s case, keeping the source code secret is one way to protect the signature of its download mechanism.

Other, more radical proposals have been mooted to distribute Library Genesis and Sci-Hub’s content. The most extreme is a move to a system called the InterPlanetary File System (IPFS) protocol (Rahalkar and Gujar). IPFS yields a distributed, addressable system that assigns a hash to every object. This hashing—a one-way system to reduce a message to a unique key—means that looking up an object’s location in the system can be done swiftly. Crucially, as the designers put it, “IPFS has no single point of failure, and nodes do not need to trust each other” (Benet).

Importantly, IPFS has various characteristics of permanence that inhere in its system of pinning. In this structure, objects are deemed immutable (pinned) and pulled down permanently to mirroring clients, thereby ensuring a type of digital preservation between clients. As the designer notes, “this also makes IPFS a Web where links are permanent, and Objects can ensure the survival of others they point to” (Benet). While various other systems have been proposed—energy-inefficient blockchains are the recurrent terrible idea beloved of overly keen technophiles (Wang and Zhao)—IPFS offers the most realistic chance for shadow libraries to implement widely distributed, geographically redundant, and disaster-resilient digital preservation.

IPFS solves many, but not all, problems for shadow libraries. It is not possible, for instance, to participate anonymously, meaning that so long as shadow libraries remain outlawed, it will still be possible to identify a potentially infringing user by Internet Protocol address. It is also unknown whether IPFS can realistically scale to the level that these shadow libraries require (for an explanation of the technical challenges, see Anna). Having to spread 33 terabytes over 2.5 million unique objects is a significant hurdle to clear. However, it will be challenging for formal publishers to shut down the shadow libraries if these difficulties can be overcome. The principles of open scholarly infrastructure would protect these shadow organizations as much as they might protect a legitimate counterpart.

One of the reasons why I believe that we should consider shadow libraries to be part of the infrastructure of contemporary scholarly communications is that they resiliently adhere to a set of formally articulated principles of such infrastructure. While they fall on several criteria—as do their legal counterparts (Bilder)—the undercover architectures of shadow libraries exhibit many of the characteristics of permanence that we expect from scholcomms infrastructure. These heretical infrastructures are also becoming embedded as a silent norm in the academy, with an entire cohort of students learning that the easiest way to access paywalled research is via piracy (Owens). The distribution and technical hardiness of these infrastructures mean that they are here to stay.

Where Next?

Shadow libraries are the symptom of ills in scholcomms. However, there are also an entire set of disagreements around their purpose. While many would argue that Sci-Hub, Library Genesis, and Anna’s Archive are potent motivators to “change the system” of scholarly communications to open access, the founder of the former disagrees. As Elbakyan puts it: “My view is completely different. For me, Sci-Hub has a value by itself, as a website where users can access knowledge. . . . The system has to be changed so that websites like Sci-Hub can work without running into problems. Sci-Hub is a goal, changing the system is one of the methods to achieve it. . . . Sci-Hub is not going to die when research articles will be free” (Elbakyan, “Sci-Hub Is a Goal”).

The digital humanities (DH) disciplines have often entirely sidestepped this problem of poor access to scholarship. By beginning from a principle of OA to new forms of research output—such as interactive digital websites—it can appear as though the challenges that shadow libraries address are irrelevant in this born-open space. This is not strictly true. We know, for instance, that one of the ways in which DH disciplines make themselves commensurate (and comeasurable) with conventional scholarship is by aping the forms of their peers (Eve, “Violins in the Subway”). DH spaces have journal articles and books, and when they adopt these forms, they are not always open by default. Further, DH scholarship has to cite existing work. It is not enough to set one’s new work aside as though it were a totally new field, with no connection to the past. No matter how hard it tries, DH finds itself enmeshed in the issues of access to which shadow libraries are often illegally devoted.

Is the future a world where centralized sites, such as Sci-Hub, amalgamate all research findings and anybody can read such work without paying? Time and court cases will tell whether they will be legal. However, in reality, that future is already here. Whether publishers like it or not, and whether legal or not, we already have a system where almost all newly published knowledge is freely available to read. The question becomes, instead, about the future of publishing labor. If this situation eventually leads to libraries and universities no longer paying for scholarship—and even open research—what is the future of research publishing? Who will perform the professionalized functions of platform development, peer review oversight, typesetting, copyediting, proofreading, digital preservation, and identifier assignment and maintenance? Are we happy to outsource these labor functions to amateur pirates funded by cryptocurrencies? Furthermore, if such outfits were professionalized, would they be so different from being publishers in their own right? It is to these questions—of the logic of extinction or exhaustion of scholarly communications—that shadow libraries apocalyptically gesture.

Notes

1. 1. In this chapter, I do not hyperlink directly to shadow libraries for multiple reasons. The first is that such links are highly unstable, as the libraries change their locations on a frequent basis to avoid detection. The second is that I do not wish to give any impression of support for such libraries.

2. 2. The official webpage of Monoskop notes, “Many of the titles in the bibliographies are linked to electronic versions of publications made available on Monoskop or other free/libre libraries.”

3. 3. Library Genesis includes not just books but scientific articles that it caches for SciHub.

4. 4. For some background on archives, see Derrida; and Caswel.

5. 5. For a critique of Maslow, though, see Tay and Diener.

Bibliography