Follow ZDNET: Add us as a favorite source On Google.
ZDNET Highlights
- Open-source repositories are collapsing under the pressure of 10 trillion downloads annually.
- To deal with this problem, all the major repositories are joining together.
- Although lack of funds is a large part of the problem, other issues also need to be addressed.
The world runs on open-source software. We all know this. But did you know that companies download more than 10 trillion (trillion with a t) open-source code files every year? According to software security provider Sonatype, they do – and the file repository sites that supply that code are being overwhelmed by demand.
As sonatype CTO Brian Fox, who oversees Maven Central Java RegistryTold me earlier this year, Maven is in danger of being overwhelmed by constant downloads. Fox & Company has found that 82% of demand comes from just 1% of IPs. This is because companies are using open-source repositories as if they were content delivery networks (CDNs).
Also: 98% of IT leaders want digital sovereignty: Now SUSE is driving it for companies everywhere
For example, the same company may download the same code hundreds of thousands of times in one day, and even the next day. What should a non-profit, open-source code repository do?
We face supply-chain resilience risks
The people running these are finally collectively saying, “This can’t be a charity forever.” Now, under Linux FoundationA new Sustaining Package Registry Working Group will try to identify solid funding, governance, and security practices to keep the code flowing as download numbers grow.
It all started with a scaling problem. Over the past few years, consumption and publication in public package registries has increased to enormous levels. Those 10 trillion downloads? That’s double Google’s annual search queries, and unlike Google, open-source sites are doing it in a much shorter amount of time.
The problem is: because software builds, continuous integration pipelines, and AI systems operate registries at machine speed rather than human speed, sites can’t keep up. That growth has brought an increase in bot traffic, automated publishing, security reports, and outright abuse, exposing what the working group apparently calls a “stability gap.” In other words, we now face a risk not just to the hosting bill, but to supply-chain resilience.
Also: New rules for AI-assisted code in the Linux kernel: What every developer needs to know
As Fox explained, “Open-source registries are no longer passive distribution points. They are operational and security-critical systems running along the way to nearly every modern software build. If we want the software supply chain to remain resilient, we need to have a serious conversation about how these platforms are funded, governed, and maintained on a global scale. Now is the time to treat registry sustainability as a shared responsibility in the software industry.”
Registry Sites Are More Than Download Mirrors
He is right. Open-source registry sites are no longer simple download mirrors. They are security-critical systems that sit directly in the path of almost every modern software build. If any of the central registries falter, whether due to cost, burnout, or a successful attack, the scope of the explosion will spread beyond open-source communities to banks, hospitals, the cloud, and governments that rarely think about where their code dependencies come from.
Christopher Robinson, CTO and Chief Security Architect Open Source Security Foundation (OpenSSF) said, “Package registries sit at the front lines of software supply chain security and resiliency. As consumption, publishing, and attack activity accelerates, the management behind these systems must evolve as well. This initiative will be an important venue for registry leaders and ecosystem stakeholders to align on practical, community-minded ways to maintain the infrastructure on which modern software depends.”
Also: Microsoft finally opens the DOS 1.0 source – and it’s much more than code
“It is bigger than any single registry,” Fox said. “What started as an operational reality on Maven Central is no longer best understood as a Maven Central story. The same pattern is visible throughout the ecosystem. More machine traffic. More automation. More scanning. Higher expectations around uptime, integrity, provenance and policy enforcement. Higher cost. Higher support burden. Over-reliance on infrastructure that the industry still talks about as if it runs on goodwill and free time.” Spoiler alert: It’s not.
To address this, Sonatype has teamed up with the Linux Foundation and other package registry leaders, including Alpha-Omega, the Eclipse Foundation (OpenVSX), the OpenJS Foundation, OpenSSF, PackageGist, the Python Software Foundation, Ruby Central (RubyGems), and the Rust Foundation (Crates). The idea is to give operators a neutral platform to openly discuss funding, governance and shared operational burden. Once that is dealt with, they will coordinate how to explain those realities to companies and organizations that have long assumed that registries are “free.” no, they’re not. They never were.
As the Linux Foundation points out, “Registries today run primarily on two things: (1) infrastructure donations and credits; and (2) the heroic efforts of small paid teams (themselves funded by donations and grants) and unpaid volunteers who operate and maintain registry services. The bulk of donations and grants come from a small group of donors and are not commensurate with the demands of the registry.”
Repositories need more than cash
The working group is clearly positioned as a space where registry leaders and ecosystem stakeholders can align on “practical, community-minded” ways to maintain that infrastructure, rather than where each operator can improvise their own survival plan in isolation.
While open-source repositories are in desperate need of more cash to keep up with demand, it’s not just about money. Many other requirements need to be addressed. These are:
Also: How AI has suddenly become more useful to open-source developers
- economic stability: Develop funding models that can truly cover infrastructure, operations, maintenance and governance, rather than relying on heroic volunteerism and a few corporate logos.
- Collective Defense: Coordinate security practices and information sharing across registries so they can rapidly detect and respond to threats as attackers automate and scale their activity.
- Governance Efficiency: Create shared policy frameworks and standardized terms that make it politically and legally possible to introduce sustainable funding models without fragmenting communities.
- Ecosystem Education and Transparency: Align messaging and educational content so that developers, companies, and policymakers finally understand what it costs to run these services, and why “unlimited free downloads forever” was never a realistic plan.
Some groups are already addressing these issues, but no one has the policies and people to do it all. By working together, it is hoped that they will develop a framework that all repositories can use without reinventing the wheel.
Also: I tried the new Linux Mint 22.3 – it’s a masterclass in polish and quality-of-life fixes
Supporting open-source repositories has become a mission-critical issue for everyone in the software business. However, until recently it was invisible. We no longer have the luxury of assuming that volunteers will keep the doors of open-source code libraries open. These sites must have our support, otherwise we’ll all be in trouble developing, building, and running the programs our companies need to keep the lights on.
