Open Technology Fund - F3 - (July 2022)

Friendly Forge Format (F3) Funding
21

Updated version with @aschrijver proposal included. Excellent.

Hi [redacted],

In order to elaborate on the high-level concept note, I chose to answer with concrete examples, facts and details. Please let me know if this answers your open questions. I’d be happy to provide additional information.

I would like to emphasize that the software forge centralization problem F3 addresses, although widely acknowledged, is not funded and only handful of people worldwide are making concrete work to solve it. It is not just relevant to Internet Freedom, but impacts the entire corpus of Free Software. In this regard F3 covers the same breadth of concerns that the reproducible software project does.

Our society, and especially our communication and information sources, depend on complex technology, which is often and increasingly centralized. The power to control them is in the hands of governments and large corporations, that may not have the best interests and needs of the human citizens and users in mind. For underprivileged people living in oppression the need for empowerment is most urgent. Open information access is at the basis of the ability for people to be informed, to reveal injustices and expose censorship. Freedom of expression and other fundamental human rights depend on the control over the technology one uses, and having the ability to participate in its creation. The means for affected people in the region to wield the full range of software development tools is crucial to support important activism work. The F3 specification is an important enabler of that, and part of a larger vision of “Liberating Free Software Development”. Making software be Free is a step toward setting people free.

Cheers

Why this effort is needed in the Internet Freedom space?

The entire software development landscape is overly dominated by just two major players, Github and Gitlab. In particular the position of Github, owned by Microsoft, is problematic when it comes to securing Internet Freedom. Github’s role in the success of open source is often lauded, and partly warranted. For Microsoft / Github providing free access was just a very successful strategy to gain market share and establish network effects. Their centralized platform lies at the heart of a huge ecosystem of software tool vendors that optimized their products to integrate with Github services.

Nowadays literally thousands of projects and millions of software developers are subjected to a strong form of de-facto vendor lock-in. Often without even realizing it. For the Free Software movement this is a threat, as Microsoft does not have their best interest at heart. They follow commercial incentives, and are bound by US regulation. By extension here we find substantial threats to Internet Freedom.

There are numerous examples on how these threats materialize in practice.

  • Geopolitical affiliation: Github as US-based corporation must comply to foreign policy and Trade Control and block people and projects from countries that are at odds with USA from accessing their platform. They apply a broad brush to assure they are in compliance, as this Tweet by Sebastian Slomski demonstrates. Once blocked it is very hard to find recourse or reparation.

  • Corporate, governmental and military influences: As a for-profit US enterprise Microsoft / Github is intent to maximize revenue and profits. Their most lucrative contracts are with partners that are not known to be favorable to the same Internet Freedoms we as humanity crave. How controversial these often secretive and shady deals are is detailed in this Article by The Atlantic.

  • Surveillance capitalism: Like all Big Tech companies Microsoft / Github is a significant player in the widespread harvesting and trade of people’s personal data. Interactions of Internet Freedom activists on the platform are no exception to that, and may provide a wealth of information. Not only do US intelligence agencies likely have backdoors to the platform, but once information enters the Wild West information markets it can end up anywhere. Like in the hands of oppressive regimes.

  • Artificial intelligence: The rise of AI has brought data collection to new heights. Github recently launched CoPilot to help with coding, and in the process ingested all open source project on their platform regardless of their license. Under “fair use” regulation you may find your open source code being regurgitated in proprietary projects. AI systems are also monitoring Terms of Service breaches, making many mistakes in the process. Policy is to err on the side of caution. Microsoft is involved the in the ongoing AI arms race and works on numerous different AI projects, where there’s no telling how they’ll affect our Internet Freedom in the long run. Not having our data available, especially for oppressed people and activists is not more than prudent.

  • Market dominance: Microsoft continues to increase and fortify their dominant position. Known for their Embrace, Extend and Extinguish (EEE) strategies they will not hesitate to bend open ecosystems to their will, thwart open standards, and increasingly monetize the services for those who are captive to their platform. Many vendor lock-in aspects are directly detrimental to the conditions needed to assure Internet Freedom:

    • Unilateral changes to development features, such as deprecating API functionality, occur at rapid pace and are hard to adapt to by open projects that have only limited resources at their disposal.

    • Proprietary nature of large parts of the product portfolio as well as the services offered by 3rd-party vendors hamper reproducible builds. For instance the continuous integration / continuous deployment Security First umbrella tools rely on CircleCI. And the anti-censorship nthLink project depends on Github Actions.

    • Github does not offer a migration path for software projects to move off their platform. There are no open data formats to export to. For example, having an intricate project, Gitea found it impossible to move off of Github and self-host their own software project. After five years the migration effort is still ongoing. Other forge software, like Github and Gitea only provide partial migration from Github for specific use cases.

To a much lesser extent the points listed above also apply to Gitlab. Its positioning is already more directed towards enterprises, and they are limiting free services they offer. Gitlab is a prime candidate for acquisition by another tech giant in the future, triggering a disruption in many open projects now using this code forge.

Stacked against these 2 giant players we find a small number of Free Software projects, like the aforementioned Gitea. Projects that have huge potential. But also ones that are deployed as lonely hard to find self-hosted islands. F3 is instrumental for bridging divides. In addition efforts are underway to make individual code forges part of the decentralized Fediverse, and thus glue them together. The F3 open data exchange format is also part of that effort.

How could this project impact the FOSS community?

Long term it would transform the FOSS online development environment from being centralized and proprietary into being a constellation of federated Free Software forges communicating with each other. It has been twenty years since SourceForge was created, with the same centralization problem as GitHub. F3 is a stepping stone for the FOSS community to reclaim ownership of the tools that they use daily to develop software.

Short term it would allow:

  • A software project to be exported in the F3 format from GitHub and imported into GitLab or Gitea using the same format
  • A developer to file a bug report on GitHub using the F3 format and importing it into GitHub without creating an account on GitHub
  • Mirroring issues from GitHub into GitLab or Gitea to receive notifications without requiring a GitHub account

How would this project provide long-term support to users at risk?

Durable self contained distribution on read-only media

It is common for people living in authoritarian regimes to procure software using physical media like CDs and flash drives. In such cases, only the source code is available since bug tracker history and pull request histories are not available in a downloadable format. F3 will allow developers in authoritarian regimes to setup self-contained, self-sustainable development shops with the full knowledge and experience of the project’s global community. When combined with Git, F3 not only provides a downloadable format but it also supports an efficient synchronization method.

Long term preservation of the software supply chain

Here is an hypothetical use case relevant to human right defenders in need of long term support:

  • In 2022 https://www.nthlink.com/ is used to setup on mobile phones and used by human right defenders in a country that is under an oppressive regime
  • Five years later, in 2027, the mobile phones need to be replaced and the application re-installed, with small modifications because the operating system has changed

With F3, the entire project including the build process that makes nthlink reproducible, has been stored in 2022. It only relies on Free Software that was also stored to make the build process durable. In 2027 they can be re-used to build a new version with small modifications and be re-installed on new phones. The effort is minimal.

Without F3, the build environment provided by GitHub has changed and it is no longer possible to use the deprecated 2022 build process. The nthlink application as it existed in 2022 can no longer be used: it is not supported for newer phones. Upgrading the application would require training the users with the new interface and functionalities. The mobile phones that broke down cannot be easily replaced, a larger effort is required although the 2022 application is still relevant and useful in this particular context. The solution designed in 2022 was made obsolete because the software project could not be archived together with its build process using an Open Standard.

What differences will this project make for developers on a practical level?

Free Software developers will be able to:

Track issues relevant to their software project across software forges and processes (see the 2021 user research report on this topic).

Migrate and mirror software projects from one software forge to another.

Reduce the complexity of implementing software forge migrations. Instead of maintaining a migration process from

  • GitHub to Gitea,
  • GitHub to GitLab,
  • GitHub to GitHub
  • Gitea to GitHub,
  • Gitea to GitLab,
  • Gitea to Gitea
  • GitLab to GitHub,
  • GitLab to Gitea,
  • GitLab to GitLab
  • etc.

With F3 it will only be necessary to maintain a migration process from:

  • GitHub to F3
  • Gitea to F3
  • GitLab to F3
  • F3 to GitHub
  • F3 to GitLab
  • F3 to Gitea

What are your thoughts on the adoption efforts?

Wide adoption of F3 is extremely difficult, long term. But can be done incrementally.

The ultimate adoption of F3 requires:

  • a concise, precise and unambiguous documentation
  • endorsement by a standard body
  • complete and reliable reference implementations in multiple programming languages
  • native integration in all major software forges

The primary adoption blocker is that GitHub is unlikely to support F3 or any other Open Format facilitating software project migration.

An incremental adoption should start by:

  • limiting the scope, with a bottom up approach, using the existing Gitea format
  • providing a reference implementation in Go
  • focusing on practical advantages this reference implementation bring to Free Software developers (i.e. cross forges issue tracking)

Further iterations would expand the scope of the F3 specifications and provide additional practical advantages to drive the change.

Can you detail the community consultation you engaged with that would support this idea/project? What specific communities are in need of this project and have expressed that need?

  • Software forge developers, system administrators and Free Software developers were interviewed as part of the user research conducted in 2021. They expressed the need for communication between software forges. They explained, by providing concrete examples from their personal experience, the practical problems that arise because such communication does not exist.
  • The communities referenced in the State of the Forge Federation: 2021 to 2023 were consulted and reviewed the document which explains F3 in context. They include software forge developers (Gitea), software forge system administrators (Codeberg) and Free Software developers
  • In 2021 the relevance of an interchange format (not yet named F3) in the context of the federation of software forges was explained during the Next Generation Internet webinar on Linked Data
  • In January 2022 the idea matured and was explained as an incremental import/export during a webinar on Forge Federation