Project Title*
This project name can be changed if a full proposal is requested.
Friendly Forge Format (F3) - an Open Standard for autonomous Free Software
Describe your project in 1-3 sentences.
Most software designed to protect HRD, journalists and whistleblowers is FLOSS. Each binary release is carefully verified by a sophisticated supply chain to spot vulnerabilities and regressions.
When a feature is missing, developers living under an oppressive regime have a strong motivation to modify the source code. But they are missing the information to run the supply chain independently. Their sub-standard release may put at risk the very people they are trying to help.
F3 is a new Open File Format that combines the source code with all the information that enables developers to produce high quality releases.
What problem will your project address?
There is no standard file format to share the content of a software forge (e.g. git repository, issues, etc.), and only some of them provide an undocumented internal format. It is possible to use the forgefed vocabulary to send a message about a particular detail (i.e. an individual commit or an issue) via the ActivityPub protocol. But the receiving forge may not know the context in which this information can be interpreted because it cannot conveniently obtain it. As a whole, a software project is a large dataset that is made of numerous interconnected elements stored in a strongly consistent state. Without a standardized file format, interoperability between forges is very difficult.
The Friendly Forge Format (abbreviated F3) is an Open File Format for storing the information from a forge such as issues, pull/merge requests, milestones, release assets, etc. as well as the associated VCS (Git, Mercurial, etc.).
F3 is designed to exchange the state of a software project between GitHub, GitLab, Gitea, etc. for backup, mirroring or federation.
F3 is essential for a forge to provide key requirements:
- Portability: the entire state of a software project can be dumped and restored at a later time, on a different development environment.
- Versatility: when published and updated as a F3 archive, a software project effectively is Open Data on which an unlimited range of applications can rely, even outside of the forge domain
- Consistency: it provides a common language to use when talking about the forge related domains
- Trust: cryptographic signatures on each F3 dump guard against malicious or unintentional tampering that could compromise the integrity of a software project
And it unlocks the following use cases:
- Analytics: data mining the contents of files is more practical than issuing a large number of queries to an API
- Mirror: issues and all other aspects of a software project can be conveniently mirrored from a forge to another by publishing a F3 archive in a VCS and acting on the changes
- Archival: storing F3 archives in a VCS makes it easier for them to be preserved in long term archives such as Software Heritage
- Reporting: forgefed messages can be created from F3 and sent via the ActivityPub protocol
If this project is funded, what form will it take?
Technology Development
Give a brief overview of the activities in this project.
Go package reference implementation
A reference implementation of F3 in Go provides:
- An API for integration in a forge written in Go
- Validation of a F3 archive (JSON Schema validation, VCS sanity checks)
- Import and Export support for Gitea and GitLab
- Dataset generators and fixtures to verify the conformance with the specifications
Milestone: The Go package is published https://pkg.go.dev/
Python package reference implementation
A reference implementation of F3 in Python provides:
- An API for integration in a forge written in Python
- Continuous deployment of the F3 documentation
- The same features as the Go package reference implementation
Milestone: The Python package is published https://pypi.org/
Specification and documentation
The F3 Specification includes:
- An introduction
- JSON Schema with embedded documentation
- Release notes
- A normative file hierarchy
- A glossary of terms and their definition
Milestones:
- JSON Schema for F3 are published in a dedicated repository
- The documentation is published at https://readthedocs.org/
First release
The first F3 release is a bundle that includes:
- The specifications and documentation
- The Go reference implementation
- The Python reference implementation
They are verified to be consistent and tagged with the same version number.
Milestone: simultaneous publication of F3 version 1.0.0 at:
- https://readthedocs.org/ for the specifications
- https://pypi.org/ for the Python reference implementation
- https://pkg.go.dev/ for the Go reference implementation
Integration in the Gitea codebase
The F3 Go reference implementation is used as a replacement of the internal format used for repositories dump and restore features in the Gitea codebase.
Milestones: pull request merged in https://forgefriends.org or https://gitea.io
Are there similar projects that exist already? How is your project different or complementary to those projects?
The State of the Forge Federation: 2021 to 2023 published in June 2022 contains a detailed description of the projects related to F3. It is designed to be a building block that can be reused by all of them to facilitate the implementation of forge federation features.
F3 is different from ForgeFed. ForgeFed is an ActivityPub extension with its own vocabulary and models represented in JSON-LD. F3 is an JSON based Open File Format providing a strongly consistent representation of a software project at a given point in time. Despite these differences, there are overlaps: they both need to define a glossary of terms and explain concepts that are common between forges. This already led to contributions to Forgefed and more are expected in the future.
The forgefriends and ForgeFlux forge federation proxies will include F3 in their upcoming releases. This integration will be a showcase demonstrating how the Go and Python API can be used for integration in other software forges.
Deploying F3 in production is challenging because it does not yet have a reassuring reputation of stability and robustness. When problems are discovered, they will require a level of understanding and an investment in time from system administrators that most service providers consider too costly. The Hostea service provider is committed to advance forge federation and will deploy F3 as soon as it is available. It will likely be the first production instance supporting F3.
How long do you estimate this project will take?
6 months to 1 year
How much funding do you estimate you will need? (In US Dollars)
60,000 USD
Who would benefit from this project?
Human Right Defenders, journalists and whistleblowers who have specific needs to protect the privacy of their communication.
During my time at the Freedom of the Press Foundation and La Maison des Lanceurs d’Alerte (2017 to 2021), I provided custom development and the digital infrastructure they required. This experience allowed me to witness first hand the problem that centralized forges such as GitHub pose to the supply chain of a software protecting the privacy of communications.
When a nation state has the means to coerce an intermediary such as GitHub to provide information about its users, attempting to develop a particular feature designed to evade surveillance in a given regional context is hazardous. The state will have advance warning of this attempt and can work around it.
The software developers who are the most motivated to work on adapting software to address the threat from an oppressive regime are the one living in the country. Thanks to numerous Free Software projects such as Tor or SecureDrop, they do not need to start from scratch. They can get the source code. They can also setup a self-hosted forge such as GitLab or Gitea and dedicate machines to continuous integration that could carefully verify their modifications is not introducing a vulnerability that would expose the very people they are trying to protect.
But here lies the problem: they have no way to conveniently get a copy of the quality assurance process that includes CI (continuous integration), CD (continuous delivery), pull requests, issues, releases. They are missing a large part of what makes the software reliable and resistant to attackers: it stays on the centralized forges. Since a software release is only as good as the tools involved in the quality & assurance process, their work is made significantly more difficult.
With F3 they will be able to get a software project as a whole, including the configuration of the CI and CD process. Every change they make will undergo the same verification than the original software and the outcome will be as robust and reliable. More importantly, they will be able to do that in complete autonomy, without frequent interactions with a centralized forge that could expose them.
Where are your intended users, or audiences located?
Global
What is your name?*
Loïc Dachary
What email address should we use to contact you?*
Why are you, and your team members, the right people to work on this project?
In 2017 and 2018 I worked on SecureDrop and authored hundreds of commits. I spearheaded the internationalization of the codebase and organized a team of translators in cooperation with Localization Lab. During this time I occasionally worked with journalists and whistleblowers, addressing their needs with training or software development. I spearheaded the User Research effort in SecureDrop in cooperation with Open Source Design.
In 2020 I worked for La Maison des Lanceurs d’Alerte and setup their information system from scratch, training and supporting the legal team, the staff as well as the volunteers. In this context I also provided technical support to whistleblowers in France and abroad when there was a suspicion they could be targeted by state actors or powerful adversaries.
Since January 2021 I worked full time with https://forgefriends.org, https://forgefed.org, https://gitea.io and https://www.softwareheritage.org/ to advance forge federation. I made contributions to the forgefed specification and played an active role in reviving the project with a broader community.
The forgefriends project is a proxy designed to enable federation for software forges that do not yet implement it natively. I participated in forgefriends from the start, in January 2021. I authored most of the code and documentation that exist today. I published activity reports on a monthly basis and organized videoconferences to keep the larger community up to date.
I have worked with the Gitea project since late 2021 to natively implement federation with code contributions.
Please upload any supporting documents to your application.
N/A
If this project is for a community gathering, what is your proposed start date?
N/A
My application will be dismissed if it does not fit within OTF’s mission, values, principles statements.
on
I have read and understand OTF’s Terms and Privacy policy.
on
I understand that all intellectual property created with support for this application must be openly licensed.
on