Open Technology Fund - F3 - (July 2022)


Project Title*

This project name can be changed if a full proposal is requested.

Friendly Forge Format (F3) - an Open Standard for autonomous Free Software

Describe your project in 1-3 sentences.

Most software designed to protect HRD, journalists and whistleblowers is FLOSS. Each binary release is carefully verified by a sophisticated supply chain to spot vulnerabilities and regressions.

When a feature is missing, developers living under an oppressive regime have a strong motivation to modify the source code. But they are missing the information to run the supply chain independently. Their sub-standard release may put at risk the very people they are trying to help.

F3 is a new Open File Format that combines the source code with all the information that enables developers to produce high quality releases.

What problem will your project address?

There is no standard file format to share the content of a software forge (e.g. git repository, issues, etc.), and only some of them provide an undocumented internal format. It is possible to use the forgefed vocabulary to send a message about a particular detail (i.e. an individual commit or an issue) via the ActivityPub protocol. But the receiving forge may not know the context in which this information can be interpreted because it cannot conveniently obtain it. As a whole, a software project is a large dataset that is made of numerous interconnected elements stored in a strongly consistent state. Without a standardized file format, interoperability between forges is very difficult.

The Friendly Forge Format (abbreviated F3) is an Open File Format for storing the information from a forge such as issues, pull/merge requests, milestones, release assets, etc. as well as the associated VCS (Git, Mercurial, etc.).

F3 is designed to exchange the state of a software project between GitHub, GitLab, Gitea, etc. for backup, mirroring or federation.

F3 is essential for a forge to provide key requirements:

  • Portability: the entire state of a software project can be dumped and restored at a later time, on a different development environment.
  • Versatility: when published and updated as a F3 archive, a software project effectively is Open Data on which an unlimited range of applications can rely, even outside of the forge domain
  • Consistency: it provides a common language to use when talking about the forge related domains
  • Trust: cryptographic signatures on each F3 dump guard against malicious or unintentional tampering that could compromise the integrity of a software project

And it unlocks the following use cases:

  • Analytics: data mining the contents of files is more practical than issuing a large number of queries to an API
  • Mirror: issues and all other aspects of a software project can be conveniently mirrored from a forge to another by publishing a F3 archive in a VCS and acting on the changes
  • Archival: storing F3 archives in a VCS makes it easier for them to be preserved in long term archives such as Software Heritage
  • Reporting: forgefed messages can be created from F3 and sent via the ActivityPub protocol

If this project is funded, what form will it take?

Technology Development

Give a brief overview of the activities in this project.

Go package reference implementation

A reference implementation of F3 in Go provides:

  • An API for integration in a forge written in Go
  • Validation of a F3 archive (JSON Schema validation, VCS sanity checks)
  • Import and Export support for Gitea and GitLab
  • Dataset generators and fixtures to verify the conformance with the specifications

Milestone: The Go package is published https://pkg.go.dev/

Python package reference implementation

A reference implementation of F3 in Python provides:

  • An API for integration in a forge written in Python
  • Continuous deployment of the F3 documentation
  • The same features as the Go package reference implementation

Milestone: The Python package is published https://pypi.org/

Specification and documentation

The F3 Specification includes:

  • An introduction
  • JSON Schema with embedded documentation
  • Release notes
  • A normative file hierarchy
  • A glossary of terms and their definition

Milestones:

First release

The first F3 release is a bundle that includes:

  • The specifications and documentation
  • The Go reference implementation
  • The Python reference implementation

They are verified to be consistent and tagged with the same version number.

Milestone: simultaneous publication of F3 version 1.0.0 at:

Integration in the Gitea codebase

The F3 Go reference implementation is used as a replacement of the internal format used for repositories dump and restore features in the Gitea codebase.

Milestones: pull request merged in https://forgefriends.org or https://gitea.io

Are there similar projects that exist already? How is your project different or complementary to those projects?

The State of the Forge Federation: 2021 to 2023 published in June 2022 contains a detailed description of the projects related to F3. It is designed to be a building block that can be reused by all of them to facilitate the implementation of forge federation features.

F3 is different from ForgeFed. ForgeFed is an ActivityPub extension with its own vocabulary and models represented in JSON-LD. F3 is an JSON based Open File Format providing a strongly consistent representation of a software project at a given point in time. Despite these differences, there are overlaps: they both need to define a glossary of terms and explain concepts that are common between forges. This already led to contributions to Forgefed and more are expected in the future.

The forgefriends and ForgeFlux forge federation proxies will include F3 in their upcoming releases. This integration will be a showcase demonstrating how the Go and Python API can be used for integration in other software forges.

Deploying F3 in production is challenging because it does not yet have a reassuring reputation of stability and robustness. When problems are discovered, they will require a level of understanding and an investment in time from system administrators that most service providers consider too costly. The Hostea service provider is committed to advance forge federation and will deploy F3 as soon as it is available. It will likely be the first production instance supporting F3.

How long do you estimate this project will take?

6 months to 1 year

How much funding do you estimate you will need? (In US Dollars)

60,000 USD

Who would benefit from this project?

Human Right Defenders, journalists and whistleblowers who have specific needs to protect the privacy of their communication.

During my time at the Freedom of the Press Foundation and La Maison des Lanceurs d’Alerte (2017 to 2021), I provided custom development and the digital infrastructure they required. This experience allowed me to witness first hand the problem that centralized forges such as GitHub pose to the supply chain of a software protecting the privacy of communications.

When a nation state has the means to coerce an intermediary such as GitHub to provide information about its users, attempting to develop a particular feature designed to evade surveillance in a given regional context is hazardous. The state will have advance warning of this attempt and can work around it.

The software developers who are the most motivated to work on adapting software to address the threat from an oppressive regime are the one living in the country. Thanks to numerous Free Software projects such as Tor or SecureDrop, they do not need to start from scratch. They can get the source code. They can also setup a self-hosted forge such as GitLab or Gitea and dedicate machines to continuous integration that could carefully verify their modifications is not introducing a vulnerability that would expose the very people they are trying to protect.

But here lies the problem: they have no way to conveniently get a copy of the quality assurance process that includes CI (continuous integration), CD (continuous delivery), pull requests, issues, releases. They are missing a large part of what makes the software reliable and resistant to attackers: it stays on the centralized forges. Since a software release is only as good as the tools involved in the quality & assurance process, their work is made significantly more difficult.

With F3 they will be able to get a software project as a whole, including the configuration of the CI and CD process. Every change they make will undergo the same verification than the original software and the outcome will be as robust and reliable. More importantly, they will be able to do that in complete autonomy, without frequent interactions with a centralized forge that could expose them.

Where are your intended users, or audiences located?

Global

What is your name?*

Loïc Dachary

What email address should we use to contact you?*

loic@dachary.org

Why are you, and your team members, the right people to work on this project?

In 2017 and 2018 I worked on SecureDrop and authored hundreds of commits. I spearheaded the internationalization of the codebase and organized a team of translators in cooperation with Localization Lab. During this time I occasionally worked with journalists and whistleblowers, addressing their needs with training or software development. I spearheaded the User Research effort in SecureDrop in cooperation with Open Source Design.

In 2020 I worked for La Maison des Lanceurs d’Alerte and setup their information system from scratch, training and supporting the legal team, the staff as well as the volunteers. In this context I also provided technical support to whistleblowers in France and abroad when there was a suspicion they could be targeted by state actors or powerful adversaries.

Since January 2021 I worked full time with https://forgefriends.org, https://forgefed.org, https://gitea.io and https://www.softwareheritage.org/ to advance forge federation. I made contributions to the forgefed specification and played an active role in reviving the project with a broader community.

The forgefriends project is a proxy designed to enable federation for software forges that do not yet implement it natively. I participated in forgefriends from the start, in January 2021. I authored most of the code and documentation that exist today. I published activity reports on a monthly basis and organized videoconferences to keep the larger community up to date.

I have worked with the Gitea project since late 2021 to natively implement federation with code contributions.

Please upload any supporting documents to your application.

N/A

If this project is for a community gathering, what is your proposed start date?

N/A

My application will be dismissed if it does not fit within OTF’s mission, values, principles statements.

on

I have read and understand OTF’s Terms and Privacy policy.

on

I understand that all intellectual property created with support for this application must be openly licensed.

on

1 Like

Mail sent today to someone who may have advice on how to get funding to help developers working on privacy related software.

Subject: OTF & format de fichiers

Salut,

Voila bientôt 18 mois que je travaille à libérer développeurs et développeuses du joug de GitHub, pour faire court. Évidement cela n’a aucun avenir financier donc je cherche de l’argent pour continuer. Une possibilité est OTF[0] parce que bon, quand la QA de SecureDrop dépend de GitHub, il y a de quoi être nerveux: Microsoft n’a pas brillé par sa résistance à la surveillance ces vingt dernières années.

Si tu as des idées de financement je suis preneur de tout conseil. Et si tu es curieux, l’intégralité de mes recherches de financement est 100% transparent[1], y compris le brouillon de la grant application à OTF[2].

A++

[0] https://www.opentech.fund/
[1] https://forum.forgefriends.org/c/funding/5
[2] Open Technology Fund concept note July 2022

Update: received an answer via mail where the person says they have no idea how to get funding.

To: hello@opentech.fund
Subject: Help: submission to the Open Technology Fund

Hi,

I submitted at concept note at OTF Apply | Internet Freedom Fund today and did not get an email from opentech.fund (I checked the spam folder). The email I used is loic@dachary.org.

Would you be so kind as to let me know if you received the application?

Thanks in advance for your help.

Cheers


Loïc Dachary, Artisan Logiciel Libre

From: app@apply.opentech.fund
Subject: Your application to Open Technology Fund: Friendly Forge Format (F3) - an Open Standard for autonomous Free Software

Dear Loic Dachary,

We appreciate your Friendly Forge Format (F3) - an Open Standard for autonomous Free Software application submission to the Open Technology Fund. We will review and reply to your submission as quickly as possible.

If you have any questions, please submit them here: https://apply.opentech.fund/apply/submissions/[redacted]/#communications

If you have issues accessing the submission system or general inquiries, please email us at hello@opentech.fund.

For more information about our support options, review process, and selection criteria, please visit our website at https://www.example.org/.

We are asking all applicants to please enable two-factor authentication for your account on OTF’s application platform. You can create or change your 2FA account by clicking on your name on the upper right hand corner. This will lead you to your personal profile. You will see the option to update your account security. Here is additional information on how to set up your 2FA in the Applicant’s Guidebook Two Factor Authentication (2FA) - OTF Application Guidebook

Project name: Friendly Forge Format (F3) - an Open Standard for autonomous Free Software
Contact name: Loic Dachary
Contact email: loic@dachary.org

Kind Regards,
The OTF Team


Open Technology Fund
https://www.opentech.fund

Received today:

From: notifications@opentech.discoursemail.com
Subject: Re: Help: submission to the Open Technology Fund

Hi,

Yes, we received your application on July 19, 2022. Thank you for checking-in about this!

Kind regards,
[redacted], Program Specialist

Application is now in review

From: app@apply.opentech.fund
Subject: Your application to Open Technology Fund: Friendly Forge Format (F3) - an Open Standard for autonomous Free Software

Dear Loic Dachary, Your application is now in “OTF Review” status (progressed from “Concept Note Received”). Please submit any questions related to your application here: https://apply.opentech.fund/apply/submissions/13789/#communications Link to your application: https://apply.opentech.fund/apply/submissions/13789/ If you have any questions, please submit them here: https://apply.opentech.fund/apply/submissions/13789/#communications See our guide for more information: General Funding Guidelines - OTF Application Guidebook If you have any issues accessing the submission system or other general inquiries, please email us at hello@opentech.fund Kind Regards, The OTF Team

– Open Technology Fund https://www.opentech.fund

1 Like

Ping asking for updates:

To: hello@opentech.fund
Subject: Re: Help: submission to the Open Technology Fund

Hi,

Thanks for the confirmation. It has been a while now and I would like to double check that I did not miss anything from you. The last mail I received was:

Subject: Your application to Open Technology Fund: Friendly Forge Format (F3) - an Open Standard for autonomous Free Software
From: app@apply.opentech.fund
Date: 19/07/2022

Thanks for your help!

On 22/07/2022 02:01, redacted via we.opentech.fund wrote:

From: we.opentech.fund incoming+ce0ec4435ed9e9f67379f2f5c3a61978@opentech.discoursemail.com

[redacted] redacted https://we.opentech.fund/u/alulling OTF Community Member
July 21

Hi,

Yes, we received your application on July 19, 2022. Thank you for checking-in about this!

Kind regards,
[redacted], Program Specialist


Loïc Dachary, Artisan Logiciel Libre

Received the following answer:

Hi there,

Your application is still under review and we anticipate following up with you soon.

Thank you for your patience.

Kind regards,

Received today.


From: app@apply.opentech.fund
Subject: Your application to Open Technology Fund: Friendly Forge Format (F3) - an Open Standard for autonomous Free Software

Dear Loic Dachary,

Your application has been reviewed and the outcome is: More information requested

We very much appreciate your submission to the Open Technology Fund for consideration. Upon evaluation of your submission, we have decided to solicit more information from you before making a determination. We have been reviewing many projects and appreciate your patience during the process.
At the end of this message, we have provided feedback from our determination for your review. Included are specific questions for you to respond to. Please provide your responses by submitting a comment under the communications tab. Please respond no later than September 28. Early responses are welcome. We very much look forward to the discussion.
Feedback:

OTF’s Reviewers appreciated the applicant’s qualifications and professional background, especially their experience working with Localization Lab.
However, OTF’s Reviewers found this concept note to be very high-level and would benefit from further elaboration on the following:

Why this effort is needed in the Internet Freedom space?
How could this project impact the FOSS community?
How would this project provide long-term support to users at risk?
What differences will this project make for developers on a practical level?
What are your thoughts on the adoption efforts?
Can you detail the community consultation you engaged with that would support this idea/project? What specific communities are in need of this project and have expressed that need?

Thank you again for your submission! Please let us know if you have any questions or concerns.

Read the full determination here: https://apply.opentech.fund/apply/submissions/[redacted]/determination/8921/

Link to your application: https://apply.opentech.fund/apply/submissions/[redacted]/
If you have any questions, please submit them here: https://apply.opentech.fund/apply/submissions/[redacted]/#communications

See our guide for more information: General Funding Guidelines - OTF Application Guidebook

If you have any issues accessing the submission system or other general inquiries, please email us at hello@opentech.fund

Kind Regards,
The OTF Team


Open Technology Fund
https://www.opentech.fund

Replied:


Hi,

Thanks for the time you spent on this application. The problem it addresses, the centralization of software forges and its consequences, dates back twenty years. There is very little funding and even less people working on creating a decentralized network of federated forges. When the vast majority of the Free Software used in the Internet Freedom space is developed and distributed by a single commercial entity (GitHub), there are associated risks. I’ll do my best to articulate what they are and why they matter, although I’m sure you already know some of them. I will also explain why creating a standard format such as F3 is a sound approach to incrementally solve that problem.

Before doing that I’d like to be able to login the portal to read the full determination but I’m unable to do so because the password reset for loic@dachary.org OTF Apply | Reset password does not send a link. I suspect something is wrong because this email was previously used, a few years ago. Be sure that I double checked the spam folder and there is nothing.

Thanks in advance for your help.

Received:


From: [redacted]@opentech.fund
CC: app@apply.opentech.fund
Subject: Re: [otf-team] Re: Your application to Open Technology Fund: Friendly Forge Format (F3) - an Open Standard for autonomous Free Software

Hi Loïc,

Apologies for the technical difficulties! I just reset your account. Would you kindly try the reset password process again and let me know if it works?

[redacted]
Program Specialist
Open Technology Fund (OTF)
[redacted]@opentech.fund
w: opentech.fund | t: @OpenTechFund

I now have access to the dashboard, which contains the past application I submitted back in 2018, and the current one.

image

image

image


I replied:

Hi [redacted],

Thanks for the quick answer: I was able to reset the passowrd, setup 2FA and now have access to the dashboard.

Cheers

The link to the determination part that was referred to in the email asking for more information contains the following, which is an integral part of the email received, no new information. It is copy/pasted.


Determination: More information requested
Determination message

We very much appreciate your submission to the Open Technology Fund for consideration. Upon evaluation of your submission, we have decided to solicit more information from you before making a determination. We have been reviewing many projects and appreciate your patience during the process.

At the end of this message, we have provided feedback from our determination for your review. Included are specific questions for you to respond to. Please provide your responses by submitting a comment under the communications tab. Please respond no later than September 28. Early responses are welcome. We very much look forward to the discussion.

Feedback:

OTF’s Reviewers appreciated the applicant’s qualifications and professional background, especially their experience working with Localization Lab.

However, OTF’s Reviewers found this concept note to be very high-level and would benefit from further elaboration on the following:

Why this effort is needed in the Internet Freedom space?
How could this project impact the FOSS community?
How would this project provide long-term support to users at risk?
What differences will this project make for developers on a practical level?
What are your thoughts on the adoption efforts?
Can you detail the community consultation you engaged with that would support this idea/project? What specific communities are in need of this project and have expressed that need?

Thank you again for your submission! Please let us know if you have any questions or concerns.

Here is a draft of my reply. I chose to explicitly name GitHub as the problem, which it is, to make things more concrete in an attempt to address the concern that the concept note is too “high level”. I think the real issue is not to understand what problem software forge centralization creates. It is probably more to figure out if that’s a problem that is for OTF to solve or not.


Hi [redacted],

In order to elaborate on the high-level concept note, I chose to answer with concrete examples, facts and details. Please let me know if this is what you are looking for. I’d be happy to try another approach if that does not help.

I would like to emphasize that the software forge centralization problem F3 addresses, although widely acknowledged, is not funded and only handful of people worldwide are making concrete work to solve it. It is not unique to Internet Freedom, it impacts the entire corpus of Free Software. In pretty much the same way reproducible software matters for all Free Software and not just those in the Internet Freedom space.

Our society, and especially our communication and information sources, depend on complex technology. Much of this technology is centralized, and the power to control it is often in the hands of governments and corporations, who often don’t have in mind the interests and needs of the human citizens and users. More than anyone, people in oppressed regimes need all the power and access they can get, to participate and collaborate in the global software development community, to create and deploy communication technology that puts the power in the people’s hands, that allows people to be informed, to express themselves, to reveal injustice, to carry out important activism work, to fight for their human rights, to live in freedom, to have control and participation in the technology they use, rather than being controlled, censored and manipulated by it.

Cheers

Why this effort is needed in the Internet Freedom space?

The overwhelming majority of FLOSS software tools currently securing Internet Freedom are distributed and developed on a single, centralized and proprietary software forge (GitHub) controlled by a global commercial company (Microsoft). This high degree of centralization is a threat to the entire FLOSS ecosystem and the Internet Freedom space in particular. The lack of Open Standard and migration paths effectively prevents software projects from migrating to alternative software forges.

How could this project impact the FOSS community?

Long term it would transform the FOSS online development environment from being centralized and proprietary into being a constellation of federated Free Software forges communicating with each other. It has been twenty years since SourceForge was created, with the same centralization problem as GitHub. F3 is a stepping stone for the FOSS community to reclaim ownership of the tools that they use daily to develop software.

Short term it would allow:

  • A software project to be exported in the F3 format from GitHub and imported into GitLab or Gitea using the same format
  • A developer to file a bug report on GitHub using the F3 format and importing it into GitHub without creating an account on GitHub
  • Mirroring issues from GitHub into GitLab or Gitea to receive notifications without requiring a GitHub account

How would this project provide long-term support to users at risk?

Here is an hypothetical use case relevant to human right defenders in need of long term support:

  • In 2022 https://www.nthlink.com/ is used to setup on mobile phones and used by human right defenders in a country that is under an oppressive regime
  • Five years later, in 2027, the mobile phones need to be replaced and the application re-installed, with small modifications because the operating system has changed

With F3, the entire project including the build process that makes nthlink reproducible, has been stored in 2022. It only relies on Free Software that was also stored to make the build process durable. In 2027 they can be re-used to build a new version with small modifications and be re-installed on new phones. The effort is minimal.

Without F3, the build environment provided by GitHub has changed and it is no longer possible to use the deprecated 2022 build process. The nthlink application as it existed in 2022 can no longer be used: it is not supported for newer phones. Upgrading the application would require training the users with the new interface and functionalities. The mobile phones that broke down cannot be easily replaced, a larger effort is required although the 2022 application is still relevant and useful in this particular context. The solution designed in 2022 was made obsolete because the software project could not be archived together with its build process using an Open Standard.

What differences will this project make for developers on a practical level?

Free Software developers will be able to:

Track issues relevant to their software project across software forges and processes (see the 2021 user research report on this topic).

Migrate and mirror software projects from one software forge to another.

Reduce the complexity of implementing software forge migrations. Instead of maintaining a migration process from

  • GitHub to Gitea,
  • GitHub to GitLab,
  • GitHub to GitHub
  • Gitea to GitHub,
  • Gitea to GitLab,
  • Gitea to Gitea
  • GitLab to GitHub,
  • GitLab to Gitea,
  • GitLab to GitLab
  • etc.

With F3 it will only be necessary to maintain a migration process from:

  • GitHub to F3
  • Gitea to F3
  • GitLab to F3
  • F3 to GitHub
  • F3 to GitLab
  • F3 to Gitea

What are your thoughts on the adoption efforts?

Wide adoption of F3 is extremely difficult, long term. But can be done incrementally.

The ultimate adoption of F3 requires:

  • a concise, precise and unambiguous documentation
  • endorsement by a standard body
  • complete and reliable reference implementations in multiple programming languages
  • native integration in all major software forges

The primary adoption blocker is that GitHub is unlikely to support F3 or any other Open Format facilitating software project migration.

An incremental adoption should start by:

  • limiting the scope, with a bottom up approach, using the existing Gitea format
  • providing a reference implementation in Go
  • focusing on practical advantages this reference implementation bring to Free Software developers (i.e. cross forges issue tracking)

Further iterations would expand the scope of the F3 specifications and provide additional practical advantages to drive the change.

Can you detail the community consultation you engaged with that would support this idea/project? What specific communities are in need of this project and have expressed that need?

  • Software forge developers, system administrators and Free Software developers were interviewed as part of the user research conducted in 2021. They expressed the need for communication between software forges. They explained, by providing concrete examples from their personal experience, the practical problems that arise because such communication does not exist.
  • The communities referenced in the State of the Forge Federation: 2021 to 2023 were consulted and reviewed the document which explains F3 in context. They include software forge developers (Gitea), software forge system administrators (Codeberg) and Free Software developers
  • In 2021 the relevance of an interchange format (not yet named F3) in the context of the federation of software forges was explained during the Next Generation Internet webinar on Linked Data
  • In January 2022 the idea matured and was explained as an incremental import/export during a webinar on Forge Federation
2 Likes

Apologies for the delay, had to some issues to deal with IRL.

Great response! When applying mCaptcha, I stated specific use cases where the vulnerable benefit the most.

As F3 gives folks the ability to check out not only the code but also related data like bug tracking history and merge history, it might be worth mentioning it with something like this:

It is common for people living in authoritarian regimes to procure software using physical media like CDs and flash drives. In such cases, only the source code is available since bug tracker history and PR histories are not available in a downloadable format.

F3 will allow developers in authoritarian regiems to setup self-contained, self-sustainable development shops with the full knowledge and experience of the project’s global community.

F3 not only provides a downloadable format but it also supports an efficient synchronisation method using Git, which allows for period synchronisation of global experience and project history using physical media too.

1 Like

(Think I already reviewed an OTF text here before… too lazy to find out now)

Reformulate. It is not a different approach. “Please let me know if this answers your open questions. I’d be happy to provide additional information.”

“It is not just relevant to Internet Freedom, but impacts the entire corpus of Free Software.”

Repetition of points. “In this regard F3 covers the same breadth of concerns that the reproducable software project does”

Reformulating. Breaking sentences differently.

“Our society, and especially our communication and information sources, depend on complex technology, which is often and increasingly centralized. The power to control them is often in the hands of governments and large corporations, that often do not have the best interests and needs of the human citizens and users in mind.”

Way too long sentence. Reformulating.

“For underprivileged people living in oppression the need for empowerment is most urgent. Open information access is at the basis of the ability for people to be informed, to reveal injustices and expose censorship. Freedom of expression and other fundamental human rights depend on the control over the technology one uses, and having the ability to participate in its creation. The means for affected people in the region to wield the full range of software development tools is crucial to support important activism work. The F3 specification is an important enabler of that, and part of a larger vision of “Liberating Free Software Development”. Making software be Free is a step toward setting people free.”

I’m time-constrained, so leave it here for now. When do you intend to send this?

PS. Note the broader context: “forging software”, FSDL as a vision, “liberating free software” as a mission. Even though F3 may not sit on that broad scope now. It can help bring your points across better.

1 Like

Absolutely, you reviewed the initial proposal, this is a followup.

Applied :+1:

Applied :+1:

Applied and added a link :+1:

Applied :+1:

The sooner the better, with a hard deadline imposed by OTF on September 27th.

The cover mail looks much better now :sparkles: The rest of the answer is the most difficult part: it took me hours to figure out how to answer sensibly. The key is to not repeat what is in the original application (that can be found at the beginning of this topic): the reviewer already has that.

I immensely appreciate your review and will wait until the last minute hoping you have the time and motivation to devote to it.

A key point, I think, is that the reviewer asked for concrete/practical reasons why it would be meaningful for OTF to fund this work. That’s what guided my answer.

Here is the revised answer with @aschrijver @realaravinth changes.


Hi [redacted],

In order to elaborate on the high-level concept note, I chose to answer with concrete examples, facts and details. Please let me know if this answers your open questions. I’d be happy to provide additional information.

I would like to emphasize that the software forge centralization problem F3 addresses, although widely acknowledged, is not funded and only handful of people worldwide are making concrete work to solve it. It is not just relevant to Internet Freedom, but impacts the entire corpus of Free Software. In this regard F3 covers the same breadth of concerns that the reproducible software project does.

Our society, and especially our communication and information sources, depend on complex technology, which is often and increasingly centralized. The power to control them is in the hands of governments and large corporations, that may not have the best interests and needs of the human citizens and users in mind. For underprivileged people living in oppression the need for empowerment is most urgent. Open information access is at the basis of the ability for people to be informed, to reveal injustices and expose censorship. Freedom of expression and other fundamental human rights depend on the control over the technology one uses, and having the ability to participate in its creation. The means for affected people in the region to wield the full range of software development tools is crucial to support important activism work. The F3 specification is an important enabler of that, and part of a larger vision of “Liberating Free Software Development”. Making software be Free is a step toward setting people free.

Cheers

Why this effort is needed in the Internet Freedom space?

The overwhelming majority of FLOSS software tools currently securing Internet Freedom are distributed and developed on a single, centralized and proprietary software forge (GitHub) controlled by a global commercial company (Microsoft). This high degree of centralization is a threat to the entire FLOSS ecosystem and the Internet Freedom space in particular. The lack of Open Standard and migration paths effectively prevents software projects from migrating to alternative software forges.

How could this project impact the FOSS community?

Long term it would transform the FOSS online development environment from being centralized and proprietary into being a constellation of federated Free Software forges communicating with each other. It has been twenty years since SourceForge was created, with the same centralization problem as GitHub. F3 is a stepping stone for the FOSS community to reclaim ownership of the tools that they use daily to develop software.

Short term it would allow:

  • A software project to be exported in the F3 format from GitHub and imported into GitLab or Gitea using the same format
  • A developer to file a bug report on GitHub using the F3 format and importing it into GitHub without creating an account on GitHub
  • Mirroring issues from GitHub into GitLab or Gitea to receive notifications without requiring a GitHub account

How would this project provide long-term support to users at risk?

Durable self contained distribution on read-only media

It is common for people living in authoritarian regimes to procure software using physical media like CDs and flash drives. In such cases, only the source code is available since bug tracker history and pull request histories are not available in a downloadable format. F3 will allow developers in authoritarian regimes to setup self-contained, self-sustainable development shops with the full knowledge and experience of the project’s global community. When combined with Git, F3 not only provides a downloadable format but it also supports an efficient synchronization method.

Long term preservation of the software supply chain

Here is an hypothetical use case relevant to human right defenders in need of long term support:

  • In 2022 https://www.nthlink.com/ is used to setup on mobile phones and used by human right defenders in a country that is under an oppressive regime
  • Five years later, in 2027, the mobile phones need to be replaced and the application re-installed, with small modifications because the operating system has changed

With F3, the entire project including the build process that makes nthlink reproducible, has been stored in 2022. It only relies on Free Software that was also stored to make the build process durable. In 2027 they can be re-used to build a new version with small modifications and be re-installed on new phones. The effort is minimal.

Without F3, the build environment provided by GitHub has changed and it is no longer possible to use the deprecated 2022 build process. The nthlink application as it existed in 2022 can no longer be used: it is not supported for newer phones. Upgrading the application would require training the users with the new interface and functionalities. The mobile phones that broke down cannot be easily replaced, a larger effort is required although the 2022 application is still relevant and useful in this particular context. The solution designed in 2022 was made obsolete because the software project could not be archived together with its build process using an Open Standard.

What differences will this project make for developers on a practical level?

Free Software developers will be able to:

Track issues relevant to their software project across software forges and processes (see the 2021 user research report on this topic).

Migrate and mirror software projects from one software forge to another.

Reduce the complexity of implementing software forge migrations. Instead of maintaining a migration process from

  • GitHub to Gitea,
  • GitHub to GitLab,
  • GitHub to GitHub
  • Gitea to GitHub,
  • Gitea to GitLab,
  • Gitea to Gitea
  • GitLab to GitHub,
  • GitLab to Gitea,
  • GitLab to GitLab
  • etc.

With F3 it will only be necessary to maintain a migration process from:

  • GitHub to F3
  • Gitea to F3
  • GitLab to F3
  • F3 to GitHub
  • F3 to GitLab
  • F3 to Gitea

What are your thoughts on the adoption efforts?

Wide adoption of F3 is extremely difficult, long term. But can be done incrementally.

The ultimate adoption of F3 requires:

  • a concise, precise and unambiguous documentation
  • endorsement by a standard body
  • complete and reliable reference implementations in multiple programming languages
  • native integration in all major software forges

The primary adoption blocker is that GitHub is unlikely to support F3 or any other Open Format facilitating software project migration.

An incremental adoption should start by:

  • limiting the scope, with a bottom up approach, using the existing Gitea format
  • providing a reference implementation in Go
  • focusing on practical advantages this reference implementation bring to Free Software developers (i.e. cross forges issue tracking)

Further iterations would expand the scope of the F3 specifications and provide additional practical advantages to drive the change.

Can you detail the community consultation you engaged with that would support this idea/project? What specific communities are in need of this project and have expressed that need?

  • Software forge developers, system administrators and Free Software developers were interviewed as part of the user research conducted in 2021. They expressed the need for communication between software forges. They explained, by providing concrete examples from their personal experience, the practical problems that arise because such communication does not exist.
  • The communities referenced in the State of the Forge Federation: 2021 to 2023 were consulted and reviewed the document which explains F3 in context. They include software forge developers (Gitea), software forge system administrators (Codeberg) and Free Software developers
  • In 2021 the relevance of an interchange format (not yet named F3) in the context of the federation of software forges was explained during the Next Generation Internet webinar on Linked Data
  • In January 2022 the idea matured and was explained as an incremental import/export during a webinar on Forge Federation
1 Like

Great example :heart: I added that, with a reformulation of the last paragraph. It helps clarify the benefits :+1:

The entire software development landscape is overly dominated by just two major players, Github and Gitlab. In particular the position of Github, owned by Microsoft, is problematic when it comes to securing Internet Freedom. Github’s role in the success of open source is often lauded, and partly warranted. For Microsoft / Github providing free access was just a very successful strategy to gain market share and establish network effects. Their centralized platform lies at the heart of a huge ecosystem of software tool vendors that optimized their products to integrate with Github services.

Nowadays literally thousands of projects and millions of software developers are subjected to a strong form of de-facto vendor lock-in. Often without even realizing it. For the Free Software movement this is a threat, as Microsoft does not have their best interest at heart. They follow commercial incentives, and are bound by US regulation. By extension here we find substantial threats to Internet Freedom.

There are numerous examples on how these threats materialize in practice.

  • Geopolitical affiliation: Github as US-based corporation must comply to foreign policy and Trade Control and block people and projects from countries that are at odds with USA from accessing their platform. They apply a broad brush to assure they are in compliance, as this Tweet by Sebastian Slomski demonstrates. Once blocked it is very hard to find recourse or reparation.

  • Corporate, governmental and military influences: As a for-profit US enterprise Microsoft / Github is intent to maximize revenue and profits. Their most lucrative contracts are with partners that are not known to be favorable to the same Internet Freedoms we as humanity crave. How controversial these often secretive and shady deals are is detailed in this Article by The Atlantic.

  • Surveillance capitalism: Like all Big Tech companies Microsoft / Github is a significant player in the widespread harvesting and trade of people’s personal data. Interactions of Internet Freedom activists on the platform are no exception to that, and may provide a wealth of information. Not only do US intelligence agencies likely have backdoors to the platform, but once information enters the Wild West information markets it can end up anywhere. Like in the hands of oppressive regimes.

  • Artificial intelligence: The rise of AI has brought data collection to new heights. Github recently launched CoPilot to help with coding, and in the process ingested all open source project on their platform regardless of their license. Under “fair use” regulation you may find your open source code being regurgitated in proprietary projects. AI systems are also monitoring Terms of Service breaches, making many mistakes in the process. Policy is to err on the side of caution. Microsoft is involved the in the ongoing AI arms race and works on numerous different AI projects, where there’s no telling how they’ll affect our Internet Freedom in the long run. Not having our data available, especially for oppressed people and activists is not more than prudent.

  • Market dominance: Microsoft continues to increase and fortify their dominant position. Known for their Embrace, Extend and Extinguish (EEE) strategies they will not hesitate to bend open ecosystems to their will, thwart open standards, and increasingly monetize the services for those who are captive to their platform. Many vendor lock-in aspects are directly detrimental to the conditions needed to assure Internet Freedom:

    • Unilateral changes to development features, such as deprecating API functionality, occur at rapid pace and are hard to adapt to by open projects that have only limited resources at their disposal.

    • Proprietary nature of large parts of the product portfolio as well as the services offered by 3rd-party vendors hamper reproducible builds. For instance the continuous integration / continuous deployment Security First umbrella tools rely on CircleCI. And the anti-censorship nthLink project depends on Github Actions.

    • Github does not offer a migration path for software projects to move off their platform. There are no open data formats to export to. For example, having an intricate project, Gitea found it impossible to move off of Github and self-host their own software project. After five years the migration effort is still ongoing. Other forge software, like Github and Gitea only provide partial migration from Github for specific use cases.

To a much lesser extent the points listed above also apply to Gitlab. Its positioning is already more directed towards enterprises, and they are limiting free services they offer. Gitlab is a prime candidate for acquisition by another tech giant in the future, triggering a disruption in many open projects now using this code forge.

Stacked against these 2 giant players we find a small number of Free Software projects, like the aforementioned Gitea. Projects that have huge potential. But also ones that are deployed as lonely hard to find self-hosted islands. F3 is instrumental for bridging divides. In addition efforts are underway to make individual code forges part of the decentralized Fediverse, and thus glue them together. The F3 open data exchange format is also part of that effort.

1 Like