Forgefriends forge format and a package capable of idempotent uploads

Bonjour,

Gitea migrations can get projects from other Gitea instances or GitLab, GitHub, etc. It cannot however migrate them back to another forge. It would be useful, for instance, to be able to migrate a single issue to another forge and update it as it changes (i.e. mirror it), for the purpose of federation.

Since the Gitea internal file format is being documented to be re-used by other forges, it could also be used to develop a forgefriends package whose purpose would be to download and upload software projects from and to any forge, re-using this file format.

Since there currently is no uploader within Gitea, this package could be bootstrapped to provide only that feature, based on the Gitea internal file format. It could become a dependency of Gitea and gradually integrate more and more of the code currently in charge of migrations.

The alternative would be to propose a new feature within Gitea to implement migrations uploads. However, this is only really useful in the context of federation. The other scenario is to migrate a project from Gitea to another Gitea instance but one could argue that it can be implemented by instructing one instance to pull from the other: there is no need to push. Yet another scenario is to migrate a project from Gitea to a forge that does not know (or want) to pull from other forges (such as GitHub) but one could argue that there is not much incentive in helping projects to move to such forges.

To summarize, I do not see how such a feature (idempotent forge upload) currently fits in Gitea. At least not until it starts implementing federation.

Thoughts?

1 Like

I’ll draft code in this repository to illustrate what it is about.

As for a nice catchy name for the data I suggest “ForgeHop” data format. This refers to “to hop”, to make a quick jump, but also to “hopper”, a freight car or coal silo used in the metal industry.

1 Like

As always I admire your creative imagination. :+1:

1 Like

@schmidt_fu on matrix suggested

  • forge
  • teapot (what you bring the tea in :slight_smile: )

I am new to federation via AP so apologies for any inaccuracies:

AP implementations currently cache foreign objects only temporarily and if there’s any interaction with a foreign object(commenting and liking liking foreign toots), I believe only a HTTP link to the object is stored in the DB.

This is different from how Matrix federation works. All participating servers retain a copy of messages on the chatroom. So even if the host server goes down, the room remains usable with all/most(Bob on server matrix.example.com joins chatroom after 100 days of chatroom creation and has only read/received message from then, then only those messages will be stored on matrix.example.com) history.

Full replication is justified in Matrix’s case, because a user joining a room will remain for prolonged periods of time but in case of forges, a user might simply want to read the contents of an issue. Full replication might inefficient there. How will you accommodate such uses where only a peek at data is required?

1 Like

There is a distinction between being able to do a full replication and actually doing it. If a forge is unable to copy some elements (issues, labels, milestones, assets, tags, commits, …) from another forge, it won’t be able to act on it and this will be problematic for federation. I think it is reasonable to begin with something that is inefficient but fully functional. And worry about optimizing later to minimize the CPU/storage/bandwidth.

Does that make sense?

1 Like

Yes, thank you for clarifying!

Another tentative name:

  • Friendly Forge Format
2 Likes

A very friendly name, product-ready. For export packages you can use the extension .fff then and refer to them as ‘Triple-F exports’.

2 Likes

FriendlyForgeFormat · GitLab here is the group that will host code and specs.

2 Likes