End to end testing of F3 could be scripted with some simple scenario:
- Create a F3 archive to act as a fixture
- Launch a supported forge (GitLab, Forgejo, Gitea, etc.)
- Round trip upload / download
- F3 => ForgeA
- ForgeA => F3
There already exist functions to create F3 fixtures in memory and the associated round trip upload / download for Forgejo. And there will be more when support for GitLab, etc. is added.
The Forgejo F3 driver that uses the Forgejo internals instead of the API also has similar testing. Native integration in GitLab will presumably duplicate this test logic as well.
To keep the test infrastructure DRY across implementations the F3 reference implementation could provide a CLI to be used for test purposes such as:
f3 test fixture /tmp/mydirto create a F3 fixture in a directory
f3 test round-trip /tmp/mydir ForgeAto import F3 found in /tmp/mydir in ForgeA, extract it and check the result is similar
This CLI could then be exposed by the forge itself, it would only need to be concerned by calling it, not by implementing it:
forgejo f3 test ...
gitlab f3 test ...
The gof3 could then implement a matrix test to verify the reference implementation is backward compatible with a range of forge instances.
Does that sound like a good and simple way to approach end to end testing?