F3 strategy for end to end testing

Friendly Forge Format (F3)
#1

Bonjour,

End to end testing of F3 could be scripted with some simple scenario:

  • Create a F3 archive to act as a fixture
  • Launch a supported forge (GitLab, Forgejo, Gitea, etc.)
  • Round trip upload / download
    • F3 => ForgeA
    • ForgeA => F3
    • Compare

There already exist functions to create F3 fixtures in memory and the associated round trip upload / download for Forgejo. And there will be more when support for GitLab, etc. is added.

The Forgejo F3 driver that uses the Forgejo internals instead of the API also has similar testing. Native integration in GitLab will presumably duplicate this test logic as well.

To keep the test infrastructure DRY across implementations the F3 reference implementation could provide a CLI to be used for test purposes such as:

  • f3 test fixture /tmp/mydir to create a F3 fixture in a directory
  • f3 test round-trip /tmp/mydir ForgeA to import F3 found in /tmp/mydir in ForgeA, extract it and check the result is similar

This CLI could then be exposed by the forge itself, it would only need to be concerned by calling it, not by implementing it:

  • forgejo f3 test ...
  • gitlab f3 test ...

The gof3 could then implement a matrix test to verify the reference implementation is backward compatible with a range of forge instances.

Does that sound like a good and simple way to approach end to end testing?

#2

Using cli package - github.com/urfave/cli - Go Packages (v1 not v2) is the best way to have an implementation compatible with Forgejo.

1 Like