-
@ DanConwayDev
2023-04-19 16:12:07In The Bull Case for a GitHub Alternative on Nostr I outlined the opportunities for a thriving ecosystem for the software development collaboration experience over nostr.
I outlined 3 core features that must be enabled:
- Authoritative git repositories, updated exclusively by authorised maintainers.
- Platform to host the surround collaboration. Comments and interactions around PRs, issues, project tracking, etc.
- CI/CD pipeline integration with tools such as CicleCI, travis, Jenkins, etc.
This article outlines some of the key challenges that we should address to achieve this and how feature 2 could be achieved whilst avoiding may of these problems.
The Group Authorisation Problem
A central feature of Nostr is permission-less account creation and authorisation of user activity through public / private key cryptography.
A key challenge that cuts across core features 1 and 2 above is how the authorisation of a state can be trustlessly verified when the list of maintainers evolves over time. From a git repository perspective the state may be changed by merging a PR, squashing commits or removing a branch. From a collaboration feature perspective the state could be changed by updating the status of an issue or moving a story across a kanban board in scenarios where such actions require authorisation.
I will shortly be publishing a deep diver into the problem of group authorisation. It applies to many more usecases. The solution I will propose is Nostr Authorisation Groups.
The Conflicting Change Problem
How is consensus reached on which of two conflicting authorised changes is accepted?
Before we get excited about honey badgers and military generals we should consider the nostrian philosophy that values simplify and pragmatism. Given that maintainers trust each other* what is the simplest solution?
Here is a simple proposal: the nostr Authorisation Event (see below) with the later timestamp takes priority. When the timestamps are identical then the repository reverts to the previous state until a new authorisation event is issued.
*Changes to the list of Authorised maintainers requires a more robust solution as trust can break down, such as when keys are compromised.
The State Reversion Problem
There are scenarios where a state change needs to be reverted. For example, a relay processes an update and then receives either: * a conflicting change with an Authorisation Event that has an identical timestamp or * a delayed Nostr Authorisation Group event that revoked the signers authorisation and was dated before the change was made.
Assuming Rewriting Git History is supported, clients and relays running a standard git server may have purged data required to revert and would need a way of recovering it.
Constructing an Authorisation Event Standard
Authorisation events would need to enable efficient verification that the state reached after any of the following actions will be reflective of the latest authorised state:
- clone a repository
- a repository with the previous authorised state, pulls changes covered by a single Authorisation Event
- a repository with a previously authorised state, pulls changes covered by many Authorisation Events
The bandwidth efficient standard transport model that git uses to push and pull changes are excellent. There is value in preserving this.
Signing and verification would happen with by either a wrapper around the standard git client or a custom git client. This would analyse changes pulled from the git server and only allow changes verified by an Authorisation Event. Similarly it would sign Authorisation Event that reflects the changes which are being pushed to the git server. The Git Server enforces the Authorisation Event Standard and will only process the request once the Authorisation Event is received.
There is a trade-off between lightweight messages and providing useful metadata that could be used for, as an example, state reversion.
Constructing a Standard for Anchoring the Social Experience
Clients focusing on the social experience of software development would need to achor messages to elements both inside a git repository (lines of code, commits, etc) and outside the git repository (PRs, issues, kamban boards, etc).
Either a soft-standard or a NIP should be constructed to establish a referencing schema for a base set of elements.
The Data Storage Problem
There are no shortage of ideas about how storing and serving the underlying data in git repositories could be achieved. From NIP-81/94 filesharing to niche relays running a git server. From torrents to trusted centralised repositories.
This is good news as storage and bandwidth is expensive and the key challenge is finding a model that incentivises actors to provide this service whilst balancing other drivers such as: 1. Resiliency / censorship resistant 2. Reliability 3. Simplicity 4. Efficiency 5. UX impact
The Interoprablilty Problem
It would be beneficial for a number of these data storage models to be built and the market could decide which is best.
Some solutions may have relays run a standard git server that only accept changes that are authorised through nostr events. Clients would issue these events before pushing to the server. Others may store the entire git metadata, such as git refs and object trees, in nostr events and download files via torrents. They could all abstracted this behind a custom git client.
These implementations could be incompatable which would be bad for the wider git-on-noster project as it would lead to fragmentation.
Agreeing an NIP for an Authorisation Event Standard as described above would prevent that. Their could even be bridges built to enable repositories to operate using multiple data storage models simultaneously.
CI/CD Integration
CI/CD tools enable secrets to be stored and accessed by CI/CD runners. nostr keys can be stored in this manner to enable jobs to be run with the outputs posted on nostr.
Repositories that require tests to pass before a PR is merged could add branch protection. Clients would only accept a change as authorised if it had a Authorisation event signed by a maintainer and an event signed by the CI/CD user that both referenced the Authorisation event and confirmed the tests passed.
Work would be needed to remove the friction in setting up integration for project using a verify of CI/CDs.
Patch vs Pull Requests
The most commonly used model for integrating changes to a code base is the Pull Request model. A contributor forks a repository, makes some changes and creates a Pull Request. There is discussion, review and changes are made to the folk. When a maintainer is happy they pull the proposed changes into the main repository.
The alternative is the Patch model. Contributors pull the latest changes from an authoritative repository. They make some changes and package them into a patch, which is sent to maintainers, often via email mailing lists. Discussions and reviews take place, revised patches are issued and when a maintainer is happy they apply the path to the authoritative repository.
A Simpler Model with Decentralisation Trade-Offs
Most of the challenges discussed so far relate to how a git repository can be trustlessly verified as being in a state that is authorised by an evolving group of maintainers so that the reliance on a centralised repository is removed. This isn't actually a requirement to enable the collaboration surrounding a git repository to move onto nostr.
An interim, or even an alternative, approach could be to establish a standard for anchoring the social experience discussed above but avoid or minimise any reliance on an Authorisation Event Standard.
The anchoring standard could aim to make it easy for clients to create a similar user experience of interacting with a Pull Request and a change request submitted using a patch. This would enable projects to support both and bolster the censorship resistant qualities of without degrading the UX. A bot could even run to take a patch, create a fork on the centralised repository and create a PR so that other users can interact with all proposed changes in the same way.
Users who are either excluded from access to the centralised repository, or choose to minimise interaction with it, could use the patch model. Whereas other users would be free to use the PR model which they are more comfortable with.
jb55 produced a tool to send patches with blobs embedded over nostr. It may be possible to recreate a repository entirely from signed patches and avoid interaction with the centralised repository completely so long as branch maintenance and Rewriting Git History is never done. A git client warpper could be done to simulate cloning a repository by downloading every patch. This would be a bandwidth and storage inefficient which might not scale well. jb55 who created the wrapper doesn't "think there should only be patches on nostr".
Ultimately the patch only model feels like a quick and dirty solution that would enable progress on the social experience which would build moment for a decentralised git repository solution to emerge.
Concluding Thoughts
We have covered a lot of ground here yet we barely mentioned incentives or spoken about barriers to adoption. These topics are important but there is perhaps not room in this article to do them justice.
We have discussed a set of problems and challenges with decentralising repositories and software development collaboration usecase over nostr that solutions will need to either address or avoid. We have set out some of the groundwork for highlighting the standards which would need to be agreed in order to enable interoprable innovation across the usecase.
I would really value your feedback and pushback on any and all aspects of this. Specifically it appears that the community has been focusing on the patch model. I wonder the extent to which this is for ease of implementation or if its not really a priority to have a decentralised git repository enabled by trustless state verification. I also haven't seen many of the above challenges clearly articulated in relation to a Github Alternative on Nostr. Perhaps I am missing obvious reasons why these are not problems, or the solutions are so simple that they haven't been articulated, or I am not looking in the right places.
Reach out to me on nostr and let me know! DanConwayDev npub15qydau2hjma6ngxkl2cyar74wzyjshvl65za5k5rl69264ar2exs5cyejr DanConwayDev@Nostr-Check.com