It’s currently too hard for users and developers to figure out which open-source software is secure and trustworthy.
The most prominent assessment tool I’ve found is OpenSSF Scorecard, which assesses projects for security risks through a series of automated checks, assigning each project a score out of 10.
Unfortunately, it hasn’t proliferated enough yet. Most users have never encountered Scorecard, so it doesn't affect their choices. And if it doesn’t directly affect weekly downloads or clout, most maintainers aren’t incentivised enough to adopt it. Scorecard also isn’t a fully comprehensive security assessment.
Whether it’s Scorecard or something else, I do believe automated assessments have strong potential. At Breaks & Ciphers I’ve been considering some features that could result in a successful movement:
- Fact-checking: a trustworthy assessment can’t currently be achieved solely through static checks or AI (hallucinations, prompt injections). There need to be incentives designed to encourage fact-checking or approval of automated results, similar to X’s Community Notes feature.
- Stages instead of scores: a scoring system may be too daunting for maintainers if their projects start out with very low scores. One alternative I like is a stage-by-stage framework, where projects work their way up through several security and trust model stages over time. I saw the effectiveness of this approach in the L2BEAT initiative, and there is a similar Walletbeat initiative in beta. Perhaps we need a Softwarebeat.
- Inform the user: the security and trust status of the project needs to be shown clearly in GitHub, app stores, and package providers. Otherwise users and developers won’t hear about it, and maintainers won’t care about it. Even if we can’t get GitHub or Google Play onboard at the beginning, we could demonstrate the effectiveness by achieving integration in smaller platforms, such as the Obtainium APK installer.