Mirrors, forks and renamed repositories: how we avoid counting a project twice

Open SourceDevn Nie3 replies

Devn NieMember#1

A question that comes up on the Open Source board: "is this a mirror?" and "is this a fork?". We exclude mirrors, and the answer is a little more involved than it looks.

Mirrors

A mirror is a copy of a repository, kept up to date automatically, usually by the same people or by a distribution. It has the same issues, the same pull requests and the same discussions, and its star count is a copy of the original's.

We exclude mirrors, and the number of them in the catalogue is currently zero — because a mirror is not something you discover by searching, it is something you follow. It appears in a search because a README links to it, and at that point you can see that the content is identical.

Forks

A fork is a different thing. A fork is a copy somebody made to change. It may have a different direction, a different maintainer, a different licence, and it may be more active than the original. microsoft/autogen and the frameworks that came out of it are the example: the fork is the interesting one.

So forks are kept, and they are listed as what they are.

Renamed repositories

The interesting case. When a project changes its name or moves to an organisation, the old URL redirects to the new one, and GitHub keeps the original's star count and issue history. The redirect is not a copy; it is the same repository at a new address.

If we treated the old address as a separate project, it would be a second row with the same stars, the same issues and the same README, and it would appear in the ranking as if it were independent growth. That is the failure we are guarding against.

The rule we use is that a project is identified by the repository's identity, not by its name. A repository that has been renamed is the same project, with the same number, and the site's link to it follows the rename.

Why the distinction matters for the ranking

The weekly list measures growth. If a renamed repository counted twice, one project's growth would be counted as two projects' growth, and a transferred repository would look like an organic rise. The ranking is only comparable across weeks if each repository contributes exactly one daily point, and the de-duplication is what makes that true.

3 replies

NicholasMember#2

The renamed-repository case is the one I have seen go wrong in other catalogues, and it is worse than a duplicate row: the old name keeps accruing stars after the move, because people still find it through old links, and both versions get reported.

The rule of "one repository identity, one row" is the right one and it is also the only one that survives a rename.

TerryRegular#3

In reply to Nicholas (#2)

Worth noting the fork case in practice, because "fork" and "successor" are not the same and the distinction matters for the licence.

A fork that changed the licence is a different project legally. The site records the licence per repository, so a fork with a different licence is recorded as a different licence, which is correct, and readers comparing the two get the right answer.

Devn NieAuthorMember#4

In reply to Terry (#3)

Both are right, and the licence-per-repository point is the one that makes the fork case safe to keep: the same codebase under two licences is two entries with two licences, and each is accurate.

The de-duplication we do is about identity, not about code, and the two questions do not get in each other's way.