Skip to content

Remove 'presentation' from version string - #179

Open
stweil wants to merge 1 commit into
konstin:mainfrom
stweil:main
Open

Remove 'presentation' from version string#179
stweil wants to merge 1 commit into
konstin:mainfrom
stweil:main

Conversation

@stweil

@stweil stweil commented May 28, 2026

Copy link
Copy Markdown

Otherwise it is misinterpreted as a pre-release.

Otherwise it is misinterpreted as a pre-release.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
@stweil

stweil commented May 28, 2026

Copy link
Copy Markdown
Author

This should fix the handling of https://www.wikidata.org/wiki/Q47007505.

@konstin

konstin commented May 28, 2026

Copy link
Copy Markdown
Owner

Can you say more about the specific example and how it fails?

@stweil

stweil commented May 28, 2026

Copy link
Copy Markdown
Author

It fails to accept valid releases:

% uv run main.py --filter kitodo.presentation
2026-05-28 21:51:29,851 INFO     Starting
2026-05-28 21:51:29,862 INFO     Logging in
2026-05-28 21:51:30,531 INFO     Logged in as Stefan_Weil@test
2026-05-28 21:51:30,638 INFO     Fetching allow and deny lists
2026-05-28 21:51:30,639 INFO     Fetching page text for User:Github-wiki-bot/Exceptions
2026-05-28 21:51:30,814 INFO     Fetching page text for User:Github-wiki-bot/Whitelist
2026-05-28 21:51:30,992 INFO     Querying Projects
2026-05-28 21:51:31,297 INFO     SPARQL query found 18221 projects
2026-05-28 21:51:31,382 INFO     1 projects remain after filtering (invalid_repo: 650, repo_filter: 17570, blacklist: 0, duplicates: 0, inconsistent: 0)
2026-05-28 21:51:31,384 INFO     Found 1 projects
2026-05-28 21:51:31,384 INFO     Querying versions
2026-05-28 21:51:31,384 INFO     Querying wikidata for project versions
2026-05-28 21:51:31,741 INFO     Found 19069 best versions for 13939 projects
2026-05-28 21:51:31,742 INFO     Processing projects
2026-05-28 21:51:31,742 INFO     ## [0/1] Kitodo.Presentation: http://www.wikidata.org/entity/Q47007505 https://github.com/kitodo/kitodo-presentation
2026-05-28 21:51:32,242 INFO     Fetched: https://api.github.com/repos/kitodo/kitodo-presentation/releases?per_page=1
2026-05-28 21:51:32,244 INFO     No fast path, wikidata: 6.0.0, releases: 7.0.1
2026-05-28 21:51:32,249 INFO     Revalidating: https://api.github.com/repos/kitodo/kitodo-presentation
2026-05-28 21:51:32,595 INFO     Not modified: https://api.github.com/repos/kitodo/kitodo-presentation
2026-05-28 21:51:32,596 INFO     Revalidated, not modified: https://api.github.com/repos/kitodo/kitodo-presentation
2026-05-28 21:51:32,599 INFO     Revalidating: https://api.github.com/repos/kitodo/kitodo-presentation/releases?page=1&per_page=100
2026-05-28 21:51:33,469 INFO     Not modified: https://api.github.com/repos/kitodo/kitodo-presentation/releases?page=1&per_page=100
2026-05-28 21:51:33,473 INFO     41 invalid releases: ('v7.0.1', 'Kitodo.Presentation 7.0.1'), ('v7.0.0', 'Kitodo.Presentation 7.0.0'), ('v6.0.0', 'Kitodo.Presentation 6.0.0'), ('v5.1.0', 'Kitodo.Presentation 5.1.0'), ('v5.0.10', 'Kitodo.Presentation...
2026-05-28 21:51:33,474 INFO     Fetching Q47007505
2026-05-28 21:51:33,780 INFO     No stable releases
2026-05-28 21:51:33,780 INFO     Kitodo.Presentation took 2.036s
2026-05-28 21:51:33,780 INFO     # Finished successfully
2026-05-28 21:51:33,783 INFO     Made 8 wikidata requests

With increased logging, the reason for the failing releases is shown:

2026-05-28 21:52:40,441 DEBUG    Conflicting versions ('stable', '7.0.1') and ('unstable', '7.0.1') for tag v7.0.1 and name Kitodo.Presentation 7.0.1 in kitodo-presentation
2026-05-28 21:52:40,441 DEBUG    Conflicting versions ('stable', '7.0.0') and ('unstable', '7.0.0') for tag v7.0.0 and name Kitodo.Presentation 7.0.0 in kitodo-presentation
2026-05-28 21:52:40,442 DEBUG    Conflicting versions ('stable', '6.0.0') and ('unstable', '6.0.0') for tag v6.0.0 and name Kitodo.Presentation 6.0.0 in kitodo-presentation
2026-05-28 21:52:40,442 DEBUG    Conflicting versions ('stable', '5.1.0') and ('unstable', '5.1.0') for tag v5.1.0 and name Kitodo.Presentation 5.1.0 in kitodo-presentation
2026-05-28 21:52:40,442 DEBUG    Conflicting versions ('stable', '5.0.10') and ('unstable', '5.0.10') for tag v5.0.10 and name Kitodo.Presentation 5.0.10 in kitodo-presentation
2026-05-28 21:52:40,442 DEBUG    Conflicting versions ('stable', '5.0.9') and ('unstable', '5.0.9') for tag v5.0.9 and name Kitodo.Presentation 5.0.9 in kitodo-presentation
2026-05-28 21:52:40,443 DEBUG    Conflicting versions ('stable', '5.0.8') and ('unstable', '5.0.8') for tag v5.0.8 and name Kitodo.Presentation 5.0.8 in kitodo-presentation
2026-05-28 21:52:40,443 DEBUG    Conflicting versions ('stable', '5.0.7') and ('unstable', '5.0.7') for tag v5.0.7 and name Kitodo.Presentation 5.0.7 in kitodo-presentation
2026-05-28 21:52:40,443 DEBUG    Conflicting versions ('stable', '5.0.6') and ('unstable', '5.0.6') for tag v5.0.6 and name Kitodo.Presentation 5.0.6 in kitodo-presentation
2026-05-28 21:52:40,443 DEBUG    Conflicting versions ('stable', '5.0.5') and ('unstable', '5.0.5') for tag v5.0.5 and name Kitodo.Presentation 5.0.5 in kitodo-presentation
2026-05-28 21:52:40,443 DEBUG    Conflicting versions ('stable', '5.0.4') and ('unstable', '5.0.4') for tag v5.0.4 and name Kitodo.Presentation 5.0.4 in kitodo-presentation
2026-05-28 21:52:40,444 DEBUG    Conflicting versions ('stable', '5.0.3') and ('unstable', '5.0.3') for tag v5.0.3 and name Kitodo.Presentation 5.0.3 in kitodo-presentation
2026-05-28 21:52:40,444 DEBUG    Conflicting versions ('stable', '5.0.2') and ('unstable', '5.0.2') for tag v5.0.2 and name Kitodo.Presentation 5.0.2 in kitodo-presentation
2026-05-28 21:52:40,444 DEBUG    Conflicting versions ('stable', '5.0.1') and ('unstable', '5.0.1') for tag v5.0.1 and name Kitodo.Presentation 5.0.1 in kitodo-presentation
2026-05-28 21:52:40,444 DEBUG    Conflicting versions ('stable', '5.0.0') and ('unstable', '5.0.0') for tag v5.0.0 and name Kitodo.Presentation 5.0.0 in kitodo-presentation
2026-05-28 21:52:40,444 DEBUG    Conflicting versions ('stable', '4.1.0') and ('unstable', '4.1.0') for tag v4.1.0 and name Kitodo.Presentation 4.1.0 in kitodo-presentation
2026-05-28 21:52:40,444 DEBUG    Conflicting versions ('stable', '4.0.1') and ('unstable', '4.0.1') for tag v4.0.1 and name Kitodo.Presentation 4.0.1 in kitodo-presentation
2026-05-28 21:52:40,444 DEBUG    Conflicting versions ('stable', '4.0.0') and ('unstable', '4.0.0') for tag v4.0.0 and name Kitodo.Presentation 4.0.0 in kitodo-presentation
2026-05-28 21:52:40,444 DEBUG    Conflicting versions ('stable', '3.3.4') and ('unstable', '3.3.4') for tag v3.3.4 and name Kitodo.Presentation 3.3.4 in kitodo-presentation
2026-05-28 21:52:40,445 DEBUG    Conflicting versions ('stable', '3.2.3') and ('unstable', '3.2.3') for tag v3.2.3 and name Kitodo.Presentation 3.2.3 in kitodo-presentation
2026-05-28 21:52:40,445 DEBUG    Conflicting versions ('stable', '2.3.2') and ('unstable', '2.3.2') for tag v2.3.2 and name Kitodo.Presentation 2.3.2 in kitodo-presentation
2026-05-28 21:52:40,445 DEBUG    Conflicting versions ('stable', '3.3.3') and ('unstable', '3.3.3') for tag v3.3.3 and name Kitodo.Presentation 3.3.3 in kitodo-presentation
2026-05-28 21:52:40,445 DEBUG    Conflicting versions ('stable', '3.3.2') and ('unstable', '3.3.2') for tag v3.3.2 and name Kitodo.Presentation 3.3.2 in kitodo-presentation
2026-05-28 21:52:40,445 DEBUG    Conflicting versions ('stable', '3.3.1') and ('unstable', '3.3.1') for tag v3.3.1 and name Kitodo.Presentation 3.3.1 in kitodo-presentation
2026-05-28 21:52:40,445 DEBUG    Conflicting versions ('stable', '3.3.0') and ('unstable', '3.3.0') for tag v3.3.0 and name Kitodo.Presentation 3.3.0 in kitodo-presentation
2026-05-28 21:52:40,445 DEBUG    Conflicting versions ('stable', '3.2.2') and ('unstable', '3.2.2') for tag v3.2.2 and name Kitodo.Presentation 3.2.2 in kitodo-presentation
2026-05-28 21:52:40,445 DEBUG    Conflicting versions ('stable', '3.2.1') and ('unstable', '3.2.1') for tag v3.2.1 and name Kitodo.Presentation 3.2.1 in kitodo-presentation
2026-05-28 21:52:40,445 DEBUG    Conflicting versions ('stable', '3.2.0') and ('unstable', '3.2.0') for tag v3.2.0 and name Kitodo.Presentation 3.2.0 in kitodo-presentation
2026-05-28 21:52:40,446 DEBUG    Conflicting versions ('stable', '2.3.1') and ('unstable', '2.3.1') for tag v2.3.1 and name Kitodo.Presentation 2.3.1 in kitodo-presentation
2026-05-28 21:52:40,446 DEBUG    Conflicting versions ('stable', '3.1.2') and ('unstable', '3.1.2') for tag v3.1.2 and name Kitodo.Presentation 3.1.2 in kitodo-presentation
2026-05-28 21:52:40,446 DEBUG    Conflicting versions ('stable', '2.3.0') and ('unstable', '2.3.0') for tag v2.3.0 and name Kitodo.Presentation 2.3.0 in kitodo-presentation
2026-05-28 21:52:40,446 DEBUG    Conflicting versions ('stable', '3.1.1') and ('unstable', '3.1.1') for tag v3.1.1 and name Kitodo.Presentation 3.1.1 in kitodo-presentation
2026-05-28 21:52:40,446 DEBUG    Conflicting versions ('stable', '3.1.0') and ('unstable', '3.1.0') for tag v3.1.0 and name Kitodo.Presentation 3.1.0 in kitodo-presentation
2026-05-28 21:52:40,446 DEBUG    Conflicting versions ('stable', '3.0.1') and ('unstable', '3.0.1') for tag v3.0.1 and name Kitodo.Presentation 3.0.1 in kitodo-presentation
2026-05-28 21:52:40,446 DEBUG    Conflicting versions ('stable', '3.0.0') and ('unstable', '3.0.0') for tag v3.0.0 and name Kitodo.Presentation 3.0.0 in kitodo-presentation
2026-05-28 21:52:40,446 DEBUG    Conflicting versions ('stable', '2.2.0') and ('unstable', '2.2.0') for tag v2.2.0 and name Kitodo.Presentation 2.2.0 in kitodo-presentation
2026-05-28 21:52:40,446 DEBUG    Conflicting versions ('stable', '2.1.0') and ('unstable', '2.1.0') for tag v2.1.0 and name Kitodo.Presentation 2.1.0 in kitodo-presentation
2026-05-28 21:52:40,446 DEBUG    Conflicting versions ('stable', '2.0.0') and ('unstable', '2.0.0') for tag v2.0.0 and name Kitodo.Presentation 2.0.0 in kitodo-presentation
2026-05-28 21:52:40,447 DEBUG    Conflicting versions ('stable', '1.3.0') and ('unstable', '1.3.0') for tag v1.3.0 and name Goobi.Presentation CE 1.3.0 in kitodo-presentation
2026-05-28 21:52:40,447 DEBUG    Conflicting versions ('stable', '1.2.1') and ('unstable', '1.2.1') for tag v1.2.1 and name Goobi.Presentation CE 1.2.1 in kitodo-presentation
2026-05-28 21:52:40,447 DEBUG    Conflicting versions ('stable', '1.2.0') and ('unstable', '1.2.0') for tag v1.2.0 and name Goobi.Presentation CE 1.2.0 in kitodo-presentation
2026-05-28 21:52:40,447 INFO     41 invalid releases: ('v7.0.1', 'Kitodo.Presentation 7.0.1'), ('v7.0.0', 'Kitodo.Presentation 7.0.0'), ('v6.0.0', 'Kitodo.Presentation 6.0.0'), ('v5.1.0', 'Kitodo.Presentation 5.1.0'), ('v5.0.10', 'Kitodo.Presentation...

The failing releases have names which start with "Kitodo.Presentation". This string includes "pre" which is interpreted as a pre-release, therefore 'unstable', although all these releases are normal stable releases.

@stweil

stweil commented May 28, 2026

Copy link
Copy Markdown
Author

@konstin, even with my patch, the script still fails, but with a different error ("Session lost (assertbotfailed)"):

2026-05-28 22:08:22,279 INFO     1 projects remain after filtering (invalid_repo: 650, repo_filter: 17570, blacklist: 0, duplicates: 0, inconsistent: 0)
2026-05-28 22:08:22,286 INFO     Found 1 projects
2026-05-28 22:08:22,287 INFO     Querying versions
2026-05-28 22:08:22,287 INFO     Querying wikidata for project versions
2026-05-28 22:08:22,457 INFO     Found 19069 best versions for 13939 projects
2026-05-28 22:08:22,458 INFO     Processing projects
2026-05-28 22:08:22,458 INFO     ## [0/1] Kitodo.Presentation: http://www.wikidata.org/entity/Q47007505 https://github.com/kitodo/kitodo-presentation
2026-05-28 22:08:22,910 INFO     Fetched: https://api.github.com/repos/kitodo/kitodo-presentation/releases?per_page=1
2026-05-28 22:08:22,912 INFO     No fast path, wikidata: 6.0.0, releases: 7.0.1
2026-05-28 22:08:22,916 INFO     Revalidating: https://api.github.com/repos/kitodo/kitodo-presentation
2026-05-28 22:08:23,372 INFO     Not modified: https://api.github.com/repos/kitodo/kitodo-presentation
2026-05-28 22:08:23,373 INFO     Revalidated, not modified: https://api.github.com/repos/kitodo/kitodo-presentation
2026-05-28 22:08:23,377 INFO     Revalidating: https://api.github.com/repos/kitodo/kitodo-presentation/releases?page=1&per_page=100
2026-05-28 22:08:24,270 INFO     Not modified: https://api.github.com/repos/kitodo/kitodo-presentation/releases?page=1&per_page=100
2026-05-28 22:08:24,272 DEBUG    Conflicting versions ('beta', '3.0.0-beta.1') and ('beta', '3.0.0-Beta.1') for tag v3.0.0-beta.1 and name Kitodo.Presentation 3.0.0-Beta.1 in kitodo-presentation
2026-05-28 22:08:24,273 INFO     1 invalid releases: ('v3.0.0-beta.1', 'Kitodo.Presentation 3.0.0-Beta.1')
2026-05-28 22:08:24,273 INFO     Fetching Q47007505
2026-05-28 22:08:24,577 INFO     There are 41 stable releases
2026-05-28 22:08:24,578 INFO     Latest version github 7.0.1, existing preferred ranks: 6.0.0
2026-05-28 22:08:24,578 INFO     Creating 2.3.0
2026-05-28 22:08:24,578 INFO     Fetching CSRF token
2026-05-28 22:08:24,946 INFO     Session lost (assertbotfailed), re-logging in
2026-05-28 22:08:24,946 INFO     Logging in
2026-05-28 22:08:25,317 ERROR    Failed to update (attempt 1/3), retrying after 3s: Login failed: {'login': {'result': 'Aborted', 'reason': 'Cannot log in when using MediaWiki\\Session\\BotPasswordSessionProvider sessions.'}}
2026-05-28 22:08:28,319 INFO     Fetching Q47007505
2026-05-28 22:08:28,514 INFO     There are 41 stable releases
2026-05-28 22:08:28,515 INFO     Latest version github 7.0.1, existing preferred ranks: 6.0.0
2026-05-28 22:08:28,515 INFO     Creating 2.3.0
2026-05-28 22:08:28,664 INFO     Session lost (assertbotfailed), re-logging in
2026-05-28 22:08:28,665 INFO     Logging in
2026-05-28 22:08:29,034 ERROR    Failed to update (attempt 2/3), retrying after 4s: Login failed: {'login': {'result': 'Aborted', 'reason': 'Cannot log in when using MediaWiki\\Session\\BotPasswordSessionProvider sessions.'}}
2026-05-28 22:08:33,036 INFO     Fetching Q47007505
2026-05-28 22:08:33,331 INFO     There are 41 stable releases
2026-05-28 22:08:33,332 INFO     Latest version github 7.0.1, existing preferred ranks: 6.0.0
2026-05-28 22:08:33,332 INFO     Creating 2.3.0
2026-05-28 22:08:33,479 INFO     Session lost (assertbotfailed), re-logging in
2026-05-28 22:08:33,480 INFO     Logging in
2026-05-28 22:08:33,856 ERROR    Failed to update: Login failed: {'login': {'result': 'Aborted', 'reason': 'Cannot log in when using MediaWiki\\Session\\BotPasswordSessionProvider sessions.'}}
2026-05-28 22:08:33,858 ERROR    Failed to update: Login failed: {'login': {'result': 'Aborted', 'reason': 'Cannot log in when using MediaWiki\\Session\\BotPasswordSessionProvider sessions.'}}
2026-05-28 22:08:33,858 INFO     # Finished successfully
2026-05-28 22:08:33,860 INFO     Made 20 wikidata requests

Do you have an idea what could cause this failure?

@stweil

stweil commented May 28, 2026

Copy link
Copy Markdown
Author

Note that this patch fixes 40 of 41 invalid releases, but there remains this "invalid" release:

2026-05-28 22:08:24,272 DEBUG    Conflicting versions ('beta', '3.0.0-beta.1') and ('beta', '3.0.0-Beta.1') for tag v3.0.0-beta.1 and name Kitodo.Presentation 3.0.0-Beta.1 in kitodo-presentation
2026-05-28 22:08:24,273 INFO     1 invalid releases: ('v3.0.0-beta.1', 'Kitodo.Presentation 3.0.0-Beta.1')

Here the conflict is caused by '3.0.0-beta.1' != '3.0.0-Beta.1'. Maybe such minor mismatches should be accepted.

@konstin

konstin commented May 31, 2026

Copy link
Copy Markdown
Owner

To solve this case, we should match with the project name, instead of replacing specific strings, which only handle this project.

@stweil

stweil commented May 31, 2026

Copy link
Copy Markdown
Author

"Presentation" can be part of any project name and included in their release strings, so this PR is not project specific. But of course there might be more strings in release name which include "pre" and which are not pre-releases. A complete run on all projects with debug output enabled would show all cases.

@konstin

konstin commented May 31, 2026

Copy link
Copy Markdown
Owner

I'd like to avoid maintaining a list of specific strings; Can we find a way to fix this misdetection by changing the string processing heuristics instead?

@stweil

stweil commented Jun 1, 2026

Copy link
Copy Markdown
Author

Instead of removing "presentation", we could remove any "pre" which is either followed by a letter or which follows a letter", because in these cases it is likely not an indication of a pre-release.

@stweil

stweil commented Jun 1, 2026

Copy link
Copy Markdown
Author

But do you have an idea why the script reports "Session lost (assertbotfailed), re-logging in" and finally fails?

@konstin

konstin commented Jun 1, 2026

Copy link
Copy Markdown
Owner

But do you have an idea why the script reports "Session lost (assertbotfailed), re-logging in" and finally fails?

The code assumes a bot account to ensure that the bot gets logged in again after timing out, instead of running into API errors for anonymous users, that can be disabled by removing the bot assertion:

api_assert: str | None = "bot"

Instead of removing "presentation", we could remove any "pre" which is either followed by a letter or which follows a letter", because in these cases it is likely not an indication of a pre-release.

Can you explain why e.g. the existing name removal heuristic doesn't work? When changing the heuristics, we need to ensure that we have the broadest heuristic possible (for maintainability, to avoid having to change it again and again for mostly similar problems) and to make reasonable sure that there are no regressions where we break cases that used to work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants