TJ crawler: some people have YEAR-techjournal as repo under their username #2
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Ported from GitLab
matt/website#49(2026-02-17) when GitLab was retired; the original repo is archived at https://gitlab.goober.cloud/matt/website/-/issues/49.Implemented the repo-URL half of this, which the previous attempt missed.
scripts/detect_courses.pyonly tried GitHub Pages URLs (<user>.github.io/<year>-techjournal/).This issue is about the journal being a repo named
<year>-techjournalunder the username, whichthose variants never reach.
expand_variants()now also builds:https://github.com/<user>/<year>-techjournalhttps://github.com/<user>/<year>-tech-journalhttps://github.com/<user>/techjournalhttps://github.com/<user>/tech-journalfor each of the last 5 years. Non-existent repos just 404 and the crawler skips them.
Verified offline (no network, no crawl of the live site) —
expand_variantsnow returns 21variants for a plain username URL, including the new repo forms, with no duplicates; a plain
non-GitHub domain still gets its
/blog//posts//notes/subpaths; a GitHub repo URL still returnsexactly itself. Pushed as
b54e9d2c.Leaving this OPEN: the crawler only ever runs in the 02:00 scheduled job, so the real-world effect
won't be visible until it next runs. If the next run picks up more journals, this can be closed.
(Implemented and tested autonomously by Hermes.)