A test for each case on the map, written from a template, none by hand.
Each case comes from the map: a route, who reaches it, and what they should get. featkpr writes one test per case: Python with pytest, calling the app through Playwright's request API. Each test keeps what it sent and what came back, then is sent again as the wrong user, to prove it checks something. No browser yet.
BookStack development code · commit 045c6a98 · featkpr's records, exported 20 Oct 2026
The test set on BookStack #
- 497 tests written so farfeatkpr's records, exported 20 Oct 2026
- 311 ran on 20 Oct 2026, 03:18 UTC, and all passedthe tests ready then; the rest were written since
- 276 proven by the wrong-user checkwrong-user run 20 Oct 2026, 03:29 UTC
- 35 unchecked: no seeded user lacks their permissionnot sent a second time
In the missed-items check of 26 Sep 2026, a fresh start with no help, 76 of the 86 cases it listed got a runnable test. The report names the other 10, each with its reason.
Each check on a route doubles its cases #
A route that checks one permission has two cases: with it and without it. Two checks make four. Each case gets one test. Drawn on BookStack's revision restore, which checks two permissions.
PUT /books/…/revisions/{revId}/restorechecks page-update and the revision permission
What happens to a test after it is written #
Each test runs on every run. When one fails, featkpr says why, and only rewrites it when the reason is the app's wording or address.
- A casewho, with which permission, gets which outcome
- A templatepicked by the kind of route; no model writes the test
- A test filePython with pytest, calling the app through Playwright's request API
- A runpass or fail per step, with the line that failed
- Why it failedthe route moved, a label changed, or it looks like a bug
- The route moved, or a label changedfeatkpr writes a new test in its place and says so on the run.
- It looks like a bugNothing is rewritten over it. It waits for a person to decide.
The wrong-user check: how a test proves it checks something #
A test that passes proves little on its own: it might pass whoever sends it. So each test is sent a second time as the wrong user, someone who should get the opposite answer. A test that expects to reach a page is sent as a user without the permission, and BookStack must refuse. A test that expects a refusal is sent as the Admin, who must get through. Either way the test must now fail. A test that still passes is not checking anything.
276 of 311 BookStack tests proven this way #
- 216 proven: the Admin got throughThese tests expect BookStack to refuse their own user. Sent as the Admin, each got through, so each failed, as it must.
- 49 proven: refused to a user without the permissionThese tests expect to get in. Sent as a user without the permission, BookStack refused each, so each failed, as it must.
- 7 proven: tests that end on a mailThese tests end on a mail BookStack sends: a password reset, an invite, a comment or page notification. The test copy sends real mail since 28 Sep. Sent as the wrong user, each failed, as it must.
- 4 proven: another kind of answerThese tests end on an answer that is neither a page reached, a refusal nor a mail. Sent as the wrong user, each failed, as it must.
- 35 uncheckedThey all expect access, and the wrong-user run signs in only as users our seed made. None of them lacks the permission, so these were not sent a second time.
- 0 got through as the wrong userNone of the tests sent a second time passed when it should have failed. Each of these would be a missing check.
- 0 failed as their own userAll 311 tests passed first as the user they were written for.
The run, step by step, in the BookStack report
Who the tests sign in as #
BookStack's own roles are Admin, Editor, Viewer and Public. The other four on this page were added by our seed, each holding one set of permissions a case needs.
The wrong-user run signs in only as users of these roles. For 35 tests that expect access, none of them lacks the permission, so those tests stay unchecked until the seed has such a user.
What it tests #
- Can each role reach each route?One test per route and per case the permission checks in the code allow, as a user the case names.
- Is the refusal the permission's?For a test that expects a refusal, the Admin sends the same request as a control and must get through, so the refusal is the permission's and not a broken page.
- Does the app refuse the way it should?A 403, a 404 that hides the thing, or a redirect away from where the allowed user lands. featkpr compares where each answer went, not only its status code, because a refusal and a success can both be a 302.
What it does not test #
- Injection and other input attacksSQL injection, cross-site scripting, file uploads: a scanner or a person.
- Sign-in, sessions and tokensHow users authenticate is not tested; the tests sign in as seeded users.
- Whether a refused request still changed dataToday the check reads the app's answer, not its database. on the plan: what each test changed · decided, no week yet
- Flows through a browserTests call the app directly. on the plan: tests through a browser · decided, no week yet
- The server, the network and their settingsOut of scope, like business rules the map has no case for.
Where it sits beside a penetration test: broken access control is first on OWASP's list of web risks, and a penetration tester checks it by hand, one route and one role at a time. The wrong-user check covers that part, per route and role, for every route with a test, on every run. It is not a full penetration test; the rest still needs a person.
Works today, and not yet #
Works today #
- A test for each caseA test per case, written from a template, none by hand. They call the app directly.497 written so far; 311 ran on 20 Oct 2026, 03:18 UTC · since 26 Sep
- Runs, with evidenceEvery test keeps what it sent and what came back, and a picture of the last page the app answered.since 26 Sep
- The wrong-user checkEach test sent a second time as the wrong user. It must now fail.276 of 311 tests proven on BookStack · since 26 Sep
- A test copy that behaves like real hostingReal mail sent and caught, a real queue, people as personas with a role and a mailbox, and a report on every run of how closely the copy matches real hosting.live 28 Sep: 11 of 17 checks match real hosting on the production-like copy · since 28 Sep
- The missed-items checkfeatkpr runs again from a fresh start with no help and compares each step with an answer key, naming every miss.BookStack: 605 missed of 2,372 · since 26 Sep
Not yet #
- What each test changedRows written, mail sent, files downloaded and calls to other hosts, recorded while each test runs.recorded on BookStack's run of 20 Oct 2026, 03:18 UTC; not shown on the site yet · decided, no week yet
- A wrong user for every testA seeded user without the permission for each test that expects to get in, so none stays unchecked.35 unchecked today · decided, no week yet
- Tests through a browserThe flows a person keeps, run in a browser. Today every test calls the app directly.decided, no week yet
- Tests written in TypeScript tooThe same tests for a TypeScript codebase, beside pytest.decided, no week yet
- Selenium and Cucumber outputsThe same cases written for the test tools you already run.decided, no week yet
from our plan of 1 Oct 2026 · every part on the roadmap
See the tests featkpr wrote for BookStack #
On a 30-minute call we open the tests, their runs and the templates that wrote them, then look at what your app would need.