A test that still passes for the wrong user guards nothing

269of 304 tests failed, as they must, when sent as the wrong userfeatkpr's records for BookStack, exported 28 Sep 2026 · run 26 Sep 2026, 17:42 UTC · commit 0f5164ec

A passing test proves less than it looks #

A test that sends a request and gets the answer it wants tells you the feature works for that user. It says nothing about whether the check that protects it exists. Delete the permission check, and the test still passes.

So each test is sent twice #

featkpr writes each test from the map of a product, per route and per role. Then it sends the same test again as the wrong user: someone who should get the opposite answer. A test that expects to reach a page is sent as a user without the permission, and BookStack must refuse. A test that expects a refusal is sent as the Admin, who must get through. Either way the test must now fail. A test that still passes is not checking the permission.

The figure at the top is the first kind: an Editor creates a book, and a reader without the permission is turned away, so the test fails, as it must.

What a test holds #

Each test carries a small block the tool writes and rewrites: who may send the request, who must be refused, and what success looks like. These lines are from one test featkpr wrote for BookStack’s “convert a book to a shelf”, the second kind. Here the test’s own user lacks the permission and must be refused. Admin sends the same request twice: once as the control, which must get through, and once as the wrong user, when the test must fail.

# --- cache block: the only part the tool rewrites ---
ACTOR_PERMISSIONS = ['book-update-all']
FLIP_ACTOR = "admin"
CONTROL = "admin"
METHOD = 'POST'
ROUTE = '/books/{bookSlug}/convert-to-shelf'
EXPECT = 'refused'
REACHED = (302, '{base}/shelves/[a-z0-9-]+')  # BookStack's success answer: status, Location of a redirect
REFUSED_BY = 'hidden'  # ACTOR cannot view what the route names: BookStack hides it
HIDDEN = 'Book not found'  # errors.book_not_found: ACTOR cannot view what the route names; BookStack hides it

The lines come from a test in featkpr’s sample data; the comments after FLIP_ACTOR and CONTROL are cut, and the figure below spells out what they say.

What happened on BookStack #

So far featkpr has written 481 tests from its map of BookStack. In the run of 26 Sep 2026, 17:42 UTC, 304 ran and all passed. Sent again as the wrong user, 269 of them failed, as they must: 51 were refused to a user without the permission, and 218 let the Admin through. None got through as the wrong user when it should not have. 35 are unchecked: they all expect access, and the wrong-user run signs in only as users our seed made; none of them lacks the permission.

Why this matters for your app #

Broken access control is first in OWASP’s 2025 list of web risks. The usual guard is a person remembering to write the negative test. Sending each test again as the wrong user makes that negative test a rule, and the count tells you how much of your app it covers.

The full numbers, step by step, are in the BookStack report.