Coding agents do well on small tasks, and lose quality as the tasks grow

1. Without the map the whole app

   To test one route, an agent reads the whole app and has to keep all of it in mind. BookStack has 312 features in 13 modules. Each dot is one of them.

   Entities 83

   Settings 73

   Exports 34

   Activity 28

   Users 27

   Uploads 22

   Access 16

   Permissions 11

   Api 8

   References 4

   Sorting 4

   Theming 1

   Search 1

   the lit dot: POST /books/{bookSlug}/convert-to-shelf, the route below

2. With the map one route and what it needs

   The map hands the agent a slice: this route, the permissions its code checks, and the users who hold them. The other 311 features stay out.

   the route

   POST /books/{bookSlug}/convert-to-shelfTurn a book into a shelf

   its checks

   • book-create-all
   • book-delete
   • book-update
   • book-view
   • bookshelf-create-all

   its users

   • Editor
   • Content-only reader
   • Reader
   • Admin
   • a role the test makes

3. One route, five permission checks, 32 tests

   featkpr reads the checks from the route's code. A user holds each one or doesn't, so there are 2 × 2 × 2 × 2 × 2 = 32 ways to hold them. featkpr writes one test for each way.

   book-create-allbook-deletebook-updatebook-viewbookshelf-create-all expects run as wrong user

   1. ✓ E C
   2. A
   3. A
   4. A
   5. A
   6. A
   7. A
   8. A
   9. A
   10. A
   11. A
   12. A
   13. A
   14. A
   15. A
   16. A
   17. A
   18. A
   19. A
   20. A
   21. A
   22. A
   23. A
   24. A
   25. A
   26. A
   27. A
   28. A
   29. A
   30. R A
   31. A
   32. C A

   1. 1 book-create-all
   2. 2 book-delete
   3. 3 book-update
   4. 4 book-view
   5. 5 bookshelf-create-all

   32 tests, one for each way to hold the five checks

   test	1	2	3	4	5	expects	run as	wrong user
   1						✓	E	C
   2								A
   3								A
   4								A
   5								A
   6								A
   7								A
   8								A

   24 more tests, the same pattern

   test	1	2	3	4	5	expects	run as	wrong user
   9								A
   10								A
   11								A
   12								A
   13								A
   14								A
   15								A
   16								A
   17								A
   18								A
   19								A
   20								A
   21								A
   22								A
   23								A
   24								A
   25								A
   26								A
   27								A
   28								A
   29								A
   30							R	A
   31								A
   32							C	A

   holds the permission doesn't hold it ✓expects to get in expects to be kept out E R C run as a user our seed made: the Editor (a BookStack role), the Reader and the Content-only reader (roles featkpr added) run as a role the test makes, holding exactly those (29 of 32) A C the wrong user: the Admin, or the Content-only reader

   Each column is one test (a row, on a narrow screen). A filled dot means its user holds that permission. Only the outlined test holds all five, so only it expects to get in. The other 31 expect to be kept out.

   Every test runs twice. First as its own user, and it must pass. Then as the wrong user, who should get the opposite answer, and now it must fail. A test that still passes wasn't checking the permission. For the outlined test, the wrong user is the Content-only reader, who holds none of the five. For the other 31, it is the Admin, who can do everything.

   32 tests, 64 runs

   That is for one route. featkpr read 343 routes in BookStack's code. Writing them takes only this route, its five checks and four users. That is the slice. Our idea is that an agent given the slice writes better tests than one given the whole app.

BookStack · featkpr's records of 28 Sep 2026: 312 features, and 32 tests for this route, 32 of them proven · the tests as featkpr's Runs screen shows them on a recorded world, at commit 0f5164ec; no outcomes drawn · routes read in the audit of 26 Sep 2026