🏆Capstone project: Project: Prove the Tests

A green suite proves that the current examples agree with current production. It does not yet prove the examples would notice a realistic regression. This project checks both directions.

Run the durable suite once more:

python -m pytest -q
cat requirements.txt

The saved project should contain your program, its input and schema documentation, the one pinned requirement, and the tests. It should not contain selected.json, catalog.json, or scratch CSV or JSON files. Pytest may create excluded tool state such as .pytest_cache; that is derived state, not an authored project file.

Break your own program on purpose

Here is the exercise, and it is the one that decides whether your suite is worth keeping.

Make a temporary backup of catalog.py outside the project, such as /tmp/catalog.py.lesson10-backup. Then go back to the original and introduce one defect. A real one, of the kind a tired person actually writes. Run the suite. Does a test go red? Does the failure message point at the thing you broke?

Work through these one at a time, restoring the known-good file between each. Finish by restoring the correct version and running the full suite green:

• change the inclusive >= back to a strict >;

• ignore the threshold entirely and keep every item;

• sort the selected items, or drop duplicates;

• write to a fixed filename instead of the requested output path;

• set ensure_ascii=True again;

• remove the trailing newline from the written file;

• let the program carry on after a read failure and write output anyway.

Seven defects. A suite that stays green through any one of them has a hole, and now you know exactly where it is.

Two ways to be fooled while doing this. A suite that is always red is not catching anything, it is just broken. And a test that fails because pytest could not collect it, or could not import your module, has told you nothing about the defect. Check that the baseline is green and the collection count is unchanged before you believe a red result.

Write the assertion yourself

One more rule for the file-behavior tests, and it matters more than it looks.

When you check what the writer produced, parse the file with json.loads and compare against a dictionary you wrote by hand in the test. Do not call your own read_catalog_json to check your own write_catalog_json. If both share a mistake, they will agree with each other perfectly and tell you everything is fine.

A test should be an independent opinion, not a second vote from the same source.

Press Check my work when the correct project is green and every file your tests touch lives under tmp_path.