Well… That’s why you test the public API and not the internals. And if you need to change the API you should have a good enough reason that changing the tests is okay too.
Well, since it’s “just” LLM time writing the tests, test the internals too, but clearly mark which tests are the public API and which tests are intnernal, so it knows to never ignore / update a public API test without consultation, but it an internal test fails it can make the call: did this test start failing because I unintentionally broke something, or because the test was testing something that no longer matters - I have seen Claude go both ways, fixing its own mistake due to a test catching an unintended consequence of a change, and also just patching the test because the old test was… old.
Well… That’s why you test the public API and not the internals. And if you need to change the API you should have a good enough reason that changing the tests is okay too.
Well, since it’s “just” LLM time writing the tests, test the internals too, but clearly mark which tests are the public API and which tests are intnernal, so it knows to never ignore / update a public API test without consultation, but it an internal test fails it can make the call: did this test start failing because I unintentionally broke something, or because the test was testing something that no longer matters - I have seen Claude go both ways, fixing its own mistake due to a test catching an unintended consequence of a change, and also just patching the test because the old test was… old.