I have worked with huge test suites.

I have worked with huge test suites. If your test suite contains tests that fail for the reasons given, and the result is a shrug, your test suite becomes useless very quickly, and then triaging which tests are allowed to fail, which are real problems, and so on becomes a much bigger waste of time and treasure. Making a big deal out of it when tests are added that mutate global state or call 3rd party APIs is an investment that pays back very quickly. Shrugging and saying "98% of the tests passed, that's probably good enough" means your test suite itself is a waste of time and resources and you could be getting the same result by just doing a quick happy path minimal set.

Posted by peteretep on Jun 21, 2015, 4:53 AM

Tags

• reliability (Other)

Related issues

• This is my biggest problem with agents, they constantly defensively code for cases that have no chance of happening and in most cases a p...
• I wish there was something like LEAN for biology.
• The issue is really the extreme difficulty of programming them, especially for anything sequential (rather than combinatorial).
• I probably could have beared with it if it was only limited to programming, but it showing up in creative fields has made me feel all sor...
• I hate that I'm in deep enough to know what you mean.
• The biggest problem with “AI” is that big corps are shoving it down our throats.