flakey tests

• when you make software, you usually want to verify the software is correct. so you write tests! sometimes, however, the tests you write are flakey. this means that even if there are no changes to the source code or test code, they sometimes pass and sometimes fail. this is not good since the whole point of writing a test is to give yourself reliable signal that the code you wrote is correct/incorrect. in general, we want our tests to be as deterministic as possible (though, 100% determinism is sometimes not possible due to clocks, non-determinsitic nature of schedulers/network calls).
• you might wonder, what causes a test to be flakey. well, lots of things!
  • race conditions
  • timeouts
  • time-based code
  • db not cleaned between tests
  • non-determistic ordering of data returned by sql queries
  • shared resources
  • network calls, and in general external APIs and you don’t control
• when testing you want to think about things that can be an axis of variation. any dimension along which the world can change from one test to the next. for example, time is an axis. timezone is another axis. filesystem state is another axis. to avoid writing flakey tests, you want to inject fixed values for any variables that could have an axis of variation.

# time is axis of variation! isLate can now be different values depending on when this logic is executed
val now = Instant.now()
val isLate = now.atZone(NY).hour >= 18

# so, a better solution is to hardcode things like time
val now = Clock.fixed(instant.parse("2026-07-10T22:00:00Z", UTC))
val isLate = now.atZone(NY).hour >= 18