Quick and dirty way to fix flakiness in Cypress tests

Overview of the problem

Frontend test automation can be fairly tricky when attempting to do full end to end coverage. People want 100% flake free tests, and only act on problems that are actionable.

If you've ever written an end to end test, you've probably had instances where a particular area of your app will have some odd flakiness to it. The test will run and succeed 95% of the times, but when it fails randomly due to a backend issue, the solution may very well be to re-run it until it passes. The simplicity of a person re-running a failing test until it passes is what causes error numbness, and frustration for having to do it. There are instances where the backend may not fully respond in time even if you put in reasonable async waits, or the backend is being flaky which can cause frontend tests to fail.

The goal of this solution is to make it so that when failures do occur, the failure is actionable. You want to cultivate a culture of reacting quickly to failures, and not have the solution be that a developer goes into the CI build to rerun the build until it passes.

[Quick and dirty temporary solution]

There is a neat temporary plugin that you can install to address this issue in the short term until there's a more permanent solution by Cypress themselves. It's an npm package named "cypress-run" that essentially forwards all arguments you already use when running "cypress run". I'll provide the instructions in yarn, as there are instructions for npm in the package link.

Full link - https://www.npmjs.com/package/cypress-run
Package description in case it ever disappears

Hopefully this module is short-lived. Right now it add retry support for parallel runs. It should work just like cypress run by forwarding all arguments, just using the Cypress Module API to support parallelization.

It uses this gist and turns it into a module and adds support for cypress run arguments: https://gist.github.com/Bkucera/4ffd05f67034176a00518df251e19f58#file-cypress-retries-js

This modules also adds retries regardless of --parallel or --record flags and doesn't need the Cypress Dashboard.

Installation of cypress-run

1. In your command line

yarn add --dev cypress-run

2. In your package.json

Update your "cypress run" command to "cypress-run"

It's that simple!

I've looked into several solutions, and the most noteworthy alternative is "cypress-plugin-retries" which is a bit more fully featured where it will only re-run the specific test that fails. I prefer re-running the entire test file if it fails.