How to attach Cypress screenshots to the Mochawesome report

15 Oct

6 Sep

Performing quality control with the default settings may not be as productive as expected. However, you can greatly streamline your process with a bit of customization and know-how. When the test fails, Cypress generates a corresponding screenshot, but it will not be added to the HTML report. Today, you’ll learn how to solve this problem.

Andrii L.

QA Lead

Hello, guys! As the title suggests, in this guide, I will show you how to properly install, set up, and run Cypress with Mochawesome reporter to make the most of HTML reports. On top of that, I will also teach you to set a custom viewport screen size when running Cypress in headless mode. So, today you will learn:

How to install and configure Cypress from scratch;
How to install and configure Mochawesome reporter plugin to get especially attractive HTML reports;
How to add failed test screenshots to Mochawesome reports;
How to fix viewport size for a headless run, as it’s 1280x720 by default;
How to set up git hook to trigger test suite.

Now that you know what you’re in for, let’s get to business.

Installing Cypress and additional components

This is the first and most obvious step, so start off by installing Cypress. If you need any help, check out the official docs.

npm install cypress --save-dev

Once that’s out of the way, install the npm-run-all package. This package will be needed to build specific logic to run tasks sequentially, even if the previous one fails.

npm install npm-run-all --save-dev

Next, install the simple-git-hooks package. It will allow you to set a trigger to run tests before pushing them to a remote repository.

npm install simple-git-hooks --save-dev

Finally, install the following components:

mochawesome — creates a report in JSON format as a result of running a spec file;
mochawesome-merge — generates one common JSON from several JSON reports;
mochawesome-report-generator — generates HTML/CSS report to visualize the test run.

npm install --save-dev mochawesome mochawesome-merge
mochawesome-report-generator

Configuring and using Cypress with Mochawesome

Open Cypress runner

Open the project’s directory and run Cypress with the npx cypress open command. This will open the test runner, initialize Cypress and create the following:

📄 cypress.json config file;

📁 cypress folder in the root directory of the project.

Set configurations in the cypress.json file:

{
"baseUrl": "<https://some-domain.com>",
"viewportHeight": 1080,
"viewportWidth": 1920,
"video": false,
"screenshotsFolder": "cypress/results/mochawesome-report/assets"
}

Check out this doc to find additional configuration settings.

Applications may experience false-negatives, and avoiding such situations is critical. These false-negative outcomes can be caused by a sudden single application failure, Internet connection problem, etc. You can find more information about this here. I suggest adding configuration parameters to rerun the tests if the first attempt was unsuccessful. You can see the retries section in the updated config:

{
"baseUrl": "<https://some-domain.com>",
"viewportHeight": 1080,
"viewportWidth": 1920,
"video": false,
	"retries":{
		"runMode": 1,
		"openMode": 1
	}
}

Set reporter configurations in the cypress.json file to let Mochawesome know how it should process the report

The cypress.json should look like this:

{
"baseUrl": "<https://some-domain.com>",
"viewportHeight": 1080,
"viewportWidth": 1920,
"video": false,
"retries":{
	"runMode": 1,
	"openMode": 1
	},
"reporter": "mochawesome",
  "reporterOptions": {
    "reportDir": "cypress/results",
    "overwrite": false,
    "html": false,
    "json": true
  }
}

Create commands to delete the folder with JSON reports in the package.json file

Delete files and folders if they exist, then recreate these folders.

{
"delete:reports": "node cypress/support/node_fs/removeDir.js",
"create:reportFolders": "node cypress/support/node_fs/createDir.js"
}

Create the removeDir.js and createDir.js files

Do that in the strictly defined path: cypress/support/node_fs/ so the npm tasks we added previously could find these scripts.

removeDir.js

// removeDir.js

var fs = require('fs')
const dir = 'cypress/results'

fs.readdir(dir, (err, files) => {
	console.log(files)
	if (files) {
		fs.rmdir(dir, { recursive: true }, (err) => {
	if (err) {
		throw err
	}
	
				console.log(`${dir} is deleted!`)
			})
	} else {
		return console.error(err)
	}
})

createDir.js

//createDir.js

var fs = require('fs')

fs.mkdir('cypress/results', (err) => {
	if (err) {
		return console.error(err)
	}
	fs.mkdir('cypress/results/mochawesome-report', (err) => {
		if (err) {
			return console.error(err)
		}
	})
})

Create a script in the package.json file to merge the Mochawesome JSON report files with a common JSON file

This file will be named mochawesome.json. Once created, the script will place it in the cypress/results/mochawesome-report folder.

{
"mochawersome:merge": "npx mochawesome-merge 'cypress/results/*.json' > cypress/results/mochawesome-report/mochawesome.json && npx marge cypress/results/mochawesome-report/mochawesome.json -f report -o cypress/results/mochawesome-report --autoOpen=true"
}

Create scripts in the package.json file

cy:chrome — runs tests in headless mode in the Chrome browser;
pretest — deletes the folder with the report of the previous run, if it exists;
test — generates an HTML report for the current run.

{
	"cy:chrome": "npx cypress run --browser=chrome",
	"pretest": "npm-run-all --sequential delete:reports create:reportFolders",
	"test": "npm-run-all --sequential cy:chrome mochawersome:merge --continue-on-error"
}

In the test and pretest scripts, you can find the following flags:

--sequential — a flag that starts the tasks sequentially;
--continue-on-error — a flag that proceeds with the task execution even if the previous one has failed.

Delete the training files

By default, the cypress/integration folder contains training data. Feel free to delete it as soon as you are comfortable with Cypress.

Edit the .gitignore file

Often there is no reason to commit test reports to the remote git repo, so it’s best to edit the .gitignore file to make the results folder invisible for git.

# Cypress testing files and folders
cypress/results

Adding screenshots of failed tests to the Mochawesome HTML report

To do that, first open cypress/support/index.js and add the following line at the very top of the file:

import addContext from "mochawesome/addContext";

Then add an event to the same file below. When a test fails, a "screenshot" constant will be created for the corresponding test. This image will then be added to the HTML report.

Cypress.on("test:after:run", (test, runnable) => {  
	if (test.state === "failed") {    
		const screenshot =`assets/${Cypress.spec.name}/${runnable.parent.title} -- ${test.title} (failed) (attempt 2).png`;    
		addContext({ test }, screenshot);  
	}
});
// pay attention to the "screenshot" constant because if you specify an incorrect file name mask, the screenshot will not be included in the report

Done! 🤠 Your report will now display screenshots.

Configuring simple-git-hooks

Have a look at the package’s readme page to get familiar with it. To run tests before every git push, you need to add the following code to the package.json file of the project:

{
"simple-git-hooks": {
	"pre-push": "npm run test"
	}
}

Don’t forget about point #3 in the simple-git-hooks readme. Perform configuration updates by running the npx simple-git-hooks command.

‍

Setting the required browser window size in headless mode

By default, the screen size (browser tab) in headless mode is set to 1280х720px. If the screen size doesn’t matter to you, congratulations, you’re all set. But there’s a huge problem if your tests rely heavily on specific screen sizes. If you set 1920 and 1080 for viewportWidth and viewportHeight parameters, the application will simply scale up. And the result will sure to disappoint you. In order to customize the screen size, you need to change your browser’s settings before running tests. Let me demonstrate this problem and show you the solution.

Suppose I need to run tests on a 1920x1080 screen at 100% zoom. If I run the default tests, I would get this:

As you can see, the scale of the page is 43%. This won’t cut it, as it is not 1920x1080. So why is this happening, and where did the 1280x720 size go? By default, Cypress covers the entire screen area during screening, and this is what we have:

So we need to increase the default screen size. We can do this by changing the browser settings, which should be applied just before running tests. To do so, you need to add the following code to this file: cypress/plugins/index.js

module.exports = (on, config) => {
	on("before:browser:launch", (browser = {}, launchOptions) => {
	console.log("launching browser %s is headless? %s", browser.name, browser.isHeadless)
	
			// the browser width and height we want to get
			// our screenshots and videos will be of that resolution
			const width = 3840
			const height = 2160
	
			console.log("setting the browser window size to %d x %d", width, height)
	
			if (browser.name === "chrome" && browser.isHeadless) {
				launchOptions.args.push(`--window-size=${width},${height}`)
	
				// force screen to be non-retina and just use our given resolution
				launchOptions.args.push("--force-device-scale-factor=1")
			}
	
			if (browser.name === "electron" && browser.isHeadless) {
				// might not work on CI for some reason
				launchOptions.preferences.width = width
				launchOptions.preferences.height = height
			}
	
			if (browser.name === "firefox" && browser.isHeadless) {
				launchOptions.args.push(`--width=${width}`)
				launchOptions.args.push(`--height=${height}`)
			}
			
			// IMPORTANT: return the updated browser launch options
			return launchOptions
	})
}

I chose a large enough screen size so that my iframe would not scale in the test runner, and this is what I got:

Take note that we are now at 100% scale, which is exactly what we need!

We can double-check this by looking at the file dimensions.

Conclusion and recap

Voilà! We have solved the problem of running tests for any viewport size in headless mode, and everything is fine now. So let’s summarize:

you can now install Cypress and configure its basic nodes;
you can install and set up Mochawesome reporter to generate great HTML reports of test runs with screenshots of relevant failed tests;
you’re not constrained anymore by the default Cypress viewport screen size when running tests in headless mode as you can now customize it for your project;
and you’ve also learned how to automatically run tests with Git hooks.

I hope this guide has helped you and improved your QA process, and if it hasn't, I'm sure it will inspire you to create an even cooler solution of your own! Either way, best of luck, and may the Cypress be with you.

Writing team: