Slack Reporting for Cypress.io

I’ve been using Cypress for front-end testing for the last year, which we have been executing in our CI pipeline with CircleCI. CircleCI offers slack notifications for builds, but it doesn’t offer the ability to customise the Slack notifications with build metadata. So I decided to write a slack reporter, that would do the following

  • Notify a channel when tests are complete
  • Display the test run status (Passed / Failed / Build Failure), plus number of tests
  • Display VCS metadata (Branch Name / Triggering Commit & Author)
  • Display VCS Pull Requesdt metadata (number and link to PR )
  • Provide a link to CI build log
  • Provide a link to a test report generated with Mochawesome
  • Provide links to screenshots / videos of failing test runs

The source code is available here :- https://github.com/YOU54F/cypress-slack-reporter

It has been released as a downloadable package from NPM, read below for details on how to get it, and how to use it.

As this is an add-on for Cypress, we still need a few pre-requisites

1. Download the npm package direct from the registry

npm install cypress-slack-reporter --save-dev

or

yarn add cypress-slack-reporter --dev

2. Create a Slack incoming webhook URL at Slack Apps

3. Setup an environment variable to hold your webhook, created in the last step and save as SLACK_WEBHOOK_URL

$ export SLACK_WEBHOOK_URL=yourWebhookUrlHere

4. Add the following in your cypress.json file

{
  ...
  "reporter": "cypress-multi-reporters",
  "reporterOptions": {
    "configFile": "reporterOpts.json"
  }
}

5. Add the following in a newly created reporterOpts.json file

{
  "reporterEnabled": "mochawesome",
  "mochawesomeReporterOptions": {
    "reportDir": "cypress/reports/mocha",
    "quiet": true,
    "overwrite": false,
    "html": false,
    "json": true
  }
}

6. Run cypress in run mode, which will generate a mochawesome test report, per spec file.

7. We now need to combine the seperate mochawesome files into a single file using mochawesome-merge

$ mkdir mochareports && npx mochawesome-merge --reportDir cypress/reports/mocha > mochareports/report-$$(date +'%Y%m%d-%H%M%S').json

8. We will now generate our test report with mochawesome, using our consolidated test report

$ npx marge mochareports/*.json -f report-$$(date +'%Y%m%d-%H%M%S') -o mochareports

9. We can now run our Slack Reporter, and set any non-default options

$ npx cypress-slack-reporter --help

  Usage: index.ts [options]

  Options:
    -v, --version            output the version number
    --vcs-provider [type]    VCS Provider [github|bitbucket|none] (default: "github")
    --ci-provider [type]     CI Provider [circleci|none] (default: "circleci")
    --report-dir [type]      mochawesome json & html test report directory, relative to your package.json (default: "mochareports")
    --screenshot-dir [type]  cypress screenshot directory, relative to your package.json (default: "cypress/screenshots")
    --video-dir [type]       cypress video directory, relative to your package.json (default: "cypress/videos")
    --verbose                show log output
    -h, --help               output usage information

Our generated slack reports will look like below.

alerts.png

Currently we support CircleCI for CI & GitHub/BitBucket VCS’s.

For other providers, please raise a GitHub issue or pass --ci-provider none provider flag to provide a simple slack message based on the mochawesome report status.

It is possible to run to run the slack-reporter programatically via a script

// tslint:disable-next-line: no-reference
/// <reference path='./node_modules/cypress/types/cypress-npm-api.d.ts'/>
import * as CypressNpmApi from "cypress";
import {slackRunner}from "cypress-slack-reporter/bin/slack/slack-alert";
// tslint:disable: no-var-requires
const marge = require("mochawesome-report-generator");
const { merge } = require("mochawesome-merge");
// tslint:disable: no-var-requires

CypressNpmApi.run({
  reporter: "cypress-multi-reporters",
  reporterOptions: {
    reporterEnabled: "mocha-junit-reporters, mochawesome",
    mochaJunitReportersReporterOptions: {
      mochaFile: "cypress/reports/junit/test_results[hash].xml",
      toConsole: false
    },
    mochawesomeReporterOptions: {
      reportDir: "cypress/reports/mocha",
      quiet: true,
      overwrite: true,
      html: false,
      json: true
    }
  }
})
  .then(async results => {
    const generatedReport =  await Promise.resolve(generateReport({
      reportDir: "cypress/reports/mocha",
      inline: true,
      saveJson: true,
    }))
    // tslint:disable-next-line: no-console
    console.log("Merged report available here:-",generatedReport);
    return generatedReport
  })
  .then(generatedReport => {
    const base = process.env.PWD || ".";
    const program: any = {
      ciProvider: "circleci",
      videoDir: `${base}/cypress/videos`,
      vcsProvider: "github",
      screenshotDir: `${base}/cypress/screenshots`,
      verbose: true,
      reportDir: `${base}/cypress/reports/mocha`
    };
    const ciProvider: string = program.ciProvider;
    const vcsProvider: string = program.vcsProvider;
    const reportDirectory: string = program.reportDir;
    const videoDirectory: string = program.videoDir;
    const screenshotDirectory: string = program.screenshotDir;
    const verbose: boolean = program.verbose;
    // tslint:disable-next-line: no-console
    console.log("Constructing Slack message with the following options", {
      ciProvider,
      vcsProvider,
      reportDirectory,
      videoDirectory,
      screenshotDirectory,
      verbose
    });
    const slack = slackRunner(
      ciProvider,
      vcsProvider,
      reportDirectory,
      videoDirectory,
      screenshotDirectory,
      verbose
    );
     // tslint:disable-next-line: no-console
     console.log("Finished slack upload")

  })
  .catch((err: any) => {
    // tslint:disable-next-line: no-console
    console.log(err);
  });

function generateReport(options: any) {
  return merge(options).then((report: any) =>
    marge.create(report, options)
  );
}

I have been extending the reporter, to allow the ability to upload the mochawesome report, and cypress artefacts (screenshots & videos) to an S3 bucket, and use the returned bucket links, for the Slack reporter. It is currently working on a PR, but needs adding to the CLI before it can be added to the master branch.

Published by

YOU54F

I have been a Software Test Engineer for 11+ years now, starting off in Accessibility Testing at a large UK Banking organisation. I have since worked with financial / medical healthcare / betting / telecommunication providers testing software and along the way helping some migrate from traditional software development methodologies to a leaner Agile based approach. I now work for a consultancy company, providing insight into tools / technologies / approaches to ensure we are testing the right things, at the right place in the stack. Less of my time these days is spent writing test code, and more time championing test processes with our developers, who have picked it up and hit the ground running.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.