Percy is an all-in-one visual testing and review platform.

In this post I'll describe how I have added it to my blog. I've first tried the recommended way in the docs but that wasn't working for me. There was an error from node that it couldn't spwan npx. I think that was related to Windows becaue Node had a problem with c:\program files\ folder. The error in the CLI was:

"gatsby-plugin-percy" threw an error while running the onPostBuild lifecycle:

Command failed: C:\Program Files\nodejs\npx.CMD percy snapshot C:\Users\alexander\GitHub\my-blog\public

Why is Percy useful?

Percy is a part of tools offerd by Broswerstack. With Percy you can visually review your last change and if every page is looking as expected.

A case that recently happened to me, an image was not loading in one blog post because Unsplash.com removed it. With-out the screenshots of Percy it wouldn't be possible for me to spot that issue but with it I've detected the issue by looking through the screenshots. The fix was easy by replacing the example image with a different one in Contentful. For more details about the why, see the docs about visual testing basics.

Setup

I don't like to set the PERCY_TOKEN with set so I've changed my npm scripts to use nps.

Install nps with npm i -SE nps. Next, run node_modules\.bin\nps init in your project folder this will copy your package.json scripts to package-scripts.js and adds start: 'nps' as only script to your package.json.

Now, you can access every script with npm start build or npm start dev etc. With this in place, we can now add percy as a script like in the following snippet:

require("dotenv").config();

module.exports = {
  scripts: {
    dev: 'gatsby develop',
    build: 'nps buildGatsby percy',
    buildGatsby: 'cross-env GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES=true ACTIVE_ENV=production gatsby build --log-pages',
    percy: `cross-env PERCY=${process.env.PERCY_TOKEN} npx percy snapshot ./public`,
    // other scripts ...
  }
};

We have added a script called percy and we're using cross-env to add our PERCY_TOKEN - cross-env is a Windows thing (not needed in Linux or MacOs) Call percy after the Gatsby build is finished, so it will be started after every build. If you don't want to call Percy with every build, just remove the percy from build and run it with npm start percy.

The percy token is needed so you can publish the generated snapshot to Percy. You can get your token on https://percy.io by going to Project Settings. Copy the token and add it to your .env file like following:

# ...other envs...
PERCY_TOKEN=your-token-here

Why is percy not called in a postbuild step? This is because nps is not supporting the npm pre- or postbuild scripts. But it's easy to mimic that by adding the scripts in a series call. The syntax is a bit different form the readme as there is no comma needed. Just use a space between the scripts like nps pre buildGatsby post.

Keep in mind, that you also have to change your build script in your deployment (in my case Netlify) to use npm start build instead of npm run build and also add the PERCY_TOKEN to your deploy environment.

First Percy run

Build your page with npm start build. After building you'll get a link in your terminal to the Percy results something like /projectName/builds/<build-no

This will open a webpage similar to the below screenshot

There is no comparison yet because it was the first build. Later it will compare the visual differences and help you spot possible issues. It will mark them with a red color if there is something to review.

For the first build, it's OK to go through each screenshot and check if it's displayed as expected.

If you're working in a team you can also add comments to the screenshots to mark issues or ask questions.

Also approval is possible and you can add it to your Github pull requests (PR). It will notify you in the PR if there are visual changes which require a review.

Other Browserstack tools

I have to check the other tools at Browserstack, too. I'm still looking for a way to check web pages on multiple devices. Browserstack got the live view but each session on a real device is just 60 seconds and that's too short to test anything.

Also automated testing would be interesting but that's not required for my blog. For a app, I think I have to do some reasearch to check what tool to use.

I think cypress.io is pretty good but I haven't tested it yet.

Responisve test

The responsive test on Browserstack wasn't working for me or am I just too impatient?

I've tried my blog, but I'm not seeing any feedback. It kept spinning and after cancelling and restarting it tells me that there is something generated (see below screenshot) - but I don't see any progress.

Screenshot test

This is working better and there is an option to get an e-mail notification once the test results are ready. I like the idea of getting an e-mail but I've waited for the results and they're ready (12 of 14 done - 2 timed-out). The e-mail wasn't send, maybe I'll get it later...

Seems like my images are not displayed correclty in Internet Explorer (IE) (they're the blurry initial draw images from Gatbsy) but I don't care as IE usage is below 1% (source https://gs.statcounter.com/). IE Edge usage is a bit higher but shouldn't be a problem as it's based on Chrome.

Conclusion

Maybe I'll check if the issue with the gatbsy-plugin-percy still exists. If it's still a problem, I'm forking the Github repo to see if there is a fix. Just wondering why this issue was closed.

Any way, I'm happy with the setup and I think it's also OK to add it in the build scripts.

For the cross-device testing, I think I have to look for other tools. Browserstack is not that bad but wasn't working for me.

Maybe I have to test Sizzy but I'm not sure about the pricing.