End to End Test Framework Tier List

Selecting a Test Framework is no easy task, especially if you're not up to date with recent tools and improvements. Although all frameworks have their Pros and Cons, there is generally some standout options.

Disclaimer: This is my personal opinion, you may not agree and that's okay.

Assessing Test Frameworks based on First Principles

I like to think about what makes a great test framework from a set of first principles, this helps to reason about what is the ideal choice. Test Frameworks have a considerable amount of features that they often share and some that set them a part from others.

First Principles

• Open Source with active maintainers
• Cutting Edge, but not Bleeding edge
• Extensive documentation and an active community
• Concise and simple API for writing tests
• Support for Testing all web primitives without hacks or workarounds.
• Excellent Developer Experience for running, writing and especially debugging tests
• Parallel Test runs, ideally with a small performance footprint
• Integrate with modern CI/CD Tools - including Docker support
• Support for running tests on all Core browsers

But it depends..

A framework can tick all the boxes from a general sense, but it can be the wrong choice for your software, team and organisation. You should consider the following additional attributes:

• Language Support should align with team skills
• Picking up a new tool is the right time for your team
• The scope of your softwares features under test

Framework Tier List

Considering the above criteria, here is my thoughts on the Framework Tier list for testing web applications. (Updated 4th of March 2024)