Playwright automated testing tool for web apps

Last updated on 12th July 2025

I am trying to quickly get an idea of Playwright which seems to be a very useful automated testing tool for web apps. An interesting aspect about Playwright is the automated test code generator - part of  Playwright Inspector - that Playwright has (like a macro recorder but smarter). So for straightforward tests, one may be able to easily generate test code for various test cases and thus have a test suite without doing any direct coding oneself (and have ability to tweak that test code with direct coding for more intricate tests).

Playwright is free and seems to be backed by Microsoft. I think it is open source but I have not checked that.

What is Playwright? (🎭 Playwright introduction tutorial, features & demo), https://www.youtube.com/watch?v=wGr5rz8WGCE, around 12 mins., pub. Jun 2021.

To skip install stuff jump to: https://youtu.be/wGr5rz8WGCE?t=236 . Covers generating test code using Playwright Inspector (Like macro recorder but smarter I think) and taking screenshot and videos of test run.

The following is a very impressive and recent video though the presentation lacks marketing polish but the presenter who is from Microsoft, seems to be very knowledgeable about Playwright and how it compares with other tools like Selenium (which seems to be another popular automated testing tool, and is open source). End-to-end Testing with #Playwright | David Paquette | The Test Tribe Calgary Meetup, https://www.youtube.com/watch?v=LT5a0Ye-wO0 , 1 hr. 32 mins, pub: Feb. 2025 . Skip intro of 1 min: https://youtu.be/LT5a0Ye-wO0?t=59

In some other video I quickly browsed through, it showed that Playwright can be integrated with GitHub Actions but I have not gone through that part properly and so don't know for sure.

May be worth checking out: AI-Powered Test Automation: Playwright MCP (Model Context Protocol) with GitHub Copilot in Action, https://www.youtube.com/watch?v=paSwmp-z9wc , 1 hr 25 mins, pub: 10 Jun 2025.

https://playwright.dev/ - Playwright enables reliable end-to-end testing for modern web apps.

https://playwright.dev/docs/intro - A startup kind of documentation page which I used to try out Playwright.

Playwright seems to be a sophisticated test automation software for web applications. In my quick trial of Playwright, I got the impression that using it to automate testing of significant size web applications, would need lot of test programming work unless some AI tools are able to create sophisticated working test scripts. If the test engineer is not familiar with Playwright then there will also be a learning curve and associated time spent. 

However, if regression testing work (after enhancements are made to system) is becoming a major burden then perhaps it may make sense to consider investing the time and effort to learn to use (if test engineer does not know) automated testing tools like Playwright and do the associated test programming work, to ease re-running of regression tests.

=====================

Notes related to my quick trial of Playwright

I followed the instructions given at https://playwright.dev/docs/intro . The test project got created properly and as part of the test project creation the minimal browsers used by Playwright were downloaded and set up in Playwright related folder. Later I was able to run the example test (tests/example.spec.js) and see the associated report. I have shared the project on GitHub: https://github.com/ravisiyer/explore-playwright .

Then I installed the Playwright VSCode extension as mentioned in https://playwright.dev/docs/getting-started-vscode . As I had already installed the browsers, I skipped some steps related to that. I explored using some of the extension features like running individual tests, running tests in debug mode, showing browser or showing trace viewer. They seem to work like it was demonstrated in the David Paquette session video mentioned earlier. Quite impressive.

Next I copied demo-todo-app.spec.js from tests-examples folder to tests and ran some of its tests. They too worked. Note that this demo-todo-app.spec.js is a far more sophisticated script than the example.spec.js script.

=====================

Next I tried to use Codegen to create a simple script for testing my Timestamp Tracker app from its live website:  https://ravisiyer.github.io/CGPTTimestamp/ . IIRC the trivial test of checking whether the app opens and shows 'Timestamp Tracker' in the page got created easily through codegen and I could test it easily. But when I tried to do more sophisticated tests of adding a timestamp and then deleting it, I ran into challenges of creating the tests using only codegen. Two issues I faced which I recall:

1. To check if the Add Timestamp did create a timestamp, using codegen I could only specify comparison with the timestamp text. But that timestamp text varies with time. So I would need to modify the check to add some regular expression matching the timestamp text the app creates. But here I would have to do the test coding myself.
2. How to check delete? Once again the timestamp text would vary. And here I would need to check deletion of that particular timestamp. Or I could check the number of timestamp entries shown on page but I did not know how to do that using codegen (without doing any test coding myself). Another tripping point here was that while doing codegen of the delete operation, the alert dialog was not shown. So once again, I would need to do coding myself to handle the alert dialog.

Then I tried to use Gemini free tier to create Playwright script for Timestamp Tracker after uploading the latter project's source code. It created what seemed to be a sophisticated testing script but on running it, I ran into multiple problems. I tried to resolve it through interactions with Gemini and some trial-and-error without really understanding the test code or the config details (as I don't want to spend time on that, as of now). But I got very little progress towards solution.

In the past, when I used Gemini to generate a React Native and Expo app (Timestamp Tracker), I could read and understand parts of the code where there were issues, and so resolve it, sometimes by myself and sometimes by directing Gemini to the issue. What this trial shows is that like in the case mentioned earlier, to use Gemini's assistance for creating working Playwright test scripts, I need to have similar level of knowledge of Playwright and the test code, so that I can fix some issues myself and get some issues fixed by directing Gemini to the specific issue.

The associated script file is: timestamp.spec.js (in tests folder) and has been put up in the GitHub repo. Even if it is not working now, one gets an idea of the code Gemini generates for such tests if Gemini is provided the app's source code. I find it to be quite sophisticated as it seems to try to test for various test cases. All these test cases were generated by Gemini itself without me prompting for the test cases. Gemini created the test cases based on its understanding of the functionality of the app after it studied the app source code. If the errors in this test script are trivial then it is very impressive to see how Gemini is generating all these tests (code for specific test cases). Sometime later if I do get on to learning Playwright test coding, I may try to spot and fix the errors in this script.

The tests in the Gemini created script (which it refers to as 'Timestamp Tracker App E2E Tests'):

1. should display initial timestamp and key buttons on launch
2. should add a new timestamp and display it
3. should toggle milliseconds display correctly
4. should add and save a note to a timestamp
5. should delete an individual timestamp
6. should trigger download when exporting timestamps
7. should clear N timestamps from the end
8. should clear all timestamps
9. should open and dismiss Info modal and navigate to blog link
10. should warn when reaching MAX_TIMESTAMPS and prevent further additions
11. should enforce MAX_NOTE_LENGTH for notes
12. should cancel note editing and revert changes

I was astonished to see all the above test cases and associated test code that Gemini generated based on me simply providing it the app's source code. If the fixes for getting above test script to work are simple then it will be an awesome capability of Gemini free tier.