Microsoft recently announced the public preview of Microsoft Playwright Testing, a new service for running Playwright tests at scale through Azure.
Playwright is a fast-growing, open-source framework that enables reliable end-to-end testing and automation for modern web apps. It uses a single API to control Chromium, Firefox, WebKit, and Opera browsers. Microsoft will offer the framework through a managed service, claiming that users can "run Playwright tests with much higher parallelization across different operating system-browser combinations simultaneously."
With the @playwright/test runner, tests run in independent, parallel worker processes, with each process starting its own browser. Moreover, increasing the number of parallel workers can reduce the time it takes to complete the full test suite. However, when running tests locally or in a continuous integration (CI) pipeline, there is a limitation to the number of central processing unit (CPU) cores on a local machine or CI agent machine.
Mandy Whaley, a partner director of product, Azure Dev Tools at Microsoft, writes:
By using Microsoft Playwright Testing service, you can increase the number of workers at cloud-scale to much bigger numbers. The worker processes orchestrated by @playwright/test continue to run locally, but the browser instances, which are resource-intensive, now run in the cloud.
With Microsoft Playwright Testing, developers can use the scalable parallelism provided by the service to run web app tests simultaneously across all modern rendering engines such as Chromium, WebKit, and Firefox on Windows and Linux and mobile emulation of Google Chrome for Android and Mobile Safari. In addition, the service-managed browsers ensure consistent and reliable results for functional and visual regression testing, whether tests run from a CI pipeline or development machine.
Overview Microsoft Playwright Testing service from CI pipelines and code editors (Source: Developer tools blog post)
To effectively use Microsoft Playwright Testing, developers must create a workspace in Azure, subsequently, generate an access token and a region-specific service endpoint. With the token and endpoint, developers can enter these in their environment variables and leverage these settings in their project, i.e., playwright.service.config.ts they need to create alongside the playwright.config.ts file. Lastly, they can run tests using the Playwright CLI or Visual Studio Code.
Running Playwright tests on remote browsers in a workspace, the CLI command can look like this:
npx playwright test --config=playwright.service.config.ts --workers=20
Alternatives for Playwright Testing are available, such as the popular Selenium, Cypress.io, and Puppeteer. These can be leveraged in the cloud for scale; for instance, Cypress.io offers the opportunity to use Cypress Cloud, and Puppeteer can be run in Cloud Functions.
With Playwright Testing available as a managed service in Azure, David Osolkowski, a software engineer at Vyne Dental, tweeted on X:
Is this a precursor to Playwright disappearing in its current form and only being available/maintained as a service? I’m tired of the enshittification of software tools, but maybe it’s time to hop back on the treadmill and look for an alternative…
Amanda Silver, a CVP of the Developer Division at Microsoft, replied in a tweet:
No. Playwright will continue to be an OSS FX unto itself.
Lastly, more details on Microsoft Playwright Testing are available on the documentation landing page and GitHub repo, and pricing can be found on the pricing page.