Since Playwright came out of shadows, there has been a lot of buzz around it lately. One of the reason for this is how Playwright simplifies handling of complex page scenarios like Iframes, Shadow DOM elements etc.
So, before Playwright, Selenium also was able to handle open Shadow DOM elements ( we’ll take about Open and Closed Shadow DOM in section below). However, it relied on the use of plain JS functions to get elements inside the Shadow DOM. Also, if you had multiple Shadow DOM hierarchies, then it became a bit tough ( although not impossible) to get inside the hierarchy.
In this blog, we’ll see how Playwright solves these issues. We’ll see how it has made us fell in love again with the Shadow DOM elements – with multiple scenarios include a combination of Shadow DOM and iFrames also.
We’ll not compare how the Playwright solves this issues with respect to Selenium, because I don’t think so that this warrants a comparison, and also I’m too lazy to create such a post 😀
DOM
Before understanding the concept of Shadow DOM, let us take a quick look at what DOM means. DOM is an abbreviation of Document Object Model
As per the Mozilla Developer Network this is what is a DOM is
The DOM represents a document with a logical tree. Each branch of the tree ends in a node, and each node contains objects. DOM methods allow programmatic access to the tree. With them, you can change the document’s structure, style, or content.
Nodes can also have event handlers attached to them. Once an event is triggered, the event handlers get executed.
Let us consider a very simple example of an HTML page.
<head>
<title>A simple web page</title>
</head>
<body>
<h1>Hello world</h1>
<p>I am rendered!</p>
</body>
</html>
As mentioned in the above mentioned definition, a logical tree of the this HTML structure would look this this
SHADOW DOM
Shadow DOM is one of the implementations of the OOPS principle of Encapsulation in an HTML document or DOM if you will. By using the concept of Shadow DOM, the style, behavior and actions on one part of the page can be completely hidden or kept separate from other part of the DOM.
Shadow DOM allows hidden DOM trees to be attached to elements in the regular DOM tree — this shadow DOM tree starts with a shadow root, underneath which can be attached to any elements you want, in the same way as the normal DOM.
There are some bits of shadow DOM terminology to be aware of:
- Shadow host: The regular DOM node that the shadow DOM is attached to.
- Shadow tree: The DOM tree inside the shadow DOM.
- Shadow boundary: the place where the shadow DOM ends, and the regular DOM begins.
- Shadow root: The root node of the shadow tree.
OPEN AND CLOSED SHADOW DOM
Consider this sample code which gives an example of a Shadow DOM tree
<div>
<div id="shell">
<div id="role-id"></div>
#shadow-root (open)
<div id="avatar"></div>
</div>
<a href="./logout.html">Logout</a>
</div>
Now consider this sample code
<div>
<div id="shell">
<div id="role-id"></div>
#shadow-root (closed)
<div id="avatar"></div>
</div>
<a href="./logout.html">Logout</a>
</div>
As you can see from above, one shadow root is open while the other one is closed. There is a difference in between open and closed shadow roots – the open shadow root can be accessed via the shadowRoot property of the HTML element, where as closed shadow root can be accessed. You can read this question on Stackoverflow about this here.
Now how does it impact us?
So, Playwright (and Selenium) both are able to access elements in the open shadow DOM. Playwright’s css and text locators pierce shadow DOM by default (documented here), which mean, you do not need explicit JS code to access the shadow DOM, which is in the case of Selenium.
One thing to note here is that Xpaths do not pierce shadow DOM – this has been mentioned here.
However, both Playwright and Selenium cannot access the elements inside the closed shadow DOM. Does that mean we cannot automate them. Well, it depends. In the blog below, we will see such a scenario and see what workaround we can have.
We’ll take Sanjay Kumar’s SelectorsHub Practice page as the AUT since it has almost all of the complex scenarios we want to cover. Btw, if you haven’t you need to take a look at SelectorsHub extension, a very helpful extension for the automation engineers, which is under the umbrella of a host of other products offered by SelectorsHub.
Scenario 1 : Element inside an open Shadow Root
So this may be one of the most common scenarios where an element is inside an open shadow root. Head over to the practice page. There is an input box with the label UserName and placeholder text enter name. See the image below
Now how to send a text to this input element using Playwright?
Since Playwright css engine pierces the shadow DOM by default, we can directly use the id of the div where the shadow root starts and the element inside the shadow root.
await page.locator("#userName #kils").fill("kancha cheena");
After running the code, we see that the code runs successfully and the element recieves the text.
We can also use the new locator methods which Playwright introduced after v1.27 onwards. So we will use the getByPlaceholder method to pass the value
await page.locator("#userName").getByPlaceholder("enter name").fill("kancha cheena");
Entire code is here
Scenario 2 : Multiple Open Shadow Roots ( Nested Shadow Roots)
Now there can be cases, when there are nested shadow roots – means one shadow root is inside of another shadow root. In that case too, Playwright can easily pierce the inner shadow DOM and get to the elements.
If you read the Playwright’s documentation regarding how to select elements in Shadow DOM , you can see these lines mentioned
In particular, in css engine, any Descendant combinator or Child combinator pierces an arbitrary number of open shadow roots, including the implicit descendant combinator at the start of the selector.
Now let’s see any example from the same web page – if you see the element which has a placeholder Enter pizza name in Chrome’s Elements tab , then you’ll see that it is inside a shadow root, which in turn is nested inside the shadow root we saw in the above scenario 1.
How can we get that element ? It is as simple as
await page.locator('#userName input#pizza').fill('Mozarella');
Since Playwright css engine can pierce the shadow DOM inside the original one, we can just use the id of the element and send our input to it. Also the new methods like getByPlaceholder can be used here also.
Scenario 3 : Element is inside an iframe inside a Shadow Root.
The thing about the practice page from Selectorshub is that there is no dearth of the complex scenarios on this page. Most of the practice pages out there have the basic scenarios, which is good, but when you’re working in an enterprise applications, then most of the times you will have to encounter more complex scenarios.
One of such scenarios is when the element is inside an iframe, which is in turn embedded inside an open shadow root. How do you handle such scenarios? Let’s see and find out.
Can we access elements inside an iframe , inside a shadow root, just like we can access the shadow root elements? The answer is No. By default, Playwright css engine will pierce the shadow root DOM but will not search elements inside iframe or closed root. This is mentioned in their documentation
In particular, in css engine, any Descendant combinator or Child combinator pierces an arbitrary number of open shadow roots, including the implicit descendant combinator at the start of the selector. It does not search inside closed shadow roots or iframes.
which means that we cannot do this
await page.locator('#userName').getByPlaceholder('Current Crush Name').fill('Scarlett Johansson');
the above code will return an error because the element will not be found. Also this will also fail
await page.locator('#userName #jex').fill('Scarlett Johansson');
To get the element inside the iframe, we’ll have to go to the iframe and then search element inside the iframe.
So, we’ll use the framelocator method in this scenario –
await page.locator('#userName').frameLocator('#pact1').getByPlaceholder('Current Crush Name').fill('Scarlett Johansson');
We get to the shadow root, then use the framelocator method to get to the iframe with id pact1 and then use the getByPlaceholder method to get the input element and then fill it with required text – and voila, the code works
Scenario 4 : Element is inside a closed shadow root
Up until now, we’ve seen scenarios where Playwright can directly or indirectly pierce the shadow root. Now if the element is inside a closed shadow root, how do we enter that element? There is a separate school of thought or approach as to why even automate that? I kind of agree to that, but any how let’s test if we can use any kind of workaround to get text to that element.
So there is an element inside a closed shadow DOM – see the image below
Now since we cannot directly access this element using the Playwright css or text selectors, since Playwright cannot pierce closed shadow root. So what we can do is – access another element and then use the TAB key functionality to may be see if we can jump on to the element.
The nearest accessible element near this closed shadow root seems to be the div with id userPass so lets click on that.
await page.locator('div#userPass').click();
This will bring the element in focus ( I think I haven’t checked it yet though). Now let’s say we use the TAB key to see if we can go to that element.
await page.keyboard.press('Tab');
Voila. The cursor is now on that element. So that means we have switched to that element successfully. However since this is inside a closed shadow root, you cannot directly use Playwright methods like fill or type commands to send the text.
Now, what to do?
The first problem is, how to get the element since Playwright methods cannot access it. So page.locator methods are of no use here.
Javascript to the rescue. So even when Playwright cannot access this element, may be we can use the pure Javascript methods to access and make changes to the element.
Since we have already jumped to the element in the previous step using TAB , that means our element is currently the active element. So we can use
document.activeElement
method in js to access this element.
Also since we have to add a new attribute we should be using the setAttribute method, to set a text value in the value attribute . Let’s see if we can do that.
In Playwright, to use plain js, we can use the evalulate command
await page.evaluate(() => document.activeElement.setAttribute('value','Top Gun Maverick'));
Now run this code and let’s see what happens –
import { chromium,test } from "@playwright/test";
test.use({ viewport: { width: 1400, height: 1000 } });
test('Launch the Selectors hub test page',async()=>{
const browser = await chromium.launch({
headless: false
});
const context = await browser.newContext();
const page = await context.newPage();
await page.goto("https://selectorshub.com/xpath-practice-page/");
await page.waitForSelector('.dropbtn',{
state: "visible"
});
await page.locator('div#userPass').click();
await page.keyboard.press('Tab');
await page.evaluate(() => document.activeElement.setAttribute('value','Top Gun Maverick'));
await page.keyboard.press('Enter');
await page.waitForTimeout(5000);The code runs successfully, without any failure but the input box doesn’t show the value. Hmm. Let’s see if the value is being set correctly.
Add this piece of code
let textcontent = await page.evaluate(()=> document.activeElement.getAttribute('value'));
await console.log(`The value set is ${textcontent}`);
This gives a console output which shows that value is set
But the UI remains same and there is no value shown in the UI. Interesting isn’t it. I tried a couple of other methods also like setting the textContent of the element
await page.evaluate(() => document.activeElement.textContent = 'Maverick');
but same thing happens as above. The UI shows nothing and the test passes. I’ve opened a new question with the Playwright team here. Will keep this post updated on whatever the resolution is.
Edit 1 : I got a response on my question above and the solution is pretty simple. We can use the type command to sent the text input. So essentially we modified our code to this
await page.locator('div#userPass').click();
await page.keyboard.press('Tab');
await page.keyboard.type('hellowrodl');
await page.waitForTimeout(5000);
So now this scenario is also working perfectly fine. Niceee.
Scenario 5 : Element is inside a closed shadow root, which is inside an open shadow root.
Now there is one more complex scenario where there may be an element, which is inside a closed shadow root, which is in turn inside an open shadow root. See the image below on the same test page.
To solve this issue also, I tried the same approach as in Scenario 4. But same result as above.
await page.locator("#userName input#pizza").click();
await page.keyboard.press('Tab');
await page.waitForTimeout(5000);
await page.evaluate(() => document.activeElement.setAttribute('value','hawke'));
const pagetext = await page.evaluate(() => document.activeElement.getAttribute('value'));
await console.log(pagetext);
In this case also the value attribute is set but the value doesn’t show up on the UI, which means that in Scenario 4, the type of input field which is password doesn’t make any difference. I’ll track this in the github issue mentioned above and let’s see what transpires.
Edit 1 : I got a response on my question above and the solution is pretty simple. We can use the type command to sent the text input. So essentially we modified our code to this
await page.locator("#userName input#pizza").click();
await page.keyboard.press('Tab');
await page.keyboard.type('I love pizza');
await page.waitForTimeout(5000);
In the above 5 scenarios, we saw how Playwright makes it easy to interact with Shadow DOM elements. For the closed ones, I’ll try to work to see if there can be any other work around, which I think should be possible, given the power of JS. In the next blog, maybe we’ll have a look at how we can work with different scenarios in iframes.
TESTEROPS 