JavaScript pop-ups and alerts are annoying enough for humans - for automated web scraping, they can block scripts and bring everything to a stop.
In this section, we’ll learn basic codes for pop-up and alert handling in Selenium, helping to eliminate them in your automated web scraping.
Thankfully, alert handling in Selenium is relatively easy, and just needs a few lines of code:
alert = Alert(driver)
alert.accept() # To accept the alert
except Exception as e:
print("No alert found.")
However, as we’ve explained in the previous section, sometimes it’s important to implement waits in Selenium. This is also often true with alert handling in Selenium, as some pop-ups might not appear right away.
If this is the case, adding a wait is one of the simplest and most effective Selenium alert methods. This will ensure the alert has time to trigger before we move on to the previous Selenium alert handling.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.alert import Alert
# Wait for the alert to appear (up to 10 seconds)
wait = WebDriverWait(driver, 10)
try:
wait.until(EC.alert_is_present())
alert = driver.switch_to.alert
alert.accept() # Accept the alert
print("Alert accepted.")
except Exception as e:
print(f"No alert found or timed out: {e}")
Note: You can change the wait time here to adjust to the respective websites you are scraping.
Now in 99% of cases, you’re going to want to close the pop-up, but we want to be thorough, so let’s first look at how to switch to a pop-up window.
# Store the main window handle
main_window = driver.current_window_handle
# Click something that opens a pop-up
driver.find_element(By.ID, 'open-popup').click()
# Switch to the new window
for handle in driver.window_handles:
if handle != main_window:
driver.switch_to.window(handle)
print("Switched to pop-up window.")
break
# Close the pop-up and switch back
driver.close()
driver.switch_to.window(main_window)
Most of the time, pop-ups are a nuisance so let’s just close them. This is a simple case of identifying the pop-up, identifying the close_button and clicking it. In this case, we’re also using WebDriverWait to ensure we give time for the pop-ups to appear.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
try:
# Wait for the pop-up close button to appear
wait = WebDriverWait(driver, 10)
close_button = wait.until(EC.element_to_be_clickable((By.CLASS_NAME, 'popup-close')))
close_button.click()
print("Pop-up closed.")
except Exception:
print("No pop-up found.")
If you prefer to be really thorough, the following code can help resolve pop-ups. It resolves not only new window pop-ups, but also in-page modal alternatives as well. When scraping dynamic websites, for example, it’s worth checking for both versions.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# Handle pop-up windows
main_window = driver.current_window_handle
try:
driver.find_element(By.ID, 'open-popup').click()
WebDriverWait(driver, 10).until(lambda d: len(d.window_handles) > 1)
for handle in driver.window_handles:
if handle != main_window:
driver.switch_to.window(handle)
print("Switched to pop-up window.")
driver.close() # Close pop-up window
break
driver.switch_to.window(main_window) # Switch back
print("Returned to main window.")
except Exception:
print("No pop-up window found.")
# Handle in-page modal pop-ups
try:
wait = WebDriverWait(driver, 10)
close_button = wait.until(EC.element_to_be_clickable((By.CLASS_NAME, 'popup-close')))
close_button.click()
print("In-page pop-up closed.")
except Exception:
print("No in-page pop-up found.")
Our community is here to support your growth, so why wait? Join now and let’s build together!