Practical Python: Using Code to Find a COVID19 Vaccine Opening at CVS
A few months ago when the first COVID19 vaccines came on the scene, I spent hours each day scouring the vaccination website to find appointments for my parents. I would refresh and refresh and refresh, hoping something would pop up, with openings being snagged in minutes if not seconds. Eventually I was able to schedule a vaccination for each of them (they are both fully vaccinated now — go science!) and could breathe a sigh of relief.
Now, with COVID vaccines becoming accessible to younger age groups, I want to be more prepared than I was previously so I can find appointments for my eligible loved ones quickly and efficiently. What better way to do so than to use code!
CVS has started vaccinating people at their locations around the United States and since they are convenient to us and add appointments daily, I thought this was a good website to focus my efforts on.
There were two elements I needed to figure out to accomplish this goal with Python. The first was how to scrape the information from the CVS website and the second was how to send an email to my friends and family when an opening came up. Below I highlight the two tools I used to solve these problems, with the final code noted below!
Using Selenium to Scrape Website Data
Selenium is an automation tool that allows you to interact with a web browser and mimic human behavior (opening browser, clicking, scrolling) while also scraping data. While usually I would use BeautifulSoup for my scraping projects, Selenium is great for websites that use Javascript or where, in order to reveal the information you’re interested in, you have to click a button/dropdown.
In order to use Selenium to operate your web browser, you’ll need to download the “driver” for your specific web browser. For example, I use Chrome, so I downloaded ChromeDriver. If you were to use Safari, FireFox or any other browser, you’d need to download the corresponding driver. The driver will allow you to automate your internet usage and movements. Additionally, when you’re writing your code, make sure that you include the path to your driver, meaning the step-by-step folder movements to get to where you’ve saved your driver (this will be noted in my code below).
Sending Emails Using Python
When first approaching this project, I didn’t have any experience sending emails using Python. I came across this written tutorial from Automate the Boring Stuff which was extremely helpful in setting me up with the Gmail API. If you’re planning on sending automated emails, I would suggest that you create a new Gmail account solely for the purpose of automation.
Once you have a separate gmail account, you’ll need to create a new project in the Google Cloud Platform by clicking “Select a Project” and then “New Project.”
You’ll be prompted to name the project and associate a location if you want. Once your new project is created, click on “Enable APIs and Services” and search for the Gmail API. Once this is enabled, navigate back to your dashboard and click “Create Credentials” on the upper right-hand side. You will need to create a OAuth client ID for a Desktop App which will allow you to download a JSON file of your credentials. These credentials should be kept in the same folder as your code — they are the equivalent of a username and password for your app so don’t share this file.
From here I followed the instructions on Automate the Boring Stuff (chapter 18) to automate sending emails via the Gmail API. This included running the ezgmail code which allowed me to log into my coding gmail account and give my app access. If you have any trouble going through the process, feel free to reach out to me via the comments below!
The Final Code
Below was the code I used to scrape the CVS website with Selenium and send emails. I’ve included an image of the sublime file (for color/formatting) as well as the text version below (for copy/paste capability).
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import ezgmail#Allow for headless browser option in Chrome, this means the code will run and scrape the data needed, but will not
#open a new browser window each time (I added this coede in at the end, when I was first troubleshooting, I allowed
#the browser to open each time so I could see what it was doing)
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
i = 1
#Used a while loop to keep the code from running forever, but to make sure it ran for at least 24 hrs
while i <2000:
#Gives a path to where the webdriver is saved and sets up headless browsing
driver = webdriver.Chrome('/Users/ek/Desktop/chromedriver', options=chrome_options)
#Gives the browser a link to navigate to
driver.get('https://www.cvs.com/immunizations/covid-19-vaccine?icid=cvs-home-hero1-link2-coronavirus-vaccine');
#Gives the page a few seconds to load
time.sleep(5)
#Finds and clicks the correct link based on the link text, in this case the state we're looking for,
#if you're interested in a different state just replace Massachusetts with another name
search_box = driver.find_element_by_link_text('Massachusetts').click()
#Gives more time for the link to load
time.sleep(10)
#Looks for the xpath location of the appointment availability text and qualifies it as status
status = driver.find_element_by_xpath('/html/body/div[2]/div/div[1]/div/div/div/div/div/div[1]/div[2]/div/div/div[2]/div/div[4]/div').text
#if statement that looks to see if text says no appointments are available
if status == "At this time, all appointments in Massachusetts are booked. We’ll add more as they become available. Please check back later.":
#if the text says there's no appointments it prints "Nope" to the terminal, I did this to know how many times it ran until it found a hit
print("Nope")
else:
#if there are appointments available, send an email with the details below (email, title, message)
ezgmail.send("example@gmail.com","CVS VACCINE OPENINGS","There are currently openings at CVS for vaccines in Massachusetts.")
#I printed yes when it found an opening to show the ratio of no's to yeses
print("Yes!")
#wait for awhile after you find a hit so that you're not inundating people with emails -- usually the appts go fast
time.sleep(500)
#quit out of your browser window
driver.quit()
#add one to i to continue the while loop
i += 1
#Wait two minutes before checking again, these sleep times can all be customized/changed
time.sleep(120)
