Linux Python Projects

Automating a Notification with Linux

matt

I wanted to work on the following areas in this project:

  • Formatting HTML
  • Automating emails in Linux
  • Requesting and working with data from APIs
  • Automating tasks in Linux with Cron

Scope for this project:

Create a Python emailer script that calls a public API to provide the day’s weather for my city and provides a quote of the day. Then schedule it with Cron to email it to my email every morning at 8 am.

Formatting HTML for the email template

This was the least technical aspect of this project. I started with a previously received email that had some decent basic formatting, edited the text to what I wanted in my template and then copied the HTML from it.

Next up I ran the code through this website to lay it out and make it easier to work with. I pasted it into a text file and saved it to the Templates folder as standard.html.

For the parts of the email that I wanted to be customisable I added a keyword in uppercase to be replaced in the script. These are the following elements:

  • RECIEVER for the name of the email recipient
  • LOCATION for where we will set the location of the recipient for time and weather forecast
  • DATETIME for the date and time
  • WEATHER for display of the weather forecast
  • QUOTE for quote of the day

Generating and sending the email

Not included in the screenshot I have also included the following code and imported methods to make it work:

#!/bin/python3

from datetime import datetime
import pytz
import yagmail

Generating the email

What this script does is open our standard.html email template we created earlier and swaps in the variable values for the uppercase keyword elements. We directly enter the values for the recipient name and location. The current date and time (when the script is run) is obtained by using the pytz library for time zone based on the location variable and then formatted into a string and put back in the datetime variable.

Sending the email

To send this email out I decided to keep it simple and go with the yagmail library, which uses gmail accounts to send email over SMTP.

To start I created a throwaway gmail account, set up 2FA and created an app password. I inserted the email address and app password into the script. There are also keywords for sender, subject and the email body which takes the changes made by the for loop.

Once I tested this a few times and was happy that the email was being generated correctly and being sent , I moved on to incorporating the APIs.

Requesting API data in the python script

Weather Forecast

For the daily weather forecast I went with Open-Meteo an open-source weather API with free access for non-commercial use. On their docs page they have a great tool that takes your parameters and spits out code or a chart and URL.

To use the code provided I had to install the following libraries to the virtual environment:

pip install openmeteo-requests
pip install requests-cache retry-requests numpy pandas

I copied the generated python code into my script and was able to print the weather forecast data on the terminal but as it was in the Pandas DataFrames format it needed to be converted into a format that would work in the HTML email body. A DateFrame is a 2 dimensional data structure such as an array or a table with rows and columns, typically used for data analysis in python.

To work with the DataFrame I installed another Python library called Pretty Table which converts into a formatted HTML table.

# build a table out of the dataframe
weather = build_table(daily_dataframe, 'blue_dark')

# save to html file
with open('pretty_table.html','w') as f:
    f.write(weather)

Quote of the day

For the quote of the day I went with Zen Quotes, a simple free API that fetches quotes into JSON format. I went through a number of other options, that either required elaborate registrations, required an API key or had a cost, before I stumbled across Zen Quotes.

The code for requesting the quote was quite simple and similar to the weather it displayed correctly in the terminal, however it required some work to get it into the email.

# Quote API Request Details
url = "https://zenquotes.io/api/today"

# Making http request
response = requests.post(url)
print(response.json())

# Get JSON response
data = response.json()

The API response data is formatted as a JSON array, with the following output:

  • q = quote text
  • a = author name
  • i = author image (key required)
  • c = character count
  • h = pre-formatted HTML quote

We want the ‘h’ value. After a few attempts I settled on the following code to convert it to a string and assign it to the quote variable:

# Get the value of "h" key (html) and convert it to a string
quote_list = []

# iterate over the list of dictionaries
for item in data:
    # check if any 'h' key existing in dict
    if 'h' in item:
        # Get the value of the 'h' key, convert it to string and append
        quote_list.append(str(item['h']))

# Convert the list into a string
quote = ' '.join(quote_list)
print(quote)

This completed my work on the script. Now to automate it to send out every day at 8am.

Automating running the script with a cronjob and creating a virtual environment

Cron is a well known and popular task scheduler for Linux. Cron was installed on my version of Ubuntu so I didn’t have to install it.

Cron jobs are commands, or in our case, a shell script, that are referenced in crontab files. These files are loaded and then monitored to see if they need to be executed.

Before setting up the cron job I decided to set up a virtual environment for the script to run in cron with all of the necessary packages and libraries. This helps avoids issues with different versions of python, keeps it isolated with only the dependencies and libraries that it needs, and keeps it simple without needing to worry about system-wide settings.

To set up the virtual environment and download the necessary packages I used the following code:

python3 -m venv ~/venv

source ~/venv/bin/activate

pip install zoneinfo pytz yagmail openmeteo_requests pandas retry_requests pretty_html_table requests_cache requests

I also had to update the shebang line in my script with the following:

#!/home/matt/venv/bin/python3

To schedule the automated email script in the cron job configuration file I first had to ensure the script was executable:

chmod +x /filepath/

Then I opened the cron tab configuration file with:

sudo crontab -e

See the above image for the scheduled cron job. The <0 8 * * *> means the job will run at 8 am every day, the next file path refers to the virtual environment and then second is the location of the script.

Now we finally have a scheduled email notification with weather and a quote each morning 🙂

This was my largest project to date with a lot of moving parts and gave me the opportunity to learn a bit about a lot of different things and gave me a lot of ideas about what skills I want to further develop.

The full code for this project can be found on my Github.

Similar Posts