24/7 Online Requests with CircuitPython

Last updated August 25, 2024

Save

Basic WiFi Setup
Settings.toml Credentials
Basics of Try/Except
wifi.radio.connected
Exception Handling Types
Multiple Excepts
Conditional Clauses
Request Error Handling
OSError errno
DNS Exception Handler (gaierror -2)
Traceback Module
Request Example Script:

Getting Started

Adafruit Playground is a wonderful and safe place to share your interests with Adafruit's vibrant community of makers and doers. Have a cool project you are working on? Have a bit of code that you think others will find useful? Want to show off your electronics workbench? You have come to the right place.

The goal of Adafruit Playground is to make it as simple as possible to share your work. On the Adafruit Playground users can create Notes. A note is a single-page space where you can document your topic using Adafruit's easy-to-use editor. Notes are like Guides on the Adafruit Learning System but guides are high-fidelity content curated and maintained by Adafuit. Notes are whatever you want them to be. Have fun and be kind.

Click here to learn more about Adafruit Playground and how to get started.

There might come a time when you want a project running 24/7 permanently online. Weather and Real Time Clock (RTC) updates are common uses. It's not enough to code a request to an API, you must know how to avoid every possible scenario that might crash your script. That's where error exception handlers come in.

Learning how to use exception handlers is the most important aspect of an always online script.

If you're a beginner please read Networking in Circuit Python then Web API's & You first. 24/7 Online Requests is a follow up guide for intermediate level Circuit Python coders. Some parts of this guide infer you already know the basics for getting your board online & interacting with Web API's.

Example of a script crashing on a TFT display (that I wrote) due to inadequate error handling.

In Circuit Python error handlers are based on a try/except paradigm. You'll often hear programmers refer to it as try/throw/catch for catching errors but that's referencing other languages such as C++ or Java. That terminology is fine to use when discussing help with errors. Just know in Circuit Python the syntax is try/except for the purposes of online API scripts.

Basic WiFi Setup

Let's say we want to connect an Adafruit ESP32-S3 Feather to WiFi.

Download File

Copy Code

import os
import adafruit_connection_manager
import wifi
import adafruit_requests

# Initalize Wifi, Socket Pool, Request Session
pool = adafruit_connection_manager.get_radio_socketpool(wifi.radio)
SSL_CONTEXT = adafruit_connection_manager.get_radio_ssl_context(wifi.radio)
# adafruit_requests.Session keep outside the main loop
# otherwise you get Out of Socket errors.
requests = adafruit_requests.Session(pool, SSL_CONTEXT)

# Use settings.toml for credentials
ssid = os.getenv("CIRCUITPY_WIFI_SSID")
password = os.getenv("CIRCUITPY_WIFI_PASSWORD")

# Connect to WiFi
wifi.radio.connect(ssid, password)

import os
import adafruit_connection_manager
import wifi
import adafruit_requests

# Initalize Wifi, Socket Pool, Request Session
pool = adafruit_connection_manager.get_radio_socketpool(wifi.radio)
SSL_CONTEXT = adafruit_connection_manager.get_radio_ssl_context(wifi.radio)
# adafruit_requests.Session keep outside the main loop
# otherwise you get Out of Socket errors.
requests = adafruit_requests.Session(pool, SSL_CONTEXT)

# Use settings.toml for credentials
ssid = os.getenv("CIRCUITPY_WIFI_SSID")
password = os.getenv("CIRCUITPY_WIFI_PASSWORD")

# Connect to WiFi
wifi.radio.connect(ssid, password)

Settings.toml Credentials

Copy Code

# Ensure your settings.toml has the following

CIRCUITPY_WIFI_SSID = "YourSSID"
CIRCUITPY_WIFI_PASSWORD = "YourPassword"

# Ensure your settings.toml has the following

CIRCUITPY_WIFI_SSID = "YourSSID"
CIRCUITPY_WIFI_PASSWORD = "YourPassword"

The above code is as basic as it gets for attempting to connect to WiFi but what happens if your WiFi access point is down or the credentials are wrong? Your script will cease to function and will crash with an error to a console. The error could be any number of possible errors such as out of sockets, out of retries, failure to get host, and so on.

Basics of Try/Except

By wrapping the WiFi connection request in a try/except, instead of crashing to REPL, it opens some important options which include (but not limited to):

print the error to serial console
ignore the error completely
retry from where try was initiated
reload code.py
restart the microcontroller

Copy Code

try:
	wifi.radio.connect(ssid, password)
except ConnectionError as e:
	print(f"❌ Connection Error: {e}")

try:
	wifi.radio.connect(ssid, password)
except ConnectionError as e:
	print(f"❌ Connection Error: {e}")

wifi.radio.connected

wifi.radio.connected is a function in the wifi library that knows if it is connected to a WiFi access point or not. It will return a boolean either True or False. We can use this "status" to know if we should attempt to reconnect or not anytime in the script.

By moving the WiFi connection attempt into the while True loop you can force it to continually check if it's connected... infinitely. The script will only continue past an exception if there is no error and a successful WiFi connection is made. In this example if you see the message

📡 Wifi!

then the connection was successful and the script will continue.

Emoji support is built into the Mu code editor and Mu REPL/Console. Emoji's are an excellent way to make certain print statements really stand out. Unfortunately not all code editors or terminal programs have emoji support such as PyCharm, VSCode, Sublime, Nano, TIO, etc..

Download File

Copy Code

while True:
	while not wifi.radio.connected:
		try:
			wifi.radio.connect(ssid, password)
		except ConnectionError as e:
			print(f"❌ Connection Error: {e}")
    print("📡 Wifi!")

while True:
	while not wifi.radio.connected:
		try:
			wifi.radio.connect(ssid, password)
		except ConnectionError as e:
			print(f"❌ Connection Error: {e}")
    print("📡 Wifi!")

Python & Circuit Python now encourages the use of f-strings instead of the older .format method. You can learn more about f-string formatting here: https://docs.python.org/3/tutorial/inputoutput.html e is a shorthand variable name for error. It's not an official variable simply one that is commonly used in Circuit Python scripts. You can name e whatever you want.

Exception Handling Types

There are many different error types. Sometimes they can be unique to a specific library. There are global error handlers built into Circuit Python such as:

ConnectionError
KeyError
OSError
RuntimeError
ValueError
and many more...

The full list of Circuit Python Exceptions are documented here.

It's perfectly fine to use multiple error handlers at the same time. For example:

Download File

Copy Code

while True:
	while not wifi.radio.connected:
		try:
			wifi.radio.connect(ssid, password)
		except (OSError, RuntimeError, ValueError, ConnectionError) as e:
			print(f"❌ Error: {e}")
    print("📡 Wifi!")

while True:
	while not wifi.radio.connected:
		try:
			wifi.radio.connect(ssid, password)
		except (OSError, RuntimeError, ValueError, ConnectionError) as e:
			print(f"❌ Error: {e}")
    print("📡 Wifi!")

Using every exception handler is a shotgun approach. The downside is it will not print the line the error spawned from or which error handler it came from. It will only print the error message. Sometimes that's a good thing, sometimes that's a bad thing.

Multiple Excepts

It is also possible to use multiple except statements with only 1 try. This is a way you can split up errors to be more specific which handler is being triggered even if it doesn't print the line number.

Download File

Copy Code

while True:
	while not wifi.radio.connected:
		try:
			wifi.radio.connect(ssid, password)
		except OSError as e:
			print(f"❌ OSError: {e}")
        except RuntimeError as e:
			print(f"❌ RuntimeError: {e}")
        except (ValueError, ConnectionError) as e:
			print(f"❌ Value or Connection Error: {e}")
    print("📡 Wifi!")

while True:
	while not wifi.radio.connected:
		try:
			wifi.radio.connect(ssid, password)
		except OSError as e:
			print(f"❌ OSError: {e}")
        except RuntimeError as e:
			print(f"❌ RuntimeError: {e}")
        except (ValueError, ConnectionError) as e:
			print(f"❌ Value or Connection Error: {e}")
    print("📡 Wifi!")

Notice parenthesis are only needed if you use multiple error handlers with a single except statement.

Let's say in a hypothetical circumstance that you will never get a RuntimeError during wifi.radio.connect. If an error handler will never be triggered then it's pointless to add it to your except statement. You'll eventually figure out which error handlers are best suited for different situations. How? By running your script until it fails. Every time you encounter a new error you add it to the list of errors to check for. This helps keep your script more efficient because every character of code costs against RAM usage. The less code you write, the less RAM you use. If you have a board with a ton of RAM to spare then absolutely, sure, you can add every exception handler in the list. Some lower power boards that can barely run Circuit Python do not have much RAM overhead. Being efficient with your code is a great habit to cultivate.

Conditional Clauses

Clauses are used within a while, for, or try loop. Since an always online script will always be within a loop these can be used at any time. You will often see try/except used with conditional clauses.

pass (does nothing and acts only as a placeholder while coding a new function)
break (breaks to try or beginning of script)
continue (continues to retry the try until successful)

For more information on conditional clauses please refer to Python Documentation Section 4.4

Here is an example use of break within an exception:

Download File

Copy Code

while True:
	while not wifi.radio.connected:
    	try:
			wifi.radio.connect(ssid, password)
		except ConnectionError as e:
			print(f"❌ Connection Error: {e}")
        	break  # this will start the script over from the beginning
    print("📡 Wifi!")

while True:
	while not wifi.radio.connected:
    	try:
			wifi.radio.connect(ssid, password)
		except ConnectionError as e:
			print(f"❌ Connection Error: {e}")
        	break  # this will start the script over from the beginning
    print("📡 Wifi!")

Request Error Handling

Now let's say we've connected to WiFi without any errors. Next let's do a get request to an API. The SOURCE will be a url that you want to access, typically one that is formatted with a .json extension or file type.

Download File

Copy Code

SOURCE = "https://httpbin.org/get"
HEADER = {"user-agent": "blinka/1.0.0"}

while True:
	while not wifi.radio.connected:
		try:
			wifi.radio.connect(ssid, password)
		except (OSError, RuntimeError, ValueError, ConnectionError) as e:
			print(f"❌ Error: {e}")
    print("📡 Wifi!")
    
	while wifi.radio.connected:
		try:
			with requests.get(url=SOURCE, headers=HEADER) as response:
				response_json = response.json()
		except KeyError as e:
			print(f"❌ Key Error: {e}")

SOURCE = "https://httpbin.org/get"
HEADER = {"user-agent": "blinka/1.0.0"}

while True:
	while not wifi.radio.connected:
		try:
			wifi.radio.connect(ssid, password)
		except (OSError, RuntimeError, ValueError, ConnectionError) as e:
			print(f"❌ Error: {e}")
    print("📡 Wifi!")
    
	while wifi.radio.connected:
		try:
			with requests.get(url=SOURCE, headers=HEADER) as response:
				response_json = response.json()
		except KeyError as e:
			print(f"❌ Key Error: {e}")

Using a try/except around the request in the same manner as the wifi connection. If a GET request fails it has its own exception handler. Splitting up exception handlers allows you to make more modular code. Using print statements for debugging will show which section an error is being thrown from. With a GET request a KeyError is a json formatting error.

I'm walking you through each step of building up try/except layers intentionally so when you see the final script it won't be intimidating at all.

The small example above is straight forward to understand. If your script becomes 1000 lines long you will have many nested try/excepts. The most important try/except is the first one that encompasses everything from directly below while True and ends at the very bottom of the entire script, so no matter what happens (in theory) any error that might slip through other error handlers will be caught at the end, preventing a crash, and allowing your script to retry from the beginning... forever.

OSError errno

errno is a subset of the CPython implementation where you can catch very specifc error numbers from an error handler. More information is available in the Circuit Python Documentation on errno.

Thanks to Deshipu in a discord discussion for figuring out how to catch and handle gaierror -2 specifically.

Download File

Copy Code

while wifi.radio.connected:
	try:
		with requests.get(url=SOURCE, headers=HEADER) as response:
			response_json = response.json()
	except (OSError, KeyError) as e:
		if e.errno == -2:
			print(f"❌ gaierror (DNS failure), waiting to hard reset {e}")
			time.sleep(240)
			microcontroller.reset()
		print(f"❌ Key Error {e}")

while wifi.radio.connected:
	try:
		with requests.get(url=SOURCE, headers=HEADER) as response:
			response_json = response.json()
	except (OSError, KeyError) as e:
		if e.errno == -2:
			print(f"❌ gaierror (DNS failure), waiting to hard reset {e}")
			time.sleep(240)
			microcontroller.reset()
		print(f"❌ Key Error {e}")

DNS Exception Handler (gaierror -2)

The OSError gaierror -2 is a quirky error in Circuit Python that stems from an upstream implementation of the ESP-IDF SDK. In my experience it can randomly occur when DNS resolution fails to resolve a host name. This is typically due to WiFi failing in the middle of a script because your router lost connection to the ISP, your DNS resolver failed, a failing ISP hop, or the host server is having an issue. You can catch the DNS error specifically using errno. The gaierror is an OSError you'll encounter seemingly randomly due to issues beyond your control.

The error message associated with gaierror -2 is:

Name or service not known

My personal preference to combat a gaierror is to force the script to sleep for a period of time. This provides time for whatever external issue is causing it, to sort itself out. Then hard reset the controller just in case the issue is wifi.radio socket related. It will take a while for your board to recover and reconnect with this method but the point is that it will recover instead of crashing to console.

Traceback Module

Traceback is a more advanced debugging helper and core module that is customizable and beyond the scope of this playground note. Here is the documentation on Traceback. If you become frustrated with exceptions (especially since they don't show the line # of the error) and want to print more verbose errors then look into the traceback module.

Request Example Script:

Download File

Copy Code

import os
import time
import microcontroller
import adafruit_connection_manager
import wifi
import adafruit_requests

# Initalize Wifi, Socket Pool, Request Session
pool = adafruit_connection_manager.get_radio_socketpool(wifi.radio)
SSL_CONTEXT = adafruit_connection_manager.get_radio_ssl_context(wifi.radio)
# adafruit_requests.Session keep outside the main loop
# otherwise you get Out of Socket errors.
requests = adafruit_requests.Session(pool, SSL_CONTEXT)

# Use settings.toml for credentials
ssid = os.getenv("CIRCUITPY_WIFI_SSID")
password = os.getenv("CIRCUITPY_WIFI_PASSWORD")

SOURCE = "https://httpbin.org/get"
HEADER = {"user-agent": "blinka/1.0.0"}

while True:
  # Connect to Wi-Fi
  print("\nConnecting to WiFi...")
  while not wifi.radio.connected:
    try:
      wifi.radio.connect(ssid, password)
    except ConnectionError as e:
      print(f"❌ Connection Error: {e}")
      time.sleep(60)
  print("📡 Wifi!")

  while wifi.radio.connected:
    try:
      
      try:
        with requests.get(url=SOURCE, headers=HEADER) as response:
          response_json = response.json()
      # Request error catcher. If script fails during request.
      except (OSError, KeyError) as e:
        if e.errno == -2:
          print(f"❌ gaierror (DNS failure), waiting to hard reset {e}")
          time.sleep(240)
          microcontroller.reset()
        print(f"❌ Key Error: {e}")
        time.sleep(60)
        
      # Your online API section goes here
      response_headers = response.headers
      print(f"Response Headers: {response_headers}")
      
    # General error catcher. If script fails after a successful request.
    except (OSError, ValueError, RuntimeError) as e:
      print(f"❌ General Exception Error: {e}")
      time.sleep(60)
      break  # breaks to while True to re-check WiFi status
    time.sleep(300)  # If everything succeeds, time until next poll update

import os
import time
import microcontroller
import adafruit_connection_manager
import wifi
import adafruit_requests

# Initalize Wifi, Socket Pool, Request Session
pool = adafruit_connection_manager.get_radio_socketpool(wifi.radio)
SSL_CONTEXT = adafruit_connection_manager.get_radio_ssl_context(wifi.radio)
# adafruit_requests.Session keep outside the main loop
# otherwise you get Out of Socket errors.
requests = adafruit_requests.Session(pool, SSL_CONTEXT)

# Use settings.toml for credentials
ssid = os.getenv("CIRCUITPY_WIFI_SSID")
password = os.getenv("CIRCUITPY_WIFI_PASSWORD")

SOURCE = "https://httpbin.org/get"
HEADER = {"user-agent": "blinka/1.0.0"}

while True:
  # Connect to Wi-Fi
  print("\nConnecting to WiFi...")
  while not wifi.radio.connected:
    try:
      wifi.radio.connect(ssid, password)
    except ConnectionError as e:
      print(f"❌ Connection Error: {e}")
      time.sleep(60)
  print("📡 Wifi!")

  while wifi.radio.connected:
    try:
      
      try:
        with requests.get(url=SOURCE, headers=HEADER) as response:
          response_json = response.json()
      # Request error catcher. If script fails during request.
      except (OSError, KeyError) as e:
        if e.errno == -2:
          print(f"❌ gaierror (DNS failure), waiting to hard reset {e}")
          time.sleep(240)
          microcontroller.reset()
        print(f"❌ Key Error: {e}")
        time.sleep(60)
        
      # Your online API section goes here
      response_headers = response.headers
      print(f"Response Headers: {response_headers}")
      
    # General error catcher. If script fails after a successful request.
    except (OSError, ValueError, RuntimeError) as e:
      print(f"❌ General Exception Error: {e}")
      time.sleep(60)
      break  # breaks to while True to re-check WiFi status
    time.sleep(300)  # If everything succeeds, time until next poll update

This is a basic boilerplate that I use in most of my online API scripts to start with. All you have to do is provide the SOURCE, HEADER, and any API calls to the host in the relevant section.

Hopefully you now have a better understanding of what it takes to keep a script online 24/7 with Circuit Python. It's simply a matter of being able to control what happens when an error occurs so you can permanently prevent your script from crashing.

This page (24/7 Online Requests with CircuitPython) was last updated on August 25, 2024.