Running Two Programs on a Single CircuitPython Device

I have a nice e-ink display on my shelf running a CircuitPython program that updates the display every fifteen minutes. Recently I wrote a second program that would also just be ideal for this display.

Thinking about my options, I could either

• copy the relevant program to the MCU everytime I want to switch
• write a wrapper application for both programs

The first option was not attractive, because I want to switch on a regular basis. The second option also has it's problems. Both existing programs are valid stand-alone and are maintained in their own Github-repositories, so adding a wrapper would make changes to the software much more complicated.

So I was thinking about a sort of "boot-switcher". Users running Linux on their PCs often have a boot-manager that allows them to switch to more obscure operating systems once in a while, and I wondered if there is a simple way to use this concept also for a CircuitPython device.

Hardware Setup

Since a MCU neither has a BIOS that can start a boot-manager before the real OS, nor (usually) a mouse or keyboard attached for the selection of the target program, this solution needs a free GPIO and a slider or button.

Our "boot-manager" is the code in boot.py. This code reads the state of the GPIO and starts one of the programs depending on the value. Using a slider will result in a "sticky" behavior: unless you flip it again, each reset will start the same program. The button is different, since you have to keep it pressed during power-on/reset to start the second program. So the second program is more of an exception while the first program runs as the default.

Software Setup

For the boot-switcher to work, the programs can't be deployed to the top-level directory of the CIRCUITPY-drive. Instead, the layout on the drive is like this:

So every application moves one level down into its own directory.

You can find the necessary boot.py in the Github-repo https://github.com/bablokb/cp-bootswitcher. In this file, you have to configure your setup, e.g.

PIN_SWITCH = board.D1
APP_NAMES  = ["_app0","_app1"]
SHARED_FILES = ["boot.py"]+APP_NAMES

The names of the application directories don't matter, but you have to add the names to APP_NAMES. You can also add more files/directories to SHARED_FILES, but all of these entries must be top-level. One use case for this is if your applications share the lib-folder.

How does it Work?

The implementation of the boot-switcher turned out to be simple and during power-on or reset you won't notice a delay. The code in boot.py only checks the state of the GPIO and then moves all files from the top-level back to the appropriate application folder and then all files from the other application folder to the toplevel. Moving is a fast operation, since it only changes the entry in the directory-table (FAT), so moving a very large lib-folder won't be slower than moving code.py.

Other Use-Cases

There are a number of other use-cases for the boot-switcher, e.g.

• providing two versions of an application, e.g. a beta version in addition to a productive version
• providing a special diagnostic application in addition to the normal application
• running the same application with two different configurations (e.g. within two different WLAN-networks)
• having two sets of libraries e.g. for CP 8.x and 9.x

Final Notes

If you only want to start different top-level programs and otherwise share the rest of your code and all settings, you don't need the boot-switcher. You can do this with a few lines of code, e.g.:

import supervisor
...
supervisor.set_next_code_file("admin.py",sticky_on_reload=True)
supervisor.reload()

Code within boot.py can do many things for you. For example, you can make the CIRCUITPY drive writable for your program or hide it for the PC. Of course you can also shoot yourself in the foot, so reading the documentation is recommended.

When playing with synthesizer patches, it helps to have a MIDI sequencer to generate note triggers for you. One option is to use a DAW on a laptop, like Ableton or whatever. But, it can also be nice to go DAWless with a hardware sequencer, because buttons and knobs and blinking lights are fun. With DIY sequencer projects in mind, I wrote a plaintext music notation sequencer module for CircuitPython, called txtseq.

The sequencer can read a song from a text file on your CIRCUITPY drive, parse the music notation into an array of MIDI note on and off events, then play the MIDI events in an event loop (allowing time to run other code). The music notation is loosely based on a subset of the abc standard. Notation for note pitch, accidentals, octave, and duration is very similar to abc. For everything else, the sequencer uses a simpler grammar and syntax that is easy to parse on a microcontroller.

CircuitPython + GarageBand Audio Demo

In my GitHub repository for the sequencer, the demonstration track I wrote is named track1.txt. To hear what the commit 9c73ec4 version of track1.txt sounds like when played from a Trinket M0 over USB MIDI into a MIDI drum instrument in GarageBand, you can listen to demos/track1-180bpm.mp3.

How to Run the Code

I've been testing this with CircuitPython 9.0.5 on a Trinket M0 (SAMD21), but most of the code (all but MIDI out) also runs on desktop python3.

CircuitPython Version

1. Prepare a host computer with something that can play sounds for incoming USB MIDI notes on channels 10, 11, 12, and 13. For example, on macOS, you can use the GarageBand app by adding a MIDI track to an empty project, then selecting a drum kit as the MIDI track's instrument.

2. Update CircuitPython and bootloader the normal way. (no additional libraries are needed)

3. Fetch a local copy of txtseq using git clone or by downloading a release archive.

4. Copy the txtseq/txtseq directory to your CIRCUITPY drive (see docs at Welcome to CircuitPython! > The CIRCUITPY Drive)

5. Copy txtseq/code.py, txtseq/boot.py, and txtseq/track1.txt to your CIRCUITPY drive

When code.py runs, it will parse music notation from track1.txt into an array of MIDI note event data, then start playing the notes over USB MIDI. The parser and playback code print a variety of debug info to the serial console to help with measuring memory and CPU use along with MIDI playback latency.

Desktop Version

This will give debug prints only, without actual MIDI playback. But, you could easily modify the code to use a library that is capable of sending MIDI. (see definition of midi_tx(data) callback function in txtseq/__main__.py)

1. Clone the txtseq repo

2. cd txtseq

3. python3 -m txtseq track1.txt

Example Output

This is from running code.py on a Trinket M0 with CircuitPython 9.0.5:

10: ppb=4
11: bpm=180
19: 1 ................
20: 1 ...................
21: 1 ...............................
22: 1 ................
23: 1 ...................
24: 1 ......................
[parse time: 517 ms]

00009924 00078924 0010992A 0017892A 00189924 001F8924 0028992E 002F892E 00309928 00378928
0040992A 0047892A 00489924 004F8924 0058992E 005F892E 00609924 00678924 0070992A 0077892A
00789924 007F8924 0088992E 008F892E 00909928 00978928 00A0992A 00A7892A 00A89924 00AF8924
00B8992E 00BF892E 00C09924 00C78924 00D0992A 00D7892A 00D89924 00DF8924 00E8992E 00EF892E
00F09928 00F78928 0100992A 0107892A 01089924 010F8924 0118992E 011F892E 01209924 0120992E
01278924 0127892E 0128992A 012F892A 0130992E 0137892E 01389924 013F8924 0140992E 0147892E
0148992A 014F892A 01509928 01578928 0160992A 0167892A 01689924 016F8924 01789924 017F8924
01809924 01838924 01849924 01878924 01889924 018B8924 018C9924 018F8924 01909924 01938924
01949924 01978924 01989924 019B8924 019C9924 019F8924 01A09924 01A38924 01A49924 01A78924
01A89924 01AB8924 01B09933 01B38933 01B49933 01B78933 01B89933 01BB8933 01BC9933 01BF8933
01C09933 01C38933 01C49933 01C78933 01C89933 01CB8933 01CC9933 01CF8933 01D09933 01D38933
01D49933 01D78933 01DC9924 01DF8924 01E09924 01E38924 01E49924 01E78924 01E89924 01EB8924
01EC9924 01EF8924 01F09924 01F38924 01F49924 01F78924 01F89924 01FB8924 01FC9924 01FF8924
02049931 020F8931 02109924 02178924 0220992A 0227892A 02289924 022F8924 0238992E 023F892E
02409928 02478928 0250992A 0257892A 02589924 025F8924 0268992E 026F892E 02709924 02778924
0280992A 0287892A 02889924 028F8924 0298992E 029F892E 02A09928 02A78928 02B0992A 02B7892A
02B89924 02BF8924 02C8992E 02CF892E 02D09924 02D78924 02E0992A 02E7892A 02E89924 02EF8924
02F8992E 02FF892E 03009928 03078928 0310992A 0317892A 03189924 031F8924 0328992E 032F892E
03309924 0330992E 03378924 0337892E 0338992A 033F892A 0340992E 0347892E 03489924 034F8924
0350992E 0357892E 0358992A 035F892A 03609928 03678928 0370992A 0377892A 03789924 037F8924
03889924 038F8924 03909924 03938924 03949924 03978924 03989924 039B8924 039C9924 039F8924
03A09924 03A38924 03A49924 03A78924 03A89924 03AB8924 03AC9924 03AF8924 03B09924 03B38924
03B49924 03B78924 03B89924 03BB8924 03C09933 03C38933 03C49933 03C78933 03C89933 03CB8933
03CC9933 03CF8933 03D09933 03D38933 03D49933 03D78933 03D89933 03DB8933 03DC9933 03DF8933
03E09933 03E38933 03E49933 03E78933 03F09931 03FB8931
[midi event dump time: 191 ms]

mem_free: 10976 10880 9728   diffs: 96 1152

Playing on USB MIDI ch10-13...
Done

1. The top section has debug print output from the parsing functions.

2. The second section is a dump of the array of timestamped midi events created by the note parsing code.

3. The third section summarizes mem_free() measurements. (see code.py)

4. The last section has debug prints from the MIDI event player.

Reading the Code

This section is for people who are interested in how the sequencer works. It's kind of technical, and I used a variety of optimization tricks to get it to fit and run well on a non-Express SAMD21 board.

1. The txtseq module exports a function, sequencer(f) (see txtseq/sequencer.py) which expects its argument, f, to be a binary mode file object (e.g. open('some-song.txt', 'rb')). For usage examples, see code.py or txtseq/__main__.py.

2. The seqencer() function parses its input file to find top level commands, which it then uses to call further parsing functions. For example, the commands 1, 2, 3, and 4 call the p_staff() function defined in txtseq/staff.py. The numbers correspond to each of the four voices (tracks). Note on and off events generated by the parser get packed as uint32 values in an array.array('L'). The most significant 16 bits have a timestamp (units of MIDI pulses). The low 16 bits have MIDI status and data bytes. This allows for adding note events one voice at a time without worrying about out-of-order events. I sort the array at the end to merge all the events from different voices into one list ordered by ascending timestamps.

3. To get all the parser code to compile and run on a SAMD21, the module is split into several files of less than 150 lines each. Also, I used several MicroPython optimization techniques from Damien George's 2018 PyCon AU talk, "Writing fast and efficient MicroPython".

4. The txtseq/util.py file holds parsing functions for dealing with comments (# ...), semantically irrelevant whitespace, and setting of header options (B for bpm, U for time unit)

5. Parsing of note pitch and duration for staff lines happens in the p_staff() function of txtseq/staff.py. The parsing style is based on state machine loops that examine one byte at a time, reading bytes with readinto() to limit heap allocations. When one of the parser functions or state machine branches recognizes a byte that should be processed by a different function or state branch, it will rewind the file's cursor position by one byte using the f.seek(mark) idiom.

6. Playback uses a generator defined in txtseq/player.py. By using the generator as the iterator for an event loop, it's easy to run your own code interleaved with the MIDI player (see main() in code.py). Also, holding playback state in the local scope of a generator makes it possible to avoid many heap allocations and dictionary lookups that would add jitter and latency if the MIDI player used a class instance.

Music Notation Grammar and Syntax

This section is for people who want to write songs for the sequencer to play as MIDI events.

For examples of how the plaintext music notation works, check out the comments in txtseq/track1.txt.

The ASCII note transcription style used here is loosely based on the abc music standard, but the two notations are not compatible. In particular, this notation uses {} for chords, requires chord durations to be specified after the closing }, and omits a lot of abc's features such as configurable key signature.

The short summary:

• Single note: <accidental><pitch><octave><duration> (e.g. C _B, c2)

• Rest: z<duration> (e.g. z z2 z16)

• Chord note: <accidental><pitch><octave>

• Chord: {<chord note><chord note>...}<duration> (e.g. {C^DA}4 {ceg})

• Accidental: _ (flat), ^ (sharp), or the empty string (natural)

• Note: C D E F G A B c d e f g a b (C is middle-c, c is 1 octave up)

• Note aliases: you can use some shorter aliases to write percussion parts: h for ^F _G (closed hi-hat), H for _B ^A (open hi-hat), and r for _e ^d (ride cymbal)

• Octave: , (lower by 1 octave), ' (raise by one octave), repeated commas or single-quotes are cumulative (,, lowers 2 octaves, ''' raises by 3). Examples: C,,, d'

• Duration: An integer representing the length of a note or chord as a multiple of the current time unit. Duration is optional with a default value of 1. With the time unit set for 1/8, C2 would mean a quarter note of middle-c, and C would be an eighth note.

• Staff: staff lines start with a voice number then have an arbitrary sequence of whitespace, bar lines (|), notes, and chords. Bar lines and whitespace are ignored by the parser, but you can use them to help organize your notes for better readability. It's fine to make long notes that last for more than one measure (e.g. C,16 with time unit set to 1/8 would be played the same as 2 tied whole notes in 4/4 time)

Example: 2 | {CDG}4 {ACD}4 | C2 C2 D2 G2 |

For more examples, see txtseq/track1.txt

Setting BPM and Time Unit

The B command sets bpm, which is a global setting that gets applied during playback. For example, a line with B 120 would set the playback speed to 120 beats per minute.

The U command sets the time unit, which relates to the duration numbers. For example, You can set the time unit to 1 eighth note with U 1/8 command. In that case, if you wrote C2 in a staff, that note's duration would be 2 eighth notes (1 quarter note). Or, for quarter note triplets, you could use the U 1/4T command. In that case, the duration of a C2 would be 2/3 of one quarter note.

The time unit options are: 1/4, 1/8, 1/16, 1/32, 1/4T, 1/8T, 1/16T, and 1/32T

Bass Clef Percussion Notes on Voice 1

The General MIDI standard includes a mapping of percussion sounds for MIDI channel 10, notes 35 to 81. The sounds for a typical Western drum kit (kick, snare, hi-hat, cymbal, etc) use notes in the bass clef, starting approximately 2 octaves below middle C. To avoid having to write things like B,,, or C,, every time you want a kick drum, voice 1 uses bass clef note names.

Bass clef note names are 2 octaves (24 MIDI notes) lower than the equivalent treble clef note names. So, bass clef C is the same MIDI note as C,, in treble clef.

Also, because the hi-hats and ride cymbal would otherwise need to be spelled with a flat or sharp prefix, I included the aliases h, H, and r. For quick reference, these are the notes for some common drum sounds:

Priority Scheduling of MIDI Messages

For a MIDI link that can move 1 message per 1 ms, playing a chord of 3 notes would take 3 ms to send. If the chord was meant to play on the same beat as a drum strike and a CC update, the whole group would take 5 ms to send. But, if the drum messages get sent first, the timing will sound tighter.

Considering that low latency matters more for percussion, giving scheduling priority to messages for the percussion (usually MIDI channel 10) should help to make the most of available MIDI bandwidth.

To allow for efficient percussion-priority sorting of MIDI events, I hardcoded the player to use the following txtseq voice to MIDI channel mapping:

I've been spending a bunch of time testing code across different MCUs and networking chips. It started with my work with Connection Manager. Then I was digging into Requests and MiniMQTT . Which of course lead me to places like NTP, AzureIoT and AWS_IOT.

As I jumped around, I needed a way to always get the right radio. Sometimes it's native WiFi, which is easy: wifi.raidio, but other times, it's some SPI based FeatherWing or maybe a M4 with and AirLift (talking to you PyPortal and MatrixPortal M4)...

Introducing: get_radio

It's not perfect, and I'm guessing there are things I haven't thought of, but it works for me so far.

The only thing a normal developer, with the right libraries installed and only one radio needs to do is:

The nice thing is it will connect for you and everything! Store your creds as WIFI_SSID? Got you covered. What about CIRCUITPY_WIFI_SSID? Got you covered there too. What about in secrets.py? Sadly not there, settings.toml is the way to go.

Now let's say you aren't using a FeatherWing, or you are connecting things to different pins? Well your set there too!

A few hours ago, I saw an announcement of a new Adafruit product #5961, connecting Stemma QT/Qwiic connector to a breadboard.

I quickly realized that by having two of these, with headers soldered on (consider using right-angle headers for this), one can use "Flat Flexible Cable" ("FFC") AF04-5-ND from Digi-Key, along with Digi-Key crimp-on connectors 609-3512-ND, to connect something like a sensor on one side of a door/operating window/etc to a microcontroller on the other side.

The FFC is MUCH thinner than a normal Stemma QT/Qwiic cable, or indeed any sort of "ribbon" cable I've seen in my nearly 60 years of "doing" electronics, so the FFC is MUCH better at getting through doors/windows/etc.  You may be familiar with FFC in a custom form (often called Flexible Printed Circuits) for connectors on various small displays.

I should mention that those "crimp on" connectors can be a tad bit ornery to get on.  I usually find that a largish pair of "pump" pliers to do the ''crimping' and another pair of pliers to "convince" the side pieces to close will do the trick.  Also, if you have a sensor exposed to the weather, you might want to consider adding some conformal coating (though the device and connections before applying the coating, and if your sensor includes humidity and/or barometric pressure, be sure to protect the "sensor hole" from the coating.

One instance of this that I've used the FFC for is a sensor outside a window while keeping an ESP-8266 safely on the window sill.  I also have sensors inside both the fridge section and freezer section of my side-by-side with an ESP-8266 on top of the unit.  (A fridge would make a pretty good Faraday Cage, so WiFi inside the fridge probably wouldn't work.)