This is still being updated, it's hot of the press, so fresh it's still wet like a just iced cake, or bar of chocolate on a summers day, unlikely to solidify any time soon... I was going to come on show and tell this week so I hurried something together, but felt not great so skipping a week
So what's this project for? Scott (CircuitPython Lead Developer a.k.a @tannewt on GitHub and Discord) has been working on ESP32 bluetooth in Circuitpython and I'm excited, so much so I wanted to test out the Bluefruit related projects in anticipation of the upcoming ESP support. I read a bunch and then thought surely we can do that with web workflow, or even web-BLE (bluetooth connections in the browser)...
Click a button in your browser and a magic panel appears on code.circuitpython.org or your web workflow circuitpython device. That panel expands to reveal all the same* screens and functionality as the Adafruit Bluefruit Connect mobile apps (plus extras), but accessible to WiFi users for the first time! *Soon there will be BLE and USB and WiFi support for all the screens/functions of the mobile app, but for now I decided to get started recreating John Parks "CircuitPython BLE Rover" which uses the Bluefruit Connect "ColorPacket" and "ButtonPacket" type of packets (the only ones I've tested so far).
Why that Guide? Well, I have an old toy tank that has been in need of repurposing for a while, and I just recently received the Crickit Featherwing (CRICKIT = Creative Robotics & Interactive Construction Kit) and that BLE Rover guide was one of the first guides that I found matching my need to quickly prototype some kind of robotic tank thing.
What's a bookmarklet?
🤫 Psst, ever wanted a DRM free version of a youtube video for offline use? Well offliberty.com had you covered for the last twenty years...various sites supported, and a handy bookmark-let (javascript instead of a URL saved as a bookmark/favourite) which would take the address from whatever page you were on and forward you over to offliberty.com to download whatever it was.
My plan is to use a bookmarklet to add extra functionality to existing webpages that interact with Serial Port / UART / BLE/USB/WiFi based circuitpython devices
First Log Entry - May 27th - Initial Scribblings:
Progress so far, it works, but there were teething issues, this video highlights a classic and I should have probably cut it although highlighting failure+learning is always good, it's related to touch versus mouse input and not cleaning up one's code. I rebound the button to initially have events fire (with Pressed=True) on mouse/touch press/down and then fire again on up/release (Pressed=False), so the buttons would be held to drive, but I hadn't appreciated the original rover project was more of an on with a press situation. I've since found a lovely guide by MakerMelissa who did a more generic robot one for CircuitPython 2022 (still a driving rover) so I may later migrate my focus to another BLE circuitpython project.
Where's the code? -- As they say in Holland: Stop, stop! This Beer is not ready yet...
Main repository is at https://github.com/tyeth/serialfruit-connect-bookmarklet however it's not version 1.0.0 yet, rather it's v0.0.80 probably by the time you read this. Not quite worthy of a release, but a fascinating experiment so far, and ready for testing by the brave or foolhardy.
There's an examples folder, with the crickit-tank example, but expect more to follow. This first example will eventually illustrate how little change is needed to allow wifi (or USB) control of previously bluetooth-only projects using the Circuitpython Web Workflow.
For now it's full of test prints and some decoding things that will be refactored into a separate file/module, but if you scroll down to the end then you can see the main button/color tank logic, and before that is the check for a BLE or WIFI or USB_Serial packet, decoding it with the adafruit_bluefruit_connect library (to reuse their packet classes - checksums and decoding logic), and then acting on a ColorPacket or ButtonPacket.
Finally the main idea of the project was to see if I could "shoe-horn in" my own side panel to any page (or eventually integrate an extra tab to the tab panel of code.circuitpython.org), with a view to reusing web-serial connections (and web workflow which uses web-sockets) but with a pretty app interface to send hex packets and remotely control pins / robots. Effectively recreating the mobile apps in javascript, but for now all the packet encoding logic in javascript along with a couple of pages.
I eventually realised while testing the bookmarklet that it required taking over the window.websocket object (saving it to a variable, then rebinding window.websocket to my own class which serves sockets from the variable instead - allowing reuse of the socket by both scripts. This requires reconnecting the current page or websocket if active, so that we can trap and reuse the websocket connections.
Similarly with the web-serial (serial ports via the browser), there are issues with no access except to the initiating script (so we rebind window.navigator.serial and keep track of each use), along with android needing a polyfill to support web-serial (but I saw the connection dialog show an S2 feather on my phone once polyfilled). Quite the journey, and I've only built about 10% of it...
Problems? Questions?
When aren't there some? Well firstly I don't have any supported bluetooth devices for circuitpython so I have no way to use the BlueFruit connect mobile apps (which I'm trying to recreate). To that end I'll get one eventually and the ESP boards will get support very soon. I also need people to eventually try the library, or just to file an issue saying what their use case is, how they use the mobile apps and what they want from the bookmarklet version.
Device Sensors - I'm always trying to get my phone's accelerometer to control things, for no good reason really, mostly because I can't. In this case it might be possible, but probably not 😅 as I thought it takes HTTPS to be able to receive deviceMotion/Orientation events (although that may not be the case and it's possibly just a site setting). If https is a requirement then we can still use it for usb/BLE via code.circuitpython.org (https), or we can cheat and pull it off via other means, but it might just work anyway at http://192.168.0.90/code/ (web workflow full code editor page for my tank).
Persistent storage? I think it makes sense to save some things against the current website your on, using browser storage, so you can have options predefined (I get the impression neopixel layouts can be defined). That would mean a device with a known website address e.g. IP/cpy-MACaddress would have it's own settings with no effort or cookie pollution.
CORS? Yeah, good question, I've seen issues once or twice when on a slow connection, like the script loads it's assets before finishing running as opposed to after triggering a different result. Generally it seems using the jsdelvr.net github urls works fine. May need to rethink after attempting bluetooth / serialports directly. Makes sense to fix the script to fire off the assets a moment later, or async/defer them at least.
Charts - Yes please, not sure what I'm expecting, simple probably, but I've not managed to find any examples of usage yet, only a single decoding example in the iOS app code. It looks like 4 data series on a chart in the mobile app.
3D Space - 🚀🛰️🌌☄️ No the other kind, positional space, i.e. representing acceleration packets, magnetometer, gyro, quaternions, I'd imagined a 3d plane on a stick like one of the 3D.js / webGL demos, or more simply a chart with 3 axis for X, Y and Z. LocationPackets also exist, so maps seem inevitable.
DFU - Looks like the mobile app supports sending DFU updates. That sounds like the perfect fit for a desktop/tablet/mobile web client, so eventually plan to support that (maybe just the NRF52 boards, but ideally consider other things like OTA in circuitpython).
PyLeap - I had a quick look at pyleap, which allows you to select from a list of learn guides to download a project to your board. I poked deep enough to find the json and the supported project list (a fair few but not that many), and the support is for NRF for bluetooth, or MagTag boards (ESP32-S2) for wifi. That list could easily be harvested and offered up to other boards, with minor tweaks it would work or potentially straight away. The projects could be updated in the Learn Guide platform to support more boards or be agnostic for those pyleap ones that make sense. I don't want to be limited to importing pyleap projects though, in my head a circuitpython project bundle can be easily got from lots of learn guides.
Custom layout screens, like app pages for users with specific needs / desires - I think with local / browser storage then this could work on a per site basis well. Export json on save would mean the screens could be shared between devices / people.
How do you use it?
Read the readme, there's one for the main repository, which gives you the code to save as a bookmark (the bookmarklet code). Then goto your device's web workflow page, normally the devices IP address, and goto the code editor (url ends with /code/ ), and then click the bookmarklet / bookmark, or type javascript: into your browser address bar and then paste the bookmarklet code and hit enter. It should make a button with Toggle Panel appear. Then disconnect and reconnect the serial using the button at the top of the page, and the panel should then work with the device.
Update - 2nd July - Finally know what I'm trying to recreate... First CircuitPython BLE experience!
Yesterday I got to test the Bluetooth on the ESP32S3, merged into main the night before so very hot off the press, and it worked!
An iPhone connected using the Adafruit BlueFruit Connect app, and then showed me graphed data coming from a sensor attached to the circuitpython device (a LilyGo/TTGO T-Display S3).
Just one drawback, not all the builds for 4MB flash boards have BLE turned on by default, in fact most have it turned off if they have a display etc as the extra flash required by the bluetooth pushed a few boards over max capacity (until the switch away from the current partition scheme [supporting 2 OTA app partitions] to another layout with a single large app partition).
Why is this a big thing? Well apart from the BLE support for ESP32 based boards (except S2), you can't use the iPhone app until a device is connected, so now I can finally test all the functionality and app screens and device configuration.
One last thing is I found two extra repositories with a desktop Bluefruit Connect app (9yrs since update - old electron/node) and a Web-BLE dashboard site, mostly I was curious as to different desktop and mobile and browser versions of the BlueFruit Connect protocols and WebBLE/WebSerial type things.
Desktop repo: https://github.com/adafruit/adafruit-bluefruit-le-desktop and web-ble dashboard:
https://adafruit.github.io/Adafruit_WebBluetooth_Dashboard/
Finally I had a play with the code.circuitpython.org site to attempt to connect my ESP32S3, but alas there is a service filter to accept the adafruit file service which isn't running on my device (I could run it, but I wanted to try BLE Workflow, which should auto start with the correct button mashing sequence although may not be enabled yet for ESP32)
Above: video showing ESP32-S3 talking bluetooth to an iPhone, sending temp+humidity data using CircuitPython
Code: adafruit/Adafruit_CircuitPython_BLE/blob/main/examples/ble_bluefruit_connect_plotter.py
Below: Web Bluetooth Dashboard showing the same sensor data, but instead running a different code.py
Code: tyeth/Adafruit_CircuitPython_BLE/examples/ble_bluefruit_web_dashboard.py
