[{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003cp\u003eThis demo uses Web Serial to receive video frames from a Pi Pico and show them in a web GUI. It works like a virtual display. The video frames go over the normal CircuitPython USB serial port as base64 encoded text with start and end markers.\u003c/p\u003e\n\u003cp\u003eI developed this technique so I could have an easy way to monitor video from my PiCowbell Camera Breakout. But, this approach could probably be adapted for use as a virtual displayio display.\u003c/p\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"user_image","content":"https://cdn-learn.adafruit.com/user_assets/assets/000/000/969/original/screenshot1.jpg?1720565534","metadata":{"caption":""}},{"element_type":"text","content":"\n  \n  \n  \n        \u003ch2\u003eHardware\u003c/h2\u003e\n      \n\n\n","metadata":{}},{"element_type":"user_image","content":"https://cdn-learn.adafruit.com/user_assets/assets/000/000/970/original/hardware.jpg?1720568078","metadata":{"caption":""}},{"element_type":"text","content":"\n  \n        \u003cp\u003eThis uses the same hardware setup as my\u0026nbsp;\u003ca href=\"https://adafruit-playground.com/u/SamBlenny/pages/camtest-picowbell-camera-breakout-demo\" target=\"_blank\"\u003eCamTest: PiCowbell Camera Breakout Demo\u003c/a\u003e project:\u003c/p\u003e\n      \n","metadata":{}},{"element_type":"product","content":"https://www.adafruit.com/product/5905","metadata":{}},{"element_type":"product","content":"https://www.adafruit.com/product/4883","metadata":{}},{"element_type":"product","content":"https://www.adafruit.com/product/5582","metadata":{}},{"element_type":"product","content":"https://www.adafruit.com/product/5945","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003ch2\u003eGetting Started\u003c/h2\u003e\n\u003cp\u003eTo begin,\u003cspan\u003e\u0026nbsp;\u003c/span\u003e\u003ca href=\"https://learn.adafruit.com/how-to-solder-headers?gad_source=1\u0026amp;gclid=EAIaIQobChMIhdCV2_-VhwMVkG1_AB1_4gmMEAAYASAAEgKgQ_D_BwE\"\u003esolder the headers\u003c/a\u003e, assemble the stack of boards,\u003cspan\u003e\u0026nbsp;\u003c/span\u003e\u003ca href=\"https://learn.adafruit.com/welcome-to-circuitpython/installing-circuitpython\"\u003einstall CircuitPython\u003c/a\u003e, then copy the project bundle code to your CIRCUITPY drive (check out the \"Download Project Bundle\" button below). Once that's all done,\u003cspan\u003e\u0026nbsp;\u003c/span\u003e\u003ccode class=\"inline\"\u003ecode.py\u003c/code\u003e\u003cspan\u003e\u0026nbsp;\u003c/span\u003ewill begin capturing camera frames and sending them over the serial console. To see the video frames, you will need to:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eMake sure your Pi Pico board is plugged in to your computer's USB port\u003c/li\u003e\n\u003cli\u003e\u0026nbsp;Load the static web GUI page, \u003ccode\u003eCIRCUITPY/index.html\u003c/code\u003e, included in the project bundle using a modern browser that supports the Web Serial API (Chrome on macOS works well)\u003c/li\u003e\n\u003cli\u003eClick the big green \"Connect\" button on the web GUI page\u003c/li\u003e\n\u003cli\u003ePick your your board from the list. It may be called \"Pico ...\"\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eOnce you pick your board, the web GUI's javascript code may take a second or two to sync with the serial stream before it begins showing video frames.\u003c/p\u003e\n\u003ch2\u003eUnderstanding the Virtual Display\u003c/h2\u003e\n\u003cp\u003eThis project has two main cooperating parts:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eOn the Pi Pico board,\u0026nbsp;\u003ccode\u003eCIRCUITPY/code.py\u003c/code\u003e captures video frames from the camera, encodes the pixels as \u003ca href=\"https://en.wikipedia.org/wiki/Base64\" target=\"_blank\"\u003ebase64 text\u003c/a\u003e, then sends them over the serial port.\u003c/li\u003e\n\u003cli\u003eIn a browser on the host computer, \u003ccode\u003eCIRCUITPY/index.html\u003c/code\u003e uses the \u003ca href=\"https://developer.mozilla.org/en-US/docs/Web/API/Web_Serial_API\" target=\"_blank\"\u003eWeb Serial API\u003c/a\u003e to receive base64 encoded video frames which it draws to an HTML canvas element.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eThe base64 encoded frame buffer data sent over the serial port looks similar to this:\u003c/p\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"markdown","content":"\u003ccode class=\"\"\u003e\u003cpre\u003e-----BEGIN FRAME-----\nNTE0NjEyNTY4MzQ0MjM1Nzg2MjE0NjY0Mzc3NjY0NTYzNjU3Nzk5Njg2NjY4Nzo8\nOTY4OTo6Ojk6OTc5OTk8PD06Ojg2NTYxLS4uKyorLCwvLC0tLzAuMTAzMS4vMS8u\n...\nMTEzNjU2MzI0NTg0NTU2MTMzMjU1MzY3NzY2NzY2Njg3NDk5OTk5Ojk4Nzk4ODg6\nNzY6Ojc4ODg3Nzs7Ozs7Oz07Ojk3NTU1ODMwMDAvLCwsLS0vLi0vLTEvMTEuLjEz\n-----END FRAME-----\nmem_free 97472\n-----BEGIN FRAME-----\nMzMyMjMzNDY0NDEyNjQ2NjUzNDQyMzQyMTM2NTMyNTQ3NTg3Nzk4NTg5ODc6Ojo5\nNzc3Nzc4ODg4OTk4ODg4OTo4OjY1NTYyMC8uLSwqKSwsKSsuLy4uLjEvLywuLi4t\n...\nLSouLy0sLC4tLCsvLiwrLiwuLSwsKisrLSksLCgoKSgnJykpJiYoKCsrJiYnJyUl\nJSgkIyQkJCMhHyIkJCYmIx4hIR4dHyIeHx4cHB0iHhsWGCAbGRgWGBwXFhkYFRYa\n-----END FRAME-----\nmem_free 97472\n\u003c/pre\u003e\u003c/code\u003e","metadata":{"markdown":"```\n-----BEGIN FRAME-----\nNTE0NjEyNTY4MzQ0MjM1Nzg2MjE0NjY0Mzc3NjY0NTYzNjU3Nzk5Njg2NjY4Nzo8\nOTY4OTo6Ojk6OTc5OTk8PD06Ojg2NTYxLS4uKyorLCwvLC0tLzAuMTAzMS4vMS8u\n...\nMTEzNjU2MzI0NTg0NTU2MTMzMjU1MzY3NzY2NzY2Njg3NDk5OTk5Ojk4Nzk4ODg6\nNzY6Ojc4ODg3Nzs7Ozs7Oz07Ojk3NTU1ODMwMDAvLCwsLS0vLi0vLTEvMTEuLjEz\n-----END FRAME-----\nmem_free 97472\n-----BEGIN FRAME-----\nMzMyMjMzNDY0NDEyNjQ2NjUzNDQyMzQyMTM2NTMyNTQ3NTg3Nzk4NTg5ODc6Ojo5\nNzc3Nzc4ODg4OTk4ODg4OTo4OjY1NTYyMC8uLSwqKSwsKSsuLy4uLjEvLywuLi4t\n...\nLSouLy0sLC4tLCsvLiwrLiwuLSwsKisrLSksLCgoKSgnJykpJiYoKCsrJiYnJyUl\nJSgkIyQkJCMhHyIkJCYmIx4hIR4dHyIeHx4cHB0iHhsWGCAbGRgWGBwXFhkYFRYa\n-----END FRAME-----\nmem_free 97472\n```"}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003ch2\u003eCircuitPython and HTML/JS Code\u003c/h2\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"button","content":"https://github.com/samblenny/webserialdisplay/releases/download/v0.2.2/webserialdisplay-6e62efb.zip","metadata":{"class":"btn btn-large btn-block btn-primary","button_type":"primary","target":"_blank","zip_folder":"false"}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003cp\u003eTo copy the project bundle files to your CIRCUITPY drive:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eUse the \"Download Project Bundle\" button above to download the project bundle zip archive file. For example, on macOS, clicking the button should put a zip file with a name like \u003ccode\u003ewebserialdisplay-6e62efb.zip\u003c/code\u003e in your Downloads folder.\u0026nbsp;\u003c/li\u003e\n\u003cli\u003eExpand the zip file by opening it, or use \u003ccode\u003eunzip\u003c/code\u003e in a Terminal. You should end up with a folder named \u003ccode\u003ewebserialdisplay\u003c/code\u003e, which should contain a \u003ccode\u003eREADME.txt\u003c/code\u003e file, a \u003ccode\u003eCircuitPython 9.x\u003c/code\u003e folder, and a \u003ccode\u003eCircuitPython 8.x\u003c/code\u003e folder.\u003c/li\u003e\n\u003cli\u003eOpen the \u003ccode\u003eCircuitPython 9.x\u003c/code\u003e folder and copy all of its contents to your CIRCUITPY drive.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eWhen you finish, your CIRCUITPY drive should look like this:\u003c/p\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"user_image","content":"https://cdn-learn.adafruit.com/user_assets/assets/000/000/971/original/CIRCUITPY-with-bundle.png?1720616348","metadata":{"caption":""}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003ch3\u003eCIRCUITPY/code.py\u003c/h3\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"embed","content":"https://github.com/samblenny/webserialdisplay/blob/v0.2.2/code.py"},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003ch3\u003eCIRCUITPY/index.html\u003c/h3\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"embed","content":"https://github.com/samblenny/webserialdisplay/blob/v0.2.2/index.html"},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003ch3\u003eCIRCUITPY/main.js\u003c/h3\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"embed","content":"https://github.com/samblenny/webserialdisplay/blob/v0.2.2/main.js"}]