This project reaches the obligatory Doom adaptation in the third act, but takes some interesting twists and turns along the way…
Background
Making my annual pilgrimage to all the Halloween stores this year, this “Jabberin’ Jack” animated Jack-o’-lantern caught my attention. It’s a fairly affordable item ($60 or less) with a tiny internal LCD projector and video player to show an animated face:
What had my interest is the combination of price and that it’s a short-throw projector. Typical LCD projectors have a focal distance of 1 meter or more, whereas short-throw varieties…normally quite spendy…focus just a few inches from the front lens, suggesting interesting and creative applications.
See, a few years back we were all agog over the Gakken Worldeye HDMI display — $150 at the time, but now discontinued and “collectible” at $400 — and were looking for economical DIY approaches which never panned out.
Both of these things are interesting because they have a shape. Shapes occlude and exhibit parallax. If you just need flat video, use a normal LCD screen or projector. But these weird displays present opportunities for some Disney Haunted Mansion-type effects.
Goals
- Jabberin’ Jack is clearly playing back videos from internal storage. At a minimum, can we get in there and substitute our own video sequences?
- Can we extract the projector and use it on other surfaces?
- Can we skip the video unit and drive the projector directly? (e.g. dynamically-rendered content such as the eyeballs)
Things to Know
- There are multiple products in this line (“ANIMAT3D”), and some have evolved through different hardware generations. This is a summary of my exploration, not a direct how-to, because your hardware might be different. First generation units are said to have used an internal microSD card. This moved to soldered flash storage in the next revision, but accessible by an internal USB port. This latest one has soldered flash and no USB. But certain aspects such as data formats are probably unchanged between them. (NOTE: the cheaper “Rockin’ Jack” does not use an LCD projector, it’s just a fixed set of LED-illuminated stencil shapes…this hack won’t work there.)
- If goal #3 above is possible, this will be done using the most affordable and accessible hardware, likely a Raspberry Pi Pico, as a simple proof of concept. The answer to most of your questions after that is “Yes,” but also, “If you write the software to do it.” Could this display video from a Pi 3? Yes, if you write the software to do it. ESP32? Yes, if. Display a live feed from a camera? Yes, if. The test case should prove it’s possible, but I don’t have the time for every permutation. If this works, you can skim the prototype code for insights to an approach.
Looking Inside
Several deeply-recessed Phillips-head screws hold the case shut. Opening this separates it into a smooth front piece (the projection surface) and a back piece holding the projector and electronics, and disc-shaped top and bottom pieces:
The electronics are comprised of three PCBs:
- The top button board (not shown here, attached to the top disc) with VOL–, SKIP and VOL+ buttons.
- Power board containing an on/off switch and USB micro-B port. A charge-only USB cable is included, but a power supply is not; it’s reasonably assumed the customer already has a spare. Even if a charge-and-data cable were included, this PCB only routes the power pins, not USB data.
- The main board, containing:
- ATJ2157 microcontroller at center (this is not an ATmega or ATSAMD part, it’s an Actions Semi chip for small media players).
- 256 MB Samsung flash storage (bottom).
- Ribbon cable to projector (right). So we can already see the display is driven directly from this board, not a simple composite input or similar.
- Several disconnects. Power (from USB board) at top left, projector backlight LED at bottom left, button board at top center, and speaker at top right.
- There is no USB port on this revision of the main board.
- The unpopulated pads (next to the power socket) is for a fan connector (presumably for the LED backlight and determined not needed) and is not for a USB port.
Disconnecting all plugs, removing and turning over the main board reveals the goods: the necessary pads are here for adding a USB connection. 5V (near center), GND (near top), and DM and DP (USB data “minus” and “plus”).
Using this requires sacrificing a proper USB charge-and-data cable, not the crummy charge-only cable included with the unit. Fortunately most everyone has a whole box full of them.
A color standard normally used in USB cables indicates which wire serves what function…but the cable I randomly chose is weird and required checking with a multimeter and USB pinout reference. The traditional colors for these connections would be:
- Red to 5V
- Black to GND
- Green to DP
- White to DM
This isn’t needed for older units, which have a USB port or a micro SD card that can be removed and popped in a USB card reader.
Jacking In
Plugging into a computer, this board appears as a flash drive. It’s a FAT32 filesystem…a Windows-native type of drive…but everything from Mac to Linux can read/write as well.
Very first thing was to make an image of the disk contents in case anything gets messed up.
The .amv files are the audio and video for the pumpkin faces. AMV is a weird format used by cheap media players, with limited resolution and fidelity, but adequate for this application. There are three pumpkin faces to choose from, and each includes a joke-telling track and a singing track. The number at the start of each filename indicates the playback order on startup and when using the SKIP button. In a moment we’ll try reading, and later writing, these files. There’s about 8 megabytes free to experiment with small clips of our own before anything else needs to be deleted.
The hidden .LIB files contain code that’s vital to to the microcontroller for playback, so never delete these!
Working with Video Files
ffmpeg is a Swiss army knife for video conversion and can turn these AMV files to MPEG4 or most anything else you need to view or edit. This is a command-line tool and installation varies by OS (on Mac this can be done with Homebrew), so check the link or Google around for instructions.
All of the AMV files were copied to a folder on my computer for experimentation. Then:
ffmpeg -i 1_Traditional\ Pumpkin\ Recut_1_1_ressized_1.amv test.mp4
After processing for a few seconds, we get a playable file!
So we can pull video off! Convert and edit if we want. Now how about the other direction? After a lot of experimentation, I arrived at:
ffmpeg -i Input.mp4 -filter:v "crop=in_h*208/176:in_h:in_w/2-in_h*104/176:0,framerate=fps=15" -c:v amv -s 208x176 -pix_fmt yuvj420p -c:a adpcm_ima_amv -ar 22050 -ac 1 -block_size 1470 1_Output.amv
(This is entered as one continuous line. It probably appears as two or more lines in your browser.)
The crop setting extracts the center part of the source video to work with the projector’s native aspect ratio, because some video editing programs (such as iMovie) can only export 16:9 video. framerate similarly adapts the video for AMVs limited throughput; depending on the source video, the setting used here will typically merge 2 or 4 frames at a time to provide a sort of motion blur.
If the file’s small (under 8 MB), this can fit on the flash drive without deleting anything else. Change the numbers on the original filenames to move them forward in the playback sequence.
Properly eject the drive, unplug USB from computer, reinstall the board atop the projector and reinstall that within the pumpkin. The USB cable you added can be taped to the inside back or just set on the pumpkin’s “floor.” Plug all of the cables back in — speaker, buttons, etc. — being very careful not to confuse the power and LED plugs! (They’re labeled on the main board.) Pop the front of the pumpkin back in place, but no need to screw it all back together yet…you’ll probably be returning for more file work later. Maybe a hole could be added for accessing the extra USB cable.
Although videos could be anything, simple subjects with a black background work best. You can find some good specimens on YouTube by searching for “Halloween Projection” and fetching the files with a tool like 4K Video Downloader. Here’s an example using EPCOT’s adorable dragon Figment:
If going back in to replace files on the flash drive, the pumpkin should be switched off and then the board powered only from the improvised USB connection.
AWKWARD: Around this time, someone brought to my attention this existing Reddit post that got pretty much to the same point, though using MPxConverterW for Windows which might be a bit easier for non-technical users. It just hadn’t turned up in Google searches, else might’ve saved me some time!
There’s still goals #2 and #3 though…
Projection Surfaces
When trying to approximate the Gakken Worldeye a few years back, a passable dome was fashioned from a tinted security camera dome, the inside coated with Rust-oleum frosted glass spray (the ideal projection is a neutral dark gray with a matte interior to reduce internal reflection). Still had that piece around, and it works fine here:
Making a dome this way might be redundant and excessive work now though, as the same company’s “Mr Chill” snowman decoration (with similar enough, though not identical, electronics inside), has a spherical shape (if the hat can be removed, unsure):
A dome is the least “shape-y” shape though. Some items at the Halloween store are sufficiently translucent and might work well with a human face projected on them. There could also be some opportunities with those big airblown costumes…while these don’t exhibit the desirable “shape” and “occlude” properties, just having a projector hidden inside a wearable is unexpected and “magic.” (Third image here is from the animated Big Hero 6, but same idea, internal projection on soft shape.)
The projector itself is about 5 inches deep, and the ideal projection distance seems to be another 4 to 5 inches:
If that’s still too deep for a specific installation (e.g. inside the human-sized head of the hanging ghost above), a 45° mirror could be used to project perpendicularly. You may need need to flip the video vertically or horizontally depending on which way the projection is “folded.”
Dynamic “Live” Content
Everything so far works with video loops. That’s probably fine for a majority of projects, and there’s space on the flash drive for a ton of material (the factory animations are about 45 minutes in total, with a few megabytes still to spare).
Suppose we want something interactive though? Projecting sports scores on something, or a spooky eye that could be puppeted? Can’t do that with AMV files. This needs code. It needs to drive the projector directly.
Inexpensive long-throw projectors have HDMI or composite video inputs…one can hook up a Raspberry Pi and call it done. But the projector in this prop has no such input. It’s this particular odd combination of wanting the short-throw capability, dynamic content, and wanting to do it inexpensively that brings us to this point…
Opening the projector itself (you do not need to do this!) reveals the LCD…a 2 inch 176x208 unit that was super common among later model flip phones, and from traces on the main driver PCB we can deduce a parallel 8-bit interface is used.
The cleanest approach would be to reprogram the microcontroller on the main PCB…but this is not a mainstream part, and there are barriers, both in documentation and software, to taking this path.
Much of the appeal in such projects is making them approachable to a wider audience. Maybe a little bit leet, but not super leet. That means Arduino or CircuitPython. So…the main PCB will be kept as it has a useful constant current driver for the projector LED…but the display flex connector will connect to another board, I’ll try an RP2040 Pico as they’re cheap, ubiquitous and easy to program. As mentioned earlier, if this works, the information is there to take it in other directions with different hardware.
As a minimum viable testbed, I’ll use a Raspberry Pi Pico RP2040 microcontroller board (about $4 from various sources) and a 24-pin 0.5mm FPC breakout adapter (about $6 from Amazon). Other options exist (including ESP32), but this is about as inexpensive as it gets:
Without a datasheet for this particular LCD, finding the pinout was a whole process of inference (based on the flash chip — which does have a datasheet — sharing the same 8-bit bus) and probing with an oscilloscope, knowing somewhat (from writing several Adafruit display libraries) how various LCD control lines operate. Also a lot of trial and error, eventually arriving at:
From the breakout/adapter board (not the RP2040), pin functions are:
PIN FUNCTION PIN FUNCTION PIN FUNCTION 1 NC (not connected) 9 DC (data/command) 17 DATA2 2 NC 10 WR (write strobe) 18 DATA1 3 GND 11 RD (read strobe) 19 DATA0 4 3.3V 12 DATA7 20 NC 5 NC 13 DATA6 21 NC 6 NC 14 DATA5 22 NC 7 CS (chip select) 15 DATA4 23 NC 8 RESET 16 DATA3 24 NC
On the RP2040 side, let’s use:
PIN FUNCTION PIN FUNCTION GPIO0 DATA0 GPIO8 RD (read strobe) GPIO1 DATA1 GPIO9 WR (write strobe) GPIO2 DATA2 GPIO10 DC (data/command) GPIO3 DATA3 GPIO11 RESET GPIO4 DATA4 GPIO12 CS (chip select) GPIO5 DATA5 GPIO6 DATA6 There are ample GND pins on the Pico board, GPIO7 DATA7 and 3.3V on the "3V3" pin
Based on the ubiquity of such displays in phones of a certain period, I could guess that the LCD driver was most likely an ILI9225.
The GFX_Library_for_Arduino has support for this driver and 8-bit parallel buses…the only “gotcha” on the latter is that it must use GPIO0 through GPIO7 on RP2040 (as shown above). The other control lines are all configurable in code though.
Wrote a minimal test program to plot random lines:
#include <Arduino_GFX_Library.h>
#define TFT_RD 8
#define TFT_WR 9
#define TFT_DC 10
#define TFT_RST 11
#define TFT_CS 12
#define TFT_ROT 0
// Display D0-D7 connect to GPIO 0-7
Arduino_DataBus *bus = new Arduino_RPiPicoPAR8(TFT_DC, TFT_CS, TFT_WR, TFT_RD);
Arduino_GFX *gfx = new Arduino_ILI9225(bus, TFT_RST, TFT_ROT);
void setup() {
Serial.begin(115200);
if (!gfx->begin()) {
Serial.println("gfx->begin() failed!");
}
gfx->fillScreen(BLACK);
}
void loop() {
gfx->drawLine(random(gfx->width()), random(gfx->height()),
random(gfx->width()), random(gfx->height()), random(65536));
}
Back-lighting the LCD “manually” during testing, this yields exactly what was hoped for:
Fantastic! This could be slotted back inside the projector, but I’ll be assembling something a little more hardy without the breadboard. The important thing is that it proves the idea is totally possible: everything we need to know to control the LCD from another device is here in the pinout above and in GFX_Library_for_Arduino.
That library also supports parallel control on ESP32 and ESP32-S2, if one has a WiFi-enabled project in mind. Or…if one digs into the source, an initialization sequence and low-level drawing operations could be extracted and something written for a Raspberry Pi computer (the Linux variety, not the microcontroller). I don’t plan on doing this, but the information is there.
To reiterate: the original controller board will still be present, to correctly power the LED inside the projector. The speaker will be disconnected (so we don’t hear the original loops playing) and the LCD’s ribbon cable will be routed to the adapter board (to RP2040) rather than the original controller.
The Inevitable Doom “Port”
I’d expected to do a simple demo to show it’s not just canned video…maybe Pong or a snake game…but The Laws of The Internet require Doom running on any and all devices. To reiterate the opening point: this isn’t a “port” to the pumpkin because we’re not using the original controller board. An RP2040 now drives the projector, which was really the goal here.
LCD was reinstalled in the projector, sealed up, and the projector reinstalled in the pumpkin. Original driver board is in place, but with speaker and LCD disconnected. For demo’s sake an I2S amplifier was added (because the RP2040 Doom port supports it)…though this adds another $6 to the otherwise around-ten-bucks parts cost. (UPDATE: an I2S Amplifier BFF can also be used — it’s a dollar less and the picoblade connector is compatible with Jabberin’ Jack’s speaker.)
PICO BOARD I2S AMP BOARD VBUS VIN (5V) GND GND GPIO26 DIN (data in) GPIO27 BCLK (bit clock) GPIO28 LRCLK (left/right clock)
The only downside maybe is that this is a Pico SDK project, which is a bit more involved than working with the Arduino IDE as in the first example. It’s adapted from rsheldiii’s “Doom in a keycap” project, in turn from another RP2040 port. The pumpkin port is here:
GitHub Repository
A pre-built .UF2 file is included there for Pico boards. Additional steps are needed to get the Doom “wad” onto the board…explained in the repo README.
This is not yet interactive; it’s just the game’s demo loop. USB keyboard support (via USB-OTG cable) should be possible but I had to disable that for the build to work.
Summary
Worth it for the hacks? Probably not, or maybe for a very select few. It does contain a short-throw projector…but with poor brightness, poor resolution and some distortion, it can’t hold a candle to the Gakken Worldeye. But the fuzzy image really would work well for certain ghostly Halloween props if that’s something you do.
Mostly this was to scratch an itch…that need to dismantle things, see how they work and spark new ideas. Other than its original intended purpose as Halloween decor, I don’t have an outstanding use case even for most maker types, nothing to warrant rushing out and paying full price just to rip apart.
If you have that same tinkering itch though, or are the type to go all-out DIY with your Halloween yard display, it could be fun (the Doom part is just silly, but the first two easier goals — different video, different projection surfaces — might have uses there). Recently spotted Jabberin’ Jack for about $40 on Amazon, and others report them turning up even cheaper at Goodwill, etc.