AWS IoT Smart LED Matrix

I was scrolling through Instagram one night when I saw an ad for a 'lo-fi connected display' called the Tidbyt.

Tidbyt - See What MattersTidbyt keeps you updated on transit, weather and more. Leave your phone in your pocket.

Tidbyt

I thought the price tag was a bit steep at $179 for what was essentially an ESP32 controlling a matrix display, so I did a bit of shopping on AliExpress. The same exact LED matrix panel was on sale for just $14!

For the firmware, I decided to use ESP-IDF for this project as I wanted to challenge myself to build an application without relying on the Arduino framework.

I started out with just an example, showing the capabilities of the display and the ESP32-HUB75-MatrixPanel-DMA library:

Now, in order to "smarten" up this thing, I wanted it to be able to render animations that were pushed over from a remote server.

The device connects over AWS IoT to securely tunnel device shadow information, such as the sprite schedule and current hash of each file. Sprites (animations) are delivered over a seperate MQTT topic, containing their size, ID number, and base64-encoded binary data. The data is in WebP format as it's compressed and easy to work with the libwebp library.

Each device authorizes to AWS using TLS client certificates, and it has IAM roles to ensure it only accesses what it is supposed to. Access is enforced by sending the esp's mac address along and it must match the CN of the cert.

Using WebP was a great exercise in trying to manage memory in such a limited environment. I had to dig around in the libwebp documentation for a bit, but eventually, I was displaying static images on the display!

I thought animations would be just adding a line or two, but I had issues when trying to decode subframes in the animation. I read the libwebp docs, and they have a special WebPAnimDecoder class:

if (isReady) {
            if (pdTICKS_TO_MS(xTaskGetTickCount()) - animStartTS > lastFrameTimestamp) {
                if (currentFrame == 0) {
                    animStartTS = pdTICKS_TO_MS(xTaskGetTickCount());
                    lastFrameTimestamp = 0;
                }

                bool hasMoreFrames = WebPAnimDecoderHasMoreFrames(dec);
                if (hasMoreFrames) {
                    uint8_t *buf;
                    if (WebPAnimDecoderGetNext(dec, &buf, &lastFrameTimestamp)) {
                        int px = 0;
                        if (!isSleeping) {
                            for (int y = 0; y < MATRIX_HEIGHT; y++) {
                                for (int x = 0; x < MATRIX_WIDTH; x++) {
                                    matrix.drawPixelRGB888(x, y, buf[px * 4], buf[px * 4 + 1], buf[px * 4 + 2]);
                                    px++;
                                }
                            }
                        } else {
                            matrix.fillScreenRGB888(0, 0, 0);
                        }
                        currentFrame++;
                        if (!WebPAnimDecoderHasMoreFrames(dec)) {
                            currentFrame = 0;
                            WebPAnimDecoderReset(dec);
                        }
                    } else {
                        ESP_LOGE(MATRIX_TAG, "we cannot get the next frame!");
                        isReady = false;
                    }
                }
            }
        }

With this change, I was able to get animated WebP files to play on the screen!

GitHub - acvigue/SmartMatrix-IDFContribute to acvigue/SmartMatrix-IDF development by creating an account on GitHub.

GitHub • acvigue

Tidying it up

Up until this point, the display was wired up using DuPont cables. This was fine for development, however, the slightest bump would cause the screen to display garbage.

I threw together a quick PCB with a USB-C connector, an ESP32-S3, and some connectors and had it manufactured with JLCPCB. I ordered the components from Mouser.

Assembly was pretty straightforward, but the USB connector required a lot of flux due to the super tiny pads.. I really enjoyed using the S3 variant of the ESP32 because I didn't need to add an additional USB-UART converter!

Server

Now, the client device works. I had to have something continuously render the sprites and send them over IoT when they updated. I decided to use AWS ECS to orchestrate spinning up instances of my SmartMatrix-Server project as necessary. The server uses a Redis-backed cache to store hashes of each devices' schedule and sprites.

Each sprite is allowed to be valid for a maximum of six hours in case decoding fails on-device and it needs to be resent. The sprites attempt to re-render every second to check for updates. Using optimized WebP, average CPU utilization of the containers is under 10%!

GitHub - acvigue/SmartMatrixServer: Server to schedule and run applets to be sent to SmartMatrixServer to schedule and run applets to be sent to SmartMatrix - GitHub - acvigue/SmartMatrixServer: Server to schedule and run applets to be sent to SmartMatrix

GitHub • acvigue

Frame

I started to work on the frame and settled on making it out of some spare 1x3s I had laying around. The sides are joined with 45-degree miter cuts and when the glue dried, I stained it a deep walnut color. At this point, I decided to attach a TSL2561 ambient light sensor to add some basic auto-brightness capabilities to the project.

With most of the code out of the way, I needed to create some of my own apps to run on the display. I originally wanted to use the same Pixlet platform as the Tidbyt to enable cross-compatibility, so I had to teach myself Starlark.

The first app that I wrote simply displayed the three scores from my Oura ring, along with a graph displaying the historical data over the past week. It ended up being a great starter app that integrated with an external web service.

I just opened a PR to add the app to the official community apps repository, so hopefully it will be in the Tidbyt app soon!

I've uploaded both the firmware and server code to my GitHub and they're linked above.. Here's a few pictures and videos of the final product though!