Realtime LED Wall With React + Websockets (I Let Strangers Control It)

Welcome to Syntax. Today, I am going to be detailing my Hack Week project, which is my real time web controlled LED grid that I put together. So Century has this thing called Hack Week Wes you get a to build whatever it is that you want, maybe a product.

Scott and CJ did some really cool stuff. I opted to do something that was a mix of both hardware and software JS well as letting the wider Internet be able to control it, and it was a super fun project. I learned a lot about it. And today, we're here to talk about all of the different pieces and get into the nitty gritty. There's a lot of really fun stuff and a lot of fun stuff you're gonna learn about it. I should say this is gonna be a pretty video heavy episode. If you're listening to this on the audio podcast, keep it going. You're gonna learn a few things. We're talking about durable objects and Wes sockets and React and microcontrollers and all that. But if you want to cut to the video version of it either on Spotify or on YouTube, I'm going to recommend that since there's gonna be a lot of visual stuff. Hell, yeah. What's up, Scott and CJ? Not much.

Not much. I brought a salad to the Sanity.

So, yeah. That's what's up. That's what's up. No. No. No. It's taken out. I'm just excited to see how you built this. I love stuff like this. I was very jealous when you started doing this project because I would love to build a little drawing interface like this. So now it's super cool. And I I gotta say just, you always, seem to bring a level to Hack Week that is, something that makes me personally want to beat you and and step up and do something better. So,

shout out for to you for always being so ambitious about Hack Week because I I'm a competitive type of Vercel. And if I I feel like if our team didn't go hard on the Hack Week, then it would be less fun. Yeah. And I I'm excited to learn how you did this because it's it JS, in a sense, a full stack project because you've got hardware, You've got a front end. You've got a server that's gotta sync these things. It's real time. Yeah. I'm excited to hear all all the parts and pieces that make this thing work. So I'm gonna show you what I built here, and then we're gonna break down all of the different pieces. So there JS, obviously, the hardware LED grid here. There's a diffuser panel that has been three d printed over top of it as as well as with a frame. Then there's a web app where you can

draw on it. If Scott and CJ, if you guys go to local.westboss.com, you'll be able to go ahead and draw on. You can upload. Yeah. One of the features is you can drag and drop images. You can obviously draw on it in real time. I do have a couple utilities, so I can paint my GitHub contributions.

Give it a sec. It's going off to the GitHub API and pulling down all of my activity.

Boom. Woah. So that's my, GitHub activity in in real time being painted to the LED. I will note Sick. That this is very washed out in the video. In person, this looks amazing.

It's, like, so crispy, but so, like, clean-cut pixels.

Yeah. And I'll explain how I got that in just a sec as Wes. But, man, it looks awesome. I've been I'm very happy with it.

That's so exciting. A couple other things you can do with it. I hope to, like, put some social stats on it or, be able to put, like, maybe build status of the the Syntax site, maybe some stats for the podcast or, century errors, a count of how many open errors we have on our century. There's just, like, a lot you can do with this. Now that I have this, like, sort of base thing built up, I am now able to just, like, put whatever it is that I I want on there. Okay. So let's get on into how it works. Let's start with the hardware, and then we'll get into the actual software side of things. So first of all, the the LED grid itself is 2,000 2,300 and something pixels. So what it is is I bought these little 16 by 16.

They're about, I don't know, $10, $8 a piece, and it's 256 different individually addressable LEDs. And and how that works is that you send power to all of these LEDs, but then you can also send data signal to every single LED, and you can change it to be any color that you want. Right? They're RGB, LEDs.

Then you can actually chain them together and and make a an an an entire matrix of them. So what I did is I I took nine of them, and I put them into, obviously, a three by three grid and soldered them all together. I actually ended up undeseoldering all of the leads on the back of them and just going doing my own soldering because these clips were just a little bit too

too big to fit behind the frame that I made. How long did that soldering take you? Because I'm so slow.

I'm getting better at it. It's just that I had to I did it, like, several times, because my initial the initial time that I did it, I I used some, like, premade frame that somebody designed, and I couldn't get access to the back. So I kept breaking it, trying to, like, put the wires underneath it. It was so annoying. So I ended up just scrapping the entire thing that I found online and designing my own frame for it. But Mhmm. I don't know. Soldering maybe maybe, like, an hour or or so,

just having to quickly desolder everything, and then I I put leads on it directly to it. And when you said you send messages to that, like, what are you able it's it's more than just an on off kinda thing, or, like, what how intricate are those messages?

The LED grid itself, obviously, it has, like, a positive, negative. It's a five volt LED. There's also 12 volts that you can get, which I kinda wish I had done because this is this is a lot of LEDs. It actually takes quite a bit of power to turn all of these on. And then what's powering it and and sending the actual data, which is the third line on these, is something called an Wes 32. And this is just a $5 microcontroller, which is absolutely wild, and it has a Wi Fi chip in it, it has a Bluetooth chip in it, and then it has all of these different pins. And these pins are used for sending serial data. Most of them are used for sending serial data out through the data line. And on this little Wes p 32 runs a little piece of software called WLED.

So WLED is just this really nice piece of software that will allow you to you can run all kinds of different animations, you can change the colors of them.

I use this in several places. The little on air sign that I have behind me, I have it off right now because it's blowing it out. The gym lighting, I'm I'm currently doing my house in these LEDs, and WLED powers all of that because it it's a piece of software that can then send the data signal from the Wes 32.

It spits it out in serial and is able to deliver it to the different types of chips that run on these these very common LED strips or LED grids.

So would you say WLED is like a custom firmware for Wes 32, or is it something that's also running on top of the firmware, like, as a separate app or something like that? No. It's a it's a totally custom firmware that you flash

directly to these chips, and it's it's written in, c c plus plus. I I when I did my controlling my Roomba, I I dipped into that as Wes, writing my own lower level code that would send serial data out of these different pins. But it's it's quite amazing because WALED also has, like, it has has Wi Fi. You can you can open it up on any browser. You can get apps that connect to it, but it also has several APIs that will accept incoming data. And those are the APIs that I used in order to send data from my application to it. It's it's running it's literally running a web server on here. Like, this that WLED is not running on my computer. It's literally running on an Wes. Not not this one, but, one that's behind here right now. Since I have so many, LEDs on here, I actually hit the limit of the number of pixels that you can, send over a data ESLint. Because the way that the data line works is that it snakes through it. So you start the data at one pixel, and it goes in and then into the next one, into the next one. And that's actually how it does it with power as well. It sends the the five volts in and out. But once you start to like, you can do a little demo with one of these really quickly. You don't even have to have an external power supply. But once you get into putting a lot of them together, you hit the limitation of how many LEDs you can possibly send data through, which is 2,048.

So I was slightly over the actual limit. So what I ended up doing is I split them into three different sections.

So one, two, three. And I have three different pins on my, GPIO pins on the microcontroller, and those are each sending three different pieces of data out. And then you can go into the software and set up the layout of it. You can see here I've got all nine of the Sanity. And then it takes care of as as far as, like, you, the user, is concerned, it's just now it's just a one single grid, and WLED figures out how to properly send the data to each of them in in which order.

Is that a a common solution, to handle that?

Yeah. Yeah. Like, to use WLED or No. To to break up, to send it in in groups like that. The the way that serial data works is that it it sends high and low voltages.

So if you're running, like, a five volt system, it might send, like, a like, a 4.5 or a five volt to show, like, a one. Right? And then it might show send, like, like, a one volt or a 0.5 volt to show a zero. Right? And you're basically just sending ones and zeros. And then, as you know, like, voltages over long runs can can start to drift, and it can start to can start to die out. So if you have a very long run or if you have a this is not a long run, but it is a lot of of pixels, then the data can start to get weak as it goes through. And there's there's solutions to boosting it, but the the better solution is to simply just chop it up into multiple segments. And then that way, you can also send data out the different pins at the same time.

It's a a single controller that is sending the data across three. Because I guess you could have also maybe had three different controllers, but this lets you use a single one Yeah. And then split up the the signal. I guess you'll probably talk about it, but, like, this requires a lot of power too. Right? So each panel is five watts. So, like, is there a separate power supply for the the panel grid itself, and then it's just getting its data signal separately?

Yes. Exactly. So Okay. The ESP 32 will run on on five volts or 3.3 volts.

And then you can also buy little adapters for them as well. I'm a big fan of this one where you can put, like, a 12 volts into it as well, and it will just it'll has a buck converter. It'll it'll dump it down. But you can power, like, one panel off of a USB, like, a one amp Mhmm. Off of USB. But once you go over one amp, it's actually funny. Mac OS X will be like, we disconnected your USB thing because it was using too much power. Yeah. So once you once you pull more than that this all of these together can pull 94 amps, which is 94 times what it should do over USB. I didn't do that because you you literally cannot buy a 94 amp five volt power supply. The the real solution when you do bigger stuff like this is instead of having to do more amps, what you do is you just increase the voltage. So I kind of regret buying five volt panels because it has required me to in absence of volts, I'm just pushing more amps.

And when when people do, like, like, big things Stadium things? Yeah. Like a stadium, they'll use, like, 48 volts. Like, that's commercial, right, or or 24 volts. And and and most hobbyists will use either 12 or five volts. So I almost do everything in in 12 volts now just cause it's you have to fuss a lot less with it. But I had bought these, I don't know, six months ago, and I just, I didn't think about that. But I I had I bought an external power supply, which JS, like, like, this big. And then I also go into my WLED settings, and I explicitly set the maximum power supply current to 18 amps right here. And and that is plenty bright. I don't even have it.

I have it at half brightness right now, and it's it is plenty bright. If any any brighter than that, you gotta start worrying about, like, thermal problems. Right? Like, it it already gets kinda hot. Yeah. If you cranked it, what would happen? Would it catch on fire, or would they explode like in, The Goofy Movie? If you cranked it, then you have to start worrying about cooling. And and the other thing about it is is, like, heat comes from resistance.

So what was happening with with these ones is if the power has to go through every single LED in order to get to the two thousand three hundredth LED, it was getting really, really hot. So what I did is there's the second wire in the middle right here. This is the power injection.

And the data has to go from pixel zero to pixel 2,300, but the power can go anywhere. You can inject power at the end, in the middle, at the start, ideally, as often JS you can. So what I did is I just I desoldered it, and I just ran power from the power supply directly to every single Sanity. And that brought the they were getting pretty hot just regularly, and then they they cool down right a lot after that because there was not as much you're not trying to pull a lot of amps through this tiny little flexible

circuit board. You're you're you're bringing it right to the panel that needs it. Yeah. Lot of fun stuff. Crazy. Yeah. Well, my problems could have been solved with code. And, like, to me, it puts me in such a place of, like, wow. I remember any time of Yarn hardware project I've ever tried to do, you know, you get things soldered up, you plug it in, and then, like, nothing happens. And then I'm just like, okay. I guess I'm done with this. Like, how do you how do you get better at debugging this stuff? You just understand how to

check values better, whether that's, like Yeah. Voltage or Like, you can add logging. Yeah. You you check voltage as Wes. Like, like, the way I knew I needed is I you you check the voltage on the input, and then you check the voltage on the output.

And then if you see a certain percentage of voltage drop, like, if it's if it's, like, 20% less, that's that's not good. Right? You should you shouldn't see that much voltage drop over it. So there's kind of a lot to CJ just uploaded some stuff to it. You kinda learn over time, and and there's just things you you understand about the basics of of electronics and and how data is is sent over the wire. A lot of, trial and error. Yeah. Scott of trial and error. Well, did you burn any circuits or bust any pixels during this project? Surprisingly, Node. Anything busted.

You Bos in. Did you release the magic smoke?

I did. Busted.

I have burned several of these out over the years, but I did not have a single anything in in this case. Nice.

There was once where I accidentally crossed the positive and negative wires.

Mhmm. And I turned it on. I plugged it in, and it wasn't working.

And then I immediately put my hand on it, and I realized it the wires were getting very hot.

And I I didn't put a fuse in it at first. I had the fuse on my desk, and I was like, I'll put that in at the end.

And I luckily, I grabbed it and unplugged it and realized I had crossed some wires, but I I had created a short circuit, and I didn't have a fuse in it. So it was just running the the amperage over and over through the things, and it was melting the wires, which would have caught on fire if I had left it. So that was my one scary moment there, and I after that, I'd quickly put a a fuse is, like, like, 10¢ I should have put in at the very first.

Well, that's the kind of thing that I would do because I just wouldn't know any better. You know? Yeah. Not put a few yeah. And the next thing you know, you're catching something on fire.

I once when I was working on the ATV at my cottage before I really understood how LEDs worked, the LED ought to to show, like, if you're in Node or reverse, they weren't working. So I tried to put the battery voltage straight to the LED, and it it literally exploded. And it LEDs are glass.

Yeah. That's what I'm saying. I want some Luckily, it didn't get it in my eye. And the the solution is LEDs, like, those little tiny little pin ones, those are 1.2 volts.

And I put 12 volts to it. So Nice. Too much.

The diffuser itself. So the LED grid is is kind of hard to look at. You look like a gamer that just bought some LED strips and and put them up. So in order to make them look nice, you have to diffuse them.

And what I ended up doing is at first, I downloaded a couple things on three d printer world and whatever and and printed them out, but I just was not happy with any of them. So I designed my own diffuser. And after a couple trials, this is what I ended up with JS it is a 0.3 millimeters of white filament.

Yeah. And then I did black, like, baffles or grid on the end. And what that did is it gave me, like, a very crisp look. I did it all in white first, and it it bled a little, which is a cool it's a vibe, but it wasn't what I was going for with the whole pixel art here. I also didn't want, like, very defined lines as well. So I I printed it. I'm not sure if you can see this or not. Let me see if I can I feel like a makeup reviewer? I printed it a couple millimeters, like, one millimeter up in up the side in white, and then that allowed me to not get nice big black lines.

And then I I did the rest of it in just black, and that was the that was the winning ticket to to do it. And I had nine of them. I ended up I ended up just gluing it. I I wanted so badly to, like, make some sort of click together system, but, like,

I was doing way too much already with this Yeah. Yeah. Just doing too much. It's like Yeah. You're saying you it. Your diffuser's got a BBL?

Yes.

Well, I Wes just saying, this is one of my favorite parts of the build because, like, I've tried to mess with, pixel arrays before, and I've never come across a good diffusing solution. So I love the fact that, like, each pixel is basically a black square. It has, like it has walls around it to prevent it from bleeding.

And then the top layer is just thin enough so that it can shine through. Like, because I don't have a three d printer, and, like, when I was trying to do this, I was, like, trying to use, like, foam board and pieces of paper or, like, bottle Gosh. You could buy, like, craft paper that's, like, slightly opaque, but then you run into the bleeding issue. So, like, it just looks blurry.

But the fact that you have like, you've isolated each pixel to then shine through, like, is awesome.

Yeah. It's it's really nice to be able to to do that. A lot of people I do regret or not. I don't don't like that there are little lines in here. I modeled it with a 0.2 millimeter lip on two of the edges.

And what that allowed me to do is to overlap them very slightly so you didn't have a very defined very defined line, but there still is a bit of a slight line. I think if you could also buy, like, like a halogen. You know, you go to, like, your dentist office, and they have those, like, tube lights.

You could buy that, and I think I might try that at one point Wes you just print the grid, and then you put a single sheet of diffuser material on the top. Yeah.

Yeah. Yeah. Yeah. That's how my it's actually funny how how my softbox works. Although the grid is on top of the diffuser, just diffuser and then grid. But that's for softbox. That's a completely different application. But oh, it's the same idea, though.

So, yeah, that might even work. I didn't even think about that to flip it around.

But that that was the hardware aspect of it. So that was was relatively fun, to play with play with all of that. And then on the the software side of things, there was kind of two pieces. There was the entire client side app, which is the real time drawing, and then there was the whole server side part, which is keeping track of what has changed, sending that data over to the LED thing.

So I'll start with the server side because I I feel like this is the most interesting. So I built a something with Cloudflare durable objects and and party kit. It's called party server now. The whole party kit thing doesn't look like it's being super maintained, but they are moving Really? Yeah. But they're they're moving it to something called Sanity server, which is a bit more of a a peeled back layer, which which I moved it in in like an hour because I initially built it in Sanity kit.

So CloudFlare durable objects are essentially a, like, a a JavaScript object or a JavaScript class. Here, let me open up some code here. You're gonna remove the photos the dogs were putting on your Oh.

Oh my goodness. Look at this, Scott. Cloudflare Durable Object solves the problem of how do you maintain state or how do you maintain, like, memory with a serverless function or or a Cloudflare worker Wes these things spin up and down when they're needed, and they spin up multiple versions of them when when you need lots of them. And it's really cool because you can simply just stick data on the object, and then that data is is available anytime you come back to that instance of the durable object, which is very similar to if you were to, like, just stick something in memory on a long running server, except that when you restart it, it it's it's it's actually saved to a database, so it it stays there.

So the durable object was really cool. And then on top of that, party server gives you the ability to do, like, real time.

What that does is when somebody is is drawing on this grid, what's happening is that there's a WebSocket connection from your browser to the Cloudflare durable object server. It's it's a Cloudflare worker. It's running locally on my machine, but you could deploy this thing to to the cloud as well.

And then every single time there JS, like like, somebody has drawn to a one of these grids right here, it's sending that data over a WebSocket to the back end. And then the back end is then saving that grid state to the the durable object, and then it is then broadcasting those changes to everybody else who currently has this application open. There there is ability to even do rooms in this one as well. I've I have it built into the app, but when you come to the Node page, it's just the the default room. And it's really cool because, like, there's you you can get into, like, doing database schemas and whatnot, but I'm just simply just I just have a, like, a like, a JavaScript map and set Mhmm. For maintaining all of this data. Right? It's just an array of of 2,300 pixels, and and that array tells you which color every single pixel JS. And then that that data source of truth is is the Cloudflare durable object.

And then anytime that somebody draws, that data comes into the server, it updates it, and then broadcast that out to to everybody else that is on it. In in fact, as I'm drawing on this thing, it's actually not painted to this grid until it sends it from the client to the server.

The server then updates it, and then it broadcasts it back to everybody, including yourself.

So the it doesn't paint it until it comes back from from the servers. It's like the opposite of Scott's. It's not Logan First at all.

This type of tech, is this primarily used for stuff like this? You wouldn't go building your whole data structure of your entire application in something. So would you or would you? I I No. No. You

you would stick your your regular application in, like like, a d one database or something on CloudFlare. The idea with the durable object is that it can maintain and share I'm calling it memory, but just like just the state of the object. Imagine you had a JavaScript object that had a property called name on it and and you set that to Scott. Right? And then you come back in six months, that JavaScript object will still be there, and then you can also share that object with with other people that have it. So that is awesome for anytime that you wanna have long running processes or come back to something because the problem with with serverless compute is that it it spins down. And and then if you wanna save literally anything or you wanna share literally anything across multiple instances of that serverless function, then you you have to end up sticking that in a database somewhere or stick it in key value, just so that you can share it amongst anyone. And and this is simply just, like, variables that exist and can be shared for everybody.

I guess on top of that, the client side library is what gives you the the sync. Because I think Wes. You could use this as, like, persistent data for your serverless functions, but being able to broadcast that state gives you the real timeness, and and that's more of, like, the WebSocket client. Is that accurate to say? Exactly. Yeah. That's so there is something on the client side. There's something called

party ESLint, and then they also are working on something called party sync, which, Scott, you'll probably like to hear as well Yeah. Which is just like, yeah, all of this this data as well. But the party client is what connects the WebSockets from the client to the actual durable object, WebSockets.

Is PartyKit a part of CloudFlare now? Yes. Okay. PartyKit was made by Sunil Pai, and he worked at Cloudflare for a long time, left, created PartyKit, and then that was acquired. So he's back there. Mhmm. Wes. And he's working on all kinds of interesting agent stuff as well. I shouldn't have said, like, Sanity Kit is not no longer maintained, but they Cloudflare, basically, they took the Party Kit repo and they haven't moved it. But now now, like, the main code Bos for Party Vercel, party client, and party sync, those all now live on on Cloudflare's own GitHub, and and they're they're they're different packages Yeah. Now, which I like because, like, Sanity kit was built on top of workers.

You don't abstract anything away now when you're using PartyServer. You can just add it to, an existing CloudFlare worker, which is I I like that as well. They're you're not they're not hiding that you're using a worker like PartyKit was. So that's the the server. It's obviously syncing state between all of the open browsers.

But then there's also the aspect of, like, how does it get to the the LED hardware.

Right? So what's happening is as part of the server, we are checking as frequently as we can for what are called dirty pixels. Right? So data JS coming in all the time, and then every now and then, we run a function that then checks what do we need to send to the microcontroller that's running WLED to tell it which pixels on this grid to now change. And the way that that works is there's a little bit of a diffing algorithm. It detects which pixels are dirty, mean meaning they have changed from what is currently on the LED grid, and then it creates just an array of all of the currently changed pixels. Right? So if I were to just take red and and paint, like, two or three of these, it's going to say, okay. Pixel 400, pixel 800, and pixel six fifty have changed, and those are red. And then what it it's going to do is it it bundles that then up into a, a request, and then it will send that to the web server that's running on the the ESP 32 with via the WLED software.

So I guess to clarify, is that just like a separate process running on your computer? It's basically another party server client

that has a direct connection to the the grid because it's on your local network? I actually did it both ways, and and I'll tell you about the problems that I had it. So what I'm doing and what you see right now is it's a function called check for update. I guess I guess I can just show you because we have the code here. Right? Yeah. Show me.

There we go. It's a funk it's just a method inside my durable object called update LED.

Mhmm. And that will then run a bunch of code that checks if there are dirty indices.

And then if so, it will send it to the WLED. And I wrote a whole another WLED library for there's a whole bunch of logic as to how it gets sent over. I'll talk about that in just a second as well. And then that will just that's a recursive function, which will try to it will call itself as soon as it gets a response back from the WLED saying it has successfully updated the pixels.

So that was the first way that I did it. And I'll I'll tell you about the the problems that I hit and and then the the second way, which you suggested.

The WLED has several different ways to send data to it.

There is a WebSocket API, and there is an HTTP, like, just a normal API endpoint that you can hit with, like, a fetch request.

I I just call that the fetch API as it's simpler.

Both of them have limitations.

The WebSocket API is is faster because you when you connect to it, you have a you have a single socket connection open. And then anytime you want, you just send updated pixels.

And that is so much faster than than the fetch based API. However, you can only send, I think, it's something like like a 100 pixel change at once. So if somebody was to take, like, a Firefox logo and drag it on here, it would very quickly overwhelm the WebSocket Sanity not so much that it would reject it, so much that it would crash this thing here because there's only so much memory in these microcontrollers.

Not that it can't hold an array of 2,300 values, but the a the WebSocket and the HTTP API are JSON based, meaning that the microcontroller has to take the JSON payload and parse it from JSON to whatever the CSS based array JS, and then it has to then send that data over GPIO via ones and zeros.

So the WebSocket API can't send too big of payloads, and then the HTTP API can send something like it says it can do 2,400, but I I tested it. I kept crashing it. I think I'm sending something like 800 or it's it's bytes. It's not pixels. But, basically, when I'm writing a full image, I have to send four or five consecutive fetch requests.

Otherwise, it will crash and and reboot the the entire microcontroller.

What happens when that crashes?

It literally just the whole screen goes blank, and then it reboots and then shows you the initial thing, and then the WebSocket reconnects. It's not a big deal. Like, it recovers, but it obviously goes goes blank entirely.

So I wrote a whole bunch of logic to say if this is a large, like, I have a bunch of variables as to, like, limits.

There we go. WebSocket byte limit, so, 1,400 bytes, and then HTTP byte limit is 5,000 bytes.

So that's I can send, I don't know, four times just under four times more via fetch Wes over than a WebSocket. So when you're drawing, like, a little bit, this is all WebSocket.

But then when you when you draw a lot of changes, then it will switch over to fetch and use those. So I wrote a bunch of logic in here to switch between the two. So that was my first two, and I wasn't really happy with either of those because I was I was doing, like, the the video, and it was, like, like, one frame a second. It's really slow to actually draw.

And, I was like, man, like, that kinda that's kinda slow. And and these things these WLEDs are not slow. Right? You can you can put, like, gradients on them and and whatnot. They're pretty powerful.

But what is slow is, I'm pretty sure, is this the JSON parsing and and the WebSocket connections and and and having to run a web server. So, ideally, you would use a lower level thing that would simply just send raw ones and zeros from whatever to it, and and that's via something called the UDP.

And UDP is is not like like TCP. UDP is a one way signal where you can just send commands and and you never get a response back from it. You just send, send, send, send. And that's how I did a video many years ago flying a drone with React.

That was a UDP socket just sending data from the React app Right. Yeah. Over Wi Fi. That's a UDP sockets.

But that still only allowed me to to do 400 pixels at once.

So that was was still a bit of overhead, and it had the the downside of Cloudflare workers, they have their runtime is not Node. Js. Their their runtime is something called Worker d, and they support most of Node. Js. Part of what they don't support is the socket API. So you cannot open a UDP socket from a CloudFlare worker. It has to either be like a a WebSocket or an HTTP request.

So I ended up making a bridge that like like CJ suggested Wes I made a server side script that basically just joined this room and watched for changes.

And as those changes happened, it would send them over the UDP socket. And then I ran into a whole bunch of trouble over, like, I was sending too many. You know? Like, if you draw, like, that was, like, 18. And then times I had a 100 people in there Okay. It was it was too much. So then what I had to end up to do is I would have to collect them as they're coming in and then shoot them out the door as fast as I could. At that point, I scrapped it. And I said, like, people who do this LED stuff professionally, they send the raw data over probably not over Wi Fi, probably not over something. They're they're sending it, like, raw over a, like, GPIO pins. Right? You're actually physically connected to it, and you're sending the data. There's no parsing or anything. You're simply just sending it straight from whatever you want to to the value. And there JS there is several other implementations aside from UDP that WLED will implement. I just haven't had the had had the time to

to dip into it. So is this as fast as you could get it refreshing with your current strategy?

Yes. Yeah. This is and in this case right now, it is a little bit slow because I have OBS open. It's totally unrelated.

If I were to close OBS, this would be butter. I this is butter smooth when I'm drawing on it, and it's it's pretty much real time just drawing on it and then uploading a photo drag and drop. It does it in two or three segments.

Who would have thought that OBS would have been the thing that makes me wanna get a new computer? That's the only time I'm like, man, my computer sucks JS when I have OBS open. I can't even figure it out because my

my resources are not pegged at all. It's just It's just something about opening OBS. It makes my it's not even, like like, look. It scrolls fine. I can hover over stuff fine. You open it, and your computer just goes, dear god, no. No. No. My computer my computer right now is butter, but there's something about the browsers. Graphics or what? And the drag or the mouse drag drop.

It's it's jacking the mouse events somehow Something. And and they are very laggy. So, yeah, this if you're not impressed by this, it's it's butter when I don't have OBS open.

I guess to clarify, that's because your server is running on the same computer as OBS. Like, if this were running in the cloud, this would you wouldn't have that limitation. Is that accurate to say? No. No. The problem is that my literally Browser. My mouse event in the browser is not being fired

quickly enough to the thing. So it's it's I see. Not a resource problem at all. It's it's that my browser and I've tried every browser on my computer does not fire these events. What if I use my built in mouse? No. Same thing. But it's it's butter, otherwise. But, obviously, it's not butter for it's not butter for, like Totally not butter. Entire images.

Yeah. It's butter, though. Not butter, though. Yeah. Yeah. But that that whole like, I'm working on a project right now where I'm decking out our whole house in these things here. I love this. Wes is turning his whole house into an LED panel. Like, my house is it's gonna be like a gamer street, but I'm gonna deck out my whole house in these, which is these are, RGBW pixels.

So they have a they have a white pixel Wes well, which allows you to get nice, warm and night much brighter.

Because if you've ever tried to use, like, an RGB light, you tried to do, like, colors, you'll notice that it gets much dimmer when you do, like, a purple versus just a a flat white.

But, anyways, I'm I'm putting these on my house, which I'm really excited about. And, like, the people that do this professionally, they they have, like, whole like, they have Elsa dancing on a grid, like, eight times the size of this, and it's, like, six 60 frames a second. So it's it's definitely possible, just not you shouldn't be using a JSON API Yeah.

Yeah. For that.

Oh, also, another thing I didn't dip into, which is I'm only sending the dirty Vercel. But one thing you can Node, like, if I were to draw from pixel three to pixel eight all blue, there is the ability to tell it, start at Vercel three, go to pixel eight, put to make them all blue. So that would compress it a little bit. But even just looking at this, like, Firefox logo, there's I guess there's, like, several instances

where it looks sense to be able to doing is you're you're that's kind of a micro optimization

when, like, the biggest optimization would simply come from Just not using it. Approach. Right. Exactly. Exactly. You're you're not gonna get that big of a a gain from that Wes where switching over protocols would be much better. Opening up this, so this is running at local.westboss.com.

It's that this process is just running on my local machine just so I can I can get it going? And also, it needs to run on my local machine because the ESP 32 is also on my home network. And then in order to send the data from my machine to that, they have to either be exposed to the Internet, which I can do, or you just have it have it locally.

I have a question. Do you have any, like, not safe for work detection on here?

I don't. The first time I did it, I did it about six months ago, and I I tried to get people to draw on it. And somebody did upload, like, a a scantily clad photo, but nothing awful.

But when I did the receipt detection yeah. People Yarn pretty pretty good. The thing that people did when I opened it up is they started writing scripts in the console that would programmatically send pixel changes, and it was kinda fun because people were drawing, like, you could tell that they were writing, like, nested loops to change the colors, and I thought that's that's pretty fun. But I think you could paint the I don't know. Is there enough detail here that you could then send that to a, like, a Wes, not safe for work, detection library. I think you could do a client side. There's gotta be a client side detection for that. Yeah. Let me let me show you what it looks like when I just upload a an actual photo here. Let me just find a photo of myself here.

Like, it's like, you can tell that's me.

Yeah.

That's that's pretty good. Yeah. Let me see if I can get the webcam stuff going real quick. Anyways, what I was saying about the the real time stuff is that this is a Vite server running locally on my computer, and then I expose that Vite server to the Internet with a Cloudflare tunnel.

And then, basically, people are visiting my my computer from the Internet without having to expose your your local IP address, which was was awesome. At one point, somebody wrote like a like a thing to send, like, a million requests through. Yeah. And my Internet was getting kinda weird where it was just kept dropping in and out, but I just I killed it at that point. I think they they went away. You probably would have to put some sort of rate limiting in there.

Here's the webcam code. And the funny part about that is, like, people were abusing the drag and drop images.

So what I did is I just went to my code and commented out that feature, and it I live updated on everybody who had it currently open's website.

You know? Like, I can work on something locally, and the the Vite dev server was serving up tons of connections. It was proxying the Wes sockets.

No problem. It just asked me if I wanted to start my camera. Oh,

you try it. You try it.

Woah.

Okay. Yeah. So I have the app open on my computer too. This is this is very real time. You just on the on the page itself, seeing his pixels.

And and that's like like, literally, it's sending the payload from Scott's computer to the durable object, and then the durable object is then sending the update to both me, CJ, and Scott before it it actually paints to the thing. So it's it's not bad. And I think if you were to if you were to, like I don't think that the WebSocket is the the hang up here. I like, I sorry. There's two web sockets at play. There's one to the durable object, and then there's another one to to this thing here. Yeah. I think if I were to take the microcontroller out of the picture, I could get it to work pretty quickly.

But Wes, essentially, am just recreating video and and and monitors, if you think so.

In your in your demo video, that's one of the last things you said. And it's so funny because, like, literally, just buy an external HDMI monitor and put images on it. Like, you don't have to do this, but it it's awesome that you're able to get low level to the Vercel. And, yeah, let's see it. That was pretty fun. The future,

I don't know about the future. This JS this is a really fun project. I think what I would like to do is is to hang it on my wall, and just to find a way for it to just be always open and put it on my website and just have people come by at any time, draw on it, or I the other thing is that I can I have admin controls where I can set which room is on there? So I can have one room which JS, like, forward slash syntax.

And if I were to draw Yeah. Syntax logo to this one, you'll see it's it's not mirrored to the LED. But if I go to the admin, I could just switch it.

Yeah. And now that room is is being painted. Cool. So I could have, like, a, like, a syntax room. I could have a drawing room. I could have, like, a stats room, and then I could just have them cycle through those rooms, or I could have buttons on my on my little stream deck here that would switch them. There's just so much so much fun stuff that you could do with it. But that was my project. That was just all it was a ton of fun. I can't believe I get to talk about it for an hour.

But I just love digging into all of this fun stuff.

Sick. Wow. That was awesome, Wes. I love it every single time.

Nice work. Cool. Alright. Let me know if you, enjoyed that or if you have any suggestions on things I said add. Leave them down in the comments below, and, I guess we'll catch you in the next one.