Rapid-fire DXOS Demo

A few weeks ago, Jess Martin and Chad Fowler livestreamed about DXOS, demoed some simple SDK-based applications, and showed off where we're going with an extensible application framework. Check out the whole 90 minute enchilada, if you like!

Or, you can get the highlights in this rapid-fire demo of DXOS. It's 12 minutes and it keeps you on the edge of your seat:

Transcript

Jess: DXOS platform For creating these shared collaborative applications that are peer to peer and have a lot of other fun properties since DXOS is a JavaScript framework, we have to start a demo in the place where all JavaScript framework demos start, which is with the to do list application.

This is a pretty standard to do list. There's nothing fancy about it. Take out the trash, get some groceries. I actually do need to feed my chickens today. If you've seen demos like this before, there's not a lot to notice other than the fact that it's fairly responsive, there's no loading spinners when I check things off when I delete something goes away immediately.

And the reason for that is that the data is being written locally to the local file system as opposed to some remote database where it's updating that database. The other nice thing about that is that I can actually close this tab. And reopen it and the data is still there. And that's because it's not in memory. This is actually being written to the local file system, IndexedDB. And so it's saved there.

If I quit the browser, come back, still there. Don't clear all of your data and cookies and everything. Cause that'll cause you problems. But it's there in IndexedDB for now. The other nice thing about this way DXOS works is that you can actually share across tabs as well. Now I've got two tabs they update together yay and it's fast. Because we're both reading from the same local system for these two tabs. The place where things start to get interesting is that you can actually share across devices as well. So I'm going to open a new profile on the right. That's the equivalent of having a new device. if you're familiar with the way chrome profiles work, arc profiles work the same way they're completely isolated from one another, have their own separate cookies, data store, all that kind of stuff. So this thing on the right doesn't know anything about any of the data that's on the left.

So if I open up the tasks app, it's going to hit me with this screen saying you need to create an identity. I don't know who you are, so I'll explain that in just a second. But I'm going to go up here first and click invite. When I click invite, I can copy this link and I go back over to this brand new device that we're trying to sync with.

And paste that invite link in. Brings me back to the create identity page and what this is saying is you need to tell me who you are. So we'll start with a name: Jess Tester 2. And then it is creating a private key for this device that's going to be used to encrypt all of the data that is both transmitted between devices, so encrypted over the wire and then eventually we'll start encrypting that data at rest. So I'm gonna go through that code and then you get this typical two factor off thing, right? So now they're in sync. And once they're in sync, it's really fast. I can be typing on either side and you can see how quick it is. So what's powering that is that It's actually taking the data in the browser's local store, and then replicating that across devices through a synchronization layer.

Chad: Is it doing it peer to peer like using WebRTC or something?

Jess: I'm glad you asked. It is doing it peer to peer. I can show you that in action using our dev tools. So this is a set of dev tools that we use for debugging our applications. It has everything about your identity, like I just mentioned.

Your private key, public keys, and then this is ECHO our database and the spaces that you're a part of a space is like the unit of replication. If I invite you into a space, you have access to all the data within the space. I have access to all the data within the space as well.

Chad: When you did the invite load just now, you had a space and then you added the new identity...

Jess: Yeah, exactly. And if I go up to invite, you'll see that there's two members in this particular space. Jess Tester. And Jess Tester 2.

Chad: And on the left then the dev tools thing, I think you actually went to like devtools.dxos.org or something, is that right?

Jess: That's true. I did. What would stand out to you as strange about that, by the way?

Chad: That you are going to an external URL to look at your own data that is not replicated to it.

Jess: Yeah, exactly. Like you can actually go into items here. select a space. This is the one space that I'm in and you can actually see the there it is: take out the trash. I went to an external domain in order to look at my own data. I've struggled with how to explain this concept. But all of these apps are just client side apps that are running on your client and domains are just a URL pointer to be able to go grab some HTML, JavaScript and CSS., I'm going to tell this client. to stop talking to all of the DXOS peers by toggling the connection. And so immediately you saw this change to disconnected, this change to closed, So if I go back over to our Tasks app and add some stuff client one, To pick up the yard. Edge. if I go back over to DevTools now, I think this is the one that I disconnected. And I click toggle connection, watch the left. Those two tasks popped in. Notice they didn't clobber each other, it actually merged the two tasks. And that's because the way that the data structure works under the hood CRDTs there.

The CRDT, that acronym expands into Conflict Free Replicated Data Type. But the easier way to think about that is if you're writing a program and you have a variable.

you want to update the contents of that variable from two different locations. Usually what happens is the last writer wins. That's what happens for most data types in programming languages, and so CRDTs is this class of data structures that allows for multiple writers to Update the same variable or the same data structure while automatically merging their changes.

And the other thing is there are trade offs here, right? There are times where the merge can't be made intelligently,

That's what programmers are most familiar with in terms of a distributed system that allows people to make changes independently and then merges them later. the reason DXOS is built on top of CRDTs is because of this peer to peer offline nature that these two peers could be disconnected and it allows you to continue working on your software. One of the things that I can't show right now without disconnecting from this live stream is, and what I'll often show when I'm demoing this in person is I'll go turn off the Wi Fi on my machine and then continue to use the application.

It feels responsive. Everything is still updating . I'm able to navigate around . Go back online and then see the changes immediately percolate over to other peers, which is pretty cool. For those of you who know what CRDTs are the particular CRDT we use in specific cases where you need one is called YJS.

It's probably the most widely production used CRDT. However CRDTs are plug and play in DXOS. You could adopt a different CRDT if you wanted to for specific data types. in specific cases, we're using YJS right now.

the thing that I'm most excited about for DXOS is that the database that is running the data store that's synchronizing across multiple clients isn't running once per application, it's running once per device.

And what happens is, instead of the database living inside of an app, and the app kind of controlling the relationship to that, the user controls the database, and the app asks for permissions to write to it. And then in that way, you end up having multiple applications that are able to write to the same data store.

I'll show a quick example of that. I wrote a Kanban app, and you'll notice that this one is also on a different domain, dxos-kanban.netlify.app. Again, I needed a place to boot up the front end code. And now you have your to do list represented as a Kanban. And so you can update either side and see it pop over.

I can come in here and pick up the yard. Was fun. And you can see updating seamlessly across. Again, these are two different apps running on two different domains on the same data store. Because the data store is running once inside of this browser. And then it's also being replicated across peers.

So there's a concept called a shared worker. You've probably heard of service workers. But we actually run the data store inside of a shared worker for browsers that support it. and the really nice thing about that is from a performance perspective, you can open 10, 15, 20 of these DXOS apps, and they're tiny little client side apps that are communicating to the shared worker.

And then all of these apps are just tiny little clients, like they're views. They're not entire full stack applications, even though they operate like full stack applications.

The code ends up being very simple because you can just assume that process is running and handling all of the hard stuff for you, like synchronization across peers and maintaining database connections, peer to peer discovery of other peers, tunneling, all of that. It's like just handled for you.

All you do is update the local data, like an IndexedDB, to it directly. And then that process handles replication, synchronization, conflict management, all that kind of stuff for you. Encryption, all of that.

We sometimes joke internally about what does DX in DXOS stand for. And obviously there's the founding idea that it stands for decentralized. But lately we've been joking that this thing is so nice to use for developers. Maybe DX stands for developer experience.

But we should look at code.

We talked about how these applications are pretty simple. So there's some boilerplate React code to be able to ship a React app.

And so pretty standard stuff: package.json and tailwind.config and ts.config and that kind of stuff. Make sure your bundler knows how to turn the HTML, CSS and JavaScript into something that your app can read. But aside from the boilerplate code this is basically the entire to do list app.

It's 140 lines, most of which is actually tailwind and modifying the state of the visual JavaScript to change the status of this from editing to that kind of stuff. there's only about 15 lines of actual DXOS code in here. I mentioned that spaces are the unit of replication and that you get identity for free.

You start out by creating a private key using use identity. You grab a space that is created by default with that identity, it gets associated with that identity. And then you use this use query. And you query for the data that you want.

And then you write to the data directly. you're not interacting with a database object, where you're saying database, write, this particular row, update this value in the database. It's just, you do a query of the data store and whatever matches that query comes back, and then you can write to those directly.

The way to read this line 16 is, I'm basically saying, give me all the to dos. This to do dot filter is empty. And then. I'm iterating over all of them when I'm rendering the page, and then in order to mark a task as completed, I just say task dot completed basically invert the value of the task But it's just that you have a JavaScript object and you update the value on the object.

And then the SDK layer handles the reactivity of that where it detects that as a change, synchronizes that across all the devices.

The other devices also see that change come in and they reactively update the UI so one of the ways that I think about this is your UI can just show the data in the state that it's in. And then whenever the data changes, through the magic of react the UI updates magically for you.

You don't have to do any sort of proactive UI management. You just. have a UI that reflects the state of the current data. And then we'll handle updating the state for you and react will handle updating your UI.

Chad: So in this case, am I right? That from like line 30 down, it's just normal react code.

And I could have just implemented something in memory that wasn't DXOS at all, and just build a simple in memory react app, and then I could swap it out and say, okay, this stuff is now in DXOS and it's getting replicated.

Jess: Yeah, we could actually do that.

We could come in and todos, it's a new todo array. Array of todos. And now we're actually storing in memory. And then pretty much everything else ends up just working because the API for updating the data is just update a JavaScript object.

It's "update the fields" which is the same as you would do. Yeah, it's pretty nice.

I think of DXOS as being both a platform. There's this distributed systems problem of we got to get this data onto other people's devices. That layer is one layer, but then there's the SDK and framework level. And the SDK framework team at DXOS has done a great job of making this really straightforward for me as a developer.

Chad: I can see that. Lovely.