Post by @dthompson@toot.cat

I have been thinking about how QUIC, the new hotness in low-level transport protocols, might apply to OCapN. My initial findings suggest it would be a poor fit but I'm still doing research.

QUIC's killer feature, stream multiplexing, doesn't do anything for OCapN because 1) OCapN mandates only one connection may be open between any two peers at a time for reasons of reference equality (in the sense of eq? in Scheme) so there can't be multiple streams at this abstraction layer 2) OCapN mandates FIFO order of message delivery and 3) unlike HTTP, OCapN is async and doesn't use a request/response cycle so once a message is delivered the pipe isn't blocked waiting for a result. The remaining advantages of QUIC are relatively minor things like faster connection due to not having to wait for a TCP handshake before doing a TLS/Noise/whatever handshake and just not being ossified like TCP.

A reliable transport built on UDP would still be very useful for OCapN because you need UDP for NAT hole punching, but idk if there's anything that can be taken off the shelf and used as-is.

3 Boosts

0 Likes

Daniel Lakeland

@dlakelan@mastodon.sdf.org yesterday

@dthompson
At a high level. What is OCapN?

Every time I hear about this stuff its at too low of a level. What sorts of things is it supposed to enable?

dave

@dthompson@toot.cat yesterday

@dlakelan The tl;dr is that OCapN is a secure RPC protocol. It's notable in that it uses the capability security model and is naturally asynchronous. Does this help?

Daniel Lakeland

@dlakelan@mastodon.sdf.org yesterday

@dthompson
It does help, makes it clear its a pretty general tech. What's the theory for where its supposed to be applied? Like, if it were mature and fully usable right now, what would be a decent example application you'd expect someone to want to build with it?

dave

@dthompson@toot.cat yesterday

@dlakelan I think OCapN fits in great with anything that you would consider to be part of the "control plane", where message order and integrity is critical. It is great fit for social programs like chat or anything ActivityPub is used for today, and even games with modest realtime requirements (not a twitchy FPS). OCapN would not be a good fit for use cases where it's fine if messages come in out of order or not at all, like in the aforementioned twitchy FPS case or realtime audio/video streaming (though it would be fantastic for all the other needs of such programs).

1+ more replies

dave

@dthompson@toot.cat yesterday

Some other reliable protocols built on UDP (or IP) are 1) RUDP, made by Bell Labs for Plan9, which has a draft RFC from 1999 and 2) SCTP, notably used by WebRTC. I think I can rule out SCTP as a good fit because it has multistreaming (like QUIC) and multihoming which are both unnecessary for OCapN. RUDP seems much simpler but anything that was ever deployed seems to use something that forks from the draft spec, but perhaps it's a good starting point.

dave

@dthompson@toot.cat 23 hr. ago

a point in favor of QUIC: it supports both reliable and unreliable communication over the same socket, so you could multiplex OCapN (reliable) and a video stream (unreliable).

Tim Panton

@steely_glint@chaos.social yesterday

@dthompson You could look at IL (also from plan9 and direct over IP but could probably run nicely over UDP or DTLS. )

Also - what are your target endpoints?

I'm playing with RTCTransport which is basically ICE+DTLS + Browser API - which might be interesting but you would have to add reliability somehow.

dave

@dthompson@toot.cat yesterday

@steely_glint thanks for the pointer re: IL. something that runs on UDP is appealing because I can implement it in userspace.

I'm thinking about this for "native" applications on linux, etc. with direct access to UDP sockets and thus can take advantage of NAT hole punching tricks.

I think I'd rather build directly on UDP than add reliability to something like RTCTransport.

dave

@dthompson@toot.cat yesterday

@steely_glint I am finding this page to be very informative https://9p.io/sys/doc/il/il.html

Vicent Ferrer

@vicentfg@mastodont.cat yesterday

@dthompson
A nitpick, SCTP doesn't run on top UDP, but IP directly.

It is widely used on telco, but Internet middle boxes usually mess up with it. Something on top of UDP would be more reliable.

dave

@dthompson@toot.cat yesterday

@vicentfg yes, you're right. should've been more clear.

dave

@dthompson@toot.cat yesterday

@vicentfg and thanks for the insight about UDP being more reliable!

Vicent Ferrer

@vicentfg@mastodont.cat yesterday

@dthompson I need to check OCap and OCapN in more detail. They seem really interesting concepts.

ArneBab

@ArneBab@rollenspiel.social yesterday

@dthompson Can there be only one peer per program? If there are multiple vats on the other node, could they use the same QUIC connection but different streams on it? That could delay running out of ports when communicating with many people. Consumer-OS’es can have quite low concurrent connection limits.

Why do you see zero roundtrip handshakes as minor? (especially if you don’t have to wait for a response)

One advantage of QUIC is that is doesn’t stand out, since HTTP3 serves as cover traffic.

dave

@dthompson@toot.cat yesterday

@ArneBab "peer" is a matter of identity, so it's possible that a program could contain multiple OCapN peer identities. Only one active connection is allowed between any two peers. Having multiple processes share one network connection to the same remote peer would be possible by using a local relay as proxy, no need for QUIC.

Zero RTT is not minor in the big picture but I just meant that in comparison to all the other stuff QUIC is doing it's relatively minor. It would be easy for another protocol built on UDP to achieve this.

For my use-case I'd want the first packet to start a Noise handshake, but QUIC incorporates a TLS handshake, so that's yet another reason why it's not a good fit.

ArneBab

@ArneBab@rollenspiel.social yesterday

@dthompson There’s always the option of just building your own layer on top of UDP.

It’s not like there is no precedent (looking at Freenet / Hyphanet …).

The ordering requirement makes that a bit expensive, though. Once you need the order preserved, just using TCP is usually a better fit than UDP, since it does all the heavy lifting, including load balancing, re-transmit, and so on.

Does UPnP not work anymore to open ports for TCP dynamically?

2+ more replies

d@nny disc@

@hipsterelectron@circumstances.run 3 days ago

@dthompson QUIC is a version of double ratchet from the signal protocol (trevor perrin was advising on both). i strongly recommend looking into that instead; that's what i'm basing a similar system on

dave

@dthompson@toot.cat 3 days ago

@hipsterelectron I think we're talking about two different QUICs. I'm talking about https://www.rfc-editor.org/rfc/rfc9000.html

RFC 9000: QUIC: A UDP-Based Multiplexed and Secure Transport

This document defines the core of the QUIC transport protocol. QUIC provides applications with flow-controlled streams for structured communication, low-latency connection establishment, and network path migration. QUIC includes security measures that ensure confidentiality, integrity, and availability in a range of deployment circumstances. Accompanying documents describe the integration of TLS for key negotiation, loss detection, and an exemplary congestion control algorithm.