Why can't I use ASIO and VST plugins in teleconference apps like Zoom?

br0d · July 13, 2020, 2:05pm

martinfinlayson:

I’m certainly no Cubase expert but having been a VoIP and video over IP engineer for the last 20 years I think I can help explain the core issues that would explain why Zoom etc can’t deliver what you expect. Leaving the question of codec selection and configuration aside (which would determine things like audio bandwidth, latency and error correction), the main issue you face is your local firewall and the NAT function built into your Internet router. For a standard SIP p2p call to work both ends must be able to send media streams directly to each other, to do this they open up local dynamically assigned network ports on the devices IP address and tell the other end of the call where to send the media to on their end (in other words they only control the receiving ports). The ports that are assigned can be anything between 1024-65535. If both endpoints are on the same network this is fine because there are no network restrictions between the two devices (firewall, NAT etc) so they can talk freely to each other. However, firewalls and NAT both break this connectivity because they are not privy to the conversation between the endpoints so have no knowledge of how the call is setup and what dynamic ports are in use.
Home router firewalls typically allow all outbound traffic as the threat is perceived to be from the outside in, not the inside out. Therefore the endpoint has no problem sending the media stream to the far end. However, when that traffic arrives at the other end’s firewall it is treated as unsolicited traffic and is discarded. this means that both clients can send easily enough but not receive.
How services like Zoom etc get around this is to have the client register with the service (when you log in) on the internet and then when a call is setup the client establishes twice the number of required outbound UDP ports and the server on the other end then send the receive media back down the additional ports. For this to work both endpoints have to be registered with the cloud service so there is no way to not route the media streams through the intermediate service.
A very long-winded explanation but hopefully you get the idea.

I kinda get it. I used to work on firewalls for a living. I have not kept up with network engineering in some years and am far from a top tier expert. But the the thing that perplexes me about that limitation is that the firewall should be able to split the traffic into a control connection (which becomes effectively authentication for the data flow) and a data connection.

A firewall maintains a TCP state table of allowed connections, and as long as the control port is in state, the bidirectional data stream ought to be fine, unless the firewall software is just badly written and is just introducing latency in processing the actual flow data, which is probably the case for most SOHO devices.

The lack of ASIO availability in software that calls itself “AV” software is cumbersome, given where we are now. Not just for the interoperability with VST FX, but for latencies. Also of course some people just have like, 40Mbps ISP connections, and the >72ms latency is just inherent to transport there.

I see the “default driver” problem in the massively deployed, business critical apps like Zoom as a more easily resolvable one than the Firewalls or bandwidth issue, because business people had acclimated too much to crappy sounding conferences during a time period where conferencing was only a minority of their time. Now, it is pretty much all their comms.