High Availability
# Goals At the highest level, our goal is to create a highly available Trino setups. More specifically, we would like to accomplish the following: * Reduce impact of crashed coordinator * Simplify deployment architecture (e.g., fast failover, load balancers, etc.) * Reduce failures due to upgrades # Work Items * ~Dispatcher split~ * ~Move queue management into separate internal service (#95)~ * ~Secure submission query API~ * ~Split queued and executing client API~ * Dispatcher-only or coordinator-only server * Add only start necessary services depending on server role * Multiple dispatchers * Proxy for dumb load balancer * Durable resource groups * Multiple coordinators in a cluster * Add configuration to run shared discovery across coordinators * Add coordination for OOM killer and reserved memory pool * Multiple clusters * Cluster/coordinator discovery (each cluster has independent discovery) * Cluster selection system for queries # Deployment Architectures There are multiple ways to combine the above work items into different setups to achieve different goals. The following sections describes the most common setups: ## Multiple coordinators with Shared Queue In this setup there is a single dispatcher containing the shared queue, and multiple coordinators in the cluster. If the dispatcher crashes, all queued queries fail, but executing queries will continue. If a coordinator crashes, all queries managed by that coordinator fail, but other q
Posted by dain on Mar 6, 2019, 12:08 AM
Tags
Related issues
- ⚠️ PHP CVEs. Update required!
- I think RTS could continue to work if someone made a massive leap of faith and put all the modern infra around it that a AAA extraction s...
- RFC: Pluggable SessionDB Provider — PostgreSQL, MySQL, and Beyond
- v2.1.113+ broken on Termux/Android: native binary requires glibc, no JS fallback
- I’m firmly anti-Tahoe and haven’t updated, and I’ve started to find Apple despicable.
- [FEATURE] Link a source control repo as the source for organization skills
Raise real-world problems, find people facing the same issue, and discuss solutions.4,000 issues0 comments
reliability (Sysadmin)
⚠️ PHP CVEs. Update required!
I think RTS could continue to work if someone made a massive leap of faith and put all the modern infra around it that a AAA extraction s...
RFC: Pluggable SessionDB Provider — PostgreSQL, MySQL, and Beyond
v2.1.113+ broken on Termux/Android: native binary requires glibc, no JS fallback
I’m firmly anti-Tahoe and haven’t updated, and I’ve started to find Apple despicable.
[FEATURE] Link a source control repo as the source for organization skills
Feature Request: Option to disable paste text collapse in input field
We went down the WASM sandboxing rabbit hole at Gobii when building our agent infra.
High Availability
# Goals At the highest level, our goal is to create a highly available Trino setups. More specifically, we would like to accomplish the following: * Reduce impact of crashed coordinator * Simplify deployment architecture (e.g., fast failover, load balancers, etc.) * Reduce failures due to upgrades # Work Items * ~Dispatcher split~ * ~Move queue management into separate internal service (#95)~ * ~Secure submission query API~ * ~Split queued and executing client API~ * Dispatcher-only or coordinator-only server * Add only start necessary services depending on server role * Multiple dispatchers * Proxy for dumb load balancer * Durable resource groups * Multiple coordinators in a cluster * Add configuration to run shared discovery across coordinators * Add coordination for OOM killer and reserved memory pool * Multiple clusters * Cluster/coordinator discovery (each cluster has independent discovery) * Cluster selection system for queries # Deployment Architectures There are multiple ways to combine the above work items into different setups to achieve different goals. The following sections describes the most common setups: ## Multiple coordinators with Shared Queue In this setup there is a single dispatcher containing the shared queue, and multiple coordinators in the cluster. If the dispatcher crashes, all queued queries fail, but executing queries will continue. If a coordinator crashes, all queries managed by that coordinator fail, but other q
Comments (0)
No comments yet. Start the discussion.