Planning Production Upgrades with minimal downtime

Many deployments of FCSDK, Live Assist or Palettes require service is always maintained.  CaféX clusters will maintain service during numerous types of server/network failure, however when a system requires an upgrade, service can become more difficult to maintain.  This article reviews how the Solution's architecture can be used to maintain service availability during a scheduled upgrade.
 
The features of the Web Gateway and Media Broker software must be used in conjunction with the existing architecture to support a seamless upgrade without a disruption. Servers must be stopped prior to an upgrade and it is recommended to keep the Media Broker & Web Gateway versions in-sync. The product Release Notes will confirm compatibility.
  
It is possible to gracefully shutdown Media Broker servers (see Administering FCSDK Guide -- Starting & Stopping Media Brokers).  This will prevent Media Brokers from processing new calls, but will not stop the Media Broker process running while there are active calls still in progress.
 
The Web Gateway does not have such a graceful process for upgrade; which means new calls could fail, while the upgrade is in progress.
 
In order to maintain service during an upgrade, we will consider an architecture that contains two FCSDK/LA Systems.  Such a solution allows the Customer to manually switch between each System without affecting established calls.  New calls will be directed to the second system.
 
 
Let us consider there were two isolated systems in place: System A & System B.
 
The customer's web application makes requests for session tokens from the Web Gateway.  The tokens are requested from System A.  The token is passed back to the client and it contains the URL (for System A) it can use to establish a WebSocket connection (to System A).
When an Upgrade is required: System B can be upgraded to the correct version (and tested using an isolated Web Application making requests from it).
Once System B is operational, the Web Application can start to make requests from System B; instead of the old System A.  Subsequent clients will be made to connect to System B to establish new calls.
 
Existing calls will be maintained on System A.  System A can be monitored to determine when all calls have ended, or by using the graceful shutdown procedure mentioned earlier.
Once all of the calls on System A have ended, this system can be upgraded for future testing, without disrupting their service.
 
The important component is the Web Application.  This is able to orchestrate which System is used and when; through configuration.
 
There are several points to consider when taking this approach:
 
The solution described is one of many possible variants and it is important to consider which components can be leveraged to define the solution.
 
Depending on scheduling System B, could be a smaller system used only while System A is upgraded.
 
Two independent systems are required, possibly with 2 public-FQDNs and 2 sets of Media Broker addresses.  Some customers use this environment for UAT, or production-like testing.  The benefit of this approach, is that infrastructure components like Load Balancers, Application Firewalls and DNS do not have to be updated to initiate the switch-over between CaféX Systems.
 
If there are changes to client-side libraries (Javascript), when the customer's web application is switched to direct traffic to the new-system, when a customer reloads their browser it may pick up the new libraries by mistake.  The web application developer must keep the old URL.
Have more questions? Submit a request

Comments

Powered by Zendesk