Dear XXX YYY,
Over the last 10 days multiple leased line and data centre faults have affected us.We have clients distributed across multiple leased lines connecting to the exchanges from our data center. Each line has one backup line as well. After almost 1 year of no issues, we have had issues crop up in rapid succession.
1. Tuesday, 3rd May at 3.20 pm, one of the lines along with the backup failed. It was too near to the closing of the market, hence we couldn't switch clients from this line to another.
2. Wednesday, 4th May at around 10.20 am, again the same lines with the backup failed. Since the exchange technology in India doesn't support active failover, took us over 20 minutes to move clients manually from one to another. Orders placed before this incident wouldn't get a status update since the clients were moved to a separate line. Hence the orders placed earlier couldn't be cancelled and status was shown pending until around 2 pm, when we could connect the old lines and reconcile.
3. Monday, 9th May at around 9.20 am. Our team worked right through the weekend to ensure the best is done so that we are ready if the leased lines were to fail again. We flew some of the team to Mumbai and worked with two different data centres and ISPs. We were also super excited about releasing the new Kite web app on Monday into which a lot of love and hard work has been put in.
You can call it extreme misfortune that our second data center in Mumbai, where Kite servers are located, developed issues with internet connectivity (although we have multiple ISPs peered to it). So connectivity was on and off for almost an hour, probably also causing an impression that it was due to the new launch. All the other platforms were working perfectly fine.
In times when there are issues, our support team is subjected to exponential increase in work load and pressure, hence slower response time. I acknowledge that we can do better to proactively inform you with regular updates in such times. We will do the same from now on
https://zerodha.com/marketintel/Bulletin/
I am sorry for the inconvenience and heartache this has brought. It's extremely unfortunate that it was completely out of our control, despite all the failsafes we have in place to avoid scenarios such as these. I want to assure all of you that, we have been doing everything possible to ensure that you have the best experience, and we will continue to do so.
Thank you for your continued support.
Regards
Nithin Kamath
Founder & CEO
