We isolated the issue down to a code issue which was deployed earlier in the day. The issue was caused by the rollup process of our data pipeline to read more data than usual from our Cassandra metrics store. This caused Cassandra stability issues and in turn the rollups to slow down. Our team will be doing a post-mortem to avoid such issues in the future. Please reach out to Sysdig support in case you have any questions. Our sincere apologies for the downtime.
Posted Dec 19, 2017 - 13:44 UTC
Identified
We are experiencing an issue with our data pipeline where rollups of data are delayed. Additionally the system might have generated incorrect alerts. The approximate start time of the issue was 12:10 UTC. We will post more updates as we make progress. Please contact Sysdig support for any questions you might have.