news.volyx.in

Incident Report: Railway Blocked by Google Cloud [resolved] (status.railway.com)

560 points by aarondf · 102 days ago · 357 comments on HN

Article summary

Railway experienced a service disruption due to Google Cloud blocking their account, causing some services to be unavailable. The issue has been resolved, and a post-mortem is available. The disruption affected users, with some experiencing errors and inability to access the dashboard. Railway is working on restoring access to all workloads and has apologized for the disruption.

Main themes

  • Cloud Service Disruption
  • Google Cloud Account Block
  • Railway Service Outage
  • Automated Account Bans
  • Cloud Dependency
  • Disaster Recovery

What commenters say

  • The outage highlights the risks of relying on a single cloud provider and the importance of having a backup plan.
  • Automated account bans can be problematic and may lead to unintended consequences, such as the disruption experienced by Railway.
  • Some argue that Google's support team is inadequate, while others have had positive experiences with their support.
  • Railway's decision to depend on Google Cloud for their infrastructure is seen as a poor planning choice by some, while others understand the complexity of the issue.
  • The incident has raised concerns about the reliability of Railway's platform and the potential consequences of putting all eggs in one basket.
  • Some commenters believe that using a multi-cloud architecture could help mitigate such outages, while others argue that it may lead to lower availability.
  • There are differing opinions on whether Railway was transparent about their infrastructure and dependencies, with some accusing them of being dishonest about owning their own data centers.