FullHost - Investigating: Connectivity issues on hypervisor tor-kvm104 – Incident details

All systems operational

Investigating: Connectivity issues on hypervisor tor-kvm104

ResolvedMajor outage1 hour 10 minutes

Affected

FullHost.Com

Major outage from 3:24 PM to 4:34 PM

Toronto

Major outage from 3:24 PM to 4:34 PM

WordPress Hosting

Major outage from 3:24 PM to 4:34 PM

Virtual Machines

Major outage from 3:24 PM to 4:34 PM

  • ResolvedUTC

    Moving this to resolved, but we are continuing to monitor this incident closely.

  • MonitoringUTC

    Our on-site engineers have corrected the issue and the performance degradation affecting services in this pod has been resolved. All affected servers are operating normally.

    We are continuing to closely monitor the situation.

  • IdentifiedUTC

    One of the redundant power feeds serving a pod at our 151 Front St. facility is currently offline. The secondary feed remains active and servers in the affected pod are powered and online. However, a firmware bug in the affected Supermicro hardware is causing CPUs to throttle when running on a single power feed, resulting in significantly degraded performance for services hosted in this pod. Our engineers are actively working on a resolution and will post an update as soon as more information is available.

  • UpdateUTC

    Incident is affecting multiple hypervisors in Toronto

  • InvestigatingUTC

    We are aware of connectivity issues affecting hypervisor tor-kvm104 in our Toronto facility. A small subset of client VMs hosted on this node may be experiencing intermittent or degraded network connectivity. Our engineers are actively investigating and will provide an update as more information becomes available.