GitHub header
All Systems Operational
Git Operations ? Operational
API Requests ? Operational
Webhooks ? Operational
Visit www.githubstatus.com for more information Operational
Issues ? Operational
Pull Requests ? Operational
Actions ? Operational
Packages ? Operational
Pages ? Operational
Codespaces ? Operational
Copilot Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Past Incidents
Apr 25, 2024

No incidents reported today.

Apr 24, 2024
Incident with Pull Requests
Resolved - This incident has been resolved.
Apr 24, 18:20 UTC
Update - The previous mitigation has been rolled back and updates to the pull request merge button should be working again. If you are still seeing issues, please attempt refreshing the pull request page.
Apr 24, 18:01 UTC
Update - One of our mitigations from the previous incident caused live updates to the pull request merge button to be disabled for some customers. Refreshing the page will update the mergability status.
Apr 24, 17:45 UTC
Investigating - We are investigating reports of degraded performance for Pull Requests
Apr 24, 17:40 UTC
Incident with Pull Requests, Git Operations, Actions, API Requests, Issues and Webhooks
Resolved - This incident has been resolved.
Apr 24, 16:16 UTC
Update - Issues is operating normally.
Apr 24, 16:13 UTC
Update - Actions is operating normally.
Apr 24, 16:13 UTC
Update - Pull Requests is operating normally.
Apr 24, 16:13 UTC
Update - Webhooks is operating normally.
Apr 24, 16:13 UTC
Update - Git Operations is operating normally.
Apr 24, 16:13 UTC
Update - API Requests is operating normally.
Apr 24, 16:12 UTC
Update - We are seeing site-wide recovery but continue to closely monitor our systems and putting additional mitigations in place to ensure we are back to full health.
Apr 24, 15:50 UTC
Update - We are continuing to see consistent impact, and we’re continuing to work on multiple mitigations to reduce load on our systems.
Apr 24, 14:08 UTC
Update - We have found an issue that may be contributing additional load to the web site and are working on mitigations. We don't see any additional impact at this time and will provide another update within an hour if we see improvements or fully mitigate the issue based on this investigation.
Apr 24, 12:47 UTC
Update - We have taken some mitigations and see less than 0.3 percent of requests failing site wide but we still see elevated 500 errors and will continue to stay statused and investigate until we are confident we have restored our error rate to base line.
Apr 24, 12:00 UTC
Update - We are seeing increased 500 errors for various GraphQL and REST APIs related to database issues. Some users may see periodic 500 errors. The team is looking into the problematic queries and mitigations now.
Apr 24, 11:13 UTC
Update - Actions is experiencing degraded performance. We are continuing to investigate.
Apr 24, 11:09 UTC
Update - Git Operations is experiencing degraded performance. We are continuing to investigate.
Apr 24, 11:06 UTC
Update - Pull Requests is experiencing degraded performance. We are continuing to investigate.
Apr 24, 10:55 UTC
Update - Webhooks is experiencing degraded performance. We are continuing to investigate.
Apr 24, 10:52 UTC
Update - Issues is experiencing degraded performance. We are continuing to investigate.
Apr 24, 10:51 UTC
Investigating - We are investigating reports of degraded performance for API Requests
Apr 24, 10:45 UTC
Incident with Git Operations
Resolved - This incident has been resolved.
Apr 24, 11:01 UTC
Investigating - We are investigating reports of degraded performance for Git Operations
Apr 24, 10:56 UTC
Apr 23, 2024

No incidents reported.

Apr 22, 2024

No incidents reported.

Apr 21, 2024

No incidents reported.

Apr 20, 2024

No incidents reported.

Apr 19, 2024

No incidents reported.

Apr 18, 2024
Incident with Codespaces
Resolved - This incident has been resolved.
Apr 18, 18:47 UTC
Update - Codespaces customers using our 16 core machines in West US 2 and West US 3 region may experience issues creating new Codespaces and resuming existing Codespaces. We suggest any customers experiencing issues switch to the East US region.
Apr 18, 18:41 UTC
Investigating - We are investigating reports of degraded performance for Codespaces
Apr 18, 18:25 UTC
Apr 17, 2024
Incident with Copilot
Resolved - This incident has been resolved.
Apr 17, 00:48 UTC
Update - We're continuing to investigate issues with Copilot
Apr 17, 00:30 UTC
Update - Copilot is experiencing degraded performance. We are continuing to investigate.
Apr 16, 23:59 UTC
Update - We're investigating issues with Copilot availability
Apr 16, 23:57 UTC
Investigating - We are currently investigating this issue.
Apr 16, 23:51 UTC
Apr 16, 2024
Apr 15, 2024
Incident with Copilot
Resolved - This incident has been resolved.
Apr 15, 14:53 UTC
Update - We have applied mitigation for Copilot in EU region and are working towards the full recovery of the service.
Apr 15, 14:13 UTC
Update - Due to an outage in one Copilot region traffic is currently being served from other regions. European users may experience higher response times.
Apr 15, 13:35 UTC
Investigating - We are investigating reports of degraded performance for Copilot
Apr 15, 12:58 UTC
Apr 14, 2024
We are investigating reports of degraded performance.
Resolved - Beginning at 17:30 UTC on April 11th and lasting until 20:30 UTC on April 14th, github.com saw significant (up to 2 hours) delays in delivering emails. At 14:21 UTC on April 14th, community reports of this were confirmed and an incident declared. Emails most impacted by the delay were password reset and unrecognized device verification, which contain time-sensitive links or verification codes, and are required to be acted on in order for password resets or unrecognized logins to proceed.

Users attempting to reset their password during the incident were unable to complete the reset. Users without two-factor authentication (2FA), signing in on an unrecognized device, were unable to complete device verification. Enterprise Managed Users, users with 2FA, and users on recognized devices or IP addresses were still able to sign in. This impacted 800-1000 user device verifications and 300-400 password resets.

The mailer delays were caused by increased usage of a shared resource pool; a separate internal job queue became unhealthy and prevented the mailer queue from being worked on.

We have made some immediate improvements to better detect and react to this type of situation again. As a short-term mitigation strategy, we have added a queue-bypass ability for time-sensitive emails, like password reset and unrecognized device verification. We can enable this setting if we observe email delays reoccurring, which will ensure that future incidents do not affect user ability to complete critical login flows. We have paused the unhealthy job queue, to prevent impact to other queues using shared resources. And we have updated our methods of detection for anomalous email delivery, to identify this issue sooner.
Apr 14, 21:53 UTC
Update - We are seeing a full recovery. Device verification and password reset emails are delivered on time.
Apr 14, 21:52 UTC
Update - We are deploying a possible mitigation to the delayed device verification and password change emails.
Apr 14, 21:34 UTC
Update - We continue to investigate issues with delays of email deliveries which is preventing users without 2FA enabled from verifying new devices. We will provide more information as it becomes available.
Apr 14, 19:54 UTC
Update - We are continuing to investigate issues with the delivery of device verification emails for users without 2FA.
Apr 14, 15:50 UTC
Update - We are continuing to investigate issues with the delivery of device verification emails for users without 2FA on new devices.
Apr 14, 15:01 UTC
Update - Device verification emails for sign-ins for users without 2FA on new devices are being sent late or not at all. This is blocking successful sign-ins for these users. We are investigating.
Apr 14, 14:27 UTC
Investigating - We are currently investigating this issue.
Apr 14, 14:21 UTC
Apr 13, 2024

No incidents reported.

Apr 12, 2024

No incidents reported.

Apr 11, 2024

No incidents reported.

← Incident History Powered by Atlassian Statuspage