When checking the Datadog dashboard widget for HTTP 500 errors, when there is a high amount of them, there is frequently the need of checking out what specific URLs are causing the errors so they can be further analysed and hopefully get rid of the errors in the long term.

HTTP 500 Dashboard:

Access to Datadog is limited only to people with access to the Cockpit. Use the same LDAP login credentials for the Cockpit to log in to Datadog.

For subscription packages created since the beginning of February 2021, all Cockpit users are automatically added to the Datadog organization. Therefore, they can access the Datadog just by clicking the link in the Cockpit -> Package overview -> Additional information -> Logs view URL. Email invitations won't be sent to these users any longer.

Step-by-step guide

In order to check the Dashboard:

  1. In Dashboards section, click on "Customer xxx-xxxx - Overview Dashboard"
  2. In the "Public instances" section click on the expand button of HTTP 500
  3. Try to reduce the time frame in order to get a clear sample of the high amount of HTTP 500 errors
  4.  More info: https://docs.datadoghq.com/dashboards/guide/custom_time_frames/

In order to check the logs to find out the URLs causing the HTTP 500 errors showed in the Dashboard:

  1. In the Logs section click on "Search"
  2. Copy paste or manually set the same time frame as the one in the Dashboard
  3. In the filter type in "source:elb" and "status:error"
  4. The results can be further filtered out to only HTTP 500 errors by adding "@http.status_code:500" to the filter. More info: https://docs.datadoghq.com/logs/search_syntax/
  5. In order to see the URLs, click on each row and check the "Event attribute" http → ssl → url or http→ url_details
  6. You can also enable the URL as a column in the list by selecting "Add a column: @http.url" in the Options button. Note: Date, host, service and content columns can be removed in order to get cleaner data.
  7.  More info: https://docs.datadoghq.com/logs/explorer/
  8. The log list can be also exported to CSV for further and more convenient analysis

Keep in mind logs are stored for 15 days in Datadog. In order to check older logs you need to enable/order "Cold storage". More info: https://docs.magnolia-cms.com/cloud/Managing-environments-using-the-Magnolia-cockpit/Monitoring-and-logging.html#_logs
Keep in mind logs can be exported to up to 5000 lines. Try to shorten the time frame in order to export less than 5000 lines. Export can be done as many times as needed with multiple time frames. More info: https://docs.datadoghq.com/logs/explorer/#export