Incident Management Configuration Settings

Users can define different Incidents per incident workflows and types. This allows you to configure settings related to incidents, postmortems, and integrations for different use cases.

Elements affecting Incident Management

note

Two key elements of Incident Management are located under the Blameless UI’s “Organization” parameters.

  • Incident Roles

  • Severity Levels

Incident Roles

Incidents within an organization are usually dealt with by an Incident Response Team (IRT), or an Incident Management Team (IMT). These team roles are often defined beforehand or (in emergency situations) during the event.

Although some of the details vary by company, their product(s), and the severity of the effect on the world outside of the company (i.e., a small company will probably not have the same effect as say a nuclear plant), the roles assigned normally consists of five main elements:

  • Commander

  • Comms Lead

  • Technical Lead

    In some cases, specialized team positions, such as public affairs and safety, can report directly to the Incident Commander (IC) when the event warrants establishment of those positions.

Severity Levels

Severity levels can be highly custom—reflecting the nature of the business and its product(s).

note

Best practices suggest tend to reflect an alphanumeric label (i.e., Sev1).

Blameless’ implementation also reflects this—allowing the user to set their own interpretation of the best case implementation.

Creating a Blameless Incident Type

Once you open the Incident Type window, you will find a list of created types and a couple of administration options.

  1. If Creating a New Incident type: Click on the Settings (gear) icon and follow the instructions.

  2. If working with an Existing Incident type: Click on Incident (fire) icon and follow the instructions.

note

Currently, the only action available under the meatball icon is to delete this Incident type.

New Incident Type

  1. Click on the Settings (gear) icon on the Blameless UI left navigation menu.

  2. Select Incident Workflows from the resulting pop-out modal.

  3. Select the “+Add Type” button in the upper right corner of the window.

note

All fields in the Incident Type creation modal with an asterisk (*) are Required Fields.

  1. Enter the Incident Type Name in the text field.

  2. Enter a Group name and associate this new Incident type with an existing group via group Email address. Groups can be used to associate permissions for Blameless resources such as incidents and postmortems and a collection of users.

    note

    The admin group is available by default.

  3. Enter your e-mail address for the group.

  4. Enter a Type description.

  5. Click on “SAVE” to add the new Incident type.

Existing Incident Type

  1. Open the Blameless UI.

  2. Select the Incident (fire) icon.

  3. Review the list of recognized Incident Types.

Blameless Incident Types

Located under the “Settings” (gear) icon, “Incident Types” are stored under the “Incident Workflows” menu option.

Incident Workflow

Blameless allows for the creation of customized types. Below are some examples:

  • DEFAULT

  • Cloud Operations

  • Compliance

  • Customer Incident

  • Error Budget Depletion

  • Hotfix

  • Release

  • Security

  • Training

    Settings for the Incident type examples appear below. Within the “Incident Types” window, you can see the created type examples, each has the following administrative action:

  • “Manage” button

  • “…” (commonly called the “ellipse” icon)

Incident Types

Managing the Incident Type

Clicking on the “Manage” button opens a new window that offers the user the ability to customize each Incident type independently of any other types they create. This starts with the naming scheme for the Incident, activating Incident creation via webhook (if desired), and identifying the Slack channel(s).

  • Force Incident creation via webhooks

  • Incident naming scheme

  • Enable private Slack channels

  • Invited Slack groups or users

  • Slack channels for Incident announcements

note

At the top of each Type Management window, you will see a tabbed menu containing the Default management window (opens to this window) as well as the Severity (SEV) levels as defined under the General Organization settings.

Default settings apply to all sev levels; If you edit settings on a sev tab, then it becomes independent of the Default for that type.

Additional sub-features available to customize the type appear below.

Important

Once complete, the user MUST click on the “SAVE” button to retain these custom values.

Incident Type settings

Force Incident creation via webhooks

This allows webhook requests from Integrations using webhooks (i.e., PagerDuty) to create a new incident even when existing incidents are in progress.

Incident naming scheme

When you click on the Incident naming scheme field, a dropdown of options appears. These include:

  • Numeric

  • Date

  • Issue Name

  • Pet Name

Numeric

This scheme follows INC-\<num>, i.e., INC-123.

Date format

The date takes on the YYMMDD format. If there are multiple Incidents, the date format will append “-1”,”-2”, etc. to the name.

Issue Name

This scheme uses \<project-Issue-ID> as the format.

Pet Name

You can create your own name to coincide with your product or naming conventions. Incident \<Adjective> \<Noun> is the format.

note

For example,

_incident-dear-camel

#_incident-glad-husky

Enable private Slack channels

Blameless allows you to create and map Incident(s) to both public (default value) and private Slack channels.

Invited Slack groups or users

You can identify which groups or users have been invited to monitor this particular Incident type.

  1. To Add groups or users: Enter the \@user / group name and press Enter. For example: \@Engineering.

Slack channels for Incident announcements

Enter the invited channels here targeting specific Incidents

  1. To Add groups or users: Enter the hashtag channel name and press Enter. For example: #Announcements.

Checklist Tasks

Continuing on down the Management window, the next section is Checklist Tasks. This is the list of subsections to be executed at each stage of the incident.

note

The double stack of six dots allows the user to drag and drop the order of the Tasks.

Incident Type settings

They are broken out as follows:

  • Investigating

  • Identified (cause)

  • Monitoring (progress)

  • Resolved

Customization of these subsections occur using the icon sets located on the right hand side of the window associated with sub tasks for each subsection.

Customization actions

  • Asterisk

  • “Ellipse” (icon)

  • Trash can (icon)

Customization options

Asterisk

Selecting the asterisk flags this subtask as a required event. If selected, the asterisk changes color to red.

note

A Required Task must be marked as completed before the Incident can be completed.

Ellipse

Selecting the “Ellipse” offers the option to add a link to this task.

Trash can

Deletes the subtask from the subsection.

Customizing Checklist Task components

Within each task is a series of steps to be completed--these include assigning roles (selectable via dropdown menu), tagging a Sub-task as required” or not, setting a recurrence frequency, adding a link to a task, and deleting the task within each sub-task.

Adding New Task(s)

You can also add a new task using the “+Add Task” link at the bottom of each subsection.

  1. To add a new Task: Click on the “+Add Task” link.

  2. Enter the name of the task.

  3. Select the Incident role that is assigned.

  4. Determine if it is a Required need or if you want to link to another resource.

  5. When you hit RETURN, the task is saved.

Incident Tag Categories

The following preliminary categories appear in the Blameless UI:

  • Contributing Factor

  • Customer

  • Product

  • Service

These values can be used as is, or as an organizer for custom values. Blameless allow allows you to create custom Incident Tag Categories using the “+Add Category” option.

Adding Custom Categories

You can also add custom category tags.

  1. To create a custom category: Click on the “+Add Category” link below the preliminary Category list.

  2. To create a predefined tag under the category: Click on the field. A drop down appears, where you are prompted top add a custom tag.

note

Adding a tag to the list of predefined tags restricts selectable options for this category.

Analysis Template

This feature allows the user to create a structured, standard template to document the results and to suggest improvements.

note

The Analysis template is built in the Settings > Incident Types section. The end product appears in the Post Mortem.

Analysis Template

Custom Postmortem Questions (defined using JSON schema)

Allows creation of a custom FAQ list using JSON Schema elements as part of the Post Mortem section.

The upper left corner of the JSON window has three tools for editing the JSON code.

  • Displaying the JSON Data

  • Displaying a compact version of the JSON Data

  • Repairing the JSON data

note

You can click on the “Example” button to see a sample FAQ using JSON.

The JSON-based Custom questions are built in the Incident Type settings window, but display in the Post Mortem window.

Analysis template output

Completion SLA

Allows the user to set up completion alerts per SLA rules regarding when the Incident and Post Mortem must be done by.

Require Postmortem

Setting the slider to active forces the Incident team to complete a Post Mortem of the incident.

Enable Reminders to complete Postmortem

Turns on reminders for the Distribution list of e-mail addresses if the PM is not published.

PagerDuty auto triggers

Enable auto-trigger of PagerDuty incidents for this incident type via the slider. When a Blameless Incident gets triggered, by activating this feature, a PagerDuty event also occurs. Refer to PagerDuty Integration Guide.

PagerDuty Services

If activating auto-triggers above, selecting the PagerDuty services (appearing in the drop down) will trigger the alerts to the selected services.

Auto assign a role to Creator

When an incident is launched, this auto assigns a role to the creator of a blameless incident.

Default JIRA project for Incident tickets

This is the project that JIRA tickets will get created in automatically when a Blameless incident is created. Refer to the JIRA Integration Guide for more information.

Default JIRA issue type for Incident tickets

Select the default JIRA issue type to assign the incident to. This is the type of issue (bug, task, story, etc.) that gets created whenever a JIRA ticket is created. Refer to the JIRA Integration Guide for more information.

Incident summary

This is a required JIRA field for the selected project and issue type for an incident JIRA ticket. Refer to the JIRA Integration Guide for more information.

SLA use traditional business week for days

As it sounds, Blameless allows you to drive the timeline based on the work week instead of the standard seven day week.