An entity is an object that represents a software construct. A defined collection of entities is known as a catalog. All entities are represented in YAML, can pull in data from integrations, and can be Scorecarded.
Each entity has its own page that centralizes data related to it, allowing you to quickly access the information you're looking for. Learn more in the Entity details page documentation.
You can create your own entity types or use one of the default entity types.
By default, Cortex supports the following entity types:
Services: The default type and the core of your catalog, used for entities such as microservices, libraries, components, etc – essentially any codebase-like module. Learn more in Add services.
Domains: An entity that allows you to group your services, resources, and other domains into hierarchical, logical units. Learn more in Add domains.
Teams: An entity to store all of your team data. Learn more in Add teams.
Cortex also supports pulling in resources directly from AWS, Azure Resources, and Google Cloud as their corresponding types out of the box.
In addition, you can create custom entity types to fit your organization's needs. Learn more in Add custom entity types.
Whether you use the UI or GitOps to manage entities, every entity in your Cortex catalogs is defined by a YAML file called the Cortex entity descriptor, also referred to as an entity's Cortex YAML
(stemming from cortex.yaml
, the name of the file that powers the GitOps approach.)
Each file is a fully compliant OpenAPI 3 spec file, with our own Cortex-specific extensions.
You can still use Cortex if you don't use OpenAPI/Swagger. We use the OpenAPI spec as a base for entity metadata, since it's an open spec with official support for extensions. That lets us extend it to be an entity descriptor spec with optional usage of actual OpenAPI fields.
Any additional metadata you want to add to your descriptor belongs in the info
section. Throughout the docs, you'll see snippets that start with x-cortex-*
– make sure to add this to the info
section.
Note that it is not currently supported to include comments in YAML files.
By default, your account is configured to edit entities through the UI. We recommend starting with the UI editor, which includes a built-in YAML editor, then switching over to GitOps when ready for a wider rollout.
If you want to use GitOps to manage your entity YAML files, you can change this setting:
In Cortex, navigate to Settings > GitOps.
Scroll down to the "Options by entity type" tile.
To only allow editing of entities via GitOps, toggle off the setting next to Enable UI editing for new entity types.
To only allow creation of entities via GitOps, toggle off the setting next to Enable UI importing for new entities.
When using GitOps, you can define any number of entities in any repository.
Domain definitions should live in the .cortex/domains
directory in your repository.
Team definitions should live in the .cortex/teams
directory in your repository.
You may store all of your entity definitions in a single repository or you can store the YAML in the repository of the corresponding service.
See the single repository example below:
Read more in Using GitOps for Cortex.
Entity tag
The entity tag - the value of x-cortex-tag
in an entity's YAML file - is a unique identifier that's used throughout Cortex. For example, the entity tag is used to declare dependencies on other entities or look up information using the Cortex Slack Bot.
The entity tag must be globally unique across all entities. It is possible to customize the entity tag.
Cortex ID
A Cortex ID (CID) is a unique, immutable entity ID that can be used outside of Cortex for historical tracking and reporting. It can be used in the API, in CQL queries, in-app, and you can use it with the Cortex Slack Bot. The ID is automatically generated by Cortex and it is 18 characters in length.
In Cortex, the CID for an entity appears in the URL for an entity and at the top of an entity page:
The CID does not appear in an entity's YAML file, as it cannot be modified.
After an entity is imported or created, it will automatically belong to a catalog based on the entity type criteria set for each catalog. When you create a custom catalog, you can set the entity type criteria.
For the default catalogs, the entity type criteria is set by default:
The Services catalog contains service
entity types
The Domains catalog contains domain
entity types
The Teams catalog contains team
entity types
The Infrastructure catalog contains any entities that are not the types service
, domain
, or team
.
By default, any custom entity types you create will belong to the Infrastructure catalog. If you do not want the entity type to belong to this catalog, you can add the entity type to a different catalog.
The All entities page lists all entities that have been imported into Cortex, across all types. This page will open to the Mine tab when a user owns any entities; otherwise, it will default to the All entities tab.
Once you’ve imported entities, you’ll be able to see the entity’s name and tag. In the beginning, you may have just a few dozen entities, but eventually you may have hundreds, if not thousands, of entities across all catalogs.
There are several ways to search and filter your entities list:
To find specific entities, use the search bar in the upper right corner of the entities list and type to search.
Click Name to select whether you want to sort by name or identifier, and whether to sort ascending or descending.
Click Display to choose whether to show archived entities in the list, and select which columns to display alongside entities. Learn more about the performance indicator columns below.
Click Filter to narrow down your list by AWS account ID or region, domain, entity type, group, team, or user.
Click Display to select whether to display the following key performance indicators as columns alongside entities:
Incidents: Displays how many active incidents are associated with a given entity. This information is pulled from FireHydrant, PagerDuty, and Rootly.
Health: When you click into this column in an entity's row, you can view more information on:
Monitors: Shows how entities are performing against monitors you’ve created in your APM. When an entity is passing all monitors, this cell will display All OK
; otherwise, it will display Failing
, Warning
, or No Data
, along with how many monitors the entity is not hitting. This information is pulled from Datadog.
Error rate: Shows the percentage of transactions that result in a failure during a pre-specified window. This information is pulled from New Relic.
If you have not set up an integration needed to populate a column, that column will not appear on the "All entities" page or on specific catalogs. If an integration is established, but the data and/or configuration for an entity do not exist, the cell will display “None.”
To view all entity types, go to Catalogs > All Entities then click the Entity types tab.
A Cortex logo appears next to built-in entity types. When you hover over the logo, you will see a "Powered by Cortex" banner:
The other entity types in the list are custom types.
To learn more about adding each type of entity, see their documentation pages:
Services are a default entity type in Cortex, used for entities such as microservices, libraries, components, etc – essentially any codebase-like module.
To view services, click Catalogs > Services from the main nav.
When you open the Services page, you'll see tabs labeled Mine and All, which denote services you own and all services visible to you in your Cortex workspace.
At the top of your services list, you can choose how the list is displayed:
Name: Click Name, then choose whether to sort by name or identifier, and whether the sort order should be ascending or descending.
Display: Click Display, then choose whether to display hierarchies, whether to show archived, and which columns to display.
You can create services:
By importing them from a connected integration
Manually in the Cortex UI
Via the entity descriptor YAML through GitOps
Via the API
You can import services directly from third-party integrations:
In Cortex, navigate to Catalogs > All entities, then click Import entities.
Select the integration to import from.
If you have a large volume of entites, click Filter in the upper right corner of the results list to select and apply entity type filters.
At the bottom of the page, click Next step.
Edit the details for the entity:
Type: Select Service.
Entity name: Enter a human readable name.
Identifier: This field is auto-populated based on your entity name. It is a unique identifier for your entity. This is also known as the x-cortex-tag
.
Description: Enter a description of the entity to help others understand its purpose.
Groups: Select groups to segment your entity.
Owners: Define ownership for your entity. We recommend selecting team owners to keep your ownership information up-to-date through any future personnel changes.
You may see owners that Cortex recommends based on repository activity. You can accept or reject the recommendations.
Links: Add links to external documentation, such as runbooks, docs, logs, or custom categories.
Parents: Define parent domains. This is where you configure the hierarchy for your entity. These can be visualized in the relationship graph.
Dependencies: Select entities that this entity depends on. These can be visualized in the relationship graph.
On-call: Configure on-call information.
Repository: Select the repository associated with this entity.
Click Confirm import.
To create a service:
In the main nav of Cortex, click Catalogs > Services.
At the top of the Services page, click +Import entities.
Configure the form:
Type: Select Service.
Entity name: Enter a human readable name.
Identifier: This field is auto-populated based on your entity name. It is a unique identifier for your entity. This is also known as the x-cortex-tag
.
Description: Enter a description of the entity to help others understand its purpose.
Groups: Select groups to segment your entity.
Owners: Define ownership for your entity. We recommend selecting team owners to keep your ownership information up-to-date through any future personnel changes.
Links: Add links to external documentation, such as runbooks, docs, logs, or custom categories.
Parents: Define parent domains. This is where you configure the hierarchy for your entity. These can be visualized in the relationship graph.
Dependencies: Select entities that this entity depends on. These can be visualized in the relationship graph.
On-call: Configure on-call information.
Repository: Select the repository associated with this entity.
When you are finished, click Confirm import at the bottom of the page.
Service entity descriptor
A barebones spec file has the OpenAPI version, along with an info
section that contains some basic details.
Required fields
The only required fields under info
are the title
, x-cortex-tag
, and x-cortex-type
. The description is optional, but we highly recommend adding descriptions to all of your entities.
Example cortex.yaml for a service
You can create, update, and delete services using the Cortex API.
It is possible to edit entities after creating them:
Navigate to the entity's page.
In the upper right corner, click Configure entity.
Make any desired changes.
Note: The only field you cannot edit is the identifier.
At the bottom of the screen, click Save changes.
Domains offer a way to group entities into hierarchical units. You can group by product area, functionality, systems, business units, or something unique to your organization. With this feature, you can cluster entities into a single, hierarchical domain that can include both parents and children.
You can define a list of other entities as children for a domain, allowing you to represent a hierarchy of how your entities are modeled across your workspace. This hierarchy is available to view in the Domains catalog, on a domain entity's details page, and in the relationship graph. The domain hierarchy can also be used to configure ownership inheritance, helping you keep track of ownership in case of personnel changes at your organization.
You can view all domains under Catalogs > Domains.
To display domains in a hierarchy:
Click Display at the top of the domains list.
Click Done.
You can also view the hierarchy for a given domain on its entity details page. If the domain has parents or children, those will appear on the Relationships page.
In the entity's side bar, click Relationships, then click the Hierachy tab.
At the top of the domain, click View in domains tree to visualize your domain hierarchy in the relationship graph.
You can create domains:
By importing them from a connected integration
Manually in the Cortex UI
Via the entity descriptor YAML through GitOps
Via the API
For simplicity, we recommend adding the highest-level domain first and then selecting it as the parent for subsequent domains. However, you can add parents and children to any domain at any point.
You can import domains directly from third-party integrations:
In Cortex, navigate to Catalogs > All entities, then click Import entities.
Select the integration to import from.
If you have a large volume of entites, click Filter in the upper right corner of the results list to select and apply entity type filters.
At the bottom of the page, click Next step.
Edit the details for the entity:
Type: Select Domain.
Entity name: Enter a human readable name.
Identifier: This field is auto-populated based on your entity name. It is a unique identifier for your entity. This is also known as the x-cortex-tag
.
Description: Enter a description of the entity to help others understand its purpose.
Groups: Optionally select groups to segment your entity.
Owners: Define ownership for your entity. We recommend selecting team owners to keep your ownership information up-to-date through any future personnel changes. You may see owners that Cortex recommends based on repository activity. You can accept or reject the recommendations.
When adding an owner, you can also configure one of the following inheritance options:
Append: Select this option to add your entity as an additional owner to all of its child entities.
Fallback: Select this option to add your entity as an owner to child entities if the child entity has no other valid owners.
None: Select this option if you do not want to configure inheritance. The owner will own the domain you are creating, but will not be configured as an appended or a fallback owner.
Parents and Children: Define parent and children domains. This is where you configure the hierarchy for your domain. These can be visualized in the relationship graph.
On-call: Configure on-call information.
Repository: Select the repository associated with this entity.
When you are finished, click Confirm import at the bottom of the page.
Click Confirm import.
In the main nav of Cortex, click Catalogs > Domains.
At the top of the Domains page, click +Import entities.
Configure the form:
Type: Select Domain.
Entity name: Enter a human readable name.
Identifier: This field is auto-populated based on your entity name. It is a unique identifier for your entity. This is also known as the x-cortex-tag
.
Description: Enter a description of the entity to help others understand its purpose.
Groups: Optionally select groups to segment your entity.
Owners: Define ownership for your entity. We recommend selecting team owners to keep your ownership information up-to-date through any future personnel changes.
When adding an owner, you can also configure one of the following inheritance options:
Append: Select this option to add your entity as an additional owner to all of its child entities.
Fallback: Select this option to add your entity as an owner to child entities if the child entity has no other valid owners.
None: Select this option if you do not want to configure inheritance. The owner will own the domain you are creating, but will not be configured as an appended or a fallback owner.
Parents and Children: Define parent and children domains. This is where you configure the hierarchy for your domain. These can be visualized in the relationship graph.
On-call: Configure on-call information.
Repository: Select the repository associated with this entity.
When you are finished, click Confirm import at the bottom of the page.
If your entity is a domain, you must specify x-cortex-type
as domain
:
Domain hierarchies
These can be defined in two ways:
From the parent entity YAML
Define children entities using the x-cortex-children
tag in the parent entity YAML.
From the child entity YAML
Define parent entities using the x-cortex-parents
tag in the child entity YAML.
While defining these relationships top-down (on the parent entity) or bottom-up (on the child entity) will both result in a hierarchy, you may choose one or the other depending on your workflow. For example, if you are frequently making changes to a group of children domains, it may be more efficient to have the children defined on the parent YAML.
Domain children
Define a list of other entities as children for a domain, allowing you to represent a hierarchy of how your entities are modeled across your workspace using the x-cortex-children
tag.
Domain parents
Define another entity as the parent for a domain, allowing you to represent a hierachy of how your entities are modeled across your workspace using the x-cortex-parents
tag.
Note: Parents must be of type domain
.
Ownership inheritance
A common use case for domains is defining ownership for the subtree of entities. Instead of defining ownership individually for every entity in your catalog, you can define ownership at the domain level and have that pass down to all of its children.
The inheritance
type for each owner can be one of APPEND
, FALLBACK
, or NONE
. If not set, inheritance is defaulted to NONE
.
APPEND
: This owner is appended to the list of owners for all child entities.
FALLBACK
: In the case where a child has no valid owners, including fallbacks, this fallback will be assigned as the owner. Note that this only applies to a child entity down the hierarchy; it is not the fallback for the parent domain itself.
NONE
: This owner owns the domain, but not necessarily any of its children (no inheritance).
Example cortex.yaml
Note: the YAML definition for a domain entity can take file names other than cortex.yaml
or cortex.yml
; see the GitOps example repository structure.
You can create, update, and delete domains using the Cortex API.
It is possible to edit entities after creating them:
Navigate to the entity's page.
In the upper right corner, click Configure entity.
Make any desired changes.
Note: The only field you cannot edit is the identifier.
At the bottom of the screen, click Save changes.
To view your teams, navigate to Catalogs > Teams.
On the right side of the Teams catalog page, see the Scorecard Leaderboard, which highlights the ten best-performing teams within your organization.
The leaderboard gamifies entity quality and encourages team members to achieve goals. This creates a culture of accountability, where everyone has visibility into how they're performing.
Each team has its own details page, where you can view key details about the team. Click a team from the Teams catalog page to view its details.
At the top of a team details page, you’ll find on-call information, Slack channels, and parent and children teams.
Additional information appears in tabs on the team's details page:
The Overview tab includes:
On-call information.
Linked Slack channels.
Where the selected team belongs within the broader hierarchy.
How the team is performing across Scorecards. By default, this will show the level that the team’s entities have reached in each Scorecard.
On the right, enable the toggle next to Aggregate children scores to include child entities in the Scorecard overview.
The Members tab includes a list of all team members, as well as their contact information. You can also add tags for each user to signify each member’s role on the team. When available, Cortex will also pull in profile photos from your Git provider.
The Entities tab shows a list of all entities that the team owns.
The Dependencies tab shows any incoming or outgoing dependencies associated with this team.
Teams not only allow you to collect vital information in a single place, but are also crucial for ownership. Rather than assign an entity to individual team members, you can assign ownership to an entire team. This makes it easy to assign multiple team members to an entity, and it ensures that when a team’s composition changes, ownership is updated accordingly.
You can create teams:
By importing them from a connected integration (e.g., Workday, GitHub)
Manually in the Cortex UI
You can configure teams to reflect the actual hierarchy at your organization while creating or importing teams in Cortex. A team can be defined as the parent of one or more teams while also being the child of another team.
In the above example, My Company
is a parent team with 7 child teams nested under it.
See the tabs below for instructions on each method.
If you have an existing source of truth for your teams and team members, we recommend importing teams. By integrating with your identity provider at this stage, Cortex will automatically sync team pages with your source of truth so you don't have to update information in more than one place when people join or leave teams.
Each integration syncs teams daily.
You can only import entities from integrations that have already been configured.
Teams from the following integrations can be imported:
After configuring an integration that includes teams, follow these steps to import teams:
In Cortex, navigate to Catalogs > All entities, then click Import entities.
Select the integration to import from.
If you have a large volume of entites, click Filter in the upper right corner of the results list to select and apply entity type filters.
At the bottom of the page, click Next step.
Edit the details for the entity:
Type: Select Team
.
Name: Enter a human readable name for your team.
Identifier: This field is auto-populated based on your entity name. It is a unique identifier for your entity. This is also known as the x-cortex-tag
.
Description: Enter a description of the team to help others understand its purpose.
Members: Select members of your team. Team members will be marked as owners of entities and receive notifications about entities owned by the team if notifications are enabled.
Links: Add links to external documentation, such as runbooks, docs, logs, or custom categories.
On-call: Configure the on-call service associated with this team. When selected, you will see the latest on-call information displayed on the team's details page.
Click Confirm import.
If you don’t have an identity provider with updated team information, you can create a team manually within Cortex. Because teams are important for effective ownership, it's recommended that you create teams in Cortex even if you don't have a single source of truth for team information.
In Cortex, navigate to Catalogs > Teams, then click Import teams.
Configure the form:
Configure the team details:
Type: Select Team
.
Name: Enter a human readable name for your team.
Identifier: This field is auto-populated based on your entity name. It is a unique identifier for your entity. This is also known as the x-cortex-tag
.
Description: Enter a description of the team to help others understand its purpose.
Members: Select members of your team. Team members will be marked as owners of entities and receive notifications about entities owned by the team if notifications are enabled.
Links: Add links to external documentation, such as runbooks, docs, logs, or custom categories.
On-call: Configure the on-call service associated with this team. When selected, you will see the latest on-call information displayed on the team's details page.
Click Confirm import.
Once you've created a team, you can view it on the Teams page within the hierarchy. If you haven't added parents or children, you can disable View as hierarchy to see the list of all teams.
You can manually create teams in Cortex and define them in the entity descriptor.
If your entity is a team, you must specify x-cortex-type
as team
.
The x-cortex-team
tag has two main sections: groups
and members
.
Adding groups to the team entity descriptor
Use groups
to link your team entity with a team on a different platform that you have integrated with Cortex. You can only link one group to a team entity.
For example, you can specify:
Under groups
, when you specify okta-security-team
, your team will contain all the members from your okta-security-team
.
Adding team members to the team entity descriptor
members
can be used to add individual members to your team when you have a use case for a team entity that doesn't correspond exactly to a team on one of your integrations. Members support roles
. For example, the following entity includes a member who has the product-manager
role and a member who has both the engineering-manager
role and manager
role:
After specifying the YAML example above, your team now will contain Product Manager
and Engineering Manager
. If product-manager@cortex.io
or engineering-manager@cortex.io
were to correspond with the email of an actual account in Cortex, they would start seeing example-team
being displayed as a team that they're a part of (i.e., it would start showing up in their Mine
tab in catalogs that contain teams).
The role
field in member is optional. In order to be considered valid, a team must have a non-empty group.
Team children
You can define a list of other teams as children, allowing you to represent a hierarchy of how your teams are modeled across your workspace, using the x-cortex-children
tag.
This hierarchy is available to look at in the Teams catalog page.
Team parents
Team children allow you to define the team relationship from the top-down, but in some cases it may be easier to define the team hierarchy from the bottom-up. For these cases, x-cortex-parents
can be added to any entity's YAML to define its parents.
Example cortex.yaml
It is possible to edit entities after creating them:
Navigate to the entity's page.
In the upper right corner, click Configure entity.
Make any desired changes.
Note: The only field you cannot edit is the identifier.
At the bottom of the screen, click Save changes.
You can manually create a team member role:
Click Add role.
In the "Add role" modal, configure the role:
Role name: Enter the role's name.
Tag: The tag - a unique identifier for the role - automatically populates based on the role name.
Role description: Enter a description of the role.
Enable notifications by default: If notifications are enabled, members with this role will receive relevant updates on Scorecards, Initiatives, and more.
Click Save.
You can apply a role only to manually-created team members. Team members who were imported from an identity integration will retain the role that was imported.
To apply a role:
In Cortex, navigate to Catalogs > Teams.
Click into a team.
In the upper right corner, click Configure entity.
Click Members.
Next to a team member, click the edit icon.
In the side panel, add a new team role.
Click Update.
In this section, you can select which identity providers will be used to sync team and team memberships into Cortex.
When enabled, any discovered teams and team relationships from Workday are automatically imported.
In order for teams to be imported, Workday needs to be enabled under the Enabled identity providers section on this settings page. The Workday report must include the managerEmail
field in order for team relationships to be auto imported.
When enabled, all team members will be granted edit access. You will not be able to turn off edit access for individual members in your teams configuration when this is enabled.
When enabled, Cortex will automatically assign roles to team members based on your enabled identity providers. Leave this option disabled to assign team roles manually.
Cortex comes with several built-in entity types, but you can create unlimited custom entity types to extend your catalogs.
Every entity in the catalog is associated with an entity type. Entity type definitions require a type
and a JSON schema that outlines the attributes that entities should conform to.
To view all entity types, go to Catalogs > All Entities then click the Entity types tab.
A Cortex logo appears next to built-in entity types. When you hover over the logo, you will see a "Powered by Cortex" banner:
The other entity types in the list are custom types.
You can create custom entity types:
When creating an entity type, keep in mind that these will ultimately dictate how you import specific entities later on.
To create, edit, and delete entity types, your user or API key must have the Edit entity types
permission.
In Cortex, navigate to Catalogs > All entities.
Click the Entity types tab, then click Create entity type.
Configure the form:
Name: Enter a human-readable name that will appear in the catalog for this entity type.
Type: Enter a unique identifier that corresponds to the entity type. This will be used to specify the type defined in the YAML definitions for imported entities.
Description: Optionally, enter a description for the entity type.
Display icon: Select an icon to appear alongside the entity type throughout the app.
Schema: Define a JSON schema that will be used to validate the x-cortex-validation
block of a given entity’s YAML.
This is not required to create an entity type, but will be useful if you want to enforce certain attributes about entities, such as a region or version number. Attributes defined in the schema will be required when creating an entity of that type, which can then be validated in Scorecards.
In the example screen shot, we’ve created a custom entity called Employee
— this is what we’ll use to represent individuals at our organization. In the schema section, you can see that each profile is required to include an employee’s location and their dream vacation. This means every time you create an Employee entity, you’ll define each employee’s location and dream vacation in a schema for that entity.
At the bottom of the page, click Create.
After saving, the entity type will appear in the Entity types tab under Catalogs > All entities, and you will be able to import entities of that type.
If you want entities of the new entity type to automatically belong to a specific catalog, you can configure the catalog's defined entity types while creating a new catalog or you can edit an existing catalog to update the entity types.
If your entity is of a different type, you must specify x-cortex-type
and x-cortex-definition
. The x-cortex-definition
field is validated against the entity type definition. If your custom entity type does not have specific requirements, entities of that type still need a non-null definition specified, like x-cortex-definition: {}
.
Example cortex.yaml
Note: the YAML definition for a custom entity can use file names other than cortex.yaml
or cortex.yml
. Please refer to the entity types for more details.
You can create, update, and delete entity types using the Cortex API.
After creating your custom entity type, you can start creating entities of that type:
In the main nav of Cortex, click Catalogs > All entities.
At the top of the Services page, click +Import entities.
Configure the form:
Type: Select your custom entity type.
Schema: Enter a JSON schema to define your custom entity. See JSON schema for custom entity types below for more information.
Entity name: Enter a human readable name for your entity.
Identifier: This field is auto-populated based on your entity name. It is a unique identifier for your entity. This is also known as the x-cortex-tag
.
Description: Enter a description of the entity to help others understand its purpose.
Groups: Select groups to segment your entity.
Owners: Define ownership for your entity. We recommend selecting team owners to keep your ownership information up-to-date through any future personnel changes.
Links: Add links to external documentation, such as runbooks, docs, logs, or custom categories.
Parents: Define parent domains. This is where you configure the hierarchy for your entity. These can be visualized in the relationship graph.
Dependencies: Select entities that this entity depends on. These can be visualized in the relationship graph.
On-call: Configure on-call information.
Repository: Select the repository associated with this entity.
When you are finished, click Confirm import at the bottom of the page.
Custom entity type definitions are powered by the open-source JSON Schema project.
The JSON schema for a custom entity type ensures consistency and validation when creating entities of that type. The attributes listed under required
in the schema must be defined for entities of that type according to the outlined properties
.
type
Type of entity and/or required component: array
, boolean
, integer
, null
, number
, object
, or string
Yes
required
Required specs for the entity type
properties
Properties of the required specs (including type
)
Yes only if the required
field is not null
Defining a JSON schema also enables visibility on an entity's detail page. The schema and the defined properties will appear under the Entity Metadata section on the Overview page, making it easy for users to identify key specs.
It is possible to edit entities after creating them:
Navigate to the entity's page.
In the upper right corner, click Configure entity.
Make any desired changes.
Note: The only field you cannot edit is the identifier.
At the bottom of the screen, click Save changes.
After an entity is imported or created, it will automatically belong to a catalog based on the entity type criteria set for each catalog. When you create a custom catalog, you can set the entity type criteria.
By default, any custom entity types you create will belong to the Infrastructure catalog. If you do not want the entity type to belong to this catalog, you can add the entity type to a different catalog's definition.
In some instances, you may want to use custom data instead of a custom entity type.
It is possible to create an entity type with an empty properties schema:
This entity type will display in the catalogs, but entities of this type won't be validated against certain specs. Such properties can instead be defined using custom data.
A schema is ideal for enforcing static properties across all entities, while custom data allows users to augment and update attributes.
In Cortex, you have the ability to define outgoing dependencies on other entities. Cortex can also automatically discover dependencies for some integrations. Having dependencies defined powers the ability to notify owners when a dependency deprecates its API or makes backwards incompatible changes, and the ability to visualize dependencies in a relationship graph.
Incoming dependencies are inferred based the outgoing dependency definitions.
In Cortex, you can visualize dependencies using the relationship graph at Tools > Relationship graphs. See the Relationship graph documentation for more information.
When a dependency deprecates its API or makes backwards incompatible changes, Cortex surfaces these issues via these methods:
Breaking API changes are listed in your Cortex workspace under Settings > Breaking API changes.
Cortex attempts to automatically make comments on PRs containing breaking OpenAPI changes that have downstream dependencies that Cortex knows about.
If a breaking change is merged to the default branch, Cortex alerts dependency owners via Slack that a breaking change was merged.
Cortex can automatically discover dependencies from your integrations:
You can define dependencies manually via an entity's YAML descriptor, via the Cortex UI (if UI editing is enabled), or via the API.
Your user or API key need the Edit entities
permission.
Navigate to the entity where you need to define a dependency.
In the upper right corner of the entity details page, click Configure entity.
Click the Hierarchy tab.
In the dropdown menu, choose an entity.
Optionally, select an endpoint.
In order for an endpoint to populate in this dropdown, it must first be defined as a path in the entity's YAML file. See "Set an endpoint for a dependency" below.
Click Add.
Before you can select an endpoint from the dropdown when manually defining a dependency, that endpoint must be defined as a path in the entity's YAML file. See the example below:
The x-cortex-dependency
field allows you to define a list of outgoing dependencies. A dependency should be directed towards an outgoing service or resource, or more granularly, to a specific endpoint of that entity.
tag
The tag
of the entity this entity depends on, i.e. the callee. See x-cortex-tag
No
method
HTTP method if depending on a specific endpoint
Required if path
is present
path
The actual endpoint this dependency refers to
Required if method
is present
description
A description of the dependency.
Yes
metadata
JSON metadata tags for the relationship. Supports arbitrary objects.
Yes
See API Docs for authentication details.
YAML is the source of truth. If a dependency has already been set through the cortex.yaml
, the API will return an error.
Endpoints are optional. A dependency optionally references an endpoint (method
and path
) of the callee, and this must already be defined in the callee's cortex.yaml
within the paths
field. If no endpoint is referenced it is assumed that the caller depends on all endpoints of the callee.
For all requests method
or path
are optional, however if one is present the other must also be present.
When interacting with an existing dependency, the method
and path
must be specified correctly to identify it.
callerTag
The tag
of the caller.
No
calleeTag
The tag
the caller depends on.
No
method
HTTP method if depending on a specific endpoint
Required if path
is present
path
The actual endpoint (as defined in the OpenAPI file) the caller depends on
Required if method
is present
description
A description of the dependency.
Yes
metadata
JSON metadata tags for the relationship. Supports arbitrary objects.
Yes
Create a dependency
POST /api/v1/catalog//dependencies/?method=&path=
Retrieve a dependency
GET /api/v1/catalog//dependencies/?method=&path=
Update a dependency
PUT /api/v1/catalog//dependencies/?method=&path=
PUT replaces entire object. The request body is considered a modified version of the already existing entity. Leaving a field out of the JSON will be interpreted as null
.
Delete a dependency
DELETE /api/v1/catalog//dependencies/?method=&path=
Bulk create or update dependencies
PUT /api/v1/catalog/dependencies
:::caution PUT replaces entire object The request body is considered a modified version of the already existing entity. Leaving a field out of the JSON will be interpreted as null
. :::
Cortex syncs AWS dependencies every day at 8:00 a.m. UTC. All other dependencies sync at 12:00 a.m. UTC.
If you need to sync dependencies manually:
Navigate to Tools > Relationship graphs.
Each entity has its own page that centralizes data related to the entity, allowing developers to reduce context switching.
While viewing entities in a catalog, or from the Catalogs > All entities page, click an entity name to open that entity's details page.
When you first open an entity, see an overview of relevant information. If there are any active incidents, you will see them at the top of the page, in addition to recent events, on-call, owners, Slack or Microsoft Teams channels, groups, repository, language, and related domains.
During an incident, the entity details page allows you to quickly find the information you need all in one place: the active incident itself, who owns the entity and who is on-call, which Slack channel to communicate in, dependencies that could be affected, and the most recent deploys.
If data is not available for a field on the entity page, it will not appear in the overview. For example, if you do not have the Slack integration configured, then the "Slack channels" field will not be displayed.
In the left sidebar of an entity, drill further into the entity's recent deploys, events, security vulnerabilities, monitoring metrics, and more, giving you the insight you need to solve an incident quickly and efficiently.
In the sidebar, click into each category for more information:
API explorer: Pulls from API documentation added to your entity.
Events: Recent events such as deploys and other integration events.
Links & docs: Documentation links you have added to the entity, along with documentation pulled from the related repository.
On-call & incidents: On-call information from Opsgenie, PagerDuty, Splunk On-Call (formerly VictorOps), and xMatters, and incidents pulled from incident.io, PagerDuty, FireHydrant, and Rootly.
Owners: See the teams who own the entity, the related Slack or Microsoft Teams channels, and a list of team members.
Learn more about how to define entity owners in Defining ownership.
Entity YAML: View the entity YAML that defines this entity.
Scorecards: A list of any Scorecards that are in progress where this entity is being scored. In additional, see the average score, median score, and a graph showing the entity's scores over time.
Initiatives: A list of all active Initiatives for this entity, including the Scorecard's current level and the due date of the Initiative.
Workflows: A list of Workflows that include this entity. If any Workflows are pending approval and you were designated as an approver when the Workflow was created, you will see the pending approval at the top of this page. This page also includes a list of recently run Workflows and the ability to run any related Workflows.
Under the "Connections" header, integration data for the entity is contextually grouped. When you configure integrations, data will be pulled in to the following sections on the entity details page:
CI/CD: Pulls from deploys API, Azure DevOps, Bitbucket, Buildkite, CircleCI, GitHub, and GitLab.
Custom data & metrics: Pulls from custom data and Eng Intelligence custom metrics.
Environments: Pulls from AWS and Kubernetes.
Issue tracking: Pulls from Azure DevOps, ClickUp, GitHub, GitLab, and Jira.
Monitoring: Pulls from Coralogix, Datadog, Dynatrace, Lightstep, New Relic, Prometheus, Splunk Observability Cloud (formerly SignalFX), and Sumo Logic.
Packages: Pulls from the packages API and any packages automatically discovered from your configured git repositories.
Repository: Pulls from Azure DevOps, Bitbucket, GitHub, and GitLab.
Integration settings: This page lists the integrations that are enabled and connected for this entity.
Under the "Plugins" header, see any plugins in your workspace that are relevant to this entity.
Learn more about entities in Manging entities.
Ownership is a core use case of Cortex, as many organizations seek to establish clear ownership of services, data, and other entities. Entities should be owned within Cortex to ensure appropriate action can be driven using Scorecards and Initiatives.
Ownership can be defined by accepting Cortex's automated recommendations for ownership, pulled in from third-party integrations, or defined manually in the Cortex UI.
By mapping every service and resource to its responsible teams, issues get resolved faster, accountability is clear, and decisions happen quickly.
Ownership drives which users will receive notifications from Cortex, including alerts for on-call changes, when verification is needed on an assigned entity, when an entity is re-evaluated and its Scorecard changes, and more.
When viewing an entity, the owners appear in the metadata bar at the top of the page:
Click into the team name to view the team's entity page, including a list of members and a list of entities owned by that team.
You can create or import the teams and users who will be defined as owners for your entities.
You can define owners based on:
A team
We recommend setting up teams as owners. If you link a group
in your YAML file from a different platform (such as Okta), the members of the team will be automatically updated in Cortex if anyone leaves your organization and is removed from your integrated identity provider.
A user email address
Owners can be defined:
By accepting Cortex's automated recommendations for owners, based on repository activity
Automatically if Cortex detects that an entity is owned by a team that does not yet exist in Cortex
If an entity's YAML references a team, but that team doesn't have a corresponding entry within Cortex, Cortex will automatically create a team. The team will include a label that says Automatically created by Cortex.
By pulling information from third-party integrations in the entity descriptor YAML
Directly in the Cortex UI
This feature is available in private beta. Please reach out to your Cortex Customer Success Manager for access. Note the following considerations:
This feature is supported for entities associated with a repository in GitHub, GitLab, or Azure DevOps. It is not supported for monorepos.
You must have at least one team in Cortex in order for Cortex to provide recommendations.
The ability to edit entities in the UI must be enabled.
To accept or reject the recommended owner, the user must have the Edit entities
permission.
Cortex analyzes a repository and automatically recommends a team owner for entities that do not have an owner.
If an entity does not have an owner and Cortex has recommendations for who the owner should be, it will be flagged in the "Owners" section of an entity details page overview, in the "Owners" sidebar link on an entity details page, and it will appear during the import process when adding entities.
In Cortex, navigate to Catalogs > All entities.
Search for and select the entity whose ownership you want to edit.
In the upper right corner of the entity's page, click Configure entity.
Click the Owners tab, then click +Add next to Teams or Users.
Add team:
Select a team from the dropdown menu, then click Add.
Add user:
Select a user from the dropdown menu, then click Add.
You can also add a user who is not listed in Cortex. To do this, enter an email address into the Email address field, then click Add.
The x-cortex-owners
field allows you to define a list of owners of type email or group.
Cortex recognizes groups from the following integrations:
The value of provider
is the name of the integration that the group is coming from. The available list is:
ACTIVE_DIRECTORY
AZURE_DEVOPS
BAMBOO_HR
CORTEX: Use when referring to a team defined in Cortex; these teams do not map to identities from a connected integration.
GITHUB
GITLAB
OKTA
OPSGENIE
SERVICE_NOW
WORKDAY
name
is a case-sensitive field that refers to the following:
if your provider is CORTEX
, name
corresponds to the x-cortex-tag
for the Cortex team you want to identify as an owner
otherwise, name
corresponds to the upstream identifier of your owner from your integration
Cortex can automatically discover ownership for your AWS resources using their owner
tag. To enable this, make sure that your AWS resources have an owner tag matching the x-cortex-tag
of the corresponding Cortex team and enable the Sync ownership from AWS toggle in Settings > AWS.
You can pull in all resources from AWS, and Cortex syncs those owners automatically based on their tags in AWS, allowing you to easily keep the resource owners up to date.
Cortex syncs ownership from AWS every day at 6 am UTC.
You can filter the entity list by owner:
Under Catalogs > All entities, click the All tab.
In the upper right corner, click Filter.
Teams can exist within hierarchies. You can view a list of all entities that are owned by the parent team and all children teams in the hierarchy:
Navigate to the parent team's page in Cortex.
Click the Entities tab.
Click Display, then enable the toggle next to Inherited Children.
Click Done.
The list will now display all entities owned by the parent and its children teams. Note that this setting does not persist when you navigate away from the page.
Read more about hierarchies in Setting up a team hierachy.
Under Settings > Entities, there are several settings relating to teams. Read more about these in the Teams documentation.
Groups are a tagging system that can be used to group a set of entities together, enabling a variety of use cases including:
Tagging
Use groups to segment entities.
For example, you could segment by tiers (tier-0
to indicate high priority), type (backend, frontend, library, API), or language (Python, Java).
Filtering throughout Cortex
Use groups to include or exclude certain entities from Scorecards, catalogs, CQL reports, and more.
For example, you may want a Scorecard to only apply to Python services or a catalog that only shows tier-0 services.
Reporting
You could aggregate Scorecard scores by a specific group, such as a tier.
For example, viewing a Production Readiness Scorecard broken down by tier.
While viewing an entity, its groups are listed at the top of the page:
Click the group name to view a list of all other entities that are tagged with this group.
You can define groups via an entity's YAML, directly in the Cortex UI, or via the API.
Note the following considerations:
If a group has been set in the entity descriptor YAML, an API call will not overwrite the existing value.
Groups created via the API will not appear in the entity descriptor YAML.
Groups created via the API cannot be removed via the Cortex UI.
To create a group in the UI:
Navigate to the entity that you want to apply a group to.
Click the magnifying glass icon at the bottom of the main nav, or click Catalogs > All entities and search for the entity from that page.
In the upper right corner of the entity page, click Configure entity.
Click into the Groups field.
To apply an existing group, type to search then click on the group.
To create a new group, type the group name into the field, then click the group name in the dropdown. The group will be created and applied to the entity.
At the bottom of the page, click Save.
Groups are defined as a list of tags.
Groups may not contain whitespace characters.
You can use the API to add groups to an entity. See the API documentation for groups here.
When should I use groups instead of custom data?
To determine whether or not to use groups or custom data, consider whether your use case would be considered "tagging" with a definite enum of values (backend
vs frontend
), or more freeform metadata availability-zones: [east, west]
.
Cortex is your source of truth to external docs. Instead of digging through wikis and other resources, you can aggregate docs on the relevant entities in Cortex in the following ways:
If you have configured the Slack integration, you can use Slack Bot commands to list an entity's docs links directly in Slack.
Links appear on an entity details page in the Links & docs section:
On entities, you can add links to docs, runbooks, tech specs, dashboards, and more.
Before editing an entity via the UI, make sure that UI editing is enabled on the Entities settings page under the GitOps tab.
In Cortex, navigate to an entity.
In the upper right corner of the entity details page, click Configure entity.
Click the Links tab.
Click +Add.
Configure the link:
Type: Select the type of link.
To create a new type, enter the type name then click the name in the dropdown.
Learn more about OpenAPI, AsyncAPI, and dashboard chart types below.
Name: Enter a name for the link.
URL: Enter the URL.
Description: Add a description of the link.
Click Add.
In the entity descriptor, add a list of link
objects:
name
, type
, and url
are required.
The value of type
can be defined based on your organization's preferences. Common examples include dashboard, documentation, healthcheck, logs, metrics, and runbook. See the sections below on this page describing OpenAPI, AsyncAPI, and dashboard charts.
You can quickly look up links using the Cortex Slack Bot.
Cortex automatically embeds markdown files from the docs
folder of your repo or from the root directory. If Cortex detects relevant files, these docs show up under the Links & docs page in an entity's sidebar.
Cortex uses the React markdown library for rendering the documents.
When adding links to an entity, Cortex supports embedding OpenAPI or AsyncAPI specs, embedding specs from your repository, adding relative links to a repository that isn't associated with the entity, and embedding charts (from Datadog, Grafana, New Relic, or other sources). You can also create a custom type while adding a link.
Cortex supports displaying Swagger/OpenAPI or AsyncAPI specs for each entity in the API explorer on an entity details page. You can also authorize your API and run queries directly in the API explorer.
You can use your entity descriptor file as an OpenAPI spec file. As mentioned in the documentation on entities, the entity descriptor file extends the OpenAPI spec, so you can implement the spec directly in this file.
You may have existing or external specs you want to display in the API explorer from sources such as Swagger Hub or GitHub Raw URL. Add these to the JSON or YAML spec file under the x-cortex-link
block, with the type openapi
.
If you have specs checked into your git repository, you can embed it by adding it as a relative URL with type openapi
. For example:
In addition to providing the ability to link to specs within the service repo, Cortex also allows for links that reference other git repositories that aren't necessarily associated with your entity but still within your organization.
Power users can also specify a branch
parameter between the second and the third colon separated group, in the form github:org/repo:branch:path/to/file
. If this parameter is omitted, we will use the repository's default branch.
Using your entity tag, you can also send your OpenAPI JSON or YAML file to the public endpoint for managing OpenAPI documentation /api/v1/catalog/documentation/openapi
for each of your entities. One API doc per entity can be uploaded and it will automatically show up as an API Doc
entry in the dropdown in the API explorer.
The request body is the following, in JSON format. See API Docs for authentication details.
spec
string
The json or yaml as a string
Converting YAML or JSON to string You can use a lightweight utility like jq to take your yaml/json and generate a string that can be used for your request:
e.g. $(cat my_file.yaml | jq -Rsa)
You may have existing or external specs you want to display in the API explorer from sources such as Swagger Hub or GitHub Raw URL. Add these to the JSON or YAML spec file under the x-cortex-link
block, with the type async_api
.
If you have specs checked into your Git repository, you can embed it by adding it as a relative URL with type async_api
. For example:
Cortex supports embedding charts from Datadog, Grafana, or New Relic directly into an entity for quick access. To view embedded charts, click the Dashboard page in the entity's sidebar.
The type
field is an optional enum of three types: datadog
, grafana
, or newrelic
.
The url
field is the src
value for the iframe
embed generated by your dashboard tool.
You must use your tool's embed URL for individual charts.
Note that you can embed individual charts, not dashboards.
If you're using Grafana and seeing errors in your Grafana embeds, make sure your Grafana instance has embeds enabled.
It is possible to display content from a source other than Datadog, Grafana, or New Relic. To do this, do not specify an option for the type
field. The URL you add must be publicly accessible or provide cross-platform authentication for Cortex to view and display the iframe. For example, if the content is accessible via VPN, then you should be able to view it in Cortex while on the VPN.
If you have configured the Slack integration, you can use Slack Bot commands to list documentation directly in Slack:
/slack links <tag-or-id>
to list all documentation links for an entity
/slack links [type] <tag-or-id>
to list specific types of documentation links for an entity
The type
corresponds to the type of documentation (e.g., openapi
, documentation
, etc.). Replace <tag-or-id>
with the entity's tag or ID.
For example:
To list all documentation links for an entity with ID en29f044598b112345
, you could run the following command in Slack:
/cortex links en29f044598b112345
To list the OpenAPI specs for an entity with tag plugin-extension
, you could run the following command in Slack:
/cortex links openapi plugin-extension
The Slack Bot links
command works for any docs links listed under x-cortex-links
. Note that it does not list the charts listed under x-cortex-dashboards
.
When you add deployment data to Cortex, it is visible on entity pages, in Eng Intelligence reporting, in executive reports, and can be accessed via CQL.
To get deploys into Cortex, you must use the Add deployment for entity API endpoint.
Adding a customData
object to your API call allows you the flexibility to add metadata that is important to your organization.
Deployment information appears in several places on an entity page:
While viewing an entity page, you can see last deployment information near the top of the page:
Near the bottom of an entity page, deploys will also appear under "Recent activity."
In the left sidebar of an entity, click Events. This page includes:
A visual chart of deploys.
By default the chart shows data from the last month. Click the Last month dropdown in the upper right to change the timeframe.
All recent events for the entity. In the upper right corner of the events list, click Filter to select and apply filters for this list. You can choose to only view the deploy
event types.
When you send deploy data into Cortex, you can use this information in Eng Intelligence reporting to gain insight into your deploy metrics. Deploy metrics include average number of deploys per week and deploy change failure rate.
In Eng Intelligence, click into an entity to open a side panel with a historical performance graph.
Read more about using Eng Intelligence in the documentation.
You can use deploy data to write rules for Scorecards and to create CQL reports.
See more examples in the CQL Explorer in Cortex.
Custom data extends the out-of-the-box metadata that Cortex provides via integrations, enhancing your use of catalogs and augmenting details available for your entities. This data can also be used to write CQL queries and Scorecard rules.
Custom data can be defined manually in the entity descriptor, added to entities programmatically via a REST API, or sent through a webhook.
There are a few ways you can add custom data to an entity: the entity descriptor, a POST
REST endpoint, or a simple webhook.
Adding custom data via entity descriptor is optimal for low-volume, human-maintained data because it requires updating the entity's YAML when the data changes.
Using a REST API is a better option for data that comes from an automated process, like a CI/CD pipeline.
Learn more about entity YAML descriptors in the Managing entities documentation.
The simplest way to describe information in the YAML is to define an object.
key
Key or title for the custom data. Anything defined before the :
will serve as the key
.
✓
value
Value for the custom data. Anything defined after the :
is the value
.
✓
Custom data can be of any type, including scalars (strings, numbers, booleans), objects, and lists.
When you add custom data to an entity's YAML, you'll see the key
and value
pairs on an entity's Custom data page.
Custom data added via entity descriptor will display with a YAML
tag.
You can add a description to data by explicitly defining it along with the key
and value
pairs. While key
and value
pairs can be defined with any terms when a description is not added, value:
must explicitly be defined when a description is included.
key
Key or title for the custom data. Anything defined before the :
will serve as the key
.
✓
value
Value for the key; should be defined explicitly with value:
✓
description
Description of the custom data
✓
Note: While the description
field is always optional, it is technically "required" to add a description to custom data.
You can pipe custom data directly into Cortex by POST
ing to /api/v1/catalog/{tag}/custom-data
where {tag}
is the x-cortex-tag
for a given entity.
The request body requires the following in JSON format:
key
string
Key or title for the custom data
✓
value
object
Value for the custom data
✓
description
string
Description of the custom data
See the API docs for authentication details.
If a key has already been set through the entity descriptor, an API call will NOT overwrite the existing value. Instead, it will simply return the existing value.
If a key
is defined in the entity descriptor, the API cannot override the key. You'll see YAML
as the source
value in the response body if you encounter this situation.
To explicitly overwrite values set in the YAML file, use the force=true
flag as a query parameter.
If you find yourself using the force
flag, it may be better to remove the field from the YAML file or update the value in the cortex.yaml
file instead to maintain the source of truth.
Custom data added via API will display with a API
tag.
You can use the bulk upload endpoint to upload multiple keys for one or more entities.
PUT
data in the following format to /api/v1/catalog/custom-data
:
You can include multiple key
and value
objects for each tag. The object conforms to the same shape as the single upload API.
You can use custom data webhooks to create unique URLs that you can use to directly POST
arbitrary JSON payloads without auth or an explicit entity tag in the URL if you do not have obvious access to the entity tag or the ability to add authentication headers.
With this method, you can tell Cortex how to process the payload and map it to an entity. For example, the payload may include a data.tag
field that corresponds to the entity tag.
Go to Custom integrations settings in Cortex.
Create a custom integration for the webhook:
Name: A human-readable name for the webhook.
Entity tag JQ: The JQ expression that Cortex will use to extract the entity tag from the payload (e.g. .data.entityTag
). **If the tag Cortex extracts from the payload is not identical to the entity tag in Cortex, the endpoint will throw a 400.
Key: The unique key that will be used in CQL queries with the syntax integration(key)
. Data will be stored under the key
for entities, just like when adding data via entity descriptor or REST API.
Save the integration and copy the provided webhook URL.
cURL any JSON your heart desires to this URL!
The data can be used exactly the same way as custom data defined through the entity descriptor or the API.
Custom data added via webhook will display with an INTEGRATION
tag.
If the webhook payload does not contain a value that maps to the exact entity tag you can tell Cortex alternative mappings to look up when processing payloads.
When processing a payload, Cortex takes the output of the Entity tag JQ field in step 2 above and searches for an entity with that value as a tag. If no such entity exists, Cortex checks whether an entity has registered the value as an alternative mapping; if so, the payload is attached to that entity.
Cortex applies a data source hierarchy to determine how to handle a case where the same key
is defined from multiple sources.
The entity descriptor is the source of truth. If a key is defined there, the API or webhooks cannot override the key.
You can override keys defined in the YAML by adding a force=true
parameter when using an API, but when the entity descriptor is eventually re-processed, the data provided via the API will be overwritten.
Catalogs contain a lot of useful metadata about entities, including ownership and data from integrations. However, you may have your own custom fields that you want to include that would make exploring the catalog easier.
These can include things like:
Which AWS zones is this deployed in?
What databases does the entity consume?
When was the last successful CI run?
If the answers to these questions fit a pre-enumerated list, consider using groups. Groups are displayed on entity detail pages and can easily be applied in catalog filters.
Custom data for cataloging makes the most sense when the data is more flexible.
Custom data can then be queried against by using the Query builder, explored with CQL reports, or viewed on an entity's details page.
Cortex makes it easy to write Scorecard rules based on custom data that exists in your catalogs. When adding a rule, you can select Custom data from Choose an integration in the form editor and follow the flow. Cortex will automatically supply variables based on the custom data you've defined.
Using the custom data API to push data into Cortex is a great way to extend your Scorecards. You can send in entire JSON payloads and use jq
to process them in a Scorecard or use it as an input to a custom OPA Policy
as a Scorecard rule.
You may have over a hundred repositories in GitHub, and eventually, you will likely import all of them into Cortex. This much information can make it difficult to know at a glance that all repositories are accounted for and all new projects are being imported into and tracked within Cortex.
The Discovery audit guarantees confidence in your catalogs by listing all changes that Cortex has detected. Cortex compares everything that exists within the system to what it discovers within your git tool, APM tool, Kubernetes cluster, and other crucial integrations, giving you insight into changes happening across your environments.
You can find the Discovery audit under Tools in the main nav.
On this page, see a list of recent changes in your environment that aren't yet reflected in Cortex, including newly created repositories, services, and resources discovered from your integrations.
The first time you use the Discovery audit, there may be a lot to review. To narrow the scope of your list and start with changes that are the highest priority for you, search or filter the list by integration or entity type.
To search, click the magnifying glass icon in the upper right corner of the list.
To filter, click Filter in the upper right corner of the list. Select your filtering criteria, then click Apply.
Cortex will tag detected changes to signify whether an entity or repository has been discovered, archived, or deleted.
You can import, delete, or ignore the entities listed in Discovery audit.
If Cortex detects a new service or resource, you can import it directly from this page:
Click +
in the row containing the new entity:
Configure the entity details.
For detailed instructions on creating a new entity, see the relevant docs page for the entity type: Services, Domains, Custom entities.
Click Confirm import.
If Cortex no longer detects a given entity, you also have the ability to delete that entity or repository directly from this page:
Click the trash can icon in the row containing the entity:
In the confirmation window, click Delete.
The confirmation window gives you the opportunity to review all potentially impacted services before deleting, so you don’t unintentionally remove something from Cortex.
If an event appears within the Discovery audit, but is irrelevant — for example, a test project that doesn’t need to be imported into Cortex — you can ignore it:
Click the hide icon in the row containing the entity:
The entity will now appear in the Ignored tab of the Discovery audit. The ignore action is persistent, so the event won’t appear again within the discovered list.
From the ignored list, you can move the entity back to the Discovered tab by clicking the eye icon.
In Cortex, use the relationship graph to better understand:
Dependencies
Visualize how entities relate to one another
Domains
See a hierarchical diagram of your software architecture
Teams
View your company's team structure
You can access the relationship graph in different ways:
From the main nav: Navigate to Tools > Relationship graphs. By default, the page displays all entities with dependencies and how they relate to one another.
From an entity details page:
While viewing an entity, click the Dependencies tab. In the row containing the entity you want to view in a graph, click the 3 dots icon, then click View in relationship graph.
While viewing a domain or team entity, locate an entity under the Hierarchy header. In the row containing the entity you want to view in a graph, click the 3 dots icon, then click View in relationship graph.
If you have more than 500 nodes in the graph, you must select a filter before the graph will display.
At the top of the page, choose which relationships you want to view: Dependencies, domains, or teams.
You can apply filters to narrow the scope of the graph:
Click Filters at the top of the page.
Apply filters for the following options:
Sources
Groups
Entity types
Degrees from selected node
The page will automatically apply your selected filters and reload the visualization.
At the top of the graph, click Display options to select:
A Scorecard to filter the graph by
Whether to hide team nodes without children
Whether to hide archived entities
In the graph, click an entity to open a modal with the ability to:
Filter by this node
Go directly to its entity details page
View a list of its child entities or dependencies
You must have the Configure settings
permission.
Under Settings > Relationship graph, you can select whether to display dependencies, domains, or teams by default when you first open the relationship graph.
Cortex’s On-call Assistant leverages the PagerDuty integration to automatically surface the most vital information about entity health and metadata when an incident is triggered. On-call Assistant notifies the user(s) responsible for an incident via Slack, including information about the affected entity, recent deployments, ownership, and links to get more details, including dependencies, runbooks, and logs.
On-call Assistant helps users respond to incidents in real time, simplifying the incident response process and helping to reduce MTTR. It can also drive adoption and engagement through links to the catalogs and Scorecards.
When an incident is triggered in PagerDuty, On-call Assistant will notify relevant users via Slack. This alert will include information about the affected entity, deploy details, and ownership information so an on-call team member can reach out to other relevant parties about the incident.
Developers can access entity information that is already in Cortex directly from the Slack notification to quickly resolve issues. On-call Assistant provides a direct link to view the alert in PagerDuty, so you can also quickly access the incident from its source.
You must have the PagerDuty integration configured.
You must create an API key in PagerDuty with the Write
permission.
If you create an API key with Read-only
permissions, you will also need to configure a webhook to get the On-call Assistant working.
To enable, navigate to Settings > PagerDuty and toggle on Enable On-call Assistant.
If you added PagerDuty API key with Write
permissions, enabling the On-Call Assistant will create a webhook subscription in PagerDuty, allowing Cortex to receive events when incidents are triggered, escalated, or unacknowledged.
If you added PagerDuty API key with Read-only
permissions, you must also configure a webhook subscription.
In Cortex, on the PagerDuty settings page, click Configure webhook.
Copy the webhook URL. You will need this in the next steps.
In PagerDuty, add a new webhook.
Paste the Cortex webhook URL into the Webhook URL field.
Choose Account
for scope type.
Select the following in Event Subscription:
incident.escalated
incident.reopened
incident.triggered
incident.unacknowledged
A secret will be generated. Copy the secret. You will need it in the next step.
Navigate back to the browser window where your Cortex instance is open. In the Webhook configuration Secret field, enter the secret that you generated in PagerDuty.
Click Save at the bottom of the side panel.
Terraform is an infrastructure-as-code tool that lets you provision, manage, and update infrastructure. Terraform can help you manage databases, s3 buckets, clusters, and every other component that comprises your infra.
Terraform’s ability to manage resources comes from providers, which are plugins that enable interaction with cloud providers, SaaS providers, and other APIs. Providers allow you to define as code what a resource looks like.
The instructions on this page apply to Terraform Cloud, the web-based interface for Terraform.
Provision a new instance
Terraform will automatically provide you with a starter workspace when you begin — our example workspace is named "tfc-guide-example".
Update the instance name in the Terraform variables.tf
file.
All Terraform modules come with a file called variables.tf
. As part of the Terraform script, we can enter variables for a given set, like region or instance type. In the example screen shot below, the variable name is "My Other Great Instance".
Note: Terraform modules also come with a main.tf
file, which contains instructions and information about the action. In our example, the main.tf
file describes the instance that we’re going to create through Terraform.
Terraform stores a “state” about your infrastructure and configuration when it’s deployed. State maps resources to this configuration, and will update any time variables or properties are changed.
In Terraform Cloud, navigate to the Run page. Verify that the changes you made to the instance name in variables.tf
have applied.
In Terraform, there are two primary commands: plan
and apply
.
The plan
command creates a plan and preview of the changes that will be made to your infrastructure. During the plan stage, Terraform assesses the main.tf
file with variables and compares it against the state. If there are differences, Terraform prompts you to approve and apply the change.
When you navigate to the run that was triggered by updating variables.tf
, you can see that the plan was automatically conducted. In this case, the plan was to create an instance with the name provided earlier. Verify that the run displays a "Plan finished" message.
In AWS, you can confirm that the instance exists and that the Terraform action was successful:
In this example, we made direct modifications to the main branch, but typically, you will edit a separate branch and create a pull request. This approach will run a plan in Terraform, but will not automatically apply changes.
Cortex not only integrates with Terraform, but can enhance your use of it. Once the integration is set up, you’ll use a Scaffolder step in a Workflow to interact with Terraform.
You must have the Configure Scaffolder templates
permission.
When using the Scaffolder, it's best to edit a secondary branch and create a pull request — the Scaffolder will actually create the pull request for you, which someone else can approve to apply the changes.
Create a Cookiecutter JSON file that is equivalent to your Terraform module.
In this file, we defined the region and instance name. You’ll then update the fields in the variables.tf
file so it knows to pull information from the Cookiecutter JSON.
Register your template in Cortex.
After you have added the template to Cortex, you can create a Workflow that includes a Scaffolder block using the template.
Run the Workflow. When you run it, Cortex will automatically open a pull request against the selected repository.
To verify that the process worked, open Terraform Cloud and navigate to Runs.
Any runs that originate from Cortex will have "[Cortex Scaffolder]" at the start of their name. When we navigate into one of these runs, we can see that it was planned and finished but not yet applied, and we can see how many changes were proposed.
Terraform Cloud also has an API that can be used to make updates without following the pull request workflow. You can use a Workflow in Cortex to execute a run through the API. If a run is set to automatically apply, then Cortex will handle the rest of the process.
Create a Workflow in Cortex.
Add a user input block.
Define inputs for Instance name
, Region
, and Instance type
.
Add an HTTP request block. Configure the following fields:
HTTP method: Select POST
.
URL: Enter the URL for the Terraform API you want to call.
Headers: Add Authorization: Bearer {{token}}
and Content-type: application/(name)
.
Payload: Build the payload by referencing the outputs of the "User input" block, e.g., {{actions.input.outputs.instance-name}}
Save the Workflow. After saving, click Run at the top of the Workflow to run it.
When the Workflow is run in Cortex, it will override data in the variables.tf
file with information that was entered in the fields.
In Terraform Cloud, you can verify that the action was successful and the run was queued. Runs triggered by actions are named "Triggered via API."
Once the run has been applied, you can also verify it in AWS. In the example screen shot below, the instance name has been changed.