DataDog Service Catalog

 Getting Started with Service Catalog


Service Catalog Reliability view showing several services and their associated MTTR, deployment metrics, issues, incidents, SLOs, and monitor statuses.

Overview

Datadog Service Catalog provides a consolidated view of your services, combining ownership metadata, performance insights, security analysis, cost allocation, and much more. Having this centralized hub allows your development teams to discover and manage critical components in your runtime environments.

This page walks you through getting started with Service Catalog in Datadog.

Prerequisites

If you have not already, create a Datadog account.

Adding entries to Service Catalog

Automatically detected services

Service Catalog automatically discovers services based on application performance telemetries such as APM, USM, and RUM. The integration with APM enables Datadog to routinely discover new services at the same frequency as your traces are collected. With USM, the Datadog Agent connects to your eBPF-compatible hosts. USM automatically detects the services running on this infrastructure and tags them using unified service tagging.

User-defined services

If you are not using these products, you can manually declare services as entries in the service.definition.yaml registry. The definitions in this file include all the metadata that the catalog uses to file your services. These can be created and updated programmatically using an internal API or with a configuration management service like Terraform. You should include this file in your version control and regularly update it whenever new resources are added to your environment.

The following example represents a shopping-cart service from an e-commerce application. It includes important metadata about the service, such as its owning team, languages used, runbook link, and code repository.

service.datadog.yaml

schema-version: v2.2
dd-service: shopping-cart
team: e-commerce
application: shopping-app
tier: "1"
type: web
languages:
  - go
  - python
contacts:
  - type: slack
    contact: https://yourorg.slack.com/archives/e-commerce
  - type: email
    contact: ecommerce@example.com
  - type: microsoft-teams
    contact: https://teams.microsoft.com/example
links:
  - name: Runbook
    type: runbook
    url: http://runbook/shopping-cart
  - name: Source
    type: repo
    provider: github
    url: https://github.com/shopping-cart
  - name: Deployment
    type: repo
    provider: github
    url: https://github.com/shopping-cart
  - name: Config
    type: repo
    provider: github
    url: https://github.com/consul-config/shopping-cart
  - name: E-Commerce Team
    type: doc
    provider: wiki
    url: https://wiki/ecommerce
  - name: Shopping Cart Architecture
    type: doc
    provider: wiki
    url: https://wiki/ecommerce/shopping-cart
  - name: Shopping Cart RFC
    type: doc
    provider: google doc
    url: https://doc.google.com/shopping-cart
tags:
  - business-unit:retail
  - cost-center:engineering
integrations:
  pagerduty:
    service-url: https://www.pagerduty.com/service-directory/PSHOPPINGCART
  opsgenie:
    service-url: "https://www.opsgenie.com/service/uuid"
    region: "US"
ci-pipeline-fingerprints:
  - id1
  - id2
extensions:
  additionalProperties:
    customField1: customValue1
    customField2: customValue2

You can also use any existing knowledge sources your organization maintains, such as Configuration Management Database (CMDB) tables (through Datadog’s ServiceNow integration) and spreadsheets, to populate your Service Catalog. Datadog also has an integration with Backstage that allows you to import any data or services registered here into Datadog directly.

Finally, you can also create entries from service tags in other Datadog products like Infrastructure Monitoring and Log Management.

Import Entries tab in the Service Catalog setup and configuration section

Managing metadata in Service Catalog

After you create these initial entries in your Service Catalog, it is important to keep the catalog updated consistently so that it remains effective. Service definition files should exist within your team’s version control to make it easier to reference new deployments and other changes to the services where an update may be required.

You can also automate the management of your service metadata by configuring a GitHub action. This will also allow you to ensure that teams are declaring services in a way that meets your standards, such as requiring all of your production services to have valid runbook links.

If your organization has an existing registry of ownership, including internal systems like Backstage or a spreadsheet, a central team can schedule updates to service definition files using API calls.

Connect telemetry from across your monitoring stack

Connect monitoring data from the rest of your observability platform to improve the utility of your catalog.

With unified service tagging, you can use the service tag to cross-reference service entities in the Service Catalog across all Datadog products. These tags can help enrich your service entities with metadata, metrics, and other context sources like Infrastructure Monitoring, RUM, Log Management, Software Delivery, and Security.

Application performance telemetry from Universal Service Monitoring and APM also provide out-of-the-box dependency mapping for your system ecosystem, so you can see how components interact with each other in all your runtime environments.

Enriching your Service Catalog

After services are populated in the catalog, you can enrich your service definitions with additional context to make them more useful. This could include adding key pieces of service metadata to your service.definition.yaml files such as:

  • Team
  • On-call engineer
  • Contact channel
  • Documentation links
  • Last deployed version
  • Code repositories
  • Runbook links
  • Library dependencies and their versions
  • Relevant dashboards

Service Catalog uses service definition schemas to store and display this metadata about your services. The schemas have built-in validation rules to ensure that only valid values are accepted. There are currently four supported schemas: v2, v2.1, v2.2, and v3.

Using Service Catalog Scorecards

Service Scorecards help you codify your organization’s best practices as rules that can be evaluated. By implementing scorecards in your catalog, your teams can measure many dimensions of service quality, including:

  • Monitoring coverage
  • Production readiness
  • Security posture
  • Adoption of the latest internal tooling
  • Integration checks

Datadog Scorecards include 10 out-of-the-box rules across observability practices, ownership tagging, and production readiness checkpoints. You can also define your own custom rules. For example, you could create a scorecard that contains a set of rules that map to the steps in your security review process, so that you can quickly audit whether each of your services is compliant. These rules might include checks related to CVE analysis, RBAC configuration, or other security parameters.

To add custom rules to your Scorecards dashboard:

  1. Click Create Rule on the Scorecards page.
  2. Specify the name of the rule, the scorecard it belongs to, a rule description, and the owning team.
  3. Send an outcome of pass, fail, or skip for each {rule, service} tuple that you are evaluating to the Scorecards API /scorecard/outcomes/batch endpoint.
  4. View an overview of outcomes in the Scorecards dashboard.
Create Rule modal to add custom rules in Scorecards dashboard

Comments

Post a Comment

Popular posts from this blog

Different Types of Reports in Scrum - Agile

Image
  Introduction to Agile Reporting I’m not going to elaborate on the Agile Framework itself here.  1. Sprint Burndown At a Sprint-level, the burndown presents the  easiest way to track and report status  (the proverbial  Red / Amber / Green ), i.e., whether your Sprint is on or off-track, and what are the chances of meeting the Sprint goals. The burndown chart – when used right – can provide  near-real time updates  on Sprint progress. “If your team do it right, then they would take in just the right amount of work into a sprint.” At the beginning of a Sprint, the Scrum team perform Sprint Planning and agree to take on development work worth a certain number of Story points. This forms the basis for the Sprint Burndown chart. The total story points agreed at the beginning of the sprint make up the y-axis, and the individual dates in the Sprint make up x-axis. If your team do it right, then they would take in just the right amount of work into ...
Read more

Terraform

Image
  Terraform is an infrastructure as code tool that lets you build, change, and version cloud and on-prem resources safely and efficiently. HashiCorp Terraform is an infrastructure as code tool that lets you define both cloud and on-prem resources in human-readable configuration files that you can version, reuse, and share. You can then use a consistent workflow to provision and manage all of your infrastructure throughout its lifecycle. Terraform can manage low-level components like compute, storage, and networking resources, as well as high-level components like DNS entries and SaaS features. How does Terraform work? Terraform creates and manages resources on cloud platforms and other services through their application programming interfaces (APIs). Providers enable Terraform to work with virtually any platform or service with an accessible API. HashiCorp and the Terraform community have already written  thousands of providers  to manage many different types of resources...
Read more

Scrum Master Interview help - Bootcamp

Understanding the MoSCoW prioritization | How to implement it into your project Projects, irrespective of their size or complexity, often juggle numerous tasks and requirements. In such a scenario,  MoSCoW prioritization  becomes essential to ensure the successful completion of the project.  Besides, it is also a very effective way to manage project priorities. With a straightforward and adaptable approach, the MoSCoW method helps manage stakeholder expectations and improve project outcomes. What is the MoSCoW prioritization technique?  The MoSCoW method is a prioritization matrix widely used in project management and software development. The term MoSCoW is an acronym that stands for  Must have ,  Should have ,  Could have , and  Won’t have , each denoting a level of priority. Here is the breakdown of the MoSCoW method:  Must have:  These are critical requirements that the project cannot be completed withou...
Read more