The Scorecards templates library provides a streamlined way to address engineering and business use cases. By using best practices from large enterprises, the library offers solutions for common engineering challenges.
Each template in the library is fully customizable to meet your organization's standards. You can modify queries, rules, and thresholds to tailor the Scorecards to your needs.
Some templates label their rules with maturity levels (L0 through L3).
Scorecard templates
Purpose: Validates if your services are ready for production deployment by checking defined objectives, alert configurations, and recent deployment activity.
Importance: Helps you find readiness gaps before deploying to production.
Prerequisites:
Key rules:
- Defined SLOs: Establishes clear targets for performance and reliability.
- APM service alerts configuration: Checks that alerts are set for prompt issue detection.
- Recent deployments: Verifies successful deployments within the last month.
- Runbook availability: Confirms that runbooks exist for alert conditions triggered in the last month.
Purpose: Assesses reliability against AWS standards for security and redundancy in cloud operations.
Importance: Helps you find reliability and security gaps in your cloud infrastructure.
Prerequisites: New Relic AWS monitoring
Key rules:
- EFS Encrypted check: Confirms encryption of Elastic File System (EFS) volumes.
- RDS Instance check retention period: Checks backup retention periods for Relational Database Service (RDS).
- RDS DeletionProtection check: Verifies deletion protection.
- RDS MultiAZ presence check: Checks for deployment across multiple availability zones (MultiAZ).
- RDS AutoMinorVersionUpgrade check: Confirms automatic minor version upgrades.
Purpose: Evaluates resource allocation and process improvements using key performance indicators.
Importance: Quantifies software delivery performance, aiding strategic decision-making.
Prerequisites:
Key rules:
- Deployment frequency: Tracks successful deployment rates.
- Mean time to recovery: Measures recovery time from failures.
- Change failure rate: Tracks the percentage of changes that result in failures.
- Lead time for changes: Measures the time taken from code commit to deployment.
Purpose: Establishes and enforces consistent tagging practices for resources within your organization.
Importance: Consistent tags help you filter and group services across views such as Maps and catalogs.
Prerequisites: APM
Key rules:
- Team Tag Exists: Checks that services carry an owning-team tag for clear ownership and accountability.
- Environment Tag Exists: Checks that services carry a consistent environment tag for filtering and grouping.
- APM Criticality is set: Checks that services have a criticality tag to prioritize by business importance.
Purpose: Establishes a comprehensive tagging strategy and complete visibility into service dependencies and relationships.
Importance: Highlights gaps in tags, relationships, trace, and synthetic coverage so you can address them.
Prerequisites:
Key rules:
- (L0) Team Tag Coverage: Identifies entities missing a
teamtag. - (L0) Environment Tag Coverage: Identifies entities missing an
environmenttag. - (L0) Uninstrumented Entities: Flags HTTP service and database entities with unknown relationships that need supplemental instrumentation.
- (L0) Distributed Trace Coverage: Measures the percentage of APM entities emitting distributed trace data.
- (L0) Synthetic Coverage: Measures the percentage of APM entities covered by synthetic monitors.
Purpose: Develops a comprehensive alerting strategy and incident management approach for service reliability.
Importance: Highlights alert gaps, noise, and SLI coverage so you can prioritize critical services.
Prerequisites:
Key rules:
- (L1) Infrastructure Alert Coverage: Identifies hosts and Kubernetes pods that lack alert coverage.
- (L1) Service Delivery Alert Coverage: Identifies services, browser, mobile, and synthetic monitors that lack alert coverage.
- (L1) Critical Alert Coverage: Flags overreliance on critical alert conditions that can cause alert fatigue.
- (L1) Alert Noise: Identifies policies that generate a high volume of incidents.
- (L2) Service Level Coverage: Checks whether entities have a defined service level indicator (SLI).
- (L2) Alerts Mean Time To Close: Evaluates the time taken to close incidents, targeting resolution within 30 minutes.
- (L2) APM Criticality Tag Coverage: Identifies entities missing a
criticalitytag. - (L3) Service Level Attainment: Evaluates the compliance score for each defined SLI, using a default threshold of 95%.
Purpose: Emphasizes efficient resource usage and robust security practices by monitoring CPU and memory usage, integrating change tracking, and detecting vulnerabilities.
Importance: Adhering to resource, change-tracking, and security standards helps you run efficient, secure infrastructure.
Prerequisites:
Key rules:
- (L1) CPU Utilization: Checks the 95th percentile of CPU usage for infrastructure entities against efficiency best practices.
- (L1) Memory Utilization: Checks the 95th percentile of memory usage for infrastructure entities against efficiency best practices.
- (L2) Change Tracking: Checks for change-tracking events across your APM entities.
- (L3) Service Vulnerabilities: Checks the percentage of APM entities with detected vulnerabilities.
Purpose: Monitors frontend performance and user interaction through JavaScript errors, HTTP 5xx errors, and Core Web Vital metrics.
Importance: Adhering to frontend error and Core Web Vitals standards helps you maintain a strong user experience.
Prerequisites:
Key rules:
- (L1) JS Error Rate: Checks the percentage of browser entities with JavaScript errors.
- (L1) Service Error Rate: Checks the percentage of APM services reporting 5xx errors in HTTP requests.
- (L2) Core Web Vitals: Largest Contentful Pane (LCP): Flags entities whose 75th-percentile LCP exceeds 2.5 seconds.
- (L2) Core Web Vitals: Interaction to Next Paint (INP): Flags entities whose 75th-percentile INP exceeds 200 milliseconds.
- (L2) Core Web Vitals: Cumulative Layout Shift (CLS): Flags entities whose 75th-percentile CLS exceeds 0.1.
- (L3) Service Level Attainment: Evaluates the compliance score for each defined SLI, using a default threshold of 95%.
Purpose: Assesses vulnerability risk across APM services and infrastructure.
Importance: Surfacing critical and high-severity vulnerabilities, active ransomware CVEs, and exploit likelihood helps prioritize remediation.
Prerequisites:
Key rules:
- APM: Under 3 Critical Severity: Checks that an APM service has fewer than 3 critical-severity vulnerabilities.
- APM: Under 5 High Severity: Checks that an APM service has fewer than 5 high-severity vulnerabilities.
- APM: Active Ransomware: Flags reported CVEs that have an active ransomware campaign.
- APM: Exploit Probability: Flags critical or high-severity vulnerabilities likely to be exploited (95th+ percentile EPSS).
- INFRA: Under 3 Critical Severity: Checks that a host or container has fewer than 3 critical-severity vulnerabilities.
- INFRA: Under 5 High Severity: Checks that a host or container has fewer than 5 high-severity vulnerabilities.
- INFRA: Active Ransomware: Flags reported CVEs that have an active ransomware campaign.
- INFRA: Exploit Probability: Flags critical or high-severity vulnerabilities likely to be exploited (95th+ percentile EPSS).
Purpose: Measures the operational quality and governance of AI responses across safety, cost efficiency, and model compliance.
Importance: Adhering to AI quality and governance standards helps you keep AI usage responsible and cost-effective.
Prerequisites: AI monitoring
Key rules:
- LLM Runtime Error Rate: Checks that LLM-enabled applications maintain a low operational error rate.
- Cost Efficiency - Average tokens per assistant response: Checks that the average token count per assistant response stays within budget.
- Model Governance - Approved model usage for assistant responses: Checks that only approved models are used for assistant responses in production.