05-Specifications / 05.11.GCP-Log-Analytics.spec

05.11.GCP Log Analytics.spec

05.11. GCP Log Analytics.spec

1. Objective

Enable the Log Analytics feature on the production GCP Logging buckets. This allows developers and operators to use standard SQL (BigQuery-compatible) to query application logs, calculate error rates, and analyze user behavior without exporting logs to an external database.

2. Infrastructure Requirements (Terraform)

The feature should be implemented in a new Terraform step: 145-gcp-log-analytics.

2.1 Resource Configuration

We will use the google_logging_project_bucket_config resource to upgrade the default _Default bucket.

resource "google_logging_project_bucket_config" "default_analytics" {
    project        = var.project_id
    location       = "global"
    bucket_id      = "_Default"
    enable_analytics = true
}

2.2 Permissions

The bnc-cpt-inf service account (used by Terraform) requires the following role to modify bucket configurations: - roles/logging.admin

The developers/users require the following role to run SQL queries: - roles/logging.viewAccessor (on the _Default bucket)

3. SQL Analytics Examples

Once enabled, logs are accessible via the "Log Analytics" page in the GCP Console or via gcloud alpha logging query.

3.1 HTTP Error Rate (Last 24 Hours)

SELECT
  httpRequest.status,
  COUNT(*) as count,
  httpRequest.requestUrl
FROM
  `PROJECT_ID.global._Default._AllLogs`
WHERE
  timestamp > TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 24 HOUR)
  AND httpRequest.status >= 400
GROUP BY 1, 3
ORDER BY count DESC

3.2 Top Active Users (By Report Generation)

SELECT
  JSON_VALUE(jsonPayload.session_id) as session,
  COUNT(*) as reports_generated
FROM
  `PROJECT_ID.global._Default._AllLogs`
WHERE
  textPayload LIKE '%Vehicle report PDF generated successfully%'
GROUP BY 1
ORDER BY reports_generated DESC

4. Implementation Strategy

1. Step 145: Create the Terraform module in bnc-cpt-inf/src/terraform/145-gcp-log-analytics.
2. Config: Add enable_log_analytics: true to prd.env.yaml.
3. Provision: Run make do-provision STEP=145-gcp-log-analytics ENV=prd.
4. Validation: Verify that the "Log Analytics" tab is active in the GCP Console for the bnc-cpt-prd project.

5. Limitations & Costs

• Retention: Analytics are only available for logs within the bucket's retention period (default 30 days).
• Cost: There is no additional cost for enabling analytics on the _Default bucket, but complex queries incur standard BigQuery-style processing costs if they exceed free tiers.

---

Technical Specification v1.0.0 — BNC CPT Observability