03.03.Infrastructure Setup

03.03. Infrastructure Setup

Step	Function	File path	Why this order / dependencies	Privileges	Notes
1	(manual or one-time)	—	Someone with rights to create projects and link billing must exist	your personal account or BUSINESS_OWNER	Usually done once per org — skip if already done
2	do_gcp_create_project	gcp-create-project.func.sh	Creates project + links billing — all later steps require project to exist	roles/resourcemanager.projectCreator + billing user	First script to run for a new environment
3	do_gcp_create_project_service_account	gcp-create-project-service-account.func.sh	Creates project-named SA + downloads key (temp disables key creation constraint)	personal account (needs org policy rights temp)	Critical — most automation uses this SA
4	do_gcp_configure_proj_sa_permissions	gcp-configure-proj-sa-permissions.func.sh	Grants owner + many powerful roles to the project SA	personal account or powerful SA	Probably what you want to run right now if project+SA already exist
5	do_gcp_project_apis_enable	gcp-project-apis-enable.func.sh	Enables APIs required by the rest of the stack (Storage, Secret Manager, Cloud Functions…)	usually the new project SA (after step 4)	Run after permissions are granted

CREATE THE PROJECTS

ORG=bnc APP=cpt ENV=dev ORG_ID=1080340024101 GCP_ACCOUNT=yordan.georgiev@csitea.net GCP_BILLING_ACCOUNT_ID=016958-C0B218-5A0B90 ./run -a do_gcp_000_create_project


ORG=bnc APP=cpt ENV=tst ORG_ID=1080340024101 GCP_ACCOUNT=yordan.georgiev@csitea.net GCP_BILLING_ACCOUNT_ID=016958-C0B218-5A0B90 ./run -a do_gcp_000_create_project

ORG=bnc APP=cpt ENV=prd ORG_ID=1080340024101 GCP_ACCOUNT=yordan.georgiev@csitea.net GCP_BILLING_ACCOUNT_ID=016958-C0B218-5A0B90 ./run -a do_gcp_000_create_project

ORG=bnc APP=cpt ENV=all ORG_ID=1080340024101 GCP_ACCOUNT=yordan.georgiev@csitea.net GCP_BILLING_ACCOUNT_ID=016958-C0B218-5A0B90 ./run -a do_gcp_000_create_project

ORG=bnc APP=cpt ENV=inf ORG_ID=1080340024101 GCP_ACCOUNT=yordan.georgiev@csitea.net GCP_BILLING_ACCOUNT_ID=016958-C0B218-5A0B90 ./run -a do_gcp_000_create_project

CLOUD RUN DOMAIN MAPPING

Cloud Run requires domain verification before custom domains can be mapped to services. This is a one-time manual setup per domain.

Why Domain Verification is Needed

GCP requires proof of domain ownership before allowing Cloud Run services to use custom domains
This prevents unauthorized domain mapping
The verification is associated with a Google account (personal or service account)
Once verified, domain mappings can be created automatically via CI/CD

Domain Verification Process

Step 1: Verify Parent Domain via Google Search Console

Verify the parent domain carpulsetracker.com to cover all subdomains (api., dev.api., tst.api., inf.api.):

Run: gcloud domains verify carpulsetracker.com --project=bnc-cpt-prd
This opens Google Search Console
Select "Domain" property type
Enter: carpulsetracker.com
Copy the provided TXT record value (e.g., google-site-verification=xxxxx)

Step 2: Add DNS TXT Record

Add the verification TXT record to Cloud DNS. If there's an existing TXT record (e.g., SPF), update it to include both:

# Authenticate
gcloud auth activate-service-account --key-file=$HOME/.gcp/.bnc/key-bnc-cpt-prd.json

# Check existing TXT records
gcloud dns record-sets list --zone=subzone-bnc-cpt-prd --project=bnc-cpt-prd --filter="type=TXT"

# Update TXT record (preserving SPF, adding verification)
gcloud dns record-sets update carpulsetracker.com. \
  --type=TXT \
  --ttl=300 \
  --rrdatas='"v=spf1 include:_spf.google.com ~all"','"google-site-verification=YOUR_TOKEN_HERE"' \
  --zone=subzone-bnc-cpt-prd \
  --project=bnc-cpt-prd

Step 3: Complete Verification in Search Console

Click "Verify" in Google Search Console after DNS propagation (may take up to 24 hours, usually minutes).

Step 4: Create Domain Mappings via GCP Console

The domain verification is tied to your personal Google account, not the service account. Create mappings via GCP Console:

Go to: https://console.cloud.google.com/run?project=bnc-cpt-prd
Click on the service (e.g., bnc-cpt-api-prd)
Go to "Manage Custom Domains" tab
Click "Add Mapping"
Enter the domain (e.g., api.carpulsetracker.com)

Repeat for all environments:

Environment	GCP Project	Service Name	Custom Domain
prd	bnc-cpt-prd	bnc-cpt-api-prd	api.carpulsetracker.com
prd (alias)	bnc-cpt-prd	bnc-cpt-api-prd	prd.api.carpulsetracker.com
dev	bnc-cpt-dev	bnc-cpt-api-dev	dev.api.carpulsetracker.com
tst	bnc-cpt-tst	bnc-cpt-api-tst	tst.api.carpulsetracker.com
inf	bnc-cpt-inf	bnc-cpt-api-inf	inf.api.carpulsetracker.com

The prd.api.carpulsetracker.com alias allows uniform URL pattern: {env}.api.carpulsetracker.com for all environments.

DNS Configuration (Already in Terraform 007-dns)

The CNAME records pointing to ghs.googlehosted.com are managed by Terraform step 007-dns:

api.carpulsetracker.com.      CNAME  ghs.googlehosted.com.
prd.api.carpulsetracker.com.  CNAME  ghs.googlehosted.com.  # alias for uniform URL pattern
dev.api.carpulsetracker.com.  CNAME  ghs.googlehosted.com.
tst.api.carpulsetracker.com.  CNAME  ghs.googlehosted.com.
inf.api.carpulsetracker.com.  CNAME  ghs.googlehosted.com.

This enables the uniform URL pattern: https://{env}.api.carpulsetracker.com for all environments.

CD Pipeline Behavior

The CD pipeline (.github/workflows/cd.yaml) handles domain mapping automatically:

If mapping exists: Skips creation, verifies mapping status
If mapping doesn't exist: Attempts to create (will warn if domain not verified)
Non-blocking: Domain mapping failures don't fail the deployment
All configuration is read from bnc-cpt-cnf/{env}.env.json

Troubleshooting

"Domain does not appear to be verified" - Verify the parent domain carpulsetracker.com in Search Console - Create the mapping via GCP Console using the account that verified the domain

CNAME conflict with TXT record - DNS doesn't allow CNAME + TXT on the same hostname - Verify the parent domain instead of the subdomain

DNS propagation delay - TXT record changes may take up to 24 hours to propagate - Custom domain health checks may fail until DNS propagates

GCP SECRETS SYNC FROM GOOGLE SHEET

This mechanism syncs secret values from a Google Sheet to GCP Secret Manager. It reads secrets from the sheet and updates them in GCP Secret Manager for each environment.

Prerequisites

Service Account Keys at ~/.gcp/.bnc/:
key-bnc-cpt-all.json - For reading Google Sheet (cross-project access)
key-bnc-cpt-{env}.json - For writing to GCP Secret Manager (env = inf, dev, tst, prd)
Google Sheet shared with the service account email bnc-cpt-all@bnc-cpt-all.iam.gserviceaccount.com
Secret containers must exist in GCP (created via Terraform step 029-create-gcp-secrets)
Docker container con-bnc-cpt-tf-runner must be running

Google Sheet Structure

Sheet URL: Configured in bnc-cpt-cnf/bnc-cpt/all.env.yaml under google_sheet_secrets.sheet_url
Worksheets:
all - Base values for all environments
inf, dev, tst, prd - Environment-specific overrides
Columns: VAR_NAME, VAR_VALUE

Worksheet Logic

Reads all worksheet first (base values)
Reads environment-specific worksheet (e.g., dev) if exists
Environment values override base values from all
Empty VAR_VALUE cells are automatically set to "n/a" (Cloud Run requires all mounted secrets to have values)

Naming Convention

VAR_NAME in sheet is converted to GCP secret ID:

VAR_NAME (Sheet)          →  GCP Secret ID
STRIPE_SECRET_KEY         →  bnc-cpt-stripe-secret-key
JWT_ACCESS_TOKEN_EXPIRE   →  bnc-cpt-jwt-access-token-expire
PAYPAL_CLIENT_SECRET      →  bnc-cpt-paypal-client-secret

Formula: {org}-{app}-{var_name.lower().replace('_', '-')}

Usage (via Make)

Run from bnc-cpt-utl/:

cd /opt/bnc/bnc-cpt/bnc-cpt-utl

# Sync secrets for a single environment
make do-gcp-sync-secrets ENV=dev

# Dry run (show what would be updated, no changes)
make do-gcp-sync-secrets ENV=dev DRY_RUN=1

# Sync all environments
for env in inf dev tst prd; do
  make do-gcp-sync-secrets ENV=$env
done

# Dry run all environments
for env in inf dev tst prd; do
  make do-gcp-sync-secrets ENV=$env DRY_RUN=1
done

Usage (Display Only - Read Sheet)

To only display secrets from the sheet without syncing:

make do-gcp-update-secrets ENV=dev

Execution Flow

Make target runs docker exec into con-bnc-cpt-tf-runner
Container executes ./run -a do_gcp_sync_secrets
Shell function activates Python via Poetry
Python script:
Authenticates to Google Sheets via service account
Reads all worksheet + environment worksheet
Merges values (env overrides all)
Authenticates to GCP via service account
Updates each secret in GCP Secret Manager

Output Example

===============================================
Syncing secrets for environment: dev
===============================================
2026-02-04 10:30:00 UTC ::: INFO: Connecting to Google Sheet: 1bYK6...
2026-02-04 10:30:01 UTC ::: INFO: Available worksheets: all, dev, tst, prd
2026-02-04 10:30:01 UTC ::: INFO: Reading 'all' worksheet (base values)...
2026-02-04 10:30:02 UTC ::: INFO:   Found 30 secrets in 'all' worksheet
2026-02-04 10:30:02 UTC ::: INFO: Reading 'dev' worksheet (environment overrides)...
2026-02-04 10:30:03 UTC ::: INFO:   Found 5 secrets (3 overrides, 2 new)
2026-02-04 10:30:03 UTC ::: INFO: Total secrets to sync: 32
2026-02-04 10:30:03 UTC ::: INFO: Syncing secrets to project: bnc-cpt-dev
2026-02-04 10:30:04 UTC ::: OK:   Updated: bnc-cpt-stripe-secret-key
2026-02-04 10:30:05 UTC ::: WARN:   Using 'n/a' for bnc-cpt-apple-pay-domain-name (VAR_NAME=APPLE_PAY_DOMAIN_NAME has no VAR_VALUE)
2026-02-04 10:30:06 UTC ::: OK:   Updated: bnc-cpt-apple-pay-domain-name
...
===============================================
2026-02-04 10:30:30 UTC ::: INFO: Summary:
2026-02-04 10:30:30 UTC ::: INFO:   Success: 22
2026-02-04 10:30:30 UTC ::: INFO:   Skipped: 10
2026-02-04 10:30:30 UTC ::: INFO:   Errors:  0
===============================================

Troubleshooting

"Secret does not exist in project" - Run Terraform step 029-create-gcp-secrets first: bash make do-generate-config-for-step ENV=dev STEP=029-create-gcp-secrets make do-provision ENV=dev STEP=029-create-gcp-secrets

"Spreadsheet not found" - Verify the sheet is shared with bnc-cpt-all@bnc-cpt-all.iam.gserviceaccount.com - Check sheet URL in all.env.yaml

"Failed to activate GCP service account" - Verify key file exists at ~/.gcp/.bnc/key-bnc-cpt-{env}.json - Check file permissions

"Missing required package: gspread" - Run Poetry install inside the container: bash docker exec -it con-bnc-cpt-tf-runner bash cd /opt/bnc/bnc-cpt/bnc-cpt-inf/src/python/gsheet-secrets-to-gcp poetry install

File	Purpose
`bnc-cpt-cnf/bnc-cpt/all.env.yaml`	Sheet URL configuration
`bnc-cpt-inf/src/python/gsheet-secrets-to-gcp/`	Python module
`bnc-cpt-utl/src/bash/run/gcp-sync-secrets.func.sh`	Shell action
`bnc-cpt-utl/src/make/tf-tasks.func.mk`	Make targets
`bnc-cpt-inf/src/terraform/029-create-gcp-secrets/`	Terraform for secret containers

WUI (FRONTEND) DEPLOYMENT

The Vue frontend is deployed to GCS buckets and served via HTTPS Load Balancer with Cloud CDN.

Infrastructure

The frontend infrastructure is provisioned via Terraform step 015-gcp-buckets-for-sites-static: - GCS bucket for static files - HTTPS Load Balancer - Cloud CDN - SSL certificate (managed) - DNS A record

Environments

Environment	URL	GCS Bucket
inf	https://inf.carpulsetracker.com	`gs://bnc-cpt-inf-site-static`
dev	https://dev.carpulsetracker.com	`gs://bnc-cpt-dev-site-static`
tst	https://tst.carpulsetracker.com	`gs://bnc-cpt-tst-site-static`
prd	https://carpulsetracker.com	`gs://bnc-cpt-prd-site-static`

Deployment Command

cd /opt/bnc/bnc-cpt/bnc-cpt-utl

# Deploy to single environment
ENV=dev ./run -a do_gcp_deploy_wui

# Deploy to all environments
for env in inf dev tst prd; do
  ENV=$env ./run -a do_gcp_deploy_wui
done

Deployment Steps

The do_gcp_deploy_wui action performs: 1. Build - Runs npm run build inside wui container 2. Verify - Checks dist directory exists 3. Sync - gsutil rsync to GCS bucket 4. CDN Invalidate - Clears CDN cache globally 5. Verify - HTTP 200 check on index.html

CDN Cache Configuration

Important: The index.html file must have no-cache headers to ensure users get the latest version:

# Set no-cache on index.html (done automatically by deploy action)
gsutil setmeta -h "Cache-Control:no-cache, no-store, must-revalidate" \
  gs://bnc-cpt-{env}-site-static/index.html

Asset files (JS, CSS) use content hashing in filenames and can be cached indefinitely.

Manual CDN Cache Invalidation

If users report seeing stale content:

gcloud auth activate-service-account --key-file=$HOME/.gcp/.bnc/key-bnc-cpt-{env}.json
gcloud compute url-maps invalidate-cdn-cache bnc-cpt-{env}-wui-https-url-map \
  --path="/*" --project=bnc-cpt-{env}

Prerequisites

WUI container running: bash cd /opt/bnc/bnc-cpt/bnc-cpt-utl make do-setup-wui
GCP service account keys at ~/.gcp/.bnc/key-bnc-cpt-{env}.json
Infrastructure provisioned: bash make do-generate-config-for-step ENV=dev STEP=015-gcp-buckets-for-sites-static make do-provision ENV=dev STEP=015-gcp-buckets-for-sites-static

File	Purpose
`bnc-cpt-utl/src/bash/run/gcp-deploy-wui.func.sh`	Deployment shell action
`bnc-cpt-cnf/bnc-cpt/{env}.env.yaml`	wui_fqdn configuration
`bnc-cpt-inf/src/terraform/015-gcp-buckets-for-sites-static/`	Infrastructure
`bnc-cpt-wui/src/vue/app/`	Vue source code

03.03.Infrastructure Setup

03.03. Infrastructure Setup

CREATE THE PROJECTS

CLOUD RUN DOMAIN MAPPING

Why Domain Verification is Needed

Domain Verification Process

Step 1: Verify Parent Domain via Google Search Console

Step 2: Add DNS TXT Record

Step 3: Complete Verification in Search Console

Step 4: Create Domain Mappings via GCP Console

DNS Configuration (Already in Terraform 007-dns)

CD Pipeline Behavior

Troubleshooting

GCP SECRETS SYNC FROM GOOGLE SHEET

Prerequisites

Google Sheet Structure

Worksheet Logic

Naming Convention

Usage (via Make)

Usage (Display Only - Read Sheet)

Execution Flow

Output Example

Troubleshooting

Related Files

WUI (FRONTEND) DEPLOYMENT

Infrastructure

Environments

Deployment Command

Deployment Steps

CDN Cache Configuration

Manual CDN Cache Invalidation

Prerequisites

Related Files