03-Development / 03.04.Utilities-Setup

03.04.Utilities Setup

03.04. Utilities Setup

Prerequisites

Initial Setup

1. Clone All Repositories

mkdir -p /opt/bnc/bnc-cpt && cd /opt/bnc/bnc-cpt
for repo in bnc-cpt-api bnc-cpt-wui bnc-cpt-utl bnc-cpt-inf bnc-cpt-cnf tpl-gen bnc-cpt-rdb; do
  git clone git@github.com:csitea/$repo.git
done

2. Verify Directory Structure

ls /opt/bnc/bnc-cpt/
# Expected: bnc-cpt-api  bnc-cpt-cnf  bnc-cpt-inf  bnc-cpt-rdb  bnc-cpt-utl  bnc-cpt-wui  tpl-gen

3. GCP Service Account Keys

Place keys at ~/.gcp/.bnc/:

mkdir -p ~/.gcp/.bnc
# Copy your service account key files here:
# key-bnc-cpt-inf.json
# key-bnc-cpt-dev.json
# key-bnc-cpt-tst.json
# key-bnc-cpt-prd.json
# key-bnc-cpt-all.json

chmod 600 ~/.gcp/.bnc/*.json

Keys are generated during GCP project setup via do_gcp_create_project_service_account (see bnc-cpt-inf/doc/bnc-cpt.ISG.md).

4. Set Shell Variables

export UID=$(id -u)
export GID=$(id -g)

Docker Container Setup

Start All Containers

cd /opt/bnc/bnc-cpt/bnc-cpt-utl
make do-setup-bnc-cpt-all

Start Individual Containers

# API backend (port 8100)
make do-setup-api

# WUI frontend (port 3333)
make do-setup-wui

# Terraform runner
make do-setup-tf-runner

# All infrastructure containers (tf-runner + tpl-gen + conf-validator)
make do-setup-app-inf

Verify Containers

docker ps | grep bnc-cpt
# Expected:
# con-bnc-cpt-cpt-api         (port 8100)
# con-bnc-cpt-wui              (port 3333)
# con-bnc-cpt-tf-runner
# con-bnc-cpt-tpl-gen
# con-bnc-cpt-conf-validator

Docker Environment File

The .env file at src/docker/.env controls Docker container configuration:

Variable Default Description
MOUNT_WORK_DIR /opt/bnc/bnc-cpt Host project root
HOME_API_PROJ_PATH /home/appusr/opt/bnc/bnc-cpt/bnc-cpt-api API path inside container
HOME_WUI_PROJ_PATH /home/appusr/opt/bnc/bnc-cpt/bnc-cpt-wui WUI path inside container
API_PORT_HOST 8100 API host port
WUI_PORT_HOST 3333 WUI host port

For CI environments, .env.cin is used instead of .env.

Terraform Setup

Generate Configuration

cd /opt/bnc/bnc-cpt/bnc-cpt-utl

# Generate config for a specific step and environment
make do-generate-config-for-step ENV=dev STEP=029-create-gcp-secrets

# Generate for all environments
for env in inf dev tst prd; do
  make do-generate-config-for-step ENV=$env STEP=029-create-gcp-secrets
done

Provision Infrastructure

# Provision a step
make do-provision ENV=dev STEP=029-create-gcp-secrets

# Full provisioning order for a new environment
for step in 001-enable-gcp-services 007-dns 015-gcp-buckets-for-sites-static 025-gcp-artifact-registry 029-create-gcp-secrets 030-gcp-cloud-run; do
  make do-generate-config-for-step ENV=dev STEP=$step
  make do-provision ENV=dev STEP=$step
done

CI/CD Setup for Dependent Projects

UTL itself has no CI/CD pipeline. It is consumed by the API and WUI CI/CD pipelines. The CI/CD setup for those projects requires:

GitHub Repository Variables (per repo)

Variable Value
ORG bnc
APP cpt
PROJ bnc-cpt-api or bnc-cpt-wui
GH_ORG csitea

GitHub Repository Secrets (per repo)

Secret Description
ALL_CSI_GITHUB_TOKEN PAT with read:org + repo scopes
GCP_SA_KEY_INF Content of key-bnc-cpt-inf.json
GCP_SA_KEY_DEV Content of key-bnc-cpt-dev.json
GCP_SA_KEY_TST Content of key-bnc-cpt-tst.json
GCP_SA_KEY_PRD Content of key-bnc-cpt-prd.json

Automated Secret Provisioning

GitHub secrets are provisioned via Terraform step 120-github-general-secrets:

cd /opt/bnc/bnc-cpt/bnc-cpt-utl
make do-generate-config-for-step ENV=all STEP=120-github-general-secrets
make do-provision ENV=all STEP=120-github-general-secrets

Tools Required on Host

Tool Purpose Install
docker Container runtime apt install docker.io
docker-compose Multi-container orchestration apt install docker-compose or Docker Desktop
make Build orchestration apt install make
git Version control apt install git
gcloud GCP CLI snap install google-cloud-cli --classic
jq JSON processing apt install jq
yq YAML processing snap install yq

GCP Secret Manager Integration

Test credentials (Stripe test card, PayPal test user) and application secrets are stored in GCP Secret Manager. They are fetched at runtime — never written to disk.

How Secrets Reach Each Context

Context Mechanism Secrets Location
Cloud Run (deployed) gcloud run deploy --set-secrets Injected as env vars at container startup
Local Docker dev Host fetches via gcloud -> docker-compose passes to containers In-memory env vars only
CI/CD pipeline GitHub runner fetches via gcloud -> passes to docker exec -e In-memory, masked in logs

Local Development Setup

Fetch secrets on the host before starting containers:

cd /opt/bnc/bnc-cpt/bnc-cpt-utl

# Fetch ALL secrets for dev environment
eval "$(ENV=dev bash src/bash/scripts/gcp-fetch-secrets.sh)"

# Or fetch only test credentials
eval "$(ENV=dev SECRETS_FILTER=test bash src/bash/scripts/gcp-fetch-secrets.sh)"

# Then start containers (docker-compose inherits host env vars)
make do-setup-app

The docker-compose environment: section passes through test credential env vars (with empty defaults) so containers work with or without secrets.

Using the Shell Action

# Fetch and export all secrets for dev (within ./run framework)
ENV=dev ./run -a do_gcp_fetch_secrets

# Fetch only test credentials
ENV=dev SECRETS_FILTER=test ./run -a do_gcp_fetch_secrets

Secret Naming Convention

GCP Secret Manager IDs map to env vars automatically:

Secret ID Env Var
bnc-cpt-stripe-secret-key STRIPE_SECRET_KEY
bnc-cpt-stripte-test-account-card-number STRIPTE_TEST_ACCOUNT_CARD_NUMBER
bnc-cpt-paypal-test-user-username PAYPAL_TEST_USER_USERNAME

Rule: strip bnc-cpt- prefix, replace - with _, uppercase.

Prerequisites

Verification

cd /opt/bnc/bnc-cpt/bnc-cpt-utl

# Check shell action framework
./run --help

# Run test action
./run -a do_hello_world

# Check Make targets
make help

# Verify GCP secrets access
ENV=dev SECRETS_FILTER=test ./run -a do_gcp_fetch_secrets