Skip to main content

Secrets & Credentials Inventory

Mirrored document

This page is one of two synchronized copies of the same content:

  • Dashboard docs copy: admin-docs/docs/architecture/secrets-and-credentials.md (this page)
  • Root repo copy: SECURITY.md (repository root)

If you edit one, edit the other in the same PR so they never drift. Public /docs/ omits Cloudflare token names, Account IDs, and secret payloads even when the repo inventory lists how those infra credentials rotate.

This is the canonical map of every secret, key, password, and credential in the FinMatch platform: what it is, where it is stored, which component owns it, and how to rotate it. No secret values appear in this document — only locations and procedures.


1. TL;DR — six secret stores​

#StoreHolds
1GCS bucket gs://finmatch-sharedPublic storefront runtime files (rate card, lender styling/overrides, analytics, css/js). Private JSON is in finmatch-secrets (§9 File 1–4).
2GCS bucket gs://finmatch-secrets (private, PAP on)Private files migrated off the public finmatch-shared bucket. Per-file Cloud Run env-var override (default still finmatch-shared). See §9.
3GCP Secret Manager (project finmatch-finance-mkt)Stripe API keys + Stripe webhook signing secrets
4Cloud Run service env varsHMAC signing secret, SendGrid, internal service-to-service secret, admin elevation secret, cache-bust secrets, per-file bucket overrides
5Cloudflare (Worker KV + Worker secrets)Per-merchant Finance Assistant key mirror, optional default FA key, allowed origins
6GitHub Actions repo secretsThe GCP service-account JSON that CI uses to deploy

Key principle: no production secret's only home is a developer laptop. Every item above lives in a managed cloud store. See §7 for the local-machine risk assessment.


2. Full inventory​

Secret / credentialStored inFormat / keyOwner (writer)Rotation path
Admin / partner / merchant loginsgs://finmatch-secrets/auth-credentials.json (AUTH_CREDENTIALS_BUCKET; public copy deleted)users[username] with password, role, apiKey, keys[]auth-apiAdmin UI (Users) / auth-api admin endpoints
Merchant Finance Assistant API keys (sk_live_…)(a) auth-credentials.json user.apiKey + typed keys[]; (b) Cloudflare KV MERCHANT_KEYS[merchantID]sk_live_…auth-api (GCS and Cloudflare KV)Regenerate in Admin → merchant detail (Copy / issue). auth-api writes the matching KV pair. Do not paste keys into KV by hand. Historical backfill: POST /admin/merchant-keys/sync-cloudflare-kv. Coverage (IDs only): GET /admin/merchant-keys/cloudflare-kv-coverage.
Partner API keys (external partners)gs://$PARTNER_API_KEYS_BUCKET/partner-api-keys.json (git-ignored). Env PARTNER_API_KEYS_BUCKET defaults to finmatch-shared; §9 File 1 cutover destination is finmatch-secrets.keys[sk_…] → { partner_id, mode, status }partner-api (lib/auth.js)Edit partner-api-keys.json in the live keys bucket; mark old status: revoked
Sandbox preview tokens (spv_…)gs://finmatch-secrets/sandbox-tokens.json (hashes only; SANDBOX_TOKENS_BUCKET; public copy deleted)hash → metadata (no plaintext)auth-apiRegenerate via Admin sandbox panel; auth-api reconcile endpoint
Stripe API keys (live + test)GCP Secret Manager: stripe-restricted-key, stripe-restricted-key-test (fallback: Cloud Run env STRIPE_API_KEY_LIVE / STRIPE_API_KEY_TEST)Stripe restricted keystripe-api service configAdd new Secret Manager version; redeploy/restart stripe-api
Stripe webhook signing secretsGCP Secret Manager: stripe-webhook-secret, stripe-webhook-secret-test (fallback: env STRIPE_WEBHOOK_SECRET*)whsec_…stripe-api service configNew Secret Manager version; update Stripe dashboard endpoint
Referral-attribution HMAC secretCloud Run env var FA_REFERRAL_HMAC_SECRET (identical on partner-api and finance-assistant)HMAC-SHA256 secretCloud Run service configUpdate env var on both services together, then redeploy
Zopa / DivideBuy per-merchant signing keysgs://finmatch-secrets/merchants.json (MERCHANTS_JSON_BUCKET; public copy deleted) → profiles[merchantId].zopa (public key, secret key, base URL)per-merchantmerchant-api (Admin FA settings)Admin → Finance Assistant settings → Save
SendGrid API key (email)Cloud Run env SENDGRID_API_KEY (on merchant-api, finance-assistant)SG.…Cloud Run service configRotate in SendGrid; update env var; redeploy
Internal service-to-service secretCloud Run env INTERNAL_API_SECRET (merchant-api, stripe-api, auth-api, contacts-api, finance-assistant — all five pinned in their deploy YAMLs)shared secretCloud Run service configUpdate env var on all consumers together
Admin elevation secretCloud Run env ADMIN_ELEVATION_SECRET (auth-api)shared secretCloud Run service configSet explicitly in prod (see §6 hardening)
Cache-bust secretsCloud Run env PARTNER_API_BUST_SECRET / CACHE_BUST_SECRETshared secretCloud Run service configUpdate env var; redeploy
CI → GCP deploy credentialGitHub Actions repo secret GCS_CREDENTIALSGCP service-account JSONGitHub repo adminsCreate new GCP SA key; update GitHub secret; delete old key
Cloudflare Worker fallback keyCloudflare Worker secret DEFAULT_FINANCE_ASSISTANT_KEY (optional)sk_live_…Cloudflare dashboard / wranglerSet/rotate in Cloudflare

3. How the pieces fit together​

Login / dashboard auth​

login.finmatch.io → auth-api POST /auth → validates against auth-credentials.json in GCS → returns role + permissions (client stores them in localStorage). Session expiry ~24h. Source: platform/AUTH_SETUP_GUIDE.md.

Merchant Finance Assistant flow (Cloudflare-fronted)​

  1. Browser → Cloudflare Worker (finance-assistant-proxy) with merchantID in the body.
  2. Worker looks up MERCHANT_KEYS.get(merchantID) in KV → the merchant's FA key. Missing ⇒ 400.
  3. Worker forwards to Cloud Run finance-assistant with Authorization: Bearer <key>.
  4. finance-assistant re-validates the key against auth-credentials.json (role merchant, status: active). Missing ⇒ 403.

So the merchant key must exist in two places and match: Cloudflare KV and auth-credentials.json. auth-api writes both after Add Merchant provision, Copy, Users → Sync Merchant API Keys, issue / revoke, or merchant ID change. merchant-api emits merchant.created and merchant.id_changed. KV writes skip until Cloudflare credentials are configured on auth-api; merchant create still succeeds. Do not paste Finance Assistant keys or Cloudflare tokens onto this public page. Source: cloudflare/finance-assistant-proxy/README.md, admin-docs/docs/merchants/merchant-users-and-finance-assistant-keys.md.

Partner API auth​

Partners send Authorization: Bearer sk_…. partner-api (lib/auth.js) validates against partner-api-keys.json in the bucket named by Cloud Run env PARTNER_API_KEYS_BUCKET (default finmatch-shared). Rate-card reads stay on finmatch-shared. Merchants / router live in finmatch-secrets. If the keys file is absent, the service falls back to built-in demo keys (dev only). Source: developer-docs/docs/partner-api/authentication.md, cloud-run/partner-api/ADD_NEW_PARTNER.md.

Server-side lender request signing​

  • Referral attribution: partner-api signs finance-offer-summary links with an HMAC over merchantId|partnerId|ts using FA_REFERRAL_HMAC_SECRET; finance-assistant verifies with the same secret. No secret ⇒ links are left unsigned and attribution falls back to P000000 (fail-safe). Source: cloud-run/*/lib/referral_signature.js, FA_REFERENCE_PLAN.md.
  • Zopa / DivideBuy: signed per-merchant using keys stored in merchants.json (profiles[merchantId].zopa), not env vars.

Stripe​

stripe-api loads live/test keys and webhook secrets from Secret Manager first, falling back to Cloud Run env vars. Source: STRIPE_INTEGRATION_GUIDE.md, cloud-run/stripe-api/index.js.

Deploys​

Every *-deploy.yml authenticates to GCP with the GitHub Actions secret GCS_CREDENTIALS (a GCP service-account JSON) — entirely GitHub-side, not tied to any laptop.


4. Protected runtime files (never edit via git)​

These live only in GCS and are written by a single owning service. Do not commit git copies (.gitignore blocks several):

  • auth-credentials.json — auth-api; lives in gs://finmatch-secrets/ (AUTH_CREDENTIALS_BUCKET; public copy deleted)
  • partner-api-keys.json — lives only in GCS; bucket from PARTNER_API_KEYS_BUCKET (default finmatch-shared; destination finmatch-secrets per §9 File 1)
  • sandbox-tokens.json — auth-api (hashes only); lives in gs://finmatch-secrets/ (SANDBOX_TOKENS_BUCKET; public copy deleted)
  • merchants.json, merchant-router.json, cors.json — merchant-api; live in gs://finmatch-secrets/ (MERCHANTS_JSON_BUCKET / MERCHANT_ROUTER_BUCKET / CORS_JSON_BUCKET; public copies deleted)
  • finmatch-rate-card.json, lender-originated-quotes.json — credit-products-api
  • contacts/contacts.json — contacts-api; lives in gs://finmatch-secrets/ (CONTACTS_BUCKET; public copy deleted)
  • feedback/notes.json + feedback/images/ — auth-api; live in gs://finmatch-secrets/ (FEEDBACK_BUCKET; public copies deleted). feedback/index.html stays on finmatch-admin (the Feedback page).

5. "Do not commit secrets" rules​

.gitignore blocks: partner-api-keys.json, **/*.live.json, **/*secret*.json, **/*credential*.json, **/API_KEYS*.md, .env*.

Rules of thumb:

  • Never put a real key/password in the repo, a Dockerfile, a workflow file, or a log.
  • Client-side code must never contain sk_… keys — merchant keys are resolved server-side by the Cloudflare Worker.
  • If you must hand a secret to someone, use a secure channel, not chat/email plaintext.

A December 2025 incident (SECURITY_INCIDENT_RESPONSE.md) was caused by committing keys to git; that runbook documents the rotation + git-history purge procedure. Those keys were rotated.


6. Known issues / hardening backlog​

Confirmed by reading the code (not yet fixed — track separately):

  1. Passwords are stored and compared in plaintext. Fixed by #400. auth-api calls verifyPassword from cloud-run/auth-api/lib/password.js, which uses bcryptjs and lazily upgrades any remaining plaintext hash on the user's next successful login. The // in production, use bcrypt! comment this item cited no longer exists in index.js. A forced reset of pre-#400 passwords has still not been done — see §9.6.
  2. ADMIN_ELEVATION_SECRET has a hardcoded dev default (auth-api falls back to finmatch-admin-elevation-dev-only if the env var is unset). Confirm it is set explicitly in production.
  3. platform/AUTH_SETUP_GUIDE.md is outdated — it lists example plaintext passwords and a pk_test_ partner-key format that no longer matches the real sk_ partner-api keys. Correct or archive it.
  4. SECURITY_INCIDENT_RESPONSE.md still contains the (rotated) exposed keys inline. Fine as history; do not copy those values elsewhere.

Suggested improvements: hash passwords (bcrypt), move Cloud Run env secrets into Secret Manager, add MFA for admin, and add login rate-limiting + audit logging.


7. Local-machine risk assessment​

Question: if a laptop is stolen or dies, could we lose access to critical security information?

Availability (loss of access): LOW. Every production secret has an authoritative home in a cloud store (GCS, Secret Manager, Cloud Run env, Cloudflare, or GitHub) — none depends on a single laptop. The realistic loss risk is only:

  • Unpushed git work — recoverable only if pushed. Run git status && git push in every worktree before wiping a machine.
  • A secret whose only copy was pasted somewhere local — rare, but if you ever generated a value (e.g. an HMAC secret) and only stored it in one local .env, back it up into a password manager / Secret Manager.

Confidentiality (theft): this is the real risk. A stolen unlocked laptop can expose live credentials that grant production access. Check for and protect:

  • GCP service-account key JSON downloaded locally (the same kind of key as GCS_CREDENTIALS) — grants deploy/write access to GCP. Prefer short-lived gcloud auth login / Workload Identity over long-lived downloaded keys.
  • Local gcloud / gsutil credentials (~/.config/gcloud).
  • Local .env files holding Cloud Run env secrets (SendGrid, HMAC, internal secrets).
  • Git-ignored local copies of partner-api-keys.json, *.live.json, *credential*, *secret* you may have edited before uploading to GCS.
  • Browser-saved passwords for the admin dashboard, Cloudflare, GCP console, GitHub, Stripe, SendGrid.

Recommended hygiene:

  • Enable full-disk encryption (FileVault / BitLocker) and a strong login password.
  • Keep the authoritative copy of every master secret in a password manager or Secret Manager, so the laptop only ever holds a working copy.
  • If a laptop is lost: rotate the GCP SA key (GCS_CREDENTIALS), any Cloudflare tokens, and revoke/rotate any keys that may have had local copies; change saved passwords.
  • Moving to Cloud Agents means you can stop keeping most creds locally — the dev environment runs on public read data and needs no secrets (privileged writes go through the deployed services / CI).

8. Source documents (authoritative per topic)​

TopicSource
Auth / loginsplatform/AUTH_SETUP_GUIDE.md (needs updating — see §6)
Merchant users + FA keysadmin-docs/docs/merchants/merchant-users-and-finance-assistant-keys.md
Cloudflare proxy + KVcloudflare/finance-assistant-proxy/README.md
Partner API authdeveloper-docs/docs/partner-api/authentication.md
Sandbox tokensadmin-docs/docs/merchants/sandbox-preview.md, architecture skill §7.1/§7.2
Stripe secretsSTRIPE_INTEGRATION_GUIDE.md, cloud-run/stripe-api/index.js
Signingcloud-run/*/lib/referral_signature.js, FA_REFERENCE_PLAN.md
Incident historySECURITY_INCIDENT_RESPONSE.md
Architecture / protected files.cursor/skills/finmatch-architecture/SKILL.md, admin-docs/docs/architecture/source-of-truth.md

9. Public-bucket exposure — remediation runbook (IN PROGRESS, opened Aug 2026)​

9.1 Severity & findings​

Two production GCS buckets are publicly readable over the internet (bucket-level allUsers / allAuthenticatedUsers IAM grants, so every object is public):

BucketExposed contentSeverity
gs://finmatch-sharedOriginal finding (Aug 2026): auth-credentials.json (passwords + keys), partner-api-keys.json, merchants.json, merchant-router.json, cors.json, sandbox-tokens.json, stripe-customers.json, users.json, users-enhanced.json, lender-originated-quotes.json, backups. Now: all return anonymous 404 except stripe-customers.json, users.json and users-enhanced.json — see §9.10.Critical as found; see §9.10 for what is still live
gs://finmatch-admincontacts/ and feedback/ data have moved to finmatch-secrets. Listing is off (anonymous list 401). Remaining confirmed anonymous GET: monitoring/merchant-config-audit/latest.json (Phase 1.5). stripe/ and merchants/history/ fail-closed because listing is off.High — leftover Phase 1.5 GET

Confirmed by anonymous HTTP 200 (headers only, no contents downloaded), e.g. https://storage.googleapis.com/finmatch-shared/auth-credentials.json.

Key facts that shape the fix:

  • No access logging on finmatch-shared (logging = null) and Cloud Audit data-access logs are effectively off → we cannot tell who downloaded the files → treat every exposed secret as compromised and rotate.
  • finmatch-shared has Uniform Bucket-Level Access OFF; public is granted at bucket IAM level, so you cannot exempt individual objects — removing public affects the whole bucket.
  • Backend services read these files as the Compute service account 238644427841-compute@developer.gserviceaccount.com, which has its own IAM read grant — so removing public access does not break the services.
  • Shoppers' browsers fetch only runtime files from finmatch-shared (finmatch-rate-card.json, finance-formulas.js, finmatch-finance-formulas.js, build-info.json, configs/lender-styling.json, configs/lender-overrides.json, configs/analytics-capture-config.json, analytics/, css/, images/, js/, scripts/). They do not fetch any credential file (the client-side merchant-router.json fetch was removed Dec 2025; getMerchantConfig reads router/config server-side).

9.2 Chosen strategy​

Move the private files out of the public buckets into a private bucket, leaving the public buckets untouched for runtime serving (zero merchant-serving risk). This avoids removing the bucket-wide public grant (which would break live storefronts) and sidesteps per-object ACL fragility.

Private destination bucket (already created): gs://finmatch-secrets (europe-west2, Uniform Bucket-Level Access ON, Public Access Prevention ENFORCED, 238644427841-compute@… granted roles/storage.objectAdmin, no public members).

Cutover is per file, using an env-var bucket override scoped to that specific file (services read a mix of public + secret files from finmatch-shared, so the override must be per-file, not per-service; e.g. PARTNER_API_KEYS_BUCKET). The default value stays finmatch-shared, so merging the code changes nothing until the env var is set at deploy — and rollback is just flipping the env var back.

9.3 Service → secret-file map (single-writer)​

Confirm with rg '<filename>' cloud-run cloud-functions before each cutover.

FileWriter (single)Readers
partner-api-keys.jsonmanual/adminpartner-api
stripe-customers.jsonstripe-apistripe-api
users.json, users-enhanced.jsonusers-apiusers-api
lender-originated-quotes.jsoncredit-products-apicredit-products-api
auth-credentials.jsonauth-apiauth-api, finance-assistant
sandbox-tokens.jsonauth-apiauth-api, getMerchantConfig
merchants.jsonmerchant-apimerchant-api, auth-api, finance-assistant, partner-api
merchant-router.jsonmerchant-apimerchant-api, partner-api, getMerchantConfig (live SDK hits getmerchantconfig-staging)
cors.jsonmerchant-apimerchant-api, finance-assistant
contacts/contacts.jsoncontacts-apicontacts-api only (merchant-api and finance-assistant call it with x-internal-auth; admin JS goes through auth-api /admin/contacts; profiles store contactIds only)
feedback/notes.json + feedback/images/auth-apiauth-api only (admin JS uses /feedback/*; API already 401)

9.4 Migration order (safest → hardest)​

  1. partner-api-keys.json — one reader, rarely written. First.
  2. stripe-customers.json; users.json + users-enhanced.json; lender-originated-quotes.json — each a single service.
  3. auth-credentials.json + sandbox-tokens.json — small coordinated group (auth-api + finance-assistant + getMerchantConfig).
  4. merchants.json + merchant-router.json + cors.json — read by almost everything; big coordinated cutover, do last in a quiet window.

9.5 Per-file cutover procedure​

For each file (or coordinated group):

  1. Code: add an env-var bucket override for that file (default finmatch-shared); open PR to finmatch-shared, review, merge. (No behaviour change until step 4.)
  2. Copy: gcloud storage cp gs://finmatch-shared/<file> gs://finmatch-secrets/<file> (immediately before cutover to minimise staleness on written files).
  3. Cut over: redeploy the writer and all readers of that file with the env var pointing at gs://finmatch-secrets.
  4. Verify: exercise the feature (e.g. partner-api quote; admin login; storefront config) and confirm it reads/writes the new bucket.
  5. Delete: once verified, gcloud storage rm gs://finmatch-shared/<file> to close the exposure for that file. (Rollback before this step = flip env var back.)

9.6 Rotate after closing (mandatory — assume compromised)​

Only after the file is private (rotating into a still-public file is pointless):

  • Partner API keys (partner-api-keys.json) — rotate all keys that existed before File 1 close (the 15 live partners in the 2026-08-17 inventory). Mark old revoked (see SECURITY_INCIDENT_RESPONSE.md). Do not rotate partners created after the public object was deleted — they were never exposed. First of these: PROXIMUS SOLUTIONS LIMITED P000116 (2026-08-17). See cloud-run/partner-api/ADD_NEW_PARTNER.md “Post-containment partners”.
  • All user passwords (auth-credentials.json) — reset; bcrypt hashing lands via the auth-api PR so new passwords are stored hashed.
  • Merchant Finance Assistant keys — rotate in Admin; auth-api writes the Cloudflare KV mirror. Do not paste keys into KV by hand.
  • stripe-customers.json holds PII (not keys — Stripe keys are in Secret Manager); assess exposure.

9.7 finmatch-admin GDPR programme (Phase 0 plan, 2026-08-18)​

Do not make gs://finmatch-admin private. It is the static website for admin.finmatch.io (HTML / JS / CSS / docs). The load balancer serves that bucket to browsers. Login is a JS check (localStorage.isLoggedIn) after the page has already downloaded. The password does not protect bucket objects at https://storage.googleapis.com/finmatch-admin/….

Data model (already correct). Merchant profiles store contactIds.primary / contactIds.technical (c_… ids) only. Name, email, and phone live in contacts-api. Do not put contact PII back onto merchants.json.

Target shape (architecture + security-by-design).

StoreHolds
gs://finmatch-adminDeploy artefacts only (dashboard + /docs/)
gs://finmatch-secretsPII / ops data the Compute SA reads
Cloud RunThe only way a browser gets that data, and only after a real login

Do not

  • Privatise the whole admin bucket (breaks admin.finmatch.io)
  • Put INTERNAL_API_SECRET (or any service secret) in admin JS
  • Mix a file move and an API lock in the same PR
  • Rotate partner keys, passwords, or FA / Cloudflare keys in this programme
  • Run two agents on this programme at once

How to execute. One checkpoint, one PR, same per-file playbook as File 1–4 (code env-var default stays on the public bucket → copy to finmatch-secrets → flip env on the writer → verify → delete the public object). Cloud Shell: one short command at a time. Confirm public close with headers-only curl (-o /dev/null). Do not cat PII JSON.

Phase 1 — hide the files (GCS)​

Confirmed 2026-08-18 (headers / listing only; no PII bodies downloaded). finmatch-admin was anonymously listable then. Listing is now off (anonymous list 401 on finmatch-shared and finmatch-admin, 2026-09-20). See §9.11.

#WhatWriterNotesPublic check (2026-08-18)
1.1contacts/contacts.jsoncontacts-apiLive in gs://finmatch-secrets/contacts/contacts.json (CONTACTS_BUCKET). Public copy deleted (anonymous HTTP 404, 2026-08-19). HTTP lock done (Phase 2.1, rev contacts-api-00008-5j4).public 404
1.2feedback/notes.json + feedback/images/…auth-apiLive in gs://finmatch-secrets/feedback/ (FEEDBACK_BUCKET). Public notes + images deleted (anonymous HTTP 404, 2026-08-20). Rev auth-00033-jzg. feedback/index.html stays public (admin page). API already 401.notes 404; images 404
1.3stripe/YYYY-MM-DD/….jsonstripe-apiLive customer snapshots. Legacy stripe-customers.json is already archived in finmatch-secrets.e.g. stripe/2026-08-18/… 200, ~601 KB
1.4merchants/history/FM-….jsonmerchant-apiPer-merchant history; prefix listable.listable
1.5monitoring/**monitoring workflows + merchant-apiDrift / audit JSON. legacy-404-watch retired Aug 2026 (historical monitoring/legacy-404-watch/ snapshots may remain until this prefix is archived). Admin must stop fetching raw storage.googleapis.com URLs (use authenticated merchant-api reads).audit latest 200 (~492 KB); in-bucket monitoring/merchant-config-audit.json 200 (~5.5 MB)
1.6emails/templates.jsonmerchant-apiLower risk; same treatment.200

Phase 1.1 and 1.2 files are private. Phase 2.1 HTTP lock is live. Remaining file groups are 1.3–1.6. Do not mix a file move with an API lock.

Phase 2 — lock the APIs (the actual control)​

Closing a GCS object does not close an unauthenticated Cloud Run path. Do not start Phase 2 in the same PR as a file move.

#WhatEvidence (2026-08-18)
2.1contacts-api list/create/update/delete. Live behind x-internal-auth (INTERNAL_API_SECRET). Admin JS uses auth-api /admin/contacts* with x-username (requireAdmin). merchant-api and finance-assistant send the service secret. Rev contacts-api-00008-5j4 (2026-08-19). /health and /_version stay open.GET /api/contacts 401
2.2merchant-api GET /api/merchants200, ~274 KB
2.3merchant-api /admin/merchant-config-audit, /admin/merchant-config-integrity, /admin/cors/summary, /admin/email-templatesall 200, no auth
2.4partner-api /admin/merchant/:idcode TODO: add admin auth

auth-api feedback routes are already 401.

Phase 3 — listing, logs, erasure​

#What
3.1Turn off anonymous list on finmatch-admin (this is how stripe/, feedback/images/, and merchants/history/ were found). Done 2026-09-20: anonymous list is HTTP 401 on finmatch-admin and finmatch-shared.
3.2GCS access logs + data-access audit logs on finmatch-admin and finmatch-secrets.
3.3Merchant delete must also delete contacts (erasure gap; documented in admin/js/merchant-crud.js).
3.4Remove leftover merchant.contacts.*.email fallback in merchant-api once no profile still has inline emails.

Staffing​

  • Phase 0 (this section) = docs only.
  • Phase 1.1 cutover = done 2026-08-19 (contacts-api-00005-8gz; public HTTP 404).
  • Phase 2.1 cutover = done 2026-08-19 (contacts-api-00008-5j4; anonymous GET /api/contacts 401).
  • Phase 1.2 cutover = done 2026-08-20 (auth-00033-jzg; public notes/images HTTP 404). Next checkpoint is 1.3 or 2.2, not both.
  • Rotation (9.6) stays paused until this programme says otherwise. PROXIMUS SOLUTIONS LIMITED P000116 is excluded from any later key rotation.

9.8 Minor tidy-ups (non-blocking)​

  • finmatch-finance-mkt_cloudbuild (US multi-region): Cloud Build staging; not public, but source tarballs leave the EU. Add a lifecycle rule to expire old objects; optionally regionalise builds to europe-west2. Do not delete it.
  • Prune finmatch-shared/backups/ and merchants.json.backup-stripe-normalize after confirming they're not needed.

9.10 Regression: CI re-published three private files for ten days​

Found 27 Aug 2026. Not a documentation error — a live exposure.

File group 2 was recorded as closed on 2026-08-17: stripe-customers.json, users.json and users-enhanced.json archived to finmatch-secrets, public copies deleted, verified 404. That verification was correct at the time.

All three were publicly downloadable again on 27 Aug, with a Last-Modified of that morning:

stripe-customers.json   HTTP 200   52,255 B   Thu, 27 Aug 2026 10:41:35 GMT
users.json HTTP 200 27,330 B Thu, 27 Aug 2026 10:41:35 GMT
users-enhanced.json HTTP 200 2,808 B Thu, 27 Aug 2026 10:41:34 GMT

Contents, by field name only:

FileRecordsFields
stripe-customers.json100address, email, name, phone, subscriptionPlan, subscriptionStatus
users.json62finmatchId, merchantContact, technicalContact
users-enhanced.json4username, password_hash, role, permissions

Cause. The remediation deleted the bucket objects but not the git copies under finmatch-shared/. The Sync finmatch-shared step in sync-to-gcs.yml rsyncs that directory to the public bucket on every run, and its -x exclude list did not name these three. Every sync since 17 Aug silently re-uploaded them. A sync ran at 10:39 on 27 Aug; the objects are stamped 10:41.

cors.json is also git-tracked and was in the exclude list — and stayed 404 throughout. The mechanism that should have protected these three was known, proven, and simply not applied to them.

Fix. The three names were added to the exclude list, with a comment above the step explaining that the list is load-bearing. Deleting a bucket object by hand is not sufficient while a git copy exists: removing the object and adding the exclude have to happen together.

Still to do:

  • delete the public objects again (the exclude stops republication, it does not remove what is already there)
  • decide whether the git copies should exist at all — no live service reads them (merchant-api calls stripe-customers.json a legacy file; only users-api reads users.json, and it has no deploy workflow)
  • treat the exposure window as 17–27 Aug 2026 for any rotation decision, not the original pre-containment window

Wider lesson for this programme. Every "public copy deleted, verified 404" row in §9.9 was verified at a point in time. Any file with a git copy under finmatch-shared/ can come back. The remaining rows should be re-checked against a current anonymous curl, not against the checklist.

9.11 Exposure sweep, 27 Aug 2026​

A full anonymous sweep of both public buckets, after §9.10. Every figure below was taken with an unauthenticated curl; contents were inspected by field name only.

On 27 Aug both buckets were anonymously listable, so nothing here needed guessing:

BucketObjects enumerable by anyone
gs://finmatch-shared4,974
gs://finmatch-admin6,919

Personal data that was anonymously readable on 27 Aug (listing-on snapshot; not the current listing-off state):

Object / prefixSizeContents (field names)Writer
finmatch-shared/stripe-customers.json52 KB100 records: address, email, name, phonenone — git copy, see §9.10
finmatch-shared/users.json27 KB62 records: merchantContact, technicalContactnone — git copy
finmatch-shared/users-enhanced.json2.8 KB4 accounts: username, password_hash, rolenone — git copy
finmatch-admin/stripe/<date>/*.json602 KB100 records: address, email, business_name, individual_name, balance, delinquent, default_sourcestripe-api — bucketName hardcoded to finmatch-admin (index.js:55)
finmatch-admin/merchants/history/*.json146 objects, 6.6 MBchange events: actor_name, actor_id, merchant_id, endpoint, detailsmerchant-api — adminBucketName hardcoded (index.js:32)
finmatch-admin/monitoring/merchant-config-audit/**5,652 objects, 698 MBaudit events: actor_name, actor_id, merchant_id, endpointmerchant-config-monitoring.yml, every 30 minutes

The Stripe snapshot on the admin bucket carries more fields than the finmatch-shared copy that triggered §9.10, including financial status.

Root causes, and why the earlier file moves worked where these do not. Every file group that was successfully moved got a *_BUCKET environment override, so the destination became configuration. These three writers hardcode the bucket name, so there is nothing to flip:

  • cloud-run/stripe-api/index.js:55 — const bucketName = 'finmatch-admin';
  • cloud-run/merchant-api/index.js:32 — const adminBucketName = 'finmatch-admin';
  • .github/workflows/merchant-config-monitoring.yml:39-42 — gsutil cp … gs://finmatch-admin/monitoring/…

Anonymous listing is the multiplier. With listing on, every object above is discoverable without guessing a filename. Turning it off does not privatise the buckets: change the allUsers binding from roles/storage.objectViewer to roles/storage.legacyObjectReader, which grants storage.objects.get but not storage.objects.list. The storefront and the admin site keep working because they request known paths.

Continuous check. tools/check-public-exposure.sh asserts all of the above anonymously, and runs from .github/workflows/public-exposure-check.yml twice a day and on any change to the sync exclude list or to finmatch-shared/. It also asserts that objects which must stay public still are, so a privatisation that would break the storefront fails the same check. It needs no credentials — anonymous is the perspective that matters. Canaries are only paths a writer actually uses (latest.json, not history.json). Dated stripe/YYYY-MM-DD/<epoch-ms>.json names cannot be guessed; when listing is denied that prefix fails closed. Merchant history ids are read at runtime from the public merchant-api list and are not stored in this repo.

Current (2026-09-20). Anonymous listing of finmatch-shared and finmatch-admin is HTTP 401 (listable on 27 Aug). stripe/ and merchants/history/ fail-closed because listing is off. Remaining confirmed anonymous GET: finmatch-admin/monitoring/merchant-config-audit/latest.json (Phase 1.5). Public Exposure Check cron stays red on that leftover. Do not paste object bodies, merchant ids, or a download recipe here.

9.9 Progress checklist​

  • Confirm exposure (both buckets) and access-log gap
  • Classify public-runtime vs private files
  • Create private bucket finmatch-secrets (private, PAP on, compute SA granted)
  • File 1 — partner-api-keys.json: code override PR → copy → deploy → verify → delete
    • Code: PARTNER_API_KEYS_BUCKET env-var override (PR #403)
    • Copy / cut over / verify / delete public copy (2026-08-17)
    • partner-api-deploy.yml pins PARTNER_API_KEYS_BUCKET=finmatch-secrets
    • Rotate the 15 pre-containment live partner keys; PROXIMUS SOLUTIONS LIMITED P000116 is excluded
  • File group 2 — stripe-customers / users / lender-originated-quotes
    • stripe-customers.json leftover: archived to finmatch-secrets, public copy deleted (anonymous HTTP 404, 2026-08-17)
    • users.json leftover: no live users-api service; archived to finmatch-secrets, public copy deleted (2026-08-17)
    • users-enhanced.json leftover: archived to finmatch-secrets, public copy deleted (2026-08-17)
    • lender-originated-quotes.json: cut over to finmatch-secrets (rev credit-products-api-00278-gzg, 30 quotes, public HTTP 404)
  • File group 3 — auth-credentials / sandbox-tokens
    • Code: AUTH_CREDENTIALS_BUCKET + SANDBOX_TOKENS_BUCKET env-var overrides (PR #411)
    • Copy / cut over writer + readers / verify login / delete public copies (anonymous HTTP 404, 2026-08-18)
    • Live storefront getMerchantConfig is getmerchantconfig-staging (SDK URL); both that service and prod getmerchantconfig now pin SANDBOX_TOKENS_BUCKET=finmatch-secrets (no CI workflow — keep the env var on the next source deploy)
    • auth-api-deploy.yml pins both env vars; finance-assistant-deploy.yml pins AUTH_CREDENTIALS_BUCKET
  • File group 4 — merchants / merchant-router / cors (quiet window)
    • Code: MERCHANTS_JSON_BUCKET + MERCHANT_ROUTER_BUCKET + CORS_JSON_BUCKET env-var overrides (PR #413)
    • Copy / cut over writer + readers / verify admin + storefront / delete public copies (anonymous HTTP 404, 2026-08-18)
    • Live storefront getMerchantConfig is getmerchantconfig-staging (SDK URL); both that service and prod getmerchantconfig now pin MERCHANT_ROUTER_BUCKET=finmatch-secrets (no CI workflow — keep SANDBOX_TOKENS_BUCKET and MERCHANT_ROUTER_BUCKET on the next source deploy)
    • Pin deploy YAMLs: merchant-api-deploy.yml (all three file buckets); auth-api-deploy.yml (MERCHANTS_JSON_BUCKET); finance-assistant-deploy.yml (MERCHANTS_JSON_BUCKET + CORS_JSON_BUCKET); partner-api-deploy.yml (MERCHANTS_JSON_BUCKET + MERCHANT_ROUTER_BUCKET)
  • Rotate all exposed credentials (paused — not in the 9.7 GDPR programme)
  • GDPR Phase 0 — plan written (do not privatise finmatch-admin; move data; lock APIs)
  • GDPR Phase 1 — hide public data files on finmatch-admin
    • 1.1 code prep: /_version + --update-env-vars (PR #415)
    • 1.1 cutover: copy / CONTACTS_BUCKET=finmatch-secrets / verify M407051 in incognito / delete public copy (anonymous HTTP 404, 2026-08-19); pin contacts-api-deploy.yml
    • 1.2 code: FEEDBACK_BUCKET override (PR #422)
    • 1.2 cutover: copy / FEEDBACK_BUCKET=finmatch-secrets / verify Feedback page / delete public notes+images (anonymous HTTP 404, 2026-08-20); pin auth-api-deploy.yml
    • 1.3 stripe/ snapshots (stripe-api)
    • 1.4 merchants/history/ (merchant-api)
    • 1.5 monitoring/ (workflows + merchant-api)
    • 1.6 emails/templates.json (merchant-api)
  • GDPR Phase 2 — lock unauthenticated APIs (after 1.1; do not mix with a file-move PR)
    • 2.1 code: auth-api proxy + callers send secret (PR #418)
    • 2.1 flip: INTERNAL_API_SECRET on contacts-api (contacts-api-00008-5j4); anonymous GET 401; M407051 save verified; pin contacts-api-deploy.yml
    • 2.2 merchant-api GET /api/merchants
    • 2.3 merchant-api open /admin/* GETs
    • 2.4 partner-api /admin/merchant/:id
  • GDPR Phase 3 — listing + logs + erasure
    • 3.1 Disable anonymous list on finmatch-admin (and finmatch-shared; anonymous list HTTP 401, 2026-09-20)
    • 3.2 Access / audit logs on finmatch-admin and finmatch-secrets
    • 3.3 Merchant delete also deletes contacts
    • 3.4 Remove leftover merchant.contacts.*.email fallback