Migration & Seeding Guide

Reference for all data seeding and migration scripts in the GrantMaster project. WARNING: Migration scripts modify live data. Always back up before running in production.

Overview

Seed scripts populate test/development data in emulators, staging, or production
Migration scripts transform existing data (schema changes, collection renames, backfills)
In-app seed helpers provide UI-triggered seeding for specific domains (audit logs, compliance, etc.)
Extension migrations run automatically via ExtensionMigrationService when an extension is upgraded

All production-targeting scripts follow the Production Guardrail Pattern (see scripts/README.md):

Must accept --confirm-project=grantlog
Must support --dry-run and --yes flags
Must implement requireProjectConfirmation() and requireInteractiveConfirmation()
Enforced by npm run check:script-safety

Quick Reference

Development Seeding

Script	Command	Purpose	Safe for Prod?
`seedEmulators.ts`	`npm run seed:emulators`	Seed all emulator data (orgs, users, projects, grants, expenses, widgets, notifications, etc.)	N/A (emulators only)
`seedJournals.ts`	`npm run seed:journals`	Seed ~525 journal entries across 3 tenants	N/A (emulators only)
`seedContacts.mjs`	`npm run seed:contacts`	Seed contacts for all relationship types across 3 orgs	N/A (emulators only)
`seedWidgets.ts`	`npm run seed:widgets`	Seed widget definitions and assignments to emulators	N/A (emulators only)
`seedWidgets.ts clear`	`npm run clear:widgets`	Clear all widget data from emulators	N/A (emulators only)
`seed-widget-library.ts`	`npm run seed:widget-library`	Seed 23 expanded widget definitions across 6 analytical domains	N/A (emulators only)
`seed-widget-assignments.ts`	`npm run seed:widget-assignments`	Seed all widget definitions and role assignments	N/A (emulators only)
`seed-enhancement-widgets.ts`	—	Seed 8 dashboard enhancement widgets	N/A (emulators only)
`seedLibrary.ts`	`npm run seed:library`	Seed document library with sample folders and documents	N/A (emulators only)
`seedCompliancePolicys.ts`	—	Seed compliance rules from mock data	N/A (emulators only)
`seed-rvo-grants.ts`	—	Fetch RVO.nl Open Data API and write grants to emulators	N/A (emulators only)
`seed-grant-source-configs.ts`	`npm run seed:grant-sources`	Seed grant source configurations for Dutch NGO ingestion pipeline	Configurable
`seedPlatformPolicys.ts`	`npm run seed:platform-policies`	Seed SuperAdmin-managed global platform rules	Configurable

Staging Seeding

Script	Command	Purpose
`seedStaging.ts`	`npm run seed:staging`	Seed staging database with test data
`clearStaging.ts`	`npm run clear:staging`	Clear all staging data
—	`npm run reseed:staging`	Clear + re-seed staging (runs both above)
`seedCompliancePolicysStaging.ts`	`npm run seed:compliance-policies:staging`	Seed compliance rules to staging
`seedWidgetsStaging.ts`	`npm run seed:widgets:staging`	Seed widgets to staging
`seed-widget-assignments.ts`	`npm run seed:widget-assignments:staging`	Seed widget assignments to staging
`seedPlatformPolicys.ts`	`npm run seed:platform-policies:staging`	Seed platform policies to staging

Production Seeding

Script	Command	Purpose	Guardrails
`seedCompliancePolicysProduction.ts`	`npm run seed:compliance-policies:production`	Seed compliance rules	Guard + confirmation
`seedWidgetsProduction.ts`	`npm run seed:widgets:production`	Seed widget definitions	Guard + confirmation
`seed-widget-assignments.ts`	`npm run seed:widget-assignments:production`	Seed widget assignments	Guard + confirmation
`seedPlatformPolicys.ts`	`npm run seed:platform-policies:production`	Seed platform policies	Guard + confirmation
`addProductionUsers.ts`	`npm run prod:add-users`	Add specific users to production	Guard + `--dry-run`

Data Migrations

Script	Command	Purpose	Dry Run?	Rollback?
`migrate-to-rbac.ts`	`npm run migrate:rbac`	RBAC: set custom claims, create dual-ID user docs, add org status	Yes (default)	Partial (additive)
`migrate-subscription-model.ts`	`npm run migrate:seats`	Convert to seat-based subscription model with Stripe sync	Yes (`--dry-run`)	Manual
`migrate-remove-grantor-from-contacts.ts`	—	Remove GRANTOR relationship from contacts	Yes (default)	Manual
`migrateGrantorsToContacts.mjs`	`npm run migrate:donors-to-contacts`	Convert donor records to contact records	No	Manual
`backfill-usage-metrics.ts`	—	Calculate and update usageMetrics for all orgs	Yes (`--dry-run`)	Manual
`normalizeSystemRoles.ts`	—	Normalize inconsistent role casing (Admin -> admin, etc.)	Yes (default, `--apply` to execute)	Manual
`syncRolePermissions.ts`	`npm run sync:role-permissions`	Sync Firestore roles with permission matrix	Guard + confirmation	Manual

Collection Migrations (scripts/migrations/)

Script	Command	Purpose	Dry Run?
`migrate-timesheets-to-journals.ts`	`npm run migrate:journals`	Copy `timesheets` to `journals` collection	Yes (default)
`verify-timesheets-to-journals.ts`	—	Verify migration completeness (diff source/target)	Read-only
`migrate-terminology-collections.ts`	`npm run migrate:terminology`	Rename collections: employees->people, retrospectives->journalSubmissions, grantPipeline->grantTracker, deadlines->milestones	Yes (default)
`moveAnalyticsWidgetsToDashboard.ts`	`npm run migrate:analytics-widgets`	Move analytics widgets to dashboard category	Confirmation prompt

Code Migrations (non-data)

Script	Purpose
`migrate-page-layout.cjs`	Codemod: migrate PageHeader/PageLayout imports and usage patterns
`migrate-domain-components.sh`	Shell script: migrate domain component imports
`update-component-imports.sh`	Shell script: update component import paths

Dashboard Management

Script	Command	Purpose
`createDefaultDashboardTemplates.ts`	`npm run dashboard:templates:generate`	Generate role-based default dashboard widget layouts
`applyDefaultDashboardTemplates.ts`	`npm run dashboard:templates:apply`	Apply dashboard templates to user profiles (`--migrate` for all, `--user-id` for one)

Seed Scripts (Detail)

seedEmulators.ts

Purpose: Primary development seed script. Populates local Firebase emulators with a comprehensive dataset spanning all features. Command: npm run seed:emulators What it creates:

3 organizations (Alpha, Beta, Gamma) with full multi-tenant isolation
Users/team members with roles and auth accounts
Projects, grants (pipeline, applications, active grants)
Expenses, journal entries, journal submissions
Contacts, donors/foundations
Compliance policies, notifications
Widget definitions and assignments
Agent runs, gamification data, M&E indicators
Grantor relationships, portal tokens, reporting deadlines
Workflow rules, approval routing, escalation configs
Module collections, mission configs
RVO grant data

Prerequisites: Firebase emulators must be running (npm run dev:start) Notes: This is the most comprehensive seed script. Other emulator seed scripts (journals, contacts, library) supplement it with domain-specific data.

seedStaging.ts

Purpose: Seeds the staging Firebase database with test data mirroring the emulator seed but targeting a remote environment. Command: npm run seed:staging Prerequisites:

Service account key for grantcontrol-staging project
Must unset emulator env vars (handled by npm script)
Run firebase use staging first (handled by npm script)

Notes: Uses the same mock data sources as the emulator seed. Includes widget priority ordering for dashboard layout.

seedJournals.ts

Purpose: Seeds realistic journal/time tracking entries for demonstration and testing. Command: npm run seed:journals What it creates (per tenant):

Org Alpha: 200 entries, Org Beta: 150, Org Gamma: 175 (~525 total)
Last 90 days, weekdays only
Status distribution: 82% approved, 15% pending, 3% rejected
6 activity types with 10 description variations each
5% audit flags, 10% marked as edited

Prerequisites: Base data must be seeded first (npm run seed:emulators) Notes: Separate from main seed to allow independent re-seeding. See README_JOURNALS.md for full details. Purpose: Seed widget ecosystem data.

seedWidgets.ts: Core widget definitions and assignments (emulator, also clear mode)
seed-widget-library.ts: Expanded library of 23 widgets across Financial, Risk & Compliance, Grant Lifecycle, Strategic Intelligence, Operational Efficiency, and Forecasting/AI domains
seed-widget-assignments.ts: Full widget definitions + role-based assignments; supports emulator, staging, and production via FIREBASE_ENV

Commands:

npm run seed:widgets / npm run clear:widgets
npm run seed:widget-library
npm run seed:widget-assignments (also :staging, :production, :dry-run variants)

seedContacts.mjs

Purpose: Seeds mock contacts for all relationship types (Donors, Partners, Beneficiaries, Government, Vendors, etc.) across all 3 organizations. Command: npm run seed:contacts Prerequisites: Emulators running

seedLibrary.ts

Purpose: Seeds the document library (Document Brain) with sample folders and documents. Command: npm run seed:library Prerequisites: Emulators running

seedCompliancePolicys*.ts

Purpose: Seeds compliance/policy rules. Variant scripts for each environment.

Variant	Target	Auth Method
`seedCompliancePolicys.ts`	Emulators	Admin SDK (emulator)
`seedCompliancePolicysCLI.ts`	Any (via Firebase CLI)	Firebase CLI auth
`seedCompliancePolicysStaging.ts`	Staging	Service account key
`seedCompliancePolicysProduction.ts`	Production	Service account key + confirmation

seedPlatformPolicys.ts

Purpose: Seeds SuperAdmin-managed global platform rules available to all tenants. Supports --env flag for environment targeting. Command: npm run seed:platform-policies (also :staging, :production)

seed-grant-source-configs.ts

Purpose: Seeds Firestore grant source configurations for the Dutch NGO ingestion pipeline (RVO, ZonMw, etc.). Command: npm run seed:grant-sources Options: --dry-run, --project-id <id>, --webhook-base-url <url>

seed-rvo-grants.ts

Purpose: Fetches live data from the RVO.nl Open Data API and writes grant opportunities to Firestore emulator. Prerequisites: Emulators running, network access to RVO API

Migration Scripts (Detail)

migrate-to-rbac.ts

Purpose: Enables full multi-tenant RBAC by setting custom claims on user auth tokens, creating dual-ID user documents, and adding organization status fields. Command:

npm run migrate:rbac (dry run, default)
npm run migrate:rbac -- --run (execute)
npm run migrate:rbac -- --run --deploy-rules (execute + deploy Firestore rules)

Prerequisites: serviceAccountKey.json in project root Safety: Dry run by default. 5-second delay before execution. Creates new documents (additive, does not delete). Rollback: Revert Firestore rules via cp firestore.rules.backup firestore.rules && firebase deploy --only firestore:rules. Custom claims can be cleared via Firebase Console. Side effects: Users must sign out and sign back in for new claims to take effect. See README-RBAC-MIGRATION.md for full details.

migrate-subscription-model.ts

Purpose: Converts organizations from old subscription model to seat-based model with Stripe synchronization. Command:

npm run migrate:seats:dry-run
npm run migrate:seats

Options: --dry-run, --org-id=<id>, --force Prerequisites: .env.local with STRIPE_SECRET_KEY and Firebase credentials Safety: Supports --dry-run for preview. Updates both Firestore and Stripe.

migrate-timesheets-to-journals.ts

Purpose: One-time collection rename. Copies legacy timesheets documents into canonical journals collection preserving document IDs and payloads. Command: npm run migrate:journals (dry run by default) Options: --dry-run (default), --execute, --org=<id>, --project=<id>, --limit=<n> Verification: Run verify-timesheets-to-journals.ts after migration to diff source/target collections.

migrate-terminology-collections.ts

Purpose: Renames collections to match updated terminology:

employees -> people
retrospectives -> journalSubmissions
grantPipeline -> grantTracker (including subcollections: tasks, documents, activities)
deadlines -> milestones

Command: npm run migrate:terminology Options: --dry-run (default), --execute Safety: Dry run by default. Copies documents (does not delete source).

migrate-remove-grantor-from-contacts.ts

Purpose: Removes the “Grantor” relationship group from contact records, since grantors are now managed exclusively in the Grantors module. Options: --environment=staging|production, --execute (dry run by default) Safety: Dry run by default. Backs up affected contacts before modification.

moveAnalyticsWidgetsToDashboard.ts

Purpose: Updates widget category from analytics to dashboard in Firestore. Consolidates all current/retrospective widgets on the Dashboard page. Command: npm run migrate:analytics-widgets Safety: Interactive confirmation prompt. Adds migrationNote field for audit trail.

backfill-usage-metrics.ts

Purpose: One-time backfill of usageMetrics on organization documents (active users, projects, storage, AI generations). Options: --dry-run, --environment=production|staging Safety: Supports dry run. Counts data from related collections and Storage.

normalizeSystemRoles.ts

Purpose: Normalizes inconsistent role casing in user documents (e.g., Admin -> admin, Super Admin -> superadmin). Options: --apply (default is read-only), --collections=employees,people, --confirm-project=<id> Safety: Read-only by default. Uses production guardrails.

syncRolePermissions.ts

Purpose: Syncs Firestore role documents with the canonical permission matrix defined in code. Command: npm run sync:role-permissions Safety: Uses production guardrails (requireProjectConfirmation, requireInteractiveConfirmation).

Data Reset Scripts

resetProduction.ts

Purpose: DANGER — Deletes ALL data in the production database and creates a clean state with 1 SuperAdmin, 1 Admin, and 1 default organization. Command: npm run reset:production -- --confirm-project=grantlog Options: --dry-run, --yes Safety: Production guardrails required. Use with extreme caution.

clearStaging.ts

Purpose: Clears all data from the staging Firebase database. Command: npm run clear:staging Prerequisites: Service account key for grantcontrol-staging

In-App Seed Helpers

These are TypeScript modules in src/lib/ that can be called programmatically from the application (typically from admin/settings pages or onboarding flows).

seedDatabase.ts

Location: src/lib/seedDatabase.ts Purpose: Seeds the Firebase database with mock data from the UI. Requires an authenticated user session. Usage: Called programmatically; accepts optional targetOrgId to scope seeding to one organization. What it creates: Organizations, projects, users, journal entries, expenses, notifications, referrals, journal submissions, module collections.

seedAuditLogs.ts

Location: src/lib/seedAuditLogs.ts Purpose: Creates 50 diverse sample audit log entries for testing the audit dashboard. Usage: Import and call seedSampleAuditLogs({ organizationId, userCount?, entriesPerUser? }) from a settings page or test button.

seedComplianceData.ts

Location: src/lib/seedComplianceData.ts Purpose: Generates realistic test data for the compliance monitoring system (summaries, alerts, metrics, trends). Usage: Import and call seedComplianceSummaries(...) etc. from test setup or admin tools.

seedSubscriptionPricing.ts

Location: src/lib/seedSubscriptionPricing.ts Purpose: Seeds default subscription pricing tiers (Free, Starter, Pro, Enterprise). Should be run once by a Super Admin during initial setup. Usage: Call seedSubscriptionPricing(superAdminId).

seedExtensionInstallations.ts

Location: src/lib/seedExtensionInstallations.ts Purpose: Auto-creates Impact module installation records for organizations on Professional or Ultimate tiers. Safe to run multiple times (skips existing installations). Usage: Call seedModuleInstallations(). Idempotent.

Extension Migration System

Extensions use ExtensionMigrationService (src/shared/platform/ExtensionMigrationService.ts) to run data migrations automatically when an extension is upgraded.

How It Works

Each extension declares migrations in its manifest with fromVersion fields
When an extension is loaded, the service compares the installed version (from Firestore ModuleInstallation) against the manifest version
If the manifest version is newer, all migrations between the two versions are executed in semver order

Each migration module exports an execute function with signature:

(context: { extensionId, organizationId, fromVersion, toVersion }) => Promise<void>

On success, the stored version is updated
On failure, the extension is suspended with reason migration_failed

Key Types

interface MigrationContext {
  extensionId: string;
  organizationId: string;
  fromVersion: string;
  toVersion: string;
}

interface MigrationResult {
  success: boolean;
  migrationsRun: number;
  fromVersion: string;
  toVersion: string;
  error?: string;
}

Adding Extension Migrations

Add a migration entry to the extension manifest with fromVersion set to the version being upgraded from
Create a migration module in the extension’s directory
Export a default or named execute function
The service handles ordering, execution, and version bookkeeping

Safety Guidelines

Always run with --dry-run first (if supported) to preview changes before writing
Back up Firestore before running production migrations (use gcloud firestore export)
Run migrations in off-peak hours to minimize user impact
Verify with health checks after migration (npm run health:production)
Keep a rollback plan ready — most migrations are additive (copies, not moves), making rollback straightforward
Use production guardrails — all production scripts must pass --confirm-project=grantlog and interactive confirmation
Check script safety before deploying new scripts: npm run check:script-safety
Never commit service account keys — they are in .gitignore but verify before pushing
Users must re-authenticate after auth claim migrations (sign out + sign in)
Test on staging first — use npm run reseed:staging to get a clean staging environment for testing migrations

Maintenance

Update this guide when:

Adding new seed or migration scripts
Changing script CLI arguments or behavior
Deprecating or archiving scripts (move to scripts/archive/)
Adding new in-app seed helpers in src/lib/
Modifying ExtensionMigrationService contracts

Get Started

Product — Domain Model

Product — Workflows

Product — Features

Product — Extensions

Product — User Guides

Product — Launch & Pricing

Engineering — Architecture

Engineering — Data

Engineering — Frontend

Engineering — Security

Engineering — Testing

Engineering — Guides

Contributing

Marketing

Documentation Index

​Migration & Seeding Guide

​Overview

​Quick Reference

​Development Seeding

​Staging Seeding

​Production Seeding

​Data Migrations

​Collection Migrations (scripts/migrations/)

​Code Migrations (non-data)

​Dashboard Management

​Seed Scripts (Detail)

​seedEmulators.ts

​seedStaging.ts

​seedJournals.ts

​seedWidgets.ts / seed-widget-library.ts / seed-widget-assignments.ts

​seedContacts.mjs

​seedLibrary.ts

​seedCompliancePolicys*.ts

​seedPlatformPolicys.ts

​seed-grant-source-configs.ts

​seed-rvo-grants.ts

​Migration Scripts (Detail)

​migrate-to-rbac.ts

​migrate-subscription-model.ts

​migrate-timesheets-to-journals.ts

​migrate-terminology-collections.ts

​migrate-remove-grantor-from-contacts.ts

​moveAnalyticsWidgetsToDashboard.ts

​backfill-usage-metrics.ts

​normalizeSystemRoles.ts

​syncRolePermissions.ts

​Data Reset Scripts

​resetProduction.ts

​clearStaging.ts

​In-App Seed Helpers

​seedDatabase.ts

​seedAuditLogs.ts

​seedComplianceData.ts

​seedSubscriptionPricing.ts

​seedExtensionInstallations.ts

​Extension Migration System

​How It Works

​Key Types

​Adding Extension Migrations

​Safety Guidelines

​Maintenance