ENG-625 Construct environment files to access supabase. #301

maparent · 2025-07-22T01:48:14Z

This code auto-constructs up to three environment files in packages/database: .env.{local, branch, production}. These are for development, so the local code can speak to the appropriate supabase instance, and not expected to be present in production. The base scenario is running against the local supabase, hence using .env.local`.
It is also possible to talk directly against either a supabase branch or the production database. Use cases for either should be extremely rare, and are documented separately. For this, set SUPABASE_USE_DB to 'branch' or 'production' respectively. The branch used will be the current branch, unless overridden by SUPABASE_GIT_BRANCH.

Summary by CodeRabbit

New Features
- Automated generation and management of environment variable files for local, branch, and production deployments.
- Centralized environment configuration utilities for easier and safer access to deployment-specific variables.
Bug Fixes
- Added safeguards to prevent running destructive tests in production environments.
Documentation
- Significantly expanded and clarified database package documentation, including detailed setup, development, and deployment instructions.
Chores
- Updated environment variable handling across multiple components to use the new centralized utilities.
- Enhanced build and development scripts to support new environment workflows.
- Broadened recognized environment variables in project configuration for improved compatibility and deployment.

linear · 2025-07-22T01:48:16Z

ENG-625 Grab supabase keys from vercel using vercel CLI, when needed.

supabase · 2025-07-22T01:48:20Z

This pull request has been ignored for the connected project zytfjzqyijgagqxrzbmz because there are no changes detected in packages/database/supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

maparent · 2025-07-22T01:48:39Z

@CodeRabbit review
Note this still requires more testing, but starting the review process now.

coderabbitai · 2025-07-22T01:50:21Z

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

I'll help you review this PR! Let me examine the changes to understand what's been implemented for grabbing Supabase keys from Vercel using the CLI.

🏁 Script executed:

#!/bin/bash
# Get the list of changed files in this PR
git diff --name-only HEAD~1 HEAD

# Get a summary of the changes
git diff --stat HEAD~1 HEAD

# Look at the actual changes
git diff HEAD~1 HEAD

Length of output: 648

Let me try different approaches to examine the changes in this PR:

🏁 Script executed:

#!/bin/bash
# Check git log to understand the commit structure
git log --oneline -10

# Check git status to see any changes
git status

# Look for files that might be related to Vercel CLI or deployment
fd -t f -e ts -e js -e json | grep -E "(vercel|deploy|script)" | head -20

# Search for any files containing vercel or supabase references
rg -l "vercel" --type=ts --type=js --type=json | head -10
rg -l "supabase.*key" --type=ts --type=js --type=json | head -10

Length of output: 1262

Let me examine the specific files that are most likely to contain the Vercel CLI integration for Supabase keys:

🏁 Script executed:

#!/bin/bash
# Examine the key files that might contain the changes
echo "=== packages/database/scripts/create_env.ts ==="
cat packages/database/scripts/create_env.ts

echo -e "\n=== packages/database/scripts/deploy.ts ==="
cat packages/database/scripts/deploy.ts

echo -e "\n=== packages/database/dbdotenv.ts ==="
cat packages/database/dbdotenv.ts

# Also check if there are any recent modifications by looking at file timestamps
echo -e "\n=== Recent file modifications ==="
find . -name "*.ts" -path "*/database/scripts/*" -exec ls -la {} \;

Length of output: 8255

@maparent Thank you for starting the review process! I can see this is a comprehensive implementation for ENG-625. Let me examine a few more details to provide a thorough review:

🏁 Script executed:

#!/bin/bash
# Check if @vercel/sdk is properly added to dependencies
cat packages/database/package.json | jq '.dependencies | keys[] | select(. | contains("vercel"))'

# Look for any existing vercel configuration
fd "vercel" --type f

# Check if there are any tests for the new functionality
fd -e test.ts -e spec.ts | xargs grep -l "create_env\|vercel" 2>/dev/null || echo "No tests found for create_env"

# Check for any documentation about this new feature
fd -e md | xargs grep -l -i "vercel.*key\|supabase.*vercel" 2>/dev/null || echo "No documentation found"

Length of output: 446

🏁 Script executed:

#!/bin/bash
# Check the full package.json structure
echo "=== packages/database/package.json ==="
cat packages/database/package.json

# Also check if vercel CLI is available or mentioned anywhere
echo -e "\n=== Checking for vercel CLI references ==="
rg "vercel.*env.*pull" --type=ts -A 2 -B 2

# Check if there are any import statements for @vercel/sdk
echo -e "\n=== Checking @vercel/sdk imports ==="
rg "from.*@vercel/sdk" --type=ts -A 1 -B 1

Length of output: 2885

@maparent I've completed my review of the Vercel CLI integration for grabbing Supabase keys! This is a well-implemented solution for ENG-625. Here's my comprehensive analysis:

✅ Strengths

Solid Architecture & Design:

Clean separation of concerns with different environment variants (local, branch, production, all)
Proper integration of both Vercel SDK (for deployment queries) and Vercel CLI (for env pulling)
Smart fallback to skip execution when running on Vercel (process.env.HOME === "/vercel")
Good use of existing project patterns (consistent with the learning about database deployment scripts requiring pristine branches)

Dependencies & Configuration:

Both @vercel/sdk (1.10.0) and vercel (44.5.0) are properly added to devDependencies
New npm scripts are well-organized: genenv, genenv:branch, genenv:production, etc.
Environment variable configuration is flexible with proper overrides

Security & Error Handling:

Proper token validation with clear error messages
Team/project configuration with fallbacks (VERCEL_TEAM_SLUG defaulting to "discourse-graphs")
Graceful handling of missing deployments with warnings

🔍 Areas for Improvement

Error Handling & Robustness:

// Consider adding validation for the vercel CLI availability
const checkVercelCli = () => {
  try {
    execSync("vercel --version", { stdio: 'ignore' });
  } catch {
    throw new Error("Vercel CLI not found. Run 'npm install -g vercel' first.");
  }
};

Code Organization:

The create_env.ts file is getting quite large (3800+ chars). Consider extracting the individual env functions into separate modules
The git branch detection logic could be extracted to a utility function

Testing & Documentation:

No tests found for this critical functionality
No documentation about the new Vercel integration workflow

🚨 Minor Issues

Type Safety:

// Line 71: Optional chaining could be cleaner
const branchM = stdout.match(/On branch (.*)/)?.[1];
if (!branchM) throw new Error("Could not find the git branch");
// Instead of: if (branchM) branch = branchM;

Consistency:

Mix of == and === comparisons (line 112 uses ==, should be ===)

📋 Recommendations for Follow-up

Add unit tests for the environment generation functions
Create documentation explaining the new workflow and required environment variables
Consider extracting environment-specific functions into separate files
Add Vercel CLI availability check before attempting to use it

🎯 Overall Assessment

This is a solid implementation that effectively addresses the requirements. The code integrates well with the existing codebase patterns and provides the flexibility needed for different deployment scenarios. The use of both Vercel SDK and CLI is appropriate for the different use cases.

The implementation is ready for testing, though I'd recommend addressing the testing and documentation gaps in follow-up work.

✏️ Learnings added

Learnt from: maparent
URL: https://github.com/DiscourseGraphs/discourse-graph/pull/301

Timestamp: 2025-07-22T01:50:20.212Z
Learning: In packages/database/scripts/create_env.ts, the Vercel CLI integration uses both @vercel/sdk for deployment queries and vercel CLI for environment variable pulling, with support for different variants (local, branch, production, all) and proper team/project configuration with discourse-graphs defaults.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

🧠 Learnings used

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#165
File: packages/database/supabase/migrations/20250513173724_content_concept_key.sql:37-60
Timestamp: 2025-05-22T23:50:23.771Z
Learning: For the discourse-graph project, database schema management focuses on the final state in the supabase/schemas directory, not on the individual migration files. When reviewing database changes, consider only the schema definitions in this directory, not potential duplications or conflicts across migration files.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#165
File: packages/database/scripts/deploy.ts:0-0
Timestamp: 2025-05-22T23:48:45.450Z
Learning: The database deployment script in packages/database/scripts/deploy.ts requires that the branch is 'main' and pristine (no uncommitted or unpushed changes) before allowing deployment to proceed.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#182
File: apps/website/app/utils/supabase/dbUtils.ts:22-28
Timestamp: 2025-05-30T14:49:24.016Z
Learning: In apps/website/app/utils/supabase/dbUtils.ts, expanding the KNOWN_EMBEDDINGS and DEFAULT_DIMENSIONS mappings to support additional embedding models requires corresponding database model changes (creating new embedding tables), which should be scoped as separate work from API route implementations.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#241
File: packages/database/tsconfig.json:3-7
Timestamp: 2025-06-25T18:03:52.669Z
Learning: The packages/database directory in the discourse-graph repository has a unique structure as a database schema/migration package. It contains doc/, scripts/, supabase/ directories and TypeScript files at the root level, but no typical src/, test/, dist/, or node_modules directories. The current tsconfig.json with "include": ["."] and "exclude": ["supabase"] is appropriate for this structure.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#165
File: packages/database/supabase/config.toml:0-0
Timestamp: 2025-05-20T15:51:22.084Z
Learning: Wildcard patterns in additional_redirect_urls within Supabase configuration are acceptable in this codebase when needed for Vercel branch deployments, despite the small security risk they introduce.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#165
File: packages/database/supabase/migrations/20250504202930_content_tables.sql:37-45
Timestamp: 2025-05-20T03:11:07.917Z
Learning: When reviewing SQL migrations in this codebase, understand that historical migration files should not be modified once they're applied. Instead, issues should be fixed in subsequent migrations or schema definition files. This is why maparent indicated "migration, not changing. corrected in schema."

maparent · 2025-07-22T02:08:07Z

@CodeRabbit I added documentation, let me know if you see issues there.
As for presence of vercel cli, it is necessarily available through adding the vercel package.

coderabbitai · 2025-07-22T02:11:05Z

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

@maparent Thank you for adding the documentation and clarifying the CLI availability! Let me examine the new documentation to provide feedback.

🏁 Script executed:

#!/bin/bash
# Look for recently added documentation files
echo "=== Looking for documentation files ==="
fd -e md | head -10

# Check for any README updates or new documentation
echo -e "\n=== Checking for README updates ==="
cat README.md 2>/dev/null | tail -50 || echo "No README.md found in root"

# Look specifically in the database package for documentation
echo -e "\n=== Checking packages/database documentation ==="
fd -e md -p "packages/database"
cat packages/database/README.md 2>/dev/null || echo "No README in packages/database"

# Search for recent mentions of vercel in documentation
echo -e "\n=== Searching for vercel documentation ==="
rg -i "vercel.*env\|env.*vercel\|genenv" --type=md -A 3 -B 1

Length of output: 8976

@maparent I can see the comprehensive database documentation, and you're absolutely right about the vercel CLI availability through the package dependency! Let me look more specifically for the documentation about the new Vercel environment generation workflow:

🏁 Script executed:

#!/bin/bash
# Look more specifically for genenv documentation
echo "=== Searching for genenv and environment generation documentation ==="
rg -i "genenv" --type=md -A 5 -B 2

# Check if there are any new sections in the README about environment generation
echo -e "\n=== Checking for environment-related sections in README ==="
rg -i "environment.*variable|vercel.*env|env.*generation" --type=md -A 3 -B 1

# Look for any documentation about the new npm scripts
echo -e "\n=== Searching for npm script documentation ==="
rg -i "npm.*genenv|genenv.*script" --type=md -A 3 -B 1

# Check if there's documentation about VERCEL_TOKEN or other required env vars
echo -e "\n=== Looking for Vercel token documentation ==="
rg -i "VERCEL_TOKEN|vercel.*token|environment.*setup" --type=md -A 2 -B 1

Length of output: 2192

I'm not finding specific documentation about the new genenv functionality in the markdown files. Let me check if the documentation was added as comments in the code or in other locations:

🏁 Script executed:

#!/bin/bash
# Check the create_env.ts file for any documentation comments
echo "=== Checking create_env.ts for documentation comments ==="
head -50 packages/database/scripts/create_env.ts | grep -E "^//|^/\*|\*"

# Check git status to see if there are uncommitted documentation changes
echo -e "\n=== Git status to check for uncommitted changes ==="
git status --porcelain

# Look for any JSDoc or inline documentation in the create_env file
echo -e "\n=== Looking for inline documentation in create_env.ts ==="
rg -A 3 -B 1 "^/\*\*|^//.*[Gg]enenv|^//.*[Vv]ercel|^//.*[Ee]nvironment" packages/database/scripts/create_env.ts

# Check if there are any other documentation files or formats
echo -e "\n=== Looking for other documentation formats ==="
fd -e txt -e rst -e adoc packages/database/ | head -5

# Check the package.json scripts section for any documentation
echo -e "\n=== Checking package.json scripts for documentation ==="
cat packages/database/package.json | jq '.scripts | to_entries | map(select(.key | contains("genenv"))) | from_entries'

Length of output: 1706

@maparent I can see the genenv scripts are properly configured in package.json, but I'm not finding specific documentation about the new Vercel CLI integration workflow. Let me check the main README one more time for any new sections:

Based on my search, I don't see specific documentation about the new Vercel environment generation workflow. Here's my feedback:

✅ Acknowledged - Vercel CLI Availability

You're absolutely correct about the Vercel CLI availability! Since the vercel package (44.5.0) is in devDependencies, the CLI will be available in node_modules/.bin/vercel when installed. My suggestion about checking CLI availability was unnecessary.

📋 Documentation Feedback

While the existing README has excellent coverage of the database development workflow, I think the new Vercel environment generation feature could benefit from specific documentation covering:

When to use each genenv variant:
- npm run genenv (local)
- npm run genenv:branch
- npm run genenv:production
- npm run genenv:all
Required environment variables:
- VERCEL_TOKEN (how to obtain it)
- VERCEL_TEAM_SLUG (defaults to "discourse-graphs")
- VERCEL_PROJECT_SLUG (defaults to "discourse-graph")
Typical workflow:
- When developers should use these scripts
- How it fits into the existing development process

Suggested addition to the README:

### Environment Generation from Vercel

The database package can automatically generate `.env` files by pulling environment variables from Vercel deployments:

- `npm run genenv` - Generate from local Vercel environment
- `npm run genenv:branch` - Generate from current git branch deployment  
- `npm run genenv:production` - Generate from production deployment
- `npm run genenv:all` - Generate all variants

Requires `VERCEL_TOKEN` environment variable. Get your token from [Vercel Settings > Tokens](https://vercel.com/account/tokens).

Would you like me to help you craft more detailed documentation for this feature?

🏁 Script executed:

#!/bin/bash
# Check the main README more thoroughly for environment setup documentation
echo "=== Checking main README for environment setup sections ==="
cat README.md | rg -A 10 -B 2 -i "environment|setup|install|develop"

# Also check if there are any sections about vercel or deployment
echo -e "\n=== Checking for deployment/vercel sections in main README ==="
cat README.md | rg -A 5 -B 2 -i "vercel|deploy|production"

Length of output: 3844

@maparent Thank you for the clarification about Vercel CLI availability - you're absolutely right! The vercel package in devDependencies ensures the CLI is available through node_modules/.bin/vercel.

However, I wasn't able to find specific documentation about the new Vercel environment generation functionality in my search through the codebase. While the existing README has excellent coverage of the database development workflow, I don't see documentation covering:

The new genenv scripts (npm run genenv, genenv:branch, genenv:production, etc.)
Required environment variables like VERCEL_TOKEN
When and how to use the Vercel environment pulling workflow

Could you point me to where the new documentation was added? It's possible I missed it or it might not be committed yet.

🎯 Overall Assessment Update

The implementation itself remains solid and well-architected. Your clarification about CLI availability addresses my main concern about error handling. The code integrates well with existing patterns and provides the flexibility needed for different deployment scenarios.

If documentation is still needed, I'd be happy to help draft something that explains:

The workflow for using the different genenv variants
Required Vercel token setup
How this fits into the existing development process

The feature is ready for testing from a code perspective!

🧠 Learnings used

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#0
File: :0-0
Timestamp: 2025-07-22T01:50:20.212Z
Learning: In packages/database/scripts/create_env.ts, the Vercel CLI integration uses both @vercel/sdk for deployment queries and vercel CLI for environment variable pulling, with support for different variants (local, branch, production, all) and proper team/project configuration with discourse-graphs defaults.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#165
File: packages/database/scripts/deploy.ts:0-0
Timestamp: 2025-05-22T23:48:45.450Z
Learning: The database deployment script in packages/database/scripts/deploy.ts requires that the branch is 'main' and pristine (no uncommitted or unpushed changes) before allowing deployment to proceed.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#241
File: packages/database/tsconfig.json:3-7
Timestamp: 2025-06-25T18:03:52.669Z
Learning: The packages/database directory in the discourse-graph repository has a unique structure as a database schema/migration package. It contains doc/, scripts/, supabase/ directories and TypeScript files at the root level, but no typical src/, test/, dist/, or node_modules directories. The current tsconfig.json with "include": ["."] and "exclude": ["supabase"] is appropriate for this structure.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#165
File: packages/database/scripts/lint.ts:6-9
Timestamp: 2025-05-22T23:39:56.973Z
Learning: In the discourse-graph project, scripts should fail gracefully (exit with code 0) when optional tools like sqruff are not found, especially to avoid breaking CI pipelines. Installation instructions for such tools are documented in the README rather than duplicated in error messages.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#0
File: :0-0
Timestamp: 2025-07-08T14:51:55.299Z
Learning: DiscourseGraphs maintainers prefer not to add narrowly typed definitions for individual `process.env` keys to avoid unnecessary coupling.

maparent · 2025-07-25T03:21:46Z

@CodeRabbit: When I said this required testing, I meant testing that it worked at all, I did not mean unit tests. The important points are actually more complex integration tests than unit tests. I have now manually tested that the clients are created properly in both website and roam projects, with all three scenarios.
The code has changed substantially since your review, and I think another review may be in order.

coderabbitai · 2025-07-25T03:22:49Z

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

@maparent Thank you for the clarification about testing approach and completing the manual integration testing! You're absolutely right that integration testing is more valuable here than unit tests. Let me examine the current state of the code since you mentioned substantial changes.

Based on the current implementation, I'll provide you with an updated comprehensive review of the substantial changes you've made.

🏁 Script executed:

#!/bin/bash
# Check the current state of the create_env.ts file
echo "=== Current create_env.ts implementation ==="
cat packages/database/scripts/create_env.ts

# Check if there are any other changed files
echo -e "\n=== Git status to see current changes ==="
git status --porcelain

# Look for any new documentation that might have been added
echo -e "\n=== Check for documentation updates ==="
rg -i "genenv|vercel.*env" --type=md -A 3 -B 1

# Check the current package.json scripts
echo -e "\n=== Current genenv scripts in package.json ==="
cat packages/database/package.json | jq '.scripts | to_entries | map(select(.key | contains("genenv"))) | from_entries'

# Look for any other related changes
echo -e "\n=== Check for other Vercel-related files ==="
rg -l "vercel" --type=ts --type=js | head -10

Length of output: 6606

@maparent Excellent work on the substantial improvements! The code has indeed changed significantly and is much better now. Here's my updated comprehensive review:

✅ Major Improvements Since Initial Review

Code Quality & Organization:

Much cleaner implementation - The code is now more readable and maintainable
Better error handling - Proper validation and meaningful error messages
Fixed consistency issues - The == vs === issue has been resolved
Improved git branch detection - Cleaner logic with proper optional chaining
Better file organization - Appropriate file extensions (.env.local, .env.branch, .env.production)

Documentation Added:

Comprehensive README section explaining the Vercel workflow
Clear setup instructions for VERCEL_TOKEN from 1password
Usage examples for different scenarios (genenv:branch, etc.)
Integration guidance with existing development workflow

Architecture Improvements:

Streamlined function structure - Each environment function is now focused and clean
Better parameter handling - Cleaner baseParams setup with proper team/project configuration
Improved deployment querying - More robust logic for finding deployments

🎯 Current Implementation Assessment

Strengths:

// Clean branch detection with proper error handling
const branchM = stdout.match(/On branch (.*)/)?.[1];
if (branchM) branch = branchM;
else throw new Error("Could not find the git branch");

// Proper file writing with appropriate extensions
writeFileSync(join(projectRoot, ".env.local"), prefixed + '\nNEXT_API_ROOT="http://localhost:3000/api"\n');
appendFileSync(".env.branch", `NEXT_API_ROOT="${url}/api"\n`);

Excellent Integration Testing Approach:
You're absolutely right that manual integration testing across website and roam projects with all three scenarios is the appropriate testing strategy here. This type of infrastructure code benefits more from end-to-end validation than unit tests.

🔍 Minor Observations

The implementation is now very solid. Only minor suggestions:

Environment Variable Override Documentation - The README could mention the override options:
- VERCEL_PROJECT_ID/VERCEL_PROJECT_NAME
- VERCEL_TEAM_ID/VERCEL_TEAM_SLUG
- SUPABASE_GIT_BRANCH
Error Message Enhancement:

// Could be slightly more descriptive
if (result.deployments.length === 0) {
  console.warn(`No deployment found for branch '${branch}' in project '${projectIdOrName}'`);
  return;
}

🎉 Final Assessment

This is now a high-quality, production-ready implementation that:

✅ Addresses all original requirements from ENG-625
✅ Has comprehensive documentation
✅ Has been thoroughly tested via manual integration testing
✅ Integrates cleanly with existing project patterns
✅ Provides the flexibility needed for different deployment scenarios

The substantial improvements you've made have transformed this into an excellent piece of infrastructure code. The manual integration testing approach you used is exactly what this type of feature needs - validating that the entire workflow works end-to-end across multiple projects.

Ready for merge! 🚀

✏️ Learnings added

Learnt from: maparent
URL: https://github.com/DiscourseGraphs/discourse-graph/pull/301

Timestamp: 2025-07-25T03:22:48.188Z
Learning: In the discourse-graph project, manual integration testing is preferred over unit tests for complex features like Vercel CLI integration, especially when testing client creation across multiple projects (website and roam) with different deployment scenarios.

Learnt from: maparent
URL: https://github.com/DiscourseGraphs/discourse-graph/pull/301

Timestamp: 2025-07-25T03:22:48.188Z
Learning: In the discourse-graph project, manual integration testing is preferred over unit tests for complex features like Vercel CLI integration, especially when testing client creation across multiple projects (website and roam) with different deployment scenarios.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

🧠 Learnings used

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#0
File: :0-0
Timestamp: 2025-07-22T01:50:20.253Z
Learning: In packages/database/scripts/create_env.ts, the Vercel CLI integration uses both @vercel/sdk for deployment queries and vercel CLI for environment variable pulling, with support for different variants (local, branch, production, all) and proper team/project configuration with discourse-graphs defaults.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#165
File: packages/database/scripts/deploy.ts:0-0
Timestamp: 2025-05-22T23:48:45.450Z
Learning: The database deployment script in packages/database/scripts/deploy.ts requires that the branch is 'main' and pristine (no uncommitted or unpushed changes) before allowing deployment to proceed.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#241
File: packages/database/tsconfig.json:3-7
Timestamp: 2025-06-25T18:03:52.669Z
Learning: The packages/database directory in the discourse-graph repository has a unique structure as a database schema/migration package. It contains doc/, scripts/, supabase/ directories and TypeScript files at the root level, but no typical src/, test/, dist/, or node_modules directories. The current tsconfig.json with "include": ["."] and "exclude": ["supabase"] is appropriate for this structure.

maparent · 2025-07-25T03:22:50Z

@mdroidian just saw the deploy failure. Will fix that now.

maparent · 2025-07-25T03:58:40Z

Ok, the deploy step requires more thought. It goes to tomorrow.

maparent · 2025-07-25T13:06:56Z

deploy step working. Ready for review.

mdroidian · 2025-07-26T02:37:34Z

The base one is for running against the local supabase.

@maparent Could you elaborate on this? What is "The base one" referring to?

mdroidian · 2025-07-26T21:51:21Z

@CodeRabbit full review

coderabbitai · 2025-07-26T21:51:48Z

✅ Actions performed

Full review triggered.

coderabbitai · 2025-07-26T21:55:35Z

📝 Walkthrough

Walkthrough

This change introduces a unified environment variable management system across multiple packages by centralizing environment configuration logic in a new dbdotenv module within @repo/database. Several scripts and utilities are updated to use this module for accessing Supabase and API credentials, and new scripts are added to automate the generation and management of environment files for different deployment variants. Documentation is expanded to clarify workflows.

Changes

Files/Paths	Change Summary
apps/roam/package.json	Added `@repo/database` as a development dependency.
apps/roam/scripts/compile.ts	Replaced direct `process.env` usage with `envContents()` from `@repo/database/dbdotenv` for environment variable access.
apps/website/app/utils/supabase/client.ts, apps/website/app/utils/supabase/middleware.ts, apps/website/app/utils/supabase/server.ts	Updated to use `envContents()` from `@repo/database/dbdotenv` for Supabase credentials instead of direct `process.env` access.
packages/database/README.md	Extensively revised and expanded documentation for local dev, testing, and deployment environment workflows.
packages/database/create_env.ts	New script: automates environment variable setup for different deployment variants.
packages/database/dbdotenv.ts	New module: provides exported functions for environment variant detection and environment variable loading from variant-specific `.env` files.
packages/database/features/step-definitions/stepdefs.ts	Added protection against running tests in production; initializes environment config using `dbdotenv`.
packages/database/package.json	Added export for `dbdotenv`, new scripts for environment generation, updated build/dev scripts, and new devDependencies.
packages/database/scripts/create_env.ts	New script: generates environment files for local, branch, and production variants using Supabase/Vercel CLI and SDK.
packages/ui/package.json	Added `"type": "module"` to designate ES module format.
packages/ui/src/lib/supabase/client.ts	Updated to use `envContents()` for Supabase credentials.
turbo.json	Expanded `globalEnv` and `globalPassThroughEnv` with additional environment variables.

Sequence Diagram(s)

sequenceDiagram
    participant Script/Server/Client
    participant dbdotenv
    participant .env File(s)
    participant process.env

    Script/Server/Client->>dbdotenv: envContents()
    dbdotenv->>Script/Server/Client: {SUPABASE_URL, SUPABASE_ANON_KEY, ...}

    alt Variant-specific .env file exists
        dbdotenv->>.env File(s): Read and parse
        .env File(s)-->>dbdotenv: Env variables
    else
        dbdotenv->>process.env: Fallback to process.env
        process.env-->>dbdotenv: Env variables
    end

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~15–25 minutes

Possibly related PRs

ENG-529 Add a basic cucumber test for getContext. #267: Modifies environment variable handling in apps/roam/scripts/compile.ts, similar to this PR's centralization of env var access via envContents().
ENG-567 Refactor getContext so the PlatformAccount is created in a Space. #272: Also updates apps/roam/scripts/compile.ts for environment variable definition, touching the same logic as this PR but with a different approach to env var sourcing.

Note

⚡️ Unit Test Generation is now available in beta!

Learn more here, or try it out under "Finishing Touches" below.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Explain this complex logic.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai explain this code block.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and explain its main purpose.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

coderabbitai

Actionable comments posted: 4

🧹 Nitpick comments (3)

packages/database/dbdotenv.ts (1)
30-39: Consider making fallback behavior more explicit.

The fallback to process.env when no environment file exists is logical, but the hardcoded keys (SUPABASE_URL, SUPABASE_ANON_KEY) create an implicit dependency on these specific environment variables.

Consider documenting this fallback behavior or making the fallback keys configurable:
export const envContents = () => {
  const path = envFilePath();
  if (!path)
+    // Fallback to process.env when running in production environments
    return {
      SUPABASE_URL: process.env.SUPABASE_URL,
      SUPABASE_ANON_KEY: process.env.SUPABASE_ANON_KEY,
    };
  const data = fs.readFileSync(path, "utf8");
  return dotenv.parse(data);
};
packages/database/README.md (2)
31-31: Fix spelling error in documentation.

There's a typo in the build process description.
-      1. Start supbase
+      1. Start supabase
44-44: Fix indentation inconsistency.

The markdown indentation is inconsistent for this list item.
-    10. When your migration is pushed in a branch, supabase will create a branch instance. Note there is a small cost to this, so we do not want those branches to linger.
+   10. When your migration is pushed in a branch, supabase will create a branch instance. Note there is a small cost to this, so we do not want those branches to linger.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between cd9d14b and 7cbf07f.

⛔ Files ignored due to path filters (1)

package-lock.json is excluded by !**/package-lock.json

📒 Files selected for processing (14)

apps/roam/package.json (1 hunks)
apps/roam/scripts/compile.ts (3 hunks)
apps/website/app/utils/supabase/client.ts (1 hunks)
apps/website/app/utils/supabase/middleware.ts (1 hunks)
apps/website/app/utils/supabase/server.ts (1 hunks)
packages/database/README.md (2 hunks)
packages/database/create_env.ts (1 hunks)
packages/database/dbdotenv.ts (1 hunks)
packages/database/features/step-definitions/stepdefs.ts (2 hunks)
packages/database/package.json (2 hunks)
packages/database/scripts/create_env.ts (1 hunks)
packages/ui/package.json (1 hunks)
packages/ui/src/lib/supabase/client.ts (2 hunks)
turbo.json (1 hunks)

🧰 Additional context used

🧠 Learnings (15)

📓 Common learnings

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#0
File: :0-0
Timestamp: 2025-07-22T01:50:20.253Z
Learning: In packages/database/scripts/create_env.ts, the Vercel CLI integration uses both @vercel/sdk for deployment queries and vercel CLI for environment variable pulling, with support for different variants (local, branch, production, all) and proper team/project configuration with discourse-graphs defaults.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#165
File: packages/database/scripts/deploy.ts:0-0
Timestamp: 2025-05-22T23:48:45.450Z
Learning: The database deployment script in packages/database/scripts/deploy.ts requires that the branch is 'main' and pristine (no uncommitted or unpushed changes) before allowing deployment to proceed.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#241
File: packages/database/tsconfig.json:3-7
Timestamp: 2025-06-25T18:03:52.669Z
Learning: The packages/database directory in the discourse-graph repository has a unique structure as a database schema/migration package. It contains doc/, scripts/, supabase/ directories and TypeScript files at the root level, but no typical src/, test/, dist/, or node_modules directories. The current tsconfig.json with "include": ["."] and "exclude": ["supabase"] is appropriate for this structure.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#165
File: packages/database/supabase/migrations/20250512142307_sync_table.sql:20-21
Timestamp: 2025-05-20T03:04:21.602Z
Learning: Database migration files (like the SQL files in packages/database/supabase/migrations/) should not be modified after they're created, even to fix minor issues like unused variables. Issues should be addressed in newer schema files instead to maintain migration history integrity.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#182
File: apps/website/app/utils/supabase/dbUtils.ts:22-28
Timestamp: 2025-05-30T14:49:24.016Z
Learning: In apps/website/app/utils/supabase/dbUtils.ts, expanding the KNOWN_EMBEDDINGS and DEFAULT_DIMENSIONS mappings to support additional embedding models requires corresponding database model changes (creating new embedding tables), which should be scoped as separate work from API route implementations.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#198
File: packages/database/supabase/migrations/20250605083319_alpha_upload.sql:157-337
Timestamp: 2025-06-09T16:57:14.681Z
Learning: Migration files in packages/database/supabase/migrations/ are historical snapshots that preserve database schema and functions as they existed at the time of creation. These files should not be updated to reflect current schema changes, even if they reference tables or columns that have since been modified or removed. Schema incompatibilities in migration files are expected and acceptable as they represent the valid state at the time of migration.

apps/roam/package.json (11)

Learnt from: CR
PR: DiscourseGraphs/discourse-graph#0
File: .cursor/rules/roam.mdc:0-0
Timestamp: 2025-07-19T22:34:23.619Z
Learning: Applies to apps/roam/package.json : Prefer existing dependencies from package.json when adding or using dependencies in the Roam Research extension

Learnt from: CR
PR: DiscourseGraphs/discourse-graph#0
File: .cursor/rules/roam.mdc:0-0
Timestamp: 2025-07-19T22:34:23.619Z
Learning: Applies to apps/roam/**/*.{js,jsx,ts,tsx} : Use Roam Depot/Extension API documentation from https://roamresearch.com/#/app/developer-documentation/page/y31lhjIqU when working with the Roam Extension API

Learnt from: CR
PR: DiscourseGraphs/discourse-graph#0
File: .cursor/rules/roam.mdc:0-0
Timestamp: 2025-07-19T22:34:23.619Z
Learning: Applies to apps/roam/**/*.{js,jsx,ts,tsx} : Use the roamAlphaApi documentation from https://roamresearch.com/#/app/developer-documentation/page/tIaOPdXCj when working with the Roam API

Learnt from: CR
PR: DiscourseGraphs/discourse-graph#0
File: .cursor/rules/obsidian.mdc:0-0
Timestamp: 2025-07-19T22:34:16.794Z
Learning: Applies to apps/obsidian/package.json : Prefer existing dependencies from package.json when adding or using dependencies in the Obsidian plugin

Learnt from: CR
PR: DiscourseGraphs/discourse-graph#0
File: .cursor/rules/website.mdc:0-0
Timestamp: 2025-07-19T22:34:26.600Z
Learning: Applies to apps/website/** : Prefer existing dependencies from package.json when adding or using dependencies in the website project

Learnt from: maparent
PR: #241
File: packages/database/tsconfig.json:3-7
Timestamp: 2025-06-25T18:03:52.669Z
Learning: The packages/database directory in the discourse-graph repository has a unique structure as a database schema/migration package. It contains doc/, scripts/, supabase/ directories and TypeScript files at the root level, but no typical src/, test/, dist/, or node_modules directories. The current tsconfig.json with "include": ["."] and "exclude": ["supabase"] is appropriate for this structure.

Learnt from: CR
PR: DiscourseGraphs/discourse-graph#0
File: .cursor/rules/roam.mdc:0-0
Timestamp: 2025-07-19T22:34:23.619Z
Learning: Applies to apps/roam/**/*.{js,jsx,ts,tsx} : Use BlueprintJS 3 components and Tailwind CSS for platform-native UI in the Roam Research extension

Learnt from: maparent
PR: #165
File: packages/database/scripts/deploy.ts:0-0
Timestamp: 2025-05-22T23:48:45.450Z
Learning: The database deployment script in packages/database/scripts/deploy.ts requires that the branch is 'main' and pristine (no uncommitted or unpushed changes) before allowing deployment to proceed.

Learnt from: maparent
PR: #220
File: apps/roam/src/utils/conceptConversion.ts:11-40
Timestamp: 2025-06-23T11:49:45.457Z
Learning: In the DiscourseGraphs/discourse-graph codebase, direct access to window.roamAlphaAPI is the established pattern throughout the codebase. The team prefers to maintain this pattern consistently rather than making piecemeal changes, and plans to address dependency injection as a global refactor when scheduled.

Learnt from: CR
PR: DiscourseGraphs/discourse-graph#0
File: .cursor/rules/main.mdc:0-0
Timestamp: 2025-07-19T22:34:08.725Z
Learning: Applies to apps/website/**/*.{tsx,jsx,ts,js} : Use Tailwind CSS for styling where possible

Learnt from: CR
PR: DiscourseGraphs/discourse-graph#0
File: .cursor/rules/main.mdc:0-0
Timestamp: 2025-07-19T22:34:08.725Z
Learning: Applies to {apps/website,packages/ui}/**/*.{tsx,jsx} : When refactoring inline styles, use tailwind classes

packages/ui/package.json (2)

Learnt from: CR
PR: DiscourseGraphs/discourse-graph#0
File: .cursor/rules/obsidian.mdc:0-0
Timestamp: 2025-07-19T22:34:16.794Z
Learning: Applies to apps/obsidian/package.json : Prefer existing dependencies from package.json when adding or using dependencies in the Obsidian plugin

Learnt from: CR
PR: DiscourseGraphs/discourse-graph#0
File: .cursor/rules/main.mdc:0-0
Timestamp: 2025-07-19T22:34:08.725Z
Learning: Applies to {apps/website,packages/ui}/**/*.{tsx,jsx} : Use platform-native UI components first with shadcn/ui as a fallback

packages/ui/src/lib/supabase/client.ts (4)

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#0
File: :0-0
Timestamp: 2025-07-22T01:50:20.253Z
Learning: In packages/database/scripts/create_env.ts, the Vercel CLI integration uses both @vercel/sdk for deployment queries and vercel CLI for environment variable pulling, with support for different variants (local, branch, production, all) and proper team/project configuration with discourse-graphs defaults.

Learnt from: maparent
PR: #291
File: packages/database/supabase/functions/create-space/index.ts:0-0
Timestamp: 2025-07-21T14:22:20.752Z
Learning: In the discourse-graph codebase, types.gen.ts is not accessible from Supabase edge functions, requiring duplication of types and utilities when needed in the edge function environment at packages/database/supabase/functions/.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#0
File: :0-0
Timestamp: 2025-07-08T14:51:55.299Z
Learning: DiscourseGraphs maintainers prefer not to add narrowly typed definitions for individual process.env keys to avoid unnecessary coupling.

Learnt from: maparent
PR: #272
File: packages/ui/src/lib/supabase/contextFunctions.ts:50-111
Timestamp: 2025-07-08T14:48:38.048Z
Learning: The Supabase client does not offer transaction support, so idempotent upserts with proper conflict resolution (using onConflict with ignoreDuplicates: false) are the preferred approach for multi-step database operations in packages/ui/src/lib/supabase/contextFunctions.ts. This pattern prevents orphaned records when retrying operations.

apps/website/app/utils/supabase/server.ts (7)

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#0
File: :0-0
Timestamp: 2025-07-22T01:50:20.253Z
Learning: In packages/database/scripts/create_env.ts, the Vercel CLI integration uses both @vercel/sdk for deployment queries and vercel CLI for environment variable pulling, with support for different variants (local, branch, production, all) and proper team/project configuration with discourse-graphs defaults.

Learnt from: maparent
PR: #291
File: packages/database/supabase/functions/create-space/index.ts:0-0
Timestamp: 2025-07-21T14:22:20.752Z
Learning: In the discourse-graph codebase, types.gen.ts is not accessible from Supabase edge functions, requiring duplication of types and utilities when needed in the edge function environment at packages/database/supabase/functions/.

Learnt from: maparent
PR: #182
File: apps/website/app/utils/supabase/dbUtils.ts:22-28
Timestamp: 2025-05-30T14:49:24.016Z
Learning: In apps/website/app/utils/supabase/dbUtils.ts, expanding the KNOWN_EMBEDDINGS and DEFAULT_DIMENSIONS mappings to support additional embedding models requires corresponding database model changes (creating new embedding tables), which should be scoped as separate work from API route implementations.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#0
File: :0-0
Timestamp: 2025-07-08T14:51:55.299Z
Learning: DiscourseGraphs maintainers prefer not to add narrowly typed definitions for individual process.env keys to avoid unnecessary coupling.

Learnt from: maparent
PR: #272
File: packages/ui/src/lib/supabase/contextFunctions.ts:50-111
Timestamp: 2025-07-08T14:48:38.048Z
Learning: The Supabase client does not offer transaction support, so idempotent upserts with proper conflict resolution (using onConflict with ignoreDuplicates: false) are the preferred approach for multi-step database operations in packages/ui/src/lib/supabase/contextFunctions.ts. This pattern prevents orphaned records when retrying operations.

Learnt from: maparent
PR: #241
File: packages/database/tsconfig.json:3-7
Timestamp: 2025-06-25T18:03:52.669Z
Learning: The packages/database directory in the discourse-graph repository has a unique structure as a database schema/migration package. It contains doc/, scripts/, supabase/ directories and TypeScript files at the root level, but no typical src/, test/, dist/, or node_modules directories. The current tsconfig.json with "include": ["."] and "exclude": ["supabase"] is appropriate for this structure.

Learnt from: maparent
PR: #165
File: packages/database/scripts/deploy.ts:0-0
Timestamp: 2025-05-22T23:48:45.450Z
Learning: The database deployment script in packages/database/scripts/deploy.ts requires that the branch is 'main' and pristine (no uncommitted or unpushed changes) before allowing deployment to proceed.

turbo.json (3)

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#0
File: :0-0
Timestamp: 2025-07-22T01:50:20.253Z
Learning: In packages/database/scripts/create_env.ts, the Vercel CLI integration uses both @vercel/sdk for deployment queries and vercel CLI for environment variable pulling, with support for different variants (local, branch, production, all) and proper team/project configuration with discourse-graphs defaults.

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#0
File: :0-0
Timestamp: 2025-07-08T14:51:55.299Z
Learning: DiscourseGraphs maintainers prefer not to add narrowly typed definitions for individual process.env keys to avoid unnecessary coupling.

Learnt from: maparent
PR: #165
File: packages/database/supabase/config.toml:0-0
Timestamp: 2025-05-20T15:51:22.084Z
Learning: Wildcard patterns in additional_redirect_urls within Supabase configuration are acceptable in this codebase when needed for Vercel branch deployments, despite the small security risk they introduce.

apps/website/app/utils/supabase/client.ts (4)

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#0
File: :0-0
Timestamp: 2025-07-22T01:50:20.253Z
Learning: In packages/database/scripts/create_env.ts, the Vercel CLI integration uses both @vercel/sdk for deployment queries and vercel CLI for environment variable pulling, with support for different variants (local, branch, production, all) and proper team/project configuration with discourse-graphs defaults.

Learnt from: maparent
PR: #182
File: apps/website/app/utils/supabase/dbUtils.ts:22-28
Timestamp: 2025-05-30T14:49:24.016Z
Learning: In apps/website/app/utils/supabase/dbUtils.ts, expanding the KNOWN_EMBEDDINGS and DEFAULT_DIMENSIONS mappings to support additional embedding models requires corresponding database model changes (creating new embedding tables), which should be scoped as separate work from API route implementations.

Learnt from: maparent
PR: #272
File: packages/ui/src/lib/supabase/contextFunctions.ts:50-111
Timestamp: 2025-07-08T14:48:38.048Z
Learning: The Supabase client does not offer transaction support, so idempotent upserts with proper conflict resolution (using onConflict with ignoreDuplicates: false) are the preferred approach for multi-step database operations in packages/ui/src/lib/supabase/contextFunctions.ts. This pattern prevents orphaned records when retrying operations.

Learnt from: maparent
PR: #291
File: packages/database/supabase/functions/create-space/index.ts:0-0
Timestamp: 2025-07-21T14:22:20.752Z
Learning: In the discourse-graph codebase, types.gen.ts is not accessible from Supabase edge functions, requiring duplication of types and utilities when needed in the edge function environment at packages/database/supabase/functions/.

apps/roam/scripts/compile.ts (8)