Eng 898 create database query functions for base dg queries

[maparent]

Base database functions for dg queries. (WIP)

Summary by CodeRabbit

• New Features
  • Expanded concept querying with filters for author, relation type, related node type, and specific related nodes.
  • Ability to fetch related concepts/relations; concept results now include schema_id.
• Database
  • Added relation access functions to traverse from a concept to its relations and vice versa.
• Tests
  • Introduced comprehensive BDD scenarios covering concept upsert and complex queries.
  • Added authenticated steps for querying nodes and upserting accounts within a space.
• Refactor
  • Streamlined test login/setup and query composition to support advanced filtering.

[linear]

ENG-898 Create database query functions for base DG queries

[supabase]

Updates to Preview Branch (eng-898-create-database-query-functions-for-base-dg-queries) ↗︎

Deployments	Status	Updated
Database	✅	Sun, 12 Oct 2025 23:21:48 UTC
Services	✅	Sun, 12 Oct 2025 23:21:48 UTC
APIs	✅	Sun, 12 Oct 2025 23:21:48 UTC

Tasks are run on every commit but only new migration files are pushed.
Close and reopen this PR if you want to apply changes from existing seed or migration files.

Tasks	Status	Updated
Configurations	✅	Sun, 12 Oct 2025 23:21:51 UTC
Migrations	✅	Sun, 12 Oct 2025 23:21:51 UTC
Seeding	✅	Sun, 12 Oct 2025 23:21:51 UTC
Edge Functions	✅	Sun, 12 Oct 2025 23:21:54 UTC

---

View logs for this Workflow Run ↗︎.
Learn more about Supabase for Git ↗︎.

[maparent]

@CodeRabbit review

[coderabbitai]

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

📝 Walkthrough

Walkthrough

Introduces relation accessor SQL functions and surfaces them in types. Expands getNodes API and adds initNodeSchemaCache. Updates step definitions to use logged-in clients, new ID-prefix semantics ($), and new query/upsert steps. Adds a comprehensive BDD feature for concept upsert/query. Minor feature table header change (@id → $id).

Changes

Cohort / File(s)	Summary
BDD features packages/database/features/contentAccess.feature	Background table: header token changed from @id to $id for Document row.
Concept query feature packages/database/features/queryConcepts.feature	New BDD spec covering concept upsert and complex node queries with filters (schemas, types, author, relations, related nodes).
Cucumber step definitions packages/database/features/step-definitions/stepdefs.ts	Adds logged-in client helper, switches env access to nullable, migrates ID semantics from @ to $, adds local-ref substitution utilities, new upsert/query steps, integrates getNodes/initNodeSchemaCache, and updates insert/lookup flows.
Query library packages/database/src/lib/queries.ts	Exports initNodeSchemaCache; extends CONCEPT_FIELDS with schema_id; broadens getNodes signature and internal composition to support author and relation-based filters and base-node constraints; maps local IDs to DB IDs.
DB types packages/database/src/dbTypes.ts	Adds function typings: concept_in_relations and concepts_of_relation returning Concept-shaped rows.
SQL migration packages/database/supabase/migrations/20250929154709_relation_access_functions.sql	Adds public SQL functions: concept_in_relations(concept) and concepts_of_relation(relation).
Schema SQL packages/database/supabase/schemas/concept.sql	Defines same public functions with comments in schema DDL.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Tester as Cucumber Scenarios
  participant Steps as stepdefs.ts
  participant Query as getNodes (queries.ts)
  participant DB as Supabase/Postgres
  participant Fns as SQL funcs

  Tester->>Steps: Given user logs in space / seeds data ($-prefixed IDs)
  Steps->>Steps: substituteLocalReferences / map $local → DB ids
  Steps->>DB: Upsert RPC / inserts (logged-in client)
  Note right of Steps: Stores world.localRefs

  Tester->>Steps: When querying nodes with parameters
  Steps->>Query: getNodes({ baseNodeLocalIds, nodeAuthor, inRels... })
  Query->>DB: Select Concepts/Contents with filters
  alt Relation filters present
    Query->>Fns: concepts_of_relation / concept_in_relations
    Fns-->>Query: Related concept rows
  end
  DB-->>Query: Result rows (+schema_id, inner content as needed)
  Query-->>Steps: Nodes list
  Steps->>Steps: Sort and local-ref substitution
  Steps-->>Tester: Then results should match

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

• ENG-833 Isolate node query functions. #427 — Also updates queries.ts and dbTypes.ts to extend node query utilities and DB function typings.
• ENG-431 Separate literal and references in concept data #200 — Modifies Concept schema and adds relation/reference-oriented DB functions and query support, overlapping with new relation accessors.

Pre-merge checks

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit's high-level summary is enabled.
Title Check	✅ Passed	The title clearly indicates that the pull request is focused on creating database query functions for base DG queries, which aligns with the primary additions of SQL functions and expanded query APIs without including unnecessary file lists or vague descriptors.
Docstring Coverage	✅ Passed	No functions found in the changes. Docstring coverage check skipped.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 4

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9ab8b30 and 93ea936.

📒 Files selected for processing (7)

• packages/database/features/contentAccess.feature (1 hunks)
• packages/database/features/queryConcepts.feature (1 hunks)
• packages/database/features/step-definitions/stepdefs.ts (10 hunks)
• packages/database/src/dbTypes.ts (1 hunks)
• packages/database/src/lib/queries.ts (6 hunks)
• packages/database/supabase/migrations/20250929154709_relation_access_functions.sql (1 hunks)
• packages/database/supabase/schemas/concept.sql (1 hunks)

🧰 Additional context used

🧠 Learnings (1)

📚 Learning: 2025-07-13T16:47:14.352Z

Learnt from: maparent
PR: DiscourseGraphs/discourse-graph#0
File: :0-0
Timestamp: 2025-07-13T16:47:14.352Z
Learning: In the discourse-graph codebase, types.gen.ts contains automatically generated database function type definitions that may have reordered signatures between versions. This reordering is expected behavior from the code generation process and should not be flagged as an issue requiring fixes.

Applied to files:

• packages/database/src/dbTypes.ts

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)

• GitHub Check: Supabase Preview
• GitHub Check: Supabase Preview

[mdroidian]

Ok, there's a lot going on here. I've made this as Request changes just so it gets back to you and tagged in linear when re-requested for review.

Could you help me understand what the purpose of this PR is?

I see the title "create database query functions for base dg queries" and in the linear ticket there is a comment with list of 7 queries. Yet in this PR I do not see any new queries nor any reference to how these changes are related or meant to apply to those 7 queries.

Is this just the base and specific queries will be created in the future? If so, is there an additional linear ticket for that? Or are the dev's supposed to use this base query and be able to construct all 7 required queries when they need to use them? Is documentation on these additional parameters forthcoming?

If it is the case that this base query is to be used by other dev's to construct a query, could you give some examples of what that would look like for a few examples:

• discourse context: query all nodes in any relation with node x, and the relation label
• query all discourse nodes of type x
• query all discourse relations containing specific discourse node x

[mdroidian]

Looks like this didn't get prettier formatted. It is helpful to keep up the format with each pr/change because when it is missed it adds a lot of noise to the subsequent pr that applies it.

[mdroidian]

Actually it looks like this is the case with most of the files in this PR.

Please go through and ensure each file is formatted.

[maparent]

Ok, I have to look into why prettier is not auto-applied.
In the case of the feature files, the zed formatter for gerkhin (the cucumber .feature file format) fights with prettier, and wants to right-align dates, which is... annoying but not wrong?
I will leave that part as-is for now to avoid churn.
But did apply prettier elsewhere.

[mdroidian]

Prettier should be applied on save.
When is the zed formatter for gerkhin applied? If it is also applied on save, I would try to turn it off so we can have a single primary formatter across all files.

[maparent]

Yes, I'll have to dig into zed configurations. Trying to punt this.

[maparent]

got zed to auto-prettify ts files, which is good. It's still unwilling to use prettier on gherkin.

[mdroidian]

Do you need to add something like this to zed's settings file?
https://www.npmjs.com/package/prettier-plugin-gherkin/v/0.0.1

[maparent]

As to your main comment:
The examples for the requested queries are in the queryConcept.feature file.
There are a few combinations of factors that require inferring from the examples in the files, but each individual factor is represented. We can discuss later today.

[maparent]

@mdroidian lmk if you feel the documentation is adequate now.

[mdroidian]

I still think there’s quite a bit of room for improvement from a DX perspective, but I won’t block on it.

For example — when I’m coding a component and need nodes of type x quickly, I’ll type in getConcepts (which already requires a small context switch), and then I’m greeted with this:

At that point, I find myself asking: “Which params do I actually need to query nodes of type x?” To figure that out, I have to open getConcepts and read through a long block of text — sorting through relation-specific params just to find what I need. The new documentation definitely helps with this, but I still think we could simplify things quite a bit. It might make sense to split it into a few smaller, more focused queries down the road.

That said, this is totally fine as a starting point.

[mdroidian]

The following files still add changes when they are saved:

• tests.md
• contentAccess.feature
• queryConcepts.feature
• README.md

Can you confirm this is the same for you?

[maparent]

Hmm... Somehow, Zed's enforced prettier settings don't agree with our prettier settings. Sigh.
Did apply prettier, will have to dig deeper into settings.

[maparent]

Re the number of arguments: I understand that is not ideal to have 17 parameters, only 4 of which would be recurrent across versions, but think of the combinatorics of having small functions for any subset of the other 13!

[mdroidian]

Re the number of arguments: I understand that is not ideal to have 17 parameters, only 4 of which would be recurrent across versions, but think of the combinatorics of having small functions for any subset of the other 13!

Good point about avoiding combinatorics. I’m not advocating a pile of tiny functions—just clearer grouping and a couple of thin presets so the common queries don’t require reading the entire param wall.

This is also probably a good time to consider adding zod.

[maparent]

Hmmm... I do not see the need for this but you do; I think this is worth a synchronous co-design session.
Or you could just add functions with less parameters that call the big function.
Also not sure Zod would help, but you may have something in mind that I don't see.

[mdroidian]

Here's an example: #488

Zod would help catch bad combinations.

We can discuss in the dev meeting today.