[DMS-877] Design Profiles Feature #770

jleiva-gap · 2025-12-10T21:29:56Z

Add design Files.
Design files based on docs generated by copilot PR to include more context about profiles (issue #763
Add examples

Copilot

Pull request overview

This PR adds comprehensive design documentation and example files for the API Profiles feature in the Ed-Fi Data Management Service. The Profiles feature enables fine-grained control over which data fields are visible or modifiable for different API consumers, supporting use cases like vendor integration, reporting systems, and data privacy compliance. The documentation was generated with assistance from Copilot to provide context for issue #763.

Key Changes:

Comprehensive design documentation covering architecture, database schema, API specifications, and implementation strategy
Quick start guide with step-by-step tutorials for using API Profiles
Five example profile XML files demonstrating common use cases (student, school, assessment, descriptor access patterns)
XML schema definition (XSD) for validating profile documents

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
docs/Profiles/API-PROFILES-DESIGN.md	Complete 2018-line design document detailing architecture, database schema, API specifications, enforcement pipeline, and implementation tickets for the Profiles feature
docs/Profiles/API-PROFILES-QUICKSTART.md	490-line quick start guide with tutorials, common patterns, testing procedures, and troubleshooting for API Profiles
docs/Profiles/examples/profiles/README.md	Overview documentation for example profiles including usage instructions, customization guidelines, and validation procedures
docs/Profiles/examples/profiles/student-write-limited.xml	Example profile allowing limited write access to student demographic and contact information
docs/Profiles/examples/profiles/student-read-only.xml	Example profile providing read-only access to basic student demographics and school associations
docs/Profiles/examples/profiles/school-minimal.xml	Example profile exposing public-facing school directory information
docs/Profiles/examples/profiles/descriptor-full-access.xml	Example profile granting complete access to descriptor resources for system administration
docs/Profiles/examples/profiles/assessment-limited.xml	Example profile restricting assessment data access to core fields only
docs/Profiles/examples/profiles/profile-schema.xsd	XML Schema Definition for validating profile document structure and constraints

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

docs/Profiles/examples/profiles/README.md

docs/Profiles/API-PROFILES-QUICKSTART.md

docs/Profiles/API-PROFILES-DESIGN.md

docs/Profiles/examples/README.md

docs/Profiles/API-PROFILES-DESIGN.md

docs/Profiles/API-PROFILES-QUICKSTART.md

simpat-adam · 2025-12-11T20:34:04Z

docs/Profiles/API-PROFILES-DESIGN.md

+
+1. **Profile Resolution**: Extract profile from `Content-Type` header
+2. **Rule Evaluation**: Load property/collection/reference rules
+3. **Input Validation**:


I don't remember how ODS behaves, but I am wondering about the requirements on the write side. What is the expectation when updating a document using a profile with limited fields? Is the expectation to do a Read-Merge-Write?

Yes, this would be similar to the read/merge/write pattern you mentioned. In ODS, if a field is explicitly marked to be excluded from writing and that field is still sent in the payload, it is simply ignored during the update. The system does not overwrite it with a blank value; instead, it updates only the remaining fields.

simpat-adam · 2025-12-11T20:37:20Z

docs/Profiles/API-PROFILES-DESIGN.md

+
+### Read Operation Enforcement
+
+For GET operations:


May be too much of an implementation detail but are we thinking middleware for read projection? Or directly in the query with JSONB operators to project only the profile fields?

The initial idea with the read operation is to apply the filter at the middleware level. This way, the query executes normally, and before assigning the response, the applicable fields are excluded according to the profile definition. This approach provides a more database‑agnostic design regarding JSONB, avoiding dependency on how each engine defines it.

simpat-adam · 2025-12-11T20:38:54Z

docs/Profiles/API-PROFILES-DESIGN.md

+
+### Write Operation Enforcement
+
+For POST/PUT/PATCH operations:


DMS doesn't support PATCH operations right now. Does ODS?

I understand. Since the use of PATCH is also uncommon in the ODS API, it was only mentioned as a reference but is not included in the endpoint definitions. I’ll remove it for clarity.

simpat-adam

My general comment is that the design probably over-explains the config service side and basic CRUD mechanics of the profiles themselves while glossing over the DMS implementation side, (I see one ticket for an "Evaluation Engine"). I posted some comment/questions about the "how" that we might want to address before we dig into tickets.

Add design Files Update files generated by copilot PR to include more context about profiles Add examples

Remove extra sections

Clean the draft

Review and update the notes

Update URL and paths

Update design document

Update request example

Remove unused section Update links

Include table Application Profile Update tickets

Update links

Update diagrams

Fix diagram

Update response

Update descriptions

jleiva-gap · 2025-12-15T20:10:28Z

My general comment is that the design probably over-explains the config service side and basic CRUD mechanics of the profiles themselves while glossing over the DMS implementation side, (I see one ticket for an "Evaluation Engine"). I posted some comment/questions about the "how" that we might want to address before we dig into tickets.

Regarding the tickets, some may not need to be included, while others might require refinement or splitting. At this stage, we can review suggestions and improvements before creating items in JIRA, ensuring everything is as clear as possible.

As for your comments on the tickets related to DMS, the following are listed in the document as part of this section: 3 (Profile Resolution), 4 (Rule Engine), 5 (Write Enforcement), 6 (Read Filtering), 8 (Profile Selection), and eventually 12, which are directly tied to the DMS process. On the Config side, tickets 1, 2, and 7 are included.

bradbanister

Spot-checking this I see a lot of inaccuracies that look like AI slop. How well was this reviewed before submitting as a PR?

docs/Profiles/API-PROFILES-DESIGN.md

bradbanister · 2025-12-15T21:07:47Z

docs/Profiles/API-PROFILES-DESIGN.md

+
+**Application-Based Profile Assignment:**
+
+When no explicit profile header is provided, DMS:


This section seems wrong. Profile application depends on relevance to the request resource and usage, ODS returns 403 not 400 and only when there are multiple relevant assigned profiles and no profile header, and I don't know what "falls back to default profile" means. Are default profiles even a thing?

Still needs to address that these are relevant profiles for this request

bradbanister · 2025-12-15T21:18:50Z

docs/Profiles/API-PROFILES-DESIGN.md

+
+    CallConfigAPI --> ProfileExists{Profile Exists?}
+    ProfileExists -->|Yes| LoadProfile[Parse Profile JSON & Rules]
+    ProfileExists -->|No| Error404[404 Profile Not Found]


This is wrong. 406 for GET, 415 for POST/PUT. I don't think Profiles ever returns 404.

docs/Profiles/API-PROFILES-DESIGN.md

bradbanister · 2025-12-15T21:38:45Z

docs/Profiles/API-PROFILES-DESIGN.md

+                  <xs:attribute name="memberSelection" type="xs:string"/>
+                </xs:complexType>
+              </xs:element>
+              <xs:element name="WriteContentType" minOccurs="0">


Is this intended to be complete? Both this and ReadContentType are missing a lot here.

Remove references to default profile Update Content Type values Update xml schema Update response codes

Update http codes Update examples Update Technical Implementation Details section to be a reference

Update diagrams

Copilot

Pull request overview

Copilot reviewed 9 out of 10 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Update xml files

bradbanister · 2025-12-17T16:24:45Z

docs/Profiles/API-PROFILES-QUICKSTART.md

+```bash
+curl -X GET \
+  https://dms-api/data/v3/ed-fi/students/{id} \
+  -H "Accept: application/json;profile=student-read-only" \


bradbanister · 2025-12-17T16:25:04Z

docs/Profiles/API-PROFILES-QUICKSTART.md

+  -H "Authorization: Bearer YOUR_TOKEN"
+
+# Verify header format
+Accept: application/json;profile=profile-name


bradbanister · 2025-12-17T16:25:20Z

docs/Profiles/examples/README.md

+# Test the profile
+curl -X GET \
+  https://dms-api/data/v3/ed-fi/students/{id} \
+  -H "Accept: application/json;profile=student-read-only" \


bradbanister · 2025-12-17T16:26:09Z

docs/Profiles/API-PROFILES-DESIGN.md

+[Auth] -> [ProfileResolution] -> [ParsePath] -> [BuildResourceInfo] -> [ParseBody] -> ...
+
+Profile Resolution Algorithm:
+1. Check Accept/Content-Type header for profile parameter


bradbanister · 2025-12-17T16:28:52Z

docs/Profiles/API-PROFILES-DESIGN.md

+1. **Profile Resolution**: Extract profile from `Content-Type` header (Ed-Fi vendor media type format: `application/vnd.ed-fi.{resource}.{profile}.writable+json`)
+2. **Rule Evaluation**: Load property/collection/reference/filter rules
+3. **Input Validation**:
+   - Check for excluded properties in request body


Excluded fields are ignored in ODS, what's this for?

bradbanister · 2025-12-17T17:25:49Z

docs/Profiles/examples/profile-schema.xsd

+        <xs:element name="Object" type="ClassDefinition"/>
+        <xs:element name="Collection" type="CollectionDefinition"/>
+      </xs:choice>
+      <xs:element name="Filter" type="FilterDefinition" minOccurs="0" maxOccurs="unbounded"/>


ODS has 0 or 1

bradbanister · 2025-12-17T17:27:16Z

docs/Profiles/API-PROFILES-QUICKSTART.md

+
+### Profile Not Found
+
+**Error**: `404 Profile 'xyz' not found`


bradbanister · 2025-12-17T17:29:22Z

docs/Profiles/API-PROFILES-DESIGN.md

+        "collections": [
+          {
+            "name": "addresses",
+            "memberSelection": "Exclude"


Is Exclude valid?

bradbanister · 2025-12-17T17:30:28Z

docs/Profiles/examples/README.md

+2. Rename the profile in the `name` attribute
+3. Modify the `Resource` name if targeting a different resource
+4. Adjust `memberSelection` strategy (IncludeOnly, ExcludeOnly, etc.)
+5. Add or remove `Property`, `Collection`, and `Reference` elements


No Reference in schema

bradbanister · 2025-12-17T17:32:14Z

docs/Profiles/examples/school-filtered-addresses.xml

+
+        <!-- Filter: Only include Physical and Mailing addresses -->
+        <Filter propertyName="AddressTypeDescriptor" filterMode="IncludeOnly">
+          <Value>Physical</Value>


Should look like real descriptor values right? Also below

jleiva-gap force-pushed the DMS-877-2 branch from 4ca521b to 20b9e2b Compare December 11, 2025 04:10

jleiva-gap requested a review from Copilot December 11, 2025 15:01

Copilot started reviewing on behalf of jleiva-gap December 11, 2025 15:02 View session

Copilot AI reviewed Dec 11, 2025

View reviewed changes

jleiva-gap marked this pull request as ready for review December 11, 2025 15:43

simpat-adam reviewed Dec 11, 2025

View reviewed changes

jleiva-gap added 9 commits December 15, 2025 09:13

[DMS-877] Design Profiles Feature

d4a5f66

Add design Files Update files generated by copilot PR to include more context about profiles Add examples

Update API-PROFILES-DESIGN.md

d9c383d

Remove extra sections

Update API-PROFILES-DESIGN.md

2b3ac34

Clean the draft

Update API-PROFILES-DESIGN.md

01300f0

Review and update the notes

Update examples

f1a5d20

Update URL and paths

Update API-PROFILES-DESIGN.md

6fc1766

Update design document

Update API-PROFILES-DESIGN.md

1051b85

Update request example

Update links

0807e79

Remove unused section Update links

Update documentation

69f9a7d

Include table Application Profile Update tickets

jleiva-gap force-pushed the DMS-877-2 branch from c032a1b to 69f9a7d Compare December 15, 2025 15:13

jleiva-gap added 5 commits December 15, 2025 09:18

Update API-PROFILES-DESIGN.md

d4860a5

Update links

Update API-PROFILES-DESIGN.md

859ba8f

Update diagrams

Update API-PROFILES-DESIGN.md

168c5fe

Fix diagram

Update API-PROFILES-DESIGN.md

351b250

Update response

Update API-PROFILES-DESIGN.md

00074d2

Update descriptions

bradbanister requested changes Dec 15, 2025

View reviewed changes

jleiva-gap added 3 commits December 15, 2025 22:16

Update API-PROFILES-DESIGN.md

fe813f8

Remove references to default profile Update Content Type values Update xml schema Update response codes

Update API-PROFILES-DESIGN.md

a5d2890

Update http codes Update examples Update Technical Implementation Details section to be a reference

Update API-PROFILES-DESIGN.md

871a50f

Update diagrams

jleiva-gap requested a review from Copilot December 16, 2025 15:27

Copilot started reviewing on behalf of jleiva-gap December 16, 2025 15:28 View session

Copilot AI reviewed Dec 16, 2025

View reviewed changes

jleiva-gap requested a review from bradbanister December 16, 2025 15:31

Sync xml examples and schema

c112011

Update xml files

bradbanister requested changes Dec 17, 2025

View reviewed changes

jleiva-gap closed this Dec 18, 2025


		### Write Operation Enforcement

		For POST/PUT/PATCH operations:


		Application-Based Profile Assignment:

		When no explicit profile header is provided, DMS:


		### Profile Not Found

		Error: `404 Profile 'xyz' not found`

[DMS-877] Design Profiles Feature #770

[DMS-877] Design Profiles Feature #770

Uh oh!

Conversation

jleiva-gap commented Dec 10, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

simpat-adam left a comment

Choose a reason for hiding this comment

Uh oh!

jleiva-gap commented Dec 15, 2025

Uh oh!

bradbanister left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone