-
-
Notifications
You must be signed in to change notification settings - Fork 1.5k
feat: Integrate StorageService for large controller data #22943
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 7 commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
01d42bc
feat: Integrate StorageService for large controller data
andrepimenta 72939b0
refactor: Update mobile adapter for new StorageService interface
andrepimenta d2ab2d1
refactor(storage-service): update adapter to handle key building and …
andrepimenta e431a4f
Add preview package of storage-service
andrepimenta 1f10b96
Merge branch 'main' into feature/storage-service
andrepimenta 7294e96
Merge branch 'feature/storage-service' of https://github.com/MetaMask…
andrepimenta 3e6f2ea
Use metamask-previews for now
andrepimenta f886613
fix(storage-service): add StorageServiceEvents to GlobalEvents
andrepimenta ba326e3
fix(storage-service): add StorageService to STATELESS_NON_CONTROLLER_…
andrepimenta 23fc5b6
test(storage-service): achieve 100% coverage for storage-service-init
andrepimenta 7d6da38
fix(storage-service): use undefined instead of null for FilesystemSto…
andrepimenta 08c5d98
test(storage-service): rename test to avoid weasel words
andrepimenta 58d0b47
fix(storage-service): throw errors consistently in all adapter methods
andrepimenta 049a5ff
refactor(storage-service): use Json type and remove wrapper
andrepimenta ca1941d
Merge branch 'main' into feature/storage-service
andrepimenta a7de4a4
Merge branch 'main' into feature/storage-service
andrepimenta 11b7710
chore: update @metamask-previews/storage-service with StorageGetResul…
andrepimenta 52d20ab
refactor(storage-service): use @metamask/storage-service with Storage…
andrepimenta 5cf519d
fix: prettier formatting in storage-service-init.test.ts
andrepimenta 71d6519
Merge branch 'main' into feature/storage-service
andrepimenta 6cdc688
Merge branch 'main' into feature/storage-service
andrepimenta 650045d
Merge branch 'main' into feature/storage-service
andrepimenta File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| import { buildControllerInitRequestMock } from '../utils/test-utils'; | ||
| import { ExtendedMessenger } from '../../ExtendedMessenger'; | ||
| import { getStorageServiceMessenger } from '../messengers/storage-service-messenger'; | ||
| import { ControllerInitRequest } from '../types'; | ||
| import { storageServiceInit } from './storage-service-init'; | ||
| import { | ||
| StorageService, | ||
| StorageServiceMessenger, | ||
| } from '@metamask-previews/storage-service'; | ||
| import { MOCK_ANY_NAMESPACE, MockAnyNamespace } from '@metamask/messenger'; | ||
|
|
||
| jest.mock('@metamask-previews/storage-service'); | ||
|
|
||
| function getInitRequestMock(): jest.Mocked< | ||
| ControllerInitRequest<StorageServiceMessenger> | ||
| > { | ||
| const baseMessenger = new ExtendedMessenger<MockAnyNamespace, never, never>({ | ||
| namespace: MOCK_ANY_NAMESPACE, | ||
| }); | ||
|
|
||
| const requestMock = { | ||
| ...buildControllerInitRequestMock(baseMessenger), | ||
| controllerMessenger: getStorageServiceMessenger(baseMessenger), | ||
| initMessenger: undefined, | ||
| }; | ||
|
|
||
| return requestMock; | ||
| } | ||
|
|
||
| describe('storageServiceInit', () => { | ||
| it('initializes the service', () => { | ||
| const { controller } = storageServiceInit(getInitRequestMock()); | ||
|
|
||
| expect(controller).toBeInstanceOf(StorageService); | ||
| }); | ||
|
|
||
| it('passes the proper arguments to the service', () => { | ||
| storageServiceInit(getInitRequestMock()); | ||
|
|
||
| const serviceMock = jest.mocked(StorageService); | ||
|
|
||
| expect(serviceMock).toHaveBeenCalledWith({ | ||
| messenger: expect.any(Object), | ||
| storage: expect.objectContaining({ | ||
| getItem: expect.any(Function), | ||
| setItem: expect.any(Function), | ||
| removeItem: expect.any(Function), | ||
| }), | ||
| }); | ||
| }); | ||
|
|
||
| it('provides FilesystemStorage adapter with required methods', () => { | ||
| storageServiceInit(getInitRequestMock()); | ||
|
|
||
| const serviceMock = jest.mocked(StorageService); | ||
| const callArguments = serviceMock.mock.calls[0][0]; | ||
|
|
||
| expect(callArguments.storage).toBeDefined(); | ||
| expect(callArguments.storage?.getItem).toBeInstanceOf(Function); | ||
| expect(callArguments.storage?.setItem).toBeInstanceOf(Function); | ||
| expect(callArguments.storage?.removeItem).toBeInstanceOf(Function); | ||
| // getAllKeys and clear are optional | ||
| }); | ||
| }); | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,189 @@ | ||
| import { ControllerInitFunction } from '../types'; | ||
| import { | ||
| StorageService, | ||
| StorageServiceMessenger, | ||
| StorageAdapter, | ||
| STORAGE_KEY_PREFIX, | ||
| } from '@metamask-previews/storage-service'; | ||
| import FilesystemStorage from 'redux-persist-filesystem-storage'; | ||
| import Device from '../../../util/device'; | ||
| import Logger from '../../../util/Logger'; | ||
|
|
||
| /** | ||
| * Wrapper for stored data with metadata. | ||
| * Each adapter defines its own wrapper structure. | ||
| */ | ||
| interface StoredDataWrapper<T = unknown> { | ||
| /** Timestamp when data was stored (milliseconds since epoch). */ | ||
| timestamp: number; | ||
| /** The actual data being stored. */ | ||
| data: T; | ||
| } | ||
|
|
||
| /** | ||
| * Mobile-specific storage adapter using FilesystemStorage. | ||
| * This provides persistent storage for large controller data. | ||
| * | ||
| * Extension will provide its own adapter using IndexedDB. | ||
| * Tests use InMemoryStorageAdapter (default when no storage provided). | ||
| */ | ||
| const mobileStorageAdapter: StorageAdapter = { | ||
| /** | ||
| * Get an item from filesystem storage. | ||
| * Deserializes and unwraps the stored data. | ||
| * | ||
| * @param namespace - The controller namespace. | ||
| * @param key - The data key. | ||
| * @returns The unwrapped data, or null if not found. | ||
| */ | ||
| async getItem(namespace: string, key: string): Promise<unknown> { | ||
| try { | ||
| // Build full key: storageService:namespace:key | ||
| const fullKey = `${STORAGE_KEY_PREFIX}${namespace}:${key}`; | ||
| const serialized = await FilesystemStorage.getItem(fullKey); | ||
|
|
||
| if (!serialized) { | ||
| return null; | ||
| } | ||
|
|
||
| const wrapper: StoredDataWrapper = JSON.parse(serialized); | ||
| return wrapper.data; | ||
| } catch (error) { | ||
| Logger.error(error as Error, { | ||
| message: `StorageService: Failed to get item: ${namespace}:${key}`, | ||
| }); | ||
NicolasMassart marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| return null; | ||
| } | ||
andrepimenta marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| }, | ||
|
|
||
| /** | ||
| * Set an item in filesystem storage. | ||
| * Wraps with metadata and serializes to string. | ||
| * | ||
| * @param namespace - The controller namespace. | ||
| * @param key - The data key. | ||
| * @param value - The value to store (will be wrapped and serialized). | ||
| */ | ||
| async setItem(namespace: string, key: string, value: unknown): Promise<void> { | ||
| try { | ||
| // Build full key: storageService:namespace:key | ||
| const fullKey = `${STORAGE_KEY_PREFIX}${namespace}:${key}`; | ||
|
|
||
| // Wrap with metadata | ||
| const wrapper: StoredDataWrapper = { | ||
| timestamp: Date.now(), | ||
| data: value, | ||
| }; | ||
|
|
||
| await FilesystemStorage.setItem( | ||
| fullKey, | ||
| JSON.stringify(wrapper), | ||
| Device.isIos(), | ||
| ); | ||
| } catch (error) { | ||
| Logger.error(error as Error, { | ||
| message: `StorageService: Failed to set item: ${namespace}:${key}`, | ||
| }); | ||
| throw error; | ||
| } | ||
| }, | ||
|
|
||
| /** | ||
| * Remove an item from filesystem storage. | ||
| * | ||
| * @param namespace - The controller namespace. | ||
| * @param key - The data key. | ||
| */ | ||
| async removeItem(namespace: string, key: string): Promise<void> { | ||
| try { | ||
| // Build full key: storageService:namespace:key | ||
| const fullKey = `${STORAGE_KEY_PREFIX}${namespace}:${key}`; | ||
| await FilesystemStorage.removeItem(fullKey); | ||
| } catch (error) { | ||
| Logger.error(error as Error, { | ||
| message: `StorageService: Failed to remove item: ${namespace}:${key}`, | ||
| }); | ||
| throw error; | ||
| } | ||
| }, | ||
|
|
||
| /** | ||
| * Get all keys for a specific namespace. | ||
| * Filters keys by namespace prefix and returns without prefix. | ||
| * | ||
| * @param namespace - The namespace to get keys for. | ||
| * @returns Array of keys (without prefix) for this namespace. | ||
| */ | ||
| async getAllKeys(namespace: string): Promise<string[]> { | ||
| try { | ||
| const allKeys = await FilesystemStorage.getAllKeys(); | ||
|
|
||
| if (!allKeys) { | ||
| return []; | ||
| } | ||
|
|
||
| const prefix = `${STORAGE_KEY_PREFIX}${namespace}:`; | ||
|
|
||
| return allKeys | ||
| .filter((key) => key.startsWith(prefix)) | ||
| .map((key) => key.slice(prefix.length)); | ||
| } catch (error) { | ||
| Logger.error(error as Error, { | ||
| message: `StorageService: Failed to get keys for ${namespace}`, | ||
| }); | ||
| return []; | ||
| } | ||
| }, | ||
|
|
||
| /** | ||
| * Clear all items for a specific namespace. | ||
| * | ||
| * @param namespace - The namespace to clear. | ||
| */ | ||
| async clear(namespace: string): Promise<void> { | ||
| try { | ||
| const allKeys = await FilesystemStorage.getAllKeys(); | ||
|
|
||
| if (!allKeys) { | ||
| return; | ||
| } | ||
|
|
||
| const prefix = `${STORAGE_KEY_PREFIX}${namespace}:`; | ||
| const keysToDelete = allKeys.filter((key) => key.startsWith(prefix)); | ||
|
|
||
| await Promise.all( | ||
| keysToDelete.map((key) => FilesystemStorage.removeItem(key)), | ||
| ); | ||
|
|
||
| Logger.log( | ||
| `StorageService: Cleared ${keysToDelete.length} keys for ${namespace}`, | ||
| ); | ||
| } catch (error) { | ||
| Logger.error(error as Error, { | ||
| message: `StorageService: Failed to clear namespace ${namespace}`, | ||
| }); | ||
| throw error; | ||
| } | ||
| }, | ||
| }; | ||
|
|
||
| /** | ||
| * Initialize the storage service. | ||
| * | ||
| * @param request - The request object. | ||
| * @param request.controllerMessenger - The messenger to use for the service. | ||
| * @returns The initialized service. | ||
| */ | ||
| export const storageServiceInit: ControllerInitFunction< | ||
| StorageService, | ||
| StorageServiceMessenger | ||
| > = ({ controllerMessenger }) => { | ||
| const controller = new StorageService({ | ||
| messenger: controllerMessenger, | ||
| storage: mobileStorageAdapter, | ||
| }); | ||
|
|
||
| return { | ||
| controller, | ||
| }; | ||
| }; | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
31 changes: 31 additions & 0 deletions
31
app/core/Engine/messengers/storage-service-messenger.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| import { ExtendedMessenger } from '../../ExtendedMessenger'; | ||
| import { getStorageServiceMessenger } from './storage-service-messenger'; | ||
| import { MOCK_ANY_NAMESPACE, MockAnyNamespace } from '@metamask/messenger'; | ||
|
|
||
| describe('getStorageServiceMessenger', () => { | ||
| it('creates a messenger with StorageService namespace', () => { | ||
| const baseMessenger = new ExtendedMessenger<MockAnyNamespace, never, never>( | ||
| { | ||
| namespace: MOCK_ANY_NAMESPACE, | ||
| }, | ||
| ); | ||
|
|
||
| const messenger = getStorageServiceMessenger(baseMessenger); | ||
|
|
||
| expect(messenger).toBeDefined(); | ||
| }); | ||
|
|
||
| it('returns a messenger with correct namespace', () => { | ||
| const baseMessenger = new ExtendedMessenger<MockAnyNamespace, never, never>( | ||
| { | ||
| namespace: MOCK_ANY_NAMESPACE, | ||
| }, | ||
| ); | ||
|
|
||
| const messenger = getStorageServiceMessenger(baseMessenger); | ||
|
|
||
| // Verify messenger has correct structure by checking it has the expected methods | ||
| expect(typeof messenger.call).toBe('function'); | ||
| expect(typeof messenger.registerActionHandler).toBe('function'); | ||
| }); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| import { | ||
| Messenger, | ||
| MessengerActions, | ||
| MessengerEvents, | ||
| } from '@metamask/messenger'; | ||
| import { StorageServiceMessenger } from '@metamask-previews/storage-service'; | ||
|
|
||
| import { RootMessenger } from '../types'; | ||
|
|
||
| /** | ||
| * Get the StorageServiceMessenger for the StorageService. | ||
| * | ||
| * @param rootMessenger - The root messenger. | ||
| * @returns The StorageServiceMessenger. | ||
| */ | ||
| export function getStorageServiceMessenger( | ||
| rootMessenger: RootMessenger, | ||
| ): StorageServiceMessenger { | ||
| const messenger = new Messenger< | ||
| 'StorageService', | ||
| MessengerActions<StorageServiceMessenger>, | ||
| MessengerEvents<StorageServiceMessenger>, | ||
| RootMessenger | ||
| >({ | ||
| namespace: 'StorageService', | ||
| parent: rootMessenger, | ||
| }); | ||
| return messenger; | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.