Skip to content

Build hook spec #955

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
auto-submit merged 12 commits into from
Feb 9, 2024
Merged

Build hook spec #955

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
43950e2
A draft at a build-hook specification
jonasfj Jan 24, 2024
d75f6ab
Iterate on the v1 spec
dcharkes Jan 25, 2024
13e9ed6
Update pkgs/native_assets_builder/doc/protocol/v1.md
dcharkes Jan 25, 2024
6f59908
Update pkgs/native_assets_builder/doc/protocol/v1.md
dcharkes Jan 25, 2024
4bc8cbd
Update pkgs/native_assets_builder/doc/protocol/v1.md
dcharkes Jan 25, 2024
5e01f81
schema version
dcharkes Jan 25, 2024
f745db4
address more comments
dcharkes Jan 25, 2024
be58730
Add path_type concepts for native_assets.yaml
dcharkes Jan 25, 2024
e1e8754
Elaborate concepts
dcharkes Jan 25, 2024
7fb1fce
Address a bunch of comments
dcharkes Jan 31, 2024
ac5a7da
Suggestions on reformulating
mosuem Feb 2, 2024
746a74d
Apply change request
mosuem Feb 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Elaborate concepts
  • Loading branch information
@dcharkes
dcharkes committed Jan 25, 2024
commit e1e8754b1372ab474a0cbb883e727c8b398501af
37 changes: 19 additions & 18 deletions pkgs/native_assets_builder/doc/protocol/v1.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,33 +6,34 @@ Version 1: As of 2024-01-24 implemented behind `--enable-experiment=native-asset
### Concepts

#### Asset
A reference to a file that is identified by an `assetId`. There may be multiple files with the
same `assetId` with different characteristics such as `target` or `link_mode`.

Note that in v1 there are _only_ native code assets, no data assets.
Native code assets are dynamic libraries.
Native data assets will be blobs of data accessible as a byte list in Dart.
An asset is a file together with metadata that tells the Dart and Flutter SDK how to bundle such asset in an application bundle.

#### AssetId
An asset must have an `assetId`. Dart code that uses an asset, references the
asset using the `assetId`.
Currently, all such assets are dynamic libraries.

A package MUST prefix all assetIds it defines with: `package:<package>/`, this
ensures assets don't conflict between packages.
An asset must have an `assetId`. Dart code that uses an asset, references the asset using the `assetId`.
Copy link
Member

@mosuem mosuem Jan 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here and in the other items: You probably mean the metadata must contain? Also, what does using mean? I assume just referencing, as an asset is a file.

Copy link
Collaborator Author

@dcharkes dcharkes Jan 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, we have conflated the Asset class that is metadata, and the file. This probs requires a full rewrite.

Since we also have a "metadata" field in the build input and build output, we shoudl probably refrain from calling everything that is not the file metadata. Maybe we should call the file the "asset file".


Conventionally, an asset referenced from `lib/src/foo.dart` in `package:foo`
has `assetId = 'package:foo/src/foo.dart'`.
An asset must have a target os and target architecture. When accessing the asset at runtime, the asset for the current os and architecture is accessed.
Copy link
Member

@mosuem mosuem Jan 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought there is not necessarily a target architecture in dry run mode?

Copy link
Collaborator Author

@dcharkes dcharkes Jan 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not as input, but there is for the output. We should probably fix that in v2.


#### Target
An asset must have a link mode. Currently, the only supported link mode is dynamic. (Static linking of native code will be added later.)

A target specifies the operating system and architecture of the targeted platform.
An asset must specify a path type for dynamic linking.

List of `platform` and `architecture` tuples supported by the
Dart and Flutter SDKs to be specified below:
* `linux_x64`
* TODO: enumerate full list of supported platform/architecture tuples.
* `absolute` paths are absolute paths in the output dir (see below).
* `system` paths are expected to resolve on the target machine PATH. In this case the asset is not a file but only metadata.
Copy link
Member

@mosuem mosuem Jan 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case the asset is not a file but only metadata. well that is not really agreeing with what an asset is. I would update the definition above.

* `process` "paths" have no path, symbols are resolved in the current process. In this case the asset is not a file but only metadata.
Copy link
Member

@mosuem mosuem Jan 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"paths" have no path - What does this mean? What are paths? What are symbols - assets are files, right?

Copy link
Collaborator Author

@dcharkes dcharkes Jan 26, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah so code assets are already much more involved than "a file". Code assets can be metadata only. And some variants of that metadata do not require any path.

path_type is probably not the right name.

Maybe it should be dynamic_library_type. But the executable and process one don't even have a dynamic library.

So then it should be dynamic_linking_type. This also sounds weird, because the linking always is the same in a way.

Maybe it should be dynamic_resolution_type. That gets closer. But the problem here is that we're also abstracting over the resolution partially. Dart and Flutter repackage dynamic libraries. So in this protocol for absolute paths, it's the absolute path at the point of this protocol. But for system paths, process, and executable the metadata gets passed in unmodified.

So then it should maybe be something like:

- asset_id: ...
  dynamic_resolution:
    type: bundled_dylib
    # no path here, because that will be decided by Flutter/Dart when making the bundle.
  path: ... # Not nested in dynamic_resolution, it's the path for the protocol.
- asset_id: ...
  dynamic_resolution:
    type: system
    path: ... # Nested in dynamic_resolution, it's the metadata to be passed into linking.
- asset_id: ...
  dynamic_resolution:
    type: process
- asset_id: ...
  dynamic_resolution:
    type: executable

wdyt?

* `executable` "paths" have no path, symbols are resolved in the current executable. In this case the asset is not a file but only metadata.

#### AssetId
An asset must have an `assetId`. Dart code that uses an asset, references the asset using the `assetId`.

A package must prefix all assetIds it defines with: `package:<package>/`, this ensures assets don't conflict between packages.
Copy link
Member

@mosuem mosuem Jan 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So it seems an assetId is a string? Also, <package> is the current package?


Conventionally, an asset referenced from `lib/src/foo.dart` in `package:foo` has `assetId = 'package:foo/src/foo.dart'`.
Copy link
Member

@mosuem mosuem Jan 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the convention if two assets would be referenced from lib/src/foo.dart? I assume the assetId is required to be unique?

Copy link
Collaborator Author

@dcharkes dcharkes Jan 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, we don't have a convention yet.

Maybe:

  • 'package:foo/src/foo.dart/some_name' if you only use it in that file.
  • 'package:foo/some_name' if you use the asset in multiple dart files.


#### Target os and architecture

A target specifies the operating system and architecture of the targeted platform.

### `build.dart`
To bundle native assets with a package, that package must define a `build.dart`
Expand Down