-
-
Notifications
You must be signed in to change notification settings - Fork 4.7k
feat: TaskProcessing API #45094
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
feat: TaskProcessing API #45094
Changes from 1 commit
Commits
Show all changes
63 commits
Select commit
Hold shift + click to select a range
00894e2
feat: first pass at TaskProcessing API
marcelklehr 8352b27
fix: weed out some psalm errors and run cs:fix
marcelklehr 4b2acee
test: Add OldTasksShouldBeCleanedUp test
marcelklehr 17486ad
fix: Add RemoveOldTasksBackgroundJob to repair step that instantiates it
marcelklehr ee7592f
fix: Run cs:fix
marcelklehr 44b896f
feat: TaskProcessing OCS API
marcelklehr b2b93e4
feat: Add getFileContents endpoint to TaskProcessing OCS API
marcelklehr 86317bb
refactor: Move validation to EShapeType
marcelklehr 29cbb3c
chore: Run cs:fix
marcelklehr 3b0925a
chore: Regenerate openapi.json
marcelklehr 1c033ae
fix(IRootFolder): Add getAppDataDirectoryName method
marcelklehr 7a94798
fix: Fix psalm issues
marcelklehr 8e56626
feat: Add ExApp endpoints
marcelklehr 5031a2e
fix: Typo
marcelklehr 843bb62
fix: LazyRoot missing method
marcelklehr c9ea537
fix: Fix psalm issues
marcelklehr b150d77
refactor: rename getTaskType to getTaskTypeId
marcelklehr 4b954d2
fix: Wire-up the new classes
marcelklehr eebeb82
fix: Small fixes
marcelklehr bd5dfd0
test: Add more tests for legacy pass-through
marcelklehr 3593d9b
fix: psalm issue
marcelklehr 9a2cd6b
fix: Expose task type on CoreTaskProcessingTask json
marcelklehr 928d04f
fix: oc_taskProcessing_tasks.identifier: notnull = false
marcelklehr 9effb55
chore: update openapi.json
marcelklehr 996e507
feat: Remove stale files in AppData as well
marcelklehr 6203c1c
fix: Check if user is authorized to use the files they mentioned
marcelklehr ef44af1
test: Test file authorization check
marcelklehr 8ccb29a
fix: psalm issues
marcelklehr a5053d3
fix: Run cs:fix
marcelklehr b85a0ed
fix: Update autoloaders
marcelklehr 5de42a5
fix: Don't use dynamic property
marcelklehr 2c87809
fix: address review comments
marcelklehr ef61c50
fix: address review comments
marcelklehr ec27c53
fix: address review comments
marcelklehr fff2fb8
fix: psalm issue
marcelklehr 4a3b9b8
refactor: identifier is now customId/custom_id
marcelklehr 20c09d1
fix: Don't check in barmani plugin in composer autoloader
marcelklehr beeee26
fix: bump OC_Version
marcelklehr 4d9a0ea
fix: update openai specs
marcelklehr f2ab6cb
fix: fix psalm issues
marcelklehr c079a61
feat: Add cancel endpoint to OCS API
marcelklehr a045e0c
fix: fix migration
marcelklehr 0e06d64
Update core/Migrations/Version30000Date20240429122720.php
marcelklehr f6f4965
fix: fix tests
marcelklehr eff862b
fix: run cs:fix
marcelklehr ec94a67
fix(ocs): change /tasktypes response to combine optional and non-opti…
marcelklehr 19a0aae
fix(TextToImage): Allow leaving the resources open
marcelklehr f3a88f0
fix(OCS-API): No csrf required for /tasks/taskId/file/fileId
marcelklehr a8afa7f
fix(OCS-API): Add endpoint to list user tasks
marcelklehr 9cc1a01
test: Put input files in user storage
marcelklehr 715245a
fix: run cs:fix
marcelklehr e4b1d23
fix(Manager#fillInputData): Load user folder if needed
marcelklehr c020490
Update lib/private/TaskProcessing/Manager.php
marcelklehr b11052f
Update lib/public/TaskProcessing/IManager.php
marcelklehr ac36c78
fix(SynchronousBackgroundJob): Only reschedule when needed
marcelklehr a9a2cbf
feat: Add some new task types
marcelklehr c1f84aa
fix(Manager#fillInputs): Try to setup user FS before access file inputs
marcelklehr 4ce5aaf
chore: Check in autoloader updates
marcelklehr 9864fc8
chore: fix htaccess
marcelklehr 8d06338
fix: Fix pass-through stt provider
marcelklehr f5a8bda
Update core/ResponseDefinitions.php
marcelklehr cac812d
fix: address review comments
marcelklehr 6c4992d
fix: expose lastUpdated in OCS API
marcelklehr File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
feat: TaskProcessing OCS API
Signed-off-by: Marcel Klehr <[email protected]>
- Loading branch information
commit 44b896f999fb1a8ed1b6c5bdfd4db6ca01f7b193
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,214 @@ | ||
| <?php | ||
|
|
||
| declare(strict_types=1); | ||
|
|
||
| /** | ||
| * @copyright Copyright (c) 2024 Marcel Klehr <[email protected]> | ||
| * | ||
| * @author Marcel Klehr <[email protected]> | ||
| * | ||
| * @license GNU AGPL version 3 or any later version | ||
| * | ||
| * This program is free software: you can redistribute it and/or modify | ||
| * it under the terms of the GNU Affero General Public License as | ||
| * published by the Free Software Foundation, either version 3 of the | ||
| * License, or (at your option) any later version. | ||
| * | ||
| * This program is distributed in the hope that it will be useful, | ||
| * but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
| * GNU Affero General Public License for more details. | ||
| * | ||
| * You should have received a copy of the GNU Affero General Public License | ||
| * along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
| */ | ||
|
|
||
|
|
||
| namespace OC\Core\Controller; | ||
|
|
||
| use OCA\Core\ResponseDefinitions; | ||
| use OCP\AppFramework\Http; | ||
| use OCP\AppFramework\Http\Attribute\AnonRateLimit; | ||
| use OCP\AppFramework\Http\Attribute\ApiRoute; | ||
| use OCP\AppFramework\Http\Attribute\NoAdminRequired; | ||
| use OCP\AppFramework\Http\Attribute\PublicPage; | ||
| use OCP\AppFramework\Http\Attribute\UserRateLimit; | ||
| use OCP\AppFramework\Http\DataResponse; | ||
| use OCP\Common\Exception\NotFoundException; | ||
| use OCP\IL10N; | ||
| use OCP\IRequest; | ||
| use OCP\PreConditionNotMetException; | ||
| use OCP\TaskProcessing\Exception\ValidationException; | ||
| use OCP\TaskProcessing\ShapeDescriptor; | ||
| use OCP\TaskProcessing\Task; | ||
| use Psr\Container\ContainerInterface; | ||
| use Psr\Log\LoggerInterface; | ||
|
|
||
| /** | ||
| * @psalm-import-type CoreTaskProcessingTask from ResponseDefinitions | ||
| * @psalm-import-type CoreTaskProcessingTaskType from ResponseDefinitions | ||
| */ | ||
| class TaskProcessingApiController extends \OCP\AppFramework\OCSController { | ||
| public function __construct( | ||
| string $appName, | ||
| IRequest $request, | ||
| private \OCP\TaskProcessing\IManager $taskProcessingManager, | ||
| private IL10N $l, | ||
| private ?string $userId, | ||
| private ContainerInterface $container, | ||
| private LoggerInterface $logger, | ||
| ) { | ||
| parent::__construct($appName, $request); | ||
| } | ||
|
|
||
| /** | ||
| * This endpoint returns all available TaskProcessing task types | ||
| * | ||
| * @return DataResponse<Http::STATUS_OK, array{types: array<string, CoreTaskProcessingTaskType>}> | ||
| * []}, array{}> | ||
| * | ||
| * 200: Task types returned | ||
| */ | ||
| #[PublicPage] | ||
| #[ApiRoute(verb: 'GET', url: '/tasktypes', root: '/taskprocessing')] | ||
| public function taskTypes(): DataResponse { | ||
| $taskTypes = $this->taskProcessingManager->getAvailableTaskTypes(); | ||
|
|
||
| /** @var string $typeClass */ | ||
| foreach ($taskTypes as $taskType) { | ||
| $taskType['inputShape'] = array_map(fn (ShapeDescriptor $descriptor) => $descriptor->jsonSerialize(), $taskType['inputShape']); | ||
| $taskType['optionalInputShape'] = array_map(fn (ShapeDescriptor $descriptor) => $descriptor->jsonSerialize(), $taskType['optionalInputShape']); | ||
| $taskType['outputShape'] = array_map(fn (ShapeDescriptor $descriptor) => $descriptor->jsonSerialize(), $taskType['outputShape']); | ||
| $taskType['optionalOutputShape'] = array_map(fn (ShapeDescriptor $descriptor) => $descriptor->jsonSerialize(), $taskType['optionalOutputShape']); | ||
| } | ||
|
|
||
| return new DataResponse([ | ||
| 'types' => $taskTypes, | ||
| ]); | ||
| } | ||
|
|
||
| /** | ||
| * This endpoint allows scheduling a task | ||
| * | ||
| * @param string $input Input text | ||
|
||
| * @param string $type Type of the task | ||
| * @param string $appId ID of the app that will execute the task | ||
| * @param string $identifier An arbitrary identifier for the task | ||
| * | ||
| * @return DataResponse<Http::STATUS_OK, array{task: CoreTaskProcessingTask}, array{}>|DataResponse<Http::STATUS_INTERNAL_SERVER_ERROR|Http::STATUS_BAD_REQUEST|Http::STATUS_PRECONDITION_FAILED, array{message: string}, array{}> | ||
|
||
| * | ||
| * 200: Task scheduled successfully | ||
| * 400: Scheduling task is not possible | ||
| * 412: Scheduling task is not possible | ||
| */ | ||
| #[PublicPage] | ||
| #[UserRateLimit(limit: 20, period: 120)] | ||
| #[AnonRateLimit(limit: 5, period: 120)] | ||
| #[ApiRoute(verb: 'POST', url: '/schedule', root: '/taskprocessing')] | ||
| public function schedule(array $input, string $type, string $appId, string $identifier = ''): DataResponse { | ||
| $task = new Task($type, $input, $appId, $this->userId, $identifier); | ||
|
||
| try { | ||
| $this->taskProcessingManager->scheduleTask($task); | ||
|
|
||
| $json = $task->jsonSerialize(); | ||
|
|
||
| return new DataResponse([ | ||
| 'task' => $json, | ||
| ]); | ||
| } catch (PreConditionNotMetException) { | ||
| return new DataResponse(['message' => $this->l->t('The given provider is not available')], Http::STATUS_PRECONDITION_FAILED); | ||
| } catch (ValidationException $e) { | ||
| return new DataResponse(['message' => $e->getMessage()], Http::STATUS_BAD_REQUEST); | ||
| } catch (\OCP\TaskProcessing\Exception\Exception $e) { | ||
| return new DataResponse(['message' => 'Internal server error'], Http::STATUS_INTERNAL_SERVER_ERROR); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * This endpoint allows checking the status and results of a task. | ||
| * Tasks are removed 1 week after receiving their last update. | ||
| * | ||
| * @param int $id The id of the task | ||
| * | ||
| * @return DataResponse<Http::STATUS_OK, array{task: CoreTaskProcessingTask}, array{}>|DataResponse<Http::STATUS_NOT_FOUND|Http::STATUS_INTERNAL_SERVER_ERROR, array{message: string}, array{}> | ||
| * | ||
| * 200: Task returned | ||
| * 404: Task not found | ||
| */ | ||
| #[PublicPage] | ||
| #[ApiRoute(verb: 'GET', url: '/task/{id}', root: '/taskprocessing')] | ||
| public function getTask(int $id): DataResponse { | ||
| try { | ||
| $task = $this->taskProcessingManager->getUserTask($id, $this->userId); | ||
|
|
||
| $json = $task->jsonSerialize(); | ||
|
|
||
| return new DataResponse([ | ||
| 'task' => $json, | ||
| ]); | ||
| } catch (NotFoundException $e) { | ||
| return new DataResponse(['message' => $this->l->t('Task not found')], Http::STATUS_NOT_FOUND); | ||
| } catch (\RuntimeException $e) { | ||
| return new DataResponse(['message' => $this->l->t('Internal error')], Http::STATUS_INTERNAL_SERVER_ERROR); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * This endpoint allows to delete a scheduled task for a user | ||
| * | ||
| * @param int $id The id of the task | ||
| * | ||
| * @return DataResponse<Http::STATUS_OK, array{task: CoreTaskProcessingTask}, array{}>|DataResponse<Http::STATUS_NOT_FOUND|Http::STATUS_INTERNAL_SERVER_ERROR, array{message: string}, array{}> | ||
| * | ||
| * 200: Task returned | ||
| * 404: Task not found | ||
| */ | ||
| #[NoAdminRequired] | ||
| #[ApiRoute(verb: 'DELETE', url: '/task/{id}', root: '/taskprocessing')] | ||
| public function deleteTask(int $id): DataResponse { | ||
| try { | ||
| $task = $this->taskProcessingManager->getUserTask($id, $this->userId); | ||
|
|
||
| $this->taskProcessingManager->deleteTask($task); | ||
|
|
||
| $json = $task->jsonSerialize(); | ||
|
|
||
| return new DataResponse([ | ||
| 'task' => $json, | ||
| ]); | ||
| } catch (\OCP\TaskProcessing\Exception\NotFoundException $e) { | ||
| return new DataResponse(['message' => $this->l->t('Task not found')], Http::STATUS_NOT_FOUND); | ||
| } catch (\OCP\TaskProcessing\Exception\Exception $e) { | ||
| return new DataResponse(['message' => $this->l->t('Internal error')], Http::STATUS_INTERNAL_SERVER_ERROR); | ||
| } | ||
| } | ||
|
|
||
|
|
||
| /** | ||
| * This endpoint returns a list of tasks of a user that are related | ||
| * with a specific appId and optionally with an identifier | ||
| * | ||
| * @param string $appId ID of the app | ||
| * @param string|null $identifier An arbitrary identifier for the task | ||
| * @return DataResponse<Http::STATUS_OK, array{tasks: CoreTaskProcessingTask[]}, array{}>|DataResponse<Http::STATUS_INTERNAL_SERVER_ERROR, array{message: string}, array{}> | ||
| * | ||
| * 200: Task list returned | ||
| */ | ||
| #[NoAdminRequired] | ||
| #[ApiRoute(verb: 'GET', url: '/tasks/app/{appId}', root: '/taskprocessing')] | ||
| public function listTasksByApp(string $appId, ?string $identifier = null): DataResponse { | ||
| try { | ||
| $tasks = $this->taskProcessingManager->getUserTasksByApp($this->userId, $appId, $identifier); | ||
| /** @var CoreTaskProcessingTask[] $json */ | ||
| $json = array_map(static function (Task $task) { | ||
| return $task->jsonSerialize(); | ||
| }, $tasks); | ||
|
|
||
| return new DataResponse([ | ||
| 'tasks' => $json, | ||
| ]); | ||
| } catch (\RuntimeException $e) { | ||
| return new DataResponse(['message' => $this->l->t('Internal error')], Http::STATUS_INTERNAL_SERVER_ERROR); | ||
| } | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.