Skip to content

[camera_avfoundation] enable more than 30 fps #7394

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
auto-submit merged 14 commits into from
Nov 5, 2024
Merged

[camera_avfoundation] enable more than 30 fps #7394

Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
6c769f9
enable more than 30 fps
misos1 Aug 7, 2024
4920305
Merge branch 'main' into more_fps
misos1 Aug 14, 2024
4f1483f
correctly set _previewSize
misos1 Aug 15, 2024
6c210c9
Merge branch 'main' into more_fps
misos1 Aug 21, 2024
30a4856
split camera changes from more_fps
misos1 Aug 21, 2024
5e1f249
extract format selection into function and more
misos1 Sep 5, 2024
5007358
revert reordering lockDevice with beginConfigurationForSession
misos1 Sep 5, 2024
65f8113
fix format
misos1 Sep 5, 2024
27868df
fix format
misos1 Sep 5, 2024
3983273
add comments and small changes
misos1 Sep 19, 2024
1d8f05a
fix format
misos1 Sep 19, 2024
cc632ec
change comments
misos1 Oct 12, 2024
f36d2dd
Merge branch 'main' into more_fps
misos1 Oct 12, 2024
0125d77
change activeFormat always if framesPerSecond is set
misos1 Oct 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix format
  • Loading branch information
@misos1
misos1 committed Sep 19, 2024
commit 1d8f05a45e9fa92beca65d5fbde19d4082066ff8
90 changes: 45 additions & 45 deletions ...s/camera/camera_avfoundation/ios/camera_avfoundation/Sources/camera_avfoundation/FLTCam.m
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,56 +155,56 @@ - (instancetype)initWithCameraName:(NSString *)cameraName

// Returns frame rate supported by format closest to targetFrameRate.
static double bestFrameRateForFormat(AVCaptureDeviceFormat *format, double targetFrameRate) {
double bestFrameRate = 0;
double minDistance = DBL_MAX;
for (AVFrameRateRange *range in format.videoSupportedFrameRateRanges) {
double frameRate = MIN(MAX(targetFrameRate, range.minFrameRate), range.maxFrameRate);
double distance = fabs(frameRate - targetFrameRate);
if (distance < minDistance) {
bestFrameRate = frameRate;
minDistance = distance;
}
}
return bestFrameRate;
double bestFrameRate = 0;
double minDistance = DBL_MAX;
for (AVFrameRateRange *range in format.videoSupportedFrameRateRanges) {
double frameRate = MIN(MAX(targetFrameRate, range.minFrameRate), range.maxFrameRate);
double distance = fabs(frameRate - targetFrameRate);
if (distance < minDistance) {
bestFrameRate = frameRate;
minDistance = distance;
}
}
return bestFrameRate;
}

// Finds format with same resolution as current activeFormat for which bestFrameRateForFormat
// returned frame rate closest to mediaSettings.framesPerSecond. Preferred are formats with the
// same subtype as current activeFormat. Sets this format as activeFormat and also updates
Copy link
Collaborator

@stuartmorgan-g stuartmorgan-g Sep 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about "Sets this format as the activeFormat in captureDevice and [...]"; I wasn't sure what activeFormat was referring to without reading the implementation.

// mediaSettings.framesPerSecond to value which bestFrameRateForFormat returned for that format.
static void selectBestFormatForRequestedFrameRate(
AVCaptureDevice *captureDevice, FCPPlatformMediaSettings *mediaSettings,
VideoDimensionsForFormat videoDimensionsForFormat) {
CMVideoDimensions targetResolution = videoDimensionsForFormat(captureDevice.activeFormat);
double targetFrameRate = mediaSettings.framesPerSecond.doubleValue;
FourCharCode preferredSubType =
CMFormatDescriptionGetMediaSubType(captureDevice.activeFormat.formatDescription);
AVCaptureDeviceFormat *bestFormat = captureDevice.activeFormat;
double bestFrameRate = bestFrameRateForFormat(bestFormat, targetFrameRate);
double minDistance = fabs(bestFrameRate - targetFrameRate);
BOOL isBestSubTypePreferred = YES;
for (AVCaptureDeviceFormat *format in captureDevice.formats) {
CMVideoDimensions resolution = videoDimensionsForFormat(format);
if (resolution.width != targetResolution.width ||
resolution.height != targetResolution.height) {
continue;
}
double frameRate = bestFrameRateForFormat(format, targetFrameRate);
double distance = fabs(frameRate - targetFrameRate);
FourCharCode subType = CMFormatDescriptionGetMediaSubType(format.formatDescription);
BOOL isSubTypePreferred = subType == preferredSubType;
if (distance < minDistance || (distance == minDistance && isSubTypePreferred &&
!isBestSubTypePreferred)) {
bestFormat = format;
bestFrameRate = frameRate;
minDistance = distance;
isBestSubTypePreferred = isSubTypePreferred;
}
}
if (![bestFormat isEqual:captureDevice.activeFormat]) {
captureDevice.activeFormat = bestFormat;
}
mediaSettings.framesPerSecond = @(bestFrameRate);
AVCaptureDevice *captureDevice, FCPPlatformMediaSettings *mediaSettings,
VideoDimensionsForFormat videoDimensionsForFormat) {
CMVideoDimensions targetResolution = videoDimensionsForFormat(captureDevice.activeFormat);
double targetFrameRate = mediaSettings.framesPerSecond.doubleValue;
FourCharCode preferredSubType =
CMFormatDescriptionGetMediaSubType(captureDevice.activeFormat.formatDescription);
AVCaptureDeviceFormat *bestFormat = captureDevice.activeFormat;
double bestFrameRate = bestFrameRateForFormat(bestFormat, targetFrameRate);
double minDistance = fabs(bestFrameRate - targetFrameRate);
BOOL isBestSubTypePreferred = YES;
for (AVCaptureDeviceFormat *format in captureDevice.formats) {
CMVideoDimensions resolution = videoDimensionsForFormat(format);
if (resolution.width != targetResolution.width ||
resolution.height != targetResolution.height) {
continue;
}
double frameRate = bestFrameRateForFormat(format, targetFrameRate);
double distance = fabs(frameRate - targetFrameRate);
FourCharCode subType = CMFormatDescriptionGetMediaSubType(format.formatDescription);
BOOL isSubTypePreferred = subType == preferredSubType;
if (distance < minDistance ||
(distance == minDistance && isSubTypePreferred && !isBestSubTypePreferred)) {
bestFormat = format;
bestFrameRate = frameRate;
minDistance = distance;
isBestSubTypePreferred = isSubTypePreferred;
}
}
if (![bestFormat isEqual:captureDevice.activeFormat]) {
Copy link
Collaborator

@stuartmorgan-g stuartmorgan-g Sep 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there evidence in docs or actual testing that this check is necessary? Standard convention for setters in Obj-C is for them to explicitly handle set-to-same-value as a no-op, so I would expect that we could just unconditionally call the setter and get exactly the same behavior.

Copy link
Contributor Author

@misos1 misos1 Oct 1, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will need to check this, I think it changes sessionPreset to AVCaptureSessionPresetInputPriority even if format is the same as it was (if I remember correctly), please wait with merging until I can get into this and other comments left.

Copy link
Contributor Author

@misos1 misos1 Oct 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it changes sessionPreset so the idea is to not change it if the format does not need any change. Although maybe it actually should be changed to prevent the session from changing to an active format which no longer supports fps set by the client: "When a client sets the session preset to anything other than AVCaptureSessionPresetInputPriority, the session resumes responsibility for configuring inputs and outputs, and is free to change its inputs' activeFormat as needed."

captureDevice.activeFormat = bestFormat;
}
mediaSettings.framesPerSecond = @(bestFrameRate);
}

- (instancetype)initWithMediaSettings:(FCPPlatformMediaSettings *)mediaSettings
Expand Down Expand Up @@ -601,8 +601,8 @@ - (AVCaptureDeviceFormat *)highestResolutionFormatForCaptureDevice:
NSUInteger pixelCount = height * width;
FourCharCode subType = CMFormatDescriptionGetMediaSubType(format.formatDescription);
BOOL isSubTypePreferred = subType == preferredSubType;
if (pixelCount > maxPixelCount || (pixelCount == maxPixelCount && isSubTypePreferred &&
!isBestSubTypePreferred)) {
if (pixelCount > maxPixelCount ||
(pixelCount == maxPixelCount && isSubTypePreferred && !isBestSubTypePreferred)) {
bestFormat = format;
maxPixelCount = pixelCount;
isBestSubTypePreferred = isSubTypePreferred;
Expand Down