Skip to content

Commit 759056b

Browse files
fabOnReactfacebook-github-bot
authored andcommitted
Override default Talkback automatic content grouping and generate a custom contentDescription (facebook#33690)
Summary: The Implementation of the functionality consists of: 1) Checking that an element has no contentDescription and no text and has other content to announce (role, state, etc.) which causes this issue (for ex. the accessibilityRole is announced before the contentDescription for ex. "Button, My text children component") 2) If Talkback finds no content to announce on the current node, a custom contentDescription is generated from the child elements that respect the following conditions: >If an AccessibilityNodeInfo is considered "actionable" (which Talkback defines as having clickable=true, longClickable=true, or focusable=true, or having AccessibilityActions for any of those), AND it has some content to read like a contentDescription or text, it will be considered focusable. >If an AccessibilityNodeInfo is considered "actionable" AND it does not have content to read like a contentDescription or text Talkback will parse descendant elements looking for non-focusable descendants to use as content. 3) implementation of a method getTalkbackDescription to generate the above contentDescription from child elements 4) over-ride parent contentDescription (accessibilityLabel) with the value returned from getTalkbackDescription Related [notes on Android: Role description is announced before the components text, rather than after https://github.com/facebook/react-native/issues/31042][51]. This issue fixes [https://github.com/facebook/react-native/issues/31042][50]. ## Changelog [Android] [Added] - Override default Talkback automatic content grouping and generate a custom contentDescription Pull Request resolved: facebook#33690 Test Plan: **PR Branch** [1]. Screenreader correctly announcing accessible/non-accessible items ([link][1]) [2]. Screenreader announces Pressability items ([link][2]) [3]. Button role is announced after child contentDescription with TouchableNativeFeedback ([link][3]) [4]. Testing for regressions in Accessibility Actions ([link][4]) [5]. Screenreader focuses on ScrollView Items ([link][5]) [6]. Recordings of complete test cases in rn-tester app main and pr branch ([link][6]) [9]. TouchableOpacity with TextInput child announces contentDescription before the Role ([link][9]) [10]. One of the child has accessibilityState (hasStateDescription triggers the announcement) ([link][10]) [11]. One of the child has accessibilityHint (hasText triggers the announcement) ([link][11]) **Main** [15]. The View does not announce the child component Text when accessibilityLabel is missing (automatic content grouping) ([link][15]) [8]. TouchableOpacity with child EditText annouces placeholder text before and after the role ([link][8]) [1]: fabOnReact/react-native-notes#14 (comment) "Screenreader correctly announcing accessible/non-accessible items" [2]: fabOnReact/react-native-notes#14 (comment) "Screenreader announces Pressability items" [3]: fabOnReact/react-native-notes#14 (comment) "Button role is announced after child contentDescription" [4]: fabOnReact/react-native-notes#14 (comment) "Testing for regressions in Accessibility Actions" [5]: fabOnReact/react-native-notes#14 (comment) "Screenreader focuses on ScrollView Items" [6]: fabOnReact/react-native-notes#14 (comment) "Recordings of complete test cases in rn-tester app main and pr branch" [7]: fabOnReact/react-native-notes#14 (comment) "TouchableOpacity with child EditText annouces Button role before the child contentDescription" [8]: fabOnReact/react-native-notes#14 (comment) "TouchableOpacity with child EditText annouces placholder text before and after the role" [9]: fabOnReact/react-native-notes#14 (comment) "TouchableOpacity with TextInput child announces contentDescription before the Role" [10]: fabOnReact/react-native-notes#14 (comment) "One of the child has accessibilityState (hasStateDescription triggers the announcement)" [11]: fabOnReact/react-native-notes#14 (comment) "One of the child has accessibilityHint (hasText triggers the announcement)" [15]: fabOnReact/react-native-notes#14 (comment) "The View does not announce the child component Text when accessibilityLabel is missing (automatic content grouping)" [50]: facebook#31042 "Android: Role description is announced before the components text, rather than after facebook#31042" [51]: fabOnReact/react-native-notes#14 "notes on Android: Role description is announced before the components text, rather than after facebook#31042" Reviewed By: cipolleschi Differential Revision: D39177512 Pulled By: blavalla fbshipit-source-id: 6bd0fba9c347bc14b3839e903184c86d2bcab3d2
1 parent fb84e09 commit 759056b

File tree

2 files changed

+455
-6
lines changed

2 files changed

+455
-6
lines changed

ReactAndroid/src/main/java/com/facebook/react/uimanager/ReactAccessibilityDelegate.java

Lines changed: 308 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,10 +16,13 @@
1616
import android.text.Layout;
1717
import android.text.Spannable;
1818
import android.text.Spanned;
19+
import android.text.TextUtils;
1920
import android.text.style.AbsoluteSizeSpan;
2021
import android.text.style.ClickableSpan;
2122
import android.view.View;
23+
import android.view.ViewGroup;
2224
import android.view.accessibility.AccessibilityEvent;
25+
import android.widget.EditText;
2326
import android.widget.TextView;
2427
import androidx.annotation.NonNull;
2528
import androidx.annotation.Nullable;
@@ -41,6 +44,7 @@
4144
import com.facebook.react.bridge.ReadableType;
4245
import com.facebook.react.bridge.UIManager;
4346
import com.facebook.react.bridge.WritableMap;
47+
import com.facebook.react.uimanager.ReactAccessibilityDelegate.AccessibilityRole;
4448
import com.facebook.react.uimanager.events.Event;
4549
import com.facebook.react.uimanager.events.EventDispatcher;
4650
import com.facebook.react.uimanager.util.ReactFindViewUtil;
@@ -59,6 +63,8 @@ public class ReactAccessibilityDelegate extends ExploreByTouchHelper {
5963
private static int sCounter = 0x3f000000;
6064
private static final int TIMEOUT_SEND_ACCESSIBILITY_EVENT = 200;
6165
private static final int SEND_EVENT = 1;
66+
private static final String delimiter = ", ";
67+
private static final int delimiterLength = delimiter.length();
6268

6369
public static final HashMap<String, Integer> sActionIdMap = new HashMap<>();
6470

@@ -356,6 +362,17 @@ public void onInitializeAccessibilityNodeInfo(View host, AccessibilityNodeInfoCo
356362
if (testId != null) {
357363
info.setViewIdResourceName(testId);
358364
}
365+
boolean missingContentDescription = TextUtils.isEmpty(info.getContentDescription());
366+
boolean missingText = TextUtils.isEmpty(info.getText());
367+
boolean missingTextAndDescription = missingContentDescription && missingText;
368+
boolean hasContentToAnnounce =
369+
accessibilityActions != null
370+
|| accessibilityState != null
371+
|| accessibilityLabelledBy != null
372+
|| accessibilityRole != null;
373+
if (missingTextAndDescription && hasContentToAnnounce) {
374+
info.setContentDescription(getTalkbackDescription(host, info));
375+
}
359376
}
360377

361378
@Override
@@ -767,4 +784,295 @@ private static class AccessibleLink {
767784

768785
return null;
769786
}
787+
788+
/**
789+
* Determines if the supplied {@link View} and {@link AccessibilityNodeInfoCompat} has any
790+
* children which are not independently accessibility focusable and also have a spoken
791+
* description.
792+
*
793+
* <p>NOTE: Accessibility services will include these children's descriptions in the closest
794+
* focusable ancestor.
795+
*
796+
* @param view The {@link View} to evaluate
797+
* @param node The {@link AccessibilityNodeInfoCompat} to evaluate
798+
* @return {@code true} if it has any non-actionable speaking descendants within its subtree
799+
*/
800+
public static boolean hasNonActionableSpeakingDescendants(
801+
@Nullable AccessibilityNodeInfoCompat node, @Nullable View view) {
802+
803+
if (node == null || view == null || !(view instanceof ViewGroup)) {
804+
return false;
805+
}
806+
807+
final ViewGroup viewGroup = (ViewGroup) view;
808+
for (int i = 0, count = viewGroup.getChildCount(); i < count; i++) {
809+
final View childView = viewGroup.getChildAt(i);
810+
811+
if (childView == null) {
812+
continue;
813+
}
814+
815+
final AccessibilityNodeInfoCompat childNode = AccessibilityNodeInfoCompat.obtain();
816+
try {
817+
ViewCompat.onInitializeAccessibilityNodeInfo(childView, childNode);
818+
819+
if (!childNode.isVisibleToUser()) {
820+
continue;
821+
}
822+
823+
if (isAccessibilityFocusable(childNode, childView)) {
824+
continue;
825+
}
826+
827+
if (isSpeakingNode(childNode, childView)) {
828+
return true;
829+
}
830+
} finally {
831+
if (childNode != null) {
832+
childNode.recycle();
833+
}
834+
}
835+
}
836+
837+
return false;
838+
}
839+
840+
/**
841+
* Returns whether the node has valid RangeInfo.
842+
*
843+
* @param node The node to check.
844+
* @return Whether the node has valid RangeInfo.
845+
*/
846+
public static boolean hasValidRangeInfo(@Nullable AccessibilityNodeInfoCompat node) {
847+
if (node == null) {
848+
return false;
849+
}
850+
851+
@Nullable final RangeInfoCompat rangeInfo = node.getRangeInfo();
852+
if (rangeInfo == null) {
853+
return false;
854+
}
855+
856+
final float maxProgress = rangeInfo.getMax();
857+
final float minProgress = rangeInfo.getMin();
858+
final float currentProgress = rangeInfo.getCurrent();
859+
final float diffProgress = maxProgress - minProgress;
860+
return (diffProgress > 0.0f)
861+
&& (currentProgress >= minProgress)
862+
&& (currentProgress <= maxProgress);
863+
}
864+
865+
/**
866+
* Returns whether the specified node has state description.
867+
*
868+
* @param node The node to check.
869+
* @return {@code true} if the node has state description.
870+
*/
871+
private static boolean hasStateDescription(@Nullable AccessibilityNodeInfoCompat node) {
872+
return node != null
873+
&& (!TextUtils.isEmpty(node.getStateDescription())
874+
|| node.isCheckable()
875+
|| hasValidRangeInfo(node));
876+
}
877+
878+
/**
879+
* Returns whether the supplied {@link View} and {@link AccessibilityNodeInfoCompat} would produce
880+
* spoken feedback if it were accessibility focused. NOTE: not all speaking nodes are focusable.
881+
*
882+
* @param view The {@link View} to evaluate
883+
* @param node The {@link AccessibilityNodeInfoCompat} to evaluate
884+
* @return {@code true} if it meets the criterion for producing spoken feedback
885+
*/
886+
public static boolean isSpeakingNode(
887+
@Nullable AccessibilityNodeInfoCompat node, @Nullable View view) {
888+
if (node == null || view == null) {
889+
return false;
890+
}
891+
892+
final int important = ViewCompat.getImportantForAccessibility(view);
893+
if (important == ViewCompat.IMPORTANT_FOR_ACCESSIBILITY_NO_HIDE_DESCENDANTS
894+
|| (important == ViewCompat.IMPORTANT_FOR_ACCESSIBILITY_NO && node.getChildCount() <= 0)) {
895+
return false;
896+
}
897+
898+
return hasText(node)
899+
|| hasStateDescription(node)
900+
|| node.isCheckable()
901+
|| hasNonActionableSpeakingDescendants(node, view);
902+
}
903+
904+
public static boolean hasText(@Nullable AccessibilityNodeInfoCompat node) {
905+
return node != null
906+
&& node.getCollectionInfo() == null
907+
&& (!TextUtils.isEmpty(node.getText())
908+
|| !TextUtils.isEmpty(node.getContentDescription())
909+
|| !TextUtils.isEmpty(node.getHintText()));
910+
}
911+
912+
/**
913+
* Determines if the provided {@link View} and {@link AccessibilityNodeInfoCompat} meet the
914+
* criteria for gaining accessibility focus.
915+
*
916+
* <p>Note: this is evaluating general focusability by accessibility services, and does not mean
917+
* this view will be guaranteed to be focused by specific services such as Talkback. For Talkback
918+
* focusability, see {@link #isTalkbackFocusable(View)}
919+
*
920+
* @param view The {@link View} to evaluate
921+
* @param node The {@link AccessibilityNodeInfoCompat} to evaluate
922+
* @return {@code true} if it is possible to gain accessibility focus
923+
*/
924+
public static boolean isAccessibilityFocusable(
925+
@Nullable AccessibilityNodeInfoCompat node, @Nullable View view) {
926+
if (node == null || view == null) {
927+
return false;
928+
}
929+
930+
// Never focus invisible nodes.
931+
if (!node.isVisibleToUser()) {
932+
return false;
933+
}
934+
935+
// Always focus "actionable" nodes.
936+
return node.isScreenReaderFocusable() || isActionableForAccessibility(node);
937+
}
938+
939+
/**
940+
* Returns whether a node is actionable. That is, the node supports one of {@link
941+
* AccessibilityNodeInfoCompat#isClickable()}, {@link AccessibilityNodeInfoCompat#isFocusable()},
942+
* or {@link AccessibilityNodeInfoCompat#isLongClickable()}.
943+
*
944+
* @param node The {@link AccessibilityNodeInfoCompat} to evaluate
945+
* @return {@code true} if node is actionable.
946+
*/
947+
public static boolean isActionableForAccessibility(@Nullable AccessibilityNodeInfoCompat node) {
948+
if (node == null) {
949+
return false;
950+
}
951+
952+
if (node.isClickable() || node.isLongClickable() || node.isFocusable()) {
953+
return true;
954+
}
955+
956+
final List actionList = node.getActionList();
957+
return actionList.contains(AccessibilityNodeInfoCompat.ACTION_CLICK)
958+
|| actionList.contains(AccessibilityNodeInfoCompat.ACTION_LONG_CLICK)
959+
|| actionList.contains(AccessibilityNodeInfoCompat.ACTION_FOCUS);
960+
}
961+
962+
/**
963+
* Returns a cached instance if such is available otherwise a new one.
964+
*
965+
* @param view The {@link View} to derive the AccessibilityNodeInfo properties from.
966+
* @return {@link FlipperObject} containing the properties.
967+
*/
968+
@Nullable
969+
public static AccessibilityNodeInfoCompat createNodeInfoFromView(View view) {
970+
if (view == null) {
971+
return null;
972+
}
973+
974+
final AccessibilityNodeInfoCompat nodeInfo = AccessibilityNodeInfoCompat.obtain();
975+
976+
// For some unknown reason, Android seems to occasionally throw a NPE from
977+
// onInitializeAccessibilityNodeInfo.
978+
try {
979+
ViewCompat.onInitializeAccessibilityNodeInfo(view, nodeInfo);
980+
} catch (NullPointerException e) {
981+
if (nodeInfo != null) {
982+
nodeInfo.recycle();
983+
}
984+
return null;
985+
}
986+
987+
return nodeInfo;
988+
}
989+
990+
/**
991+
* Creates the text that Google's TalkBack screen reader will read aloud for a given {@link View}.
992+
* This may be any combination of the {@link View}'s {@code text}, {@code contentDescription}, and
993+
* the {@code text} and {@code contentDescription} of any ancestor {@link View}.
994+
*
995+
* <p>This description is generally ported over from Google's TalkBack screen reader, and this
996+
* should be kept up to date with their implementation (as much as necessary). Details can be seen
997+
* in their source code here:
998+
*
999+
* <p>https://github.com/google/talkback/compositor/src/main/res/raw/compositor.json - search for
1000+
* "get_description_for_tree", "append_description_for_tree", "description_for_tree_nodes"
1001+
*
1002+
* @param view The {@link View} to evaluate.
1003+
* @param info The default {@link AccessibilityNodeInfoCompat}.
1004+
* @return {@code String} representing what talkback will say when a {@link View} is focused.
1005+
*/
1006+
@Nullable
1007+
public static CharSequence getTalkbackDescription(
1008+
View view, @Nullable AccessibilityNodeInfoCompat info) {
1009+
final AccessibilityNodeInfoCompat node =
1010+
info == null ? createNodeInfoFromView(view) : AccessibilityNodeInfoCompat.obtain(info);
1011+
1012+
if (node == null) {
1013+
return null;
1014+
}
1015+
try {
1016+
final CharSequence contentDescription = node.getContentDescription();
1017+
final CharSequence nodeText = node.getText();
1018+
1019+
final boolean hasNodeText = !TextUtils.isEmpty(nodeText);
1020+
final boolean isEditText = view instanceof EditText;
1021+
1022+
StringBuilder talkbackSegments = new StringBuilder();
1023+
1024+
// EditText's prioritize their own text content over a contentDescription so skip this
1025+
if (!TextUtils.isEmpty(contentDescription) && (!isEditText || !hasNodeText)) {
1026+
// next add content description
1027+
talkbackSegments.append(contentDescription);
1028+
return talkbackSegments;
1029+
}
1030+
1031+
// EditText
1032+
if (hasNodeText) {
1033+
// skipped status checks above for EditText
1034+
1035+
// description
1036+
talkbackSegments.append(nodeText);
1037+
return talkbackSegments;
1038+
}
1039+
1040+
// If there are child views and no contentDescription the text of all non-focusable children,
1041+
// comma separated, becomes the description.
1042+
if (view instanceof ViewGroup) {
1043+
final StringBuilder concatChildDescription = new StringBuilder();
1044+
final ViewGroup viewGroup = (ViewGroup) view;
1045+
1046+
for (int i = 0, count = viewGroup.getChildCount(); i < count; i++) {
1047+
final View child = viewGroup.getChildAt(i);
1048+
1049+
final AccessibilityNodeInfoCompat childNodeInfo = AccessibilityNodeInfoCompat.obtain();
1050+
ViewCompat.onInitializeAccessibilityNodeInfo(child, childNodeInfo);
1051+
1052+
if (isSpeakingNode(childNodeInfo, child)
1053+
&& !isAccessibilityFocusable(childNodeInfo, child)) {
1054+
CharSequence childNodeDescription = getTalkbackDescription(child, null);
1055+
if (!TextUtils.isEmpty(childNodeDescription)) {
1056+
concatChildDescription.append(childNodeDescription + delimiter);
1057+
}
1058+
}
1059+
childNodeInfo.recycle();
1060+
}
1061+
1062+
return removeFinalDelimiter(concatChildDescription);
1063+
}
1064+
1065+
return null;
1066+
} finally {
1067+
node.recycle();
1068+
}
1069+
}
1070+
1071+
private static String removeFinalDelimiter(StringBuilder builder) {
1072+
int end = builder.length();
1073+
if (end > 0) {
1074+
builder.delete(end - delimiterLength, end);
1075+
}
1076+
return builder.toString();
1077+
}
7701078
}

0 commit comments

Comments
 (0)