Skip to content
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions docs/workflow/testing/libraries/testing-android.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Testing Libraries on Android

To build the tests and run them on Android (devices or emulators) you need the following prerequisites.

- [Android NDK](https://developer.android.com/ndk/downloads)
- [Android SDK](https://developer.android.com/studio)
- OpenJDK

(TODO: provide command-line steps to download them with dependencies)

Once SDKs are downloaded, set `ANDROID_NDK_ROOT` and `ANDROID_SDK_ROOT`.
Example:
```
export ANDROID_SDK_ROOT=/Users/egorbo/Library/Android/sdk
export ANDROID_NDK_ROOT=/Users/egorbo/android-ndk-r21b
```

Next, we need OpenSSL binaries and headers, we haven't properly integrated this dependency yet (it's an ongoing discussion) but there is a workaround:

- Download and unzip https://maven.google.com/com/android/ndk/thirdparty/openssl/1.1.1g-alpha-1/openssl-1.1.1g-alpha-1.aar
- Set these env variables:
```
GOOGLE_OPENSSL=/Users/egorbo/prj/openssl-1.1.1g-alpha-1.aar/prefab/modules
export AndroidOpenSslHeaders="$GOOGLE_OPENSSL/ssl/include"
export AndroidOpenSslCryptoLib="$GOOGLE_OPENSSL/crypto/libs/android.x86_64/libcrypto.so"
export AndroidOpenSslLib="$GOOGLE_OPENSSL/ssl/libs/android.x86_64/libssl.so"
```
**IMPORTANT:** make sure correct ABIs are used in the path, e.g. `-arch x64` -> `android.x86_64`

Now we're ready to build everything for Android:
```
./build.sh mono+libs -os Android -arch x64
```
and even run tests one by one for each test suite:
```
./build.sh libs.tests -os Android -arch x64 -test
```
Make sure an emulator is booted (see `AVD Manager`) or a device is plugged in and unlocked.

**NOTE**: Xharness doesn't run any UI on Android and runs tests using headless testing API so the device/emulator won't show anything (but still must stay active).

### Running individual test suites
- The following shows how to run tests for a specific library
```
./dotnet.sh build /t:Test src/libraries/System.Numerics.Vectors/tests /p:TargetOS=Android /p:TargetArchitecture=x64
```

### How the tests work
Android app is basically a [Java Instrumentation](https://github.com/dotnet/runtime/blob/master/src/mono/msbuild/AndroidAppBuilder/Templates/MonoRunner.java) and a simple Activity that inits the Mono Runtime via JNI. This Mono Runtime starts a simple xunit test
runner called XHarness TestRunner which runs tests for all *.Tests.dll libs in the bundle. There is also XHarness.CLI tool with ADB inside to deploy `*.apk` to a target (device or emulator) and obtain logs once tests are completed.

### Obtaining the logs
XHarness for Android doesn't talk much and only saves tests result to a file once tests finished but you can also subscribe to live logs via the following command:
```
adb logcat -s "DOTNET"
```
Or simply open `logcat` window in Android Studio or Visual Stuido.

### Known Issues
- We don't support `-os Android` on Windows yet (`WSL` can be used instead)
32 changes: 32 additions & 0 deletions docs/workflow/testing/libraries/testing-apple.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Testing Libraries on iOS and tvOS

In order to build libraries and tests for iOS or tvOS you only need some fresh version of XCode installed (e.g. 11.3 or higher).

Build Libraries for iOS:
```
./build.sh mono+libs -os iOS -arch x64
```
Run tests one by one for each test suite on a simulator:
```
./build.sh libs.tests -os iOS -arch x64 -test
```
Unlike XHarness-Android, XHarness for iOS is able to boot simulators by its own.

### Running individual test suites
- The following shows how to run tests for a specific library on a simulator
```
cd src/libraries/System.Numerics.Vectors/tests
./dotnet.sh build /t:Test /p:TargetOS=iOS /p:TargetArchitecture=x64
```
for devices you need to specify `DevTeamProvisioning`:
```
./dotnet.sh build /t:Test /p:TargetOS=iOS /p:TargetArchitecture=arm64 /p:DevTeamProvisioning=...
```
AppleAppBuilder generates temp Xcode projects you can manually open and resolve provisioning there using native UI and deploy to your devices and start debugging.

### How the tests work
iOS/tvOS *.app (or *.ipa) is basically a simple [ObjC app](https://github.com/dotnet/runtime/blob/master/src/mono/msbuild/AppleAppBuilder/Templates/main-console.m) that inits the Mono Runtime. This Mono Runtime starts a simple xunit test
runner called XHarness TestRunner which runs tests for all *.Tests.dll libs in the bundle. There is also XHarness.CLI tool with `mlaunch` inside to deploy `*.app` and `*.ipa` to a target (device or simulator) and listens for logs via sockets.

### Known Issues
- Most of the test suites crash on devices due to [#35674)(https://github.com/dotnet/runtime/issues/35674)