This page introduces tips and limitations regarding creating test cases for mobile Flutter apps with MagicPod.
1. Prerequisites
- Flutter 3.3 or later
- Xcode 14.1 or later
In addition to the requirements above, make sure that your Flutter app complies with this guideline.
2. Status of operation of test steps
We have tested operations of the main test steps involved in testing Flutter apps on cloud devices, Browserstack and SauceLabs. The following icons are used.
✅ | Confirmed to work |
⚠️ | Partially problematic |
❓ | Unconfirmed to work |
Note that a Flutter app built with the following commands is used to check the operation.
- .apk file: flutter build apk --debug
- .aab file: flutter build appbundle --debug
- .app file: flutter build ios --simulator
- .ipa file: flutter build ipa --export-method development
Cloud device
Type | Command | Android | iOS |
UI operation | Tap | ✅ | ✅ |
Double tap | ✅ | ⚠️ | |
Flick | ✅ | ✅ | |
Drag & drop | ✅ | ⚠️ | |
Long press & drag | ✅ | ✅ | |
Pinch out | ✅ | ✅ | |
Pinch in | ✅ | ✅ | |
Swipe | ✅ | ✅ | |
Draw a figure by freehand | ✅ | ✅ | |
Continue swipe | ✅ | ✅ | |
Long tap | ✅ | ⚠️ | |
Tap specified position | ✅ | ✅ | |
Multi touch action at the screen center |
❓ | ✅ | |
UI input | Input | ✅ | ✅ |
Input to picker | ❓ | ❓ | |
Input keyboard key | ✅ | ✅ | |
Toggle switch | ✅ | ✅ | |
Others | Store value | ⚠️ | ✅ |
WebView testing | ✅ | ✅ |
Browserstack
Type | Command | Android | iOS |
UI operation | Tap | ✅ | ✅ |
Double tap | ✅ | ⚠️ | |
Flick | ✅ | ✅ | |
Drag & drop | ✅ | ⚠️ | |
Long press & drag | ✅ | ✅ | |
Pinch out | ✅ | ✅ | |
Pinch in | ✅ | ✅ | |
Swipe | ✅ | ✅ | |
Draw a figure by freehand | ✅ | ✅ | |
Continue swipe | ✅ | ✅ | |
Long tap | ✅ | ⚠️ | |
Tap specified position | ✅ | ✅ | |
Multi touch action at the screen center |
❓ | ✅ | |
UI input | Input | ✅ | ✅ |
Input to picker | ❓ | ❓ | |
Input keyboard key | ✅ | ✅ | |
Toggle switch | ✅ | ✅ | |
Others | Store value | ⚠️ | ✅ |
WebView testing | ⚠️ | ⚠️ |
SauceLabs
Type | Command | Android | iOS |
UI operation | Tap | ✅ | ✅ |
Double tap | ✅ | ✅ | |
Flick | ✅ | ✅ | |
Drag & drop | ✅ | ⚠️ | |
Long press & drag | ✅ | ⚠️ | |
Pinch out | ✅ | ⚠️ | |
Pinch in | ✅ | ⚠️ | |
Swipe | ✅ | ✅ | |
Draw a figure by freehand | ✅ | ✅ | |
Continue swipe | ✅ | ✅ | |
Long tap | ✅ | ⚠️ | |
Tap specified position | ✅ | ✅ | |
Multi touch action at the screen center |
❓ | ✅ | |
UI input | Input | ✅ | ✅ |
Input to picker | ❓ | ❓ | |
Input keyboard key | ✅ | ✅ | |
Toggle switch | ✅ | ✅ | |
Others | Store value | ⚠️ | ✅ |
WebView testing | ⚠️ | ⚠️ |
3. Tips for creating test cases
3.1. Common to iOS and Android
3.1.1. Testing both iOS and Android apps in one test case
Like building multi-platform Flutter apps, you may want to run one test case on both platforms. In such cases, please refer to https://support.magic-pod.com/hc/en-us/articles/16904866116249#native_apps.
3.1.2. Cases of text input failure
Text input may fail when the focus is not on entering a text field. In this case, add a Tap command before the Input command.
- Depending on the implementation of the application, the locators required before and after the tap may change. In such cases, you can assign a common accessibility ID (https://support.magic-pod.com/hc/en-us/articles/16904866116249#assign_common_accessibility_id) or utilize XPath improvements (https://support.magic-pod.com/hc/en-us/articles/16904866116249#improve_xpath) to reuse the locator before and after a tap.
- On iOS, it may not be possible to focus on a text field with a Tap command. In this case, change the Tap command to a Tap specified position command and ensure that the text field will have focus.
- Even after focusing on a text field, text input may still fail (especially on iOS). In this case, consider changing the Wait policy to "Put emphasis on stability".
3.2. iOS
3.2.1. Cases of tap failure
- Tap operations may fail due to a problem Xcode (https://github.com/appium/appium/issues/16913). In this case, change the Tap command to a Tap specified position command.
- If the Tap specified position also fails, consider changing the Wait policy to "Put emphasis on stability".
3.3 Android
3.3.1. Unable to retrieve UI element values, empty strings are being retrieved
Due to changes on Flutter side (https://github.com/flutter/flutter/issues/124493), you may not be able to retrieve UI element values in Flutter 3.3.10 or later. In this case, you might be able to retrieve an equivalent UI element value by using the content-desc or hint attribute.
However, a content-desc attribute may differ from a value of a UI element displayed on the screen, as this information is originally used for speech-to-text features for people with sight impairments.
4. Limitations
- As of 2023/05, MagicPod does not support self-healing for UI captured on different platforms. For example, if you use a locator acquired on iOS and the locator does not find an element on Android, self-healing will not be triggered.
- As of 2023/05, MagicPod does not support interacting elements on the basis of Key class.
- On iOS, if a Long press & drag command is executed more than once on the same screen, the second and subsequent Long press & drag commands may fail.