Start Guide: 3. How to Create a Test - Part 2 (Mobile App)

Continuing on, we shall explain the basic method of creating a test using the mobile app test version of MagicPod.

1. Setting daily build + batch test run schedules
   1. Creating MagicPod batch run settings
   2. Test results email notification settings
   3. Test results Slack notification settings
   4. Test schedule run settings (in case of Bitrise)
      1. Running MagicPod from Bitrise build
      2. Running Bitrise builds periodically
      3. Running builds for every Push or Pull Request to the repository
   5. Test schedule run settings (in case of CircleCI)
2. Introduction of functions for ease of use

1. Setting daily build + batch test run schedules

Once you have created several tests, you can schedule them to run as a build + batch test every day, and configure the results to be sent using email or Slack notifications.

1-1 Creating MagicPod batch run settings

First, set up MagicPod for batch execution.

Press “Settings▼” within the “Batch runs” tab in the project you wish to configure.

From "Settings:" in the upper left corner, either select an existing setting or create a new one.

Set the Environment to “Cloud” and the App Type to “App file (cloud upload)”*1.

 

1-2 Test results email notification settings

To configure email notifications, go to “Common settings” > ”Notify test result by email”. The default is to provide notifications.

 

1-3 Test results Slack notification settings

To configure Slack notifications, go to “Common settings” > “Notify test result by Slack” > “Add”

When the popup window appears, input your desired name and click “Add to Slack”.

Select the channels for which you wish to send notifications, and click “Allow”.

For the method of setting whether to provide notification(s) of Success/Failure/Requires Confirmation/Suspension, see Notification of test results.

 

1-4 Test schedule run settings (in case of Bitrise)

First, create a build configuration for Bitrise to create an App file from the source code according to the procedure in How to create an App file - 1. In case of Bitrise.

1-4-1 Running MagicPod from Bitrise builds

Link Bitrise and Magic Pod. Open Bitrise’s Workflow Editor.

Press the “+” button below the “Xcode build for simulator” step,

and search for the “Magic Pod” step to add.

Once added, click and set the “Enter value” for each item.

• Magic Pod API token
  • Set the values copied from “MAGIC_POD_API_TOKEN” in “Key” and the Magic Pod API token page in “Value”, and click “Add new”
• Organization name
  • Specify the organization name used in Magic Pod. Please note that this is not the “display name”, but the name in alphabet characters used by the organization URL. )
• Project name
  • Specify the project name used in Magic Pod. Please note that this is not the “display name”, but the name in alphabet characters used by the project URL. )
• Test settings number
  • Specify the number of the settings created in 1−1. Creating MagicPod batch run settings

Although there is a wide variety of other parameters available, it will still work with these other parameters in their default value state*2. Apps built in Bitrise are uploaded to the Magic Pod side as well, and tests will be run at the time of CI.
When finished, save the modifications,

move to the build screen,

and run the build again.

Once the test finishes successfully,

in addition to the Bitrise test result email,

the Magic Pod test results email will be sent.

After adjusting the settings, the following types of tests will be possible.

• Tests can be run in parallel in multiple device patterns for uploaded apps (MagicPod test batch execution tab > Settings ▼ > Add using + in lower left corner > Common settings > Execution method "Run in parallel")
• Change the uploaded app from an app file to an IPA file and test the app you have built on an external Cloud service such as SauceLabs/BrowserStack on an actual device

 

1-4-2. Running Bitrise builds periodically

Click “Start/Schedule a Build” at the top-right of the Bitrise build screen.

The build configuration pop-up window will be displayed, so enable the "Schedule this Build" switch.

Specify the time, day of the week, and branch you want to build, and press "Start Build" after selecting the Workflow you just created.

This completes the build schedule setup.

 

1-4-3. Run the build every time there is a repository Push or Pull Request

If Webhook is set in the initial settings, the build will be automatically run whenever there is a Push or Pull Request to the Git repository. This timing can be adjusted using the following method.

• You can adjust the timing of the build start, such as Push and Pull Requests from the Workflow Editor Triggers tab *3.
• Build results are automatically linked to screens such as GitHub, and this makes it easy to check build results in response to Push and Pull Requests.
  • Displaying build results in relation to Push requests
  • Displaying build results in relation to Pull requests

 

1-5. Test schedule run settings (in case of CircleCI)

CircleCI works in conjunction with MagicPod to schedule builds and test runs.

Refer to the following for details on the setup method.

• Collaboration with CircleCI
• Running batch tests/running command line tests

 

2. Introduction of features for ease of use

If you wish to know all the commands→Command list

• Method of operating screens for which scrolling is required
• Email tests
• WebView tests
• Utilization of shared steps　This makes running tests easier. No limit on the number of creations
• Utilizing variables
• Utilizing conditional branching
• Visual Diff Checks
• Innovations in application implementation to simplify automated testing
• Running batch tests/running command line tests (Local PC test)
• I want to run only specific test cases as a batch
• Mobile App Test Frequently Asked Questions

*Notes / Sources

1. Since the apps to be tested are uploaded from Bitrise, the settings made for the batch execution settings screen are not relevant.
2. By changing the settings, you can select whether to wait until the test is complete or whether to delete the apps uploaded to the Magic Pod after the test is complete.
3. Build triggers

 

Next, we shall explain the screens and terms used for the various settings.

→4. Description of Screens and Terms