On this page, we shall explain how to run batch tests on the local PC in MagicPod from the command line.
In a local PC environment, batch test execution uses MagicPodDesktop. For installation instructions, see Before performing a browser test on local PC.
Table of contents
- Run batch tests on Mac
- Run batch tests on Windows
- Run batch tests on Linux
- Run batch tests on parallel real devices on the Mac
- Write magic_pod_config.json with setting number
1. Run batch tests on Mac
Write a configuration file
First, you need to write the configuration file required for command line execution of the test.
As an easy way to do this, we recommend that you set all items except for Test result language in the test run settings panel in MagicPod, and write the remaining items in the configuration file. If you want to write all items in the configuration file, please delete the testSettingsNumber item from the configuration file.
Once you launch MagicPodDesktop once, the magic_pod_config.json file should be found in the /Users/<Username>/Library/Application Support/magic_pod_desktop directory (this is the configuration file used by MagicPodDesktop). Copy this to an appropriate place, such as the Desktop.
Once copied, write the configuration file in JSON format, following the format.
{
"owner": <Organization name. This is not the display name, but the alphabet description in the URL>,
"project": <Project name. This is not the display name, but the alphabet description in the URL>,
"capabilities": <Target OS and target app settings. Same as Appium Capabilities>,
"sendMail": <whether to distribute a test results e-mail. If omitted, this is false>,
"captureType": <test results screen capture setup. Any one of on_error, on_ui_transit, or on_each_step If omitted, this is on_each_step>,
"xmlTestOutput": <output the test results XML in JUnit format. Mainly in coordination with Jenkins. If omitted, this is false>,
"workDir": <full path to the work directory. If omitted, this is “magicPod” on the Desktop>,
"authTokenFilePath": <full path to authentication token file. When omitted, set to ~/.magic_pod_token>
"envVars": <list of environmental variable strings in KEY=VALUE format. If omitted, this is a blank list>,
"testNumber": <describes the test number to be run delimited by commas. When blank or omitted, run all tests. If you want to run a specific test multiple times, you can use something like "1*100" (in this example, test number 1 is run 100 times)>,
"includedTestCaseLabels": <specifies the label for the test in character string list format>,
"excludedTestCaseLabels": <specifies the label for the test in character string list format>,
"logLevel": <sets the volume of log to output. Any one of beginner, expert, or magic_pod_bug_investigation. If omitted, this is beginner>,
"retryCount": <how many times to retry failed tests. If omitted, this is 0 (in other words, do not retry)>,
"externalAppium": <Whether to use the MagicPodDesktop external Appium Server. If omitted, this is false>,
"externalAppiumUrl": <Specifies the URL of the Appium Server to connect to if externalAppium is true. >,
"language": <display language for the test execution log. ja or en. If omitted, this is en>,
"visualDiffPattern": <Image difference pattern name. It is possible to specify skip. When omitted, this is handled as unspecified>
}
Refer to here when specifying shared variables.
Refer here when using variables for values in the configuration file.
Based on the above, we would write the settings as follows.
{
"owner": "SampleCompany",
"project": "SampleApp",
"capabilities": {
"platformName": "iOS",
"platformVersion": "17.2",
"deviceName": "iPhone 15",
"app": "/Users/<yourUserName>/magicpod_demo_app.app",
"automationName": "XCUITest"
},
"sendMail": true,
"workDir": "/Users/<yourUserName>/magicpod-work"
}
Run tests
Once the configuration file is complete, run the test according to the following procedure.
- Close MagicPodDesktop if it has been launched.
- On the Mac Terminal, navigate to the MagicPodDesktop directory.
- Run the following command.
# Update MagicPodDesktop to the latest version if available
"MagicPodDesktop.app/Contents/MacOS/MagicPodDesktop" update --magic_pod_config="<Full path to magic_pod_config.json>"
# Test batch execution
"MagicPodDesktop.app/Contents/MacOS/MagicPodDesktop" run --magic_pod_config="<Full path to magic_pod_config.json>"
- Run the test in accordance with the magic_pod_config.json configuration. If the test is successful, the value returned by the process will be 0, whereas if it fails, this will be 1. If it ends abnormally, this will be -1.
If you copy magic_pod_config.json, command line execution is possible with other Mac PCs in the same way.
However, as the magic_pod_config.json is not saved with authentication information alone, you will need to launch MagicPodDesktop once, enter your email address and password, and save the authentication information on the PC.
2. Run batch tests on Windows
Test batch execution within the project can also be performed from Windows PowerShell.
Write a configuration file
First, you need to write the configuration file required for command line execution of the test.
Once you launch MagicPodDesktop once, the magic_pod_config.json file should be found in the C:\Users\<Username>\AppData\Roaming\magic_pod_desktop directory (this is the configuration file used by MagicPodDesktop). Copy this to an appropriate place, such as the Desktop.
After you copy this, following the format below, write the configuration file in JSON format.
{
"owner": <Organization name. This is not the display name, but the alphabet description in the URL>,
"project": <Project name. This is not the display name, but the alphabet description in the URL>,
"capabilities": <setup of the target browser, etc. This is the same as Desired Capabilities in Selenium>,
"sendMail": <whether to distribute a test results e-mail. If omitted, this is false>,
"baseUrl": <base URL>,
"captureType": <test results screen capture setup. Any one of on_error, on_ui_transit, or on_each_step If omitted, this is on_each_step>,
"stepCaptureScope": <Whether to include only the HTML content in each line of screen captures or the entire browser window. Either html or window If omitted, this is html>
"xmlTestOutput": <output the test results XML in JUnit format. Mainly in coordination with Jenkins. If omitted, this is false>,
"workDir": <full path to the work directory. If omitted, this is “magicPod” on the Desktop>,
"authTokenFilePath": <full path to authentication token file. If omitted, this is C:\Users\ [Username] \.magic_pod_token>
"envVars": <list of environmental variable strings in KEY=VALUE format. If omitted, this is a blank list>,
"testNumber": <describes the test number to be run delimited by commas. If this is blank or omitted, all tests are run>,
"includedTestCaseLabels": <specifies the label for the test in character string list format>,
"excludedTestCaseLabels": <specifies the label for the test in character string list format>,
"logLevel": <sets the volume of log to output. Any one of beginner, expert, or magic_pod_bug_investigation. If omitted, this is beginner>,
"retryCount": <how many times to retry failed tests. If omitted, this is 0 (in other words, do not retry)>,
"language": <display language for the test execution log. ja or en. If omitted, this is en>,
"visualDiffPattern": <Image difference pattern name. It is possible to specify skip. If omitted, this is treated as unspecified>,
"testCondition": <conditions when running tests (*see example below)>
}
Refer to here when specifying shared variables.
Refer here when using variables for values in the configuration file.
Based on the above, the configuration description may be, for example, as follows.
{
"owner": "MyCompany",
"project": "SampleWebApp",
"capabilities": {
"browserName": "chrome"
},
"sendMail": true,
"testCondition": {
"browser": "chrome"
}
}
- The current specification is such that similar content must be written for capabilities and testCondition, but we plan to improve this in the future. The test may be run even if there is no testCondition, but browser information may be missing when viewing the test results on the Web.
Run tests
Once the configuration file is complete, run the test according to the following procedure.
- Close MagicPodDesktop if it has been launched.
- Open Windows PowerShell. There are several ways of doing this, but the simplest way is to open Windows PowerShell from the start menu.
- Run the following command. For the MagicPodDesktop version section, enter (0.36.0, etc.) according to the folder in which it actually exists. This can be completed on PowerShell. Note that an error will occur if there is no & at the start (when entering from 1 on PowerShell, this is inserted automatically)
& "C:\Users\<Username>\AppData\Local\magic_pod_desktop\app-<MagicPodDesktop version>\MagicPodDesktop.exe" run --magic_pod_config="<magic_pod_config.json full path>"; Wait-Process -Name MagicPodDesktop -Timeout 3600
The 3600 in the command is the maximum number of seconds to wait for the test to complete, and means to wait up to 3600 seconds. This wait time may be changed based on the length of the test. - Run the test in accordance with the magic_pod_config.json configuration.
If you copy magic_pod_config.json, it is possible to run the command line in the same way on other Windows PCs.
However, as only the authentication information is not saved in magic_pod_config.json, it is necessary to start MagicPodDesktop once, enter the e-mail address and password, and save the authentication information on the PC.
3. Run batch tests on Linux
It is possible to run a test batch on Linux, using a procedure similar to the Windows one.
Install the required libraries
First, install xvfb (required libraries for browser rendering) and imagemagick (required libraries to acquire screen shots for the entire browser window). For Ubuntu, you can install these using the following commands.
sudo apt-get update
sudo apt-get install -y unzip xvfb libxi6 libgconf-2-4 imagemagick
In certain CI services, such as CircleCI, xvfb and imagemagick are installed from the start, so this procedure is not necessary.
Next, if we are using Chrome to run the test, we must first install Chrome. For Ubuntu, you can install these using the following commands.
wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
sudo sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list'
sudo apt-get update
sudo apt-get install google-chrome-stable
Next, we need to install Japanese language fonts so that there will be no garbled characters in Japanese language screen captures. In the case of Ubuntu, you can install these using the following commands.
sudo apt -qqy --no-install-recommends install -y locales-all
sudo locale-gen ja_JP.UTF-8
sudo update-locale LANG=ja_JP.UTF-8
sudo update-locale LANGUAGE="ja_JP:ja"
sudo apt -qqy --no-install-recommends install -y fonts-takao-gothic fonts-takao-mincho
sudo dpkg-reconfigure --frontend noninteractive locales
sudo fc-cache -fv
Download magicpod-command-line
Execute the following command on the command line to download.
curl -L "https://app.magicpod.com/api/v1.0/magicpod-clients/local/linux/latest/" -H "Authorization: Token {apikey}" --output {filename}.zip
- {apikey}: API token
- {filename}: Any file name you like
You can also acquire the zip file for magicpod-command-line for Linux from the Download page unzip it, and place the resulting magicpod-command-line and node_modules directories in an appropriate location (both files need to be in the same place).
Write a configuration file
Create magic_pod_config.json in the same way as in Windows (There is no original file to copy, so you need to create it using a text editor, etc.).
Set authentication information
If you are using a PC on which you have successfully logged into MagicPodDesktop, below the Mac folder /Users/<Username> or for Windows, in the C:\Users\<Username> folder, there should be a file called .magic_pod_token. This file is the authentication information file for the login user, so copy this below “~/” on a Linux machine.
Alternatively, you can set the authentication information by running the following command and acquiring the token information called {"token":"****"}, and saving the contents of the **** part in ~/.magic_pod_token (only for users not configured for SAML authentication).
#Specify the e-mail address used for MagicPod in xxxx and the password in yyyy.
curl -X POST -L --data-urlencode "email=xxxx" --data-urlencode "password=yyyy" https://app.magicpod.com/api/v1.0/internal/token-auth/
The path for ~/.magic_pod_token can be changed using authTokenFilePath in magic_pod_config.json.
Launch Xvfb
You need to launch Xvfb before running a test, and close it when it is finished. As this process is carried out automatically by some Cloud CI services, such as CircleCI, this is not required. If you are using Jenkins, the Xvfb plug-in will automatically launch Xvfb when running a job. In other cases, by changing the command when running the test, it is possible to launch Xvfb and the test together (to be described later).
Run tests
Once you are ready, you can run the test using the following command. This is the command when Xvfb is already running on CircleCI, Jenkins, etc.
magicpod-command-line --magic_pod_config="<magic_pod-config.json full path>"
The command to run when launching Xvfb and the test at the same time is as follows.
xvfb-run magicpod-command-line --magic_pod_config="<magic_pod-config.json full path>"
In any case, it is necessary either to use the full path to magicpod-command-line on execution, or specify a relative path to magicpod-command-line.
Save the test results as a video
It is only possible to save a video of the test in progress when running local tests in a Linux environment.
Before running the test, install a tool called ffmpeg, which is required for saving the video.
sudo apt install ffmpeg
Next, add the option to save videos to the test configuration file. Note that this option cannot be added as a screen operation, so it is necessary to edit the configuration file manually. If you specify record_type as the key and always as the value, it will be saved as a video every time you run a test. If you specify only_failed as the value, it will only save if the test fails, and if you specify never, the setting is for it not to be saved.
{
"owner": "MyCompany",
"project": "SampleWebApp",
"capabilities": {
"browserName": "chrome"
},
"sendMail": true,
"testCondition": {
"browser": "chrome",
"record_type": "always"
}
}
Specify several environment variables as necessary when running the test.
# Mandatory in principle
export MAGIC_POD_RECORDING_SCREEN_SIZE=1280x1024
# Option
export MAGIC_POD_RECORDING_DISPLAY=:99.0
export MAGIC_POD_RECORDING_FRAME_RATE=8
xvfb-run magicpod-command-line --magic_pod_config=""
|
Variable name |
Meaning |
Default value |
|
MAGIC_POD_RECORDING_SCREEN_SIZE |
Specifies the area size of the display rendered by Xvfb, to be saved as a video. (method of looking this up is described later) |
None |
|
MAGIC_POD_RECORDING_DISPLAY |
This is the number of the display depicted by Xvfb. (method of looking this up is described later) |
:99.0 |
|
MAGIC_POD_RECORDING_FRAME_RATE |
Specifies the video frame rate (number of frames per second). The larger the value, the more detailed the movement that can be saved. However, as this will greatly increase the file size, uploads may take more time, and if the file size limit (50 MB) is exceeded, it may not be possible to complete the upload. |
8 |
To find out what to specify for the above two environment variables, check the display numbers, etc., used, with Xvfb running.
# Example when launching Xvfb and Chrome manually. Launch these using method used for ordinary tests
xvfb-run google-chrome --no-sandbox &
# Check the running Xfvb processes and parameters
ps -ef | grep Xvfb
(Output example)
root 213 202 0 02:17 pts/1 00:00:00 Xvfb :99 -screen 0 640 x 480 x 16 -nolisten tcp -auth /tmp/xvfb-run.uyTR8o/Xauthority
root 810 13 0 02:18 pts/1 00:00:00 grep --color=auto Xvfb
640 x 480 x 16 is where the screen resolution is specified. In this case, the MAGIC_POD_RECORDING_SCREEN_SIZE is 640 x 480. The display number is :99.0, in accordance with the first parameter and screen option. As the default value is :99.0, it will work even if not specified, but you can specify this as an environment variable if the value is different. Be sure to close related processes (Xvfb, xvfb-run, google-chrome, etc.) after checking the variable value.
4. Run batch tests on parallel real devices on Mac
If you want to execute multiple tests on real devices connected to the local PC in parallel, prepare multiple magic_pod_config.json files as follows, execute MagicPodDesktop with multiple actual devices connected to a single PC, and specify each of the magic_pod_config.json file separately. Note that when running batch tests on parallel real devices on Mac, it is not supported to use testSettingsNumber.
iOS parallel execution settings
{
....
"capabilities": {
"udid": "<device UDID>",
"wdaLocalPort": <port number that does not duplicate any of the MagicPod settings in the PC>,
....
},
"internalAppiumPort": <port number that does not duplicate any of the MagicPod settings in the PC>,
"internalMagicPodPort": <port number that does not duplicate any of the MagicPod settings in the PC>,
"workDir": "<directory that does not duplicate any of the MagicPod settings in the PC>",
....
}
Android parallel execution settings
{
....
"capabilities": {
"udid": "<device UDID>",
"systemPort": <port number that does not duplicate any of the MagicPod settings in the PC>,
"chromeDriverPort": <port number that does not duplicate any of the MagicPod settings in the PC>,
....
},
"internalAppiumPort": <port number that does not duplicate any of the MagicPod settings in the PC>,
"internalMagicPodPort": <port number that does not duplicate any of the MagicPod settings in the PC>,
"workDir": "<directory that does not duplicate any of the MagicPod settings in the PC>",
....
}
- Ports and workDirs should not overlap within the same PC throughout the course of the browser test, Android test, and iOS test.
- The device udid can be obtained by connecting the device to the machine via USB and using the command instruments -s devices in the case of iOS (note that a lot of information about the iOS simulator is displayed at the same time.) For Android, this can be obtained using the command ~/Library/Android/sdk/platform-tools/adb devices.
5. Write magic_pod_config.json with setting number
By writing magic_pod_config.json in the following format, it is possible to refer to the test run settings already created on the Batch runs page.
- Requires a client with version 0.99.12 or higher.
{
"owner": <Organization name. This is not the display name, but the alphabet description in the URL>,
"project": <Project name. This is not the display name, but the alphabet description in the URL>,
"testSettingsNumber": <Test Settings Number>,
"testSettingsPatternName": <Test Settings Pattern Name. Required when there are multiple patterns>
}
The items referenced by testSettingsNumber
testSettingsNumber refers to items other than Test result language.
If magic_pod_config.json contains duplicate properties referenced by testSettingsNumber, the items referenced by testSettingsNumber will be used.
For example, the following testCondition is ignored.
{
"owner": "MagicPodOrganization",
"project": "MagicPodProject",
"testSettingsNumber": 1,
"testCondition": {
"browser": "chrome",
}
}