Continuous Integration (CI) is a software quality control method that automatically builds and tests applications whenever there are changes to the source code. Tools designed for efficient CI are called CI tools, and major tools include GitHub Actions, CircleCI, and Bitrise. Due to the numerous benefits of regular execution, we recommend integrating MagicPod's automated testing into CI services.
Table of contents
- Benefits of running tests regularly using CI services
- How to integrate with each CI service
- Frequently asked questions
Benefits of running tests regularly using CI services
Integrating MagicPod's automated testing into your CI service enables regular test execution. The benefits include:
- Quick detection of defects and issues.
- Increased productivity of the development team.
- Reduced effort for manual executions.
For more details, please view the following guides:
Start guide: 5. Establish test automation habits as quickly as possible (Mobile App)
Start guide: 5. Establish test automation habits as quickly as possible (Browser)
How to integrate with each CI service
Please check each guide. It's also possible to use CI services not listed below such as GitLab. In such cases, follow the procedure in 'Integrate with CircleCI - Create 'run_magicpod_test.sh', create a .sh file, and then integrate this file into your job according to each CI tool's method.
Run daily tests with Jenkins (Mac/Local PC Test)
Frequently asked questions
Batch test runs may fail to start because all cloud devices are in use
When automating test execution with CI services, you can add a script to check the number of available cloud devices before running tests, allowing the test execution to wait until devices become available.
For more details, please refer to this help page: Queuing batch runs when the cloud device usage limit is reached
How to choose between MagicPod's scheduled runs and CI service execution
For mobile app testing, CI service execution is recommended because build files need to be uploaded to MagicPod. By integrating app builds with test execution in your CI pipeline, you can automatically run tests against the latest builds. Generate build files (.app, .apk files, etc.) within CI and automate the upload and test execution using the MagicPod API to enable efficient testing integrated into your development workflow.
For browser testing, build file uploads are not required—you can run tests simply by specifying a URL. This makes MagicPod's scheduled run feature ideal for setting up automatic execution on daily or weekly schedules. However, CI service execution is also effective when you want to run tests in coordination with deployments or for each pull request.
For both mobile app and browser testing, we recommend varying test volume to improve development speed:
- Nightly runs: Use MagicPod's scheduled run feature to test all flows (execution time: long)
- On deployment: Use CI service to test basic flows before deployment (execution time: medium)
- On pull requests: Use CI service to test minimal flows before merging (execution time: short)