Once the test is ready, set it to run daily. By doing this, you can detect problems that occur during the development period at an early stage. Even if they are not run every day, it is good if they are run every time there is a change to the source code. As the first step, though, set the test to run once per day.
Here, we shall explain how to periodically run a test using the CI (continuous integration) tool, Jenkins. On this occasion we will explain how to simply set up Jenkins on a Mac PC and test.
Setup
- Install Java 9 in accordance with this procedure.
- Next, we shall set up Jenkins on a Mac PC in accordance with the article Let us try to install and use Jenkins.
- Follow the content of the article, and “Enter the administrator password,” “install the recommended plug-ins,” and “Register the initial users”. The screen shown in figure 1 will be displayed.
- Next, following the “Let’s try running the test” section of the same article, “Register the Job” and “run the Job”. If the Job is successful, setup is complete (figure 2).
Figure 1. Jenkins launch screen.
Figure 2. Job success screen.
Create a MagicPod Test Execution Job
Once setup is complete, it is time to create the job to run the test.
Create the Job
Next, in the same way as before, create a “Freestyle project” job using an appropriate name.
MagicPodDesktop Setup
Place MagicPodDesktop on the same Mac PC as Jenkins, right-click on it, and select “Open” to launch. Once it has launched, enter the authentication information and log in (once you log in, authentication information shall be saved on the PC, so you will not need to log in again).
When this task is complete, close MagicPodDesktop. If you do this while it is launched, execution of the test shall fail.
Edit the Settings File
Following the procedure here, edit the settings file, magic_pod_config.json, required for running the test.
- Set “xmlTestOutput# to true. This will output the test results to the working directory as an XML file to be read by Jenkins.
- In “workDir”, you should specify <Jenkins worksplace>/magic-pod-work(example: /Users/<User name>/.jenkins/workspace/<Job name>/magic-pod-work).
Add the build step “Execute Shell”
Click the “Settings” for the created Job, and move to the Settings screen, and select “Execute Shell” from “Add build step” to add it (figure 3).
Once added, the description should be written like "<path to MagicPodDesktop.app
>/Contents/MacOS/MagicPodDesktop" run --magic_pod_config="<path to magic_pod_config.json>" in the “Command” field (figure 4).
Figure 3. Adding “Execute Shell”.
Figure 4. Setting “Command”.
Add the post-build actions “Publish JUnit test result report”
In the same way, select and add “Publish JUnit test result report” from “Post-build Actions”, and specify the relative path <workDir location>/*.xml to the “Test report XMLs”, as shown in figure 5. If you do not run any test, a warning message that the directory does not exist will be shown in red. However, once you run the test once, XML is generated and the error shall be resolved.
Figure 5. “Publish JUnit test result report” setting.
Run JOB
Save the settings and try to run JOB. Once JOB completes, a link to the results for each test shall be created on the results screen (figure 6).
Figure 6. Link to test results.
Email/chat notification of the test results
The test execution results notification can be provided by email or chat, such as Slack, etc.
To provide notification by email, follow the procedure in How to notify to Gmail when the build by Jenkins fails and run “Email Server settings” and “Job-side settings”. This will cause an email to be sent upon test failure.
To provide Slack notification, configure the settings on Slack and Jenkins, following the procedure in Notifying the Jenkins build results to Slack.
Run the test periodically
To run the test periodically, configure the settings in Running a job periodically with Jenkins.
For example, if you want to run the test at noon every day, you should enter H 12 * * * in “Scheduling”.
Further improvements
The settings introduced here are the most basic ones. For actual operations, the following additional improvements can be considered.
- Assign a fixed IP to the Jenkins server, so it can be seen by anyone within the company
- Use the Jenkins “Extended E-mail notification” to further specify the method by which emails are sent