Email tests with Gmail Web API

You can create Email tests using the Gmail Web API, which is part of the Google Workspace (formerly G Suite) organization. Since the free version of Gmail is difficult to obtain tokens that do not expire, please consider subscribing to Google Workspace or use a web service called MailSlurp.

Table of contents

1. How to Create Email Tests
   
   1. Obtain the "Client ID" and "Client Secret"
   2. Obtain the "Refresh token"
   3. Obtain the "Access token" by the "Web API call" command
   4. Obtain the "Mail ID" by the "Web API call" command
      
      1. Specify search conditions for retrieving email
      2. Retrieving a previous email
   5. Obtain the body of the email by the "Web API call" command
2. Examples of Test Case Steps
   
   1. Transition to the URL included in the body of the email
   2. Enter the authentication code included in the body of the email in the app
   3. Test using a random email address
3. Troubleshooting
   1. Failure to obtain the "Access token"
   2. Failure to obtain the "Mail ID"
   3. Failure to retrieve the email body

1. How to Create Email Tests

1-1. Obtain the "Client ID" and "Client Secret"

First, obtain the Client ID and Client Secret required for Web API authentication.

Prepare one test Google account (= Gmail account) belonging to Google Workspace, which is different from the account you normally use, since the Web API will be able to retrieve your email content.

Then, sign in to Google's Developer Console with the Google account you created, and create a project from the drop-down list in the upper left corner of the screen with a project name of your choice.

Then, from the APIs & Services > Dashboard menu in the upper left corner of the screen, select　Enable APIs & services and choose the Gmail API to enable.

To authenticate the API, you must first enable OAuth. First, select APIs & Services > OAuth consent screen from the menu in the upper left corner of the screen; if you have a Google Workspace email address, you can select Internal for User Type. You can also use External for test submissions, but the token will expire in a few days.

If the email address is not a Google Workspace email address, Internal cannot be selected. If you select External and get confirmation by Google, you can probably get a non-Google Workspace email address with a non-expiring token, but it would be very difficult to get it approved.

On the next screen, specify the appropriate App name, User support email, and Developer contact information and press SAVE AND CONTINUE.

On the next screen, press ADD OR REMOVE SCOPES, select View your email messages and settings, and press UPDATE. OAuth is now enabled.

Next, create Credentials. From the APIs & Services > Credentials menu in the upper left corner of the screen, select CREATE CREDENTIALS > OAuth client ID, set Application type to Desktop app and press CREATE.

Then, Client ID, Client secret, and a button to download the JSON file will appear on the screen. Save the JSON file in an appropriate directory with the name credentials.json.

1-2. Obtain the "Refresh token"

Next, we will obtain a Refresh token, which is also required for Web API authentication.

The procedure is explained using Python. (Other APIs can also be found here.)

First, if Python is not installed on your machine, follow the instructions here.

Next, open a command prompt on Windows or Terminal on Mac and execute the following command.

 pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib

Now you have downloaded Google client library.

Next, start an editor in an appropriate directory on your machine and create an empty file. Press the Copy Code Sample button on this page to copy and paste the contents.

Save the created file with the file name quickstart.py.

Move credentials.json saved earlier to the same directory as quickstart.py.

Execute the following from the command line.

cd <Directory with quickstart.py>

python quickstart.py

The following page will appear. Click Allow to close the window.

A file named token.json will be created in the same directory, from which you can check Client ID, Client secret, and Refresh token.

This completes the preparation for using the Gmail API. Now that we have the Client ID, Client secret, and Refresh token, we can call the Gmail API from the Magic Pod test script.

1-3. Obtain "Access token" by the "Web API call" command

First, define three secret shared variables CLIENT_ID, CLIENT_SECRET, and REFRESH_TOKEN, and set each to the value obtained earlier.

Next, copy this json type text and paste it into the test case edit screen with ctrl + V (command + V on Mac OS). A Web API call step will then be added to retrieve the latest Access token based on the Refresh token.

{"command_calls": [{"command": "web_api","val": [{"url": "https://accounts.google.com/o/oauth2/token","method": "post","header": [{"key": "Content-Type","value": "application/x-www-form-urlencoded"}],"body": {"format": "raw","form_data": [],"raw": "client_id=${CLIENT_ID}&client_secret=${CLIENT_SECRET}&refresh_token=${REFRESH_TOKEN}&grant_type=refresh_token"},"result": {"params": [{"key": "ACCESS_TOKEN","value": "jsonResponse[\"access_token\"]"}]}}]}],"project_id": 0,"version": "0"}

Executing this will set the variable ACCESS_TOKEN to the value of the Access token (the retrieved Access token can be used over and over again within 30 minutes).

If adding a step by copy and paste fails, add a Web API call step and set the following items:

Method: POST
URL: https://accounts.google.com/o/oauth2/token
Header: "Content-Type" as key, "application/x-www-form-urlencoded" as value
Body: Select "raw data" and set "client_id=${CLIENT_ID}&client_secret=${CLIENT_SECRET}&refresh_token=${REFRESH_TOKEN}&grant_type=refresh_token" in body 
Results: "ACCESS_TOKEN" as Variable, "jsonResponse["access_token"]" as Javascript

1-4. Obtain "Mail ID" by the "Web API call" command

Next, use the Access token obtained in step 1-3 to obtain one mail ID. Copy this json type text and paste it into the test case edit screen with ctrl + V (command + V on Mac OS). A Web API call step will then be added to retrieve the latest email in your inbox.

{"command_calls": [{"command": "web_api","val": [{"url": "https://www.googleapis.com/gmail/v1/users/me/messages/","method": "get","header": [{"key": "Authorization","value": "Bearer ${ACCESS_TOKEN}"}],"result": {"params": [{"key": "MAIL_ID","value": "jsonResponse[\"messages\"][0][\"id\"]"}]}}]}],"project_id": 0,"version": "0"}

Executing this will set the variable MAIL_ID to the latest mail ID matches search conditions.

If adding a step by copy and paste fails, add a Web API call step and set the following items:

Method: GET
URL: https://www.googleapis.com/gmail/v1/users/me/messages/
Header: "Authorization" as key, "Bearer ${ACCESS_TOKEN}" as value
Results: "MAIL_ID" as Variable, "jsonResponse["messages"][0]["id"]" as Javascript

Tips: Specify search conditions for retrieving email

You can specify search conditions such as To/From mail address or subject and retrieve a specific email by appending ?q=<search conditions> like https://www.googleapis.com/gmail/v1/users/me/messages/?q=<search conditions>.

The <search conditions> section can be written in Gmail's search condition notation. For example, you can use the following notation. For more details, please refer to this page.

• ?q="subject:New user registration" (retrieve an email with "New user registration" in the title)
• ?q="from:no-reply@magic-pod.com" (retrieve an email sent from "no-reply@magic-pod.com")
• ?q="to:new_user%2B1890537491%40magic-pod.com" (retrieve an email sent to "new_user+1890537491@magic-pod.com". If you use an alias email address, you must specify + as "%2B" and @ as "%40" in URL encoding)
• ?q=new_user12345678 (Retrieve an email containing "new_user12345678" in the body, etc.)

Tips: Retrieving a previous email

It is possible to retrieve a previous mail ID that is not the latest by changing the numeric part of the Javascript "jsonResponse["messages"][0]["id"]" in the Result tab of the Web API call settings screen.

• To retrieve the mail ID before the latest one: "jsonResponse["messages"][1]["id"]"
• To retrieve the mail ID before the second latest one: "jsonResponse["messages"][2]["id"]"

1-5. Retrieve the body of the email by the "Web API call" command

Finally, retrieve the body of the email based on the mail ID obtained in step 1-4. Copy this json type text and paste it into the test case edit screen with ctrl + V (command + V on Mac OS). A Web API call step will then be added to retrieve the body of the email based on the mail ID obtained in steps 1-4.

{"command_calls": [{"command": "web_api","val": [{"url": "https://www.googleapis.com/gmail/v1/users/me/messages/${MAIL_ID}","method": "get","header": [{"key": "Authorization","value": "Bearer ${ACCESS_TOKEN}"}],"result": {"params": [{"key": "MAIL_CONTENTS","value": "const inlineMessagePart = function(messagePart) { let messagePartBodies = [{\"mimeType\": messagePart.mimeType, \"body\": messagePart.body}]; if (\"parts\" in messagePart) { for (const subMsgPart of messagePart[\"parts\"]) { messagePartBodies = messagePartBodies.concat(inlineMessagePart(subMsgPart)); } } return messagePartBodies; }; atob([jsonResponse['payload']].map(inlineMessagePart).flat().filter(function(item) { return item.mimeType === \"text/plain\"; })[0]['body']['data']);"}]}}]}],"project_id": 0,"version": "0"}

The contents of the body of the email will now be saved in the variable MAIL_CONTENTS.

For details on how to save only the necessary information from the body of the email into a variable, see 2. Examples of Test Case Steps.

If adding a step by copy and paste fails, add a Web API call step and set the following items:

• Method: GET
• URL: https://www.googleapis.com/gmail/v1/users/me/messages/${MAIL_ID}
• Header: "Authorization" as key, "Bearer ${ACCESS_TOKEN}" as value
• Results: "MAIL_CONTENTS" for the results variable, and copy and paste the following into the resulting Javascript (line feed characters will be lost, but this will not affect the operation)

const inlineMessagePart = function(messagePart) {
   let messagePartBodies = [{"mimeType": messagePart.mimeType, "body": messagePart.body}];
   if ("parts" in messagePart) {
     for (const subMsgPart of messagePart["parts"]) {
       messagePartBodies = messagePartBodies.concat(inlineMessagePart(subMsgPart));
     }
   }
   return messagePartBodies;
};
atob([jsonResponse['payload']].map(inlineMessagePart).flat().filter(function(item) { return item.mimeType === "text/plain"; })[0]['body']['data']);

2. Examples of Test Case Steps

2-1. Transition to the URL included in the body of the email

Let's create a test in which the user registers as a new member by navigating to the URL included in the email.

1. Click the "Register" button on the application to be tested.
2. An email is sent to the newly registered email address.
3. Click on the URL in the email to complete the membership registration.

Dear XXX
Thank you for registering.
Please log in using the link below within one hour of receipt.
https://example.com/qwertyuiop123456789

In this case, please add the following test case steps.

In this case, you can store just the URL into the URL variable by specifying the following regular expression in the step11 Store substring matching regular expression command.

https:\/\/example.com\/.+

2-2. Enter the authentication code included in the body of the email in the app

Let's create a test in which new member registration is performed by entering the authentication code contained in the email into the app to be tested.

1. Press the "Register" button in the app to be tested.
2. An email will be sent to the newly registered email address.
3. Enter the authentication code in the email into the app to complete the membership registration.

Dear XXX
Thank you for registering.
Please log in within 1 hour of receipt and enter the authentication code below.
Authentication code: 123456

In this case, please add the following test case steps.

In this case, you can store just the 6-digit number into the auth_code variable by specifying the following regular expression in the step11 Store substring matching regular expression command.

[0-9]{6}

If you want to accurately extract the 6-digit number that follows the "Authentication code: ", you can use the following regular expression.

(?<=Authentication code: )([0-9]{6})

2-3. Test using a random email address

You can use Gmail email aliases to test with a different email address each time. You can create a new email address associated with the original address by appending a value after the "+" like "new_user+20240621175024@magic-pod.com".

In the example below, the "Store unique value generated based on the current time" command is used to create a unique value, and the "Store fixed value" command is used to generate an email address used for new member registration.

3. Troubleshooting

3-1. Failure to obtain the "Access token"

When making a Web API call to obtain an access token, the call may fail with a 401 error as shown below.

Client error. Status code 401 Unauthorized
Response data: {"error":"invalid_client","error_description":"The OAuth client was not found."}

In this case, one of the shared variables CLIENT_ID, CLIENT_SECRET, or REFRESH_TOKEN is most likely incorrect. Check the following

• The three shared variables are defined correctly.
• Whether you are using a test run setting with the three shared variables defined.

3-2. Failure to obtain the "Mail ID"

If no email matches the search criteria during the execution of the Web API to obtain the mail ID, the Gmail Web API returns a response of {"resultSizeEstimate": 0}. In this case, it is most likely that the email has not yet been received by the time the Web API call to obtain the Mail ID is made.

Please insert a Wait for fixed seconds command between the step that triggers the email sending (e.g., "Register") and the Web API call step that obtains the Mail ID, as shown below. Adjust the wait time as necessary.

3-3. Failure to retrieve the email body

When handling email as a program, you need to be aware of its data structure. In the case of HTML email, this data may have a hierarchical structure (in the case of a simple text email, this will not have a hierarchical structure). Therefore, the Javascript used to assign the "MAIL_CONTENTS" variable is processed as follows.

1. Flatten the data structure of the email
2. Search for a message part for which the MIME type is text/plain. This is used to retrieve the body of an email from a simple text email, or to retrieve alternate text from an HTML email for environments where the HTML email cannot be viewed.
3. Acquire data from two message parts (replacement text for main text of email or HTML email). This data is a Base64 character string, so convert it to UTF8.

In the following two cases, the Javascript may fail to retrieve the main text of the email.

1. If the email in the test is an HTML email and there is no alternative text
If there is no message part where the MIME type is text/plain. In this case, if the Javascript ‘item.mimeType === "text/plain"’ described above is replaced by ‘item.mimeType === "text/html"’, it should be possible to obtain the main text of the email.

2. In case there are multiple places where the message parts are MIME type text/plain
If this is the case, it is possible that unintended email text may be obtained. In this case, we apologize for the inconvenience and request that you contact MagicPod support. They will provide a different workaround.

Reference
Refer here for more information on the Gmail Web API data structure.