Exercise 4.3 | Building Updates Automation |
Data | Building footprints (Esri Shapefile) |
Overall Goal | Provide email-driven triggers for building updates |
Demonstrates | Email Triggers |
Start Workspace | N/A |
Start Server Project | C:\FMEData2020\Projects\ServerAuthoring\RealTime-Ex3-Start.fsproject |
End Workspace | N/A |
End Server Project | C:\FMEData2020\Projects\ServerAuthoring\RealTime-Ex3-Complete.fsproject |
As a technical analyst in the GIS department, you were involved in a recent assignment to set up a Directory Watch solution for users to automatically update the corporate database.
Having learned that not all users are able to access the internal network where FME Server is hosted, you think that it should be possible to also set up a system that uses email-based automation to handle the same updates.
FME Lizard says... |
This exercise continues where Exercise 2 left off. You must have completed the previous chapter Exercises to carry out this exercise, or upload the Begin Server Project to FME Server to start where Exercise 2 left off. |
1) Create Resource Folder
The first step is to create another Resource folder where all the email attachments will be saved. Log into the FME Server web interface, navigate to Resources > Data > BuildingUpdates, and then create a new folder called Emails.
2) Update Automation
Next, Navigate to the Automations page, since we are triggering the same workspace we can expand on the existing Automation created in the previous exercises. Add a new Trigger by dragging and dropping a Trigger node (green) onto the canvas. Before configuring the Email Trigger add a connection from its output port to the Workspace Action.
The new Trigger can be created to use either the Email (SMTP) protocol or the Email (IMAP) protocol.
SMTP is easier to set up, but FME Server must reside on a server with a proper DNS record (all FME Cloud and Training machines will have this). IMAP is necessary when FME Server resides on an internal network or you would like to monitor an email account sitting on an external server.
SMTP Protocol
To use the SMTP protocol select Email (SMTP) as the Trigger. This will reveal the Email User Name parameter. Enter a name for receiving email, for example: fmeshapeprocessing
For the Download Attachments To parameter browse to the Emails folder created in step 1.
Clicking Apply will create an email address fmeshapeprocessing@<hostname> - for example:
Host | Example Email Address |
---|---|
FME Cloud | fmeshapeprocessing@myfmeserver.fmecloud.com |
Amazon AWS | fmeshapeprocessing@ec1-23-456-789-012.compute-1.amazonaws.com |
localhost | fmeshapeprocessing@IP Address |
To quickly find your IP Address, navigate to Google in your browser and search "My IP". The search results will return your public IP address as the first result.
IMAP Protocol
To use the IMAP protocol select Email (IMAP) as the Trigger. This will open up a number of other parameters. Enter them according to your email account.
In case it is of use, the server information for Gmail, Outlook, and Yahoo! are as follows:
IMAP Server Host | imap.gmail.com | imap-mail.outlook.com | imap.mail.yahoo.com |
Server Port | 993 | 993 | 993 |
Connection Security | SSL/TLS | SSL/TLS | SSL/TLS |
Verify SSL Certificates | Yes | Yes | Yes |
You will also need to check the settings in your email account to make sure IMAP is turned on. Regardless of the email provider, you should set these parameters as follows:
Parameter | Value |
---|---|
Poll Interval | 30 Seconds |
Emails to Fetch | New Emails Only |
Download Attachments To | Data > BuildingUpdates > Emails |
You could select any Resource folder for attachments to be saved to; but for this exercise series we have created a separate Emails folder to ensure we don't choose the BuildingUpdates folder as this would cause the workspace to be triggered by both the Email and Directory Watch Trigger!
The Automation is not quite ready yet because the JSON message from the Email Trigger stores the incoming file path in an attribute named Email Attachment, however the Source dataset for the run workspace is set to pick up a value from the Directory Watch File Path attribute. Therefore in order for the workspace to be able to process the file path from both Triggers we need to create an attribute in each Trigger with a common name that can be used downstream in the workflow.
3) Create Custom Key
We will need to create an Output Key on both Triggers. First select the Email trigger and click on the Output Keys tab. Event Keys list all the standard output keys that come with that action, the second, custom section called User Keys.
To add a new User Key click on the plus icon. Set the key name to user.shapefile and for the Value select Email > Email Attachment in the Text Editor window.
Repeat this process in the Directory Watch trigger but this time set the Value to Directory > File Path.
Lastly return to the Workspace Action and change the Source Dataset from file.path to the user.shapefile attribute listed under the User Keys dropdown menu. Now, the workspace will be able to parse the Shapefile path to the workspace no matter what the trigger is.
FME Lizard says... |
User keys can only be used in the Automation they are defined in, and can be set to an element of the JSON from the Trigger or Action message.
To create values that can be used across all workspaces you can create Global Keys by selected the globe icon in the menu ribbon. These can only be set to plain text values, but an example of their use might be a value your organization commonly refers to, such as a UNC path to an external location where all data is stored. |
4) Test Automation
Now let's test the Automation.
This time, because we connected the Email Trigger directly to the Workspace Action, let's zip up the four .shp, and associated file types, to add as an attachment.
Send an email with an attachment to the address created in the Email Trigger. When the email is received by FME Server (SMTP), or FME Server fetches it (IMAP), the Automation will send a message to the Workspace Action. (Remember that an IMAP publication only checks for an email at the specified poll interval, so the result might not be immediate!)
CONGRATULATIONS |
By completing this exercise you have learned how to:
|