Custom Transformers and Loops

A loop is a programming structure that allows an action to be implemented repeatedly. In most cases, a loop is linked to a condition; i.e., the action continues until a certain condition is met.

An aircraft circles the airport as a delaying action. However, rarely in computing do we wish to slow processing. Instead, each iteration carries out a different action, or produces a slightly different result. Often each loop provides a closer and closer approximation of a solution, halting when it is within a set tolerance.

Loops and FME

In FME, loops are a way to repeat a section of transformers multiple times, without having to duplicate that section. For technical reasons, loops are only permitted inside a custom transformer.

As you know, FME processes one feature at a time. Therefore, when you create a loop, each feature is being sent around the loop individually.

So, to be worthwhile, each iteration of the loop must do something different. Often each iteration processes a different component of a feature (for example, reads records from a list) or repeats the same process using the results of the previous loop.

FME Lizard says…

It's stating the obvious I know, but you use a loop to repeat only actions that need repeating. One-off actions should take place outside of a loop. For example, here is an aircraft landing sequence: The "wheels down" part comes outside the circling loop because we don't want to raise/lower the wheels every time we circle the airport. That would be very inefficient.

Setting up a Custom Transformer Loop

A loop in a custom transformer requires various components. The required components are:

• The start point of the loop
• The end point of the loop

Other components relate to the conditions for ending the loop. At least one of these is required:

• An absolute condition by which to test for the end of processing ("Circle the airport until I tell you")
• The number of times to carry out the loop ("Circle the airport five times, then land")
• A running count of the number of iterations ("I've circled the airport three times and have two more times to go")

Loop Start Points

The start of the loop is identified by an Input port object. Although it can be the same input port as used for features to enter by, this does not have to be the case. For example, here there is an input port for features to arrive into, and another one for the start of the loop:

This allows the loop point to be other than the beginning of the custom transformer.

By default, this second input port also appears on the transformer itself, like this:

If you don't require this, then you simply have to 'unpublish' it in the input port's parameters:

Loop End Points

The end of a loop is identified by a Loop object. You can insert one by selecting it from the canvas context menu in a custom transformer:

When a loop object is placed you are asked which Input object it is to be looped to:

And then the loop is complete:

Of course, this example is an infinite loop. The action is repeated, but there is no condition being tested to stop it. FME won't let an infinite loop run forever - it will recognize the problem and stop it - but we should set up something to force an ending.

Loop Conditions

There are two general types of condition for which we can test. Firstly we can loop a set number of times. Secondly, we can loop until a specific condition is met.

Here is a custom transformer that loops a set number of times:

Notice that we have an attribute that is a counter for the number of times we have looped (LoopCounter), and an attribute that tells us the maximum number of loops to carry out (MaxLoops).

In each loop, the counter attribute is incremented by 1. When LoopCounter < MaxLoops, then we loop back and process the data again. When LoopCounter = MaxLoops, then we exit the transformer.

Instead of a simple count of iterations, another method is to test a specific measure of data quality. For example, here an ellipse representing a park is adjusted (resized smaller and smaller) until it's size is less than a certain quantity:

Loops and Transformer Types

As you should already know, transformers that operate on one feature at a time are called Feature-Based, and transformers that operate on multiple features at a time are called Group-Based.

We can also call a loop "Feature-Based" because it only processes one feature at a time. Unfortunately, that means that using a group-based transformer inside a (feature-based) custom transformer loop is not a simple task.

If you attempt to create a loop inside an embedded custom transformer, when it includes a group-based FME transformer, then you receive an error message. Group-based transformers are only permitted inside a loop in a linked custom transformer. There are technical reasons for this that we won't go into right now.

This is the error message you get:

So, inside a linked custom transformer definition, you'll see a particular parameter (in the Navigator window) called Enable Blocked Looping:

When set to Yes then other parameters allow you to set the number of iterations, and to set an attribute that will hold that value. Parallel processing is turned off (the parameters are removed) for custom transformers that are being looped, and the Insert Mode is automatically changed to "Linked Only".