Feature Caching

Sometimes it's important to be able to inspect data at any step of the translation. This behavior is activated using Run > Run with Feature Caching on the menu bar:

With this option active, FME generates caches at every step of the translation when the workspace is run:

The caches are indicated by the small icons on each object. In the above screenshot, the caches are green, but if any parameters change that could affect the data in the cache, they turn yellow. Yellow caches are considered invalid; you can still inspect them, but their data may no longer accurately represent the results of the translation.


Inspecting Cached Data

With Toggle Automatic Inspect on Selection enabled, if you click a feature type or transformer with a cache, it will automatically be displayed in Visual Preview (covered in the next section):

Specific data caches (for example a single output port on a transformer) can be inspected by clicking on the feature cache icon:

Inspecting data this way can allow you to quickly debug and develop workspaces, but keep in mind that if you are inspecting a large number of features or features with complex geometry, the caching required for automatic inspection might introduce performance challenges.


TIP
Be aware that caching data causes the translation to be slower, and to use system resources such as disk space.

Data caching is very useful while developing a workspace, but should be turned off before putting a workspace into production.

Partial Runs

When caching is turned on, running a translation causes data to be cached at every part of the workspace. In subsequent runs, those caches can be used instead of having to re-run entire sections of the workspace.

Here, for example, a workspace has been run with caching turned on:

Now the author makes a change to the AreaCalculator parameters:

Notice that the caches change color (to yellow) on the AreaCalculator and subsequent transformers. This color denotes that caches are stale; their data contents no longer match what the workspace would produce.

To get the new results, the author must re-run the workspace. However, they do not have to re-run the entire workspace; they can start the workspace at the point of change - the AreaCalculator:


NEW
New in FME 2019.0: the keyboard shortcut F5 now runs the workspace from any existing caches, instead of running the entire workspace, which you can do with Shift + F5.

Run From This causes the workspace to run from that point only, using data cached up until that point. Notice how hovering over the option causes all "downstream" transformers to be highlighted. They are the only ones that will be run. That makes the translation quicker.

The other option is Run To This. The author could use that option on the writer feature type and get much the same effect:

...but notice how the second branch from the StatisticsCalculator does not get highlighted. It will not be run. That shows how you can avoid running a particular section of workspace, in much the same way as if that connection had been disabled.


TIP
A partial run is particularly useful in avoiding re-reading data from its source; especially when the data comes from a slow, remote location such as a web service.

Also, caches can be saved with the workspace, when it is saved as a template. That means the workspace can be re-run using the caches from a previous session or even from another author!

results matching ""

    No results matching ""