Configuration and Setup Information
Configuration and setup messages in the FME log tell us vital information about FME's version and configuration, the system resources and how FME intends to use them, and what system paths are being used.
Starting translation... FME 2019.0.0.0 (20190328 - Build 19238 - WIN64) FME_HOME is 'C:\Program Files\FME2019\' FME Desktop Smallworld Edition (floating) Permanent License. Machine host name is: Training2019
For example, here you can see which version of FME is being used (FME2019.0.0.0, 64-bit), its license type (floating), and the machine name (Training2019). If you do have multiple FME versions installed, it’s a second location to check the correct one is running.
Following further configuration messages (about shared folders) is information about the working and temporary folders:
FME Configuration: Current working folder is `C:\FMEData2019\Workspaces\InstructorUse\DesktopAdv' FME Configuration: Temporary folder is `C:\Users\mireland\AppData\Local\Temp\2', set from environment variable `TEMP' System Status: 10.42 GB of disk space available in the FME temporary folder (C:\Users\mireland\AppData\Local\Temp\2) System Status: 128.00 TB of virtual memory available
This is important because (as we'll see) the temporary folder location has performance implications. Then there is information about the operating system:
Operating System: Microsoft Windows Server 10 Server 4.0 64-bit (Build 14393) FME Platform: WIN64 Locale: en_US Code Page: 1252 (ANSI - Latin I)
Language and encoding aren't performance indicators, but they can help us diagnose problems with encoding in the output.
Later in the log is important information about the system resources and FME’s memory management:
FME Configuration: Process limit is 8.00 GB of physical memory FME Configuration: Start freeing memory when process usage exceeds 24.00 GB of virtual memory FME Configuration: Stop freeing memory when process usage is below 18.00 GB of virtual memory
In this case, there is a limit of 8GB memory per process. The following numbers indicate how FME will manage memory resources. It will use 24GB of virtual memory and then it will start to release memory by caching features to disk. This caching will stop once virtual memory use is less than 18GB.
This way FME will perform to its potential automatically, while not taking so much memory that the system may fail or other processes on the system would suffer.
FME Lizard says… |
There can be understandable confusion about terms such as physical memory, virtual memory, address space, and swap space; so let's take a moment to explain them.
On a 64-bit system, address space (which is where data is actually stored) has an impractically high maximum (128TB) that has no practical value; other failures would occur long before that number is breached. So instead of reporting address space, 64-bit FME reports "Virtual Memory". This is a more realistic number that combines physical memory (the actual chips on your motherboard) with a portion of disk called Swap Space. |
Temporary Folder
For performance tuning one of the most critical parts of the log reports the temporary folder being used. When physical memory resources become low, FME starts to cache data to disk (the Swap Space) and swap that data in and out of physical memory as required.
The temporary folder is where it writes data to, so there are two important considerations for that folder.
Firstly it's important to ensure this folder has enough temporary disk space for the translation being carried out. Depending on the workspace, the transformations being carried out, and other processes happening on the computer; temporary disk space requirements may be many times greater than the size of the original dataset.
Secondly, it's useful if the disk written to is both fast and unused by any other process. It will not, for example, help performance to share the temporary disk with the operating system; additionally, a solid-state disk is much faster than a traditional hard drive.
FME Lizard says… |
The comparative benefits of RAM vs. SSD vs. HDD are hard to quantify. Do a web search if you don't believe me; people quote RAM as being 4x as fast, 20x as fast, 100,000x as fast!
In reality, it depends on so many factors. However, in general, use as much memory as you can to avoid caching, and use SSD so that - where caching/swapping is unavoidable - it runs as fast as possible. |