CMS event visualization FAQ

# CMS event visualization (unofficial) FAQ

What tools do I have to display CMS events

Right now , there are three available iguana,frog and fireworks.Iguana and Fireworks are officially supported by CMS management and have behind small teams of developers. Frog has no CMS official support and is essentially the result of the work of a single person: Loic Quertenmont.

Why so many tools?

Considering that an ideal event display should allow the casual user:

- The display of each event in the CMS datasets with a click
- We should be able to scan in a minute hundreds of events for relevant features.
- The display should be possible everywhere even in an Internet cafe'
- The program should also produce quality images/animations for press and publishing.

The task is so complex that we need many approaches.Each one of the three programs is better for some tasks and the three make us more near to the goal of the ideal event display.

## Real data visualization

Ok, I just need to see what a RECO dataset from Castor contains. What is the easiest way to do it?

(Complete and up-to-date instructions available from https://twiki.cern.ch/twiki/bin/view/CMS/WorkBookFireworks )

First I suppose that you have a PC with adequate hardware resources. If not you are out of luck. Then you are on a network with a fast connection to Cern. In this case the easiest way (but not the fastest or the best concerning the quality of images) is:

    ssh -l your_user_name lxplus
    /afs/cern.ch/cms/fireworks/cmsShow21/cmsShow -c /afs/cern.ch/cms/fireworks/cosmics.fwc rootfile (add rfio:/castor/cern.ch/cms at the beginning if the dataset is on castor)

This was easy to use but I want to have access to all data collections of a single event.What you suggest:

Always with the same caveat(adequate hardware+fast connection).

    ssh -l your_user_name lxplus
    cd /tmp/your_user_name (or some other directory where you have lot of space)
    scramv1 p CMSSW CMSSW_2_1_10 (release compatible with the data you want to see)
    cd CMSSW_2_1_10/src
    project CMSSW
    eval `scramv1 runtime -csh`
    iguana rootfile (add rfio:/castor/cern.ch/cms at the beginning if dataset on castor)
    (select next event)

You get an empty 3D window on the right and a huge tree on the left that describes everything you have in the event. The idea is that you check an entry and you get it displayed on the right window. You select the text of the entry and you get it printed in the small white window on the right bottom. This is true but unfortunately it is very easy to get stuck when you do this. So to get something done you should proceed with big caution and know in advance what you are doing. If you try to follow the instructions given below you should produce no disaster:(the first time it is better to start with an event that you know is interesting when you examined the sample with fireworks)

    in the "Event" menu goto the desired event.You must write run number and event number correctly.
    select the tracks collections that interest you
    select "Reco Tracks Det"
    select "Draw style" "still draw style" "wireframe" with right mouse button on 3D window.
    select sistrip clusters and pixel clusters
    select hcal endcap
    position the image

Now you can start scanning events using the key CTRL+N

It is possible to create rz and rphi windows and to look at the event with multiple windows but unfortunately this can be done "safely" only with a local installation of iguana.And this installation requiring full CMSSW isn't straightforward.

Instead of installing locally iguana there are other ways to scan fast 3D events?

An alternative is to use frog. Frog consists of two components :a frog `visualizer` and a `producer`. The visualizer can be quickly installed locally. It displays events only in a special format contained in .vis file created by the producer. Howevere there are some .vis files already available. . To see them:

    cd frog_directory
    edit config.txt (change the URL in the line "InputVisFile=")
    ./frog

Frog seems promising.But how do I install it on my computer ?

(Complete and up-to-date instructions available from http://projects.hepforge.org/frog/ )

Under windows: download zip file. Uncompress it in some directory and you are in businnes! Alternatively download from CVS repository soft directory with all subdirectories. The file to execute is `frog.exe`.

Under linux: it can be more difficult. Although frog doesn't need CMSSW, installing it on a arbitrary linux distribution may sometimes require the installation of additional software. This consists of standard Linux libraries and,if this happens , you should know how to install these.

If instead, you are running linux slc4 on your computer and you can access CMSSW software then , by following the steps indicated below, you are sure to have the necessary libraries installed.

    cd directory_with_enough_space

    project CMSSW
    scramv1 project CMSSW CMSSW_2_1_11 (choose always the last)
    cd CMSSW_2_1_11/src
    cvs co -r Version_1_104 -d Visualisation/Frog UserCode/FastOpenGlDisplayer (check on the frog site which is the last)
    cd Visualisation/Frog/soft/
    make

What if I want to see what a RAW dataset contain?

In this case fireworks isn't of much help and iguana is too slow. The problem here is that you need to redo the reconstrunction using CMSSW. This being the case, then the fastest way to scan raw data is to use frog. You have to run a CMSSW cfg like the one needed by iguana but this will produce in the python directory a temporary .vis (InLive.vis) file that you can immediately inspect from a remote computer with the frog visualizer installed . You run the CMSSW job on lxplus and write this temporary file on $HOME/scratchx/ where it is accessible from every computer that can access locally /afs/cern.ch/user/. Here the recipe to follow:

    (on lxplus , if not yet installed, you have to install the latest frog:)
    cd directory_with_enough_space
    project CMSSW
    scramv1 project CMSSW CMSSW_2_1_9 (choose always the last)
    cd CMSSW_2_1_9/src
    cvs co -r Version_1_104 -d Visualisation/Frog UserCode/FastOpenGlDisplayer (check on the frog site which is the last)
    scramv1 b
    cd Visualization/Frog/python
    (end of installation: to be done only once)
    cd frog_directory
    cd CMSSW_2_1_9/src
    project CMSSW
    eval `scramv1 runtime -csh`
    cd Visualization/Frog/python
    edit Frog_Analyzer_Cosmics_FromRaw.py indicating the name of input dataset
    cmsRun   Frog_Analyzer_Cosmics_FromRaw.py
    (you should see the job printing a line for each event processed)
    (on your pc)
    (if not yet done) change InputViSFile to point to temporary file
    start frog

You can start looking at events as soon as they are processed on lxplus.Just push the R (for reload) key when you want the latest events to be accessible. Or even better, edit frog configuration file (config.txt) with these 3 parameters. These make frog always display the latest event in the .vis file and will reload .vis files every 1sec. (So displayed event will always be the latest available event in the .vis)

    InputVisFile = {../python/InLive.vis}; // In order to load events in the tempory .vis file (Name InLive.vis is hardcoded, so you have to use this)
    Event_Number = -1; // This will always display the latest event in the file
    Event_Time = 1; // This will check the .vis file every 1 sec.

Can I use frog in the same way to look at RECO data?

Yes. The procedure is the same : replace only the file `Frog_Analyzer_Cosmics_FromRaw.py` with `Frog_Analyzer_Cosmics.py`

I have the latest release of Iguana installed locally on my pc. So it should be fast enough. How can I use it to look at RAW data?

(Complete and up-to-date instructions available from Tracker visualization wiki )

    cd CMSSW_2_1_10/src/
    eval `scramv1 runtime -csh`
    cmscvsroot CMSSW
    cvs login   (give 98passwd as password)
    cvs co  VisReco/VisCustomTracker/test
    cd VisReco/VisCustomTracker/test
    (edit iguanaraw.cfg changing input dataset)
    iguana iguanaraw.cfg

Alternatively if you want to use the plugin VisCustomTracker and to look only at tracker data , then use `trackerIguanaraw.py` in the same directory.

Can I use a cfg file working with RAW data on RECO data?

Yes but you have to do some simple changes:

Replace

    process.p = cms.Path( process.RawToDigi_woGCT * process.reconstructionCosmics)

with

    process.mypath = cms.Path(process.siPixelRecHits*process.siStripMatchedRecHits*process.TrackRefitter)

Add at the beginning:

    process.load("RecoTracker.TrackProducer.RefitterWithMaterial_cff")

So which of the three programs is the easiest to use?

Frog. Fireworks comes second. And then iguana.

Which is the easiest to install?

Fireworks. The frog visualizer is also very easy to install and can be used to look at events contained in .vis files available through Internet or accessible locally through `afs`.

Which is the fastest to scan many events?

Frog and Fireworks installed locally are the fastest and have almost the same speed. But Fireworks can be used only for RECO data.Iguana installed locally with enough resources can , in principle , reach the same speed. In practice , since in most cases it needs some CMSSW reconstruction, the real speed can be ten times slower. This is the case for example of raw data.

Which is the best to bring out the most relevant features of an event at first sight ?

Frog uses in a clever way the 3D representation, animation and fast exclusion ,inclusion of parts of signal/detector. For this reason is the best for the casual (non expert) user. Fireworks has the cleanest layout. Iguana with a black initial window is easy to beat (-:) but puts the complete event at your hand. After you become expert on how to configure it and on which signal/detector use, you will prefer it to the other two.

Which is the best if you are on a slow network connection to Cern and have no CMSSW installed locally?

Then the only alternative is frog. Frog is also the only one that can,in principle, also be run from an Internet cafe'.

Which is the best to produce nice pictures for the press or for publishing?

Iguana.

Which is the best to filter events with arbitrary cuts?

fireworks

What other tasks can be performed only by iguana?

All visualization tasks where its integration with CMSSW software is crucial.

If you need to know exactly what each event contains. The other two programs are more easy to use because they shield you from the complexity of the event. This means for example that displaying what is in some exotic data collection is a click away in iguana, requires some work with the menus and some writing with fireworks, and can be pretty impossible in frog (because the collection isn't in the .vis file) needing some expert work on the producer.

Another use of iguana impossible with the other two is the building of integrated signal trackermaps by VisCustomTracker.

What is the minimum hardware I need to run them?

A graphics card that understands opengl and is fast enough is mandatory for all three programs. A memory size > 1Gb also is very important. Big screen resolution then helps getting the most from these three tools. Frog is the software that needs less resources. For this , it is the only one that can run also on a Internet cafe' computer (provided that it has an adequate graphics card).

On which platform can I run them?

All three run on Linux SLC4. The frog visualizer can also run on Windows and fireworks on MAC. As said before frog is the only one that can be run on a "internet cafe'" computer: i.e. it doesn't need any CMSSW software and can get event information though a web adress.

I am interested in the events of some run. How can I find the datasets where it is stored?

Go to dbs . Click on "DBS discovery page". Then write in the first textfield "find run where run=xxxx". Choose the dataset in the page that you get. Click on "details" to know the Lumi section of each file in a dataset. This number gives the order in which the events where read.

The number of events in a dataset can be obtained by a query like this:

    find sum(file.numevents) where dataset like /Cosmics/Commissioning08-PromptReco-v2/RECO  and run=68021

Once you have chosen the dataset , check that it is really present on castor by using the command:

    rfdir /castor/cern.ch/cms/dataset_name

I am interested in a specific event of some run. How can I find the datasets where it is stored?

It seems that the information about first-last event in a dataset isn't written anywhere. The datasets are also listed on dbs in random order. The ordered list can be obtained if you sort them by lumi section number.This doesn't seem to be done automatically but ,once you know the dasaset name, you can have a list containing a line for each file name with the lumi sector. The search to be written in the textfield is:

    find lumi,file where run=XXXXX AND dataset=dataset_name(for example:/Cosmics/Commissioning08-v1/RAW)

Using this list you have to reason in this way. I search event 38099. The mean number of events in each dataset seem to be 30000 (this is reported in details list) so it should be in the file with lumi=2.

The web interface of dbs is very slooow:there is a way to get quick answers?

Yes. Just go in a directory where CMSSW is installed and give the usual "eval ...." . At this point a command "dbs" is defined that allows you to make searches by a command line. Here is an example that can be used as template for any request:

    dbs --search --query "find lumi,file,file.numevents where dataset = */FEVT and run=122294 " --url=http://cmsdbsprod.cern.ch/cms_dbs_caf_analysis_01/servlet/DBSServlet

How I can find the correct global tag to be used with a specific run?

Most of the CMSSW cfg files used by iguana and frog have the following line:

    process.GlobalTag.globaltag = "CRUZET4_V6P::All"

where the magic word in the " " is the famous global tag. The list of spells can be reached from here and depends on CMSSW release and run number among other things. For example , for the normal offline use, for a CRAFT run you would use the global tag "CRAFT_VxP::All" with x the highest available when you run the program.

How I can know the correct magnetic field for a run?

Look this service.

How I can find the DQM summary plots of a run?

Use the DQM gui. More detailed plots can be examined by giving the following command on lxplus (in a directory CMSSW enabled):

    root /afs/cern.ch/cms/CAF/CMSCOMM/COMM_DQM/data/Cosmics/068/021/DQM_V0001_R000068021__Cosmics__Commissioning08-PromptReco-v2__RECO.root

(in this example we looked at run 68021 but it is quite obvious how to change run:the directory contains only 1 file and it is enough to give TAB to select it).

How can I know if a run is good?

Use the list of all runs with GOOD/BAD rating for all subdetectors.. Also this.

How can I plot some quantity that can be computed from DQM histograms for each run to see how it changes?

Luca Malgeri in a post has provided a recipe based on a python program that uses a cfg file and a list of runs. Just run it (in a CMSSW enabled directory on lxplus) giving:

    python DQMscan.py

and then open in a browser dqcosm.html that should look like this

How I can know if during (a long run) there was some important change of working conditions?

There is this study by Andrea Venturi on SiStrip

There is a way to display only events with tracks?

The best way to access only events with tracks is to access skims. You can find them on dbs doing a search like this:

    find dataset where dataset=/Cosmics/*Pointing*

You get in this way a list of RECO datasets . If you are interested in a specific run you should select "Run info" and then the Run that interests you.

There is a way to display only events filtered by an arbitrary cut?

Yes, fireworks allow you to write arbitrary cuts in CINT and display only events that obey the cuts. For example , if in the "Filter" text field you write:

    recoTracks_ctfWithMaterialTracksP5__Rec.@obj.size()>1 | SiStripClusteredmNewDetSetVector_siStripClusters__Rec.obj.@m_data.size()>40

you will get displayed only events with more than one track or more than 40 tracker modules hit. For other examples of cuts just look this : List of useful commands to explore dataset

The same result is obtained in frog by running the Analyzer with a CMSSW filter. A CMSSW filter should in principle work also for Iguana but I was ,until now, unable to use it.

With iguana and frog you can use also another simple trick: give a list of events in the cfg file. This is an example on the way how you do it:

    eventsToProcess = cms.untracked.VEventID('68021:15254','68021:29485','68021:83753','68021:89755','68021:113003','68021:365284','68021:369973','68021:365284'),

This list could be created automatically by a root macro where you apply the cut.

How can I access the image server on cmstac05 outside Cern?

I suppose that you have root access to your Linux box outside Cern. Access as root /etc/hosts/ adding the line

    127.0.0.1 cmstac05.cern.ch

Then you should open the following terminal on lxplus:

    ssh -l your_username -L 80:cmstac05.cern.ch:80 lxplus.cern.ch

What if I am on Windows and I want to access cmstac05 outside Cern?

There is a way to know how many events with tracks are in a specific dataset?

The easiest way is:

    cd /tmp/Your_user_name/ (or other directory with enough space)
    scramv1 p CMSSW CMSSW_2_1_10 (a release compatible with the dataset you want to inspect)
    cd CMSSW_2_1_10/src/
    root dataset_name (if in castor then add rfio:/castor/cern.ch/cms at beginning)
    TBrowser b
    (select root files)
     (select dataset)
     (select Events)
     (select track collection)
     (select obj)
     (select size)
     you should get the histogram of number of tracks per event in collection

The same trick can be used to find if the sample has some other peculiar feature. For example events with big number of clusters. But keep in mind that root doesn't seem to work well with rfio: datasets. If it starts crashing for every histogram you request, then copy first the dataset locally on /tmp/ using

    rfcp rfio_dataset /tmp/

and then work on the local copy.

There is a way to know which are the events with highest number of tracks?

Proceed like we did before to plot the number of tracks for example in collection ctfWithMaterialTracks. Let's suppose you see there are 3 events with number of tracks > 20. Give in root the following command:

    Events->Scan("recoTracks_ctfWithMaterialTracksP5__Rec.@obj.size():EventAuxiliary.id_.event_","recoTracks_ctfWithMaterialTracksP5__Rec.@obj.size()>20")

You should get a list like this:

    ***********************************
    *    Row    * recoTrack* eventAuxi*
    ***********************************
    *    9107   *        22*    212750*
    ....

9107 is the number of event in the dataset. 212750 the event number in run.

List of useful commands to explore dataset ;directory containg sample macros

If you want to save with Events->Scan lists of events, then use

    Events->SetScanField(0);
    Events->Scan(...);>eventlist.txt

How can I get an ascii file with all castor files of a run ordered by lumi sector?

On dbs do the following search:

    find lumi,file where run=68021 and dataset=/Cosmics/Commissioning08-PromptReco-v2/RECO

Select "all" then cut the result and paste it creating a file "filelist.txt". Now (on linux) give the command:

    cat filelist.txt | sort -n | cut -d , -f 2 > sfilelist.txt

sfilelist.txt contains the sorted list of files.

How can build an ntuple containing for each event of a run some very basic information like event number , LS, number of digi, number of clusters, number of tracks, etc: so I can quickly have a list of possibly interesting events?

How can I move events around if I don't have access on my graphics computer to castor?

Normally you have to do this in two steps:

    ssh lxplus -l your_username
    cd /tmp/your_username/
    rfcp /castor/cern.ch/cms/dataset_name .
    scp dataset_name your_username@your_computername:/your_datadirectory/

If you want to copy only a few events , then :

    cd CMSSW_ ../src/
    eval ...
    edit the following cfg file as copy.py:
    import FWCore.ParameterSet.Config as cms

    process = cms.Process("COPY")

    process.maxEvents = cms.untracked.PSet( input = cms.untracked.int32(10) )

    process.source = cms.Source("PoolSource",
        fileNames = cms.untracked.vstring("/store/relval/CMSSW_2_1_6/RelValTTbar/GEN-SIM-DIGI-RAW-HLTDEBUG-RECO/IDEAL_V9_v1/0000/00B3662C-DA78-DD11-88E6-001617C3B70E.root")
        )

    process.copyAll = cms.OutputModule("PoolOutputModule", fileName = cms.untracked.string("reco.root") )
    process.printEventNumber = cms.EDAnalyzer("AsciiOutputModule")
    process.out = cms.EndPath(process.copyAll + process.printEventNumber)

    cmsRun copy.py
    scp outputfile  your_username@your_computer:/your_datadirectory/

## Montecarlo events visualization

What is the easiest way to look at Montecarlo events?

As usual I would suggest to start with fireworks on lxplus. This will give you a rough idea on how the events look like.

    ssh lxplus -l your_username
    cd /tmp/your_username/
    /afs/cern.ch/cms/fireworks/cmsShow21/cmsShow rootfile

Ok done: now I want to look at the events in detail using iguana. I can run it locally on adequate machine. What I do?

Start in the usual way:

    cd directory_with_enough_space
    scramv1 p CMSSW CMSSW_2_1_10
    cd CMSSW_2_1_10/src
    iguana rootfile
    (next event)

If you start playing with the menu on the left you see there is a problem:you can't show tracks. This is because now rechits are no more stored on disk. The thing to do is to run the track refitter before iguana. In practice:

    stop iguana
    mv iguana(Tab) iguana.py
    vi iguana.py
    (add the following lines at the beginning)
    process.load("TrackingTools.TrackRefitter.TracksToTrajectories_cff")

    process.load("RecoTracker.TrackProducer.RefitterWithMaterial_cff")
    process.load("RecoTracker.TransientTrackingRecHit.TransientTrackingRecHitBuilderWithoutRefit_cfi")
    process.TrackRefitter.src = 'generalTracks'
    process.TrackRefitter.TrajectoryInEvent = True

     (and this other as last line)
     process.p = cms.Path(process.TrackRefitter)
    iguana iguana.py
    (using File menu create new rphi and rz window)
    (maximize display then Window/tile)
    (select parts of detector to display using the entries "Muon" "ECAL" "HCAL" "Tracker"at the end:don't use the Recodetector entries!)
    (zoom and position the detector in 3D window)
    (select next event)
    (select signal that you want to display: i.e. generaltracks, Calotowers (all), muons, dt and csc segments)
    (you may need also to change some parameters using the menu Configure/Edit : for example
    "VisCaloEnergyScale" )

Can I use frog to look at MC data?

Yes. The procedure is the same used before for RAW data : replace only the file `Frog_Analyzer_Cosmics_FromRaw.py` with `Frog_Analyzer.py`

Where can I find MC events to test my cfg file?

Just look for most recent information at the Forum: RelVal Samples and Release Testing. Also you can try yourself searching on lxplus:

    rfdir /castor/cern.ch/cms/store/relval/

## Furter advice

How I can know if a new CMSSW release requests changes to my visualization cfg files?

Look in this directory(accessible only from Cern) . Starting from 229 I am trying to store some sample images for each release plus the cfg file used to produce them.

Bibliography:

- Cruzet 3 event display recipe
- Cruzet 4 event display recipe
- Beam Halo event display recipe
- CRAFT Events at 3.8T
- Event time separation distribution
- Results from visual scan
- Visual summary of CRAFT data
- SiStripQualitySummaryReport's images
- Validating CMS event display tools with real data

Back to Old fart guide to CMS software

Maintained by Giuseppe Zito:Giuseppe.Zito@cern.ch.Last revised