Detailed Description

Running the processes efficiently with fantastic display.

This class implements REST's main functionality - process running. multi- thread is enabled here which improves efficiency. Pause menu and progress bar is also provided which makes the work easier!

RESTsoft - Software for Rare Event Searches with TPCs

History of developments:

2014-june: First concept. As part of conceptualization of previous REST code (REST v2) Igor G. Irastorza

2017-Aug: Major change: added for multi-thread capability Kaixiang Ni

Definition at line 28 of file TRestProcessRunner.h.

#include <TRestProcessRunner.h>

Inheritance diagram for TRestProcessRunner:

Public Member Functions
void	BeginOfInit ()
	Reads information from rml config file. More...

	ClassDefOverride (TRestProcessRunner, 7)

void	ConfigOutputFile ()
	Forming an output file. More...

void	EndOfInit ()
	Ending of the startup procedure. More...

void	FillThreadEventFunc (TRestThread *t)
	Calling back the FillEvent() method in TRestThread. More...

Long64_t	GetFileSplitSize () const

TRestAnalysisTree *	GetInputAnalysisTree ()

TRestEvent *	GetInputEvent ()

Int_t	GetNextevtFunc (TRestEvent targetevt, TRestAnalysisTree targettree)
	Get next event and copy it to the address of targetevt. More...

int	GetNProcessedEvents () const

int	GetNProcesses () const

int	GetNThreads () const

TRestAnalysisTree *	GetOutputAnalysisTree ()

TFile *	GetOutputDataFile ()

std::string	GetProcInfo (std::string infoname)

double	GetReadingSpeed ()

ProcStatus	GetStatus () const

void	InitFromConfigFile () override
	To make settings from rml file. This method must be implemented in the derived class.

void	Initialize () override
	REST run class. More...

TRestEventProcess *	InstantiateProcess (TString type, TiXmlElement *ele)
	InstantiateProcess in sequential start up. More...

std::string	MakeProgressBar (int progress100, int length=100)
	Make a string of progress bar with given length and percentage.

void	MergeOutputFile ()
	Calls TRestRun::MergeOutputFile() to merge the main file with process's tmp file. More...

void	PauseMenu ()
	A pause menu providing some functions during the process. More...

void	PrintMetadata () override
	Implemented it in the derived metadata class to print out specific metadata information. More...

void	PrintProcessedEvents (Int_t rateE)
	Print number of events processed, file read speed, ETA and a progress bar.

Int_t	ReadConfig (const std::string &keydeclare, TiXmlElement *e)
	method to deal with iterated child elements More...

void	ReadProcInfo ()
	Create a process info list which used called by TRestRun::FormFormat(). More...

void	ResetRunTimes ()
	Reset running time count to 0.

void	RunProcess ()
	The main executer of event process. More...

bool	UseTestRun () const

void	WriteProcessesMetadata ()
	Write process metadata to fOutputDataFile.

Public Member Functions inherited from TRestMetadata
void	AddLog (std::string log="", bool print=true)
	Add logs to messageBuffer.

void	DoNotStore ()
	If this method is called the metadata information will not be stored in disk.

TVector2	Get2DVectorParameterWithUnits (std::string parName, TVector2 defaultValue=TVector2(-1, -1))

TVector3	Get3DVectorParameterWithUnits (std::string parName, TVector3 defaultValue=TVector3(-1, -1, -1))

TString	GetCommit ()
	Returns the REST commit value stored in fCommit.

std::string	GetConfigBuffer ()
	Returns the config section of this class.

std::string	GetDataMemberValue (std::string memberName)
	Get the value of data member as string. More...

std::vector< std::string >	GetDataMemberValues (std::string memberName, Int_t precision=0)
	Get the value of datamember as a vector of strings. More...

TString	GetDataPath ()
	Returns a std::string with the path used for data storage.

Double_t	GetDblParameterWithUnits (std::string parName, Double_t defaultValue=PARAMETER_NOT_FOUND_DBL)
	Gets the value of the parameter name parName, after applying unit conversion. More...

Bool_t	GetError () const
	It returns true if an error was identified by a derived metadata class.

TString	GetErrorMessage ()
	Returns a std::string containing the error message.

TString	GetLibraryVersion ()
	Returns the REST libraty version stored in fLibraryVersion.

TString	GetMainDataPath ()
	Gets a std::string with the path used for data storage.

Int_t	GetNumberOfErrors () const

Int_t	GetNumberOfWarnings () const

std::string	GetParameter (std::string parName, TString defaultValue=PARAMETER_NOT_FOUND_STR)
	Returns corresponding REST Metadata parameter from multiple sources. More...

std::string	GetSectionName ()
	Returns the section name of this class, defined at the beginning of fSectionName.

TRestStringOutput::REST_Verbose_Level	GetVerboseLevel ()
	returns the verboselevel in type of REST_Verbose_Level enumerator

TString	GetVerboseLevelString ()
	returns the verbose level in type of TString More...

TString	GetVersion ()
	Returns the REST version stored in fVersion.

Int_t	GetVersionCode ()

UInt_t	GetVersionMajor () const

UInt_t	GetVersionMinor () const

UInt_t	GetVersionPatch () const

Bool_t	GetWarning () const
	It returns true if an error was identified by a derived metadata class.

TString	GetWarningMessage ()
	Returns a std::string containing the warning message.

TRestMetadata *	InstantiateChildMetadata (int index, std::string pattern="")
	This method will retrieve a new TRestMetadata instance of a child element of the present TRestMetadata instance based on the `index` given by argument, which defines the element order to be retrieved, 0 for first element found, 1 for the second element found, etc. More...

TRestMetadata *	InstantiateChildMetadata (std::string pattern="", std::string name="")
	This method will retrieve a new TRestMetadata instance of a child element of the present TRestMetadata instance based on the `name` given by argument. More...

Bool_t	isCleanState () const

Bool_t	isOfficialRelease () const

Int_t	LoadConfigFromBuffer ()
	Initialize data from a string element buffer. More...

Int_t	LoadConfigFromElement (TiXmlElement eSectional, TiXmlElement eGlobal, std::map< std::string, std::string > envs={})
	Main starter method. More...

Int_t	LoadConfigFromFile (const std::string &configFilename, const std::string &sectionName="")
	Give the file name, find out the corresponding section. Then call the main starter.

virtual void	Merge (const TRestMetadata &)

TRestMetadata &	operator= (const TRestMetadata &)

void	Print ()
	Implementing TObject::Print() method.

void	PrintConfigBuffer ()
	Print the config xml section stored in the class. More...

void	PrintMessageBuffer ()
	Print the buffered message.

void	PrintTimeStamp (Double_t timeStamp)
	Print the current time on local machine. More...

void	SetConfigFile (std::string configFilename)
	set config file path from external

void	SetError (std::string message="", bool print=true, int maxPrint=5)
	A metadata class may use this method to signal that something went wrong.

void	SetHostmgr (TRestManager *m)
	Set the host manager for this class.

void	SetSectionName (std::string sName)
	set the section name, clear the section content

void	SetVerboseLevel (TRestStringOutput::REST_Verbose_Level v)
	sets the verbose level

void	SetWarning (std::string message="", bool print=true, int maxPrint=5)
	A metadata class may use this method to signal that something went wrong.

void	Store ()
	If this method is called the metadata information will be stored in disk.

	TRestMetadata (const TRestMetadata &)

virtual void	UpdateMetadataMembers ()
	Method to allow implementation of specific metadata members updates at inherited classes.

virtual Int_t	Write (const char *name=nullptr, Int_t option=0, Int_t bufsize=0)
	overwriting the write() method with fStore considered

void	WriteConfigBuffer (std::string fName)
	Writes the config buffer to a file in append mode.

	~TRestMetadata ()
	TRestMetadata default destructor.

Private Attributes
TRestAnalysisTree *	fAnalysisTree

Int_t	fEventsToProcess

TTree *	fEventTree

Int_t	fFileCompression

Long64_t	fFileSplitSize

Int_t	fFirstEntry

Bool_t	fInputAnalysisStorage

TRestEvent *	fInputEvent

Bool_t	fInputEventStorage

Int_t	fNBranches

Int_t	fNFilesSplit

Bool_t	fOutputAnalysisStorage

TFile *	fOutputDataFile

TString	fOutputDataFileName
	the TFile pointer being used

TRestEvent *	fOutputEvent

Bool_t	fOutputEventStorage

Int_t	fProcessedEvents

std::map< std::string, std::string >	fProcessInfo

Int_t	fProcessNumber

ProcStatus	fProcStatus

TRestRun *	fRunInfo

Bool_t	fSortOutputEvents

Int_t	fThreadNumber

std::vector< TRestThread * >	fThreads

Bool_t	fUsePauseMenu

Bool_t	fUseTestRun
	Number of files being split.

Bool_t	fValidateObservables

Additional Inherited Members
Protected Member Functions inherited from TRestMetadata
std::string	ElementToString (TiXmlElement *ele)
	Convert an TiXmlElement object to string. More...

TVector2	Get2DVectorParameterWithUnits (std::string parName, TiXmlElement *e, TVector2 defaultValue=TVector2(-1, -1))

TVector3	Get3DVectorParameterWithUnits (std::string parName, TiXmlElement *e, TVector3 defaultValue=TVector3(-1, -1, -1))

Double_t	GetDblParameterWithUnits (std::string parName, TiXmlElement *e, Double_t defaultVal=PARAMETER_NOT_FOUND_DBL)

TiXmlElement *	GetElement (std::string eleDeclare, TiXmlElement *e=nullptr)
	Get an xml element from a given parent element, according to its declaration.

TiXmlElement *	GetElementFromFile (std::string configFilename, std::string NameOrDecalre="")
	Open an xml encoded file and find its element. More...

TiXmlElement *	GetElementWithName (std::string eleDeclare, std::string eleName)
	Get an xml element from the default location, according to its declaration and its field "name".

TiXmlElement *	GetElementWithName (std::string eleDeclare, std::string eleName, TiXmlElement *e)
	Get an xml element from a given parent element, according to its declaration and its field "name".

std::string	GetFieldValue (std::string fieldName, std::string definition, size_t fromPosition=0)
	Gets field value in an xml element string by parsing it as TiXmlElement.

std::string	GetFieldValue (std::string parName, TiXmlElement *e)
	Returns the field value of an xml element which has the specified name. More...

std::string	GetKEYDefinition (std::string keyName)
	Gets the first key definition for keyName found inside buffer starting at fromPosition. More...

std::string	GetKEYDefinition (std::string keyName, size_t &Position)

std::string	GetKEYDefinition (std::string keyName, size_t &Position, std::string buffer)

std::string	GetKEYDefinition (std::string keyName, std::string buffer)

std::string	GetKEYStructure (std::string keyName)
	Gets the first key structure for keyName found inside buffer after fromPosition. More...

std::string	GetKEYStructure (std::string keyName, size_t &Position)

std::string	GetKEYStructure (std::string keyName, size_t &Position, std::string buffer)

std::string	GetKEYStructure (std::string keyName, size_t &Position, TiXmlElement *ele)

std::string	GetKEYStructure (std::string keyName, std::string buffer)

TiXmlElement *	GetNextElement (TiXmlElement *e)
	Get the next sibling xml element of this element, with same eleDeclare.

std::string	GetParameter (std::string parName, size_t &pos, std::string inputString)
	Returns the value for the parameter name parName found in inputString. More...

std::string	GetParameter (std::string parName, TiXmlElement *e, TString defaultValue=PARAMETER_NOT_FOUND_STR)
	Returns the value for the parameter named parName in the given section. More...

std::pair< std::string, std::string >	GetParameterAndUnits (std::string parname, TiXmlElement *e=nullptr)
	Returns the unit string of the given parameter of the given xml section. More...

std::map< std::string, std::string >	GetParametersList ()
	It retrieves a map of all parameter:value found in the metadata class.

TString	GetSearchPath ()

virtual void	InitFromRootFile ()
	Method called after the object is retrieved from root file.

virtual Int_t	LoadSectionMetadata ()
	This method does some preparation of xml section. More...

void	ReadAllParameters ()
	Reflection methods, Set value of a datamember in class according to TRestMetadata::fElement. More...

void	ReadParametersList (std::map< std::string, std::string > &list)
	It reads a parameter list and associates it to its corresponding metadata member. par0 --> fPar0.

std::string	ReplaceConstants (const std::string buffer)
	Identifies "constants" in the input buffer, and replace them with corresponding value. More...

std::string	ReplaceVariables (const std::string buffer)
	Identifies environmental variable replacing marks in the input buffer, and replace them with corresponding value. More...

void	ReSetVersion ()
	Resets the version of TRestRun to REST_RELEASE. Only TRestRun is allowed to update version.

std::string	SearchFile (std::string filename)
	Search files in current directory and directories specified in "searchPath" section. More...

void	SetLibraryVersion (TString version)
	Set the library version of this metadata class.

TiXmlElement *	StringToElement (std::string definition)
	Parsing a string into TiXmlElement object. More...

	TRestMetadata ()
	TRestMetadata default constructor.

	TRestMetadata (const char *configFilename)
	constructor

void	UnSetVersion ()
	Resets the version of TRestRun to -1, in case the file is old REST file. Only TRestRun is allowed to update version.

Protected Attributes inherited from TRestMetadata
std::string	configBuffer
	The buffer where the corresponding metadata section is stored. Filled only during Write()

std::string	fConfigFileName
	Full name of the rml file. More...

std::map< std::string, std::string >	fConstants
	Saving a list of rml constants. name-value std::pair. Constants are temporary for this class only.

TiXmlElement *	fElement
	Saving the sectional element together with global element.

TiXmlElement *	fElementGlobal
	Saving the global element, to be passed to the resident class, if necessary.

Bool_t	fError = false
	It can be used as a way to identify that something went wrong using SetError method.

TString	fErrorMessage = ""
	A std::string to store an optional error message through method SetError.

TRestManager *	fHostmgr
	All metadata classes can be initialized and managed by TRestManager.

Int_t	fNErrors = 0
	It counts the number of errors notified.

Int_t	fNWarnings = 0
	It counts the number of warnings notified.

std::string	fSectionName
	Section name given in the constructor of the derived metadata class.

Bool_t	fStore
	This variable is used to determine if the metadata structure should be stored in the ROOT file.

std::map< std::string, std::string >	fVariables
	Saving a list of rml variables. name-value std::pair.

TRestStringOutput::REST_Verbose_Level	fVerboseLevel
	Verbose level used to print debug info.

Bool_t	fWarning = false
	It can be used as a way to identify that something went wrong using SetWarning method.

TString	fWarningMessage = ""
	It can be used as a way to identify that something went wrong using SetWarning method.

std::string	messageBuffer
	The buffer to store the output message through TRestStringOutput in this class.

endl_t	RESTendl
	Termination flag object for TRestStringOutput.

Member Function Documentation

◆ BeginOfInit()

void TRestProcessRunner::BeginOfInit ( )

Reads information from rml config file.

It first checks if a friendly TRestRun object is initialized in TRestManager, if so, it reads the following configuration items:

firstEntry, lastEntry, eventsToProcess. These indicates how many events we need to process.
Tree branch list. can be inputAnalysis, inputEvent, outputEvent.
Number of thread needed. A list TRestThread will then be instantiated.

Definition at line 114 of file TRestProcessRunner.cxx.

◆ ConfigOutputFile()

void TRestProcessRunner::ConfigOutputFile ( )

Forming an output file.

Merge all the process's file together, write TRestRun metadata, and then save process metadata to that file

Definition at line 951 of file TRestProcessRunner.cxx.

◆ EndOfInit()

void TRestProcessRunner::EndOfInit ( )

Ending of the startup procedure.

It first sets input event as the first event in process chain, and then do a Validation of the process chain. Finally it calls ReadProcInfo() and create a process info list

Definition at line 282 of file TRestProcessRunner.cxx.

◆ FillThreadEventFunc()

void TRestProcessRunner::FillThreadEventFunc ( TRestThread * t )

Calling back the FillEvent() method in TRestThread.

This method is locked by mutex. There can never be two of it running simultaneously in two threads. As a result threads will not write their files together, thus preventing segmentaion violation.

Definition at line 792 of file TRestProcessRunner.cxx.

◆ GetNextevtFunc()

Int_t TRestProcessRunner::GetNextevtFunc	(	TRestEvent *	targetevt,
		TRestAnalysisTree *	targettree
	)

Get next event and copy it to the address of targetevt.

If can, it will also set the observal value of targettree according to the local analysis tree.

This method is locked by mutex. There can never be two of it running simultaneously in two threads.

If there is a single thread process, the local input event will be set to the out put of this process. The targettree will not be changed.

If not, the local input event and analysis tree will be updated after calling TTree::GetEntry(). The observables in the local tree will be copyed to the targettree.

Finally the data in the input event will get cloned to the targetevt by root streamer.

If the current entry is the last entry of the input tree, or the single thread process stops to give a concret pointer as the output, the process is over. This method returns -1.

Definition at line 759 of file TRestProcessRunner.cxx.

◆ Initialize()

void TRestProcessRunner::Initialize ( )

overridevirtual

REST run class.

Setting default values of class' data member.

Reimplemented from TRestMetadata.

Definition at line 73 of file TRestProcessRunner.cxx.

◆ InstantiateProcess()

TRestEventProcess * TRestProcessRunner::InstantiateProcess	(	TString	type,
		TiXmlElement *	ele
	)

InstantiateProcess in sequential start up.

It instantiates a the object by the method TClass::GetClass(), giving it type name. Then it asks the process object to LoadConfigFromFile() with an xml section.

Definition at line 1049 of file TRestProcessRunner.cxx.

◆ MergeOutputFile()

void TRestProcessRunner::MergeOutputFile ( )

Calls TRestRun::MergeOutputFile() to merge the main file with process's tmp file.

After this operation, fOutputDataFile will be set to TRestRun's output file

Definition at line 1006 of file TRestProcessRunner.cxx.

◆ PauseMenu()

void TRestProcessRunner::PauseMenu ( )

A pause menu providing some functions during the process.

It can:

Change verbose level
Change number of events to process
Push forward event by event
Print the latest processed event
Quit the process directly with file saved

Definition at line 554 of file TRestProcessRunner.cxx.

◆ PrintMetadata()

void TRestProcessRunner::PrintMetadata ( )

overridevirtual

Implemented it in the derived metadata class to print out specific metadata information.

Prints metadata content on screen. Usually overloaded by the derived metadata class.

Reimplemented from TRestMetadata.

Definition at line 1174 of file TRestProcessRunner.cxx.

◆ ReadConfig()

Int_t TRestProcessRunner::ReadConfig	(	const std::string &	keydeclare,
		TiXmlElement *	e
	)

method to deal with iterated child elements

If child element is declared as "addProcess", then multiple new process will be instantiated using sequential startup method, by calling InstantiateProcess() The processes will be added into each TRestThread instance. If the process is external process, then it will be sent to TRestRun.

Definition at line 213 of file TRestProcessRunner.cxx.

◆ ReadProcInfo()

void TRestProcessRunner::ReadProcInfo ( )

Create a process info list which used called by TRestRun::FormFormat().

Items: FirstProcess, LastProcess, ProcNumber

Definition at line 314 of file TRestProcessRunner.cxx.

◆ RunProcess()

void TRestProcessRunner::RunProcess ( )

The main executer of event process.

Things doing in this method:

Call each thread to prepare their process chain, output tree and output file. The method is PrepareToProcess().
Print metadata for each process
Set output tree by cloning the TRestThread output tree
Reset run count, modify ROOT mutex to make it support multi-thread.
Call each thread to start. The threads running the processes are detatched after this calling.
The main thread loops for progress bar while waiting the child threads to finish.
After finished, print some information, reset ROOT mutexs.
Call ConfigOutputFile() method to save the output file

!!!!!!!!!!!Important!!!!!!!!!!!!

Definition at line 342 of file TRestProcessRunner.cxx.

Field Documentation

◆ fEventTree

TTree* TRestProcessRunner::fEventTree

private

indicates the name of the first file created as output data file. The actual output file maybe changed if tree is too large

Definition at line 42 of file TRestProcessRunner.h.

The documentation for this class was generated from the following files: