In-situ creation and addition of containers to the EventStore

[hegner]

Define a policy about enabling "on-demand" creation and addition of containers

(a follow up on #109)

[tmadlener]

I have just had a brief look into the possibility of having something like "per-event collections", respectively the possibility of not having to register collections in an initialization phase prior to the actual processing. For context: the initialization phase approach is something that Gaudi does in order to e.g. facilitate scheduling of the different algorithms, because the dependencies can be resolved in this initialization phase. The "per-event collections" is similar to what LCIO does in ILCSoft, where the Event is the main entry point and collections can be added to it without first having to initialize the collections.

From what I gathered, it seems that this will not be easy with a "naive" root approach of storing the event data in a TTree as it is currently done. The problem is that the TTree has the notion of this columnar data storage. It is easily possible to add additional branches at any point in the data processing, but this branch will be visible for all events that are stored in the tree.

For example

auto* tree = new TTree("test_tree", "test tree");

int index = 0;
double val = 0;
tree->Branch("index", &index);
tree->Branch("val", &val);

while (index++ < 11) {
  val = 0.5 * index;
  tree->Fill();
}

double lateVal = 0;
tree->Branch("lateVal", &lateVal);
lateVal = 2.5;
tree->Fill();

while (index++ < 20) {
  val = 0.75 * index;
  tree->Fill();
}

results in a TTree with 20 entries, where lateVal is available in all 20 entries with the value of 2.5 (even in those that have been previously filled).

I am not yet sure how or if this can be easily solved.

Another thing that came to my mind after the discussion on Friday: While I think that podio should in principle be able to support both use cases, it might be beneficial to not have to support both simultaneously, i.e. the user has to decide at initialization (or maybe even compile time), which way the data should be handled. Or is there any "realistic" use case where a mixed style would be necessary?

[tmadlener]

This is a general ROOT "issue". It is documented for the FrameWriter:

podio/include/podio/ROOTFrameWriter.h

Lines 32 to 38 in 705721d

	/** Store the given frame with the given category. Store all available
	* collections from the Frame.
	*
	* NOTE: The contents of the first Frame that is written in this way
	* determines the contents that will be written for all subsequent Frames.
	*/
	void writeFrame(const podio::Frame& frame, const std::string& category);