Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document I/O interfaces #648

Merged
merged 4 commits into from
Jul 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 105 additions & 4 deletions include/podio/Reader.h
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,17 @@

namespace podio {

/// Generic (type erased) reader class that can handle different I/O backends
/// transparently
///
/// Offers some more high level functionality compared to the lower level
/// backend specific readers that this class wraps. In contrast to the lower
/// level readers that usually return arbitrary FrameData, this interface class
/// will return fully constructed Frames. In addition, it provides convenience
/// methods to deal specifically with the "events" frame category.
///
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition it provides convenience methods to deal specifically with the event frame category...

/// @note The recommended way to construct is to use the makeReader() functions
/// since they handle the instantiation of the correct low level readers
class Reader {
private:
struct ReaderConcept {
Expand Down Expand Up @@ -73,51 +84,141 @@ class Reader {
std::unique_ptr<ReaderConcept> m_self{nullptr};

public:
/// Create a reader from a low level reader
///
/// @tparam T The type of the low level reader (will bededuced)
/// @param actualReader a low level reader that provides access to FrameDataT
template <typename T>
Reader(std::unique_ptr<T>);
Reader(std::unique_ptr<T> actualReader);

Reader(const Reader&) = delete;
Reader& operator=(const Reader&) = delete;

Reader(Reader&&) = default;
Reader& operator=(Reader&&) = default;

~Reader() = default;

/// Read the next frame of a given category
///
/// @param name The category name for which to read the next frame
///
/// @returns A fully constructed Frame with the contents read from file
///
/// @throws std::invalid_argument in case the category is not available or in
/// case no more entries are available
podio::Frame readNextFrame(const std::string& name) {
return m_self->readNextFrame(name);
}

/// Read the next frame of the "events" category
///
/// @returns A fully constructed Frame with the contents read from file
///
/// @throws std::invalid_argument in case no (more) events are available
podio::Frame readNextEvent() {
return readNextFrame(podio::Category::Event);
}

/// Read a specific frame for a given category
///
/// @param name The category name for which to read the next entry
/// @param index The entry number to read
///
/// @returns A fully constructed Frame with the contents read from file
///
/// @throws std::invalid_argument in case the category is not available or in
/// case the specified entry is not available
podio::Frame readFrame(const std::string& name, size_t index) {
return m_self->readFrame(name, index);
}

/// Read a specific frame of the "events" category
///
/// @param index The event number to read
///
/// @returns A fully constructed Frame with the contents read from file
///
/// @throws std::invalid_argument in case the desired event is not available
podio::Frame readEvent(size_t index) {
return readFrame(podio::Category::Event, index);
}

/// Get the number of entries for the given name
///
/// @param name The name of the category
///
/// @returns The number of entries that are available for the category
size_t getEntries(const std::string& name) const {
return m_self->getEntries(name);
}

/// Get the number of events
///
/// @returns The number of entries that are available for the category
size_t getEvents() const {
return getEntries(podio::Category::Event);
}

/// Get the build version of podio that has been used to write the current
/// file
///
/// @returns The podio build version
podio::version::Version currentFileVersion() const {
return m_self->currentFileVersion();
}

/// Get the names of all the available Frame categories in the current file(s).
///
/// @returns The names of the available categories from the file
std::vector<std::string_view> getAvailableCategories() const {
return m_self->getAvailableCategories();
}

/// Get the datamodel definition for the given name
///
/// @param name The name of the datamodel
///
/// @returns The high level definition of the datamodel in JSON format
const std::string_view getDatamodelDefinition(const std::string& name) const {
return m_self->getDatamodelDefinition(name);
}

/// Get all names of the datamodels that are available from this reader
///
/// @returns The names of the datamodels
std::vector<std::string> getAvailableDatamodels() const {
return m_self->getAvailableDatamodels();
}
};

/// Create a Reader is able to read the file
///
/// This will inspect the filename as well as peek at the file contents to
/// instantiate the correct low level reader to open and read the file
///
/// @param filename The (path to the) file to read from
///
/// @returns A Reader that has been initialized and that can be used for reading
/// data from the passed file
Reader makeReader(const std::string& filename);
Reader makeReader(const std::vector<std::string>& filename);

/// Create a Reader that is able to read the files
///
/// This will inspect the filenames as well as peek into the **first file only**
/// to decide based on the contents which low level reader to instantiate for
/// reading. All files are assumed to be of the same I/O format, no switching
/// between formats is possible.
///
/// @note For SIO files this will only work with exactly one file!
///
/// @param filenames The (paths to the) files to read from
///
/// @returns A Reader that has been initialized and that can be used for reading
/// data from the passed files
///
/// @throws std::runtime_error in case the file extensions differ or in case
/// support for the necessary I/O backend has not been built or in case
/// multiple files for the SIO backend are passed
Reader makeReader(const std::vector<std::string>& filenames);

} // namespace podio

Expand Down
72 changes: 71 additions & 1 deletion include/podio/Writer.h
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,19 @@

namespace podio {

/// Generic (type erased) writer class that can handle different I/O backends
/// (almost) transparently
///
/// Offers some more high level functionality compared to the lower level
/// backend specific writers that this class wraps. In addition, it provides
/// convenience methods to deal specifically with the "events" frame category.
///
/// @note Since this simply wraps lower level writers, some of the limitations
/// of the wrapped writers will still apply, e.g. if used for writing ROOT files
/// frames of a given category will have to have the same contents.
///
/// @note The recommended way to construct is to use the makeWriter() function
/// since that handles the instantiation of the correct low level writers
class Writer {
private:
struct WriterConcept {
Expand Down Expand Up @@ -40,34 +53,91 @@ class Writer {
std::unique_ptr<WriterConcept> m_self{nullptr};

public:
/// Create a Writer from a lower level writer
///
/// @tparam T the type of the low level writer (will be deduced)
/// @param writer A low level writer that does the actual work
template <typename T>
Writer(std::unique_ptr<T> reader) : m_self(std::make_unique<WriterModel<T>>(std::move(reader))) {
Writer(std::unique_ptr<T> writer) : m_self(std::make_unique<WriterModel<T>>(std::move(writer))) {
}

Writer(const Writer&) = delete;
Writer& operator=(const Writer&) = delete;
Writer(Writer&&) = default;
Writer& operator=(Writer&&) = default;

/// Destructor
///
/// This also takes care of writing all the necessary metadata to read files
/// back again.
~Writer() = default;

/// Store the given frame with the given category
///
/// This stores all avaialble categories from the passed frame
///
/// @param frame The frame to write
/// @param category The category name under which this frame should be stored
void writeFrame(const podio::Frame& frame, const std::string& category) {
return m_self->writeFrame(frame, category, frame.getAvailableCollections());
}

/// Store the given Frame with the given category.
///
/// This stores only the desired collections and not the complete frame.
///
/// @param frame The Frame to store
/// @param category The category name under which this Frame should be
/// stored
/// @param collsToWrite The collection names that should be written
void writeFrame(const podio::Frame& frame, const std::string& category, const std::vector<std::string>& collections) {
return m_self->writeFrame(frame, category, collections);
}

/// Store the given frame under the "events" category
///
/// This stores all avaialble categories from the passed frame
///
/// @param frame The frame to write
void writeEvent(const podio::Frame& frame) {
writeFrame(frame, podio::Category::Event, frame.getAvailableCollections());
}

/// Store the given Frame under the "events" category
///
/// This stores only the desired collections and not the complete frame.
///
/// @param frame The Frame to store
/// @param collsToWrite The collection names that should be written
void writeEvent(const podio::Frame& frame, const std::vector<std::string>& collections) {
writeFrame(frame, podio::Category::Event, collections);
}

/// Write the current file, including all the necessary metadata to read it
/// again.
///
/// @note The destructor will also call this, so letting a Writer go out of
/// scope is also a viable way to write a readable file
void finish() {
return m_self->finish();
}
};

/// Create a Writer that is able to write files for the desired backend
///
/// Will look at the desired filename as well as the type argument to decide on
/// the backend.
///
/// @param filename The filename of the output file that will be created.
/// @param type The (optional) type argument to switch between RNTuple and TTree
/// based backend in case the suffix is ".root". Will be ignored
/// in case the suffix is ".sio"
///
/// @returns A fully initialized Writer for the I/O backend that has been
/// determined
///
/// @throws std::runtime_error In case the suffix can not be associated to an
/// I/O backend or if support for the desired I/O backend has not been built
Writer makeWriter(const std::string& filename, const std::string& type = "default");

} // namespace podio
Expand Down
3 changes: 3 additions & 0 deletions src/Reader.cc
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,9 @@ Reader makeReader(const std::vector<std::string>& filenames) {
}
} else if (suffix == "sio") {
#if PODIO_ENABLE_SIO
if (filenames.size() > 1) {
throw std::runtime_error("The SIO reader does currently not support reading multiple files");
}
auto actualReader = std::make_unique<SIOReader>();
actualReader->openFile(filenames[0]);
Reader reader{std::move(actualReader)};
Expand Down