Preservation of the Direct Photon and Neutral Meson Analysis in the PHENIX Experiment at RHIC

PHENIX [8] relies on its host (Brookhaven National Laboratory) for archiving of its raw data and a subset of the derived data products [9]. A special area within the disk mass storage has been allocated for keeping intermediate data crucial for Analysis Preservation, including the analysis of direct photons in d+Au collisions. Conditions and calibrations data are stored in a database, and the issue of preserving these data will be addressed at a later time. The publication-level data such as data points used in tables in plots is reliably preserved by PHENIX Collaboration, by leveraging the HEPData Portal at CERN. A vast majority of all published PHENIX results are now preserved in this manner.

The PHENIX software stack, the result of over 25 years of continuous development, is inherently complex and includes a number of legacy components. Preserving this software presents significant challenges, particularly in maintaining compatibility amid ongoing component upgrades and evolving operating systems. To address these challenges, the PHENIX Collaboration has adopted containerization technology, specifically Docker, which provides an efficient solution for preserving the software environment. Docker enables the encapsulation of the entire software stack, ensuring both its continued operation and portability across different systems.

In addition to containerization, we have prioritized reproducibility in our computational workflows for physics analysis. To achieve this, we integrated the REANA platform [10], which facilitates the orchestration of complex workflows within a containerized environment. REANA not only ensures that our analyses are reproducible but also that they can be reliably executed in the future, irrespective of changes in the underlying infrastructure.

This combined approach of Docker-based containerization and the REANA platform has proven effective in overcoming the challenges of software preservation, ensuring that the PHENIX software stack remains functional and accessible for future analyses.

Even a perfectly preserved software can’t be useful without accompanying “know-how” for specific applications and related configuration details. This is being addressed via the PHENIX Knowledge Management effort. PHENIX has successfully migrated a large number – more than 620 – of its research documents (including PhD theses, technical write-ups, conference presentations etc) to a modern digital repository — the Zenodo platform at CERN [11]. The previously fragmented Web materials have been curated and migrated to a new website, created with the goals of simplicity, security, performance and ease of long-term management [8]. All these measures combined allowed us to create a functioning foundation for Analysis Preservation.

The complete description of the context, significance, issues and physics consequences drawn from the analysis of $d$+Au data are beyond the scope of this paper and are described in [6], as well as in a dedicated write-up for this analysis [12]. Some of the key results in heavy-ion experiments such as PHENIX are the nuclear modification factors (essentially the qualitative difference between particle production in heavy ion vs. proton-proton collisions) which are studied as a function of centrality (the amount of overlap between the colliding ions). Conclusions are based in a large part on the measured invariant yields (spectra) of $\pi^{0}$ and direct $\gamma$ as a function of centrality.

The steps of the analysis leading to those invariant $\pi^{0}$ and direct $\gamma$ yields are shown in Fig. 2 and preserved in our REANA implementation. This workflow is designed such that can be run on a batch system and has been demonstrated to reproduce the fundamental quantities being studied, such the spectra in [6], thus satisfying the first and foremost goal of DAP: reproducibility of a published result. This is illustrated in Fig. 3. While the analysis as a whole is preserved, some of its parameters, cuts and conditions in the individual macros can be changed, making somewhat different analyses of the same data possible, so it can be viewed as a meaningful step toward the more ambitious, extended DAP goal of new analyses of preserved old data. The entire workflow and all relevant code, along with the necessary configuration files is preserved is a version control system accessible to users with BNL SDCC credentials here [13], with the necessary input files stored in [14]. The foundational software stack used in PHENIX has been preserved in a Docker image, ensuring portability.

The REANA implementation of our analysis workflow, as illustrated in Fig. 2, is designed to encapsulate and automate the complex sequence of steps required for data analysis in PHENIX.

In this workflow, blue boxes represent operations performed on a condensed output derived from a comprehensive scan of all available data, stored as a series of multidimensional histograms. These condensed outputs, essential for various stages of the analysis, are stored within the containerized environment. Pink boxes indicate operations involving simulated particles embedded in real events, generating what are known as ”response matrices,” which are 2D histograms that capture the detector’s response to different physical scenarios. These matrices are also preserved within the container, ensuring that all necessary inputs for analysis are readily accessible. Green boxes correspond to the actual analysis steps, which involve executing the preserved ROOT macros. Analysts can edit these macros to modify parameters, apply different cuts, or adjust conditions according to their specific analysis objectives. Each step in the workflow is accompanied by a verbal description to guide users through the process.

To validate the robustness of our preservation approach, the containerized analysis was tested by a computer-literate individual who is not a PHENIX collaborator. This individual successfully executed the entire analysis chain, with the results shown in Fig. 3. The left panel displays the $\pi^{0}$ and direct $\gamma$ spectra (and their ratio) as originally published, while the right panel shows the same spectra generated from the preserved analysis chain. This successful replication by an external user underscores the effectiveness of the REANA implementation in preserving not just the data, but also the full analytical capability for future research.

The PHENIX Collaboration has been largely successful in its Data and Analysis Preservation program (DAP), however this required a substantial amount of effort, against the background of limited personnel available for work in this area. Adding to the challenge, the PHENIX hardware and software design started in early 1990s, and for obvious reasons the software environment underwent major changes since then. Over the years, the software stack became very large, with complex code dependencies and in some cases redundancies. In view of this, we conclude that the single most important lesson from our efforts is that a well-designed, long-term DAP component of the experiment needs to be created by professionals from the very beginning of the experiment and the required standards meticulously followed during its entire lifetime, even if it appears costly or counterproductive at any particular moment. We recognize that a robust analysis preservation effort may be considered a hurdle on the path to quickly obtaining new and exciting publishable results. For that reason, another important lesson is that clear, stable guidance is needed from the very beginning, on how to balance there contradictory requirements. One other dimension of the work we are doing is balancing the legitimate requirements of openness (since the data, the code and the analysis techniques need to be eventually made public due to the policies of the funding agencies), against the also legitimate requirements of data protection and cybersecurity. Reconciling all of this adds to the already significant challenge, and once again, clear guidance and sufficient funding are instrumental in achieving the worthy goals of the Data and Analysis Preservation.