Automation Framework Hackathon 2.0

This is the second Automation Framework Hackathon [1] dedicated to review and push forward the ongoing developments towards automating the calibration workflows of subsystems using this framework by working together collectively. The hackathon would provide the opportunity for brainstorming discussions, debugging and solutions to common challenges, and improvement in the workflows by exchange of ideas and knowledge with a collaborative effort - in a focused and uninterrupted environment, free from distractions.

A brief overview of the framework and it's potential advantages:

The framework was initially developed by the ECAL DPG during the LHC LS2 to automatize the data processing for deriving calibrations and it's monitoring. The main goal was to improve the quality of prompt reconstructed data and reduce the time needed to derive conditions. The system has been designed following these principles:

Allows greater flexibility with the use of the market software
Allows integration the existing calibration workflows limiting the amount of code re-writing
Provide simple monitoring tools to oversee the automation

Owing to the successful implementation and smooth operations of the ECAL calibration workflows in 2022, the framework was also adopted by other DPGs to streamline their calibration workflows. In general, the framework can be used for a wide range of data-processing tasks including:

Producing and analyzing ntuples
Making DQM ntuples and plots
Running prompt offline analysis workflows
Creating, validating, and uploading conditions to database

Further details can also be found in the past presentations [2] [3] and dedicated documentation in [4] . In the long-term goal, a common calibration workflow infrastructure for the subsystems like this would allow to minimize the repetitive data processing steps that are common to some POGs and also to eradicate the inter-dependency between different POGs and DPGs for transfer of conditions.

Hackathon Details

During the hackathon, participants will work on the proposed target project with a daily summary of the progress. The hackathon would mainly consist of two parts:

Introductory talks and dedicated presentations from the developers and experts who are already using the framework
Hands-on hackathon sessions to incorporate and develop new workflows for subsystems

The hackathon will take place in hybrid mode - both in-person and remotely via zoom, but physical presence would be preferred to profit from active discussions. To help us keep track of the number of participants and physical attendees, please register yourself with the link provided below in the agenda. The deadline for registration is 25th November, 2023.

Prior to coming to the hackathon, please fill out the checklist for your subsystem according to the target goal to be achieved using the framework in this gDoc https://docs.google.com/document/d/1n5SE05-DnhV6MepQtf91jtQxM58wrqwLzI445kCjAuM/edit

If case you have further questions, please get in touch with cms-PPD-conveners-AlCaDB@cern.ch