main.tex

\documentclass[a4paper]{jpconf}

\usepackage{url}
\usepackage{graphicx}
\usepackage{float}

\newcommand{\quotes}[1]{``#1''}

\begin{document}

\title{StoRM-2: initial design and development activities} 

\author{
  A~Ceccanti,
  F~Giacomini,
  E~Vianello and
  E~Ronchieri
}

\address{INFN-CNAF, Bologna, IT}

\ead{
  andrea.ceccanti@cnaf.infn.it
}

\begin{abstract}
  StoRM is the storage element solution that powers the CNAF Tier-1
  data center as well as more than 30 other sites.  Experience in
  developing, maintaining and operating it at scale suggests that a
  significant refactoring of the codebase is necessary to improve
  StoRM maintainability, reliability, scalability and ease of
  operation in order to meet the data management requirements coming
  from HL-LHC and other communities served by the CNAF Tier-1 data
  center.  In this contribution we highlight the initial StoRM-2
  design and development activities.
\end{abstract}

\section{Introduction}
\label{sec:introduction}

Andrea, Giaco, ...:

Main reasons for the refactoring/rewrite:

\begin{itemize}
    \item Reduce maintenance and evolution costs, simplify the codebase and introduce appropriate testing
    \item Provide horizontal scalability for all StoRM services (the backend cannot be currently replicated)
    \item Simplify service operation and deployment (containerization)
\end{itemize}

\section{StoRM 2 high-level architecture}

Andrea:

Describe new StoRM 2 architecture. 
\begin{figure}
    \centering
    \includegraphics[width=.6\textwidth]{high-level-arch.png}
    \caption{\label{fig:oidc-providers}The StoRM 2 high level architecture.}
\end{figure}

Cite the Nginx VOMS module~\cite{ref:nginx-voms}.

Briefly describe the communication between FE and BE, based on
Protobuf and gRPC. Show an example of Request/Response specification.

\section{Principles guiding the development work}

Andrea: 

\begin{itemize}
    \item Test-driven development: $>90\%$ coverage on all code
    \item Single repository for all the components (fe,be, clis, etc.)
    \item Single versions for all the components
    \item Git-flowish branching model
    \item Keep a README.md to describe directory contents
    \item Keep a CHANGELOG to describe code changes
    \item Dockerized development/testing environment
    \item Continuous Integration
    \item Tracing
    \item Uniform logging in all components
    \item Uniform configuration for all components
\end{itemize}

\section{The build environment}

Andrea, Giaco:

Describe here the work done on build images. Challenges/solutions/lessons learned.

\section{The new StoRM frontend component}
Andrea, Enrico:

Describe here the new StoRM frontend component

\section{The StoRM-2 backend component}

The StoRM-2 backend is a gRPC server that provides multiple
services. One service responds to \textit{version} requests. Another
service responds to storage-related requests, which represent the main
scope of StoRM. In general there is no direct, one-to-one mapping
between SRM requests arriving at the frontend and requests addressed
to the backend; rather, these represent building blocks that the
frontend can compose in order to prepare the responses to SRM clients.

Among the storage requests addressed to the backend, at the moment
only a couple are implemented: \textit{ls}, in its multiple variations
(for a file or a directory, recursive, up to a given depth, etc.),
returns information about files and directories; \textit{pin},
\textit{unpin} and \textit{pin status} manage the
\verb|user.storm.pinned| attribute of filesystem entities, which is
essential for the implementation of the more complex
\textit{srmPrepareToGet} SRM request.

All the backend requests are currently blocking: a response is sent
back to the frontend only when the request has been fully processed.

The backend also incorporates sub-components of more general utility
to operate on Filesystem Extended Attributes and POSIX Access Control
Lists~\cite{p1003.1e}, adding a layer of safety and expressivity on
top of the native C APIs. They allow to define attributes and ACLs
respectively and to apply them to or read them from filesystem
entities.

For example the following sets the attribute \verb|user.storm.pinned|
of file \verb|myFile.txt| to the pin duration:

{\small
\begin{verbatim}
set_xattr(
  storage_dir / "myFile.txt",
  StormXAttrName{"pinned"},
  XAttrValue{duration}
);
\end{verbatim}
}

The following instead extends the ACL currently assigned to
\verb|myFile.txt| with some additional entries:

{\small
\begin{verbatim}
add_to_access_acl(
  storage_dir / "myFile.txt",
  {
    {User{"storm"}, Perms::Read | Perms::Write},
    {Group{"storm"}, Perms::Read},
    {other, Perms::None}
  }
);
\end{verbatim}
}

The backend is implemented in C++, in the latest standard version
supported by the toolset installed in the reference platform
(currently C++17). The build system is based on CMake.

The backend relies on some other third-party dependencies, the most
important being for interaction with the filesystem (Boost
Filesystem~\cite{ref:boost.fs}), for logging (Boost
Log~\cite{ref:boost.log}) and for handling configuration
(yaml-cpp~\cite{ref:yaml-cpp}).

\section{Test suites and continuous integration}
Eli:

Describe here work done on test suites.

\section{Conclusions and future work}
All:

Long road ahead, no effort. Will StoRM 2 ever see the light 
with this sort of commitment from INFN?


\section*{References}

\bibliographystyle{iopart-num}
\bibliography{biblio}

\end{document}