Skip to content
Snippets Groups Projects
Commit 11858421 authored by Alessandro Costantini's avatar Alessandro Costantini
Browse files

Merge branch 'infn_cc'

parents f30cfede 39a04452
No related branches found
No related tags found
1 merge request!1DS contributions
Expand all Hide whitespace changes
Inline Side-by-side
Showing
with 1358 additions and 264 deletions
build.sh
+ 3
− 2
View file @ 11858421
......@@ -131,8 +131,9 @@ build_from_source virgo AdV_computing_CNAF.tex
#build_from_source st StatMet.tex sm2017.bib
build_from_source ds_eoscpilot ds_eoscpilot.tex
build_from_source ds_eoschub ds_eoschub.tex
build_from_source ds_cloud_c ds_cloud_c.tex
build_from_source ds_infn_cc ds_infn_cc.tex
#build_from_source ds_cloud_c ds_cloud_c.tex
#build_from_source ds_cloud_c ds_cloud_c.tex
build_from_source ds_infn_cc ds_infn_cc.tex *.png
build_from_source ds_devops_pe ds_devops_pe.tex
#build_from_source cloud_b cloud_b.tex *.png *.jpg
#build_from_source cloud_c cloud_c.tex *.png *.pdf
......
cnaf-annual-report-2018.tex
+ 2
− 2
View file @ 11858421
......@@ -213,8 +213,8 @@ Introducing the sixth annual report of CNAF...
\ia{Common software lifecycle management in external projects: Placeholder}{ds_devops_pe}
\ia{EOSC-hub: Placeholder}{ds_eoschub}
\ia{EOSCpilot - Interoperability aspects and results}{ds_eoscpilot}
\ia{Cloud@CNAF Management and Evolution: Placeholder}{ds_cloud_c}
\ia{INFN CorporateCloud: Placeholder}{ds_infn_cc}
%\ia{Cloud@CNAF Management and Evolution}{ds_cloud_c}
\ia{INFN CorporateCloud: Management and evolution}{ds_infn_cc}
\ia{eXtreme DataCloud project: Advanced data management services for distributed e-infrastructures}{sdds-xdc}
\ia{DEEP-HybridDataCloud project: Hybrid services for distributed e-infrastructures}{sdds-deep}
......
contributions/ds_cloud_c/ds_cloud_c.tex deleted 100644 → 0
+ 0
− 155
View file @ f30cfede
\documentclass[a4paper]{jpconf}
\usepackage{graphicx}
\begin{document}
\title{Cloud@CNAF Management and Evolution: placeholder}
\author{C. Duma$^1$, A. Costantini$^1$, D. Michelotto$^1$ and D. Salomoni$^1$}
\address{$^1$INFN Division CNAF, Bologna, Italy}
%\address{$^2$IFCA, Consejo Superior de Investigaciones Cientificas-CSIC, Santander, Spain}
\ead{ds@cnaf.infn.it}
\begin{abstract}
This paper describes the achievements of the H2020 project INDIGO-DataCloud in the field of software lifecycle management, the Continuous Integration and Delivery systems setup to manage the new releases, as a first step towards the implementation of a DevOps approach.
\end{abstract}
\section{Introduction}
Text
\section{First section}
\label{sec:release}
TOCHANGE
The software development lifecycle (SDL) process (Figure~\ref{fig:1}) in INDIGO has been supported by a continuous
software improvement process that regarded the software quality assurance, software maintenance,
including release management, support services, and the management of pilot infrastructures
needed for software integration and acceptance testing.
%\begin{figure}
% \centering
% \includegraphics[width=\textwidth]{Figure5.pdf}
% \caption{Software development lifecycle implementation}
% \label{fig:1}
%\end{figure}
Preview releases are made available for evaluation by user communities and
resource providers through the pilot infrastructures. Release
candidates are subjected to integration testing, which may include the
%\subsection{Software development lifecycle management}
Software lifecycle management is performed mostly via automated actions orchestrated.
In Figure we depict the project's software lifecycle management services and
activities and their interdependencies:
\begin{itemize}
\item Version Control System (VCS) - Source Code is made available through public VCS
repositories, hosted externally in GitHub repositories, guaranteeing in this
way the software openness and visibility, simplifying the exploitation beyond the
project lifetime. The INDIGO-DataCloud software is released under the Apache 2.0
software license and can be deployed on both public and private Cloud infrastructures.
\item Software quality assurence criteria and control activities and services to enable them:
\begin{itemize}
\item Continuous Integration service using {\bf Jenkins}: Service to automate the building,
packaging (where applicable) and execution of unit and functional tests of software components.
\item Code review service using GitHub: Code review of software source code is one integral part of the SQA\@. This service facilitates the code review proces. It records the
comments and allows the reviewer to verify the software modification.
\item Code metrics services using {\bf Grimoire}: To collect and visualize several metrics about the software components.
\end{itemize}
\item Software release and maintenance activities, services and supporting infrastructures
\begin{itemize}
\item A project management service using {\bf openproject.org} is made available by the
project: It provides tools such as an issue tracker, wiki, a placeholder for documents and a project management timeline.
\item Artifacts repositories for RPM and Debian packages, and Docker Hub for containers:
In INDIGO-DataCloud there are two types of artifacts, packaged software and virtual images.
The software can be downloaded from our public repository\footnote{http://repo.indigo-datacloud.eu}.
\item Release notes, installation and configuration guides, user and development manuals are made
available on {\bf GitBook}\footnote{https://indigo-dc.gitbooks.io/indigo-datacloud-releases}.
\item Bug trackers using GitHub issues tracker: Services to track issues and bugs of INDIGO-DataCloud software components.
\item Integration infrastructure: this infrastructure is composed of computing resources to support directly
the Continuous Integration service. It's the place where building and packaging of software
occurs as well as the execution of unit and functional tests. These resources are provided by INDIGO partners.
\item Testing infrastructure: this infrastructure aims to provide several types of environment. A stable environment
for users where they can preview the software and services developed by INDIGO-DataCloud, prior to its public release.
\item Preview infrastructure: where the released artifacts are deployed and made available for testing and validation by the use-cases.
\end{itemize}
\end{itemize}
The first INDIGO-DataCloud major release (codename {\tt MidnightBlue}) was released 1st of August 2016 (see table~\ref{tab:1} for the fact sheet). The
second INDIGO-DataCloud major release (codename {\tt ElectricIndigo}) was made publicly available on April 14th 2017 (see table~\ref{tab:2} for the fact sheet).
\section{DevOps approach in INDIGO}
Progressive levels of automation were adopted throughout the different phases of
the INDIGO-DataCloud project software development and delivery processes.
\subsection{Services for continuous integration and SQA}
The INDIGO-DataCloud CI process is schematically shown
in Figure~\ref{fig:3}. The process, in its different steps, reflects some of
the main and important achievements of the software integration team, such as:
\begin{itemize}
\item New features are developed independently from the
production version in \textit{feature branches}. The creation of
a pull request for a specific feature branch marks the start of
the automated validation process through the execution of the
SQA jobs.
\item The SQA jobs perform the code style verification and calculate unit
and functional test coverage.
\begin{itemize}
\item The tools necessary for tackling these tests are packaged in
Docker images, available in DockerHub.
\item Each test then initiates a new container that provides a
clean environment for its execution.
\item This is an innovative approach that provides the flexibility
needed to cope with the INDIGO-DataCloud software diversity.
\end{itemize}
\item The results of the several SQA jobs are made available in the Jenkins
service which notifies back to GitHub their exit status.
\begin{itemize}
\item Only if the tests have succeeded, the source code is
validated and is ready to be merged into the production branch.
\end{itemize}
\item The last step in the workflow is the code review, where a human
review of the change is performed. After code review the source code
can be merged and becomes ready for integration and later release.
\end{itemize}
As a general rule, the described CI process must be followed by all the PTs
contributing code to INDIGO-DataCloud. However there are exceptions to this rule that fall into two main categories:
\subsection{Continuous delivery}
Continuous delivery adds, on top of the software development chain, a seamless
manufacturing of software packages ready to be deployed into production
services. Therefore, fast, frequent and small releases can be taken over thus
promoting the reliability of the software.
\subsection{DevOps adoption from user communities}
The experience gathered throughout the project with regards to the adoption of different DevOps
practices is not only useful and suitable for the software related to the core services in the
INDIGO-DataCloud solution, but also applicable to the development and distribution of the applications coming from the user communities.
\section{Conclusions}
Thanks to the new common solutions developed by the INDIGO project, teams of first-line
researchers in Europe are using public and private Cloud resources to get new results in Physics, Biology, Astronomy, Medicine, Humanities and other disciplines.
%\section*{Acknowledgments}
%eXtreme-DataCloud has been funded by the European Commision H2020 research and innovation program under grant agreement RIA XXXXXXX.
\end{document}
contributions/ds_infn_cc/artifact/ds_infn_cc.pdf 0 → 100644
+ 0
− 0
View file @ 11858421
File added
contributions/ds_infn_cc/artifact/iopams.sty 0 → 100644
+ 87
− 0
View file @ 11858421
%%
%% This is file `iopams.sty'
%% File to include AMS fonts and extra definitions for bold greek
%% characters for use with iopart.cls
%%
\NeedsTeXFormat{LaTeX2e}
\ProvidesPackage{iopams}[1997/02/13 v1.0]
\RequirePackage{amsgen}[1995/01/01]
\RequirePackage{amsfonts}[1995/01/01]
\RequirePackage{amssymb}[1995/01/01]
\RequirePackage{amsbsy}[1995/01/01]
%
\iopamstrue % \newif\ifiopams in iopart.cls & iopbk2e.cls
% % allows optional text to be in author guidelines
%
% Bold lower case Greek letters
%
\newcommand{\balpha}{\boldsymbol{\alpha}}
\newcommand{\bbeta}{\boldsymbol{\beta}}
\newcommand{\bgamma}{\boldsymbol{\gamma}}
\newcommand{\bdelta}{\boldsymbol{\delta}}
\newcommand{\bepsilon}{\boldsymbol{\epsilon}}
\newcommand{\bzeta}{\boldsymbol{\zeta}}
\newcommand{\bfeta}{\boldsymbol{\eta}}
\newcommand{\btheta}{\boldsymbol{\theta}}
\newcommand{\biota}{\boldsymbol{\iota}}
\newcommand{\bkappa}{\boldsymbol{\kappa}}
\newcommand{\blambda}{\boldsymbol{\lambda}}
\newcommand{\bmu}{\boldsymbol{\mu}}
\newcommand{\bnu}{\boldsymbol{\nu}}
\newcommand{\bxi}{\boldsymbol{\xi}}
\newcommand{\bpi}{\boldsymbol{\pi}}
\newcommand{\brho}{\boldsymbol{\rho}}
\newcommand{\bsigma}{\boldsymbol{\sigma}}
\newcommand{\btau}{\boldsymbol{\tau}}
\newcommand{\bupsilon}{\boldsymbol{\upsilon}}
\newcommand{\bphi}{\boldsymbol{\phi}}
\newcommand{\bchi}{\boldsymbol{\chi}}
\newcommand{\bpsi}{\boldsymbol{\psi}}
\newcommand{\bomega}{\boldsymbol{\omega}}
\newcommand{\bvarepsilon}{\boldsymbol{\varepsilon}}
\newcommand{\bvartheta}{\boldsymbol{\vartheta}}
\newcommand{\bvaromega}{\boldsymbol{\varomega}}
\newcommand{\bvarrho}{\boldsymbol{\varrho}}
\newcommand{\bvarzeta}{\boldsymbol{\varsigma}} %NB really sigma
\newcommand{\bvarsigma}{\boldsymbol{\varsigma}}
\newcommand{\bvarphi}{\boldsymbol{\varphi}}
%
% Bold upright capital Greek letters
%
\newcommand{\bGamma}{\boldsymbol{\Gamma}}
\newcommand{\bDelta}{\boldsymbol{\Delta}}
\newcommand{\bTheta}{\boldsymbol{\Theta}}
\newcommand{\bLambda}{\boldsymbol{\Lambda}}
\newcommand{\bXi}{\boldsymbol{\Xi}}
\newcommand{\bPi}{\boldsymbol{\Pi}}
\newcommand{\bSigma}{\boldsymbol{\Sigma}}
\newcommand{\bUpsilon}{\boldsymbol{\Upsilon}}
\newcommand{\bPhi}{\boldsymbol{\Phi}}
\newcommand{\bPsi}{\boldsymbol{\Psi}}
\newcommand{\bOmega}{\boldsymbol{\Omega}}
%
% Bold versions of miscellaneous symbols
%
\newcommand{\bpartial}{\boldsymbol{\partial}}
\newcommand{\bell}{\boldsymbol{\ell}}
\newcommand{\bimath}{\boldsymbol{\imath}}
\newcommand{\bjmath}{\boldsymbol{\jmath}}
\newcommand{\binfty}{\boldsymbol{\infty}}
\newcommand{\bnabla}{\boldsymbol{\nabla}}
\newcommand{\bdot}{\boldsymbol{\cdot}}
%
% Symbols for caption
%
\renewcommand{\opensquare}{\mbox{$\square$}}
\renewcommand{\opentriangle}{\mbox{$\vartriangle$}}
\renewcommand{\opentriangledown}{\mbox{$\triangledown$}}
\renewcommand{\opendiamond}{\mbox{$\lozenge$}}
\renewcommand{\fullsquare}{\mbox{$\blacksquare$}}
\newcommand{\fulldiamond}{\mbox{$\blacklozenge$}}
\newcommand{\fullstar}{\mbox{$\bigstar$}}
\newcommand{\fulltriangle}{\mbox{$\blacktriangle$}}
\newcommand{\fulltriangledown}{\mbox{$\blacktriangledown$}}
\endinput
%%
%% End of file `iopams.sty'.
contributions/ds_infn_cc/artifact/jpconf.cls 0 → 100644
+ 957
− 0
View file @ 11858421
This diff is collapsed.
contributions/ds_infn_cc/artifact/jpconf11.clo 0 → 100644
+ 141
− 0
View file @ 11858421
%%
%% This is file `jpconf11.clo'
%%
%% This file is distributed in the hope that it will be useful,
%% but WITHOUT ANY WARRANTY; without even the implied warranty of
%% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
%%
%% \CharacterTable
%% {Upper-case \A\B\C\D\E\F\G\H\I\J\K\L\M\N\O\P\Q\R\S\T\U\V\W\X\Y\Z
%% Lower-case \a\b\c\d\e\f\g\h\i\j\k\l\m\n\o\p\q\r\s\t\u\v\w\x\y\z
%% Digits \0\1\2\3\4\5\6\7\8\9
%% Exclamation \! Double quote \" Hash (number) \#
%% Dollar \$ Percent \% Ampersand \&
%% Acute accent \' Left paren \( Right paren \)
%% Asterisk \* Plus \+ Comma \,
%% Minus \- Point \. Solidus \/
%% Colon \: Semicolon \; Less than \<
%% Equals \= Greater than \> Question mark \?
%% Commercial at \@ Left bracket \[ Backslash \\
%% Right bracket \] Circumflex \^ Underscore \_
%% Grave accent \` Left brace \{ Vertical bar \|
%% Right brace \} Tilde \~}
\ProvidesFile{jpconf11.clo}[2005/05/04 v1.0 LaTeX2e file (size option)]
\renewcommand\normalsize{%
\@setfontsize\normalsize\@xipt{13}%
\abovedisplayskip 12\p@ \@plus3\p@ \@minus7\p@
\abovedisplayshortskip \z@ \@plus3\p@
\belowdisplayshortskip 6.5\p@ \@plus3.5\p@ \@minus3\p@
\belowdisplayskip \abovedisplayskip
\let\@listi\@listI}
\normalsize
\newcommand\small{%
\@setfontsize\small\@xpt{12}%
\abovedisplayskip 11\p@ \@plus3\p@ \@minus6\p@
\abovedisplayshortskip \z@ \@plus3\p@
\belowdisplayshortskip 6.5\p@ \@plus3.5\p@ \@minus3\p@
\def\@listi{\leftmargin\leftmargini
\topsep 9\p@ \@plus3\p@ \@minus5\p@
\parsep 4.5\p@ \@plus2\p@ \@minus\p@
\itemsep \parsep}%
\belowdisplayskip \abovedisplayskip}
\newcommand\footnotesize{%
% \@setfontsize\footnotesize\@xpt\@xiipt
\@setfontsize\footnotesize\@ixpt{11}%
\abovedisplayskip 10\p@ \@plus2\p@ \@minus5\p@
\abovedisplayshortskip \z@ \@plus3\p@
\belowdisplayshortskip 6\p@ \@plus3\p@ \@minus3\p@
\def\@listi{\leftmargin\leftmargini
\topsep 6\p@ \@plus2\p@ \@minus2\p@
\parsep 3\p@ \@plus2\p@ \@minus\p@
\itemsep \parsep}%
\belowdisplayskip \abovedisplayskip
}
\newcommand\scriptsize{\@setfontsize\scriptsize\@viiipt{9.5}}
\newcommand\tiny{\@setfontsize\tiny\@vipt\@viipt}
\newcommand\large{\@setfontsize\large\@xivpt{18}}
\newcommand\Large{\@setfontsize\Large\@xviipt{22}}
\newcommand\LARGE{\@setfontsize\LARGE\@xxpt{25}}
\newcommand\huge{\@setfontsize\huge\@xxvpt{30}}
\let\Huge=\huge
\if@twocolumn
\setlength\parindent{14\p@}
\else
\setlength\parindent{18\p@}
\fi
\if@letterpaper%
%\input{letmarg.tex}%
\setlength{\hoffset}{0mm}
\setlength{\marginparsep}{0mm}
\setlength{\marginparwidth}{0mm}
\setlength{\textwidth}{160mm}
\setlength{\oddsidemargin}{-0.4mm}
\setlength{\evensidemargin}{-0.4mm}
\setlength{\voffset}{0mm}
\setlength{\headheight}{8mm}
\setlength{\headsep}{5mm}
\setlength{\footskip}{0mm}
\setlength{\textheight}{230mm}
\setlength{\topmargin}{1.6mm}
\else
%\input{a4marg.tex}%
\setlength{\hoffset}{0mm}
\setlength{\marginparsep}{0mm}
\setlength{\marginparwidth}{0mm}
\setlength{\textwidth}{160mm}
\setlength{\oddsidemargin}{-0.4mm}
\setlength{\evensidemargin}{-0.4mm}
\setlength{\voffset}{0mm}
\setlength{\headheight}{8mm}
\setlength{\headsep}{5mm}
\setlength{\footskip}{0mm}
\setlength{\textheight}{230mm}
\setlength{\topmargin}{1.6mm}
\fi
\setlength\maxdepth{.5\topskip}
\setlength\@maxdepth\maxdepth
\setlength\footnotesep{8.4\p@}
\setlength{\skip\footins} {10.8\p@ \@plus 4\p@ \@minus 2\p@}
\setlength\floatsep {14\p@ \@plus 2\p@ \@minus 4\p@}
\setlength\textfloatsep {24\p@ \@plus 2\p@ \@minus 4\p@}
\setlength\intextsep {16\p@ \@plus 4\p@ \@minus 4\p@}
\setlength\dblfloatsep {16\p@ \@plus 2\p@ \@minus 4\p@}
\setlength\dbltextfloatsep{24\p@ \@plus 2\p@ \@minus 4\p@}
\setlength\@fptop{0\p@}
\setlength\@fpsep{10\p@ \@plus 1fil}
\setlength\@fpbot{0\p@}
\setlength\@dblfptop{0\p@}
\setlength\@dblfpsep{10\p@ \@plus 1fil}
\setlength\@dblfpbot{0\p@}
\setlength\partopsep{3\p@ \@plus 2\p@ \@minus 2\p@}
\def\@listI{\leftmargin\leftmargini
\parsep=\z@
\topsep=6\p@ \@plus3\p@ \@minus3\p@
\itemsep=3\p@ \@plus2\p@ \@minus1\p@}
\let\@listi\@listI
\@listi
\def\@listii {\leftmargin\leftmarginii
\labelwidth\leftmarginii
\advance\labelwidth-\labelsep
\topsep=3\p@ \@plus2\p@ \@minus\p@
\parsep=\z@
\itemsep=\parsep}
\def\@listiii{\leftmargin\leftmarginiii
\labelwidth\leftmarginiii
\advance\labelwidth-\labelsep
\topsep=\z@
\parsep=\z@
\partopsep=\z@
\itemsep=\z@}
\def\@listiv {\leftmargin\leftmarginiv
\labelwidth\leftmarginiv
\advance\labelwidth-\labelsep}
\def\@listv{\leftmargin\leftmarginv
\labelwidth\leftmarginv
\advance\labelwidth-\labelsep}
\def\@listvi {\leftmargin\leftmarginvi
\labelwidth\leftmarginvi
\advance\labelwidth-\labelsep}
\endinput
%%
%% End of file `iopart12.clo'.
contributions/ds_infn_cc/ds_infn_cc.tex
+ 168
− 105
View file @ 11858421
This diff is collapsed.
contributions/ds_infn_cc/infncc-net.png 0 → 100644
+ 0
− 0
View file @ 11858421
contributions/ds_infn_cc/infncc-net.png

198 KiB

contributions/ds_infn_cc/infncc-services.png 0 → 100644
+ 0
− 0
View file @ 11858421
contributions/ds_infn_cc/infncc-services.png

128 KiB

0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment