@@ -17,14 +17,19 @@ This paper describes the common procedure defined and adopted in the field of So
...
@@ -17,14 +17,19 @@ This paper describes the common procedure defined and adopted in the field of So
the quality of the provided solutions, services and components, while strengthening the collaboration between
the quality of the provided solutions, services and components, while strengthening the collaboration between
developers and operations teams among different external projects.
developers and operations teams among different external projects.
In particular, the paper analyses the common software lifecycle management procedure developed during the
In particular, the paper analyses the common software lifecycle management procedure developed during the
INDIO-DataCloud \cite{indigo}project and recently improved and adopted in two EC funded
INDIO-DataCloud project and recently improved and adopted in two EC funded
projects: eXtreme DataCloud \cite{xdc}and DEEP Hybrid DataCloud\cite{deep}.
projects: eXtreme DataCloud and DEEP Hybrid DataCloud.
\end{abstract}
\end{abstract}
\section{Introduction}
\section{Introduction}
A relevant activity in the software-oriented projects is the definition and implementation of the entire Software
The eXtreme-DataCloud (XDC) \cite{xdc} and DEEP-HybridDataCloud (DEEP-HDC) \cite{deep} projects are aimed at addressing requirements from a wide range of User Communities belonging to several disciplines and test the developed software solutions against the real life use cases.
Lifecycle Management process. As the software components envisaged by the project have a history of development
The software solutions carried out by the both projects are released as Open Source and are based on already existing components (TRL8+) that the projects will enrich with new functionalities and plugins.
in previous successful European projects implementing different types of modern software development techniques,
The use of standards and protocols widely available on the state-of-the-art distributed computing ecosystems may be not enough to guarantee that the released components can be easily plugged into the European e-Infrastructures and in general on cloud based computing environments and the definition and implementation of the entire Software
Lifecycle Management process becames mandatory in such projects.
As the software components envisaged by both projects have a history of development
in previous successful European projects (such as the INDIGO-DataCloud \cite{indigo} project) implementing different types of modern software development techniques,
the natural choice was to complement the previous, individual, Continuous Development and Integration services
the natural choice was to complement the previous, individual, Continuous Development and Integration services
with a Continuous Testing, Deployment and Monitoring as part of a DevOps approach:
with a Continuous Testing, Deployment and Monitoring as part of a DevOps approach:
\begin{itemize}
\begin{itemize}
...
@@ -59,6 +64,7 @@ These DevOps activities are carried out on loop continuously until the desired p
...
@@ -59,6 +64,7 @@ These DevOps activities are carried out on loop continuously until the desired p
Automation will play a central role in all the activities in order to achieve a complete release automation, moving the
Automation will play a central role in all the activities in order to achieve a complete release automation, moving the
software from the developers through build and quality assurance checks, to deployment into integration testbeds
software from the developers through build and quality assurance checks, to deployment into integration testbeds
and finally to production sites part of the Pilot Infrastructures.
and finally to production sites part of the Pilot Infrastructures.
In the following sections, an overview of the recentli defined best practices that have been adopted in both XDC and DEEP-XDC projects for the Software Lifecycle Management and Continuous Integration and Delivery are presented and described.
\section{Software Quality Assurance and Control}
\section{Software Quality Assurance and Control}
...
@@ -83,7 +89,7 @@ The following particularities characterize the corresponding development teams:
...
@@ -83,7 +89,7 @@ The following particularities characterize the corresponding development teams:
\item More focus on development activities, with limited resources, if any, available for quality assurance activities.
\item More focus on development activities, with limited resources, if any, available for quality assurance activities.
\end{itemize}
\end{itemize}
The Quality Assurance process has to take all these factors into account to define the Software Quality Assurance Plan (SQAP).
The Quality Assurance process has to take all above described factors into account to define the Software Quality Assurance Plan (SQAP).
A set of "QA Policies” have also to be defined to guide the development teams towards uniform practices and processes.
A set of "QA Policies” have also to be defined to guide the development teams towards uniform practices and processes.
These QA Policies define the main activities of the software lifecycle, such as releasing, tracking, packaging and documenting
These QA Policies define the main activities of the software lifecycle, such as releasing, tracking, packaging and documenting
the software carried out by the project. This is done in collaboration with development teams, making sure they are flexible
the software carried out by the project. This is done in collaboration with development teams, making sure they are flexible
...
@@ -110,7 +116,7 @@ Those SQA criteria \ref{R22} have the goal to
...
@@ -110,7 +116,7 @@ Those SQA criteria \ref{R22} have the goal to
\section{Software Maintenance and Support}
\section{Software Maintenance and Support}
Regarding the software maintenance and support area of the software lifecycle management,
Regarding the software maintenance and support area of the software lifecycle management,
the main objectives that should be covered by the project - and described in the Maintenance plan - are:
the main objectives that should be and described in the Maintenance plan are:
\begin{itemize}
\begin{itemize}
\item To increase the quality levels of the software by contributing to the implementation and automation
\item To increase the quality levels of the software by contributing to the implementation and automation
of the Quality Assurance (QA) and Control procedures defined by the project.
of the Quality Assurance (QA) and Control procedures defined by the project.
...
@@ -120,8 +126,8 @@ of the Quality Assurance (QA) and Control procedures defined by the project.
...
@@ -120,8 +126,8 @@ of the Quality Assurance (QA) and Control procedures defined by the project.
\item To guarantee the stability of services already deployed in production and the increase of their readiness
\item To guarantee the stability of services already deployed in production and the increase of their readiness
levels, where needed.
levels, where needed.
\end{itemize}
\end{itemize}
Moreover the common practices deal with the definition of all processes and procedures regarding the software maintenance and
Moreover the common practices deal with the definition of those processes and procedures related to the software maintenance and
support, and their continuous execution:
support and their continuous execution:
\begin{itemize}
\begin{itemize}
\item Software Maintenance - regarding software preparation \& transition from the developers to production
\item Software Maintenance - regarding software preparation \& transition from the developers to production
repositories and final users.
repositories and final users.
...
@@ -132,30 +138,30 @@ repositories and final users.
...
@@ -132,30 +138,30 @@ repositories and final users.
repositories, defining policies and release cycles.
repositories, defining policies and release cycles.
\end{itemize}
\end{itemize}
The plan regarding the software maintenance and support management have to follow the guidelines of the
The plan regarding the software maintenance and support management have to follow the guidelines of the
ISO/IEC 14764:2006 standard \cite{R30}, and includes a set of organizational roles and administrative roles to handle
ISO/IEC 14764:2006 standard \cite{R30}, and includes a set of organizational and administrative roles to handle
maintenance implementation, change management and validation, software release, migration and retirement, support
maintenance implementation, change management and validation, software release, migration and retirement, support
and helpdesk activities.
and helpdesk activities.
Component releases are classified in major, minor, revision and emergency, based on the impact of the changes on the
Component releases are classified in major, minor, revision and emergency, based on the impact of the changes on the
component interface and behavior. Requests for Change (RfC) are managed adopting a priority-driven approach,
component interface and behavior. Requests for Change (RfC) are managed adopting a priority-driven approach,
so that the risk of compromising the stability of the software deployed in a production environment is minimized.
so that the risk of compromising the stability of the software deployed in a production environment is minimized.
The User Support activity deals with the coordination of the support, to users of the software components developed
The User Support activity deals, instead, with the coordination of the support to the users that make use of the software components (developed within the project activities) and included in the main project software distributions.
within the project and included in the main project software distributions.
\section{Services for continuous integration and SQA}
\section{Services for continuous integration and SQA}
A set of tools and services are needed to support the Software Quality Assurance, the
To support the Software Quality Assurance, the
Continuous Integration and the software release and maintenance.
Continuous Integration and the software release and maintenance activities, a set of tools and services are needed.
The choice of using publicly available cloud services has three main reasons:
Usually, those tools and services are provided by using publicly available cloud services due to the following reasons:
\begin{itemize}
\begin{itemize}
\item Higher public visibility and in line with project objectives for open source software
\item Higher public visibility and in line with project objectives for open source software,
\item Provides a path to further development, support and exploitation beyond the end of the project.
\item Provides a path to further development, support and exploitation beyond the end of the project,
\item Smaller effort needed inside the project to operate and manage those services.
\item Smaller effort needed inside the project to operate and manage those services.
\end{itemize}
\end{itemize}
The list of services needed is given below with a small description for each service and the Web link.
The list of services needed is given in Table 1 with a small description for each service and the related Web link.
\begin{figure}[h]
\begin{figure}[h!]
\centering
\centering
Table 1: Tools and services to support DevOps.
\includegraphics[width=10cm,clip]{CI-tools.png}
\includegraphics[width=10cm,clip]{CI-tools.png}
%\caption{The list of services.}
%\caption{The list of services.}
\label{citools}
\label{citools}
...
@@ -165,8 +171,7 @@ The list of services needed is given below with a small description for each ser
...
@@ -165,8 +171,7 @@ The list of services needed is given below with a small description for each ser
\section{Key Performance Indicators}
\section{Key Performance Indicators}
Defining appropriate KPIs for maintenance, release and support activities, and monitor them during the project lifetime
Defining appropriate KPIs for maintenance, release and support activities, and monitor them during the project lifetime
may help in highlight the project achievements and put in place the appropriate corrective actions in case of deviation
may help in highlight the project achievements and put in place the appropriate corrective actions in case of deviations.
of the project activities.
In principle, the KPIs should address the following impact areas and reflect the related goal:
In principle, the KPIs should address the following impact areas and reflect the related goal:
\begin{itemize}
\begin{itemize}
\item Prepare data and computing e-Infrastructures to absorb the needs of communities that push the envelope in terms of data and intensive computing
\item Prepare data and computing e-Infrastructures to absorb the needs of communities that push the envelope in terms of data and intensive computing
...
@@ -180,16 +185,17 @@ In principle, the KPIs should address the following impact areas and reflect th
...
@@ -180,16 +185,17 @@ In principle, the KPIs should address the following impact areas and reflect th
\end{itemize}
\end{itemize}
\section{Conclusions}
\section{Conclusions}
The paper describes the common procedures to be applied in the field of software lifecycle management aimed at managing
The paper describes the common procedures to be applied in the field of software lifecycle management, aimed at managing
the new releases and ensure the quality of the provided solutions, services or components.
the new releases and ensure the quality of the provided solutions, services and components provided by the project.
In particular, the paper described the best practices to adopt in order to i) foster the quality and reliability of the software produced,
In particular, the paper described the best practices to adopt in order to i) foster the quality and reliability of the software produced,
ii) to define the processes and procedures regarding the software maintenance and support, iii) identify the services needed
ii) to define the processes and procedures regarding the software maintenance and support, iii) identify the services needed
to support the Software Quality Assurance, the Continuous Integration and the software release and maintenance
to support the Software Quality Assurance, the Continuous Integration and the software release and maintenance
and iv) define appropriate KPIs to monitor the project achievements.
and iv) define appropriate KPIs to monitor the project achievements.
The experience gathered throughout this activity with regards to the adoption of different DevOps
The experience gathered throughout this activity with regards to
practices is not only useful and suitable for the software related to the core services in any software development project, but
can be also applicable to the development and distribution of the applications coming, for example, from the user communities.
The adoption of different DevOps practices is becaming mandatory for software development projects,
the experience gathered throughout this activity can be also applicable to the development and distribution of software products coming, for example, from the user communities and other software product activities.
\section*{Acknowledgments}
\section*{Acknowledgments}
...
@@ -205,6 +211,8 @@ eXtreme DataCloud has been funded by the European Commission H2020 research and
...
@@ -205,6 +211,8 @@ eXtreme DataCloud has been funded by the European Commission H2020 research and
