.gitlab-ci.yml

+image: blang/latex
+
+stages:
+  - build
+
+job:
+  stage: build
+  script:
+    - mkdir build
+    - cd build
+    - sh ../build.sh
+  artifacts:
+    paths:
+    - build/cnaf-annual-report-2018.pdf
+  tags: 
+    - docker

README.md

-L'Annual Report 2017 del CNAF contiene la descrizione delle attività
-del Centro nell'anno 2017, sotto forma di brevi articoli raccolti in
+L'Annual Report 2018 del CNAF contiene la descrizione delle attività
+del Centro nell'anno 2018, sotto forma di brevi articoli raccolti in
 un unico documento.
 
 Questa repository ha lo scopo di supportare la preparazione del
@@ -16,8 +16,9 @@ I template sono disponibili sia per LaTeX che per Microsoft Word, ma è
 fortemente raccomandato che i contributi siano scritti in
 LaTeX. Questo permette una maggiore qualità del prodotto finale e una
 maggiore automazione nella produzione del report. Il comitato
-editoriale, quest'anno composto da Lucia Morganti, Luca dell'Agnello 
-ed Elisabetta Ronchieri, è a disposizione per fornire chiarimenti
+editoriale, quest'anno composto da Lucia Morganti, Alessandro Costantini, Luca dell'Agnello, 
+Federico Fornari e Elisabetta Ronchieri
+è a disposizione per fornire chiarimenti
 e supporto in caso di necessità.
 
 Il documento finale verrà ottenuto integrando direttamente i

build.sh

@@ -63,76 +63,94 @@ fi
 cd ${builddir}
 
 # prepare cover
-#link_pdf cover cover.pdf
-#link_pdf experiment experiment.pdf
-#link_pdf datacenter datacenter.pdf
-#link_pdf research research.pdf
-#link_pdf transfer transfer.pdf
-#link_pdf additional additional.pdf
-
-#build_from_source user-support user_support.tex *.png
-#build_from_source ams ams.tex  AMS_nuovo.pdf  contributors.pdf  He-MC.pdf  He-MC.tiff  input_output.jpg  production_jobs.jpg
-#build_from_source alice alice.tex *.png *.eps
-#build_from_source atlas    atlas.tex
-#build_from_source borexino borexino.tex
-#build_from_source cms report-cms-feb-2018.tex cms-jobs.eps tier-1-sr-2017.eps
+link_pdf cover cover.pdf
+link_pdf experiment experiment.pdf
+link_pdf datacenter datacenter.pdf
+link_pdf research research.pdf
+link_pdf transfer transfer.pdf
+link_pdf additional additional.pdf
+
+build_from_source user-support main.tex *.PNG
+build_from_source ams AMS-report-2019.tex  AMS_nuovo.pdf  contributors.pdf  He-MC.pdf  input_output.jpg  production_jobs.jpg
+build_from_source alice main.tex *.png 
+build_from_source atlas    atlas.tex
+build_from_source borexino Borexino_CNAFreport2018.tex
+build_from_source cms report-cms-feb-2019.tex tier1-jobs-2018.pdf tier1-readiness-2018.pdf
+link_pdf belle Cnaf-2019-5.0.pdf
 #build_from_source cosa      cosa.tex  biblio.bib beegfs.PNG
-#build_from_source cnprov cnprov.tex
-#build_from_source cta      cta.tex  *.eps
-#build_from_source cuore    cnaf_cuore.tex cnaf_cuore.bib
-#build_from_source cupid    cupid.tex cupid.bib 
-#link_pdf dampe    dampe.pdf
-#link_pdf darkside ds.pdf
+build_from_source cnprov cnprov.tex
+build_from_source cta CTA_annualreport_2018_v1.tex *.eps
+build_from_source cuore cuore.tex cuore.bib
+build_from_source cupid    main.tex cupid-biblio.bib 
+build_from_source dampe    main.tex *.jpg *.png
+build_from_source darkside ds-annual-report-2019.tex
 #build_from_source eee eee.tex  EEEarch.eps  EEEmonitor.eps  EEEtracks.png  ELOGquery.png  request.png
 #build_from_source exanest   exanest.tex  biblio.bib monitoring.PNG storage.png
-#build_from_source famu famu.tex  fig1.eps fig2.eps
 #build_from_source fazia fazia.tex
+build_from_source fermi fermi.tex
+build_from_source gamma gamma.tex
+build_from_source icarus report_2018.tex *.png
 #build_from_source gerda    gerda.tex *.pdf
 #build_from_source glast 	glast.tex 
-#link_pdf juno juno.pdf
-#build_from_source km3net km3net.tex compmodel.png  threetier.png
-#build_from_source lhcb lhcb.tex *.jpg
-#build_from_source lhcf  lhcf.tex
-#build_from_source limadou  limadou.tex
+link_pdf juno juno-annual-report-2019.pdf
+build_from_source km3net km3net.tex compmodel.png  threetier.png
+build_from_source na62 main.tex 
+build_from_source newchim repnewchim18.tex fig1.png
+build_from_source lhcb lhcb.tex *.png
+build_from_source lhcf  lhcf.tex
+build_from_source limadou limadou.tex
 #build_from_source lowcostdev lowcostdev.tex *.jpg
 #build_from_source lspe lspe.tex biblio.bib lspe_data_path.pdf
+build_from_source virgo AdV_computing_CNAF.tex
+build_from_source xenon main.tex xenon-computing-model.pdf
 
-#build_from_source mw-esaco     mw-esaco.tex *.png
-#build_from_source mw-kube      mw-kube.tex
-#build_from_source mw-cdmi-storm mw-cdmi-storm.tex *.png *.jpeg
-#build_from_source mw-software  mw-software.tex
-#build_from_source mw-iam       mw-iam.tex
+build_from_source sc18 SC18.tex *.png
+
+## Research and Developments
+build_from_source sd_iam	main.tex biblio.bib *.png
+build_from_source sd_storm	main.tex biblio.bib *.png
+build_from_source sd_storm2	main.tex biblio.bib *.png
+build_from_source sd_nginx_voms	main.tex biblio.bib *.png
 
 #build_from_source na62	 na62.tex
-#link_pdf padme padme.pdf
+link_pdf padme 2019_PADMEcontribution.pdf
 #build_from_source xenon xenon.tex  xenon-computing-model.pdf
-#build_from_source sysinfo  sysinfo.tex pres_rundeck.png deploy_grafana.png
+build_from_source sysinfo  sysinfo.tex *.png
 #link_pdf virgo VirgoComputing.pdf
 
-#build_from_source tier1 tier1.tex
+build_from_source tier1 tier1.tex *.png
 #build_from_source flood theflood.tex *.png
-#build_from_source farming farming.tex
+build_from_source HTC_testbed HTC_testbed_AR2018.tex
+build_from_source farming ARFarming2018.tex *.png *.jpg
 #build_from_source dynfarm dynfarm.tex 
-#build_from_source storage storage.tex *.png Huawei_rack.JPG
+build_from_source storage storage.tex *.PNG
 #build_from_source seagate seagate.tex biblio.bib  *.png *.jpg
 #build_from_source dataclient dataclient.tex
 #build_from_source ltpd ltpd.tex *.png
-#build_from_source net net.tex *.png
+build_from_source net main.tex *.png
 #build_from_source ssnn1 ssnn.tex *.jpg
 #build_from_source ssnn2 vmware.tex *.JPG *.jpg
 
 #build_from_source infra Chiller.tex  chiller-location.png
-
+build_from_source audit Audit-2018.tex image.png
 #build_from_source cloud_cnaf cloud_cnaf.tex *.png
-#build_from_source srp SoftRel.tex ar2017.bib
+build_from_source dmsq dmsq2018.tex ar2018.bib
 #build_from_source st StatMet.tex sm2017.bib
-#build_from_source cloud_a cloud_a.tex *.pdf
+build_from_source ds_eoscpilot ds_eoscpilot.tex *.png
+build_from_source ds_eoschub ds_eoschub.tex *.png
+build_from_source ds_cloud_c ds_cloud_c.tex *.png 
+build_from_source ds_infn_cc ds_infn_cc.tex *.png
+build_from_source ds_devops_pe ds_devops_pe.tex *.png 
 #build_from_source cloud_b cloud_b.tex *.png *.jpg
 #build_from_source cloud_c cloud_c.tex *.png *.pdf
 #build_from_source cloud_d cloud_d.tex *.png 
+build_from_source sdds-xdc SDDS-XDC.tex *.png
+build_from_source sdds-deep SDDS-DEEP.tex *.png
+build_from_source PhD_DataScience_2018 PhD-DataScience-2018.tex 
+build_from_source chnet dhlab.tex *.png 
 
-#build_from_source pett pett.tex bibliopett.bib
-#build_from_source iso iso.tex 27001.png  biblioiso.bib
+build_from_source pett pett.tex bibliopett.bib
+build_from_source summerstudent summerstudent.tex *.png  
 
 pdflatex ${topdir}/cnaf-annual-report-2018.tex \
 && pdflatex ${topdir}/cnaf-annual-report-2018.tex 2> /dev/null \

contributions/HTC_testbed/HTC_testbed_AR2018.tex

+\documentclass[a4paper]{jpconf}
+\usepackage[english]{babel}
+% \usepackage{cite}
+\usepackage{biblatex}
+%\bibliographystyle{abnt-num}
+
+%%%%%%%%%% Start TeXmacs macros
+\newcommand{\tmtextit}[1]{{\itshape{#1}}}
+\newenvironment{itemizedot}{\begin{itemize} \renewcommand{\labelitemi}{$\bullet$}\renewcommand{\labelitemii}{$\bullet$}\renewcommand{\labelitemiii}{$\bullet$}\renewcommand{\labelitemiv}{$\bullet$}}{\end{itemize}}
+%%%%%%%%%% End TeXmacs macros
+
+\begin{document}
+
+\title{Evaluating Migration of INFN--Tier 1 from CREAM-CE/LSF to
+HTCondor-CE/HTCondor}
+\author{Stefano Dal Pra$^1$}
+\address{$^1$ INFN-CNAF, Bologna, IT}
+\ead{stefano.dalpra@cnaf.infn.it}
+
+
+\begin{abstract}
+  The Tier 1 data center provides computing resources for a variety of HEP and
+  Astrophysics experiments, organized in Virtual Organization submitting their
+  jobs to our computing facilities through Computing Elements, acting as Grid
+  interfaces to the Local Resource Manager. We planned to phase--out our
+  current LRMS (IBM/Platform LSF 9.1.3) and CEs (CREAM) to adopt HTCondor as a
+  replacement of LSF and HTCondor--CE instead of CREAM. A small cluster has
+  been set up to practice the management and evaluate a migration plan to a
+  new LRMS and CE set. This document reports about our early experience on
+  this. 
+\end{abstract}
+
+\section{Introduction}
+
+The INFN-Tier 1 currently provides a computing power of about 400 kHS06, 35000
+slots on one thousand physical Worker Nodes. These resources are accessed
+through Grid by 24 Grid VOs and locally by 25 user groups.
+
+The IBM/Platform LSF 9.1.3 batch system arbitrate access to all the competing
+users groups, both Grid and local, according to a \tmtextit{fairshare} policy,
+designed to prevent underutilization of the available resources or starvation
+of lower priority groups, while ensuring a medium--term share proportional to
+configured quotas.
+
+The CREAM--CEs act as frontend for Grid users to the underlying LSF batch
+system, submitting their jobs on behalf of them. This setup has proven to be
+an effective solution for several years. However, the compatibility between
+CREAM and HTCondor seems to be less tight than with LSF. Moreover, active
+development of CREAM has recently ceased and thus we cannot expect new
+versions to be released, nor better HTCondor support to be implemented by an
+official development team. We decided to migrate our batch system solution
+from LSF to HTCondor, thus we need to also change our CEs. We have selected
+HTCondor-CE as a natural choice, because it is maintained by the same
+development team of HTCondor. In the following we provide a report about our
+experience with HTCondor and HTCondor--CE.
+
+\section{The HTCondor cluster}
+
+To get acquainted with the new batch system and CEs, to evaluate how these can
+work together, how other components, such as monitoring, provisioning and
+accounting systems can be integrated with HTCondor and HTCondor--CE and
+finally to devise a reasonable migration plan, a simple small HTCondor 8.6.13
+cluster has been set up during spring 2018. A HTCondor--CE was soon added, in
+late April. HTCondor is a very mature opensource product, deployed at several
+major Tier 1 for years, thus we already know that it will certainly fit our
+use cases. The HTCondor--CE, on the other hand, is a more recent product, and
+a number of issues might be too problematic for us to deal with. Our focus is
+about ensuring that this CE implementation can be a viable solution for us.
+
+\subsection{The testbed}
+
+The test cluster consists of:
+\begin{itemizedot}
+  \item a HTCondor--CE on top of
+  
+  \item a HTCondor \ Central Manager and Collector
+  
+  \item 3 Worker Nodes (Compute Nodes, in HTCondor terms), 16 slot each.
+\end{itemizedot}
+
+\subsection{HTCondor--CE Installation and setup}
+
+The first CE installation was a bit tricky. The RPMs were available from OSG
+repositories only, meaning that a number of default settings and dependencies
+were unmet for EGI standards. Short after, however, HTCondor--CE RPMs were made
+available on the same official repository of HTCondor.
+
+\subsubsection{Setup}
+
+To setup the configuration for the HTCondor and HTCondor--CE, puppet modules
+are available. Unfortunately the puppet system at our site is not compatible
+with these modules as they depend on \tmtextit{hiera}, which is not supported
+at our site. These were later adapted to make them compatible with our
+configuration management system. In the meanwhile, the setup was finalized
+looking at the official documentation.
+
+\subsubsection{Configuration}
+
+The first configuration was completed manually. The main documentation
+source for the HTCondor--CE is that of the OSG website~\cite{OSGDOC},
+which refers to a tool \tmtextit{osg-configure} not present on the
+general HTCondor--CE release.  Because of this, the setup was completed
+by trial and error. Once a working setup was obtained, a set of
+integration notes were added to a public wiki~\cite{INFNWIKI}.  This
+should help other non OSG users to get some supplementary hint to
+complete their installation.
+
+\subsubsection{Accounting}
+
+As of 2018, the official EGI accounting tool, APEL~\cite{APEL}, has no support for
+HTCondor--CE. On the other hand, INFN--T1 has a custom accounting tool in
+place for several years now~\cite{DGAS}. Thus, it's all about finding a suitable way to
+retrieve from HTCondor the same information that we retrieve from CREAM--CE
+and LSF.
+
+A working way to do so has been by using python and the \tmtextit{python
+bindings}, a set of api interfaces to the HTCondor daemons. These can be used
+to query the SCHEDD at the CE and retrieve a specified set of data\quad about
+recently finished jobs, which are subsequently inserted to our local
+accounting database. A noticeable fact to note, is that the grid information
+(User DN, VO, etc.) are directly available together with all the needed
+accounting data. This simplifies the accounting problem, as it is no more
+necessary to collect grid data separately from the BLAH component and then
+look for matches with the corresponding grid counterpart.
+
+This solution have been used during 2018 to provide accounting for
+HTCondor--CE testbed cluster.
+
+\subsection{Running HTCondor--CE}
+
+After some time to become confident with the main configuration tasks, the
+testbed begun working with jobs submitted by the 4 LHC experiments from
+September 2018. The system proved to be stable and smooth, being able to work
+unattended. This confirms that this system can be a reliable substitute for
+CREAM--CE and LSF.
+
+\subsection{Running HTCondor}
+
+The HTCondor batch system is a mature product with a large user base. We have
+put less effort at investigating it deeply. We already know that most or all
+of needed features will work well. Rather, some effort have been put on
+dealing with configuration management.
+
+\subsubsection{Configuration management}
+
+Eventhoutgh a standard base of puppet classes have been adapted to our
+management system, an additional python tool have been written to improve
+flexibility and readiness. The tool works by reading and enforcing on each
+node of the cluster a set of configuration directives written on text files
+accessible from a shared filesystem. The actual set and the read order depends
+on the host role and name. Doing so, a large cluster can be quite easily
+managed as a collection of set of host sets. The tool is quite simple and
+limited but it can be improved as needed when more complex requirements should
+arise.
+
+\subsection{The migration plan}
+
+After using the testbed cluster a possible plan for a smooth migration have
+been devised:
+\begin{itemizedot}
+  \item Install and setup a new HTCondor cluster, with a few more HTCondor--CE
+  and an initial small set of Worker Nodes
+  \item Enable the LHC VOs on the new cluster
+  \item Add more WN to the new cluster gradually
+  \item Enable other Grid VOs
+  \item Finally, enable submission from local submissions. These are made from
+  a heterogenous set of users, with a potentially rich set of individual needs
+  and can require a considerable administrative effort to meet all of them.
+\end{itemizedot}
+
+\subsection{Conclusion}
+
+A testbed cluster based on HTCondor--CE on top of HTCondor batch system has
+been deployed to evaluate these as a substitute for CREAM--CE and LSF. The
+evaluation has mostly focused on the HTCondor--CE, as it is the most recent
+product. Apart for a few minor issues, mainly related to gaps in the available
+documentation, the CE proved to be a stable component. The possibility to
+perform accounting has been verified.
+
+\section*{References}
+\begin{thebibliography}{9}
+\bibitem{OSGDOC} \url{https://opensciencegrid.org/docs/compute-element/install-htcondor-ce/}
+\bibitem{INFNWIKI} \url{http://wiki.infn.it/progetti/htcondor-tf/htcondor-ce_setup}
+
+\bibitem{DGAS} S. Dal Pra, ``Accounting Data Recovery. A Case Report from
+  INFN-T1'' Nota interna, Commissione Calcolo e Reti dell'INFN, {\tt CCR-48/2014/P}
+\bibitem{APEL} \url{https://wiki.egi.eu/wiki/APEL}
+\end{thebibliography}
+
+\end{document}

contributions/PhD_DataScience_2018/PhD-DataScience-2018.tex

+\documentclass[a4paper]{jpconf}
+\usepackage{graphicx}
+\begin{document}
+\title{ Infrastructures and Big Data processing as pillars in the XXXIII PhD course in Data Sciece and Computation}
+
+%\address{Production Editor, \jpcs, \iopp, Dirac House, Temple Back, Bristol BS1~6BE, UK}
+\author{D. Salomoni$^1$, A. Costantini$^1$, C. D. Duma$^1$, B. Martelli$^1$, D. Cesini$^1$, E. Fattibene$^1$, D. Michelotto $^1$
+        % etc.
+}
+
+\address{$^1$ INFN-CNAF, Bologna, IT}
+
+\ead{davide.salomoni@cnaf.infn.it}
+
+\begin{abstract}
+During the Academic year 2017-2018 the Alma Mater Studiorum, Universuty of Bologna (IT) activated the XXXIII PhD course in Data Science and Computation. 
+The course runs for four years and it is devoted to those students graduated in the field of  Mathematical Physical, Chemical and Astronomical Sciences. 
+This course builds upon fundamental data science disciplines to train candidates that should to become able to carry out academic and industrial research 
+at a higher level of abstraction, with different final specializations in several different fields where data analysis and computation becomes prominent. 
+In this respect, INFN-CNAF was responsible for two courses: Infrastructure for Big Data processing Basic (IBDB) and Advanced (IBDA)
+\end{abstract}
+
+
+\section{Introduction}
+During the Academic year 2017-2018 the Alma Mater Studiorum, Universuty od Bologna (IT) activated the PhD XXXIII course in Data Science and Computation. 
+
+The PhD course starts off based on a joint collaboration of the University of Bologna with politecnico di Milano, the Golinelli Foundation, the Italian Institute 
+of Technology, Cineca, the ISI Foundation and INFN. Even though they are all Italian, each of the aforementioned institutions has already achieved a renown
+ international role in the upcoming field of scientific management and processing of data. Nonetheless, during its lifetime the Course is intended to discuss, 
+design and establish a series of international initiatives that include the possibility to reach agreements with foreign Universities and Research Institutions to 
+issue, for example: joint doctoral degrees, co-­tutorship and student exchanges. These activities will be carried out also based on the contribution that each 
+member of the Course Board will provide. 
+
+The PhD course runs for four years and is aimed at train people to become able to carry out academic and industrial research at a level of abstraction that 
+builds atop each single scientific skill which lies at the basis of the field of ``Data Science''. 
+
+Drawing on this, students graduated in the field of  Mathematical Physical, Chemical and Astronomical Sciences should produce original and significant 
+researches in terms of scientific publications and innovative applications, blending basic disciplines and finally specializing in specific fields as from those
+ provided in the following ``Curricula and Research'' topics
+\begin{itemize}
+\item Quantitative Finance and Economics 
+\item Materials and Industry 4.0
+\item Genomics and bioinformatics
+\item Personalised medicine
+\item Hardware and Infrastructure
+\item Machine learning and deep learning
+\item Computational physics
+\item Big Data, Smart Cities \& Society
+\end{itemize}
+
+In this respect, INFN-CNAF was responsible for two courses: Infrastructure for Big Data processing Basic (IBDB) and Advanced (IBDA).
+ Davide Salomoni has been the responsible in charge for both courses.
+
+\section{Activities to be carried out during the Course}
+
+At the beginning of the course each student is supported by a supervisor, member of the Collegio dei Docenti (Faculty Board), who guides him throughout 
+the Ph.D. studies. The first 24 months are devoted to the integration and deepening of the student expertise, according to a personalized learning plan 
+(drawn up by the student in agreement with the supervisor and then submitted to the Board for approval). The learning plan foresees reaching 40 CFU
+ (credits) by attending courses and passing the corresponding exams. By the 20th month (from the beginning of the course) the student must submit a
+ written thesis proposal to the Board for approval. By the end of the 24th month the student must have completed the personalized learning plan and 
+must report on the progress of the thesis draft. The admission (from the first) to the second is taken into consideration by the Board (and approved in 
+the positive case) on the basis of the fact that the candidate has obtained an adequate number of CFU. The admission (from the second) to the third 
+is taken into consideration by the Board (and approved in the positive case) if the candidate has obtained all the CFU and on the basis of a candidate's 
+public presentation regarding his/her thesis proposal. The third and the fourth years are entirely devoted to the thesis work. he admission (from the third) 
+to the fourth is is taken into consideration by the Board (and approved in the positive case) on the basis of a candidate's public presentation regarding the 
+current status of his/her thesis. The Board finally approves the admission to the final exam, on the basis of the reviewers' comments. The Board may 
+authorize a student to spend a period in Italy at universities, research centers or companies. It is mandatory for the student to spend a period of at 
+least 3 months abroad, during the 3rd/4th year of the course.
+
+\section{Infrastructure for Big Data processing}
+As already mentioned, the didactical units Infrastructure for Big Data processing Basic (IBDB) and Advanced (IBDA), headed by Davide Salomoni with the 
+support of the authors, have been an integral part of the PhD course and constituted the personalized learning plan of some PhD students.
+In order to made available the teaching material and to made possible an active interaction among the teachers and the students, a Content 
+Management System have been deployed and made available. The CMS elected for such activity have been Moodle \cite{moodle} and the entire courses 
+have been made available trough it via a dedicated link (https://moodle.cloud.cnaf.infn.it/).   
+
+\subsection{Infrastructure for Big Data processing Basic}
+The course is aimed at providing basic concepts of Cloud computing at the Infrastructure-as-a-Service level. The course started with an introduction to
+ Big Data It will continue with a description of the building blocks of modern data centers and how they are abstracted by the Cloud paradigm. A real-life 
+computational challenge was also given and students have to create (during the course) a cloud-based computing model to solve this challenge. Access
+ to a limited set of Cloud resources and services was granted to students in order to complete the exercises. A very brief introduction to High Performance 
+Computing (HPC) has been also be given. Notions about the emerging “fog” and “edge” computing paradigms and how they are linked to Cloud infrastructures concluded the course.
+The course foresees an oral exam focusing on the presented topics. Students have been requested to prepare a small project discussed during the exam.
+The course IBDA covered the wollowing arguments
+\begin{itemize}
+\item  Introduction to IBDB: Here, an introduction to the course and its objective are described to the students. Moreover, a presentation of the computational challenges  during the course.
+\item Datacenter building blocks: Basic concepts related to batch system, queues, allocation policies, quota, etc. and a description of the different storage system have been provided. 
+Moreover, an overview of networking, monitoring and provisioning concepts have been given. 
+\item Infrastructures for Parallel Computing: High Throughput Vs High Performance computing have been described and analysed.
+\item Cloud Computing: An introduction to Cloud IaaS have been provided and some comparisons among public and private cloud have been given. Hands-on have been provided 
+on how to use the IaaS stack layer, deploy virtual resources and create different components.
+\item Creating a computing model in distributed infrastructures and multi-sites Cloud: Here an overview of the common strategies for Job Submission, Data Management, Failover 
+and Disaster Recovery have been described. Moreover, a discussion on computing model creation and introduction to the projects requested for the examination have been started.
+\item Computing Continuum: Here an introduction to Low Power devices, Edge Computing, Fog Computing and Computing Continuum for Big Data Infrastructures have been presented.
+\end{itemize}
+
+
+\subsection{Infrastructure for Big Data processing Advanced}
+The course is aimed at discussing the foundations of Cloud computing and storage services beyond IaaS (PaaS and SaaS) leading the students to understand how to 
+exploit distributed infrastructures for Big Data processing. 
+The IBDA course is intended as an evolution of the IBDB and, therefore, before following this course the IBDB should have already been achieved, or having familiarity with the covered topics.
+At the end of the course, the student had practical and theoretical knowledge on distributed computing and storage infrastructures, cloud computing and virtualization,
+ parallel computing and their application to Big Data Analysis.
+The course foresees an oral exam focusing on the presented topics. Students have been requested to prepare a small project discussed during the exam.
+The course IBDA covered the wollowing arguments
+\begin{itemize}
+\item Introduction to IBDA. Here, an introduction to the course and its objective are described to the students. Moreover, a general presentation about Clouds beyond 
+the IaaS and the INDIGO-DataCloud architecture as a concrete example have been discussed.
+\item Authentication and Authorization: Here principles of Cloud authentication and authorization (X.509, SAML, OpenID-Connect, LDAP, Kerberos, Username/password, 
+OAuth) have been presented, with a focus on the INDIGO-IAM (Identity and Access Management) tool \cite{iam}. The session envisaged also a set of hands-on related to 
+1)Connecting to INDIGO IAM, 2)Adapting a web-based application to use IAM, 3)Connecting multiple AuthN methods.
+\item Cloud PaaS. Here an overview of PaaS and related examples have bee provided, together with a hogh-level description of the TOSCA \cite{tosca} standard for PaaS automation. 
+Hands-on related to TOSCA template and Alien4Cloud \cite{a4c}.
+\item Non-Posix Cloud Storage. Lessons are intended tp provide to the students the basic concepts on POSIX and Object storage with pracital examples and hands-on on CEPH \cite{ceph}
+\item Containers. The origin of containers, Docker \cite{docker} and dockerfiles, automation with Docker Swarm and security considerations about containers are provided. Moreover, 
+a description of how to run docker containers in userspace with udocker \cite{udocker}. Hands-on on how to create a container, working with docker versions and deploy a container
+ in a Cloud have been carried out to conplete the session.
+\item Resource orchestration. Here the local orchestration of resources in Kubernetes \cite{kubernetes}, Mesos \cite{mesos} have been described, with a focus on how Information 
+Sysytems and the INDIGO Orcehstrator \cite{orchestrator} can be used to orchestrate resources remotely. The hands-on to create and deploy an HTCondor cluster over a Cloud has been also provided to the students.
+\item Distributed File Systems. Storj, ipfs and Onedata basic concepts have been described. For the laest topic, an hands-on on how to store and replicate files at multiple sites with Onedata have been provided.
+\item Cloud automation. The basic concepts of configuration management automation have been described, focusing the session on Ansible \cite{ansible} configuration manager and its relation with the TOSCA templates.
+\end{itemize}
+
+\section{Conclusions}                
+Based on a joint collaboration of the University of Bologna with Politecnico di Milano, the Golinelli Foundation, the Italian Institute of Technology, 
+Cineca, the ISI Foundation and INFN, the XXXIII PhD course in Data Sciece and Computation has been activated.
+The course is aimed at train people to become able to carry out academic and industrial research at a level of abstraction 
+that builds atop each single scientific skill which lies at the basis of the field of Data Science.
+
+As part of the PhD course, the teaching units Infrastructure for Big Data processing Basic (IBDB) and Advanced (IBDA) has 
+been included in the personalized learning plan of some PhD students. The teaching units are aimed at providing foundations of Cloud 
+computing and storage services beyond IaaS, PaaS, SaaS, leading the students to understand how to exploit distributed infrastructures for Big Data processing.
+
+As an expect result, original, relevant and significant research activities are due by the end of the Course that can take different forms including
+ for example: scientific publications, system and software design, realization and production, and any kind of innovative applications specializing on a
+ broad gamut of topics, such as for example: Quantitative Finance and Economics; Materials and Industry 4.0; Genomics and bioinformatics; Personalised 
+medicine; Hardware and Infrastructure; Machine learning and deep learning; Computational physics; Big Data, Smart Cities \& Society.
+
+\section{References} 
+\begin{thebibliography}{}
+\bibitem{moodle}
+Web site: https://moodle.org
+\bibitem{iam}
+Web site: https://www.indigo-datacloud.eu/identity-and-access-management
+\bibitem{tosca}
+Web site: https://github.com/indigo-dc/tosca-types
+\bibitem{a4c}
+Web site: https://github.com/indigo-dc/alien4cloud-deep
+\bibitem{ceph}
+Web site: https://ceph.com
+\bibitem{docker}
+Web site: https://www.docker.com/
+\bibitem{udocker}
+Web site: https://github.com/indigo-dc/udocker
+\bibitem{kubernetes}
+Web site: https://kubernetes.io/
+\bibitem{mesos}
+Web site: mesos.apache.org
+\bibitem{orchestrator}
+Web site: https://www.indigo-datacloud.eu/paas-orchestrator
+\bibitem{ansible}
+Web site: https://www.ansible.com
+\end{thebibliography}
+
+
+\end{document}
+
+

contributions/README.md

+This directory contains for each contribution a folder