\documentclass[a4paper]{jpconf}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{graphicx}

\bibliographystyle{iopart-num}

\begin{document}

\title{A VOMS module for the Nginx web server}
\author{A. Ceccanti$^1$, F. Giacomini$^1$, E. Ronchieri$^1$, N. Terranova$^1$}
\address{$^1$ INFN-CNAF, Bologna, IT}
\ead{francesco.giacomini@cnaf.infn.it}

\begin{abstract} We present the motivation, the design and some
  implementation hints of a software module for the Nginx web server
  aimed at extracting the attributes of a VOMS-issued Attribute
  Certificate during a client authentication based on an X.509 Proxy
  Certificate. The module enables the use of Nginx as a reverse proxy
  to a Grid service, relieving it from most of the work necessary to
  authenticate a client.
\end{abstract}

\section{Introduction}

Every Grid service needs to authenticate and possibly authorize every
request that it receives. The authentication is historically based on
X.509 Proxy Certificates~\cite{rfc5280}, extended with Attribute
Certificates~\cite{rfc5755} obtained from a VOMS
service~\cite{voms}. An Attribute Certificate is conceptually an
assertion signed by the VOMS service that declares the groups the user
submitting the request belongs to within a Virtual Organization
(e.g. a scientific collaboration) and the possible roles they have
within those groups.

As a consequence, a Grid service must carry out a number of
security-related steps before even starting its own business logic:
\begin{itemize}
\item offer an HTTPS endpoint;
\item perform X.509 certificate-based client authentication;
\item extract the VOMS attributes, on which it could later base an
  authorization decision.
\end{itemize}

The purpose of this work is to factor those three actions out of a
Grid service into a common service-independent module to be run by a
reverse proxy deployed in front of the service. The ideal deployment
model is shown in Figure~\ref{fig:deployment}. If the reverse proxy
and the actual service run in a trusted zone, the communication
between the two can even happen over plain HTTP.

\begin{figure}
  \begin{center}
    \includegraphics[width=.9\textwidth]{deployment}
    \caption{\label{fig:deployment}Deployment model enabled by the
      VOMS module run in a front-end service acting as a reverse proxy
      towards the actual service running as the back-end.}
  \end{center}
\end{figure}

After a first prototype~\cite{nginxvoms-cnafar-2016} was prepared as
part of a master thesis, a properly engineered version has been
developed for production use.

\section{Nginx}

The reverse proxy of choice is Nginx~\cite{nginx}. Nginx is an efficient HTTP and
reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy
server, which serves or proxies a large fraction of the busiest
sites on the Internet.

Nginx has a modular architecture: the core software components provide
a solid and efficient foundation to dispatch events (e.g. from
networking) to one or more worker processes to be asynchronously
managed. Additional functionality is further provided by
\textit{modules} that are linked to the Nginx executable. Modules can
also be loaded dynamically at run time, yet they need to be known at
compile time. As a consequence, the typical way to obtain an Nginx
executable is to compile the corresponding source code, specifying the
exact selection of modules needed for the specific deployment.

To simplify our build and testing setup, we have opted to base our
work on top of the OpenResty~\cite{openresty} distribution of Nginx,
which includes already a selection of useful modules.

However we cannot use the Nginx code as-is, for it is unaware of the
delegation mechanism designed by Globus for the Grid Security
Infrastructure on which all Grid services are based. The delegation
mechanism introduces a control character in the HTTP protocol, between
the SSL/TLS handshake and the transmission of the method. Although we
do not intend to support the delegation mechanism in this work, we
wrote a minimal patch to the Nginx request-parsing code, so that it
can accept that extra character and ignore it if it means ``no
delegation''.

An additional patch has been proposed for Nginx upstream, to enable
the support for proxy certificates via a flag in the configuration
file, after recent versions of OpenSSL~\cite{openssl} have removed the
possibility to enable it via an environment variable. Since proxy
certificates are not widely used outside of the Grid world, the patch
has not been accepted; despite being a more convenient and clearer
solution, the patch is not even applied locally, since there is a
workaround applicable directly in our module's code, as described
below.

An instance of Nginx can be configured (and re-configured) through a
file read by the master process. The configuration file can specify,
among many other things, that Nginx runs in reverse-proxy mode and how
the request should be passed to the upstream server (i.e. the
back-end). The syntax of the configuration file allows for the use of
\textit{variables} (in our case, for example, there would be one
called \texttt{\$voms\_fqans}), whose value is provided by a specific
handler that gets executed when the corresponding variable is used.

\section{Building and testing}

Though the development can happen on any platform, the reference
platform for the deployment at the moment is CentOS 7. Moreover, as
mentioned above, we have elected the OpenResty distribution as the
basis for our own Nginx module. A Docker image combining CentOs 7 and
OpenResty is available both for local builds and for the continuous
integration builds bound to the \textit{git} repository that hosts the
source code.

The Docker image is augmented with the necessary dependencies (such as
the VOMS libraries) and with the elected development tools, mostly
installed through the software collections \textit{Developer Toolset
  7} (to have a compiler that supports C++ 2017 and various
sanitizers) and \textit{LLVM Toolset 7} (to have the code formatter
and the static analyzer).

The testing is based on \texttt{Test::Nginx}~\cite{test::nginx}, a
Perl-based testing framework that comes with OpenResty, that allows
the specification of tests with a declarative syntax. We struggled to
find a way to enable TLS client authentication, but the result is very
satisfactory.

The build and the tests are automatically run every time some code is
pushed to the reference git repository~\cite{module-baltig}.

\section{The VOMS module}

The purpose of the VOMS module is to extract the information available
in the Attribute Certificate embedded in the X.509 Proxy Certificate
used for the authentication of the client and make it available as
Nginx variables. The variables can then be used in the configuration
file to form the request that is passed upstream by the reverse proxy.

The variables correspond very closely to the fields of the
\texttt{voms} data structure found in the API of the VOMS C++
library~\cite{voms-github}. For example:

\begin{description}
\item[\texttt{voms\_user}] The Subject of the End-Entity certificate, used to sign the proxy.
\item[\texttt{voms\_fqans}] A comma-separated list of Fully Qualified Attribute
  Names.
\item[\texttt{voms\_vo}] The name of the Virtual Organization (VO) to which the End Entity belongs.
\item[\texttt{voms\_not\_before}] The date before which the Attribute
  Certificate is not yet valid.
\item[\texttt{voms\_not\_after}] The date after which the Attribute Certificate
  is not valid anymore.
\end{description}

The module consists mainly of the handlers that are called when a
variable is referenced in the configuration file. The information
needed to give a value to the variables comes from invocations of the
VOMS library API; that information is obtained as a by-product of the
validation of the X.509 certificate chain presented by the client,
including the VOMS Attribute Certificate. Since such a validation is
expensive, performing it every time a handler is called is best
avoided and a caching strategy is preferable. The caching can be
applied at multiple levels: for each request, for each connection
(multiple requests can be sent over the same connection), for multiple
connections authenticated with the same client proxy certificate. At
the moment the caching is applied at connection level, but moving it
to the next level is already planned.

As mentioned above, the module also enables the support for proxy
certificates in OpenSSL, which in recent versions of the library is
not available any more through setting an environment variable. This
is done calling the appropriate OpenSSL API functions in a handler
that gets executed at the end of the Nginx configuration phase, at a
time when the SSL certificate store, the data structure containing the
flag that enables the acceptance of proxies, is initialized and
available for manipulation.

\section{Deployment}

Once the module described in this work is linked to the Nginx
executable, in order to deploy a Grid service according to the model
sketched in the Introduction, the typical configuration for an Nginx
instance used as a reverse proxy in front of the actual Grid service
would include directives similar to the following.

{\small
\begin{verbatim}

server {

  listen                  443 ssl;
  ssl_certificate         /certs/cert.pem;
  ssl_certificate_key     /certs/key.pem;
  ssl_client_certificate  /etc/pki/tls/certs/ca-bundle.crt;
  ssl_verify_client       on;
  ssl_verify_depth        100;

  location / {

    proxy_set_header      Voms-User       $voms_user;
    proxy_set_header      Voms-Fqans      $voms_fqans;
    proxy_set_header      Voms-Vo         $voms_vo;
    proxy_set_header      Voms-Not-Before $voms_not_before;
    proxy_set_header      Voms-Not-After  $voms_not_after;

    proxy_pass            http://back-end;
  }
}

\end{verbatim}
}
The service running on the back-end would then receive requests over
plain HTTP and will find among its headers all the VOMS information
needed to apply its own authorization policies.

\section{Conclusions and future work}

In this paper we have shown how an Nginx reverse proxy equipped with
the described module could relieve a Grid service from most of the
work necessary to authenticate a client presenting credentials based
on X.509 Proxy Certificates extended with a VOMS-issued Attribute
Certificate.

The module is ready for production use and is currently part of the
development effort aimed at revising the implementation of the StoRM
service.

The main planned development concerns the improvement of the caching
of the information obtained during the validation of VOMS Attribute
Certificates.

\section*{References}
\bibliography{biblio}

\end{document}