dmsq2018.tex

\documentclass[a4paper]{jpconf}
\usepackage{graphicx}

\usepackage{booktabs}
\bibliographystyle{iopart-num}

\begin{document}
\title{Comparing Data Mining Techniques for Software Defect Prediction}

\author{Marco Canaparo, Elisabetta Ronchieri}

\address{INFN CNAF, Bologna, Italy}

\ead{marco.canaparo@cnaf.infn.it, elisabetta.ronchieri@cnaf.infn.it}

\begin{abstract}
In the last decades, the role of Data Mining techniques have grown in the field of software engineering to cover various tasks, such as software defect prediction and test code generation. In this contribution, we describe the work done in 2018 to compare these techniques for software defect prediction in order to identify the ones that perform the best.
\end{abstract}

%--------------------------------------------------------------------------------
\section{Introduction}
\label{sec_intro}
Over the past years, the use of Data Mining techniques have been growing in various applications of  software engineering. In this field, typical tasks \cite{Ali2013} are source code generation \cite{joulin} and software defect prediction \cite{tong}: the former is usually for the test activity, the latter is for the quality assessment. They rely on software datasets compose of a set of features for the various instances. The features include software metrics and other information, like the instances defectiveness. Through data mining algorithms, software developers may detect violation in the code. Existing literature shows promising approaches to address software engineering issues. Our contribution aims to compare these techniques for software defect prediction in order to identify the ones that perform the best.

During 2018, the main focus of our work was to define a methodology to compare data mining techniques in the context of open source and HEP software.
The following paragraphs summarize the research procedure and the preliminary results.

\section{Research Procedure}
\label{sec_rproc}

The research procedure is composed of five steps:
\begin{enumerate}
\item collection of data mining techniques in the software engineering field;
\item collection of software metrics;
\item collection of datasets;
\item identification of data mining tools and library for this study;
\item identification of performance criteria.
\end{enumerate}

\subsection{Data Mining Techniques}
For the first step, we have considered existing literature focusing on defect prediction. 
\noindent\textbf{Support Vector Machine} (e.g. SMO): is a supervised techniques that searches for the optimal hyperplane to separate training data. The hyperplane found is intuitive: it is the one which is maximally distant from the two classes of labelled points located in each side \cite{vapnik, campbell}.

\noindent\textbf{Decision Tree} (e.g. J48): is a flow-chart like tree structure. It is composed of: nodes which represent a test on a attribute value; branches which show the outcome of the tests; leaves, that indicate the resulting classes \cite{Han2006}.

\noindent\textbf{Naive Bayes}: relies on the Bayesian rule of conditional probability. It assumes that all the attributes are independent and analyses each of them individually \cite{mccallum}.

\noindent\textbf{Ensemble Classifier} (e.g. Random Forest): consists of training multiple classifiers and then combining their predictions \cite{dietterich}. This technique leads to a generalized improvement of the ability of each classifier \cite{he}. According to the way the component classifiers are trained, parallel or sequential, we can distinguish two different categories of ensemble. Bagging \cite{breiman} and Random Forest \cite{breiman2} are both parallel classifiers. Bagging creates multiple version of the classifier by replicating the learning set in parallel from the original on and the final decision is made by majority voting strategy. Random Forest adopts a combination of tree predictors, each depending on the values of a random vector sampled independently and with the same distribution for all trees in the forest. Adaboost \cite{freund} is an example of a sequential classifier since each classifier of this technique is applied sequentially on the training samples misclassified by the previous one.

\noindent\textbf{Deep Learning}: is applied to feature hierarchy where features of higher levels are formed by the composition of lower level ones. Deep learning techniques leverage learning intermediate representations that can be shared across tasks and, as a consequence, they can exploit unsupervised data and data from similar tasks to improve performance on problems characterised by scarcity of labelled data \cite{bengio, raina, collobert}.

\subsection{Software Metrics}
Concerning software metrics, we have collected all the metrics used in literature over time, some of them are:

\noindent\textbf{McCabe} (e.g. Cyclomatic Complexity, Essential Complexity): is used to evaluate the complexity of a software program. It is derived from a flow graph and is mathematically computed using graph theory. Basically, it is determined by counting the number of decision statements in a program \cite{McCabe1976, McCabe1989}.

\noindent\textbf{Halstead} (e.g. Base Measures, Derived Measures): is used to measure some characteristics of a program module - such as the "Length", the "Potential Volume", "Difficulty", the "Programming Time" - by employing some basic metrics like number of unique operators, number of unique operands, total occurrences of operators, total occurrences of operands \cite{shen, Halstead1977}.

\noindent\textbf{Size} (e.g. Lines of Code, Comment Lines of Code): the Lines of Code (LOC) is used to measure a software module and the accumulated LOC of all the modules for measuring a program \cite{li}.

\noindent\textbf{Chidamber and Kemerer} (e.g. Number of Children, Depth of Inheritance): is used for object-oriented programs and is the most popular for performing software analysis and prediction. It has been adopted by many software tool vendors and computer scientists \cite{Chidamber1994, catal}. Some metrics of the suite are: Weighted Method Per Class, which measures the number of methods which is in each class; Depth of Inheritance Tree, which measures the distance of the longest path from a class to the root in the inheritance tree; Number Of Children, which measures the number of classes that are direct descendants of each class.

For the third step, we have focused on the NASA Defect Dataset \cite{Shepperd2013, zhang, song, Gray2011, zhou, Aleem2015, nasadataset, AR}, Eclipse \cite{eclipsedataset}, and Android and Elastic Search \cite{androiddataset}. Table \ref{tab:datasets} shows a summary of the most important characteristics of these datasets in terms of number of projects, metrics, modules and percentage of defective modules per projects, reporting their range whenever possible. Modules refer to instances and represent e.g. classes, files, functions and so on.
%\vspace*{-\baselineskip}
  \begin{table}[h]
  \begin{center}
  \caption{Summary of the datasets employed}
  \label{tab:datasets}
%  \scriptsize
    \begin{tabular}{rrrrr}
      \toprule
      Repository & \#Projects & \#Metrics & \#Modules & \%Defective Modules\\
      \midrule
      NASA Defect Datasets & 11 & [30,41] &  [101, 5589] & [0.41, 48.80]\%\\
      Eclipse Datasets & 5 & 17 each &  [324, 1863] &  [9.26, 39.81]\%\\
      Android Datasets & 6 & 102 each &  [74, 124] &   [0, 27.02]\%\\
      Elastic Search Datasets & 12 &102 each &  [1860, 7263]& [0.16, 11.47]\%\\
      \bottomrule
    \end{tabular}
    \end{center}
  \end{table}
  %\vspace*{-\baselineskip}
  
\subsection{Data Mining Tools}
In relation to the data mining tools, we have employed Weka \cite{weka}, scikit-learn \cite{scikit} and R \cite{r}. They are based on java, python and R and they are characterized by a different learning curve.
 
\subsection{Performance Criteria}  
Finally, for the performance criteria we have taken into account what is available in literature. All the criteria are defined on the basis of the \textbf{confusion matrix} that summarizes the performance of a classification algorithm. According to confusion matrix, we have defined: True Positive (TP) all the instances predicted as defective and that are actually defective; True Negative (TN) all the instances predicted as non-defective and that are actually non-defective; False Positive (FP) all the instances predicted as defective and that are actually non-defective; False Negative (FN) all the instances predicted as non-defective and that are actually defective. In the following, criteria are reported.

\textbf{Accuracy} is the percentage of modules correctly classified as either faulty or non-faulty. 

\textbf{Precision} is the percentage of modules classified as faulty that are actually faulty.

\textbf{Recall} or \textbf{Completeness} is the percentage of faulty modules that are predicted as faulty.

\textbf{Mean Absolute Error} determines how close the values of predicted and actual fault rate differ. 

\textbf{F-measure} is a combined measure of recall and precision, the higher value of this indicator the better is the quality of the learning method for software prediction.

\section{Preliminary Results and Future Works}
\label{sec_pr}

Bagging and Random Forest have achieved the best average accuracy over all the datasets \cite{canaparo}. In the future, we will focus on (semi-)unsupervised machine learning techniques that enable us to include them in the software development process since the majority of software datasets lacks of instance categorizations \cite{ronchieri}. Furthermore, we will investigate in the adoption of various machine learning frameworks on different resource infrastructure, such as cloud and GPU-equipped resources \cite{salomoni}.


% ------------------------------------------------------------------------

\section*{References}
\bibliography{ar2018}

%\section{Introduction}
%These guidelines show how to prepare articles for publication in \jpcs\ using \LaTeX\ so they can be published quickly and accurately. Articles will be refereed by the \corg s but the accepted PDF will be published with no editing, proofreading or changes to layout. It is, therefore, the author's responsibility to ensure that the content and layout are correct.  This document has been prepared using \cls\ so serves as a sample document. The class file and accompanying documentation are available from \verb"http://jpcs.iop.org".
%
%\section{Preparing your paper}
%\verb"jpconf" requires \LaTeXe\ and  can be used with other package files such
%as those loading the AMS extension fonts
%\verb"msam" and \verb"msbm" (these fonts provide the
%blackboard bold alphabet and various extra maths symbols as well as
%symbols useful in figure captions); an extra style file \verb"iopams.sty" is
%provided to load these packages and provide extra definitions for bold Greek letters.
%\subsection{Headers, footers and page numbers}
%Authors should {\it not} add headers, footers or page numbers to the pages of their article---they will
%be added by \iopp\ as part of the production process.
%
%\subsection{{\cls\ }package options}
%The \cls\ class file has two options `a4paper' and `letterpaper':
%\begin{verbatim}
%\documentclass[a4paper]{jpconf}
%\end{verbatim}
%
%or \begin{verbatim}
%\documentclass[letterpaper]{jpconf}
%\end{verbatim}
%
%\begin{center}
%\begin{table}[h]
%\caption{\label{opt}\cls\ class file options.}
%%\footnotesize\rm
%\centering
%\begin{tabular}{@{}*{7}{l}}
%\br
%Option&Description\\
%\mr
%\verb"a4paper"&Set the paper size and margins for A4 paper.\\
%\verb"letterpaper"&Set the paper size and margins for US letter paper.\\
%\br
%\end{tabular}
%\end{table}
%\end{center}
%
%The default paper size is A4 (i.e., the default option is {\tt a4paper}) but this can be changed to Letter by
%using \verb"\documentclass[letterpaper]{jpconf}". It is essential that you do not put macros into the text which alter the page dimensions.
%
%\section{The title, authors, addresses and abstract}
%The code for setting the title page information is slightly different from
%the normal default in \LaTeX\ but please follow these instructions as carefully as possible so all articles within a conference have the same style to the title page.
%The title is set in bold unjustified type using the command
%\verb"\title{#1}", where \verb"#1" is the title of the article. The
%first letter of the title should be capitalized with the rest in lower case.
%The next information required is the list of all authors' names followed by
%the affiliations. For the authors' names type \verb"\author{#1}",
%where \verb"#1" is the
%list of all authors' names. The style for the names is initials then
%surname, with a comma after all but the last
%two names, which are separated by `and'. Initials should {\it not} have
%full stops. First names may be used if desired. The command \verb"\maketitle" is not
%required.
%
%The addresses of the authors' affiliations follow the list of authors.
%Each address should be set by using
%\verb"\address{#1}" with the address as the single parameter in braces.
%If there is more
%than one address then a superscripted number, followed by a space, should come at the start of
%each address. In this case each author should also have a superscripted number or numbers following their name to indicate which address is the appropriate one for them.
%
%Please also provide e-mail addresses for any or all of the authors using an \verb"\ead{#1}" command after the last address. \verb"\ead{#1}" provides the text Email: so \verb"#1" is just the e-mail address or a list of emails.
%
%The abstract follows the addresses and
%should give readers concise information about the content
%of the article and should not normally exceed 200
%words. {\bf All articles must include an abstract}. To indicate the start
%of the abstract type \verb"\begin{abstract}" followed by the text of the
%abstract.  The abstract should normally be restricted
%to a single paragraph and is terminated by the command
%\verb"\end{abstract}"
%
%\subsection{Sample coding for the start of an article}
%\label{startsample}
%The code for the start of a title page of a typical paper might read:
%\begin{verbatim}
%\title{The anomalous magnetic moment of the
%neutrino and its relation to the solar neutrino problem}
%
%\author{P J Smith$^1$, T M Collins$^2$,
%R J Jones$^{3,}$\footnote[4]{Present address:
%Department of Physics, University of Bristol, Tyndalls Park Road,
%Bristol BS8 1TS, UK.} and Janet Williams$^3$}
%
%\address{$^1$ Mathematics Faculty, Open University,
%Milton Keynes MK7~6AA, UK}
%\address{$^2$ Department of Mathematics,
%Imperial College, Prince Consort Road, London SW7~2BZ, UK}
%\address{$^3$ Department of Computer Science,
%University College London, Gower Street, London WC1E~6BT, UK}
%
%\ead{williams@ucl.ac.uk}
%
%\begin{abstract}
%The abstract appears here.
%\end{abstract}
%\end{verbatim}
%
%\section{The text}
%The text of the article should should be produced using standard \LaTeX\ formatting. Articles may be divided into sections and subsections, but the length limit provided by the \corg\ should be adhered to.
%
%\subsection{Acknowledgments}
%Authors wishing to acknowledge assistance or encouragement from
%colleagues, special work by technical staff or financial support from
%organizations should do so in an unnumbered Acknowledgments section
%immediately following the last numbered section of the paper. The
%command \verb"\ack" sets the acknowledgments heading as an unnumbered
%section.
%
%\subsection{Appendices}
%Technical detail that it is necessary to include, but that interrupts
%the flow of the article, may be consigned to an appendix.
%Any appendices should be included at the end of the main text of the paper, after the acknowledgments section (if any) but before the reference list.
%If there are two or more appendices they will be called Appendix A, Appendix B, etc.
%Numbered equations will be in the form (A.1), (A.2), etc,
%figures will appear as figure A1, figure B1, etc and tables as table A1,
%table B1, etc.
%
%The command \verb"\appendix" is used to signify the start of the
%appendixes. Thereafter \verb"\section", \verb"\subsection", etc, will
%give headings appropriate for an appendix. To obtain a simple heading of
%`Appendix' use the code \verb"\section*{Appendix}". If it contains
%numbered equations, figures or tables the command \verb"\appendix" should
%precede it and \verb"\setcounter{section}{1}" must follow it.
%
%\section{References}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%In the online version of \jpcs\ references will be linked to their original source or to the article within a secondary service such as INSPEC or ChemPort wherever possible. To facilitate this linking extra care should be taken when preparing reference lists.
%
%Two different styles of referencing are in common use: the Harvard alphabetical system and the Vancouver numerical system.  For \jpcs, the Vancouver numerical system is preferred but authors should use the Harvard alphabetical system if they wish to do so. In the numerical system references are numbered sequentially throughout the text within square brackets, like this [2], and one number can be used to designate several references.
%
%\subsection{Using \BibTeX}
%We highly recommend the {\ttfamily\textbf\selectfont iopart-num} \BibTeX\ package by Mark~A~Caprio \cite{iopartnum}, which is included with this documentation.
%
%\subsection{Reference lists}
%A complete reference should provide the reader with enough information to locate the article concerned, whether published in print or electronic form, and should, depending on the type of reference, consist of:
%
%\begin{itemize}
%\item name(s) and initials;
%\item date published;
%\item title of journal, book or other publication;
%\item titles of journal articles may also be included (optional);
%\item volume number;
%\item editors, if any;
%\item town of publication and publisher in parentheses for {\it books};
%\item the page numbers.
%\end{itemize}
%
%Up to ten authors may be given in a particular reference; where
%there are more than ten only the first should be given followed by
%`{\it et al}'. If an author is unsure of a particular journal's abbreviated title it is best to leave the title in
%full. The terms {\it loc.\ cit.\ }and {\it ibid.\ }should not be used.
%Unpublished conferences and reports should generally not be included
%in the reference list and articles in the course of publication should
%be entered only if the journal of publication is known.
%A thesis submitted for a higher degree may be included
%in the reference list if it has not been superseded by a published
%paper and is available through a library; sufficient information
%should be given for it to be traced readily.
%
%\subsection{Formatting reference lists}
%Numeric reference lists should contain the references within an unnumbered section (such as \verb"\section*{References}"). The
%reference list itself is started by the code
%\verb"\begin{thebibliography}{<num>}", where \verb"<num>" is the largest
%number in the reference list and is completed by
%\verb"\end{thebibliography}".
%Each reference starts with \verb"\bibitem{<label>}", where `label' is the label used for cross-referencing. Each \verb"\bibitem" should only contain a reference to a single article (or a single article and a preprint reference to the same article).  When one number actually covers a group of two or more references to different articles, \verb"\nonum"
%should replace \verb"\bibitem{<label>}" at
%the start of each reference in the group after the first.
%
%For an alphabetic reference list use \verb"\begin{thereferences}" ... \verb"\end{thereferences}" instead of the
%`thebibliography' environment and each reference can be start with just \verb"\item" instead of \verb"\bibitem{label}"
%as cross referencing is less useful for alphabetic references.
%
%\subsection {References to printed journal articles}
%A normal reference to a journal article contains three changes of font (see table \ref{jfonts}) and is constructed as follows:
%
%\begin{itemize}
%\item the authors should be in the form surname (with only the first letter capitalized) followed by the initials with no periods after the initials. Authors should be separated by a comma except for the last two which should be separated by `and' with no comma preceding it;
%\item the article title (if given) should be in lower case letters, except for an initial capital, and should follow the date;
%\item the journal title is in italic and is abbreviated. If a journal has several parts denoted by different letters the part letter should be inserted after the journal in Roman type, e.g. {\it Phys. Rev.} A;
%\item the volume number should be in bold type;
%\item both the initial and final page numbers should be given where possible. The final page number should be in the shortest possible form and separated from the initial page number by an en rule `-- ', e.g. 1203--14, i.e. the numbers `12' are not repeated.
%\end{itemize}
%
%A typical (numerical) reference list might begin
%
%\medskip
%\begin{thebibliography}{9}
%\item Strite S and Morkoc H 1992 {\it J. Vac. Sci. Technol.} B {\bf 10} 1237
%\item Jain S C, Willander M, Narayan J and van Overstraeten R 2000
%{\it J. Appl. Phys}. {\bf 87} 965
%\item Nakamura S, Senoh M, Nagahama S, Iwase N, Yamada T, Matsushita T, Kiyoku H
%and 	Sugimoto Y 1996 {\it Japan. J. Appl. Phys.} {\bf 35} L74
%\item Akasaki I, Sota S, Sakai H, Tanaka T, Koike M and Amano H 1996
%{\it Electron. Lett.} {\bf 32} 1105
%\item O'Leary S K, Foutz B E, Shur M S, Bhapkar U V and Eastman L F 1998
%{\it J. Appl. Phys.} {\bf 83} 826
%\item Jenkins D W and Dow J D 1989 {\it Phys. Rev.} B {\bf 39} 3317
%\end{thebibliography}
%\smallskip
%
%\noindent which would be obtained by typing
%
%\begin{verbatim}
%\begin{\thebibliography}{9}
%\item Strite S and Morkoc H 1992 {\it J. Vac. Sci. Technol.} B {\bf 10} 1237
%\item Jain S C, Willander M, Narayan J and van Overstraeten R 2000
%{\it J. Appl. Phys}. {\bf 87} 965
%\item Nakamura S, Senoh M, Nagahama S, Iwase N, Yamada T, Matsushita T, Kiyoku H
%and 	Sugimoto Y 1996 {\it Japan. J. Appl. Phys.} {\bf 35} L74
%\item Akasaki I, Sota S, Sakai H, Tanaka T, Koike M and Amano H 1996
%{\it Electron. Lett.} {\bf 32} 1105
%\item O'Leary S K, Foutz B E, Shur M S, Bhapkar U V and Eastman L F 1998
%{\it J. Appl. Phys.} {\bf 83} 826
%\item Jenkins D W and Dow J D 1989 {\it Phys. Rev.} B {\bf 39} 3317
%\end{\thebibliography}
%\end{verbatim}
%
%\begin{center}
%\begin{table}[h]
%\centering
%\caption{\label{jfonts}Font styles for a reference to a journal article.}
%\begin{tabular}{@{}l*{15}{l}}
%\br
%Element&Style\\
%\mr
%Authors&Roman type\\
%Date&Roman type\\
%Article title (optional)&Roman type\\
%Journal title&Italic type\\
%Volume number&Bold type\\
%Page numbers&Roman type\\
%\br
%\end{tabular}
%\end{table}
%\end{center}
%
%\subsection{References to \jpcs\ articles}
%Each conference proceeding published in \jpcs\ will be a separate {\it volume};
%references should follow the style for conventional printed journals. For example:\vspace{6pt}
%\numrefs{1}
%\item Douglas G 2004 \textit{J. Phys.: Conf. Series} \textbf{1} 23--36
%\endnumrefs
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\subsection{References to preprints}
%For preprints there are two distinct cases:
%\renewcommand{\theenumi}{\arabic{enumi}}
%\begin{enumerate}
%\item Where the article has been published in a journal and the preprint is supplementary reference information. In this case it should be presented as:
%\medskip
%\numrefs{1}
%\item Kunze K 2003 T-duality and Penrose limits of spatially homogeneous and inhomogeneous cosmologies {\it Phys. Rev.} D {\bf 68} 063517 ({\it Preprint} gr-qc/0303038)
%\endnumrefs
%\item Where the only reference available is the preprint. In this case it should be presented as
%\medskip
%\numrefs{1}
%\item Milson R, Coley A, Pravda V and Pravdova A 2004 Alignment and algebraically special tensors {\it Preprint} gr-qc/0401010
%\endnumrefs
%\end{enumerate}
%
%\subsection{References to electronic-only journals}
%In general article numbers are given, and no page ranges, as most electronic-only journals start each article on page 1.
%
%\begin{itemize}
%\item For {\it New Journal of Physics} (article number may have from one to three digits)
%\numrefs{1}
%\item Fischer R 2004 Bayesian group analysis of plasma-enhanced chemical vapour deposition data {\it New. J. Phys.} {\bf 6} 25
%\endnumrefs
%\item For SISSA journals the volume is divided into monthly issues and these form part of the article number
%
%\numrefs{2}
%\item Horowitz G T and Maldacena J 2004 The black hole final state {\it J. High Energy Phys.}  	JHEP02(2004)008
%\item Bentivegna E, Bonanno A and Reuter M 2004 Confronting the IR fixed point cosmology 	with 	high-redshift observations {\it J. Cosmol. Astropart. Phys.} JCAP01(2004)001
%\endnumrefs
%\end{itemize}
%
%\subsection{References to books, conference proceedings and reports}
%References to books, proceedings and reports are similar to journal references, but have
%only two changes of font (see table~\ref{book}).
%
%\begin{table}
%\centering
%\caption{\label{book}Font styles for references to books, conference proceedings and reports.}
%\begin{tabular}{@{}l*{15}{l}}
%\br
%Element&Style\\
%\mr
%Authors&Roman type\\
%Date&Roman type\\
%Book title (optional)&Italic type\\
%Editors&Roman type\\
%Place (city, town etc) of publication&Roman type\\
%Publisher&Roman type\\
%Volume&Roman type\\
%Page numbers&Roman type\\
%\br
%\end{tabular}
%\end{table}
%
%Points to note are:
%\medskip
%\begin{itemize}
%\item Book titles are in italic and should be spelt out in full with initial capital letters for all except minor words. Words such as Proceedings, Symposium, International, Conference, Second, etc should be abbreviated to {\it Proc.}, {\it Symp.}, {\it Int.}, {\it Conf.}, {\it 2nd}, respectively, but the rest of the title should be given in full, followed by the date of the conference and the town or city where the conference was held. For Laboratory Reports the Laboratory should be spelt out wherever possible, e.g. {\it Argonne National Laboratory Report}.
%\item The volume number, for example vol 2, should be followed by the editors, if any, in a form such as `ed A J Smith and P R Jones'. Use {\it et al} if there are more than two editors. Next comes the town of publication and publisher, within brackets and separated by a colon, and finally the page numbers preceded by p if only one number is given or pp if both the initial and final numbers are given.
%\end{itemize}
%
%Examples taken from published papers:
%\medskip
%
%\numrefs{99}
%\item Kurata M 1982 {\it Numerical Analysis for Semiconductor Devices} (Lexington, MA: Heath)
%\item Selberherr S 1984 {\it Analysis and Simulation of Semiconductor Devices} (Berlin: Springer)
%\item Sze S M 1969 {\it Physics of Semiconductor Devices} (New York: Wiley-Interscience)
%\item Dorman L I 1975 {\it Variations of Galactic Cosmic Rays} (Moscow: Moscow State University Press) p 103
%\item Caplar R and Kulisic P 1973 {\it Proc. Int. Conf. on Nuclear Physics (Munich)} vol 1 (Amsterdam: 	North-Holland/American Elsevier) p 517
%\item Cheng G X 2001 {\it Raman and Brillouin Scattering-Principles and Applications} (Beijing: Scientific)
%\item Szytula A and Leciejewicz J 1989 {\it Handbook on the Physics and Chemistry of Rare Earths} vol 12, ed K A Gschneidner Jr and L Erwin (Amsterdam: Elsevier) p 133
%\item Kuhn T 1998 {\it Density matrix theory of coherent ultrafast dynamics Theory of Transport Properties of Semiconductor Nanostructures} (Electronic Materials vol 4) ed E Sch\"oll (London: Chapman and Hall) chapter 6 pp 173--214
%\endnumrefs
%
%\section{Tables and table captions}
%Tables should be numbered serially and referred to in the text
%by number (table 1, etc, {\bf rather than} tab. 1). Each table should be a float and be positioned within the text at the most convenient place near to where it is first mentioned in the text. It should have an
%explanatory caption which should be as concise as possible.
%
%\subsection{The basic table format}
%The standard form for a table is:
%\begin{verbatim}
%\begin{table}
%\caption{\label{label}Table caption.}
%\begin{center}
%\begin{tabular}{llll}
%\br
%Head 1&Head 2&Head 3&Head 4\\
%\mr
%1.1&1.2&1.3&1.4\\
%2.1&2.2&2.3&2.4\\
%\br
%\end{tabular}
%\end{center}
%\end{table}
%\end{verbatim}
%
%The above code produces table~\ref{ex}.
%
%\begin{table}[h]
%\caption{\label{ex}Table caption.}
%\begin{center}
%\begin{tabular}{llll}
%\br
%Head 1&Head 2&Head 3&Head 4\\
%\mr
%1.1&1.2&1.3&1.4\\
%2.1&2.2&2.3&2.4\\
%\br
%\end{tabular}
%\end{center}
%\end{table}
%
%Points to note are:
%\medskip
%\begin{enumerate}
%\item The caption comes before the table.
%\item The normal style is for tables to be centred in the same way as
%equations. This is accomplished
%by using \verb"\begin{center}" \dots\ \verb"\end{center}".
%
%\item The default alignment of columns should be aligned left.
%
%\item Tables should have only horizontal rules and no vertical ones. The rules at
%the top and bottom are thicker than internal rules and are set with
%\verb"\br" (bold rule).
%The rule separating the headings from the entries is set with
%\verb"\mr" (medium rule). These commands do not need a following double backslash.
%
%\item Numbers in columns should be aligned as appropriate, usually on the decimal point;
%to help do this a control sequence \verb"\lineup" has been defined
%which sets \verb"\0" equal to a space the size of a digit, \verb"\m"
%to be a space the width of a minus sign, and \verb"\-" to be a left
%overlapping minus sign. \verb"\-" is for use in text mode while the other
%two commands may be used in maths or text.
%(\verb"\lineup" should only be used within a table
%environment after the caption so that \verb"\-" has its normal meaning
%elsewhere.) See table~\ref{tabone} for an example of a table where
%\verb"\lineup" has been used.
%\end{enumerate}
%
%\begin{table}[h]
%\caption{\label{tabone}A simple example produced using the standard table commands
%and $\backslash${\tt lineup} to assist in aligning columns on the
%decimal point. The width of the
%table and rules is set automatically by the
%preamble.}
%
%\begin{center}
%\lineup
%\begin{tabular}{*{7}{l}}
%\br
%$\0\0A$&$B$&$C$&\m$D$&\m$E$&$F$&$\0G$\cr
%\mr
%\0\023.5&60  &0.53&$-20.2$&$-0.22$ &\01.7&\014.5\cr
%\0\039.7&\-60&0.74&$-51.9$&$-0.208$&47.2 &146\cr
%\0123.7 &\00 &0.75&$-57.2$&\m---   &---  &---\cr
%3241.56 &60  &0.60&$-48.1$&$-0.29$ &41   &\015\cr
%\br
%\end{tabular}
%\end{center}
%\end{table}
%
%\section{Figures and figure captions}
%Figures must be included in the source code of an article at the appropriate place in the text not grouped together at the end.
%
%Each figure should have a brief caption describing it and, if
%necessary, interpreting the various lines and symbols on the figure.
%As much lettering as possible should be removed from the figure itself and
%included in the caption. If a figure has parts, these should be
%labelled ($a$), ($b$), ($c$), etc.
%\Tref{blobs} gives the definitions for describing symbols and lines often
%used within figure captions (more symbols are available
%when using the optional packages loading the AMS extension fonts).
%
%\begin{table}[h]
%\caption{\label{blobs}Control sequences to describe lines and symbols in figure
%captions.}
%\begin{center}
%\begin{tabular}{lllll}
%\br
%Control sequence&Output&&Control sequence&Output\\
%\mr
%\verb"\dotted"&\dotted        &&\verb"\opencircle"&\opencircle\\
%\verb"\dashed"&\dashed        &&\verb"\opentriangle"&\opentriangle\\
%\verb"\broken"&\broken&&\verb"\opentriangledown"&\opentriangledown\\
%\verb"\longbroken"&\longbroken&&\verb"\fullsquare"&\fullsquare\\
%\verb"\chain"&\chain          &&\verb"\opensquare"&\opensquare\\
%\verb"\dashddot"&\dashddot    &&\verb"\fullcircle"&\fullcircle\\
%\verb"\full"&\full            &&\verb"\opendiamond"&\opendiamond\\
%\br
%\end{tabular}
%\end{center}
%\end{table}
%
%
%Authors should try and use the space allocated to them as economically as possible. At times it may be convenient to put two figures side by side or the caption at the side of a figure. To put figures side by side, within a figure environment, put each figure and its caption into a minipage with an appropriate width (e.g. 3in or 18pc if the figures are of equal size) and then separate the figures slightly by adding some horizontal space between the two minipages (e.g. \verb"\hspace{.2in}" or \verb"\hspace{1.5pc}". To get the caption at the side of the figure add the small horizontal space after the \verb"\includegraphics" command and then put the \verb"\caption" within a minipage of the appropriate width aligned bottom, i.e. \verb"\begin{minipage}[b]{3in}" etc (see code in this file used to generate figures 1--3).
%
%Note that it may be necessary to adjust the size of the figures (using optional arguments to \verb"\includegraphics", for instance \verb"[width=3in]") to get you article to fit within your page allowance or to obtain good page breaks.
%
%\begin{figure}[h]
%\begin{minipage}{14pc}
%\includegraphics[width=14pc]{name.eps}
%\caption{\label{label}Figure caption for first of two sided figures.}
%\end{minipage}\hspace{2pc}%
%\begin{minipage}{14pc}
%\includegraphics[width=14pc]{name.eps}
%\caption{\label{label}Figure caption for second of two sided figures.}
%\end{minipage}
%\end{figure}
%
%\begin{figure}[h]
%\includegraphics[width=14pc]{name.eps}\hspace{2pc}%
%\begin{minipage}[b]{14pc}\caption{\label{label}Figure caption for a narrow figure where the caption is put at the side of the figure.}
%\end{minipage}
%\end{figure}
%
%Using the graphicx package figures can be included using code such as:
%\begin{verbatim}
%\begin{figure}
%\begin{center}
%\includegraphics{file.eps}
%\end{center}
%\caption{\label{label}Figure caption}
%\end{figure}
%\end{verbatim}
%
%\section*{References}
%\begin{thebibliography}{9}
%\bibitem{iopartnum} IOP Publishing is to grateful Mark A Caprio, Center for Theoretical Physics, Yale University, for permission to include the {\tt iopart-num} \BibTeX package (version 2.0, December 21, 2006) with  this documentation. Updates and new releases of {\tt iopart-num} can be found on \verb"www.ctan.org" (CTAN).
%\end{thebibliography}

\end{document}