VASP Compilation
=======================


VASP Source Code 
*****************
As of 8/8/2014, all vasp source code and PAW info are located on grandcentral
at /home/cam1/vasp/.  The time stamps are on everything which can help you
decipher which potcars are the new one, in case it is not obvious. If an
authorized group member needs to download new source from vasp, instructions can
be found in the vasp_download subdirectory in the sphinx doc source tree (obtained by cloning restricted git repository).


Instructions for compiling VASP pure open source
**********************************************************
These instructions are basically `copied`__ from Peter Larsson with a few minor details of differences.
This might be useful if you want to compile somewhere that you do not have access to the Intel suite.

.. code-block:: bash 

  System info
  - Vasp5.2
  - Ubuntu 12.04 
  - gfortran, g++
  - openmpi1.5
  - scalapack
  - OpenBLAS
  - fftw

1. use apt-get to install the following:

.. code-block:: bash 

  sudo apt-get install gfortran git build-essential openmpi1.5-bin libopenmpi1.5-dev


2. Install OpenBlas (formerly Goto Blas)

.. code-block:: bash 

  git clone git://github.com/xianyi/OpenBLAS
  cd OpenBlas
  make FC=gfortran CC=gcc USE_THREAD=0 

3. Install scalapack

.. code-block:: bash 

  wget http://www.netlib.org/scalapack/scalapack.tgz
  tar -xvzf scalapack.tgz
  cd scalapack-*
  cp SLmake.inc.example SLmake.inc

Change the following two lines in SLmake.inc as below, but use the proper path for OpenBLAS:

.. code-block:: bash 

  BLASLIB       = -L/home/cam1/vasp/OpenBLAS -lopenblas
  LAPACKLIB     = $(BLASLIB)

Now Compile:

.. code-block:: bash 

  make

4. Install fftw

.. code-block:: bash 

  wget http://www.fftw.org/fftw-3.3.4.tar.gz
  tar -xvzf fftw-3.3.4.tar.gz
  cd fftw-3.3.4
  configure 
  make 
  sudo make install

5. compile vasp using the following two makefiles (:download:`main <dalal_custom/makefile.5.2_os>` and :download:`lib <dalal_custom/makefile.5.lib_os>`).

__ http://www.nsc.liu.se/~pla/blog/2013/11/05/compile-vasp-on-ubuntu/

Rondinelli's Makefile
**********************************************************
These are the Makefiles that were used to build the executable that was used to benchmark on Rondinelli's cluster (:download:`main <rondinelli/Makefile.5.3.5_rondinelli>` and :download:`lib <rondinelli/Makefile.5.lib_rondinelli>`). 

Pure Intel Compilation
******************************
(Deprecated, see below) In this case we are compiling using the Intel Cluster Studio XE. The resulting executable is the one that is currently run on our local machines.
It should be noted that these are always compiled in parallel,
which is optimal for basically any modern multicore processor. If you want to use spin-orbit coupling, you need to simply remove the string "-DNGZhalf" from the main
makefile and recompile. You do not need to recompile the lib. 

.. code-block:: bash 

  System info
  - Vasp5.2
  - Ubuntu 12.04 
  - Intel Cluster Studio XE

The makefiles can be found here (:download:`main <pure_intel_5.2/makefile_main>` and :download:`lib <pure_intel_5.2/makefile_lib>`).

.. code-block:: bash 

  System info
  - Vasp.4.6
  - Ubuntu 12.04 
  - Intel Cluster Studio XE

The makefiles can be found here (:download:`main <pure_intel_4.6/makefile_main>` and :download:`lib <pure_intel_4.6/makefile_lib>`).

.. _mordechai_compile_vasp:

Mordechai's Instructions for compiling VASP 5.4.1
**************************************************

Thanks to a reorganziation of the files in 5.4.1, it is much easier to compile.

#. Copy the linux intel file
 
  .. code-block:: bash 

     cp arch/makefile.include.linux_intel makefile.include

#. In that file, change ``mpif90`` (which uses gfortran) to ``mpiifort`` (which uses ifort)

#. For FFT, use the VASP modules. Change OBJECTS and INCS to::

    OBJECTS= fftmpi.o fftmpi_map.o fft3dfurth.o fft3dlib.o
    INCS=

#. For Intel MPI, change BLACS/SCALAPACK linking to::

    BLACS      = -lmkl_blacs_intelmpi_lp64
    SCALAPACK  = -lmkl_scalapack_lp64 $(BLACS)

    LLIBS      = -L$(MKL_PATH) $(SCALAPACK) $(LAPACK) $(BLAS)

#. ``make all``


.. _lyuwens_compile_vasp:

Lyuwen's Instructions for compiling VASP 5.4.1
***********************************************

It is a little different from the Mordechai's version, should be working for the most cases

#. The first step is to build the libfftw3xf_intel.a

   .. code-block:: bash

      cd $MKLROOT/interfaces/fftw3xf
      make libintel64 compiler=intel INSTALL_DIR=${HOME}/.local/lib
    
   Now you should find ``${HOME}/.local/lib/libfftw3xf_intel.a`` built and ready.

#. Then we have the vasp.5.4.1.tar.gz and all the patches from the vasp wiki

   .. code-block:: bash

      tar zxvf vasp.5.4.1.tar.gz
      wget  http://cms.mpi.univie.ac.at/patches/patch.5.4.1.08072015.gz \
            http://cms.mpi.univie.ac.at/patches/patch.5.4.1.27082015.gz \
            http://cms.mpi.univie.ac.at/patches/patch.5.4.1.06112015.gz
      gunzip patch.5.4.1.*.gz
      cd vasp.5.4.1; for i in $(ls ../patch.5.4.1.*);do patch -p1 < $i;done
      cp arch/makefile.include.linux_intel makefile.include

#. Now change the FFTW directory

   ::

    OBJECTS    = fftmpiw.o fftmpi_map.o fftw3d.o fft3dlib.o \
                 $(HOME)/.local/lib/libfftw3xf_intel.a
    INCS       =-I$(MKLROOT)/include/fftw

#. Compile everything

   .. code-block:: bash

      make all

#. To build the VASP on MacOS system you need to first install 
   the intel Parallel Studio XE Composer Edition for C++ OS X and 
   the intel Parallel Studio XE Composer Edition for Fortran OS X 
   respectively and remember to check the scalapack in fortran installation (By default it is off). 
   And then compile the MPICH with the intel composers since the intel does not support impi for MacOS. 
   Then we made a few changes to the makefile.include.linux_intel

   ::

      FC = mpifort #by default the mpi version of ifort mpich is this name
      FCL = mpifort -mkl

      BLACS=-lmkl_blacs_mpich_lp64
      OBJECTS    = fftmpiw.o fftmpi_map.o fftw3d.o fft3dlib.o \
                   $(HOME)/.local/lib/libfftw3xf_intel.a

Compiling VASP 5.4.1 with Wannier90 enabled
***********************************************
In order to use Wannier90 with VASP, you need to compile with an additional flag linking to the wannier90 library file libwannier.a.
Begin by compiling the Wannier90 library (ie. see our docs on compiling Wannier90). VASP 5.4.1 appears not to work with the latest
version of Wannier90 (I tried), which is consistent with the VASP docs (`link <https://cms.mpi.univie.ac.at/wiki/index.php/LWANNIER90>`_).
Therefore, compile WANNIER90 v1.2, and then link this directory into the VASP directory.

   .. code-block:: bash

      cam1@morningside:~/vasp/vasp.5.4.1$ ls
      arch  bin  build  makefile  makefile.include  README  src  wannier90-1.2  

First we need to set the flag -DVASP2WANNIER90:

   .. code-block:: bash

    CPP_OPTIONS= -DMPI -DHOST=\"IFC91_ompi\" -DIFC \
                 -DCACHE_SIZE=4000 -DPGF90 -Davoidalloc \
                 -DMPI_BLOCK=8000 -DscaLAPACK -Duse_collective \
                 -DnoAugXCmeta -Duse_bse_te \
                 -Duse_shmem -Dtbdyn -DVASP2WANNIER90


Now we need to modify the makefile such that it knows to link in the wannier library file.

   .. code-block:: bash

      LLIBS      = -L$(MKL_PATH) $(SCALAPACK) $(LAPACK) $(BLAS) ../../wannier90-1.2/libwannier.a

If you have already compiled, you can simply remove build/std/wannier*.o and recompile. 
VASP will now generate the necessary Wannier90 files if requested, and then the Wannier90 code
can be used to actually generate the maximially localized Wannier. Please note that it is
recommended to use the latest version of Wannier90 with these files given that it is backwards 
compatible and it has features that version 1.2 does not (ie. printing out Wannier Hamiltonian, etc).


Compiling VASP 5.4.4 with or without Wannier90 enabled
**********************************************************
The ``vasp.5.4.4.pl2.tgz`` file is available on grandcentral ``/home/cam1/vasp/``.

Please note that VASP 5.4.4 requires subroutines implemented in MPI-3 which is not available in prior versions.
Thus Intel Parallel Studio 2013 would not work with VASP 5.4.4.

The pre-compiler flag ``-DnoAugXCmeta`` use in VASP 5.4.1 has been marked deprecated on 
`VASP wiki <https://www.vasp.at/wiki/wiki/index.php/Precompiler_flags>`_,
and has beed tested to cause divergence in calulation of ThO2 with SCAN functional.
It will not be used in this compilation.
But keep in mind some of the compiled VASP 5.4.4 binaries may be compiled with the flag.

* [OPTIONAL] If you want to compile vasp with Wannier90, compile Wannier90 v1.2 and append ``-DVASP2WANNIER90`` to the ``CPP_OPTIONS`` variable
  in the makefile.include, and append the path to the library ``libwannier.a`` to the ``LLIBS`` variable. 

* [OPTIONAL] If you want to include Intel's fftw3xf interface into the build, build the interface by the following command. 
  (See :ref:`below <notes_on_fft>` for details about fftw)

   .. code-block:: bash

      cd $MKLROOT/interfaces/fftw3xf
      make libintel64 compiler=intel INSTALL_DIR=SOMEWHERE
    
   then append the path to the compiled static library ``libfftw3xf_intel.a`` located in ``SOMEWHERE`` to the ``LLIBS`` variable.


* If you want to use parallel compiling, make sure you have ``makedepf90`` installed through ``apt`` or ``conda``, and change the makefile
  in the root directory to

   .. code-block:: bash

      VERSIONS = std gam ncl gpu gpu_ncl

      .PHONY: all veryclean versions $(VERSIONS)

      all: std gam ncl

      versions: $(VERSIONS)

      $(VERSIONS):
              mkdir build/$@ ; \
              cp src/makefile src/.objects makefile.include build/$@ ; \
              $(MAKE) -C build/$@ VERSION=$@ dependencies -j1 ; \
              $(MAKE) -C build/$@ VERSION=$@ all

      veryclean:
              rm -rf build/*


* Build vasp by running command ``make all``, if you want to use parallel compiling, then you can use command like ``make -j4 std``.

* Notes on FFT libraries: 
.. _notes_on_fft:

   According to `VASP wiki <https://www.vasp.at/wiki/index.php/Installing_VASP.5.X.X#Fast-Fourier-Transforms>`_,
   There are two choices of implementation when linking VASP with FFT:

      1. To use Intel's MKL wrapper of fftw (and compiling with CPP_OPTIONS= .. -DMPI ..):

      .. code-block:: bash

         OBJECTS= fftmpiw.o fftmpi_map.o fftw3d.o fft3dlib.o \
                  <path to>/libfftw3xf_intel.a
         INCS=-I$(MKLROOT)/include/fftw

      2. Or to use Juergen Furtmueller's FFT implementation (and -DMPI):

      .. code-block:: bash

         OBJECTS= fftmpi.o fftmpi_map.o fft3dfurth.o fft3dlib.o
         INCS=

   The compilation guide by :ref:`Mordechai <mordechai_compile_vasp>` used the second implementation 
   while :ref:`Lyuwen <lyuwens_compile_vasp>`'s guide and this VASP 5.4.4 compilation guide recommands the first
   version. According to past runs the first implementation is faster.
   
   When using Intel MKL libraries to build VASP, all FFTW wrappers can be found in the dynamic library ``libmkl_intel_*.so`` 
   or the static library ``libmkl_intel_*.a`` under ``$MKLROOT/lib/intel64``.  Thus building the ``fftw3xf`` interface is not necessary,
   and VASP can be dynamically linked to the FFTW wrappers with the following configuration.

   .. code-block:: bash

      OBJECTS= fftmpiw.o fftmpi_map.o fftw3d.o fft3dlib.o
      INCS=-I$(MKLROOT)/include/fftw