{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "76badf4a",
   "metadata": {},
   "source": [
    "This notebook shows how to generate the benchmark plots used in the paper. \n",
    "\n",
    "To generate the plots using the benchmarks of the original paper, download the following gist:\n",
    "\n",
    "[stavros11/ffb88a5b914b60213515f0256c0e8aa4](https://gist.github.com/stavros11/ffb88a5b914b60213515f0256c0e8aa4)\n",
    "\n",
    "copy the contents in a folder named `/data` in the directory of this notebook and execute all cells.\n",
    "\n",
    "The logs provided in the gist serve as a template.\n",
    "The same plotting functionality should work with logs generated from different machines.\n",
    "To generate new logs from scratch one can use the bash scripts provided in the `scripts/` directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "97f43ed0",
   "metadata": {},
   "outputs": [],
   "source": [
    "from utils import load_data, load_evolution_data\n",
    "\n",
    "save = False # if ``True`` plots will be saved in the current directory as pdfs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4d6d97f",
   "metadata": {},
   "source": [
    "## Figure 3\n",
    "\n",
    "Bar plot with import breakdown and dry run vs simulation comparison for qibojit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ba717379",
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "from barplots import plot_breakdown_nqubits\n",
    "\n",
    "data = load_data(f\"./data/qibojit_breakdown.dat\")\n",
    "plot_breakdown_nqubits(data, \"supremacy\", save=save)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e2eeb0b",
   "metadata": {},
   "source": [
    "## Figure 4\n",
    "\n",
    "Scaling plots of execution time as a function of the number of qubits for all qibo backends."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1ba0a87d",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from scaling import plot_scaling\n",
    "\n",
    "cpu_data = load_data(\"./data/qibo_scaling_cpu.dat\")\n",
    "gpu_data = load_data(\"./data/qibo_scaling_gpu.dat\")\n",
    "\n",
    "plot_scaling(cpu_data, gpu_data, \"qft\", \"total_dry_time\", legend=False, save=save)\n",
    "plot_scaling(cpu_data, gpu_data, \"qft\", \"total_simulation_time\", save=save)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8cb5555a",
   "metadata": {},
   "source": [
    "## Figure 5\n",
    "\n",
    "qibojit backend performance on different CPU and GPU devices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "895cb582",
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "import seaborn as sns # used for color palettes\n",
    "from devices import Line, plot_devices\n",
    "\n",
    "oranges = sns.color_palette(\"Oranges\", 2)\n",
    "greens = sns.color_palette(\"Greens\", 2)\n",
    "blues = sns.color_palette(\"Blues\", 3)\n",
    "\n",
    "lines = [\n",
    "    Line(\"NVIDIA RTX A6000 (cupy)\", load_data(\"./data/qibo_scaling_gpu.dat\", qibojit_only=True), blues[2], \"o\"),\n",
    "    Line(\"NVIDIA DGX V100 (cupy)\", load_data(\"./data/dgx_qibojit.dat\", qibojit_only=True), blues[1], \"^\"),\n",
    "    Line(\"NVIDIA GTX 1650 (cupy)\", load_data(\"./data/gtx1650_qibojit.dat\", qibojit_only=True), blues[0], \"d\"),\n",
    "    Line(\"AMD Radeon VII (cupy)\", load_data(\"./data/rocm_qibojit.dat\"), greens[1], \"v\"),\n",
    "    Line(\"NVIDIA RTX A6000 (cupy-multigpu)\", load_data(\"./data/rtx_multigpu.dat\", qibojit_only=True), blues[2], \"o\", linestyle=\"--\"),\n",
    "    Line(\"AMD EPYC 7742, 128 th., 2TB (numba)\", load_data(\"./data/qibo_scaling_cpu.dat\", qibojit_only=True), oranges[1], \"\"),\n",
    "    Line(\"ATOS QLM, 384 th., 6TB (numba)\", load_data(\"./data/qlm_qibojit.dat\", qibojit_only=True), oranges[0], \"\"),\n",
    "]\n",
    "# filter qibojit-cupy only data if the log file contains more (eg. qibojit-cuquantum)\n",
    "is_cupy = lines[0].data[\"library_options\"].apply(lambda x: \"cupy\" in x)\n",
    "lines[0].data = lines[0].data[is_cupy == True]\n",
    "\n",
    "plot_devices(lines, \"qft\", \"total_dry_time\", save=save)\n",
    "plot_devices(lines, \"qft\", \"total_simulation_time\", legendfont=26, save=save)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5cd4c946",
   "metadata": {},
   "source": [
    "## Figure 6\n",
    "\n",
    "Bar plot with different multigpu configurations and qibojit vs qibotf comparison."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "15099a54",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from barplots import plot_multigpu\n",
    "\n",
    "data = load_data(\"./data/dgx_multigpu.dat\")\n",
    "plot_multigpu(data, 32, \"total_dry_time\", save=save)\n",
    "plot_multigpu(data, 32, \"total_simulation_time\", legend=True, save=save)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64538a61",
   "metadata": {},
   "source": [
    "## Figure 7\n",
    "\n",
    "Bar plot with comparisons between different simulation libraries on various circuits."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "81035c0e",
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "import seaborn as sns\n",
    "from libraries import Library, plot_libraries\n",
    "\n",
    "cpu_data = load_data(\"./data/libraries_cpu.dat\")\n",
    "gpu_data = load_data(\"./data/libraries_gpu.dat\")\n",
    "\n",
    "palette = sns.color_palette(\"bright\", 7)\n",
    "libraries = [\n",
    "    Library(\"qibo\", palette[0], \"/\", \"Qibo\"),\n",
    "    Library(\"qibo GPU\", palette[0], \"/\", \"Qibo GPU\"),\n",
    "    Library(\"qiskit\", palette[1], \"-\", \"Qiskit\"),\n",
    "    Library(\"qiskit-gpu GPU\", palette[1], \"-\", \"Qiskit GPU\"),\n",
    "    Library(\"hybridq\", palette[2], \"x\", \"HybridQ\"),\n",
    "    Library(\"hybridq-gpu GPU\", palette[2], \"x\", \"HybridQ GPU\"),\n",
    "    Library(\"qulacs\", palette[4], \"\\\\\", \"Qulacs\", has_single=False),\n",
    "    Library(\"qulacs-gpu GPU\", palette[4], \"\\\\\", \"Qulacs GPU\", has_single=False),\n",
    "    Library(\"projectq\", palette[3], \"o\", \"ProjectQ\", has_single=False),\n",
    "    Library(\"qcgpu GPU\", palette[3], \"o\", \"QCGPU\", has_double=False)\n",
    "]\n",
    "\n",
    "plot_libraries(libraries, cpu_data, gpu_data, \"total_dry_time\", 20, precision=\"single\", legend=False, save=save)\n",
    "plot_libraries(libraries, cpu_data, gpu_data, \"total_dry_time\", 30, precision=\"single\", legend=True, save=save)\n",
    "\n",
    "plot_libraries(libraries, cpu_data, gpu_data, \"total_dry_time\", 20, precision=\"double\", legend=False, save=save)\n",
    "plot_libraries(libraries, cpu_data, gpu_data, \"total_dry_time\", 30, precision=\"double\", legend=True, save=save)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20eb1827",
   "metadata": {},
   "source": [
    "## Figure 8\n",
    "\n",
    "Bar plot comparing fusion vs no fusion for all qibojit platforms and circuits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e0a127b8",
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "from fusion import plot_fusion_circuits\n",
    "\n",
    "data = load_data(f\"./data/qibojit_fusion.dat\")\n",
    "plot_fusion_circuits(data, 30, \"total_dry_time\", fontsize=38, legend=True, save=True)\n",
    "plot_fusion_circuits(data, 30, \"total_simulation_time\", fontsize=38, legend=True, save=save)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "909fbaeb",
   "metadata": {},
   "source": [
    "## Figure 9\n",
    "\n",
    "Bar plot comparing two-qubit fusion for different libraries (qibo, qiskit, qsim)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60e32824",
   "metadata": {},
   "outputs": [],
   "source": [
    "import seaborn as sns\n",
    "from libraries import Library, plot_libraries\n",
    "\n",
    "cpu_data = load_data(\"./data/libraries_fusion_cpu.dat\")\n",
    "gpu_data = load_data(\"./data/libraries_fusion_gpu.dat\")\n",
    "\n",
    "palette = sns.color_palette(\"bright\", 7)\n",
    "libraries = [\n",
    "    Library(\"qibo\", palette[0], \"/\", \"Qibo (numba)\", alpha=0.3),\n",
    "    Library(\"qibo GPU\", palette[0], \"/\", \"Qibo (cupy) GPU\", alpha=1.0),\n",
    "    Library(\"qibo-cuquantum GPU\", palette[0], \"/\", \"Qibo (cuquantum) GPU\", alpha=0.6),\n",
    "    Library(\"qiskit\", palette[1], \"-\", \"Qiskit\", alpha=0.3),\n",
    "    Library(\"qiskit-gpu GPU\", palette[1], \"-\", \"Qiskit GPU\", alpha=1.0),\n",
    "    Library(\"qsim\", palette[2], \"\\\\\", \"qsim\", has_double=False, alpha=0.3),\n",
    "    Library(\"qsim-gpu GPU\", palette[2], \"\\\\\", \"qsim GPU\", has_double=False, alpha=1.0),\n",
    "    Library(\"qsim-cuquantum GPU\", palette[2], \"\\\\\", \"qsim (cuquantum) GPU\", has_double=False, alpha=0.6),\n",
    "]\n",
    "\n",
    "plot_libraries(libraries, cpu_data, gpu_data, \"total_dry_time\", 30, precision=\"single\", \n",
    "               legend=True, fontsize=45, logscale=True, fusion=True, save=save)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5392b2d",
   "metadata": {},
   "source": [
    "## Figure 10\n",
    "\n",
    "Scaling plot vs time dt for adiabatic evolution of TFIM Hamiltonian using the dense form."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6d7c77f0",
   "metadata": {},
   "outputs": [],
   "source": [
    "from evolution import plot_dense\n",
    "\n",
    "data = load_evolution_data(\"data/evolution.dat\")\n",
    "plot_dense(data, \"total_dry_time\", 10, save=save)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b8e88e7",
   "metadata": {},
   "source": [
    "## Figure 11\n",
    "\n",
    "Scaling plot vs time dt for adiabatic evolution of TFIM Hamiltonian using the Trotter decomposition."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67f4884c",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from evolution import plot_trotter\n",
    "\n",
    "data = load_evolution_data(\"data/evolution.dat\")\n",
    "plot_trotter(data, \"total_dry_time\", 10, yticks=[1, 10], legend=True, save=save)\n",
    "plot_trotter(data, \"total_dry_time\", 20, yticks=[1, 10, 100], legend=False, save=save)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}