final first commit
This commit is contained in:
167
qibojit-benchmarks/scripts/quimb_mpi/README.md
Normal file
167
qibojit-benchmarks/scripts/quimb_mpi/README.md
Normal file
@@ -0,0 +1,167 @@
|
||||
# Quimb MPI Benchmark Suite
|
||||
|
||||
Demonstrates MPI-enabled tensor network simulation using quimb with 2 MPI processes.
|
||||
|
||||
## Files
|
||||
|
||||
- **quimb_mpi.sh** — Main benchmark sweep script using `mpirun -np 2`
|
||||
- **expectation_mps_mpi.json** — Expectation mode with MPS (default for expectation)
|
||||
- **expectation_dense_mpi.json** — Expectation mode, dense tensor network
|
||||
- **dense_vector_mps_mpi.json** — State extraction with MPS (default for state)
|
||||
- **dense_vector_dense_mpi.json** — State extraction, dense tensor network
|
||||
|
||||
## Key Differences from `/scripts/quimb/`
|
||||
|
||||
- Each `python compare.py` call wrapped with `mpirun -np 2` to enable MPI parallelization
|
||||
- All JSON configs include `"MPI_enabled": true`
|
||||
- Default file names include `_mpi` suffix to distinguish from single-process configs
|
||||
- Filenames in logs clearly show MPI mode (e.g., `expectation_mps_mpi.json`)
|
||||
|
||||
## Configuration Files
|
||||
|
||||
### Expectation Modes (computing Hamiltonian expectation values)
|
||||
- **expectation_mps_mpi.json**: MPI-enabled, uses Matrix Product State (MPS) tensor network ansatz; good for larger systems with limited bond dimension
|
||||
- **expectation_dense_mpi.json**: MPI-enabled, uses dense tensor network; full contraction without MPS truncation
|
||||
|
||||
### State Modes (full state vector computation)
|
||||
- **dense_vector_mps_mpi.json**: MPI-enabled, extracts full state using MPS contraction (faster, bounded memory)
|
||||
- **dense_vector_dense_mpi.json**: MPI-enabled, extracts full state using dense tensor (maximum memory usage)
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
cd /ssd_data/tankya2/code/ASC_2026_prism/ASC-2026/qibojit-benchmarks/scripts/quimb_mpi
|
||||
|
||||
# Run with defaults (all matching dense_vector*.json and expectation*.json configs)
|
||||
./quimb_mpi.sh
|
||||
|
||||
# Test only expectation mode, custom qubit sizes
|
||||
modes="expectation" nqubits_list="8 10 12" ./quimb_mpi.sh
|
||||
|
||||
# Test dense vector only with one explicit config
|
||||
state_cfg=dense_vector_dense_mpi.json modes="dense_vector" ./quimb_mpi.sh
|
||||
|
||||
# Restrict the sweep to a chosen subset of configs
|
||||
EXPECTATION_CONFIGS="expectation_mps_mpi.json expectation_dense_mpi.json" ./quimb_mpi.sh
|
||||
|
||||
# Use 4 MPI processes (if testing scalability)
|
||||
np=4 ./quimb_mpi.sh
|
||||
```
|
||||
|
||||
## Supported Circuits
|
||||
|
||||
### ✓ Fully Supported (Numerically Validated)
|
||||
|
||||
| Circuit | Alias | Parameters | Status | Notes |
|
||||
|---------|-------|------------|--------|-------|
|
||||
| **qft** | — | none | ✓ Works | Quantum Fourier Transform; aligned with qibojit |
|
||||
| **variational** | — | nlayers | ✓ Works | Parameterized circuit; aligned with qibojit |
|
||||
| **bv** | bernstein-vazirani | none | ✓ Works | Bernstein-Vazirani algorithm; aligned with qibojit |
|
||||
| **hs** | hidden-shift | none | ✓ Works | Hidden-shift problem; aligned with qibojit |
|
||||
| **qaoa** | — | nlayers | ✓ Works | QAOA with RZZ gate support; aligned with qibojit |
|
||||
|
||||
### ❌ Not Supported
|
||||
|
||||
| Circuit | Issue | Notes |
|
||||
|---------|-------|-------|
|
||||
| **qv** | Qiskit API compatibility | quantum-volume uses deprecated qiskit `.qasm()` method (v3.0+); not specific to quimb |
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `nqubits_list` | `6 8 10` | Space-separated qubit counts to benchmark |
|
||||
| `circuits` | `qaoa` | Space-separated circuit names |
|
||||
| `modes` | `dense_vector expectation` | Benchmark modes (dense_vector, expectation) |
|
||||
| `nlayers` | `2` | Layers for layered circuits (variational, qaoa) |
|
||||
| `np` | `2` | Number of MPI processes (mpirun -np) |
|
||||
| `nreps` | `1` | Repetitions per benchmark |
|
||||
| `filename` | `quimb_benchmark_mpi.dat` | Output log file |
|
||||
| `precision` | `complex128` | NumPy dtype (complex128 or complex64) |
|
||||
| `exp_cfg` | unset | Optional single expectation config; when unset, all `expectation*.json` files are swept |
|
||||
| `state_cfg` | unset | Optional single dense-vector config; when unset, all `dense_vector*.json` files are swept |
|
||||
| `EXPECTATION_CONFIGS` | unset | Optional space-separated subset of expectation configs to run |
|
||||
| `STATE_CONFIGS` | unset | Optional space-separated subset of dense-vector configs to run |
|
||||
|
||||
## Results
|
||||
|
||||
Results are stored in `/qibojit-benchmarks/benchmarks/results/` with filename prefix `quimb_benchmark_mpi.dat` by default (configurable via `filename=` env var).
|
||||
|
||||
Each benchmark includes:
|
||||
- **import_time** — Backend initialization time
|
||||
- **creation_time** — Circuit creation time
|
||||
- **dry_run_time** — First execution time (warm-up)
|
||||
- **simulation_times** — Execution times for nreps runs
|
||||
- **expectation_result** — Observable expectation values (for expectation mode)
|
||||
|
||||
## Observable
|
||||
|
||||
By default, expectation uses:
|
||||
- **X on qubit 0**, identity on remaining qubits
|
||||
- Configurable via `pauli_pattern` in JSON config (e.g., `"XIIII"` for 5 qubits)
|
||||
|
||||
## Reference Validation
|
||||
|
||||
Each benchmark run automatically includes a **qibojit (numba) reference** with the same observable configuration, enabling direct numerical comparison for validation.
|
||||
|
||||
Example:
|
||||
```
|
||||
[expectation MPI] quimb + variational @ 6q: 0.2019...
|
||||
[qibojit ref] numba + variational @ 6q: 0.2019... ✓ aligned
|
||||
```
|
||||
|
||||
## Supported Circuits
|
||||
|
||||
### ✓ Fully Supported (Numerically Validated)
|
||||
|
||||
| Circuit | Alias | Parameters | Status | Notes |
|
||||
|---------|-------|------------|--------|-------|
|
||||
| **qft** | — | none | ✓ Works | Quantum Fourier Transform; aligned with qibojit |
|
||||
| **variational** | — | nlayers | ✓ Works | Parameterized circuit; aligned with qibojit |
|
||||
| **bv** | bernstein-vazirani | none | ✓ Works | Bernstein-Vazirani algorithm; aligned with qibojit |
|
||||
| **hs** | hidden-shift | none | ✓ Works | Hidden-shift problem; aligned with qibojit |
|
||||
| **qaoa** | — | nlayers | ✓ Works | QAOA with RZZ gate support; aligned with qibojit |
|
||||
|
||||
### ❌ Not Supported
|
||||
|
||||
| Circuit | Issue | Notes |
|
||||
|---------|-------|-------|
|
||||
| **qv** | Qiskit API compatibility | quantum-volume uses deprecated qiskit `.qasm()` method (v3.0+); not specific to quimb |
|
||||
|
||||
## Notes
|
||||
|
||||
- **Intel MPI Fabric:** On Prism, automatically sets `I_MPI_FABRICS=shm` (shared-memory fabric) for single-node local runs
|
||||
- **Observable:** X on first qubit, identity on rest (configurable via `pauli_pattern` in JSON)
|
||||
- **Reference:** qibojit (numba backend) included automatically with aligned observable for validation
|
||||
- **RZZ Gate:** Full support for QAOA and algorithms using two-qubit Ising rotations
|
||||
- **cu1 Gate:** Fully supported in expectation value mode (QFT compatible)
|
||||
- **Numerical Alignment:** All benchmarks automatically include qibojit reference with identical observable for validation
|
||||
|
||||
## MPI Enable/Disable
|
||||
|
||||
To test without MPI, use configs from `/scripts/quimb/` instead:
|
||||
|
||||
```bash
|
||||
# Compare: MPI vs no MPI
|
||||
./quimb_mpi.sh # with MPI (2 processes)
|
||||
cd ../quimb && ./quimb_local.sh # without MPI (single process)
|
||||
```
|
||||
|
||||
## Tensor Network Options
|
||||
|
||||
Quimb supports both MPS (Matrix Product State) and dense tensor network contractions. By default the script sweeps every matching JSON file in this folder, or you can target individual configs:
|
||||
|
||||
```bash
|
||||
# Compare MPS vs dense at same nqubits with MPI
|
||||
exp_cfg=expectation_mps_mpi.json nqubits_list="12" ./quimb_mpi.sh
|
||||
exp_cfg=expectation_dense_mpi.json nqubits_list="12" ./quimb_mpi.sh
|
||||
```
|
||||
|
||||
MPS is typically faster and uses bounded memory, while dense provides full accuracy without truncation.
|
||||
|
||||
## Additional Notes
|
||||
|
||||
- **Bond Dimension:** MPS modes use max_bond_dimension=20 by default; adjust in JSON for memory/accuracy tradeoff
|
||||
- **SVD Cutoff:** Set to 1e-10 for high accuracy; increase for faster execution with acceptable loss tolerance
|
||||
- **Scaling:** MPI parallelization enables larger systems; test with np=4, np=8, etc. for scaling studies
|
||||
- **Single-Node MPI:** Default `-np 2` demonstrates MPI capability on local machine; for cluster runs, adjust `-np` and node configuration
|
||||
@@ -0,0 +1 @@
|
||||
{"MPI_enabled": true, "expectation_enabled": false, "use_mps": false}
|
||||
@@ -0,0 +1 @@
|
||||
{"MPI_enabled": true, "expectation_enabled": false, "use_mps": true, "max_bond_dimension": 20, "svd_cutoff": 1e-10}
|
||||
@@ -0,0 +1 @@
|
||||
{"MPI_enabled": true, "expectation_enabled": true, "use_mps": false, "pauli_pattern": "XIIII"}
|
||||
@@ -0,0 +1 @@
|
||||
{"MPI_enabled": true, "expectation_enabled": true, "use_mps": true, "max_bond_dimension": 20, "svd_cutoff": 1e-10, "pauli_pattern": "XIIII"}
|
||||
216
qibojit-benchmarks/scripts/quimb_mpi/quimb_mpi.sh
Executable file
216
qibojit-benchmarks/scripts/quimb_mpi/quimb_mpi.sh
Executable file
@@ -0,0 +1,216 @@
|
||||
#!/usr/bin/env bash
|
||||
# MPI-enabled quimb benchmark sweep (2 processes).
|
||||
# Tests both dense vector (non-expectation) and expectation across multiple
|
||||
# qubit sizes and circuit types, using mpirun -np 2 to demonstrate MPI capability.
|
||||
#
|
||||
# Usage examples:
|
||||
# ./quimb_mpi.sh
|
||||
# nqubits_list="6 8 10" circuits="variational bv" ./quimb_mpi.sh
|
||||
# modes="expectation" nreps=3 ./quimb_mpi.sh
|
||||
# state_cfg=dense_vector_dense_mpi.json modes="dense_vector" ./quimb_mpi.sh
|
||||
set -euo pipefail
|
||||
|
||||
: "${precision:=complex128}"
|
||||
: "${nreps:=3}"
|
||||
: "${filename:=quimb_benchmark_mpi.dat}"
|
||||
: "${nlayers:=2}"
|
||||
: "${nqubits_list:=6 8 10}"
|
||||
: "${circuits:=supremacy qft variational qaoa}"
|
||||
: "${modes:=dense_vector expectation}"
|
||||
: "${np:=2}"
|
||||
: "${exp_cfg:=}"
|
||||
: "${state_cfg:=}"
|
||||
: "${EXPECTATION_CONFIGS:=}"
|
||||
: "${STATE_CONFIGS:=}"
|
||||
|
||||
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
repo_root="$(cd "${script_dir}/../.." && pwd)"
|
||||
cd "${repo_root}"
|
||||
|
||||
config_list_from_prefix() {
|
||||
local prefix="$1"
|
||||
local single_cfg="$2"
|
||||
local override_list="$3"
|
||||
|
||||
if [[ -n "${override_list}" ]]; then
|
||||
for cfg in ${override_list}; do
|
||||
printf '%s\n' "${cfg}"
|
||||
done
|
||||
return
|
||||
fi
|
||||
|
||||
if [[ -n "${single_cfg}" ]]; then
|
||||
printf '%s\n' "${single_cfg}"
|
||||
return
|
||||
fi
|
||||
|
||||
shopt -s nullglob
|
||||
local matches=( "${script_dir}/${prefix}"*.json )
|
||||
shopt -u nullglob
|
||||
|
||||
for path in "${matches[@]}"; do
|
||||
basename "${path}"
|
||||
done
|
||||
}
|
||||
|
||||
require_existing_configs() {
|
||||
local mode="$1"
|
||||
shift
|
||||
|
||||
for cfg in "$@"; do
|
||||
if [[ ! -f "${script_dir}/${cfg}" ]]; then
|
||||
echo "Missing ${mode} config: ${script_dir}/${cfg}" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
read_pauli_pattern() {
|
||||
local cfg_path="$1"
|
||||
|
||||
python - "${cfg_path}" <<'PY'
|
||||
import json
|
||||
import sys
|
||||
|
||||
try:
|
||||
with open(sys.argv[1], "r") as f:
|
||||
data = json.load(f)
|
||||
print(data.get("pauli_pattern", ""))
|
||||
except Exception:
|
||||
print("")
|
||||
PY
|
||||
}
|
||||
|
||||
build_ref_pattern() {
|
||||
local base_pauli_pattern="$1"
|
||||
local nqubits="$2"
|
||||
|
||||
python - "${base_pauli_pattern}" "${nqubits}" <<'PY'
|
||||
import sys
|
||||
|
||||
p = sys.argv[1].upper()
|
||||
n = int(sys.argv[2])
|
||||
|
||||
if not p:
|
||||
p = "X" + "I" * max(n - 1, 0)
|
||||
if len(p) < n:
|
||||
p = p + ("I" * (n - len(p)))
|
||||
else:
|
||||
p = p[:n]
|
||||
print(p)
|
||||
PY
|
||||
}
|
||||
|
||||
run_qibojit_reference() {
|
||||
local nqubits="$1"
|
||||
local circuit="$2"
|
||||
local circuit_opts="$3"
|
||||
local exp_cfg_name="$4"
|
||||
local exp_cfg_path="${script_dir}/${exp_cfg_name}"
|
||||
local base_pauli_pattern
|
||||
local ref_pattern
|
||||
|
||||
base_pauli_pattern="$(read_pauli_pattern "${exp_cfg_path}")"
|
||||
ref_pattern="$(build_ref_pattern "${base_pauli_pattern}" "${nqubits}")"
|
||||
|
||||
echo " [qibojit reference] cfg=${exp_cfg_name}"
|
||||
python compare.py \
|
||||
--circuit "${circuit}" \
|
||||
${circuit_opts:+--circuit-options "${circuit_opts}"} \
|
||||
--nqubits "${nqubits}" \
|
||||
--filename "${filename}" \
|
||||
--library-options backend=qibojit,platform=numba,expectation=${ref_pattern} \
|
||||
--nreps "${nreps}" \
|
||||
--precision "${precision}"
|
||||
}
|
||||
|
||||
mapfile -t expectation_cfgs < <(
|
||||
config_list_from_prefix "expectation" "${exp_cfg}" "${EXPECTATION_CONFIGS}"
|
||||
)
|
||||
mapfile -t state_cfgs < <(
|
||||
config_list_from_prefix "dense_vector" "${state_cfg}" "${STATE_CONFIGS}"
|
||||
)
|
||||
|
||||
run_dense_mode=false
|
||||
run_expectation_mode=false
|
||||
|
||||
if echo "${modes}" | grep -Eqw "dense_vector|statevector"; then
|
||||
run_dense_mode=true
|
||||
fi
|
||||
|
||||
if echo "${modes}" | grep -qw "expectation"; then
|
||||
run_expectation_mode=true
|
||||
fi
|
||||
|
||||
if [[ "${run_dense_mode}" == true && ${#state_cfgs[@]} -eq 0 ]]; then
|
||||
echo "No dense-vector JSON configs found in ${script_dir}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ "${run_expectation_mode}" == true && ${#expectation_cfgs[@]} -eq 0 ]]; then
|
||||
echo "No expectation JSON configs found in ${script_dir}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
require_existing_configs "dense-vector" "${state_cfgs[@]}"
|
||||
require_existing_configs "expectation" "${expectation_cfgs[@]}"
|
||||
|
||||
# On Prism, Intel MPI needs shared-memory fabric for single-node runs.
|
||||
if [[ -z "${I_MPI_FABRICS:-}" ]]; then
|
||||
export I_MPI_FABRICS=shm
|
||||
fi
|
||||
|
||||
echo "Dense-vector configs : ${state_cfgs[*]:-none}"
|
||||
echo "Expectation configs : ${expectation_cfgs[*]:-none}"
|
||||
|
||||
for nqubits in ${nqubits_list}; do
|
||||
for circuit in ${circuits}; do
|
||||
circuit_opts="nlayers=${nlayers}"
|
||||
# These circuits do not use nlayers — omit it to avoid unknown-option errors.
|
||||
if [[ "${circuit}" == "qft" || "${circuit}" == "QFT" || \
|
||||
"${circuit}" == "supremacy" || "${circuit}" == "Supremacy" || \
|
||||
"${circuit}" == "bv" || "${circuit}" == "bernstein-vazirani" || \
|
||||
"${circuit}" == "hs" || "${circuit}" == "hidden-shift" || \
|
||||
"${circuit}" == "qaoa" || "${circuit}" == "qv" || \
|
||||
"${circuit}" == "quantum-volume" ]]; then
|
||||
circuit_opts=""
|
||||
fi
|
||||
|
||||
echo "===== nqubits=${nqubits} circuit=${circuit} np=${np} ====="
|
||||
|
||||
# dense_vector is the preferred name; statevector is kept as a legacy alias.
|
||||
if [[ "${run_dense_mode}" == true ]]; then
|
||||
for cfg_name in "${state_cfgs[@]}"; do
|
||||
state_cfg_path="${script_dir}/${cfg_name}"
|
||||
echo " [dense_vector MPI] cfg=${cfg_name}"
|
||||
mpirun -np "${np}" python compare.py \
|
||||
--circuit "${circuit}" \
|
||||
${circuit_opts:+--circuit-options "${circuit_opts}"} \
|
||||
--nqubits "${nqubits}" \
|
||||
--filename "${filename}" \
|
||||
--library-options backend=qibotn,platform=quimb,computation_settings=${state_cfg_path} \
|
||||
--nreps "${nreps}" \
|
||||
--precision "${precision}"
|
||||
done
|
||||
fi
|
||||
|
||||
if [[ "${run_expectation_mode}" == true ]]; then
|
||||
for cfg_name in "${expectation_cfgs[@]}"; do
|
||||
exp_cfg_path="${script_dir}/${cfg_name}"
|
||||
echo " [expectation MPI] cfg=${cfg_name}"
|
||||
mpirun -np "${np}" python compare.py \
|
||||
--circuit "${circuit}" \
|
||||
${circuit_opts:+--circuit-options "${circuit_opts}"} \
|
||||
--nqubits "${nqubits}" \
|
||||
--filename "${filename}" \
|
||||
--library-options backend=qibotn,platform=quimb,computation_settings=${exp_cfg_path} \
|
||||
--nreps "${nreps}" \
|
||||
--precision "${precision}"
|
||||
|
||||
run_qibojit_reference "${nqubits}" "${circuit}" "${circuit_opts}" "${cfg_name}"
|
||||
done
|
||||
fi
|
||||
|
||||
echo
|
||||
done
|
||||
done
|
||||
Reference in New Issue
Block a user