qiskit-documentation/docs/guides/serverless-manage-resources.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6b7abc7b-b435-43d1-9fd8-c349ee8710f3",
   "metadata": {},
   "source": [
    "# Manage Qiskit Serverless compute and data resources\n",
    "\n",
    "Qiskit Serverless allows you to manage compute and data across your [Qiskit pattern](./intro-to-patterns), including CPUs, QPUs, and other compute accelerators.\n",
    "\n",
    "## Parallel workflows"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2d40a63-3359-46e9-8f1b-4746b449b407",
   "metadata": {},
   "source": [
    "For classical tasks that can be parallelized, use the `@distribute_task` decorater to define compute requirements needed to perform a task. Start by recalling the `transpile_parallel.py` example from the [Write your first Qiskit Serverless program](./serverless-first-program) topic:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "475d82f0-15cc-4db3-b3b0-54b07822b2a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# /source_files/transpile_remote.py\n",
    "\n",
    "from qiskit.transpiler.preset_passmanagers import generate_preset_pass_manager\n",
    "from qiskit_serverless import distribute_task\n",
    "\n",
    "@distribute_task(target={\"cpu\": 1})\n",
    "def transpile_remote(circuit, optimization_level, backend):\n",
    "    \"\"\"Transpiles an abstract circuit into an ISA circuit for a given backend.\"\"\"\n",
    "    pass_manager = generate_preset_pass_manager(\n",
    "        optimization_level=optimization_level,\n",
    "\t\tbackend=backend\n",
    "    )\n",
    "    isa_circuit = pass_manager.run(circuit)\n",
    "    return isa_circuit"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5914f1d-f898-4db4-8d1e-ccc8081883b9",
   "metadata": {},
   "source": [
    "In this example, you decorated the `transpile_remote()` method with `@distribute_task(target={\"cpu\": 1})`. When run, this creates an asynchronous parallel worker task with a single CPU core, and returns with a reference to track the worker. To fetch the result, pass the reference to the `get()` method:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74fdcd4a-01cd-46ca-aa24-2a8a3605346f",
   "metadata": {},
   "source": [
    "```python\n",
    "from qiskit_serverless import get\n",
    "\n",
    "transpile_worker_reference = transpile_remote(circuit, optimization_level, backend)\n",
    "result = get(transpile_worker_reference)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "268d5ef6-12cf-4ba5-95a3-f7460e4b3bfc",
   "metadata": {},
   "source": [
    "You can also create and run multiple parallel tasks as follows:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac99b4a0-4a42-4c43-869d-265344b70359",
   "metadata": {},
   "source": [
    "```python\n",
    "transpile_worker_references = [\n",
    "    transpile_remote(circuit, optimization_level, backend)\n",
    "    for circuit in circuit_list\n",
    "]\n",
    "\n",
    "results = get(transpile_worker_references)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "611fe030-4494-46b5-9ea1-9678ac513210",
   "metadata": {},
   "source": [
    "### Explore different task configurations\n",
    "\n",
    "You can flexibly allocate CPU, GPU, and memory for your tasks via `@distribute_task()`. For Qiskit Serverless on IBM Quantum™ Platform, each program is equipped with 16 CPU cores and 32 GB RAM, which can be allocated dynamically as needed.\n",
    "\n",
    "CPU cores can be allocated as full CPU cores, or even fractional allocations, as shown in the following.\n",
    "\n",
    "Memory is allocated in number of bytes. Recall that there are 1024 bytes in a kilobyte, 1024 kilobytes in a megabyte, and 1024 megabytes in a gigabyte. To allocate 2 GB of memory for your worker, you need to allocate `\"mem\": 2 * 1024 * 1024 * 1024`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cea90969-cfbf-4181-9ffa-524f3709dc69",
   "metadata": {},
   "outputs": [],
   "source": [
    "@distribute_task(target={\n",
    "    \"cpu\": 16,\n",
    "    \"mem\": 32 * 1024 * 1024 * 1024\n",
    "})\n",
    "def transpile_remote(circuit, optimization_level, backend):\n",
    "    return None"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6fff174-301f-47ca-b7c6-69bb5298fd14",
   "metadata": {},
   "source": [
    "## Automatic QPU selection\n",
    "\n",
    "This example demonstrates how to use `IBMQPUSelector` to automate the process of selecting which qubits to use from a set of available QPUs. Instead of manually selecting a QPU, Qiskit Serverless automatically allocates a QPU according to desired criteria. `IBMQPUSelector`s are imported from server-side `qiskit_serverless_tools.selectors` modules, and must be invoked from your uploaded program.\n",
    "\n",
    "Here, `IBMLeastNoisyQPUSelector` finds the QPU that yields the least-noisy qubit subgraph for the input circuit, from among the QPUs available to you through your IBM Quantum account.\n",
    "\n",
    "For each `IBMQPUSelector`, the context is set in the constructor. All `IBMQPUSelectors` require Qiskit Runtime credentials. The `IBMLeastNoisyQPUSelector` requires a circuit and transpile options specifying how the circuit should be optimized for each QPU, to determine the most optimal QPU and qubit layout.\n",
    "\n",
    "All `IBMQPUSelectors` implement a `get_backend` method, which retrieves the optimal QPU with respect to the given context. The `get_backend` method also allows for additional filtering of the QPUs. It is implemented using the same interface as the [QiskitRuntimeService.backends](/api/qiskit-ibm-runtime/qiskit_ibm_runtime.QiskitRuntimeService#backends) method."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9b38d02-1c4b-4598-abc8-f2ff2f3c1d23",
   "metadata": {},
   "source": [
    "```python\n",
    "# source_files/transpile_parallel.py\n",
    "\n",
    "from qiskit_ibm_runtime import QiskitRuntimeService\n",
    "from qiskit.circuit.random import random_circuit\n",
    "from qiskit_serverless_tools.selectors import IBMLeastNoisyQPUSelector\n",
    "\n",
    "service = QiskitRuntimeService(channel='ibm_quantum', token=API_TOKEN)\n",
    "\n",
    "abstract_circuit = random_circuit(\n",
    "    num_qubits=100, depth=4, measure=True\n",
    ")\n",
    "\n",
    "selector = IBMLeastNoisyQPUSelector(\n",
    "    service, circuit=abstract_circuit, transpile_options={\"optimization_level\": 3}\n",
    ")\n",
    "backend = selector.get_backend(min_num_qubits=127)\n",
    "target_circuit = selector.optimized_circuit\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8eda9b3e-31c3-4783-b5f6-a72bd31e41b6",
   "metadata": {},
   "source": [
    "You can also use the `IBMLeastBusyQPUSelector` to find a QPU that can support the circuit width but with the shortest queue."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10bdb211-0d53-4269-aa92-51ecf494bd63",
   "metadata": {},
   "outputs": [],
   "source": [
    "# source_files/transpile_parallel.py\n",
    "\n",
    "from qiskit.transpiler.preset_passmanagers import generate_preset_pass_manager\n",
    "from qiskit_serverless_tools.selectors import IBMLeastBusyQPUSelector\n",
    "\n",
    "backend = IBMLeastBusyQPUSelector(service).get_backend(min_num_qubits=127)\n",
    "pm = generate_preset_pass_manager(optimization_level=3, backend=backend)\n",
    "target_circuit = pm.run(abstract_circuit)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6bc45489-56d0-4f46-8659-9df4d1555516",
   "metadata": {},
   "source": [
    "## Manage data across your program\n",
    "\n",
    "Qiskit Serverless allows you to manage files in the `/data` directory across all your programs. This includes several limitations:\n",
    "\n",
    "- Only `tar` and `h5` files are supported today\n",
    "- This is only a flat `/data` storage, and cannot have `/data/folder/` subdirectories\n",
    "\n",
    "The following shows how to upload files. Be sure you have authenticated to Qiskit Serverless with your [IBM Quantum account](https://quantum.ibm.com/account) (see [Deploy to IBM Quantum Platform](./serverless-first-program#deploy-to-ibm-quantum-platform) for instructions)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "0183278f-8ce3-4466-9255-097b2d211052",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'{\"message\":\"/usr/src/app/media/5f37582aa306c50013fac285/transpile_demo.tar\"}'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import tarfile\n",
    "\n",
    "# Create a tar\n",
    "filename = \"transpile_demo.tar\"\n",
    "file = tarfile.open(filename,\"w\")\n",
    "file.add(\"source_files/transpile_remote.py\")\n",
    "file.close()\n",
    "\n",
    "# Upload the tar to Serverless data directory\n",
    "serverless.file_upload(filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f762470-945f-48d5-a65b-c60d3b2dae3f",
   "metadata": {},
   "source": [
    "Next, you can list all the files in your `data` directory. This data is accessible to all programs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "14241fc4-d0cb-4803-8752-a460e1f48708",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['transpile_demo.tar']"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "serverless.files()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a97bd83e-8250-43bb-b1c4-d40d822c7ba2",
   "metadata": {},
   "source": [
    "This can be done from a program by using `file_download()` to download the file to the program environment, and uncompressing the `tar`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "ef649b2a-ed95-4dd2-89d9-61438faa7c1e",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 10.2k/10.2k [00:00<00:00, 10.0MiB/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "downloaded_3d3c4bf6_transpile_demo.tar\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "# transpile_parallel.py\n",
    "\n",
    "import tarfile\n",
    "\n",
    "files = serverless.files()\n",
    "demo_file = files[0]\n",
    "downloaded_tar = serverless.file_download(demo_file)\n",
    "\n",
    "print(downloaded_tar)\n",
    "with tarfile.open(downloaded_tar, 'r') as tar:\n",
    "    tar.extractall()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b93dbdb-2060-468b-8496-ba98142a780b",
   "metadata": {},
   "source": [
    "At this point, your program can interact with the files, as you would a local experiment. `file_upload()` , `file_download()`, and `file_delete()` can be called from your local experiment, or your uploaded program, for consistent and flexible data management."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a004dd78-0e0a-4a3b-83cb-333469533ef6",
   "metadata": {},
   "source": [
    "## Next steps\n",
    "\n",
    "<Admonition type=\"info\" title=\"Recommendations\">\n",
    "\n",
    "- See a full example that [ports existing code to Qiskit Serverless ](./serverless-port-code).\n",
    "- Read a paper in which researchers used Qiskit Serverless and quantum-centric supercomputing to [explore quantum chemistry](https://arxiv.org/abs/2405.05068v1).\n",
    "\n",
    "</Admonition>"
   ]
  }
 ],
 "metadata": {
  "description": "Manage compute and data across your Qiskit pattern with Qiskit Serverless.",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3"
  },
  "title": "Manage Qiskit Serverless compute and data resources"
 },
 "nbformat": 4,
 "nbformat_minor": 2
}