chore: start the bones of the project (#5)

Signed-off-by: Joan Fontanals Martinez <joan.martinez@jina.ai>

.github/CLA.md

@@ -0,0 +1,111 @@
+# Contributor License Agreement
+
+Thank you for your interest in contributing to software projects 
+managed by Jina AI Limited ("JINA", "We" or "Us"). In order to 
+clarify the intellectual property license granted with Contributions 
+from any person or entity, JINA
+must have a Contributor License Agreement ("CLA") on file that has
+been signed by each Contributor, indicating agreement to the license
+terms below. This license is for your protection as a Contributor as
+well as the protection of JINA and its users; it does not
+change your rights to use your own Contributions for any other purpose.
+
+
+You accept and agree to the following terms and conditions for Your
+present and future Contributions submitted to JINA.
+
+
+1. Definitions.
+ "You" (or "Your") shall mean the copyright owner or legal entity
+ authorized by the copyright owner that is making this Agreement
+ with JINA. For legal entities, the entity making a
+ Contribution and all other entities that control, are controlled
+ by, or are under common control with that entity are considered to
+ be a single Contributor. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+ "Contribution" shall mean any original work of authorship,
+ including any modifications or additions to an existing work, that
+ is intentionally submitted by You to JINA for inclusion
+ in, or documentation of, any of the products owned or managed by
+ JINA (the "Work"). For the purposes of this definition,
+ "submitted" means any form of electronic, verbal, or written
+ communication sent to JINA or its representatives,
+ including but not limited to communication on electronic mailing
+ lists, source code control systems, and issue tracking systems that
+ are managed by, or on behalf of, JINA for the purpose of
+ discussing and improving the Work, but excluding communication that
+ is conspicuously marked or otherwise designated in writing by You
+ as "Not a Contribution."
+ 
+ 
+2. Grant of Copyright License. Subject to the terms and conditions of
+ this Agreement, You hereby grant to JINA and to
+ recipients of software distributed by JINA a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare derivative works of,
+ publicly display, publicly perform, sublicense, and distribute Your
+ Contributions and such derivative works.
+ 
+ 
+3. Grant of Patent License. Subject to the terms and conditions of
+ this Agreement, You hereby grant to JINA and to
+ recipients of software distributed by JINA a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have
+ made, use, offer to sell, sell, import, and otherwise transfer the
+ Work, where such license applies only to those patent claims
+ licensable by You that are necessarily infringed by Your
+ Contribution(s) alone or by combination of Your Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If any
+ entity institutes patent litigation against You or any other entity
+ (including a cross-claim or counterclaim in a lawsuit) alleging
+ that your Contribution, or the Work to which you have contributed,
+ constitutes direct or contributory patent infringement, then any
+ patent licenses granted to that entity under this Agreement for
+ that Contribution or Work shall terminate as of the date such
+ litigation is filed.
+ 
+ 
+4. You represent that you are legally entitled to grant the above
+ license. If your employer(s) has rights to intellectual property
+ that you create that includes your Contributions, you represent
+ that you have received permission to make Contributions on behalf
+ of that employer, that your employer has waived such rights for
+ your Contributions to JINA, or that your employer has
+ executed a separate Corporate CLA with JINA.
+ 
+ 
+5. You represent that each of Your Contributions is Your original
+ creation (see section 7 for submissions on behalf of others). You
+ represent that Your Contribution submissions include complete
+ details of any third-party license or other restriction (including,
+ but not limited to, related patents and trademarks) of which you
+ are personally aware and which are associated with any part of Your
+ Contributions.
+ 
+ 
+6. You are not expected to provide support for Your Contributions,
+ except to the extent You desire to provide support. You may provide
+ support for free, for a fee, or not at all. Unless required by
+ applicable law or agreed to in writing, You provide Your
+ Contributions on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS
+ OF ANY KIND, either express or implied, including, without
+ limitation, any warranties or conditions of TITLE, NON-
+ INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE.
+ 
+ 
+7. Should You wish to submit work that is not Your original creation,
+ You may submit it to JINA separately from any
+ Contribution, identifying the complete details of its source and of
+ any license or other restriction (including, but not limited to,
+ related patents, trademarks, and license agreements) of which you
+ are personally aware, and conspicuously marking the work as
+ "Submitted on behalf of a third-party: [named here]".
+ 
+ 
+8. You agree to notify JINA of any facts or circumstances of
+ which you become aware that would make these representations
+ inaccurate in any respect.

.github/CODE_OF_CONDUCT.md

@@ -0,0 +1,78 @@
+
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to make participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, sex characteristics, gender identity and expression,
+level of experience, education, socio-economic status, nationality, personal
+appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+  advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies within all project spaces, and it also applies when
+an individual is representing the project or its community in public spaces.
+Examples of representing a project or community include using an official
+project e-mail address, posting via an official social media account, or acting
+as an appointed representative at an online or offline event. Representation of
+a project may be further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at `hello@jina.ai`. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see
+https://www.contributor-covenant.org/faq
+

.github/CODE_REVIEW_GUIDELINES.md

@@ -0,0 +1,82 @@
+# Code Review Guidelines  
+
+## PyGuide  
+
+- [Google guide](https://google.github.io/styleguide/pyguide.html)
+- [Antipatterns](https://docs.quantifiedcode.com/python-anti-patterns/index.html)
+- [Zen of python](https://zen-of-python.info/)
+- [Code review for humans](https://phauer.com/2018/code-review-guidelines/)
+- [Code review culture](https://www.giladpeleg.com/blog/better-code-review/)
+
+## General PR process  
+
+- The scope of PR should be simple, unique and well-defined. PR should not contain unrelated changes
+- Approve PR only if you are sure about the scope
+- Be respectful and reply asap
+- Avoid spending too much time on trivial changes
+- Avoid premature optimisation
+- For community PR, check CLA by typing `recheckcla`
+- Ask for jina env if needed by `jina -vf`
+- Suggest changes and check back ASAP. Get PR merged soon
+- Suggest adding appropriate documentation for new features
+- Don't forget to praise when PR is ready with something like LGTM
+
+## TLDR for pyguide  
+
+- Variables names need to be informative. No k, v or x.
+- Use standard import [order](https://stackoverflow.com/questions/20762662/whats-the-correct-way-to-sort-python-import-x-and-from-x-import-y-statement)
+- Define global variables in CAPS
+- Use appropriate underscore for variable naming (leading, lagging, single, double)
+- Add docstrings
+- Add type hints
+- Use codetags like [#TODO](https://www.python.org/dev/peps/pep-0350/#mnemonics) in code wherever needed
+- Raise appropriate exceptions
+- Don't rename functions & arguments exposed to the user unless necessary. Have appropriate depreciation strategy
+
+## Jina specific  
+
+- Use this to create random ports for servers
+
+    `from jina.helper import random_port`
+
+- You can keep function specific imports inside functions
+- Jina ENV variables should start with **JINA_**
+- For type hints which require import, you can do it like this
+
+```python
+if False:
+    from PIL import Image
+
+def func(arr: 'Image'):
+    from PIL import Image
+    ...
+```
+
+## Adding tests  
+
+- Add new unit & integration test if you add a new feature
+- Use pytest tmpdir fixture for temporary directories
+- Check test locally before pushing
+- If a `Flow()` needs to be built in the test and is not a test of the Flow module, the test should go under `/tests/integration`
+
+## Core  
+
+**Single Responsibility Principle**  
+
+Do not place more than one responsibility into a single class or function. Instead, refactor into separate classes and functions.
+
+**Open Closed Principle**  
+
+While adding new functionality, existing code should not be modified. New functionality should be written in new classes and functions.
+
+**Liskov substitutability principle**  
+
+The child class should not change the behavior (meaning) of the parent class. The child class can be used as a substitute for a base class.
+
+**Interface segregation**  
+
+Do not create lengthy interfaces. Instead, split them into smaller interfaces based on the functionality. The interface should not contain any dependencies (parameters) which are not required for the expected functionality.
+
+**Dependency Injection**  
+
+Inject dependencies instead of hard-coding them.

.github/codecov.yml

@@ -0,0 +1,21 @@
+codecov:
+  # https://docs.codecov.io/docs/comparing-commits
+  allow_coverage_offsets: true
+ignore:
+  - "jina/resources"
+  - "jina/serve/instrumentation/_aio_client.py"
+  - "jina/serve/instrumentation/_aio_server.py"
+coverage:
+  status:
+    project:
+      default:
+        informational: true
+        target: auto  # auto compares coverage to the previous base commit
+        flags:
+          - jina
+  comment:
+    layout: "reach, diff, flags, files"
+    behavior: default
+    require_changes: false  # if true: only post the comment if coverage changes
+    branches:               # branch names that can post comment
+      - "master"

.github/labeler.yml

@@ -0,0 +1,105 @@
+# Add 'label1' to any changes within 'example' folder or any subfolders
+area/docs:
+  - docs/**/*
+  - ./*.md
+
+area/testing:
+  - tests/**/*
+
+area/setup:
+  - setup.py
+  - extra-requirements.txt
+  - requirements.txt
+  - MANIFEST.in
+  - fastentrypoints.py
+
+area/core:
+  - jina/**/*
+
+area/helper:
+  - jina/*.py
+
+area/entrypoint:
+  - jina/__init__.py
+
+
+area/network:
+  - jina/peapods/**/*
+  - jina/proto/**/*
+
+area/housekeeping:
+  - .github/**/*
+  - ./.gitignore
+
+area/cicd:
+  - .github/workflows/**/*
+
+area/docker:
+  - Dockerfiles/**/*
+  - ./.dockerignore
+
+area/cli:
+  - jina_cli/**/*
+
+area/docarray:
+  - docarray/**/*
+
+area/script:
+  - script/**/*
+
+component/client:
+  - jina/clients/**/*
+
+component/driver:
+  - jina/drivers/**/*
+
+component/executor:
+  - jina/executors/**/*
+
+component/docker:
+  - jina/docker/**/*
+
+executor/meta:
+  - jina/executors/*.py
+
+executor/encoder:
+  - jina/executors/encoders/**/*
+
+executor/indexer:
+  - jina/executors/indexers/**/*
+
+executor/ranker:
+  - jina/executors/rankers/**/*
+
+executor/crafter:
+  - jina/executors/crafters/**/*
+
+component/flow:
+  - jina/flow/**/*
+
+component/logging:
+  - jina/logging/**/*
+
+component/entrypoint:
+  - jina/main/**/*
+
+component/peapod:
+  - jina/peapods/**/*
+
+component/proto:
+  - jina/proto/**/*
+
+component/type:
+  - jina/types/**/*
+
+component/math:
+  - jina/math/**/*
+
+component/resource:
+  - jina/resources/**/*
+
+component/hub:
+  - jina/hub/**/*
+
+component/jaml:
+  - jina/jaml/**/*

.github/pull_request_template.md

@@ -0,0 +1,14 @@
+<!---Decomposing the complex issue into subtasks can help you build it step-by-step. Thanks for your pull request! :rocket: --->
+<!---We know that dev life is hectic, but **please provide a (brief) description** of what your PR does, and how it does it. **Otherwise, your PR cannot be reviewed!** --->
+<!---This policy was agreed upon in a past company retro, and makes everyone's life a little easier. Thanks for your collaboration!--->
+
+**Goals:**
+<!---https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword--->
+
+- resolves #ISSUE-NUMBER
+- ...
+- ...
+
+- ...
+- ...
+- [ ] check and update documentation. See [guide](https://github.com/jina-ai/jina/blob/master/CONTRIBUTING.md#-contributing-documentation) and ask the team.

.github/release-template.ejs

@@ -0,0 +1,173 @@
+<% var groupCommits = [
+{
+    name: 'breaking',
+    show: true,
+    list: []
+}, {
+    name: 'feat',
+    show: true,
+    list: []
+}, {
+    name: 'perf',
+    show: true,
+    list: []
+}, {
+    name: 'fix',
+    show: true,
+    list: []
+}, {
+    name: 'refactor',
+    show: true,
+    list: []
+}, {
+    name: 'docs',
+    show: true,
+    list: []
+},  {
+    name: 'test',
+    show: true,
+    list: []
+}, {
+    name: 'other',
+    show: true,
+    list: []
+}
+]
+
+var all_titles = {};
+var all_commiters = {};
+var commitHref = "https://github.com/jina-ai/jina/commit/"
+
+commits.forEach(function (commit) {
+
+    var result = (commit.title).match(/^(\w*)(\((.*)\))?\: (.*)$/);
+
+    var type = result && result[1];
+    var scope = result && result[3];
+    var title = result && result[4];
+    var committer = commit.authorName
+
+    if (!(committer in all_commiters)) {
+        all_commiters[committer] = 1
+    }
+
+    if (!(title in all_titles)) {
+        all_titles[title] = 1
+        if( title != null && (title.indexOf('💥')>-1 || title.indexOf(':boom:')>-1) ){
+            groupCommits.find(item => item.name === 'breaking').list.push({
+                type: type,
+                scope: scope,
+                title: title,
+                commit: commit
+            })
+        } else if(type == 'fix' || type == 'fixed'){
+            groupCommits.find(item => item.name === 'fix').list.push({
+                type: type,
+                scope: scope,
+                title: title,
+                commit: commit
+            })
+        } else if(type == 'perf' || type == 'performance'){
+            groupCommits.find(item => item.name === 'perf').list.push({
+                type: type,
+                scope: scope,
+                title: title,
+                commit: commit
+            })
+        } else if(type == 'feat' || type == 'feature'){
+            groupCommits.find(item => item.name === 'feat').list.push({
+                type: type,
+                scope: scope,
+                title: title,
+                commit: commit
+            })
+        } else if(type == 'refactor'){
+            groupCommits.find(item => item.name === 'refactor').list.push({
+                type: type,
+                scope: scope,
+                title: title,
+                commit: commit
+            })
+        } else if(type == 'docs' || type == 'doc'){
+            groupCommits.find(item => item.name === 'docs').list.push({
+                type: type,
+                scope: scope,
+                title: title,
+                commit: commit
+            })
+        } else if(type == 'test' || type == 'tests' || type == 'ci'){
+            groupCommits.find(item => item.name === 'test').list.push({
+                type: type,
+                scope: scope,
+                title: title,
+                commit: commit
+            })
+        } else {
+            groupCommits.find(item => item.name === 'other').list.push({
+                type: type,
+                scope: scope,
+                title: title,
+                commit: commit
+            })
+        }
+    }
+
+
+});
+
+
+var listCommits = function(list, key){
+
+list.forEach(function (ct) {
+
+    var type = ct.type;
+    var scope = ct.scope;
+    var title = '';
+    var commit = ct.commit;
+
+    if(type){
+        if(key != 'other'){
+            title = (scope? '__'+scope+'__: ':'') + ct.title;
+        }else{
+            title = '__' + type + (scope? '('+scope+')':'') + '__ : ' + ct.title;
+        }
+    }else{
+        title = commit.title;
+    }
+%> - <% if(typeof commitHref === 'undefined' || commitHref === '') { %>[```<%=commit.sha1.slice(0,8)%>```]<% } else { %>[[```<%=commit.sha1.slice(0,8)%>```](<%=commitHref%><%=commit.sha1%>)]<%}%> __-__ <%=title%> (*<%= commit.authorName %>*)
+<% })} %>
+
+🙇 We'd like to thank all contributors for this new release! In particular,
+<%  Object.keys(all_commiters).forEach(function (key) { %> <%= key %>, <% }) %> 🙇
+
+<%
+        for(var i of groupCommits){
+    if(i.list.length == 0) continue;
+
+if (i.name === 'breaking' && i.show) { %>
+### 💥 Breaking changes
+
+<%	} else if (i.name === 'fix' && i.show) { %>
+### 🐞 Bug fixes
+
+<%	} else if( i.name === 'feat' && i.show) { %>
+### 🆕 New Features
+
+<%	} else if(i.name === 'perf' && i.show) { %>
+### ⚡ Performance Improvements
+
+<%	} else if(i.name === 'refactor' && i.show) { %>
+### 🧼 Code Refactoring
+
+<%	} else if(i.name === 'docs' && i.show) { %>
+### 📗 Documentation
+
+<%	} else if(i.name === 'test' && i.show) { %>
+### 🏁 Unit Test and CICD
+
+<%	} else if (i.name === 'other' && i.show) { %>
+### 🍹 Other Improvements
+
+<%	}
+    i.show && listCommits(i.list, i);
+} %>

.github/slack-pypi.json

@@ -0,0 +1,51 @@
+{
+  "blocks": [
+    {
+      "type": "header",
+      "text": {
+        "type": "plain_text",
+        "text": ":tada: Jina $RELEASE_VER is just released on PyPi!",
+        "emoji": true
+      }
+    },
+    {
+      "type": "section",
+      "text": {
+        "type": "mrkdwn",
+        "text": "You can now use `pip install -U jina` to upgrade."
+      },
+      "accessory": {
+        "type": "image",
+        "image_url": "https://docs.jina.ai/_static/favicon.png",
+        "alt_text": "cute cat"
+      }
+    },
+    {
+      "type": "context",
+      "elements": [
+        {
+          "type": "mrkdwn",
+          "text": "<https://pypi.org/project/jina/|:package: Check out the PyPi release>"
+        }
+      ]
+    },
+    {
+      "type": "context",
+      "elements": [
+        {
+          "type": "mrkdwn",
+          "text": "<https://github.com/jina-ai/jina/blob/master/CHANGELOG.md|:notebook_with_decorative_cover: Read the change log>"
+        }
+      ]
+    },
+    {
+      "type": "context",
+      "elements": [
+        {
+          "type": "mrkdwn",
+          "text": "<https://get.jina.ai|:star: Give Jina a star>"
+        }
+      ]
+    }
+  ]
+}

.github/workflows/cd.yml

@@ -0,0 +1,27 @@
+name: CD
+
+on:
+  push:
+    branches:
+      - main
+
+jobs:
+  prerelease:
+    if: |
+      !startsWith(github.event.head_commit.message, 'chore') &&
+      !startsWith(github.event.head_commit.message, 'build: hotfix') &&
+      !endsWith(github.event.head_commit.message, 'reformatted by jina-dev-bot')
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+        with:
+          fetch-depth: 100
+      - name: Pre-release (.devN)
+        run: |
+          git fetch --depth=1 origin +refs/tags/*:refs/tags/*
+          pip install twine wheel
+          ./scripts/release.sh
+        env:
+          TWINE_USERNAME: ${{ secrets.TWINE_USERNAME }}
+          TWINE_PASSWORD: ${{ secrets.TWINE_PASSWORD }}
+          JINA_SLACK_WEBHOOK: ${{ secrets.JINA_SLACK_WEBHOOK }}

.github/workflows/ci-no-code-related.yml

@@ -0,0 +1,16 @@
+name: CI for Code Un-related Changes
+
+on:
+  pull_request:
+    paths:
+      - 'docs/**'
+      - 'README.md'
+
+jobs:
+  commit-lint:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2.5.0
+        with:
+          fetch-depth: 0
+      - uses: wagoid/commitlint-github-action@v4

.github/workflows/ci.yml

@@ -0,0 +1,150 @@
+name: CI
+
+on:
+  pull_request:
+    paths-ignore:
+      - 'docs/**'
+      - 'README.md'
+
+jobs:
+  commit-lint:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2.5.0
+        with:
+          fetch-depth: 0
+      - uses: wagoid/commitlint-github-action@v4
+
+  lint-flake-8:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2.5.0
+      - name: Set up Python 3.9
+        uses: actions/setup-python@v4
+        with:
+          python-version: 3.9
+      - name: Lint with flake8
+        run: |
+          pip install flake8
+          # stop the build if there are Python syntax errors or undefined names
+          flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics --exclude .git,__pycache__,docs/source/conf.py,old,build,dist,tests/
+          # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
+          flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics --exclude .git,__pycache__,docs/source/conf.py,old,build,dist,tests/
+
+  check-black:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2.5.0
+        with:
+          fetch-depth: 0
+      - name: Set up Python 3.9
+        uses: actions/setup-python@v4
+        with:
+          python-version: 3.9
+      - id: file_changes
+        uses: Ana06/get-changed-files@v1.2
+      - name: check black
+        run: ./scripts/black.sh
+        env:
+          CHANGED_FILES: ${{ steps.file_changes.outputs.added_modified }}
+
+  prep-testbed:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+        with:
+          fetch-depth: 0
+      - name: Install Dependency
+        run: |
+          sudo apt-get install jq
+      - id: set-matrix-unit
+        run: |
+          echo "::set-output name=matrix::$(bash scripts/get-all-test-paths.sh unit 1)"
+      - id: set-matrix-integration
+        run: |
+          echo "::set-output name=matrix::$(bash scripts/get-all-test-paths.sh integration_local 1)"
+    outputs:
+      matrix-unit: ${{ steps.set-matrix-unit.outputs.matrix }}
+      matrix-integration: ${{ steps.set-matrix-integration.outputs.matrix }}
+
+  unit-tests:
+    needs: prep-testbed
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: [3.9]
+        test-path: ${{fromJson(needs.prep-testbed.outputs.matrix-unit)}}
+    steps:
+      - uses: actions/checkout@v2
+        with:
+          fetch-depth: 0
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Prepare environment
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install wheel
+          pip install -r requirements.txt
+          pip install --no-cache-dir ".[test]"
+          sudo apt-get install libsndfile1
+      - name: Test
+        id: test
+        run: |
+          pytest -v -s --log-cli-level=DEBUG ${{ matrix.test-path }}
+        timeout-minutes: 30
+
+  integration-tests:
+    needs: prep-testbed
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: [3.9]
+        test-path: ${{fromJson(needs.prep-testbed.outputs.matrix-integration)}}
+    steps:
+      - uses: actions/checkout@v2
+        with:
+          fetch-depth: 0
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Prepare environment
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install wheel
+          pip install -r requirements.txt
+          pip install --no-cache-dir ".[test]"
+          sudo apt-get install libsndfile1
+      - name: Setup monitoring stack
+        run: |
+          cd $GITHUB_WORKSPACE
+          docker-compose -f tests/integration/docker-compose.yml  --project-directory . up  --build -d --remove-orphans
+      - name: Test
+        id: test
+        run: |
+          pytest -v -s --log-cli-level=DEBUG ${{ matrix.test-path }}
+        timeout-minutes: 30
+      - name: Cleanup monitoring stack
+        run: |
+          cd $GITHUB_WORKSPACE
+          docker-compose -f tests/integration/docker-compose.yml  --project-directory . down
+
+  # just for blocking the merge until all parallel integration-tests are successful
+  success-all-test:
+    needs:
+      - unit-tests
+      - integration-tests
+    if: always()
+    runs-on: ubuntu-latest
+    steps:
+      - uses: technote-space/workflow-conclusion-action@v2
+      - name: Check Failure
+        if: env.WORKFLOW_CONCLUSION == 'failure'
+        run: exit 1
+      - name: Success
+        if: ${{ success() }}
+        run: echo "All Done"

.github/workflows/cla.yml

@@ -0,0 +1,25 @@
+name: CLA
+
+on:
+  push:
+    branches-ignore:
+      - '**'  # temporally ignore all
+
+jobs:
+  cla-check:
+    runs-on: ubuntu-latest
+    steps:
+      - name: "CLA Assistant"
+        if: (github.event.comment.body == 'recheckcla' || github.event.comment.body == 'I have read the CLA Document and I hereby sign the CLA') || github.event_name == 'pull_request_target'
+        # Alpha Release
+        uses: cla-assistant/github-action@v2.1.3-beta
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          # the below token should have repo scope and must be manually added by you in the repository's secret
+          PERSONAL_ACCESS_TOKEN: ${{ secrets.CLA_TOKEN }}
+        with:
+          path-to-signatures: '.github/signatures/v1/cla.json'
+          path-to-document: 'https://github.com/jina-ai/jina/blob/master/.github/CLA.md'
+          branch: 'cla'
+          allowlist: jina-bot
+          signed-commit-message: '$contributorName has signed the CLA in #$pullRequestNo'

.github/workflows/commit-lint.yml

@@ -0,0 +1,53 @@
+# commit-lint needs "pull_request_target" access,
+# therefore separate from main ci.yml
+name: PR
+
+on:
+  push:
+    branches-ignore:
+      - '**'  # temporally ignore all, as https://github.com/wagoid/commitlint-github-action/issues/66
+
+jobs:
+  commit-lint-check:
+    runs-on: ubuntu-latest
+    steps:
+      - name: find the prev warning if exist
+        uses: peter-evans/find-comment@v1
+        id: fc
+        with:
+          issue-number: ${{ github.event.pull_request.number }}
+          comment-author: 'github-actions[bot]'
+          body-includes: 'bad commit message'
+      - name: Delete comment if exist
+        if: ${{ steps.fc.outputs.comment-id != 0 }}
+        uses: actions/github-script@v3
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          script: |
+            github.issues.deleteComment({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              comment_id: ${{ steps.fc.outputs.comment-id }},
+            })
+      - uses: actions/checkout@v2.5.0
+        with:
+          fetch-depth: 0
+      - run: "echo \"module.exports = {extends: ['@commitlint/config-conventional']}\" > commitlint.config.js"
+      - uses: wagoid/commitlint-github-action@v1
+        env:
+          GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
+      - name: if lint failed
+        if: ${{ failure() }}
+        uses: peter-evans/create-or-update-comment@v1
+        with:
+          issue-number: ${{ github.event.pull_request.number }}
+          body: |
+            Thanks for your contribution :heart:
+            :broken_heart: Unfortunately, this PR has one ore more **bad commit messages**, it can not be merged. To fix this problem, please refer to:
+            - [Commit Message Guideline for the First Time Contributor](https://github.com/jina-ai/jina/issues/553)
+            - [Contributing Guideline](https://github.com/jina-ai/jina/blob/master/CONTRIBUTING.md)
+
+            Note, other CI tests will *not* *start* until the commit messages get fixed.
+
+            This message will be deleted automatically when the commit messages get fixed.
+          reaction-type: "eyes"

.github/workflows/force-docker-build.yml

@@ -0,0 +1,167 @@
+name: Manual Docker Build
+
+on:
+  workflow_dispatch:
+    inputs:
+      release_token:
+        description: 'Your release token'
+        required: true
+      triggered_by:
+        description: 'CD | TAG | MANUAL'
+        required: false
+        default: MANUAL
+
+jobs:
+  token-check:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check release token
+        id: token-check
+        run: | 
+          touch SUCCESS
+        if: inputs.release_token == env.release_token
+        env:
+          release_token: ${{ secrets.JINA_CORE_RELEASE_TOKEN }}
+      - name: Fail release token
+        run: |
+          [[ -f SUCCESS ]]
+
+
+  regular-release:
+    needs: token-check
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        pip_tag: [ "", "perf", "standard", "devel"]  # default: "" = core
+        py_version: [ "3.7", "3.8", "3.9" , "3.10", "3.11"]  # default "" = 3.7
+    steps:
+      - uses: actions/checkout@v2.5.0
+        with:
+          fetch-depth: 100
+      - name: Set envs and versions
+        run: |
+          DEFAULT_PY_VERSION="3.8"
+          VCS_REF=${{ github.ref }}
+          echo "VCS_REF=$VCS_REF" >> $GITHUB_ENV
+          echo "Will build $VCS_REF"
+          echo "BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_ENV
+          echo "BUILD_TARGET=jina" >> $GITHUB_ENV
+
+          if [[ "${{ matrix.pip_tag }}" == "perf" ]]; then
+            echo "JINA_PIP_INSTALL_PERF=1" >> $GITHUB_ENV
+          fi
+
+          if [[ "${{ matrix.pip_tag }}" == "" ]]; then
+            echo "JINA_PIP_INSTALL_CORE=1" >> $GITHUB_ENV
+          fi
+
+          JINA_VERSION=$(sed -n '/^__version__/p' ./jina/__init__.py | cut -d \' -f2)
+          V_JINA_VERSION=v${JINA_VERSION}
+          JINA_MINOR_VERSION=${JINA_VERSION%.*}
+          JINA_MAJOR_VERSION=${JINA_MINOR_VERSION%.*}
+
+          PY_TAG=${{matrix.py_version}}
+          if [ -n "${PY_TAG}" ]; then
+            PY_TAG=-py${PY_TAG//./}
+          fi
+
+          PIP_TAG=${{ matrix.pip_tag }}
+          if [ -n "${PIP_TAG}" ]; then
+              PIP_TAG=-${PIP_TAG}
+          fi
+          
+          git fetch --depth=1 origin +refs/tags/*:refs/tags/*
+          LAST_VER_TAG=$(git tag -l | sort -V | tail -n1)
+          PRE_VERSION=-dev$(git rev-list $LAST_VER_TAG..HEAD --count)
+
+          if [[ "${{ github.event.inputs.triggered_by }}" == "CD" ]]; then
+
+            if [[ "${{ matrix.py_version }}" == "$DEFAULT_PY_VERSION" ]]; then
+              echo "TAG_ALIAS=\
+                              jinaai/jina:master${PY_TAG}${PIP_TAG}, \
+                              jinaai/jina:master${PIP_TAG}, \
+                              jinaai/jina:${JINA_VERSION}${PRE_VERSION}${PIP_TAG}, \
+                              jinaai/jina:${JINA_VERSION}${PRE_VERSION}${PY_TAG}${PIP_TAG}" \
+                              >> $GITHUB_ENV
+            else
+              # on every CD
+              echo "TAG_ALIAS=\
+                              jinaai/jina:master${PY_TAG}${PIP_TAG}, \
+                              jinaai/jina:${JINA_VERSION}${PRE_VERSION}${PY_TAG}${PIP_TAG}" \
+                              >> $GITHUB_ENV
+            fi
+
+          elif [[ "${{ github.event.inputs.triggered_by }}" == "TAG" ]]; then
+            # on every tag release
+
+            if [[ "${{ matrix.py_version }}" == "$DEFAULT_PY_VERSION" ]]; then
+              echo "TAG_ALIAS=\
+                              jinaai/jina:latest${PY_TAG}${PIP_TAG}, \
+                              jinaai/jina:${JINA_VERSION}${PY_TAG}${PIP_TAG}, \
+                              jinaai/jina:${JINA_MINOR_VERSION}${PY_TAG}${PIP_TAG}, \
+                              jinaai/jina:${JINA_MAJOR_VERSION}${PY_TAG}${PIP_TAG}, \
+                              jinaai/jina:latest${PIP_TAG}, \
+                              jinaai/jina:${JINA_VERSION}${PIP_TAG}, \
+                              jinaai/jina:${JINA_MINOR_VERSION}${PIP_TAG}, \
+                              jinaai/jina:${JINA_MAJOR_VERSION}${PIP_TAG} \
+                              " >> $GITHUB_ENV
+            else
+              echo "TAG_ALIAS=\
+                              jinaai/jina:latest${PY_TAG}${PIP_TAG}, \
+                              jinaai/jina:${JINA_VERSION}${PY_TAG}${PIP_TAG}, \
+                              jinaai/jina:${JINA_MINOR_VERSION}${PY_TAG}${PIP_TAG}, \
+                              jinaai/jina:${JINA_MAJOR_VERSION}${PY_TAG}${PIP_TAG} \
+                              " >> $GITHUB_ENV
+            fi
+          elif [[ "${{ github.event.inputs.triggered_by }}" == "MANUAL" ]]; then
+            # on every manual release
+            if [[ "${{ matrix.py_version }}" == "$DEFAULT_PY_VERSION" ]]; then
+              echo "TAG_ALIAS=\
+                              jinaai/jina:${JINA_VERSION}${PIP_TAG}, \
+                              jinaai/jina:${JINA_VERSION}${PY_TAG}${PIP_TAG} \
+                              " >> $GITHUB_ENV
+            else
+              echo "TAG_ALIAS=\
+                              jinaai/jina:${JINA_VERSION}${PY_TAG}${PIP_TAG} \
+                              " >> $GITHUB_ENV
+            fi
+          else
+            echo "Bad triggered_by: ${{ github.event.inputs.triggered_by }}!"
+            exit 1
+          fi
+
+          echo "JINA_VERSION=${JINA_VERSION}" >> $GITHUB_ENV
+
+      - name: Set up Docker Buildx
+        id: buildx
+        uses: docker/setup-buildx-action@v1
+        with:
+          install: true
+      - name: Login to DockerHub
+        uses: docker/login-action@v1
+        with:
+          username: ${{ secrets.DOCKERHUB_DEVBOT_USER }}
+          password: ${{ secrets.DOCKERHUB_DEVBOT_TOKEN }}
+      - run: |
+          # https://github.com/docker/buildx/issues/464#issuecomment-741507760
+          # https://github.com/kubernetes-sigs/azuredisk-csi-driver/pull/808/files
+          docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-aarch64
+          docker run --rm --privileged tonistiigi/binfmt --install all
+      - name: Build and push
+        uses: docker/build-push-action@v2
+        with:
+          context: .
+          file: Dockerfiles/debianx.Dockerfile
+          platforms: linux/amd64,linux/arm64
+          push: true
+          tags: ${{env.TAG_ALIAS}}
+          build-args: |
+            BUILD_DATE=${{env.BUILD_DATE}}
+            JINA_VERSION=${{env.JINA_VERSION}}
+            VCS_REF=${{env.VCS_REF}}
+            PIP_INSTALL_CORE=${{env.JINA_PIP_INSTALL_CORE}}
+            PIP_INSTALL_PERF=${{env.JINA_PIP_INSTALL_PERF}}
+            PY_VERSION=${{matrix.py_version}}
+            PIP_TAG=${{matrix.pip_tag}}
+          target: ${{env.BUILD_TARGET}}

.github/workflows/force-docs-build.yml

@@ -0,0 +1,113 @@
+name: Manual Docs Build
+
+on:
+  workflow_dispatch:
+    inputs:
+      release_token:
+        description: 'Your release token'
+        required: true
+      triggered_by:
+        description: 'CD | TAG | MANUAL'
+        required: false
+        default: MANUAL
+      build_old_docs:
+        description: 'Whether to build old docs (TRUE | FALSE)'
+        type: string
+        default: 'FALSE'
+      package:
+        description: The name of the repo to build documentation for.
+        type: string
+        default: jina
+      repo_owner:
+        description: The owner of the repo to build documentation for. Defaults to 'jina-ai'.
+        type: string
+        default: jina-ai
+      pages_branch:
+        description: Branch that Github Pages observes
+        type: string
+        default: gh-pages
+      git_config_name:
+        type: string
+        default: Jina Dev Bot
+      git_config_email:
+        type: string
+        default: dev-bot@jina.ai
+
+jobs:
+  token-check:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check release token
+        id: token-check
+        run: | 
+          touch SUCCESS
+        if: inputs.release_token == env.release_token
+        env:
+          release_token: ${{ secrets.JINA_CORE_RELEASE_TOKEN }}
+      - name: Fail release token
+        run: |
+          [[ -f SUCCESS ]]
+
+  build-and-push-latest-docs:
+    needs: token-check
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 1
+      - uses: actions/setup-python@v4
+        with:
+          python-version: '3.7'
+      - name: Install Dependencies
+        run: |
+          pip install .[devel]
+          cd docs
+          pip install -r requirements.txt
+          pip install --pre -U furo
+          pip install sphinx-markdown-tables==0.0.17
+      - name: Sphinx Build
+        run: |
+          cd docs
+          bash makedoc.sh local-only
+          mv ./_build/dirhtml /tmp/gen-html
+          cd ..
+      - name: Checkout to GH pages branch (${{ inputs.pages_branch }})
+        run: |
+          git fetch origin ${{ inputs.pages_branch }}:${{ inputs.pages_branch }} --depth 1
+          git checkout -f ${{ inputs.pages_branch }}
+          git reset --hard HEAD
+      - name: Small config stuff
+        run: |
+          touch /tmp/gen-html/.nojekyll
+          cp ./docs/_versions.json /tmp/gen-html/_versions.json
+          cp ./docs/CNAME /tmp/gen-html/CNAME
+          cp /tmp/gen-html/404/index.html /tmp/gen-html/404.html
+          sed -i 's/href="\.\./href="/' /tmp/gen-html/404.html # fix asset urls that needs to be updated in 404.html
+      - name: Moving old doc versions
+        run: |
+          cd docs
+          for i in $(cat _versions.json | jq '.[].version' | tr -d '"'); do if [ -d "$i" ]; then mv "$i" /tmp/gen-html; fi; done
+      - name: Swap in new docs
+        run: |
+          rm -rf ./docs
+          mv /tmp/gen-html ./docs
+      - name: Push it up!
+        run: |
+          git config --local user.email "${{ inputs.git_config_email }}"
+          git config --local user.name "${{ inputs.git_config_name }}"
+          git show --summary
+          git add ./docs && git commit -m "chore(docs): update docs due to ${{github.event_name}} on ${{github.repository}}"
+          git push origin ${{ inputs.pages_branch }}
+
+  build-old-docs:
+    needs: build-and-push-latest-docs
+    runs-on: ubuntu-latest
+    if: inputs.build_old_docs == 'TRUE'
+    steps:
+      - uses: benc-uk/workflow-dispatch@v1
+        with:
+          workflow: Build old docs
+          token: ${{ secrets.JINA_DEV_BOT }}
+          inputs: '{ "release_token": "${{ env.release_token }}", "triggered_by": "TAG"}'
+        env:
+          release_token: ${{ secrets.JINA_CORE_RELEASE_TOKEN }}

.github/workflows/force-release.yml

@@ -0,0 +1,65 @@
+name: Manual Release
+
+on:
+  workflow_dispatch:
+    inputs:
+      release_token:
+        description: 'Your release token'
+        required: true
+      release_reason:
+        description: 'Short reason for this manual release'
+        required: true
+
+jobs:
+  token-check:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/github-script@v3
+        with:
+          script: |
+            core.setFailed('token are not equivalent!')
+        if: github.event.inputs.release_token != env.release_token
+        env:
+          release_token: ${{ secrets.LCSERVE_RELEASE_TOKEN }}
+
+  update-docker:
+    needs: token-check
+    runs-on: ubuntu-latest
+    steps:
+      - uses: benc-uk/workflow-dispatch@v1
+        with:
+          workflow: Manual Docker Build
+          token: ${{ secrets.JINA_DEV_BOT }}
+          inputs: '{ "release_token": "${{ env.release_token }}", "triggered_by": "TAG"}'
+        env:
+          release_token: ${{ secrets.LCSERVE_RELEASE_TOKEN }}
+
+  regular-release:
+    needs: token-check
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+        with:
+          token: ${{ secrets.JINA_DEV_BOT }}
+          fetch-depth: 100  # means max contribute history is limited to 100 lines
+#          submodules: true
+      - uses: actions/setup-python@v2
+        with:
+          python-version: 3.7
+      - run: |
+          git fetch --depth=1 origin +refs/tags/*:refs/tags/*
+          npm install git-release-notes
+          pip install twine wheel
+          ./scripts/release.sh final "${{ github.event.inputs.release_reason }}" "${{github.actor}}"
+        env:
+          TWINE_USERNAME: ${{ secrets.TWINE_USERNAME }}
+          TWINE_PASSWORD: ${{ secrets.TWINE_PASSWORD }}
+          JINA_SLACK_WEBHOOK: ${{ secrets.JINA_SLACK_WEBHOOK }}
+      - if: failure()
+        run: echo "nothing to release"
+      - name: bumping master version
+        uses: ad-m/github-push-action@v0.6.0
+        with:
+          github_token: ${{ secrets.JINA_DEV_BOT }}
+          tags: true
+          branch: main

requirements.txt

@@ -0,0 +1,2 @@
+jina
+docarray[hnswlib]

scripts/black.sh

@@ -0,0 +1,17 @@
+#!/bin/bash
+pip install black==22.3.0
+arrVar=()
+echo we ignore non-*.py files
+excluded_files=(
+)
+for changed_file in $CHANGED_FILES; do
+  if [[ ${changed_file} == *.py ]] && ! [[ " ${excluded_files[@]} " =~ " ${changed_file} " ]]; then
+    echo checking ${changed_file}
+    arrVar+=(${changed_file})
+  fi
+done
+if (( ${#arrVar[@]} )); then
+  black -S --check "${arrVar[@]}"
+fi
+echo "no files left to check"
+exit 0

scripts/get-all-test-paths.sh

@@ -0,0 +1,24 @@
+#!/usr/bin/env bash
+
+set -ex
+
+TEST_SUITE=$1
+DEFAULT_BATCH_SIZE=5
+BATCH_SIZE="${2:-$DEFAULT_BATCH_SIZE}"
+
+declare -a unit_tests=($(find tests/unit -name "test_*.py"))
+declare -a integration_tests_local=($(find tests/integration/local -name "test_*.py"))
+declare -a integration_tests_jcloud=($(find tests/integration/jcloud -name "test_*.py"))
+declare -a all_tests=("${unit_tests[@]}" "${integration_tests[@]}")
+
+if [ "$TEST_SUITE" == "unit" ]; then
+    dest="$(echo "${unit_tests[@]}" | xargs -n$BATCH_SIZE)"
+elif [[ "$TEST_SUITE" == "integration_local" ]]; then
+    dest="$(echo "${integration_tests_local[@]}" | xargs -n$BATCH_SIZE)"
+elif [[ "$TEST_SUITE" == "integration_jcloud" ]]; then
+    dest="$(echo "${integration_tests_jcloud[@]}" | xargs -n$BATCH_SIZE)"
+else
+    dest="$(echo "${all_tests[@]}" | xargs -n$BATCH_SIZE)"
+fi
+
+printf '%s\n' "${dest[@]}" | jq -R . | jq -cs .

scripts/get-last-release-note.py

@@ -0,0 +1,13 @@
+## under jina root dir
+# python scripts/get-last-release-note.py
+## result in root/tmp.md
+
+with open('CHANGELOG.md') as fp:
+    n = []
+    for v in fp:
+        if v.startswith('## Release Note'):
+            n.clear()
+        n.append(v)
+
+with open('tmp.md', 'w') as fp:
+    fp.writelines(n)

scripts/release.sh

@@ -0,0 +1,120 @@
+#!/usr/bin/env bash
+
+# Requirements
+# brew install hub
+# npm install -g git-release-notes
+# pip install twine wheel
+
+set -ex
+
+INIT_FILE='vectordb/__init__.py'
+VER_TAG='__version__ = '
+RELEASENOTE='./node_modules/.bin/git-release-notes'
+
+function escape_slashes {
+    sed 's/\//\\\//g'
+}
+
+function update_ver_line {
+    local OLD_LINE_PATTERN=$1
+    local NEW_LINE=$2
+    local FILE=$3
+
+    local NEW=$(echo "${NEW_LINE}" | escape_slashes)
+    sed -i '/'"${OLD_LINE_PATTERN}"'/s/.*/'"${NEW}"'/' "${FILE}"
+    head -n10 ${FILE}
+}
+
+
+function clean_build {
+    rm -rf dist
+    rm -rf *.egg-info
+    rm -rf build
+}
+
+function pub_pypi {
+    # publish to pypi
+    clean_build
+    python setup.py sdist
+    twine upload dist/*
+    clean_build
+}
+
+function git_commit {
+    git config --local user.email "dev-bot@jina.ai"
+    git config --local user.name "Jina Dev Bot"
+    git tag "v$RELEASE_VER" -m "$(cat ./CHANGELOG.tmp)"
+    git add $INIT_FILE ./CHANGELOG.md
+    git commit -m "chore(version): the next version will be $NEXT_VER" -m "build($RELEASE_ACTOR): $RELEASE_REASON"
+}
+
+
+
+function make_release_note {
+    ${RELEASENOTE} ${LAST_VER}..HEAD .github/release-template.ejs > ./CHANGELOG.tmp
+    head -n10 ./CHANGELOG.tmp
+    printf '\n%s\n\n%s\n%s\n\n%s\n\n%s\n\n' "$(cat ./CHANGELOG.md)" "<a name="release-note-${RELEASE_VER//\./-}"></a>" "## Release Note (\`${RELEASE_VER}\`)" "> Release time: $(date +'%Y-%m-%d %H:%M:%S')" "$(cat ./CHANGELOG.tmp)" > ./CHANGELOG.md
+}
+
+BRANCH=$(git rev-parse --abbrev-ref HEAD)
+
+if [[ "$BRANCH" != "main" ]]; then
+  printf "You are not at main branch, exit\n";
+  exit 1;
+fi
+
+LAST_UPDATE=`git show --no-notes --format=format:"%H" $BRANCH | head -n 1`
+LAST_COMMIT=`git show --no-notes --format=format:"%H" origin/$BRANCH | head -n 1`
+
+if [ $LAST_COMMIT != $LAST_UPDATE ]; then
+    printf "Your local $BRANCH is behind the remote master, exit\n"
+    exit 1;
+fi
+
+# release the current version
+export RELEASE_VER=$(sed -n '/^__version__/p' $INIT_FILE | cut -d \' -f2)
+LAST_VER=$(git tag -l | sort -V | tail -n1)
+printf "last version: \e[1;32m$LAST_VER\e[0m\n"
+
+if [[ $1 == "final" ]]; then
+  printf "this will be a final release: \e[1;33m$RELEASE_VER\e[0m\n"
+
+  NEXT_VER=$(echo $RELEASE_VER | awk -F. -v OFS=. 'NF==1{print ++$NF}; NF>1{$NF=sprintf("%0*d", length($NF), ($NF+1)); print}')
+  printf "bump master version to: \e[1;32m$NEXT_VER\e[0m\n"
+
+  make_release_note
+
+  pub_pypi
+
+  VER_TAG_NEXT=$VER_TAG\'${NEXT_VER}\'
+  update_ver_line "$VER_TAG" "$VER_TAG_NEXT" "$INIT_FILE"
+  RELEASE_REASON="$2"
+  RELEASE_ACTOR="$3"
+  git_commit
+elif [[ $1 == 'rc' ]]; then
+  printf "this will be a release candidate: \e[1;33m$RELEASE_VER\e[0m\n"
+  DOT_RELEASE_VER=$(echo $RELEASE_VER | sed "s/rc/\./")
+  NEXT_VER=$(echo $DOT_RELEASE_VER | awk -F. -v OFS=. 'NF==1{print ++$NF}; NF>1{$NF=sprintf("%0*d", length($NF), ($NF+1)); print}')
+  NEXT_VER=$(echo $NEXT_VER | sed "s/\.\([^.]*\)$/rc\1/")
+  printf "bump master version to: \e[1;32m$NEXT_VER\e[0m, this will be the next version\n"
+
+  make_release_note
+
+  pub_pypi
+
+  VER_TAG_NEXT=$VER_TAG\'${NEXT_VER}\'
+  update_ver_line "$VER_TAG" "$VER_TAG_NEXT" "$INIT_FILE"
+  RELEASE_REASON="$2"
+  RELEASE_ACTOR="$3"
+  git_commit
+else
+  # as a prerelease, pypi update only, no back commit etc.
+  COMMITS_SINCE_LAST_VER=$(git rev-list $LAST_VER..HEAD --count)
+  NEXT_VER=$RELEASE_VER".dev"$COMMITS_SINCE_LAST_VER
+  printf "this will be a developmental release: \e[1;33m$NEXT_VER\e[0m\n"
+
+  VER_TAG_NEXT=$VER_TAG\'${NEXT_VER}\'
+  update_ver_line "$VER_TAG" "$VER_TAG_NEXT" "$INIT_FILE"
+
+  pub_pypi
+fi

setup.py

@@ -0,0 +1,38 @@
+from setuptools import setup, find_packages
+from vectordb import __version__
+
+# Read the contents of requirements.txt
+with open('requirements.txt', 'r') as f:
+    requirements = f.read().splitlines()
+
+setup(
+    name='vectordb',
+    version=__version__,
+    description='The Python VectorDB. Build your vector database from working as a library to scaling as a database in the cloud',
+    author='Jina AI',
+    author_email='hello@jina.ai',
+    license='Apache 2.0',
+    url='https://github.com/jina-ai/vectordb/',
+    download_url='https://github.com/jina-ai/vectordb/tags',
+    packages=find_packages(),
+    classifiers=[
+        'Development Status :: 5 - Production/Stable',
+        'Intended Audience :: Developers',
+        'License :: OSI Approved :: MIT License',
+        'Programming Language :: Python',
+        'Programming Language :: Python :: 3',
+        'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
+        'Programming Language :: Python :: 3.9',
+        'Programming Language :: Python :: 3.10',
+        'Programming Language :: Python :: 3.11',
+    ],
+    python_requires='>=3.7',
+    extras_require={
+        'test': [
+            'pytest',
+            'pytest-asyncio',
+        ],
+    },
+    install_requires=requirements,
+)

vectordb/__init__.py

@@ -0,0 +1 @@
+__version__ = '0.0.1'