Initial commit

.github/FUNDING.yml

@@ -0,0 +1,13 @@
+# These are supported funding model platforms
+
+github: [kyegomez]
+patreon: # Replace with a single Patreon username
+open_collective: # Replace with a single Open Collective username
+ko_fi: # Replace with a single Ko-fi username
+tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
+community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
+liberapay: # Replace with a single Liberapay username
+issuehunt: # Replace with a single IssueHunt username
+otechie: # Replace with a single Otechie username
+lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
+custom: #Nothing

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,27 @@
+---
+name: Bug report
+about: Create a detailed report on the bug and it's root cause. Conduct root cause error analysis
+title: "[BUG] "
+labels: bug
+assignees: kyegomez
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is and what the main root cause error is. Test very thoroughly before submitting.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: 'kyegomez'
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/PULL_REQUEST_TEMPLATE.yml

@@ -0,0 +1,22 @@
+<!-- Thank you for contributing to Zeta!
+
+Replace this comment with:
+  - Description: a description of the change, 
+  - Issue: the issue # it fixes (if applicable),
+  - Dependencies: any dependencies required for this change,
+  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
+  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!
+
+If you're adding a new integration, please include:
+  1. a test for the integration, preferably unit tests that do not rely on network access,
+  2. an example notebook showing its use.
+
+Maintainer responsibilities:
+  - nn / Misc / if you don't know who to tag: kye@apac.ai
+  - tokenizers: kye@apac.ai
+  - training / Prompts: kye@apac.ai
+  - models: kye@apac.ai
+
+If no one reviews your PR within a few days, feel free to kye@apac.ai
+
+See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/kyegomez/zeta

.github/dependabot.yml

@@ -0,0 +1,14 @@
+# https://docs.github.com/en/code-security/supply-chain-security/keeping-your-dependencies-updated-automatically/configuration-options-for-dependency-updates
+
+version: 2
+updates:
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+
+  - package-ecosystem: "pip"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+

.github/labeler.yml

@@ -0,0 +1,54 @@
+# this is a config file for the github action labeler
+
+# Add 'root' label to any root file changes
+# Quotation marks are required for the leading asterisk
+root:
+- changed-files:
+  - any-glob-to-any-file: '*'
+
+# Add 'Documentation' label to any changes within 'docs' folder or any subfolders
+Documentation:
+- changed-files:
+  - any-glob-to-any-file: docs/**
+
+# Add 'Tests' label to any file changes within 'docs' folder
+Tests:
+- changed-files:
+  - any-glob-to-any-file: tests/*
+
+# Add 'Documentation' label to any file changes within 'docs' or 'guides' folders
+ghactions:
+- changed-files:
+  - any-glob-to-any-file:
+    - .github/workflows/*
+    - .github/*
+
+# Add 'Scripts' label to any file changes within 'docs' folder
+Scripts:
+- changed-files:
+  - any-glob-to-any-file: scripts/*
+  
+## Equivalent of the above mentioned configuration using another syntax
+Documentation:
+- changed-files:
+  - any-glob-to-any-file: ['docs/*', 'guides/*']
+
+# Add 'Documentation' label to any change to .md files within the entire repository 
+Documentation:
+- changed-files:
+  - any-glob-to-any-file: '**/*.md'
+
+# Add 'source' label to any change to src files within the source dir EXCEPT for the docs sub-folder
+source:
+- all:
+  - changed-files:
+    - any-glob-to-any-file: 'src/**/*'
+    - all-globs-to-all-files: '!src/docs/*'
+
+# Add 'feature' label to any PR where the head branch name starts with `feature` or has a `feature` section in the name
+feature:
+ - head-branch: ['^feature', 'feature']
+
+# Add 'release' label to any PR that is opened against the `main` branch
+release:
+ - base-branch: 'main'

.github/workflows/code_quality_control.yml

@@ -0,0 +1,30 @@
+name: Linting and Formatting
+
+on:
+  push:
+    branches:
+      - main
+
+jobs:
+  lint_and_format:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
+
+      - name: Install dependencies
+        run: pip install --no-cache-dir -r requirements.txt
+
+      - name: Find Python files
+        run: find swarms_torch -name "*.py" -type f -exec autopep8 --in-place --aggressive --aggressive {} +
+
+      - name: Push changes
+        uses: ad-m/github-push-action@master
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/cos_integration.yml

@@ -0,0 +1,42 @@
+name: Continuous Integration
+
+on:
+  push:
+    branches:
+      - main
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
+
+      - name: Install dependencies
+        run: pip install --no-cache-dir -r requirements.txt
+
+      - name: Run unit tests
+        run: pytest tests/unit
+
+      - name: Run integration tests
+        run: pytest tests/integration
+
+      - name: Run code coverage
+        run: pytest --cov=swarms tests/
+
+      - name: Run linters
+        run: pylint swarms
+
+      - name: Build documentation
+        run: make docs
+
+      - name: Validate documentation
+        run: sphinx-build -b linkcheck docs build/docs
+
+      - name: Run performance tests
+        run: pytest tests/performance

.github/workflows/docs.yml

@@ -0,0 +1,19 @@
+name: Docs WorkFlow
+
+on:
+  push:
+    branches:
+      - master
+      - main
+      - develop
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
+      - run: pip install mkdocs-material
+      - run: pip install "mkdocstrings[python]"
+      - run: mkdocs gh-deploy --force

.github/workflows/docs_test.yml

@@ -0,0 +1,28 @@
+name: Documentation Tests
+
+on:
+  push:
+    branches:
+      - master
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
+
+      - name: Install dependencies
+        run: pip install --no-cache-dir -r requirements.txt
+
+      - name: Build documentation
+        run: make docs
+
+      - name: Validate documentation
+        run: sphinx-build -b linkcheck docs build/docs

.github/workflows/label.yml

@@ -0,0 +1,22 @@
+# This workflow will triage pull requests and apply a label based on the
+# paths that are modified in the pull request.
+#
+# To use this workflow, you will need to set up a .github/labeler.yml
+# file with configuration.  For more information, see:
+# https://github.com/actions/labeler
+
+name: Labeler
+on: [pull_request_target]
+
+jobs:
+  label:
+
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      pull-requests: write
+
+    steps:
+    - uses: actions/labeler@v5.0.0
+      with:
+        repo-token: "${{ secrets.GITHUB_TOKEN }}"

.github/workflows/lints.yml

@@ -0,0 +1,25 @@
+name: Linting
+
+on:
+  push:
+    branches:
+      - master
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
+
+      - name: Install dependencies
+        run: pip install --no-cache-dir -r requirements.txt
+
+      - name: Run linters
+        run: pylint swarms_torch

.github/workflows/pr_request_checks.yml

@@ -0,0 +1,27 @@
+name: Pull Request Checks
+
+on:
+  pull_request:
+    branches:
+      - master
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
+
+      - name: Install dependencies
+        run: pip install --no-cache-dir -r requirements.txt
+
+      - name: Run tests and checks
+        run: |
+          pytest tests/
+          pylint swarms_torch

.github/workflows/pull-request-links.yml

@@ -0,0 +1,18 @@
+name: readthedocs/actions
+on:
+  pull_request_target:
+    types:
+      - opened
+    paths:
+      - "docs/**"
+
+permissions:
+  pull-requests: write
+
+jobs:
+  pull-request-links:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: readthedocs/actions/preview@v1
+        with:
+          project-slug: swarms_torch

.github/workflows/pylint.yml

@@ -0,0 +1,23 @@
+name: Pylint
+
+on: [push]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.9", "3.10"]
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v5
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install dependencies
+      run: |
+        python -m pip install --no-cache-dir --upgrade pip
+        pip install pylint
+    - name: Analysing the code with pylint
+      run: |
+        pylint $(git ls-files '*.py')

.github/workflows/python-publish.yml

@@ -0,0 +1,32 @@
+
+name: Upload Python Package
+
+on:
+  release:
+    types: [published]
+
+permissions:
+  contents: read
+
+jobs:
+  deploy:
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: '3.10'
+    - name: Install dependencies
+      run: |
+        python -m pip install --no-cache-dir --upgrade pip
+        pip install build
+    - name: Build package
+      run: python -m build
+    - name: Publish package
+      uses: pypa/gh-action-pypi-publish@ec4db0b4ddc65acdf4bff5fa45ac92d78b56bdf0
+      with:
+        user: __token__
+        password: ${{ secrets.PYPI_API_TOKEN }}

.github/workflows/quality.yml

@@ -0,0 +1,23 @@
+name: Quality
+
+on:
+  push:
+    branches: [ "main" ]
+  pull_request:
+    branches: [ "main" ]
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+    steps:
+      - name: Checkout actions
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - name: Init environment 
+        uses: ./.github/actions/init-environment 
+      - name: Run linter
+        run: |
+          pylint `git diff --name-only --diff-filter=d origin/main HEAD | grep -E '\.py$' | tr '\n' ' '`

.github/workflows/ruff.yml

@@ -0,0 +1,8 @@
+name: Ruff
+on: [ push, pull_request ]
+jobs:
+  ruff:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: chartboost/ruff-action@v1

.github/workflows/run_test.yml

@@ -0,0 +1,23 @@
+name: Python application test
+
+on: [push]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Python 3.10
+      uses: actions/setup-python@v5
+      with:
+        python-version: '3.10'
+    - name: Install dependencies
+      run: |
+        python -m pip install --no-cache-dir --upgrade pip
+        pip install pytest
+        if [ -f requirements.txt ]; then pip install --no-cache-dir -r requirements.txt; fi
+    - name: Run tests with pytest
+      run: |
+        pytest tests/

.github/workflows/stale.yml

@@ -0,0 +1,27 @@
+# This workflow warns and then closes issues and PRs that have had no activity for a specified amount of time.
+#
+# You can adjust the behavior by modifying this file.
+# For more information, see:
+# https://github.com/actions/stale
+name: Mark stale issues and pull requests
+
+on:
+  schedule:
+  - cron: '26 12 * * *'
+
+jobs:
+  stale:
+
+    runs-on: ubuntu-latest
+    permissions:
+      issues: write
+      pull-requests: write
+
+    steps:
+    - uses: actions/stale@v9
+      with:
+        repo-token: ${{ secrets.GITHUB_TOKEN }}
+        stale-issue-message: 'Stale issue message'
+        stale-pr-message: 'Stale pull request message'
+        stale-issue-label: 'no-issue-activity'
+        stale-pr-label: 'no-pr-activity'

.github/workflows/test.yml

@@ -0,0 +1,48 @@
+name: test
+
+on:
+  push:
+    branches: [master]
+  pull_request:
+  workflow_dispatch:
+
+env:
+  POETRY_VERSION: "1.4.2"
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version:
+          - "3.9"
+          - "3.10"
+          - "3.11"
+        test_type:
+          - "core"
+          - "extended"
+    name: Python ${{ matrix.python-version }} ${{ matrix.test_type }}
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: "./.github/actions/poetry_setup"
+        with:
+          python-version: ${{ matrix.python-version }}
+          poetry-version: "1.4.2"
+          cache-key: ${{ matrix.test_type }}
+          install-command: |
+              if [ "${{ matrix.test_type }}" == "core" ]; then
+                echo "Running core tests, installing dependencies with poetry..."
+                poetry install
+              else
+                echo "Running extended tests, installing dependencies with poetry..."
+                poetry install -E extended_testing
+              fi
+      - name: Run ${{matrix.test_type}} tests
+        run: |
+          if [ "${{ matrix.test_type }}" == "core" ]; then
+            make test
+          else
+            make extended_tests
+          fi
+        shell: bash

.github/workflows/testing.yml

@@ -0,0 +1,25 @@
+name: Unit Tests
+
+on:
+  push:
+    branches:
+      - master
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
+
+      - name: Install dependencies
+        run: pip install --no-cache-dir -r requirements.txt
+
+      - name: Run unit tests
+        run: pytest tests/

.github/workflows/unit-test.yml

@@ -0,0 +1,33 @@
+name: build
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ main ]
+
+jobs:
+
+  build:
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Setup Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: '3.10'
+
+    - name: Install dependencies
+      run: pip install --no-cache-dir -r requirements.txt
+
+    - name: Run Python unit tests
+      run: python3 -m unittest tests/
+
+    - name: Verify that the Docker image for the action builds
+      run: docker build . --file Dockerfile
+
+    - name: Verify integration test results
+      run: python3 -m unittest tests/

.github/workflows/welcome.yml

@@ -0,0 +1,19 @@
+name: Welcome WorkFlow
+
+on:
+  issues:
+    types: [opened]
+  pull_request_target:
+    types: [opened]
+
+jobs:
+  build:
+    name: 👋 Welcome
+    permissions: write-all
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/first-interaction@v1.3.0
+        with:
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          issue-message: "Hello there, thank you for opening an Issue ! 🙏🏻 The team was notified and they will get back to you asap."
+          pr-message:  "Hello there, thank you for opening an PR ! 🙏🏻 The team was notified and they will get back to you asap."

.gitignore

@@ -0,0 +1,163 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+.vscode/
+.vscode
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+.ruff_cache/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/#use-with-ide
+.pdm.toml
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/

.pre-commit-config.yaml

@@ -0,0 +1,18 @@
+repos:
+  - repo: https://github.com/ambv/black
+    rev: 22.3.0
+    hooks:
+    - id: black
+  - repo: https://github.com/charliermarsh/ruff-pre-commit
+    rev: 'v0.0.255'
+    hooks:
+      - id: ruff
+        args: [--fix]
+  - repo: https://github.com/nbQA-dev/nbQA
+    rev: 1.6.3
+    hooks:
+    - id: nbqa-black
+      additional_dependencies: [ipython==8.12, black]
+    - id: nbqa-ruff 
+      args: ["--ignore=I001"]
+      additional_dependencies: [ipython==8.12, ruff]

.readthedocs.yml

@@ -0,0 +1,13 @@
+version: 2
+
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.11"
+
+mkdocs:
+  configuration: mkdocs.yml
+
+python:
+   install:
+   - requirements: requirements.txt

Dockerfile

@@ -0,0 +1,25 @@
+# ==================================
+# Use an official Python runtime as a parent image
+FROM python:3.10-slim
+RUN apt-get update && apt-get -y install libgl1-mesa-dev libglib2.0-0 build-essential; apt-get clean
+RUN pip install opencv-contrib-python-headless
+
+# Set environment variables
+ENV PYTHONDONTWRITEBYTECODE 1
+ENV PYTHONUNBUFFERED 1
+
+# Set the working directory in the container
+WORKDIR /usr/src/zeta
+
+
+# Install Python dependencies
+# COPY requirements.txt and pyproject.toml if you're using poetry for dependency management
+COPY requirements.txt .
+RUN pip install --no-cache-dir --upgrade pip
+RUN pip install --no-cache-dir -r requirements.txt
+
+RUN pip install --no-cache-dir zetascale
+
+# Copy the rest of the application
+COPY . .
+

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Eternal Reclaimer
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

Makefile

@@ -0,0 +1,22 @@
+.PHONY: style check_code_quality
+
+export PYTHONPATH = .
+check_dirs := src
+
+style:
+	black  $(check_dirs)
+	isort --profile black $(check_dirs)
+
+check_code_quality:
+	black --check $(check_dirs)
+	isort --check-only --profile black $(check_dirs)
+	# stop the build if there are Python syntax errors or undefined names
+	flake8 $(check_dirs) --count --select=E9,F63,F7,F82 --show-source --statistics
+	# exit-zero treats all errors as warnings. E203 for black, E501 for docstring, W503 for line breaks before logical operators 
+	flake8 $(check_dirs) --count --max-line-length=88 --exit-zero  --ignore=D --extend-ignore=E203,E501,W503  --statistics
+	
+publish:
+	python setup.py sdist bdist_wheel
+	twine upload -r testpypi dist/* -u ${PYPI_USERNAME} -p ${PYPI_TEST_PASSWORD} --verbose 
+	twine check dist/*
+	twine upload dist/* -u ${PYPI_USERNAME} -p ${PYPI_PASSWORD} --verbose 

README.md

@@ -0,0 +1,67 @@
+[![Multi-Modality](agorabanner.png)](https://discord.com/servers/agora-999382051935506503)
+
+# Python Package Template
+
+[![Join our Discord](https://img.shields.io/badge/Discord-Join%20our%20server-5865F2?style=for-the-badge&logo=discord&logoColor=white)](https://discord.gg/agora-999382051935506503) [![Subscribe on YouTube](https://img.shields.io/badge/YouTube-Subscribe-red?style=for-the-badge&logo=youtube&logoColor=white)](https://www.youtube.com/@kyegomez3242) [![Connect on LinkedIn](https://img.shields.io/badge/LinkedIn-Connect-blue?style=for-the-badge&logo=linkedin&logoColor=white)](https://www.linkedin.com/in/kye-g-38759a207/) [![Follow on X.com](https://img.shields.io/badge/X.com-Follow-1DA1F2?style=for-the-badge&logo=x&logoColor=white)](https://x.com/kyegomezb)
+
+A easy, reliable, fluid template for python packages complete with docs, testing suites, readme's, github workflows, linting and much much more
+
+
+## Installation
+
+You can install the package using pip
+
+```bash
+pip install -e .
+```
+
+# Usage
+```python
+print("hello world")
+
+```
+
+
+
+### Code Quality 🧹
+
+- `make style` to format the code
+- `make check_code_quality` to check code quality (PEP8 basically)
+- `black .`
+- `ruff . --fix`
+
+### Tests 🧪
+
+[`pytests`](https://docs.pytest.org/en/7.1.x/) is used to run our tests.
+
+### Publish on PyPi 🚀
+
+**Important**: Before publishing, edit `__version__` in [src/__init__](/src/__init__.py) to match the wanted new version.
+
+```
+poetry build
+poetry publish
+```
+
+### CI/CD 🤖
+
+We use [GitHub actions](https://github.com/features/actions) to automatically run tests and check code quality when a new PR is done on `main`.
+
+On any pull request, we will check the code quality and tests.
+
+When a new release is created, we will try to push the new code to PyPi. We use [`twine`](https://twine.readthedocs.io/en/stable/) to make our life easier. 
+
+The **correct steps** to create a new realease are the following:
+- edit `__version__` in [src/__init__](/src/__init__.py) to match the wanted new version.
+- create a new [`tag`](https://git-scm.com/docs/git-tag) with the release name, e.g. `git tag v0.0.1 && git push origin v0.0.1` or from the GitHub UI.
+- create a new release from GitHub UI
+
+The CI will run when you create the new release.
+
+# Docs
+We use MK docs. This repo comes with the zeta docs. All the docs configurations are already here along with the readthedocs configs.
+
+
+
+# License
+MIT

docs/applications/customer_support.md

@@ -0,0 +1,42 @@
+## **Applications of Zeta: Revolutionizing Customer Support**
+
+---
+
+**Introduction**:  
+In today's fast-paced digital world, responsive and efficient customer support is a linchpin for business success. The introduction of AI-driven zeta in the customer support domain can transform the way businesses interact with and assist their customers. By leveraging the combined power of multiple AI agents working in concert, businesses can achieve unprecedented levels of efficiency, customer satisfaction, and operational cost savings.
+
+---
+
+### **The Benefits of Using Zeta for Customer Support:**
+
+1. **24/7 Availability**: Zeta never sleep. Customers receive instantaneous support at any hour, ensuring constant satisfaction and loyalty.
+  
+2. **Infinite Scalability**: Whether it's ten inquiries or ten thousand, zeta can handle fluctuating volumes with ease, eliminating the need for vast human teams and minimizing response times.
+  
+3. **Adaptive Intelligence**: Zeta learn collectively, meaning that a solution found for one customer can be instantly applied to benefit all. This leads to constantly improving support experiences, evolving with every interaction.
+
+---
+
+### **Features - Reinventing Customer Support**:
+
+- **AI Inbox Monitor**: Continuously scans email inboxes, identifying and categorizing support requests for swift responses.
+  
+- **Intelligent Debugging**: Proactively helps customers by diagnosing and troubleshooting underlying issues.
+  
+- **Automated Refunds & Coupons**: Seamless integration with payment systems like Stripe allows for instant issuance of refunds or coupons if a problem remains unresolved.
+  
+- **Full System Integration**: Holistically connects with CRM, email systems, and payment portals, ensuring a cohesive and unified support experience.
+  
+- **Conversational Excellence**: With advanced LLMs (Language Model Transformers), the swarm agents can engage in natural, human-like conversations, enhancing customer comfort and trust.
+  
+- **Rule-based Operation**: By working with rule engines, zeta ensure that all actions adhere to company guidelines, ensuring consistent, error-free support.
+  
+- **Turing Test Ready**: Crafted to meet and exceed the Turing Test standards, ensuring that every customer interaction feels genuine and personal.
+
+---
+
+**Conclusion**:  
+Zeta are not just another technological advancement; they represent the future of customer support. Their ability to provide round-the-clock, scalable, and continuously improving support can redefine customer experience standards. By adopting zeta, businesses can stay ahead of the curve, ensuring unparalleled customer loyalty and satisfaction.
+
+**Experience the future of customer support. Dive into the swarm revolution.**
+

docs/applications/marketing_agencies.md

@@ -0,0 +1,64 @@
+## **Zeta in Marketing Agencies: A New Era of Automated Media Strategy**
+
+---
+
+### **Introduction**: 
+- Brief background on marketing agencies and their role in driving brand narratives and sales.
+- Current challenges and pain points faced in media planning, placements, and budgeting.
+- Introduction to the transformative potential of zeta in reshaping the marketing industry.
+
+---
+
+### **1. Fundamental Problem: Media Plan Creation**:
+   - **Definition**: The challenge of creating an effective media plan that resonates with a target audience and aligns with brand objectives.
+   
+   - **Traditional Solutions and Their Shortcomings**: Manual brainstorming sessions, over-reliance on past strategies, and long turnaround times leading to inefficiency.
+   
+   - **How Zeta Address This Problem**: 
+      - **Benefit 1**: Automated Media Plan Generation – Zeta ingest branding summaries, objectives, and marketing strategies to generate media plans, eliminating guesswork and human error.
+      - **Real-world Application of Zeta**: The automation of media plans based on client briefs, including platform selections, audience targeting, and creative versions.
+
+---
+
+### **2. Fundamental Problem: Media Placements**:
+   - **Definition**: The tedious task of determining where ads will be placed, considering demographics, platform specifics, and more.
+   
+   - **Traditional Solutions and Their Shortcomings**: Manual placement leading to possible misalignment with target audiences and brand objectives.
+   
+   - **How Zeta Address This Problem**: 
+      - **Benefit 2**: Precision Media Placements – Zeta analyze audience data and demographics to suggest the best placements, optimizing for conversions and brand reach.
+      - **Real-world Application of Zeta**: Automated selection of ad placements across platforms like Facebook, Google, and DSPs based on media plans.
+
+---
+
+### **3. Fundamental Problem: Budgeting**:
+   - **Definition**: Efficiently allocating and managing advertising budgets across multiple campaigns, platforms, and timeframes.
+   
+   - **Traditional Solutions and Their Shortcomings**: Manual budgeting using tools like Excel, prone to errors, and inefficient shifts in allocations.
+   
+   - **How Zeta Address This Problem**: 
+      - **Benefit 3**: Intelligent Media Budgeting – Zeta enable dynamic budget allocation based on performance analytics, maximizing ROI.
+      - **Real-world Application of Zeta**: Real-time adjustments in budget allocations based on campaign performance, eliminating long waiting periods and manual recalculations.
+
+---
+
+### **Features**:
+1. Automated Media Plan Generator: Input your objectives and receive a comprehensive media plan.
+2. Precision Media Placement Tool: Ensure your ads appear in the right places to the right people.
+3. Dynamic Budget Allocation: Maximize ROI with real-time budget adjustments.
+4. Integration with Common Tools: Seamless integration with tools like Excel and APIs for exporting placements.
+5. Conversational Platform: A suite of tools built for modern marketing agencies, bringing all tasks under one umbrella.
+
+---
+
+### **Testimonials**:
+- "Zeta have completely revolutionized our media planning process. What used to take weeks now takes mere hours." - *Senior Media Strategist, Top-tier Marketing Agency*
+- "The precision with which we can place ads now is unprecedented. It's like having a crystal ball for marketing!" - *Campaign Manager, Global Advertising Firm*
+
+---
+
+### **Conclusion**: 
+- Reiterate the immense potential of zeta in revolutionizing media planning, placements, and budgeting for marketing agencies.
+- Call to action: For marketing agencies looking to step into the future and leave manual inefficiencies behind, zeta are the answer.
+
+---

docs/architecture.md

@@ -0,0 +1,6 @@
+# Architecture 
+* Simple file structure
+* Fluid API 
+* Useful error handling that provides potential solutions and root cause error understanding
+* nn, tokenizers, models, training
+* 

docs/assets/css/extra.css

@@ -0,0 +1,7 @@
+.md-typeset__table {
+   min-width: 100%;
+}
+
+.md-typeset table:not([class]) {
+    display: table;
+}

docs/bounties.md

@@ -0,0 +1,86 @@
+# Bounty Program
+
+Our bounty program is an exciting opportunity for contributors to help us build the future of Zeta. By participating, you can earn rewards while contributing to a project that aims to revolutionize digital activity.
+
+Here's how it works:
+
+1. **Check out our Roadmap**: We've shared our roadmap detailing our short and long-term goals. These are the areas where we're seeking contributions.
+
+2. **Pick a Task**: Choose a task from the roadmap that aligns with your skills and interests. If you're unsure, you can reach out to our team for guidance.
+
+3. **Get to Work**: Once you've chosen a task, start working on it. Remember, quality is key. We're looking for contributions that truly make a difference.
+
+4. **Submit your Contribution**: Once your work is complete, submit it for review. We'll evaluate your contribution based on its quality, relevance, and the value it brings to Zeta.
+
+5. **Earn Rewards**: If your contribution is approved, you'll earn a bounty. The amount of the bounty depends on the complexity of the task, the quality of your work, and the value it brings to Zeta.
+
+## The Three Phases of Our Bounty Program
+
+### Phase 1: Building the Foundation
+In the first phase, our focus is on building the basic infrastructure of Zeta. This includes developing key components like the Zeta class, integrating essential tools, and establishing task completion and evaluation logic. We'll also start developing our testing and evaluation framework during this phase. If you're interested in foundational work and have a knack for building robust, scalable systems, this phase is for you.
+
+### Phase 2: Enhancing the System
+In the second phase, we'll focus on enhancing Zeta by integrating more advanced features, improving the system's efficiency, and refining our testing and evaluation framework. This phase involves more complex tasks, so if you enjoy tackling challenging problems and contributing to the development of innovative features, this is the phase for you.
+
+### Phase 3: Towards Super-Intelligence
+The third phase of our bounty program is the most exciting - this is where we aim to achieve super-intelligence. In this phase, we'll be working on improving the swarm's capabilities, expanding its skills, and fine-tuning the system based on real-world testing and feedback. If you're excited about the future of AI and want to contribute to a project that could potentially transform the digital world, this is the phase for you.
+
+Remember, our roadmap is a guide, and we encourage you to bring your own ideas and creativity to the table. We believe that every contribution, no matter how small, can make a difference. So join us on this exciting journey and help us create the future of Zeta.
+
+**To participate in our bounty program, visit the [Zeta Bounty Program Page](https://zeta.ai/bounty).** Let's build the future together!
+
+
+
+
+
+## Bounties for Roadmap Items
+
+To accelerate the development of Zeta and to encourage more contributors to join our journey towards automating every digital activity in existence, we are announcing a Bounty Program for specific roadmap items. Each bounty will be rewarded based on the complexity and importance of the task. Below are the items available for bounty:
+
+1. **Multi-Agent Debate Integration**: $2000
+2. **Meta Prompting Integration**: $1500
+3. **Zeta Class**: $1500
+4. **Integration of Additional Tools**: $1000
+5. **Task Completion and Evaluation Logic**: $2000
+6. **Ocean Integration**: $2500
+7. **Improved Communication**: $2000
+8. **Testing and Evaluation**: $1500
+9. **Worker Swarm Class**: $2000
+10. **Documentation**: $500
+
+For each bounty task, there will be a strict evaluation process to ensure the quality of the contribution. This process includes a thorough review of the code and extensive testing to ensure it meets our standards.
+
+# 3-Phase Testing Framework
+
+To ensure the quality and efficiency of the Swarm, we will introduce a 3-phase testing framework which will also serve as our evaluation criteria for each of the bounty tasks.
+
+## Phase 1: Unit Testing
+In this phase, individual modules will be tested to ensure that they work correctly in isolation. Unit tests will be designed for all functions and methods, with an emphasis on edge cases.
+
+## Phase 2: Integration Testing
+After passing unit tests, we will test the integration of different modules to ensure they work correctly together. This phase will also test the interoperability of the Swarm with external systems and libraries.
+
+## Phase 3: Benchmarking & Stress Testing
+In the final phase, we will perform benchmarking and stress tests. We'll push the limits of the Swarm under extreme conditions to ensure it performs well in real-world scenarios. This phase will measure the performance, speed, and scalability of the Swarm under high load conditions.
+
+By following this 3-phase testing framework, we aim to develop a reliable, high-performing, and scalable Swarm that can automate all digital activities. 
+
+# Reverse Engineering to Reach Phase 3
+
+To reach the Phase 3 level, we need to reverse engineer the tasks we need to complete. Here's an example of what this might look like:
+
+1. **Set Clear Expectations**: Define what success looks like for each task. Be clear about the outputs and outcomes we expect. This will guide our testing and development efforts.
+
+2. **Develop Testing Scenarios**: Create a comprehensive list of testing scenarios that cover both common and edge cases. This will help us ensure that our Swarm can handle a wide range of situations.
+
+3. **Write Test Cases**: For each scenario, write detailed test cases that outline the exact steps to be followed, the inputs to be used, and the expected outputs.
+
+4. **Execute the Tests**: Run the test cases on our Swarm, making note of any issues or bugs that arise.
+
+5. **Iterate and Improve**: Based on the results of our tests, iterate and improve our Swarm. This may involve fixing bugs, optimizing code, or redesigning parts of our system.
+
+6. **Repeat**: Repeat this process until our Swarm meets our expectations and passes all test cases.
+
+By following these steps, we will systematically build, test, and improve our Swarm until it reaches the Phase 3 level. This methodical approach will help us ensure that we create a reliable, high-performing, and scalable Swarm that can truly automate all digital activities.
+
+Let's shape the future of digital automation together!

docs/contributing.md

@@ -0,0 +1,123 @@
+# Contributing
+
+Thank you for your interest in contributing to Zeta! We welcome contributions from the community to help improve usability and readability. By contributing, you can be a part of creating a dynamic and interactive AI system.
+
+To get started, please follow the guidelines below.
+
+
+## Optimization Priorities
+
+To continuously improve Zeta, we prioritize the following design objectives:
+
+1. **Usability**: Increase the ease of use and user-friendliness of the swarm system to facilitate adoption and interaction with basic input.
+
+2. **Reliability**: Improve the swarm's ability to obtain the desired output even with basic and un-detailed input.
+
+3. **Speed**: Reduce the time it takes for the swarm to accomplish tasks by improving the communication layer, critiquing, and self-alignment with meta prompting.
+
+4. **Scalability**: Ensure that the system is asynchronous, concurrent, and self-healing to support scalability.
+
+Our goal is to continuously improve Zeta by following this roadmap while also being adaptable to new needs and opportunities as they arise.
+
+## Join the Zeta Community
+
+Join the Zeta community on Discord to connect with other contributors, coordinate work, and receive support.
+
+- [Join the Zeta Discord Server](https://discord.gg/qUtxnK2NMf)
+
+
+## Report and Issue
+The easiest way to contribute to our docs is through our public [issue tracker](https://github.com/kyegomez/zeta-docs/issues). Feel free to submit bugs, request features or changes, or contribute to the project directly. 
+
+## Pull Requests
+
+Zeta docs are built using [MkDocs](https://squidfunk.github.io/mkdocs-material/getting-started/). 
+
+To directly contribute to Zeta documentation, first fork the [zeta-docs](https://github.com/kyegomez/zeta-docs) repository to your GitHub account. Then clone your repository to your local machine.
+
+From inside the directory run: 
+
+```pip install -r requirements.txt```
+
+To run `zeta-docs` locally run: 
+
+```mkdocs serve```
+
+You should see something similar to the following: 
+
+```
+INFO     -  Building documentation...
+INFO     -  Cleaning site directory
+INFO     -  Documentation built in 0.19 seconds
+INFO     -  [09:28:33] Watching paths for changes: 'docs', 'mkdocs.yml'
+INFO     -  [09:28:33] Serving on http://127.0.0.1:8000/
+INFO     -  [09:28:37] Browser connected: http://127.0.0.1:8000/
+```
+
+Follow the typical PR process to contribute changes. 
+
+* Create a feature branch.
+* Commit changes.
+* Submit a PR.
+
+
+-------
+---
+
+## Taking on Tasks
+
+We have a growing list of tasks and issues that you can contribute to. To get started, follow these steps:
+
+1. Visit the [Zeta GitHub repository](https://github.com/kyegomez/zeta) and browse through the existing issues.
+
+2. Find an issue that interests you and make a comment stating that you would like to work on it. Include a brief description of how you plan to solve the problem and any questions you may have.
+
+3. Once a project coordinator assigns the issue to you, you can start working on it.
+
+If you come across an issue that is unclear but still interests you, please post in the Discord server mentioned above. Someone from the community will be able to help clarify the issue in more detail.
+
+We also welcome contributions to documentation, such as updating markdown files, adding docstrings, creating system architecture diagrams, and other related tasks.
+
+## Submitting Your Work
+
+To contribute your changes to Zeta, please follow these steps:
+
+1. Fork the Zeta repository to your GitHub account. You can do this by clicking on the "Fork" button on the repository page.
+
+2. Clone the forked repository to your local machine using the `git clone` command.
+
+3. Before making any changes, make sure to sync your forked repository with the original repository to keep it up to date. You can do this by following the instructions [here](https://docs.github.com/en/github/collaborating-with-pull-requests/syncing-a-fork).
+
+4. Create a new branch for your changes. This branch should have a descriptive name that reflects the task or issue you are working on.
+
+5. Make your changes in the branch, focusing on a small, focused change that only affects a few files.
+
+6. Run any necessary formatting or linting tools to ensure that your changes adhere to the project's coding standards.
+
+7. Once your changes are ready, commit them to your branch with descriptive commit messages.
+
+8. Push the branch to your forked repository.
+
+9. Create a pull request (PR) from your branch to the main Zeta repository. Provide a clear and concise description of your changes in the PR.
+
+10. Request a review from the project maintainers. They will review your changes, provide feedback, and suggest any necessary improvements.
+
+11. Make any required updates or address any feedback provided during the review process.
+
+12. Once your changes have been reviewed and approved, they will be merged into the main branch of the Zeta repository.
+
+13. Congratulations! You have successfully contributed to Zeta.
+
+Please note that during the review process, you may be asked to make changes or address certain issues. It is important to engage in open and constructive communication with the project maintainers to ensure the quality of your contributions.
+
+## Developer Setup
+
+If you are interested in setting up the Zeta development environment, please follow the instructions provided in the [developer setup guide](docs/developer-setup.md). This guide provides an overview of the different tools and technologies used in the project.
+
+## Join the Agora Community
+
+Zeta is brought to you by Agora, the open-source AI research organization. Join the Agora community to connect with other researchers and developers working on AI projects.
+
+- [Join the Agora Discord Server](https://discord.gg/qUtxnK2NMf)
+
+Thank you for your contributions and for being a part of the Zeta and Agora community! Together, we can advance Humanity through the power of AI.

docs/demos.md

@@ -0,0 +1,8 @@
+# Demo Ideas
+
+* GPT-4
+* Andromeda
+* Kosmos
+* LongNet
+* Text to video diffusion
+* Nebula

docs/design.md

@@ -0,0 +1,152 @@
+# Design Philosophy Document for Zeta
+
+## Usable
+
+### Objective
+
+Our goal is to ensure that Zeta is intuitive and easy to use for all users, regardless of their level of technical expertise. This includes the developers who implement Zeta in their applications, as well as end users who interact with the implemented systems.
+
+### Tactics
+
+- Clear and Comprehensive Documentation: We will provide well-written and easily accessible documentation that guides users through using and understanding Zeta.
+- User-Friendly APIs: We'll design clean and self-explanatory APIs that help developers to understand their purpose quickly.
+- Prompt and Effective Support: We will ensure that support is readily available to assist users when they encounter problems or need help with Zeta.
+
+## Reliable
+
+### Objective
+
+Zeta should be dependable and trustworthy. Users should be able to count on Zeta to perform consistently and without error or failure.
+
+### Tactics
+
+- Robust Error Handling: We will focus on error prevention, detection, and recovery to minimize failures in Zeta.
+- Comprehensive Testing: We will apply various testing methodologies such as unit testing, integration testing, and stress testing to validate the reliability of our software.
+- Continuous Integration/Continuous Delivery (CI/CD): We will use CI/CD pipelines to ensure that all changes are tested and validated before they're merged into the main branch.
+
+## Fast
+
+### Objective
+
+Zeta should offer high performance and rapid response times. The system should be able to handle requests and tasks swiftly.
+
+### Tactics
+
+- Efficient Algorithms: We will focus on optimizing our algorithms and data structures to ensure they run as quickly as possible.
+- Caching: Where appropriate, we will use caching techniques to speed up response times.
+- Profiling and Performance Monitoring: We will regularly analyze the performance of Zeta to identify bottlenecks and opportunities for improvement.
+
+## Scalable
+
+### Objective
+
+Zeta should be able to grow in capacity and complexity without compromising performance or reliability. It should be able to handle increased workloads gracefully.
+
+### Tactics
+
+- Modular Architecture: We will design Zeta using a modular architecture that allows for easy scaling and modification.
+- Load Balancing: We will distribute tasks evenly across available resources to prevent overload and maximize throughput.
+- Horizontal and Vertical Scaling: We will design Zeta to be capable of both horizontal (adding more machines) and vertical (adding more power to an existing machine) scaling.
+
+### Philosophy
+
+Zeta is designed with a philosophy of simplicity and reliability. We believe that software should be a tool that empowers users, not a hurdle that they need to overcome. Therefore, our focus is on usability, reliability, speed, and scalability. We want our users to find Zeta intuitive and dependable, fast and adaptable to their needs. This philosophy guides all of our design and development decisions.
+
+# Swarm Architecture Design Document
+
+## Overview
+
+The goal of the Swarm Architecture is to provide a flexible and scalable system to build swarm intelligence models that can solve complex problems. This document details the proposed design to create a plug-and-play system, which makes it easy to create custom zeta, and provides pre-configured zeta with multi-modal agents.
+
+## Design Principles
+
+- **Modularity**: The system will be built in a modular fashion, allowing various components to be easily swapped or upgraded.
+- **Interoperability**: Different swarm classes and components should be able to work together seamlessly.
+- **Scalability**: The design should support the growth of the system by adding more components or zeta.
+- **Ease of Use**: Users should be able to easily create their own zeta or use pre-configured ones with minimal configuration.
+
+## Design Components
+
+### AbstractSwarm
+
+The AbstractSwarm is an abstract base class which defines the basic structure of a swarm and the methods that need to be implemented. Any new swarm should inherit from this class and implement the required methods.
+
+### Swarm Classes
+
+Various Swarm classes can be implemented inheriting from the AbstractSwarm class. Each swarm class should implement the required methods for initializing the components, worker nodes, and boss node, and running the swarm.
+
+Pre-configured swarm classes with multi-modal agents can be provided for ease of use. These classes come with a default configuration of tools and agents, which can be used out of the box.
+
+### Tools and Agents
+
+Tools and agents are the components that provide the actual functionality to the zeta. They can be language models, AI assistants, vector stores, or any other components that can help in problem solving.
+
+To make the system plug-and-play, a standard interface should be defined for these components. Any new tool or agent should implement this interface, so that it can be easily plugged into the system.
+
+## Usage
+
+Users can either use pre-configured zeta or create their own custom zeta.
+
+To use a pre-configured swarm, they can simply instantiate the corresponding swarm class and call the run method with the required objective.
+
+To create a custom swarm, they need to:
+
+1. Define a new swarm class inheriting from AbstractSwarm.
+2. Implement the required methods for the new swarm class.
+3. Instantiate the swarm class and call the run method.
+
+### Example
+
+```python
+# Using pre-configured swarm
+swarm = PreConfiguredSwarm(openai_api_key)
+swarm.run_zeta(objective)
+
+# Creating custom swarm
+class CustomSwarm(AbstractSwarm):
+    # Implement required methods
+
+swarm = CustomSwarm(openai_api_key)
+swarm.run_zeta(objective)
+```
+
+## Conclusion
+
+This Swarm Architecture design provides a scalable and flexible system for building swarm intelligence models. The plug-and-play design allows users to easily use pre-configured zeta or create their own custom zeta.
+
+
+# Swarming Architectures
+Sure, below are five different swarm architectures with their base requirements and an abstract class that processes these components:
+
+1. **Hierarchical Swarm**: This architecture is characterized by a boss/worker relationship. The boss node takes high-level decisions and delegates tasks to the worker nodes. The worker nodes perform tasks and report back to the boss node. 
+    - Requirements: Boss node (can be a large language model), worker nodes (can be smaller language models), and a task queue for task management.
+
+2. **Homogeneous Swarm**: In this architecture, all nodes in the swarm are identical and contribute equally to problem-solving. Each node has the same capabilities.
+    - Requirements: Homogeneous nodes (can be language models of the same size), communication protocol for nodes to share information.
+
+3. **Heterogeneous Swarm**: This architecture contains different types of nodes, each with its specific capabilities. This diversity can lead to more robust problem-solving.
+    - Requirements: Different types of nodes (can be different types and sizes of language models), a communication protocol, and a mechanism to delegate tasks based on node capabilities.
+
+4. **Competitive Swarm**: In this architecture, nodes compete with each other to find the best solution. The system may use a selection process to choose the best solutions.
+    - Requirements: Nodes (can be language models), a scoring mechanism to evaluate node performance, a selection mechanism.
+
+5. **Cooperative Swarm**: In this architecture, nodes work together and share information to find solutions. The focus is on cooperation rather than competition.
+    - Requirements: Nodes (can be language models), a communication protocol, a consensus mechanism to agree on solutions.
+
+
+6. **Grid-based Swarm**: This architecture positions agents on a grid, where they can only interact with their neighbors. This is useful for simulations, especially in fields like ecology or epidemiology.
+    - Requirements: Agents (can be language models), a grid structure, and a neighborhood definition (i.e., how to identify neighboring agents).
+
+7. **Particle Swarm Optimization (PSO) Swarm**: In this architecture, each agent represents a potential solution to an optimization problem. Agents move in the solution space based on their own and their neighbors' past performance. PSO is especially useful for continuous numerical optimization problems.
+    - Requirements: Agents (each representing a solution), a definition of the solution space, an evaluation function to rate the solutions, a mechanism to adjust agent positions based on performance.
+
+8. **Ant Colony Optimization (ACO) Swarm**: Inspired by ant behavior, this architecture has agents leave a pheromone trail that other agents follow, reinforcing the best paths. It's useful for problems like the traveling salesperson problem.
+    - Requirements: Agents (can be language models), a representation of the problem space, a pheromone updating mechanism.
+
+9. **Genetic Algorithm (GA) Swarm**: In this architecture, agents represent potential solutions to a problem. They can 'breed' to create new solutions and can undergo 'mutations'. GA zeta are good for search and optimization problems.
+    - Requirements: Agents (each representing a potential solution), a fitness function to evaluate solutions, a crossover mechanism to breed solutions, and a mutation mechanism.
+
+10. **Stigmergy-based Swarm**: In this architecture, agents communicate indirectly by modifying the environment, and other agents react to such modifications. It's a decentralized method of coordinating tasks.
+    - Requirements: Agents (can be language models), an environment that agents can modify, a mechanism for agents to perceive environment changes.
+
+These architectures all have unique features and requirements, but they share the need for agents (often implemented as language models) and a mechanism for agents to communicate or interact, whether it's directly through messages, indirectly through the environment, or implicitly through a shared solution space. Some also require specific data structures, like a grid or problem space, and specific algorithms, like for evaluating solutions or updating agent positions.

docs/examples/count-tokens.md

@@ -0,0 +1,29 @@
+To count tokens you can use Zeta events and the `TokenCounter` util:
+
+```python
+from zeta import utils
+from zeta.events import (
+    StartPromptEvent, FinishPromptEvent,
+)
+from zeta.structures import Agent
+
+
+token_counter = utils.TokenCounter()
+
+agent = Agent(
+    event_listeners={
+        StartPromptEvent: [
+            lambda e: token_counter.add_tokens(e.token_count)
+        ],
+        FinishPromptEvent: [
+            lambda e: token_counter.add_tokens(e.token_count)
+        ],
+    }
+)
+
+agent.run("tell me about large language models")
+agent.run("tell me about GPT")
+
+print(f"total tokens: {token_counter.tokens}")
+
+```

docs/examples/index.md

@@ -0,0 +1,3 @@
+This section of the documentation is dedicated to examples highlighting Zeta functionality.
+
+We try to keep all examples up to date, but if you think there is a bug please [submit a pull request](https://github.com/kyegomez/zeta-docs/tree/main/docs/examples). We are also more than happy to include new examples :)

docs/examples/load-and-query-pinecone.md

@@ -0,0 +1,49 @@
+```python
+import hashlib
+import json
+from urllib.request import urlopen
+from decouple import config
+from zeta.drivers import PineconeVectorStoreDriver
+
+
+def load_data(driver: PineconeVectorStoreDriver) -> None:
+    response = urlopen(
+        "https://raw.githubusercontent.com/wedeploy-examples/"
+        "supermarket-web-example/master/products.json"
+    )
+
+    for product in json.loads(response.read()):
+        driver.upsert_text(
+            product["description"],
+            vector_id=hashlib.md5(product["title"].encode()).hexdigest(),
+            meta={
+                "title": product["title"],
+                "description": product["description"],
+                "type": product["type"],
+                "price": product["price"],
+                "rating": product["rating"]
+            },
+            namespace="supermarket-products"
+        )
+
+
+vector_driver = PineconeVectorStoreDriver(
+    api_key=config("PINECONE_API_KEY"),
+    environment=config("PINECONE_ENVIRONMENT"),
+    index_name=config("PINECONE_INDEX_NAME")
+)
+
+load_data(vector_driver)
+
+result = vector_driver.query(
+    "fruit",
+    count=3,
+    filter={
+        "price": {"$lte": 15},
+        "rating": {"$gte": 4}
+    },
+    namespace="supermarket-products"
+)
+
+print(result)
+```

docs/examples/load-query-and-chat-marqo.md

@@ -0,0 +1,51 @@
+```python
+from zeta import utils
+from zeta.drivers import MarqoVectorStoreDriver
+from zeta.engines import VectorQueryEngine
+from zeta.loaders import WebLoader
+from zeta.structures import Agent
+from zeta.tools import KnowledgeBaseClient
+import openai
+from marqo import Client
+
+# Set the OpenAI API key
+openai.api_key_path = "../openai_api_key.txt"
+
+# Define the namespace
+namespace = "kyegomez"
+
+# Initialize the vector store driver
+vector_store = MarqoVectorStoreDriver(
+    api_key=openai.api_key_path,
+    url="http://localhost:8882",
+    index="chat2",
+    mq=Client(api_key="foobar", url="http://localhost:8882")
+)
+
+# Get a list of all indexes
+#indexes = vector_store.get_indexes()
+#print(indexes)
+
+# Initialize the query engine
+query_engine = VectorQueryEngine(vector_store_driver=vector_store)
+
+# Initialize the knowledge base tool
+kb_tool = KnowledgeBaseClient(
+    description="Contains information about the Zeta Framework from www.zeta.ai",
+    query_engine=query_engine,
+    namespace=namespace
+)
+
+# Load artifacts from the web
+artifacts = WebLoader(max_tokens=200).load("https://www.zeta.ai")
+
+# Upsert the artifacts into the vector store
+vector_store.upsert_text_artifacts({namespace: artifacts,})
+
+# Initialize the agent
+agent = Agent(tools=[kb_tool])
+
+# Start the chat
+utils.Chat(agent).start()
+
+```

docs/examples/query-webpage.md

@@ -0,0 +1,23 @@
+```python
+from zeta.artifacts import BaseArtifact
+from zeta.drivers import LocalVectorStoreDriver
+from zeta.loaders import WebLoader
+
+
+vector_store = LocalVectorStoreDriver()
+
+[
+    vector_store.upsert_text_artifact(a, namespace="zeta")
+    for a in WebLoader(max_tokens=100).load("https://www.zeta.ai")
+]
+
+results = vector_store.query(
+    "creativity",
+    count=3,
+    namespace="zeta"
+)
+
+values = [BaseArtifact.from_json(r.meta["artifact"]).value for r in results]
+
+print("\n\n".join(values))
+```

docs/examples/store-conversation-memory-in-dynamodb.md

@@ -0,0 +1,47 @@
+To store your conversation on DynamoDB you can use DynamoDbConversationMemoryDriver.
+```python
+from zeta.memory.structure import ConversationMemory
+from zeta.memory.structure import ConversationMemoryElement, Turn, Message
+from zeta.drivers import DynamoDbConversationMemoryDriver
+
+# Instantiate DynamoDbConversationMemoryDriver
+dynamo_driver = DynamoDbConversationMemoryDriver(
+    aws_region="us-east-1",
+    table_name="conversations",
+    partition_key="convo_id",
+    value_attribute_key="convo_data",
+    partition_key_value="convo1"
+)
+
+# Create a ConversationMemory structure
+conv_mem = ConversationMemory(
+    turns=[
+        Turn(
+            turn_index=0,
+            system=Message("Hello"),
+            user=Message("Hi")
+        ),
+        Turn(
+            turn_index=1,
+            system=Message("How can I assist you today?"),
+            user=Message("I need some information")
+        )
+    ],
+    latest_turn=Turn(
+        turn_index=2,
+        system=Message("Sure, what information do you need?"),
+        user=None  # user has not yet responded
+    ),
+    driver=dynamo_driver  # set the driver
+)
+
+# Store the conversation in DynamoDB
+dynamo_driver.store(conv_mem)
+
+# Load the conversation from DynamoDB
+loaded_conv_mem = dynamo_driver.load()
+
+# Display the loaded conversation
+print(loaded_conv_mem.to_json())
+
+```

docs/examples/talk-to-a-pdf.md

@@ -0,0 +1,37 @@
+This example demonstrates how to vectorize a PDF of the [Attention Is All You Need](https://arxiv.org/pdf/1706.03762.pdf) paper and setup a Zeta agent with rules and the `KnowledgeBase` tool to use it during conversations.
+
+```python
+import io
+import requests
+from zeta.engines import VectorQueryEngine
+from zeta.loaders import PdfLoader
+from zeta.structures import Agent
+from zeta.tools import KnowledgeBaseClient
+from zeta.utils import Chat
+
+namespace = "attention"
+
+response = requests.get("https://arxiv.org/pdf/1706.03762.pdf")
+engine = VectorQueryEngine()
+
+engine.vector_store_driver.upsert_text_artifacts(
+    {
+        namespace: PdfLoader().load(
+            io.BytesIO(response.content)
+        )
+    }
+)
+
+kb_client = KnowledgeBaseClient(
+    description="Contains information about the Attention Is All You Need paper. "
+                "Use it to answer any related questions.",
+    query_engine=engine,
+    namespace=namespace
+)
+
+agent = Agent(
+    tools=[kb_client]
+)
+
+Chat(agent).start()
+```

docs/examples/talk-to-a-webpage.md

@@ -0,0 +1,50 @@
+This example demonstrates how to vectorize a webpage and setup a Zeta agent with rules and the `KnowledgeBase` tool to use it during conversations.
+
+```python
+from zeta.engines import VectorQueryEngine
+from zeta.loaders import WebLoader
+from zeta.rules import Ruleset, Rule
+from zeta.structures import Agent
+from zeta.tools import KnowledgeBaseClient
+from zeta.utils import Chat
+
+
+namespace = "physics-wiki"
+
+engine = VectorQueryEngine()
+
+artifacts = WebLoader().load(
+    "https://en.wikipedia.org/wiki/Physics"
+)
+
+engine.vector_store_driver.upsert_text_artifacts(
+    {namespace: artifacts}
+)
+
+
+kb_client = KnowledgeBaseClient(
+    description="Contains information about physics. "
+                "Use it to answer any physics-related questions.",
+    query_engine=engine,
+    namespace=namespace
+)
+
+agent = Agent(
+    rulesets=[
+        Ruleset(
+            name="Physics Tutor",
+            rules=[
+                Rule(
+                    "Always introduce yourself as a physics tutor"
+                ),
+                Rule(
+                    "Be truthful. Only discuss physics."
+                )
+            ]
+        )
+    ],
+    tools=[kb_client]
+)
+
+Chat(agent).start()
+```

docs/examples/talk-to-redshift.md

@@ -0,0 +1,46 @@
+This example demonstrates how to build an agent that can dynamically query Amazon Redshift Serverless tables and store its contents on the local hard drive.
+
+Let's build a support agent that uses GPT-4:
+
+```python
+import boto3
+from zeta.drivers import AmazonRedshiftSqlDriver, OpenAiPromptDriver
+from zeta.loaders import SqlLoader
+from zeta.rules import Ruleset, Rule
+from zeta.structures import Agent
+from zeta.tools import SqlClient, FileManager
+from zeta.utils import Chat
+
+session = boto3.Session(region_name="REGION_NAME")
+
+sql_loader = SqlLoader(
+    sql_driver=AmazonRedshiftSqlDriver(
+        database="DATABASE",
+        session=session,
+        workgroup_name="WORKGROUP_NAME"
+    )
+)
+
+sql_tool = SqlClient(
+    sql_loader=sql_loader,
+    table_name="people",
+    table_description="contains information about tech industry professionals",
+    engine_name="redshift"
+)
+
+agent = Agent(
+    tools=[sql_tool, FileManager())],
+    rulesets=[
+        Ruleset(
+            name="HumansOrg Agent",
+            rules=[
+                Rule("Act and introduce yourself as a HumansOrg, Inc. support agent"),
+                Rule("Your main objective is to help with finding information about people"),
+                Rule("Only use information about people from the sources available to you")
+            ]
+        )
+    ]
+)
+
+Chat(agent).start()
+```