Skip to content

Commit

Permalink
Updates to README to be more accurate. (#267)
Browse files Browse the repository at this point in the history
* Update README prod deploy section for clarity

* Update models Canopy works with

* Better wording

* Update README.md

Changed the production recommendation - docker first, running gunicorn as alternative

* Update README.md

Finalized reversing docker and gunicorn order

---------

Co-authored-by: igiloh-pinecone <118673156+igiloh-pinecone@users.noreply.github.com>
  • Loading branch information
aulorbe and igiloh-pinecone authored Feb 8, 2024
1 parent 04b4c7c commit a2cf274
Showing 1 changed file with 9 additions and 10 deletions.
19 changes: 9 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@ This will open a similar chat interface window, but will show both the RAG and n

## Considerations

* Canopy currently only supports OpenAI as the backend for both the embedding model and the LLM. Rate limits and pricing set by OpenAI will apply.
* Rate limits and pricing set by model providers apply to Canopy usage. Canopy currently works with OpenAI, Azure OpenAI, Anyscale, and Cohere models.
* More integrations will be supported in the near future.

## Contributing
Expand Down Expand Up @@ -248,19 +248,18 @@ client = OpenAI(base_url="http://localhost:8000/v1/my-namespace")

### Running Canopy server in production

Canopy is using FastAPI as the web framework and Uvicorn as the ASGI server. It is recommended to use Gunicorn as the production server, mainly because it supports multiple worker processes and can handle multiple requests in parallel, more details can be found [here](https://www.uvicorn.org/deployment/#using-a-process-manager).

To run the canopy server for production, please run:
Canopy is using FastAPI as the web framework and Uvicorn as the ASGI server.
To use Canopy in production, it is recommended to utilize Canopy's docker image, available on [GitHub Packages](https://github.com/pinecone-io/canopy/pkgs/container/canopy),
for your production needs.
For guidance on deploying Canopy on the Google Cloud Platform (GCP), refer to the example provided in the
[Deployment to GCP](docs/deployment-gcp.md) documentation.

Alternatively, you can use Gunicorn as production-grade WSGI, more details [here](https://www.uvicorn.org/deployment/#using-a-process-manager).
Set your desired `PORT` and `WORKER_COUNT` envrionment variables, and start the server with:
```bash
gunicorn canopy_server.app:app --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:PORT --workers WORKER_COUNT
gunicorn canopy_server.app:app --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:$PORT --workers $WORKER_COUNT
```

Alternatively, consider utilizing the Canopy Docker image available on [GitHub Packages](https://github.com/pinecone-io/canopy/pkgs/container/canopy)
for your production needs. For guidance on deploying Canopy on the Google Cloud Platform (GCP), refer to the example provided in the
[Deployment to GCP](docs/deployment-gcp.md) documentation.


> [!IMPORTANT]
> The server interacts with services like Pinecone and OpenAI using your own authentication credentials.
When deploying the server on a public web hosting provider, it is recommended to enable an authentication mechanism,
Expand Down

0 comments on commit a2cf274

Please sign in to comment.