velocitatem/raccoon
1
0
Fork 0
mirror of https://github.com/velocitatem/raccoon.git synced 2026-05-31 16:53:37 +00:00
Code Issues Packages Projects Releases Wiki Activity

Re-structure

Browse Source
This commit is contained in:
Daniel Rosel
2023-02-23 15:55:29 +01:00
parent 58abb8e530
commit 22d6e5df3c
3 changed files with 31 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
README.md
View File

@@ -41,3 +41,27 @@ That is what this project proposes. We can stress-test the prompt by giving it a
|----------------------------|-----------------------|
| Can be used with any model | Can be time-consuming |
| Saves possible costs | Not 100% accurate |
## How to use
Currently, this stress-test was designed to interface with [suppress.js](https://github.com/velocitatem/suppress), but it can easily be modified to work with anything else.
First, you should configure the way you want to evaluate the prompts in `run.py`. In this file, you can configure the following:
```python
def runPrompt(prompt):
return "The evaluation of that prompt in a given model or application"
```
This function should take in a prompt and return a string that represents the evaluation of that prompt. For example, if you are using [suppress.js](https://github.com/velocitatem/suppress), you can use the existing function, and just modify the endpoint.
Then, you can run the stress-test with:
```bash
python3 cps-stress-test.py
```
This will run the stress-test of the prompts on your model or application. It will output the results of the stress-test.
## Contributing
Always looking for more prompts to test! If you have any prompts that you want to add, please make a pull request. Any other contributions are also welcome.