This commit is contained in:
2023-02-24 21:52:32 +01:00
parent 6cb6b95743
commit 8ec3e78f84
5 changed files with 341 additions and 73 deletions

View File

@@ -42,6 +42,9 @@ That is what this project proposes. We can stress-test the prompt by giving it a
| Can be used with any model | Can be time-consuming |
| Saves possible costs | Not 100% accurate |
### Other
It is also important to understand how to design a good prompt, you can read more about that [here](https://github.com/dair-ai/Prompt-Engineering-Guide).
## How to use
Currently, this stress-test was designed to interface with [suppress.js](https://github.com/velocitatem/suppress), but it can easily be modified to work with anything else.
@@ -55,10 +58,12 @@ def runPrompt(prompt):
This function should take in a prompt and return a string that represents the evaluation of that prompt. For example, if you are using [suppress.js](https://github.com/velocitatem/suppress), you can use the existing function, and just modify the endpoint.
Alternatively, you can pass your own method as a parameter to the `run(method=method)` function in `cps.py`. This method should take in a prompt and return a string that represents the evaluation of that prompt.
Then, you can run the stress-test with:
```bash
python3 cps-stress-test.py
python3 cps.py
```
This will run the stress-test of the prompts on your model or application. It will output the results of the stress-test.
@@ -85,7 +90,7 @@ node server.js
Now the backend is all set, you can run the stress-test with:
```bash
python3 cps-stress-test.py
python3 cps.py
```
You should get a response similar to this:
@@ -106,6 +111,7 @@ result: False
Percentage of malicious inputs that passed: 0.5
```
## Customization
## Contributing