Re-structure

This commit is contained in:
2023-02-23 15:55:29 +01:00
parent 58abb8e530
commit 22d6e5df3c
3 changed files with 31 additions and 6 deletions

View File

@@ -41,3 +41,27 @@ That is what this project proposes. We can stress-test the prompt by giving it a
|----------------------------|-----------------------| |----------------------------|-----------------------|
| Can be used with any model | Can be time-consuming | | Can be used with any model | Can be time-consuming |
| Saves possible costs | Not 100% accurate | | Saves possible costs | Not 100% accurate |
## How to use
Currently, this stress-test was designed to interface with [suppress.js](https://github.com/velocitatem/suppress), but it can easily be modified to work with anything else.
First, you should configure the way you want to evaluate the prompts in `run.py`. In this file, you can configure the following:
```python
def runPrompt(prompt):
return "The evaluation of that prompt in a given model or application"
```
This function should take in a prompt and return a string that represents the evaluation of that prompt. For example, if you are using [suppress.js](https://github.com/velocitatem/suppress), you can use the existing function, and just modify the endpoint.
Then, you can run the stress-test with:
```bash
python3 cps-stress-test.py
```
This will run the stress-test of the prompts on your model or application. It will output the results of the stress-test.
## Contributing
Always looking for more prompts to test! If you have any prompts that you want to add, please make a pull request. Any other contributions are also welcome.

View File

@@ -2,6 +2,7 @@ import json
import sys import sys
import os import os
import subprocess import subprocess
from run import runPrompt
# read the file malicous.csv # read the file malicous.csv
# each item is like this: [malicious prompt, expected malicous response] # each item is like this: [malicious prompt, expected malicous response]
@@ -16,12 +17,8 @@ def readMaliciousFile():
return malicious return malicious
data = readMaliciousFile() data = readMaliciousFile()
import requests
def runPrompt(prompt):
# send a get request to localhost:3042/new/person/:introduction
# the introduction is the prompt
r = requests.get('http://localhost:3042/capital/' + prompt)
return r.json().get('response')

4
run.py Normal file
View File

@@ -0,0 +1,4 @@
import requests
def runPrompt(prompt):
r = requests.get('http://localhost:3042/capital/' + prompt)
return r.json().get('response')